HLT exports with DashWare, TrackVision and RaceRender

lunat1ck · Post by **lunat1ck** » Mon Jul 02, 2012 2:15 pm

One of my goals is to create track videos that I can watch to study how I drive, learn where I'm making mistakes, and then go even faster. To accomplish this, I am trying to figure out the most efficient way to process the two video streams I take at the track (two GoPro cameras) and overlay them with the data gathered by HLT.

I'm looking at three software products to do the data overlay right now: DashWare, TrackVision and RaceRender, and unfortunately, I have not had a successfully generated final video yet, but the learning process has been interesting. First of all, I am doing all of this work on a MacBook Pro, running Windows 7 inside a VMWare virtual machine, although the issues I'm having appear to be independent of the underlying operating system. The two video streams are being combined into a single one by using iMovie 9.0.6 and it's slick picture-in-picture feature. The input video format for all the rendering software is 720p.

The main challenge with DashWare and TrackVision is getting either software product to read the output files generated by HLT. Using HLT 15.0.5, you have several choices for the export format, and can export either individual laps, an entire day, or all laps. HLT does support a format specific for both DashWare and/or TrackVision, but only when individual laps are exported. Since I have something like 70 laps worth of data from 6 track sessions over 2 days, exporting them individually is obviously very tedious, but I tried working with data exported both ways.

First, I exported the full set of laps, and then created truncated versions of the output file by editing it in Excel. To create a single file with all the data for a given session, I simply identified the lap numbers for that session, and deleted all of the rows in the file that were for unwanted laps. This way, the single big file with ALL of the laps can be carved up manually and a set of session-specific data files created. However, only RaceRender was able to read this data file and do anything useful with it. Both DashWare and/or TrackVision threw exceptions when reading this file.

Second, I went through the pain and agony of exporting every single lap, individually, using the DashWare/TrackVision CSV format. For each session, this gave me a set of files, one per lap. While the DashWare interface allows you to load and synchronize multiple file, the resulting output would never render data from more than the first lap. TrackVision doesn't support multiple data files (as far as I can see), but using instructions I received from TV support, I was able to create a data file that *should* have worked, but simply didn't. I'm waiting for some feedback form the TV folks about that. I also tried combining multiple single-lap files into a single file using Excel, but had no luck.

So, for both DashWare and TrackVision, I was unable to get multiple laps into either product. I suspect you could do this one lap at a time, but that is far too time consuming. This can only be a feasible process (I want to to do this for ALL of my track sessions, to study the data) if it can be done a session at a time.

RaceRender was able to read the multi-lap output CSV format just fine, with no modifications, and on screen seemed to created the complete video I wanted. However, when creating the output video, the resulting MP4 file is truncated. It only has a subset of the first lap rendered, and then the video stops on the last frame and never changes.

So, I got pretty far with RaceRender, and now I have to work with the developers to figure out why the generated MP4 file is useless (I have not yet tried exporting to WMV or AVI). I have to work with the DW and TV developers to figure out why they are not accepting the data formats from HLT. I suspect that some data massaging in Excel might workaround the issues I've found.

Sadly, both DashWare and TrackVision have *far* superior gauge choices, so I still want to figure out how to get the data into a format they can work with. RR can also do PIP natively, and it's possible the output file from iMovie is triggering bugs in the RR MP4 rendering, so I might play around with using different input formats.

In summary, I am unable to get ANY of these products to produce the video I want, but I'm chasing down the various issues I've found with each product's development team and I hope to get this all worked out and write up the how-to docs I promised.

Post by **Harry** » Mon Jul 02, 2012 7:22 pm

Hi,

LapTimer v16 will unify the versatile CSV and DashWare / TrackVision CSV export. It will allow session export in addition (continues laps). I have passed the new format to DW and TV for verification and expect it to work fine with these.

- Harry

Tony@TrackVision · Post by **Tony@TrackVision** » Tue Jul 03, 2012 2:58 pm

Harry's v16 format works nicely with TrackVision, and will be a good fit for user who want to make video of a single track session or race. TrackVision lets you trim the start and the end of the video, so its easy to remove the paddock and grid section and the cool down/return to paddock if you want. You can also save individual individual laps or parts fo laps within a session, of course.

lunat1ck · Post by **lunat1ck** » Fri Jul 13, 2012 4:04 pm

As promised, here's the video's I made using data acquired with Harry's Lap Timer, and exported for use by Race Render.

http://www.youtube.com/watch?v=LNJGAk0wp0k
http://www.youtube.com/watch?v=-tlCIJLuXO0

I've got one more that shows all the cars I chased down and passed uploading right now, too....

First of all, I want to thank Tony at TrackVision for helping me resolve the issues I had with reading the data into TV. The problems I had were a result of manipulating the CSV file in Excel on a MacOS desktop, and then trying to use it on Windows 7. MacOS and Windows have different conventions for representing newlines, and in order to share data between the two platforms you MUST be sure to save the files as "Windows CSV" (which uses the correct newline representation), or to manipulate them on Windows natively. I created this problem for myself, since I have a Mac Book Pro, and I run Windows 7 in a VM.

Having said that, I did not use TV for these videos, because I spent all my time learning how to do both picture-in-picture as well as data overlay in Race Render. I am still unable to get DashWare to read the data at all, so I haven't been able to evaluate that software yet. Now that I have RaceRender figured out, I'll spend some time playing with TrackVision in the next couple of weeks, and share that experience.

First of all, RR's support for picture in picture is a pretty compelling feature, if you have multiple cameras. I initially used iMovie 11 to generate the PIP, but the resulting composite video created problems for Race Render, as it would generate truncated output video. According to the RR support folks, there are known issues with the time codes and metadata for videos generated like this, so I had to switch to using RR to do the PIP instead of iMovie. Tony at TV warned of similar issues, so the end result is that if you want to do PIP, you should either use RR, or post process the overlayed video generated by something like TV (which has no native PIP support) to add the additional camera angles.

However, editing PIP in iMovie and getting the two camera in sync was extremely tedious. In RR, it was very easy. Plus, RR makes it pretty easy to crop the images, using specific numeric values, while iMovie only let's to crop using the mouse. That makes it very difficult to apply the same cropping window to multiple video streams. In this case, iMovie's general purpose editing features are far less suited for this specific application, and RR makes this basic operation very easy.

Once I had a video layout I liked, it was easy to export it as a template, and then reuse it to fairly quickly create additional videos using data from different track sessions. I did not spend much time customizing the guages, but RR appears to have a lot of support for this. One thing missing from RR is a running "best lap" display, showing sector times and splits. I believe both DW and TV can do this, though.

A couple of notes on manipulating the HLT data: I used the export of ALL the laps, and then used Excel to add a "SPEED[MPH]" column, since 15.0.5 only exports KPH in the "ALL laps" export. I can post the instructions on how to do this, but I need to ask the author of those hints if it's OK to share them. IIRC, v16 will include improved per-session export, and I need to ask Harry if this will include MPH by default (sorry -- the US will never use metric speeds... Not my fault!!

I then made session-specific copies of the files, and simply deleted all the rows for laps that were not in that session. The resulting CSV file could then be used in RR to overlay data for the videos from that session.

In summary, I like Race Render, and it met my immediate goal of being able to post-process a lot of video and data into a useful form that I can study to learn how to drive better. Please let me know if this is useful or interesting information for the HLT community, and I'll continue to share what I'm learning as I progress. I'd be more than happy to answer any questions, assuming I know the answers.

Post by **Harry** » Fri Jul 13, 2012 4:29 pm

Thanks for the report. LT 16 has all data metric and imperial. As said before, I'd thought overlaying tools are able to do the conversion on their own. Anyway, LT 16 will have it

Harry

toolo4sno · Post by **toolo4sno** » Thu Jul 26, 2012 4:48 pm

I am having issues with my csv export to Dashware. The gauges freeze at lap 8 for me (lap timer keeps going on and on). I notice in the .csv that there is a strange data point for the time (see below, 15:47:46.:0). It happens right around the time and speed (44mph) that the odd gauge freezing happens. Does this strange time data point have anything to do with it and can I edit it somehow. My attempts at minor edits to the .csv seem to make DW not import it.

25-Jul-12 47:46.1 63.29 65.5 40.699813 1.6428 1.020789 0.04 -0.98 42.708013 -83.390268 297 41.6 0 0 0 0 0 0
25-Jul-12 47:46.3 63.49 67.7 42.06683 1.6477 1.023833 0.05 -0.95 42.708037 -83.390217 297 46.2 0 0 0 0 0 0
25-Jul-12 47:46.7 63.89 71.6 44.490177 1.6574 1.029861 0.04 -0.93 42.708082 -83.390113 298 52 0 0 0 0 0 0
25-Jul-12 15:47:46.:0 64.17 73.3 45.546508 1.664 1.033962 0.02 -0.56 42.708114 -83.390044 298 53.5 0 0 0 0 0 0
25-Jul-12 47:47.1 64.29 73.8 45.857194 1.6668 1.035701 0.08 -0.87 42.708128 -83.390015 299 53.8 0 0 0 0 0 0
25-Jul-12 47:47.3 64.49 74.4 46.230017 1.6714 1.03856 0.09 -0.76 42.708153 -83.38997 299 53.4 0 0 0 0 0 0
25-Jul-12 47:47.5 64.69 74.6 46.354291 1.6759 1.041356 0.05 -0.62 42.70818 -83.389928 299 52.5 0 0 0 0 0 0

Issue two: I think this is getting resolved in V16 (can't wait for that by the way), but how do I edit my LapTimerGPSRecDB4Dashboard.csv file for each session so that lap count and such are correct for the session I'm on? Is there a tutorial I missed?

Post by **Harry** » Fri Jul 27, 2012 11:18 am

On #2: not currently. Like LapTimer's own overlay, export is currently designed to overlay each lap.

Harry

toolo4sno · Post by **toolo4sno** » Sat Jul 28, 2012 10:26 pm

Can any modifications be made to the LapTimerGPSRecDB4Dashboard.csv file, saved, then opened in DW? It doesn't appear to be the case. I've noticed that the file size changes dramatically just saving it after a minor change; compared to unzipped size.

The 3rd session of my most recent track day also has an odd gauge locking. This time it starts frozen and then rejoins logging/displaying. A similar oddly formatted time code is in the 3rd session's data. This time its 19:33:54.:0.

Could this all be caused by the way I'm ending each session. From what I recall I hit "cancel" in Racers mode when coming in on the cool down lap near the pit entrance. Most of my sessions the data has been acting odd around the lap before cool down. That's aside from the 1st and 3rd session frozen gauges.

Also, lap count doesn't appear to be on your output spreadsheet, but DW can display it. Is that something they are doing or is it in your sheet and I just haven't looked hard enough?

Although I turned off the gauges when they were acting up and edited out most of the odd behavior, here is my Youtube page for reference.
http://www.youtube.com/user/2lo4sno

Post by **Harry** » Sun Jul 29, 2012 10:47 am

Hi,

The csv files are plain text files. The only thing that might be a little complicated is that line feeds are different in UNIX systems (iOS) and Windows PCs. But a good text editor can handle this.

In case you see odd data, simply remove the line.

Canceling the last round is o.k.

As stated above, v16 will have the ability to export sessions, it will include everything needed for multi laps.

Harry

toolo4sno · Post by **toolo4sno** » Mon Jul 30, 2012 12:15 am

I was able to edit the csv appropriately, thanks for the tip. I could see in the text editor how the layout/formatting of a unix vs windows (saved thru excel) was drastically different. My videos are great now.

Thanks, Harry.