Major stability and data quality issues with 17.0.2

Any discussion on using LapTimer. Please use this forum in case you need guidance on how to use LapTimer or perform a certain operation
Post Reply
lunat1ck
20 or more Posts ★★★
20 or more Posts ★★★
Posts: 42
Joined: Thu Apr 19, 2012 4:01 pm
Location: New York, NY, USA
Contact:

Major stability and data quality issues with 17.0.2

Post by lunat1ck »

After a long, cold, boring winter I finally had my first track day at Lime Rock Park this past Monday, and I had a very bad experience with HLT 17.0.2. Last year, I managed to work out the kinks of using the software, but it took a couple of trips to the track to do so. By June, it was all working, and I managed to put together a series of very cool videos using HLT's data with Race Render 2. I was expecting things to keep working, at least as well as they did with the previous release, but as you will see, that was not the case.

Before I go further, here's my setup:

iPhone 4 running iOS 5 (I'm staying away from 6 until absolutely necessary -- possible issue?)
XGPS150, with the upgraded firmware, running at 5Hz
PLX Devices Kiwi2
HLT 17.0.2 Grand Prix Edition
5 GoPro cameras (I don't use the iPhone for video)
Post processing in Race Render 2

The day started out bad, when HLT crashed the first time I started it, and then took several minutes to fix all the databases and sanity check the old lap data (I haven't deleted *anything* from last year, so there's a lot of old data on my phone). Throughout the day, it seemed to switch from forward to reverse video at random, which was pretty annoying. During the track sessions, I kept seeing really strange numbers when I would cross the start finish line, numbers that were clearly NOT my actual lap time. I don't generally look at HLT while I'm on track, though, so I didn't really see a pattern, just observed a few strange numbers. For example, once it showed my lap time as around 30 seconds, and since the track record is in the upper 40s (in a car just a tad more capable than mine :-) it's pretty obviously bogus.

After each session, though, I always check the lap list to get an idea of what times I was running, and when viewed on HLT the times all looked about right. The next day, when I did a sanity check of those times against the times measured in the video, they were within .01 seconds. At the track, my biggest problem was the day/night display switching at random, and a LOT of crashes. About every 3rd time I started HLT, whether to set up for a new track session, or to show off my personal best 1.04:64 lap time to friends, it would go into the "Checking databases...." code. My understanding was that HLT only did this after a crash, otherwise, I don't why it happened some times and not others.

The real headaches started when I exported the data for use in Race Render. First of all, HLT crashes roughly 25% of the time when I try to export, and it does so in different places (sometimes when I select the lap, sometimes when I touch the export button, sometimes during the generation of the data)
I always edit the exported CSV data file to re-index the laps starting at 1, and I happened to notice that the LAST data point for some of the laps had a time value that was several orders of magnitude too large. This would totally throw RR2 off, but the simple workaround was to just delete that offending row. For the 4 sessions I exported, I had at least one such bogus row in all 4 of the files, IIRC. Once I had corrected that glitch, I then found that the actual data was *horrible* quality. My XGPS150 is providing data at 5Hz, and yet when you watch the cute little red dot wander the track map generated by RR2, it looks like I was driving on the hill across the river from the front straight, where the spectators sit. I'm not that bad :-P

In the data file for the first session the accelerometer data just disappears, and those values drop to 0 and stay there. The measured speed is also WAY off -- in some places, it would DROP while I was accelerating, and since I *know* my apex speeds on all 7 corners at LRP, I can see it was off by as much as 20 mph in some places. I will assume that since measured speed is derived from the GPS data, that the root cause here is bad information from the GPS. Whether it's the fault of the XGPS150 or HLT I can't tell, but clearly HLT has other issues as well, since it generates bogus data in the output files. Also, the accelerometer data has nothing to with the GPS (I assume...). FWIW, I have POI Detection set to Wide, too.

The data for the 2nd session seemed a little better, but the GPS was wandering WAY off track again, and this time, it started to miss the start/finish trigger as a result. Several laps were combined in pairs into a single lap, as a result of this. The data for the 3rd and 4th sessions had no merged laps, but the accuracy of the GPS data is still questionable, since the measured speeds are clearly very wrong.

As a result of this, the only data overlay that was worth anything in RR2 was the track map, and only in the sessions where that data wasn't so bad that the track map didn't even resemble Lime Rock Park?

I have uploaded the exported data files, and a sample video file that shows how poor the GPS data is to:

http://ftp.openefs.org/hlt (Abusing the ftp server for one of my open source projects :-P)

Right now, I am *extremely* frustrated with this software, because my track days are precious: I only get on track once or twice a month, and this data is critical for maximizing the learning experience. Last year, I had everything working wonderfully and I have been promoting HLT to other track drivers, but had this been my trial run with the software, I'd almost ask for my money back.

My next track day is May 2nd, and if I have no workarounds or solutions for these problems by then, I'm going to split my 4 track sessions between HLT and the TrackAddictHD product from the Race Render guys, and see if that does any better. Right now, the lack of stability and the data quality are forcing me to look for other options.

Harry -- I am a BIG fan of the work you've done, and I'm not going to give up on HLT just yet, so if there's anything else I can do to help you get to the root cause of these problems, please let me know.
User avatar
Harry
Site Admin
Site Admin
Posts: 10638
Joined: Sun Sep 12, 2010 10:32 am
Location: Siegum, Germany
Contact:

Re: Major stability and data quality issues with 17.0.2

Post by Harry »

Hi,

Sorry to hear you had trouble... Here are the reasons for your observations:

Both from looking into the data, and from the points you raised, it is pretty clear there has been a severe issue with the GPS. Step by step: the day / night skin changes colors when the sun changes from above horizon to below horizon (and vice versa). The time the sun sets and raises are calculated from the position you are. With a valid GPS position, LapTimer switches to high contrast in the morning, and back in the evening. Whenever the skin changes to black during daytime, you have lost GPS position. So actually this skin is a pretty good indicator something is wrong (you can change the skin to solid colors in LapTimer Settings if you want). The data linked show two other indicators: the HDOP value jumps up to 6 (instead of 1 - which means the receiver is operating at 100% of accuracy you can expect) - which means accuracy has been very bad. At this time LapTimer started over calibration (error condition detected) - which leads to 0 acceleration as the "standing still" first step cannot be performed while driving. The named wrong value is a consequence from error condition too.

So despite the other two items below, GPS is the root cause for wrong values displayed, black / white switch / wrong speeds / wrong timing. It is in the nature of an app like LapTimer to be blamed for the results - but it is nothing one can change on the app side in this situation. In addition, please keep in mind LapTimer never recalculates GPS data except when asked for. So whenever you see wrong positions or speeds, check the GPS situation. The usual approach for doing this is to exclude components to identify the source of the problem. So in case you see effects like the named, start for example by removing the XGPS and check how the system behaves for the internal sensor. Power on / off may help too here.

There are actually two item you can blame LapTimer for ;-) and both should be fixed in 17.0.3
Submitted bug fix release 17.0.3 to Apple last Saturday. It includes the following fixes / improvements:
- Some fixes to English, French and German localizations
- Fixed vehicle image association on iPads
- Some export improvement: crash for very "long" laps fixed, improved VBO format
- GPS View optimization: for historic values, slope and approach are hidden
- Fixed mis-detection of Otto / Diesel engines from OBD
- Performance improvements for start up and resume
- Improved iCloud status message
- Improved OBD compatibility for BMWs, added PID exclusion to Expert Settings (workaround Lotus)
- Several intermittent bugs fixed
The frequent database checks happened for users with very big database: when the app resumed (so you sent it to background, and pull it forward again), LapTimer created a copy of the database to have a backup in place in case data gets corrupted. When this action takes too long, iOS believes the app is not alive anymore and kills it. During the next startup, this made LapTimer believe it crashed - leading to the length recovery process. The above point "Performance improvements for start up and resume" fixes this issue starting with 17.0.3. Please note that despite the waiting time, this should not have any other effect.

The other item is on export. For large sets of data, 17.0.2 generated indeed crashes - which is fixed in 17.0.3 too ("crash for very "long" laps fixed"). Please export again once 17.0.3 becomes available.

Hope that at least clarifies the situation. In case you have questions on the above, please let me know.

- Harry
Image Image Image Image
lunat1ck
20 or more Posts ★★★
20 or more Posts ★★★
Posts: 42
Joined: Thu Apr 19, 2012 4:01 pm
Location: New York, NY, USA
Contact:

Re: Major stability and data quality issues with 17.0.2

Post by lunat1ck »

Thanks for the reply, Harry -- that all makes sense (you usually do :-). I'll be sure to upgrade to 17.0.3 before the next track day, and I'll play around with the GPS and a fake track around my neighborhood and see how it fares. Once I got the firmware for the XGPS150 upgraded last year, I had no problems at all, so I naturally blamed the software that had changed.

I'll also clean up the database, and get rid of last years old laps. I really have no further use for them, since I learned everything I could from those sessions already, and I just don't see any value in the old information. Having said that, I'll export the data and stash it somewhere in case I change my mind.

At least I know where to focus my diagnostic efforts. Thanks again.
User avatar
Harry
Site Admin
Site Admin
Posts: 10638
Joined: Sun Sep 12, 2010 10:32 am
Location: Siegum, Germany
Contact:

Re: Major stability and data quality issues with 17.0.2

Post by Harry »

Fine, but please run the export with 17.0.3 :-D
Image Image Image Image
Post Reply