Smooth OBD data?

Help on issues you run into with LapTimer; in case you have a question on how to use LapTimer, use the forum "Using LapTimer" instead
pawliger
20 or more Posts ★★★
20 or more Posts ★★★
Posts: 20
Joined: Wed Apr 02, 2014 6:43 pm

Smooth OBD data?

Post by pawliger »

I am using Petrolhead on an Android N5 with a BT-linked OBDLink MX and BT-linked Qstarz 818XT GPS unit.
I created an overlay using GoPro video as the primary video source.
The throttle and RPM data seems extremely janky and jumpy and often wrong. Could it be incorrect, and, if so, the bad data smoothed or rejected?

The video: https://www.youtube.com/watch?v=Lu5lQlCAHAE
User avatar
Harry
Site Admin
Site Admin
Posts: 10518
Joined: Sun Sep 12, 2010 10:32 am
Location: Siegum, Germany
Contact:

Re: Smooth OBD data?

Post by Harry »

Please check the Debugging OBD Connections available in the Accessories/OBD forum. When sending the log, please add a link to this thread so I get what needs to be done.

Harry
Image Image Image Image
foo_fighter
10 or more Posts ★
10 or more Posts ★
Posts: 11
Joined: Fri May 23, 2014 7:17 pm

Re: Smooth OBD data?

Post by foo_fighter »

I'm having similar issues with an nearly identical setup. I noticed that there are often spikes of very high RPM for 1 or 2 readings. Some as high as 20,000 RPM. Even if you remove the spikes, the increase in RPMs is very noisy(not physical)


Image

8499 5 31-May-14 16:33:27.400 107.25 36.587923 -121.753943 112.4 69.842122 236.8 776.902888 337.9 2 2 9 1.12 1.9 3.173 1.971611 1 -0.19 -0.6 36.587955 -121.753845 3456 25.41 94 58.408892 0.08 4 0.9 93 0 -37 0 5722 3456 5722
8500 5 31-May-14 16:33:27.600 107.45 36.587975 -121.753968 107.2 66.610992 236.8 776.902888 338.1 2 2 9 1.12 1.9 3.1791 1.975401 1 -0.19 -0.63 36.588007 -121.75387 11649 20.46 87 54.059294 1 1 0.9 93 0 -37 0 11649 3456 5722
8501 5 31-May-14 16:33:28.000 107.85 36.588065 -121.754018 95.5 59.340949 236.7 776.574804 337.2 2 2 9 1.12 1.9 3.19 1.982174 1 -0.2 -0.64 36.5881 -121.753915 3701 20.46 84 52.19518 0 3 0.9 93 0 -37 0 11649 3701 5722
Last edited by foo_fighter on Sun Jun 01, 2014 7:24 pm, edited 1 time in total.
User avatar
Harry
Site Admin
Site Admin
Posts: 10518
Joined: Sun Sep 12, 2010 10:32 am
Location: Siegum, Germany
Contact:

Re: Smooth OBD data?

Post by Harry »

This is most probably due to drop outs in OBD delivery. Please check the OBD Debugging thread and send a log: viewtopic.php?f=20&t=1500

- Harry
Image Image Image Image
foo_fighter
10 or more Posts ★
10 or more Posts ★
Posts: 11
Joined: Fri May 23, 2014 7:17 pm

Re: Smooth OBD data?

Post by foo_fighter »

Hi Harry,

Log file sent.

Thanks.
User avatar
Harry
Site Admin
Site Admin
Posts: 10518
Joined: Sun Sep 12, 2010 10:32 am
Location: Siegum, Germany
Contact:

Re: Smooth OBD data?

Post by Harry »

Thanks for sending the data...

I had a look into it. The logged data is raw data, while the data discussed in the thread is processed / interpolated data - we need to keep that in mind for this discussion. Other than the data in the post above, the raw data shows no spikes. My assumption is these peaks are generated by LapTimer when the OBD input drops for a short time with jumpy values before. With the high update rate GPS used (10 Hz), an OBD drop out of e.g. 1 second requires LapTimer to interpolate / extrapolate 10 OBD values. In case the last two values before the drop out show a big difference / are noisy, this noisyness is multiplied and leads to spikes.

In the log sent, the average OBD update rate is 7 to 8 Hz. The longest "drop" is 1.2 seconds...

The "jumpiness" in the data appears in the logged / raw data already, so this is indeed the characteristic of you car's sensors.

Lessons learned: high GPS update rates can lead to incorrect extrapolations for noisy data coming in at a lower update rate. I will keep that item for some future improvements. While I plan to provide "after the fact" filters for all GPS / OBD data anyway (available for lat/lon only currently), I'm not sure how to treat realtime / automatic smoothing... In general, LapTimer keeps raw data and does not manipulate it without the user asking for. But the rpm values in the log should probably get smoothed...

- Harry
Image Image Image Image
foo_fighter
10 or more Posts ★
10 or more Posts ★
Posts: 11
Joined: Fri May 23, 2014 7:17 pm

Re: Smooth OBD data?

Post by foo_fighter »

Would it help to send the exported data from this session?

I'm not sure I understand how the interpolation/extrapolation is creating 1 or 2 huge spikes between 2 readings:

For example 18 subsequent RPM outputs 0.2seconds apart:
3953
3947
3975
3920
3561
3008
2663
13589
3911
3911
3509
3592
3611
3396
3240
3159
3065
3025

How did 13,589 get interpolated or extrapolated from the sea of ~3000 RPM entries. If I could understand it, I could maybe write a filter in perl or excel but right now I'm just limiting the entries to less than 7800 RPM. Even with that the data jumps around a lot like in the video posted above.

Thanks.
Harry wrote:Thanks for sending the data...

I had a look into it. The logged data is raw data, while the data discussed in the thread is processed / interpolated data - we need to keep that in mind for this discussion. Other than the data in the post above, the raw data shows no spikes. My assumption is these peaks are generated by LapTimer when the OBD input drops for a short time with jumpy values before. With the high update rate GPS used (10 Hz), an OBD drop out of e.g. 1 second requires LapTimer to interpolate / extrapolate 10 OBD values. In case the last two values before the drop out show a big difference / are noisy, this noisyness is multiplied and leads to spikes.

In the log sent, the average OBD update rate is 7 to 8 Hz. The longest "drop" is 1.2 seconds...

The "jumpiness" in the data appears in the logged / raw data already, so this is indeed the characteristic of you car's sensors.

Lessons learned: high GPS update rates can lead to incorrect extrapolations for noisy data coming in at a lower update rate. I will keep that item for some future improvements. While I plan to provide "after the fact" filters for all GPS / OBD data anyway (available for lat/lon only currently), I'm not sure how to treat realtime / automatic smoothing... In general, LapTimer keeps raw data and does not manipulate it without the user asking for. But the rpm values in the log should probably get smoothed...

- Harry
User avatar
Harry
Site Admin
Site Admin
Posts: 10518
Joined: Sun Sep 12, 2010 10:32 am
Location: Siegum, Germany
Contact:

Re: Smooth OBD data?

Post by Harry »

You are right, in case it is a single value like the peak above, it is probably a different issue... As it is not possible to derive the original data from interpolated values, both a log of raw input and the result in LapTimer recordings (with a peak) is required. In case you want to filter existing data, you may consider to cut values changing too fast (I doubt it is possible to change rpms from 2663 to 13589 in 0.2 seconds for road going cars).

Something like

Code: Select all

for rpm[i] in recordings
        if abs(rpm[i]-rpm[i-1])>2000
                rpm[i] = rpm[i-1]
- Harry
Image Image Image Image
gplracerx
20 or more Posts ★★★
20 or more Posts ★★★
Posts: 836
Joined: Thu May 03, 2012 5:26 am
Location: Kingsport, TN USA

Re: Smooth OBD data?

Post by gplracerx »

I don't see the need to extrapolate OBD data in the first place. Why not just use the last valid point until you get a new point? The graph won't look as smooth, but you won't have the spikes you get when you try to extrapolate noisy data. Interpolation between valid points after the fact implies the data are valid when they may not be. I'd rather see a step change on the graph and know the data between the points is missing than have to dig through the data file later.
User avatar
TooManyIDs
20 or more Posts ★★★
20 or more Posts ★★★
Posts: 33
Joined: Fri Nov 29, 2013 5:16 pm

Re: Smooth OBD data?

Post by TooManyIDs »

OBD present in all recordings today.

Removed some slower laps first, had to pit three times to get the external GPS back up on the dash (guess I need a better mounting solution for it - use the rubber pad now - guess it needed cleaning - along with my dash). :lol:

Did have to trim master video to 23:32 ~ 1.99GB before I could process any overlays (ffmpeg option 2 worked well, thank you). Unfortunately, my fastest lap was in the part beyond 2GB.

I am seeing the jumpy OBD throttle position on the track, as I did in the neighborhood when beta testing. You suggested a different sync value, if I recall.

Much movement in RPM, as expected, but seems to be approximately what I would expect.

Surprised to notice today that the gear selection representation was consistently wrong (almost as if -1 from actual, but don't believe it's that simple). Only one section of today's course is taken in second gear (esses), most of the track in third, with end of the longer straights seeing fourth gear for a short while (you can hear the gear shifts in the video and rpm registering throttle blip while downshifting).

I had a look at the OBD Diagnostic thread, doesn't seem to be connection related. Didn't create a trace yet, seems you may have enough data from others. Otherwise, let me know, will create one.

Uploaded overlaid video, hlptrz, and csv. Let me know if any of these would be helpful. My setup details in my signature. Thanks.
Al
Apple iPhone 6 Plus (128GB)
Dual Electronics XGPS150A
PLX Kiwi 2
Post Reply