Smooth OBD data?
Smooth OBD data?
I am using Petrolhead on an Android N5 with a BT-linked OBDLink MX and BT-linked Qstarz 818XT GPS unit.
I created an overlay using GoPro video as the primary video source.
The throttle and RPM data seems extremely janky and jumpy and often wrong. Could it be incorrect, and, if so, the bad data smoothed or rejected?
The video: https://www.youtube.com/watch?v=Lu5lQlCAHAE
I created an overlay using GoPro video as the primary video source.
The throttle and RPM data seems extremely janky and jumpy and often wrong. Could it be incorrect, and, if so, the bad data smoothed or rejected?
The video: https://www.youtube.com/watch?v=Lu5lQlCAHAE
Re: Smooth OBD data?
Please check the Debugging OBD Connections available in the Accessories/OBD forum. When sending the log, please add a link to this thread so I get what needs to be done.
Harry
Harry
-
- 10 or more Posts ★
- Posts: 11
- Joined: Fri May 23, 2014 7:17 pm
Re: Smooth OBD data?
I'm having similar issues with an nearly identical setup. I noticed that there are often spikes of very high RPM for 1 or 2 readings. Some as high as 20,000 RPM. Even if you remove the spikes, the increase in RPMs is very noisy(not physical)
8499 5 31-May-14 16:33:27.400 107.25 36.587923 -121.753943 112.4 69.842122 236.8 776.902888 337.9 2 2 9 1.12 1.9 3.173 1.971611 1 -0.19 -0.6 36.587955 -121.753845 3456 25.41 94 58.408892 0.08 4 0.9 93 0 -37 0 5722 3456 5722
8500 5 31-May-14 16:33:27.600 107.45 36.587975 -121.753968 107.2 66.610992 236.8 776.902888 338.1 2 2 9 1.12 1.9 3.1791 1.975401 1 -0.19 -0.63 36.588007 -121.75387 11649 20.46 87 54.059294 1 1 0.9 93 0 -37 0 11649 3456 5722
8501 5 31-May-14 16:33:28.000 107.85 36.588065 -121.754018 95.5 59.340949 236.7 776.574804 337.2 2 2 9 1.12 1.9 3.19 1.982174 1 -0.2 -0.64 36.5881 -121.753915 3701 20.46 84 52.19518 0 3 0.9 93 0 -37 0 11649 3701 5722
8499 5 31-May-14 16:33:27.400 107.25 36.587923 -121.753943 112.4 69.842122 236.8 776.902888 337.9 2 2 9 1.12 1.9 3.173 1.971611 1 -0.19 -0.6 36.587955 -121.753845 3456 25.41 94 58.408892 0.08 4 0.9 93 0 -37 0 5722 3456 5722
8500 5 31-May-14 16:33:27.600 107.45 36.587975 -121.753968 107.2 66.610992 236.8 776.902888 338.1 2 2 9 1.12 1.9 3.1791 1.975401 1 -0.19 -0.63 36.588007 -121.75387 11649 20.46 87 54.059294 1 1 0.9 93 0 -37 0 11649 3456 5722
8501 5 31-May-14 16:33:28.000 107.85 36.588065 -121.754018 95.5 59.340949 236.7 776.574804 337.2 2 2 9 1.12 1.9 3.19 1.982174 1 -0.2 -0.64 36.5881 -121.753915 3701 20.46 84 52.19518 0 3 0.9 93 0 -37 0 11649 3701 5722
Last edited by foo_fighter on Sun Jun 01, 2014 7:24 pm, edited 1 time in total.
Re: Smooth OBD data?
This is most probably due to drop outs in OBD delivery. Please check the OBD Debugging thread and send a log: viewtopic.php?f=20&t=1500
- Harry
- Harry
-
- 10 or more Posts ★
- Posts: 11
- Joined: Fri May 23, 2014 7:17 pm
Re: Smooth OBD data?
Hi Harry,
Log file sent.
Thanks.
Log file sent.
Thanks.
Re: Smooth OBD data?
Thanks for sending the data...
I had a look into it. The logged data is raw data, while the data discussed in the thread is processed / interpolated data - we need to keep that in mind for this discussion. Other than the data in the post above, the raw data shows no spikes. My assumption is these peaks are generated by LapTimer when the OBD input drops for a short time with jumpy values before. With the high update rate GPS used (10 Hz), an OBD drop out of e.g. 1 second requires LapTimer to interpolate / extrapolate 10 OBD values. In case the last two values before the drop out show a big difference / are noisy, this noisyness is multiplied and leads to spikes.
In the log sent, the average OBD update rate is 7 to 8 Hz. The longest "drop" is 1.2 seconds...
The "jumpiness" in the data appears in the logged / raw data already, so this is indeed the characteristic of you car's sensors.
Lessons learned: high GPS update rates can lead to incorrect extrapolations for noisy data coming in at a lower update rate. I will keep that item for some future improvements. While I plan to provide "after the fact" filters for all GPS / OBD data anyway (available for lat/lon only currently), I'm not sure how to treat realtime / automatic smoothing... In general, LapTimer keeps raw data and does not manipulate it without the user asking for. But the rpm values in the log should probably get smoothed...
- Harry
I had a look into it. The logged data is raw data, while the data discussed in the thread is processed / interpolated data - we need to keep that in mind for this discussion. Other than the data in the post above, the raw data shows no spikes. My assumption is these peaks are generated by LapTimer when the OBD input drops for a short time with jumpy values before. With the high update rate GPS used (10 Hz), an OBD drop out of e.g. 1 second requires LapTimer to interpolate / extrapolate 10 OBD values. In case the last two values before the drop out show a big difference / are noisy, this noisyness is multiplied and leads to spikes.
In the log sent, the average OBD update rate is 7 to 8 Hz. The longest "drop" is 1.2 seconds...
The "jumpiness" in the data appears in the logged / raw data already, so this is indeed the characteristic of you car's sensors.
Lessons learned: high GPS update rates can lead to incorrect extrapolations for noisy data coming in at a lower update rate. I will keep that item for some future improvements. While I plan to provide "after the fact" filters for all GPS / OBD data anyway (available for lat/lon only currently), I'm not sure how to treat realtime / automatic smoothing... In general, LapTimer keeps raw data and does not manipulate it without the user asking for. But the rpm values in the log should probably get smoothed...
- Harry
-
- 10 or more Posts ★
- Posts: 11
- Joined: Fri May 23, 2014 7:17 pm
Re: Smooth OBD data?
Would it help to send the exported data from this session?
I'm not sure I understand how the interpolation/extrapolation is creating 1 or 2 huge spikes between 2 readings:
For example 18 subsequent RPM outputs 0.2seconds apart:
3953
3947
3975
3920
3561
3008
2663
13589
3911
3911
3509
3592
3611
3396
3240
3159
3065
3025
How did 13,589 get interpolated or extrapolated from the sea of ~3000 RPM entries. If I could understand it, I could maybe write a filter in perl or excel but right now I'm just limiting the entries to less than 7800 RPM. Even with that the data jumps around a lot like in the video posted above.
Thanks.
I'm not sure I understand how the interpolation/extrapolation is creating 1 or 2 huge spikes between 2 readings:
For example 18 subsequent RPM outputs 0.2seconds apart:
3953
3947
3975
3920
3561
3008
2663
13589
3911
3911
3509
3592
3611
3396
3240
3159
3065
3025
How did 13,589 get interpolated or extrapolated from the sea of ~3000 RPM entries. If I could understand it, I could maybe write a filter in perl or excel but right now I'm just limiting the entries to less than 7800 RPM. Even with that the data jumps around a lot like in the video posted above.
Thanks.
Harry wrote:Thanks for sending the data...
I had a look into it. The logged data is raw data, while the data discussed in the thread is processed / interpolated data - we need to keep that in mind for this discussion. Other than the data in the post above, the raw data shows no spikes. My assumption is these peaks are generated by LapTimer when the OBD input drops for a short time with jumpy values before. With the high update rate GPS used (10 Hz), an OBD drop out of e.g. 1 second requires LapTimer to interpolate / extrapolate 10 OBD values. In case the last two values before the drop out show a big difference / are noisy, this noisyness is multiplied and leads to spikes.
In the log sent, the average OBD update rate is 7 to 8 Hz. The longest "drop" is 1.2 seconds...
The "jumpiness" in the data appears in the logged / raw data already, so this is indeed the characteristic of you car's sensors.
Lessons learned: high GPS update rates can lead to incorrect extrapolations for noisy data coming in at a lower update rate. I will keep that item for some future improvements. While I plan to provide "after the fact" filters for all GPS / OBD data anyway (available for lat/lon only currently), I'm not sure how to treat realtime / automatic smoothing... In general, LapTimer keeps raw data and does not manipulate it without the user asking for. But the rpm values in the log should probably get smoothed...
- Harry
Re: Smooth OBD data?
You are right, in case it is a single value like the peak above, it is probably a different issue... As it is not possible to derive the original data from interpolated values, both a log of raw input and the result in LapTimer recordings (with a peak) is required. In case you want to filter existing data, you may consider to cut values changing too fast (I doubt it is possible to change rpms from 2663 to 13589 in 0.2 seconds for road going cars).
Something like
- Harry
Something like
Code: Select all
for rpm[i] in recordings
if abs(rpm[i]-rpm[i-1])>2000
rpm[i] = rpm[i-1]
-
- 20 or more Posts ★★★
- Posts: 836
- Joined: Thu May 03, 2012 5:26 am
- Location: Kingsport, TN USA
Re: Smooth OBD data?
I don't see the need to extrapolate OBD data in the first place. Why not just use the last valid point until you get a new point? The graph won't look as smooth, but you won't have the spikes you get when you try to extrapolate noisy data. Interpolation between valid points after the fact implies the data are valid when they may not be. I'd rather see a step change on the graph and know the data between the points is missing than have to dig through the data file later.
- TooManyIDs
- 20 or more Posts ★★★
- Posts: 33
- Joined: Fri Nov 29, 2013 5:16 pm
Re: Smooth OBD data?
OBD present in all recordings today.
Removed some slower laps first, had to pit three times to get the external GPS back up on the dash (guess I need a better mounting solution for it - use the rubber pad now - guess it needed cleaning - along with my dash).
Did have to trim master video to 23:32 ~ 1.99GB before I could process any overlays (ffmpeg option 2 worked well, thank you). Unfortunately, my fastest lap was in the part beyond 2GB.
I am seeing the jumpy OBD throttle position on the track, as I did in the neighborhood when beta testing. You suggested a different sync value, if I recall.
Much movement in RPM, as expected, but seems to be approximately what I would expect.
Surprised to notice today that the gear selection representation was consistently wrong (almost as if -1 from actual, but don't believe it's that simple). Only one section of today's course is taken in second gear (esses), most of the track in third, with end of the longer straights seeing fourth gear for a short while (you can hear the gear shifts in the video and rpm registering throttle blip while downshifting).
I had a look at the OBD Diagnostic thread, doesn't seem to be connection related. Didn't create a trace yet, seems you may have enough data from others. Otherwise, let me know, will create one.
Uploaded overlaid video, hlptrz, and csv. Let me know if any of these would be helpful. My setup details in my signature. Thanks.
Removed some slower laps first, had to pit three times to get the external GPS back up on the dash (guess I need a better mounting solution for it - use the rubber pad now - guess it needed cleaning - along with my dash).
Did have to trim master video to 23:32 ~ 1.99GB before I could process any overlays (ffmpeg option 2 worked well, thank you). Unfortunately, my fastest lap was in the part beyond 2GB.
I am seeing the jumpy OBD throttle position on the track, as I did in the neighborhood when beta testing. You suggested a different sync value, if I recall.
Much movement in RPM, as expected, but seems to be approximately what I would expect.
Surprised to notice today that the gear selection representation was consistently wrong (almost as if -1 from actual, but don't believe it's that simple). Only one section of today's course is taken in second gear (esses), most of the track in third, with end of the longer straights seeing fourth gear for a short while (you can hear the gear shifts in the video and rpm registering throttle blip while downshifting).
I had a look at the OBD Diagnostic thread, doesn't seem to be connection related. Didn't create a trace yet, seems you may have enough data from others. Otherwise, let me know, will create one.
Uploaded overlaid video, hlptrz, and csv. Let me know if any of these would be helpful. My setup details in my signature. Thanks.
Al
Apple iPhone 6 Plus (128GB)
Dual Electronics XGPS150A
PLX Kiwi 2
Apple iPhone 6 Plus (128GB)
Dual Electronics XGPS150A
PLX Kiwi 2