sample track

As, I’ve mentioned earlier, we had a pleasant trip through Italy the last summer. I have recorded the whole trip to my GPS unit using i-GO navigation program. Seems like 10 second recording intervals are fine and the track is quite precise and smooth.
Unfortunately, during upgrade from WM5 to WM6 I had to reinstall i-Go and all tracks just dissapeared from track list in i-GO. Stupid me, i HAD to save ‘igo.db’ file somewhere.
I found no way to put them back, so let the fun begin.
A short glimpse into one of the files and..

...
$GPGSA,A,3,04,20,13,23,31,11,,,,,,,2.0,1.2,1.6*33
$GPRMC,061120.287,A,5414.2734,N,02337.5314,E,54.08,261.55,290707,,,A*50
$GPGGA,061121.000,5414.2726,N,02337.5281,E,1,06,1.2,152.2,M,31.0,M,,0000*54
...

Hey, do You think the same way? Yes, it’s NMEA data in plain text.

Ok, the Tool – GPSbabel is ready. It can read and write a lot of GPS data formats. Let’s try:

$ sudo apt-get install gpsbabel

, a handy script ‘nmea2gpx’ from here
and let the fun begin:

$ for i in `ls *.bin`; do ./nmea2gpx $i; done

The conversion is smooth. You can discard files files up to about 400B – they usually contains only empty GPX structure.

But the resulting data set is HUGE: initial data set is 182MB, the resulting GPX files uses MB.
It’s fine if you need very detailed track info, but two problems arise:

  • due to the nature of GPS it’s inaccurate, if you use the receiver in dense urban areas
  • you do not need this amount of information to create daily track or a track for the whole journey

And again, let’s turn back to GPSbabel. It has a lot of filters to improve overall usability of saved data:

$ gpsbabel -i gpx -f 2007-08-07.gpx -x simplify,crosstract,error=0.002k -o gpx -F 2007-08-07-e2m.gpx
$ gpsbabel -i gpx -f 2007-08-07.gpx -x simplify,crosstract,error=0.005k -o gpx -F 2007-08-07-e5m.gpx
$ gpsbabel -i gpx -f 2007-08-07.gpx -x simplify,crosstract,error=0.01k -o gpx -F 2007-08-07-e10m.gpx
$ gpsbabel -i gpx -f 2007-08-07.gpx -x simplify,crosstract,error=0.05k -o gpx -F 2007-08-07-e50m.gpx
$ gpsbabel -i gpx -f 2007-08-07.gpx -x simplify,crosstract,error=0.1k -o gpx -F 2007-08-07-e100m.gpx
$ gpsbabel -i gpx -f 2007-08-07.gpx -x discard,hdop=10 -o gpx -F 2007-08-07-hdop10.gpx
$ gpsbabel -i gpx -f 2007-08-07.gpx -x discard,hdop=6 -o gpx -F 2007-08-07-hdop6.gpx

 $ ls -al *08-07*.gpx
-rw-r--r-- 1 ejs ejs 5321544 2008-02-24 20:41 2007-08-07.gpx
-rw-r--r-- 1 ejs ejs  343808 2008-02-24 21:00 2007-08-07-e10m.gpx
-rw-r--r-- 1 ejs ejs  909181 2008-02-24 20:56 2007-08-07-e2m.gpx
-rw-r--r-- 1 ejs ejs  541929 2008-02-24 20:59 2007-08-07-e5m.gpx
-rw-r--r-- 1 ejs ejs  117973 2008-02-24 21:31 2007-08-07-e50m.gpx
-rw-r--r-- 1 ejs ejs   69973 2008-02-24 21:38 2007-08-07-e100m.gpx
-rw-r--r-- 1 ejs ejs 5072358 2008-02-24 20:52 2007-08-07-hdop10.gpx
-rw-r--r-- 1 ejs ejs 4983846 2008-02-24 20:52 2007-08-07-hdop6.gpx

Summarize the above:

100 meter error, data rate reduced 75 times

  • reducing maximum allowable Horizontal Dilution of Precision (HDOP) makes almost no change to data size
  • using ‘crosstrack’ filter and removing points maximum 50 meters away from track, reduced dataset size 10 times. Using 100 meters error reduces dataset almost 75 times, but you can see in the image and here, the data it is still usable.

The perfect and free tool to overlay the GPX track onto the globe is GPS visualizer . Use it.

The proposed workflow is:

  1. download tracks the tracks you need to the computer
  2. convert them to GPX
  3. merge day track to single file
  4. apply ‘crosstalk’ filter with maximum error of 100 – 200 meters to the daily track file. Your goal is to get 150-500kB file, depending on the road network and conditions.
  5. merge daily track files to single trip file
  6. apply ‘crosstalk’ filter with maximum error of 1 – 2 kilometers to the trip file, depending on the road network and conditions.
  7. register at Google Maps to get your API key, as it’s stated in GPS visualizer FAQ
  8. proceed your files with GPS visualizer
  9. donate
  10. enter your Google Maps API key into every saved HTML file from GPSvisualizer.com. You HAVE saved the files, have You?
  11. arrange the presentation of your wanderings
  12. enjoy
$ gpsbabel -i gpx -f 2007-07-31.gpx -i gpx -f 2007-08-01.gpx -i gpx -f 2007-08-02.gpx \
 -i gpx -f 2007-08-03.gpx -i gpx -f 2007-08-04.gpx -i gpx -f 2007-08-05.gpx \
 -i gpx -f 2007-08-06.gpx -i gpx -f 2007-08-07.gpx -i gpx -f 2007-08-08.gpx \
 -i gpx -f 2007-08-09.gpx -i gpx -f 2007-08-10.gpx -i gpx -f 2007-08-11.gpx \
 -i gpx -f 2007-08-13.gpx -i gpx -f 2007-08-14.gpx -i gpx -f 2007-08-15.gpx -o gpx -F full.gpx

$ gpsbabel -i gpx -f full.gpx -x simplify,crosstract,error=0.5k -o gpx -F full-e500m.gpx
$ gpsbabel -i gpx -f full.gpx -x simplify,crosstract,error=1k -o gpx -F full-e1km.gpx
$ gpsbabel -i gpx -f full.gpx -x simplify,crosstract,error=2k -o gpx -F full-e2km.gpx
$ls -al

lrwxrwxrwx 1 ejs ejs        34 2008-02-26 02:38 2007-07-31.gpx -> ../daily/2007-07-31/2007-07-31.gpx
lrwxrwxrwx 1 ejs ejs        34 2008-02-26 02:38 2007-08-01.gpx -> ../daily/2007-08-01/2007-08-01.gpx
lrwxrwxrwx 1 ejs ejs        34 2008-02-26 02:38 2007-08-02.gpx -> ../daily/2007-08-02/2007-08-02.gpx
lrwxrwxrwx 1 ejs ejs        34 2008-02-26 02:38 2007-08-03.gpx -> ../daily/2007-08-03/2007-08-03.gpx
lrwxrwxrwx 1 ejs ejs        34 2008-02-26 02:38 2007-08-04.gpx -> ../daily/2007-08-04/2007-08-04.gpx
lrwxrwxrwx 1 ejs ejs        34 2008-02-26 02:38 2007-08-05.gpx -> ../daily/2007-08-05/2007-08-05.gpx
lrwxrwxrwx 1 ejs ejs        34 2008-02-26 02:38 2007-08-06.gpx -> ../daily/2007-08-06/2007-08-06.gpx
lrwxrwxrwx 1 ejs ejs        34 2008-02-26 02:38 2007-08-07.gpx -> ../daily/2007-08-07/2007-08-07.gpx
lrwxrwxrwx 1 ejs ejs        34 2008-02-26 02:39 2007-08-08.gpx -> ../daily/2007-08-08/2007-08-08.gpx
lrwxrwxrwx 1 ejs ejs        34 2008-02-26 02:39 2007-08-09.gpx -> ../daily/2007-08-09/2007-08-09.gpx
lrwxrwxrwx 1 ejs ejs        34 2008-02-26 02:39 2007-08-10.gpx -> ../daily/2007-08-10/2007-08-10.gpx
lrwxrwxrwx 1 ejs ejs        34 2008-02-26 02:39 2007-08-11.gpx -> ../daily/2007-08-11/2007-08-11.gpx
lrwxrwxrwx 1 ejs ejs        34 2008-02-26 02:39 2007-08-13.gpx -> ../daily/2007-08-13/2007-08-13.gpx
lrwxrwxrwx 1 ejs ejs        34 2008-02-26 02:39 2007-08-14.gpx -> ../daily/2007-08-14/2007-08-14.gpx
lrwxrwxrwx 1 ejs ejs        34 2008-02-26 02:39 2007-08-15.gpx -> ../daily/2007-08-15/2007-08-15.gpx
-rw-r--r-- 1 ejs ejs    186733 2008-02-26 02:59 full-e1km.gpx
-rw-r--r-- 1 ejs ejs    111056 2008-02-26 03:01 full-e2km.gpx
-rw-r--r-- 1 ejs ejs    329869 2008-02-26 02:53 full-e500m.gpx
-rw-r--r-- 1 ejs ejs 107624591 2008-02-26 02:45 full.gpx

by, the way, while processing the last two gpsbabel commands and running Xorg , KDE and Iceape as usual:

top - 02:58:54 up  6:52,  6 users,  load average: 2.66, 1.93, 1.19
Tasks: 121 total,   4 running, 117 sleeping,   0 stopped,   0 zombie
Cpu(s): 99.3%us,  0.7%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:    516620k total,   508624k used,     7996k free,     2004k buffers
Swap:  1052216k total,   157092k used,   895124k free,    93384k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                     
 6173 ejs       25   0 86368  82m  840 R 48.9 16.3   2:59.09 gpsbabel                                                    
 6175 ejs       25   0 86372  82m  840 R 48.9 16.3   1:28.35 gpsbabel   

The resulting GPS data with 1km error is presented here.