published: May 17, 2019, 6:48 p.m.
gpx2dzg - visualize and convert GPS marks to GSSI's proprietary NMEA file, DZG
I discuss a piece of open-source software I wrote for use as a visualization and converter tool for converting GPS and SIR-4000 marks to GSSI's proprietary file format called DZG.
By far the most frustrating datasets I've had to deal with in the past few years are GPR projects that had no GPS input directly in the field. GSSI devices have the ability to record NMEA sentences and correlate them with mark numbers, but only if the GPS you're using has the ability to output NMEA through a serial cable, a finicky and antiquated interface that can be tough to get working. Several of my projects have not had GPS directly wired into the GPR controller, which means the field team must concurrently mark GPS waypoints and user marks on a handheld GPS and the control unit every designated number of meters. There are many ways this can go awry. If a user misses a mark for some reason, if a mark is late, or if the team forgets to take the mark entirely. The process becomes even more frustrating in postprocessing, where the user must go through and manually correlate GPS points with mark points in the GSSI files to make sure there are equal numbers. There often aren't. Then you have to comb through and try to guess where the missing mark is. It's an absolute mess.
For a long time I lamented that there must be a solution, before eventually biting the bullet and creating one myself. The key was understanding the construction of GSSI's DZG file, an ASCII file with both standard and proprietary NMEA sentences. NMEA is a pretty easy format to read, but it's not all that easy to write, as in addition to the actual sentence construction, each NMEA sentence has a hexadecimal checksum at the end, which is a real pain to encode if (like me) you have no idea what you're doing.
While I was in my office late one night earlier this year I stumbled upon
pynmea2, a piece of python brilliance that only comes around every once in a while. I like it for many reasons, all of which are essentially not true of the software I myself write: it's lightweight, doesn't rely on too many other things, and can convert things both in and out of the format it's meant to read. This means, critically, that I just have to supply the numbers for the NMEA sentence, and the hex checksum is essentially an afterthought. Because of this it didn't take me long to figure out what a valuable tool
pynmea2 was. Prior to figuring out it existed, I had been thinking how easy it might be to spoof a GSSI DZG file into existence, if only encoding NMEA strings were easier.
These discoveries led me to begin to plan how a DZG encoder might look. There was not much in it: all I would have to do is figure out some acrobatics to get the scan numbers of each mark recorded by the GSSI control unit to correlate with a latitude, longitude, and time. When I finally got around to writing it, it took just over 24 hours to go from essentially no code to a working release on PyPI. Not bad for someone who knew zero python a few years ago.
gpx2dzg is a pretty efficient piece of code compared to what I've built in the past. It likely still has some GSSI DZX file formats that it can't handle, but I'm always looking for ways to improve, so if you come across something it chokes on, please attach the problematic DZX/GPX files to a new github issue and I'll see what I can do.