December 2017 - mkgmap-dev - The mkgmap lists

Re: [mkgmap-dev] DEM File Format and mkgmap
by Guy Vandervorst 19 Dec '17

19 Dec '17

I have 2 shortcuts for Basecamp: Basecamp and Basecamp 2D. Rightclick on Basecamp 2D gives me: "C:\Program Files (x86)\Garmin\BaseCamp" /Disable3D Starting 2D gives me Basecamp with 3D-maps greyed out. Guy

3 2

Maps in *.gmap format installation directory
by Carlos Dávila 19 Dec '17

19 Dec '17

Is there a unique directory where maps in *.gmap format can be installed? I would like to change my current installers to use gmap format instead of the old format, but I don't know what is the right directory to install them. In my XP virtual box if I copy gmap folder in c:\archivos de programa(program files)\Garmin\Maps maps are read by MapSource, but not by BaseCamp. If I copy gmap folder in C:\Archivos de programa\Garmin\Basecamp\Maps then they are read by BaseCamp, but not by MapSource. And what about more recent versions of Windows?

5 4

Re: [mkgmap-dev] DEM File Format and mkgmap
by osm＠pinns 18 Dec '17

18 Dec '17

Hi Andreas I have been able to make it work (tapping on the arrows) on my Oregon 600 but its very sluggish and at one stage it just freezes. I think you are right ; there is an issue. On 15/12/2017 19:30, Andreas Schmidt wrote: > Hi Frank, > > with ypur great tool I am able to > > - have3d - View in basecamp > - hillshading on Oregon600 > > Thats really amazing ! > > In Oregon there is also a menu item „3D“. Even if the map Shows Up > hillshading,, this function doesnt work. Oregon complains that the map > does Not have enough hight information. > > Could you make it work on your device? > > Andreas > > > > > > > > Gesendet mit der Telekom Mail App > <http://www.t-online.de/service/redir/emailmobilapp_ios_smartphone_footerlin…> > > > > -----Original-Nachricht----- > Von: Frank Stinner <frank.stinner(a)leipzig.de> > Betreff: Re: [mkgmap-dev] DEM File Format and mkgmap > Datum: 08.12.2017, 09:27 Uhr > An: mkgmap-dev(a)lists.mkgmap.org.uk > Hi, > > i must confess, i have never tried to decompilea garmin dem map. I > have only playing with bit-patterns and then lookingfor the result in > mapsource. But i'm sure, that we have different algorithmsor one > algorithm with different special cases. > > My description in the pdf is (only) validfor the „TOPO Deutschland > v3“. Please see the pages 3 and 4. There area few "unknown values". > > For example see "Codierungstyp"(coding type) for a 64x64 tile. I found > in different garmin maps valuesfrom 0 to 6. 0 is the standard. It > seems to be, that the coding type haveno influence of the algorithm, > but i'm not sure. Presumable is this typeonly important for different > interpretations of the height values. BuildDEMFileuse type 1 for areas > with "unknown height". This is in this casethe interpretation for > height=maxvalue. > > Unknown is also the byte on position 0x25in the dem header. In the > „TOPO Deutschland v3“ there we have 0x1 buti have also seen in other > maps 0x0. > > Interesting is also the word in 0x12 in evereyzoomlevel definition > (see page 26 in the pdf). In the „TOPO Deutschlandv3“ thats all 0. But > i have also found 0x100, 0x200, 0x400. Perhaps akind of multiplier for > heights? > > Very important is the bytes 0x0 (and 0x1)in everey zoomlevel > definition. In the „TOPO Deutschland v3“ 0x0 is ever0. 0x1 seems to be > a number as "link" to a maplevel. But i don'treally understand the > sense ot that. We can have 6 maplevels and we cansay the maplevel 2 > and 4 are without dem's? > > In a little demo map from garmin i have see"twins" of > zoomlevelnumbers, one with value 0 on 0x0 and onewith 1 on 0x0. > Perhaps 0x0 is for different and optimized output on pcand gps? > > I think, a decompiler with the algo frommy pdf is working only for > this special case: > 0x15 in dem header should be 0 (meter) > 0x25 in dem header should be 1 > 0x0 and 0x12 in every zoomlevel should 0 > > By the way, have anybody see a dem with valuesin foot? The values in > such a dem should be greater then for a dem in meter.Have anybody > height values with a precision of foot? The conversion fromfoot to > meter would be a first compression step. If garmin really use > "foot",then a different/better compression algo make sense. > > > Frank > > > _______________________________________________ > mkgmap-dev mailing list > mkgmap-dev(a)lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

4 10

Re: [mkgmap-dev] DEM File Format and mkgmap
by Andreas Schmidt 15 Dec '17

15 Dec '17

Hi Frank, with ypur great tool I am able to - have3d - View in basecamp - hillshading on Oregon600 Thats really amazing ! In Oregon there is also a menu item „3D“. Even if the map Shows Up hillshading,, this function doesnt work. Oregon complains that the map does Not have enough hight information. Could you make it work on your device? Andreas Gesendet mit der Telekom Mail App <http://www.t-online.de/service/redir/emailmobilapp_ios_smartphone_footerlin…> -----Original-Nachricht----- Von: Frank Stinner <frank.stinner(a)leipzig.de> Betreff: Re: [mkgmap-dev] DEM File Format and mkgmap Datum: 08.12.2017, 09:27 Uhr An: mkgmap-dev(a)lists.mkgmap.org.uk Hi, i must confess, i have never tried to decompilea garmin dem map. I have only playing with bit-patterns and then lookingfor the result in mapsource. But i'm sure, that we have different algorithmsor one algorithm with different special cases. My description in the pdf is (only) validfor the „TOPO Deutschland v3“. Please see the pages 3 and 4. There area few "unknown values". For example see "Codierungstyp"(coding type) for a 64x64 tile. I found in different garmin maps valuesfrom 0 to 6. 0 is the standard. It seems to be, that the coding type haveno influence of the algorithm, but i'm not sure. Presumable is this typeonly important for different interpretations of the height values. BuildDEMFileuse type 1 for areas with "unknown height". This is in this casethe interpretation for height=maxvalue. Unknown is also the byte on position 0x25in the dem header. In the „TOPO Deutschland v3“ there we have 0x1 buti have also seen in other maps 0x0. Interesting is also the word in 0x12 in evereyzoomlevel definition (see page 26 in the pdf). In the „TOPO Deutschlandv3“ thats all 0. But i have also found 0x100, 0x200, 0x400. Perhaps akind of multiplier for heights? Very important is the bytes 0x0 (and 0x1)in everey zoomlevel definition. In the „TOPO Deutschland v3“ 0x0 is ever0. 0x1 seems to be a number as "link" to a maplevel. But i don'treally understand the sense ot that. We can have 6 maplevels and we cansay the maplevel 2 and 4 are without dem's? In a little demo map from garmin i have see"twins" of zoomlevelnumbers, one with value 0 on 0x0 and onewith 1 on 0x0. Perhaps 0x0 is for different and optimized output on pcand gps? I think, a decompiler with the algo frommy pdf is working only for this special case: 0x15 in dem header should be 0 (meter) 0x25 in dem header should be 1 0x0 and 0x12 in every zoomlevel should 0 By the way, have anybody see a dem with valuesin foot? The values in such a dem should be greater then for a dem in meter.Have anybody height values with a precision of foot? The conversion fromfoot to meter would be a first compression step. If garmin really use "foot",then a different/better compression algo make sense. Frank

1 0

expression not-contained
by demon.box 15 Dec '17

15 Dec '17

hi, in my style, in "relations" I have this code: set route_ref1='$(route_ref1), ${ref|not-contained:, :route_ref1}' | '${ref}'; and I have a way that is member of 3 relations with ref 3, 362 and 363 it seems like the command not-contained process only the first character because the result for route_ref1 is only 3 instead 3, 362, 363 it's a bug of mkmap or it's something desired and known? thanks --enrico -- Sent from: http://gis.19327.n8.nabble.com/Mkgmap-Development-f5324443.html

4 12

bug?
by lig fietser 13 Dec '17

13 Dec '17

Hi, I have added a few hacks on barriers to make more info rendering on my maps. Therefore I misuse the housenumber option to make it visible, like barriers: barrier=* & barrier ~ '(gate|entrance|lift_gate|stile|block|cycle_barrier|kissing_gate|toll_booth|turnstile|full-height_turnstile|border_control|swing_gate|log|bump_gate)' { add mkgmap:housenumber='(${barrier})' ; add mkgmap:street='bicycle:${bicycle}' ; add mkgmap:postal_code='open: ${opening_hours}' ; } In this picture you can see it went wrong somewhere, mkgmap translated the value "full-heigth_turnstile" into "Full # Eight_turnstile. So somewhere in the code mkgmap:housenumber='(${barrier})' -h is converted into #. Is this a bug or is this intended?

3 3

BuildDEMFile: system out of memory
by lig fietser 09 Dec '17

09 Dec '17

lig fietser has shared a OneDrive file with you. To view it, click the link below. <https://1drv.ms/u/s!ApArbaVZkx5rbZCEfL0GVPRjSz8> [https://r1.res.office365.com/owa/prem/images/dc-jpg_20.png]<https://1drv.ms/u/s!ApArbaVZkx5rbZCEfL0GVPRjSz8> atlantic_DEM.jpg<https://1drv.ms/u/s!ApArbaVZkx5rbZCEfL0GVPRjSz8> Hi Frank, I was running your tool on my Atlantic map and got a system out of memory error. The script did continue to build the dem data but in my map a few islands in the center of the map were missing DEM data (see the red area in the attachment, Sao Miguel on the Azores and La Palma and El Hierro on the Canary Islands). Any idea how to solve this issue? The map OFM_Atlantic(13-10-2017_gmap you can download at http://www.openfietsmap.nl/downloads/atlantic DEM tiles I downloaded from http://www.javawa.nl/srtm/index.php?lang=en (viewfinder panoramas) <http://ftp.nluug.nl/pub/maps/openfietsmap/Atlantic/OFM_Atlantic(13-10-2017_…> The command that I used is: for /D %%i in (34351*.*) do (BuildDEMFile.exe -d %%i\%%i.DEM --hgtpath=%hgtpath% --tre=%%i\%%i.TRE --lastcolstd --usedummydata --dlon=0.00027761 )

2 4

Commit r4000: fix possible NPE when POI for overview map has no label
by svn commit 09 Dec '17

09 Dec '17

Version mkgmap-r4000 was committed by gerd on Sat, 09 Dec 2017 fix possible NPE when POI for overview map has no label http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4000

1 0

Re: [mkgmap-dev] DEM File Format and mkgmap
by Frank Stinner 08 Dec '17

08 Dec '17

Hi, i must confess, i have never tried to decompile a garmin dem map. I have only playing with bit-patterns and then looking for the result in mapsource. But i'm sure, that we have different algorithms or one algorithm with different special cases. My description in the pdf is (only) valid for the ?TOPO Deutschland v3?. Please see the pages 3 and 4. There are a few "unknown values". For example see "Codierungstyp" (coding type) for a 64x64 tile. I found in different garmin maps values from 0 to 6. 0 is the standard. It seems to be, that the coding type have no influence of the algorithm, but i'm not sure. Presumable is this type only important for different interpretations of the height values. BuildDEMFile use type 1 for areas with "unknown height". This is in this case the interpretation for height=maxvalue. Unknown is also the byte on position 0x25 in the dem header. In the ?TOPO Deutschland v3? there we have 0x1 but i have also seen in other maps 0x0. Interesting is also the word in 0x12 in everey zoomlevel definition (see page 26 in the pdf). In the ?TOPO Deutschland v3? thats all 0. But i have also found 0x100, 0x200, 0x400. Perhaps a kind of multiplier for heights? Very important is the bytes 0x0 (and 0x1) in everey zoomlevel definition. In the ?TOPO Deutschland v3? 0x0 is ever 0. 0x1 seems to be a number as "link" to a maplevel. But i don't really understand the sense ot that. We can have 6 maplevels and we can say the maplevel 2 and 4 are without dem's? In a little demo map from garmin i have see "twins" of zoomlevelnumbers, one with value 0 on 0x0 and one with 1 on 0x0. Perhaps 0x0 is for different and optimized output on pc and gps? I think, a decompiler with the algo from my pdf is working only for this special case: 0x15 in dem header should be 0 (meter) 0x25 in dem header should be 1 0x0 and 0x12 in every zoomlevel should 0 By the way, have anybody see a dem with values in foot? The values in such a dem should be greater then for a dem in meter. Have anybody height values with a precision of foot? The conversion from foot to meter would be a first compression step. If garmin really use "foot", then a different/better compression algo make sense. Frank

3 2

DEM File Format and mkgmap
by Gerd Petermann 07 Dec '17

07 Dec '17

Hi all, I think I understand now much more about this format. I used Frank Stinners code for BuildDEMFile and his pdf to write a decoder in the display tool. I still fails to read some Garmin DEM files but seems to work with the files produced by BuildDEMFile. My current work flow is to generate DEM files with BuildDEMFile and read them with my decoder. If the decoder fails I can find out why because I can trace step by step what BuildDEMFile does. I guess that I did not yet hit all special cases identified by Frank, but I am making good progress now, so I hope I'll be able to start implementing a DEM encoder in mkgmap soon. Reg. the format: It is quite different to those used in other GARMIN sub files like NOD, NET, RGN etc. In short: The data is organized in tiles, typically 64*64 values. Instead of storing each value Garmin uses some statistics to predict the next value based on the already processed values. This predicted value is subtracted from the real value and only the delta is stored. The better the prediction, the closer to 0 is the delta. The prediction also calculates the error-probability of the prediction and uses this to select the number of bits which are needed to store the value. If the delta is larger than expected the so called BigBin encoding is used. The format to store the delta is optimized for values close to 0, so that many deltas are stored with only 1 or 2 bits. A 2nd trick to save space is to use run length encoding for repeated values. So, the tricky part is to find out exactly how the prediction algo by Garmin works. Frank has done really great work here. For those who want to try the display tool: You have to compile on your own and you also need an installed mgkmap.jar. If that is available a command like this (replace d:\... with your paths) should produce a large txt file: java -ea -Xmx1000m -cp d:\display\dist\display.jar;d:\mkgmap\dist\mkgmap.jar test.display.DemDisplay test.DEM > dem_decode.txt I am now searching for DEM files produced by BuildDEMFile which are not properly processed by display tool. When it fails you'll see a traceback like this Exception in thread "main" java.lang.RuntimeException: did not read all avalaible bytes at test.display.DemDisplay$DemTile.decodeBitStreamData(DemDisplay.java:532) at test.display.DemDisplay$DemSection.decodeBitstream(DemDisplay.java:831) at test.display.DemDisplay.readFileHeader(DemDisplay.java:887) at test.display.DemDisplay.print(DemDisplay.java:845) at test.display.CommonDisplay.display(CommonDisplay.java:171) at test.display.CommonDisplay.display(CommonDisplay.java:196) at test.display.DemDisplay.main(DemDisplay.java:903) Gerd

5 21