- mkgmap-dev - The mkgmap lists

bug?
by lig fietser 13 Dec '17

13 Dec '17

Hi, I have added a few hacks on barriers to make more info rendering on my maps. Therefore I misuse the housenumber option to make it visible, like barriers: barrier=* & barrier ~ '(gate|entrance|lift_gate|stile|block|cycle_barrier|kissing_gate|toll_booth|turnstile|full-height_turnstile|border_control|swing_gate|log|bump_gate)' { add mkgmap:housenumber='(${barrier})' ; add mkgmap:street='bicycle:${bicycle}' ; add mkgmap:postal_code='open: ${opening_hours}' ; } In this picture you can see it went wrong somewhere, mkgmap translated the value "full-heigth_turnstile" into "Full # Eight_turnstile. So somewhere in the code mkgmap:housenumber='(${barrier})' -h is converted into #. Is this a bug or is this intended?

3 3

BuildDEMFile: system out of memory
by lig fietser 09 Dec '17

09 Dec '17

lig fietser has shared a OneDrive file with you. To view it, click the link below. <https://1drv.ms/u/s!ApArbaVZkx5rbZCEfL0GVPRjSz8> [https://r1.res.office365.com/owa/prem/images/dc-jpg_20.png]<https://1drv.ms/u/s!ApArbaVZkx5rbZCEfL0GVPRjSz8> atlantic_DEM.jpg<https://1drv.ms/u/s!ApArbaVZkx5rbZCEfL0GVPRjSz8> Hi Frank, I was running your tool on my Atlantic map and got a system out of memory error. The script did continue to build the dem data but in my map a few islands in the center of the map were missing DEM data (see the red area in the attachment, Sao Miguel on the Azores and La Palma and El Hierro on the Canary Islands). Any idea how to solve this issue? The map OFM_Atlantic(13-10-2017_gmap you can download at http://www.openfietsmap.nl/downloads/atlantic DEM tiles I downloaded from http://www.javawa.nl/srtm/index.php?lang=en (viewfinder panoramas) <http://ftp.nluug.nl/pub/maps/openfietsmap/Atlantic/OFM_Atlantic(13-10-2017_…> The command that I used is: for /D %%i in (34351*.*) do (BuildDEMFile.exe -d %%i\%%i.DEM --hgtpath=%hgtpath% --tre=%%i\%%i.TRE --lastcolstd --usedummydata --dlon=0.00027761 )

2 4

Commit r4000: fix possible NPE when POI for overview map has no label
by svn commit 09 Dec '17

09 Dec '17

Version mkgmap-r4000 was committed by gerd on Sat, 09 Dec 2017 fix possible NPE when POI for overview map has no label http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4000

1 0

Re: [mkgmap-dev] DEM File Format and mkgmap
by Frank Stinner 08 Dec '17

08 Dec '17

Hi, i must confess, i have never tried to decompile a garmin dem map. I have only playing with bit-patterns and then looking for the result in mapsource. But i'm sure, that we have different algorithms or one algorithm with different special cases. My description in the pdf is (only) valid for the ?TOPO Deutschland v3?. Please see the pages 3 and 4. There are a few "unknown values". For example see "Codierungstyp" (coding type) for a 64x64 tile. I found in different garmin maps values from 0 to 6. 0 is the standard. It seems to be, that the coding type have no influence of the algorithm, but i'm not sure. Presumable is this type only important for different interpretations of the height values. BuildDEMFile use type 1 for areas with "unknown height". This is in this case the interpretation for height=maxvalue. Unknown is also the byte on position 0x25 in the dem header. In the ?TOPO Deutschland v3? there we have 0x1 but i have also seen in other maps 0x0. Interesting is also the word in 0x12 in everey zoomlevel definition (see page 26 in the pdf). In the ?TOPO Deutschland v3? thats all 0. But i have also found 0x100, 0x200, 0x400. Perhaps a kind of multiplier for heights? Very important is the bytes 0x0 (and 0x1) in everey zoomlevel definition. In the ?TOPO Deutschland v3? 0x0 is ever 0. 0x1 seems to be a number as "link" to a maplevel. But i don't really understand the sense ot that. We can have 6 maplevels and we can say the maplevel 2 and 4 are without dem's? In a little demo map from garmin i have see "twins" of zoomlevelnumbers, one with value 0 on 0x0 and one with 1 on 0x0. Perhaps 0x0 is for different and optimized output on pc and gps? I think, a decompiler with the algo from my pdf is working only for this special case: 0x15 in dem header should be 0 (meter) 0x25 in dem header should be 1 0x0 and 0x12 in every zoomlevel should 0 By the way, have anybody see a dem with values in foot? The values in such a dem should be greater then for a dem in meter. Have anybody height values with a precision of foot? The conversion from foot to meter would be a first compression step. If garmin really use "foot", then a different/better compression algo make sense. Frank

3 2

DEM File Format and mkgmap
by Gerd Petermann 07 Dec '17

07 Dec '17

Hi all, I think I understand now much more about this format. I used Frank Stinners code for BuildDEMFile and his pdf to write a decoder in the display tool. I still fails to read some Garmin DEM files but seems to work with the files produced by BuildDEMFile. My current work flow is to generate DEM files with BuildDEMFile and read them with my decoder. If the decoder fails I can find out why because I can trace step by step what BuildDEMFile does. I guess that I did not yet hit all special cases identified by Frank, but I am making good progress now, so I hope I'll be able to start implementing a DEM encoder in mkgmap soon. Reg. the format: It is quite different to those used in other GARMIN sub files like NOD, NET, RGN etc. In short: The data is organized in tiles, typically 64*64 values. Instead of storing each value Garmin uses some statistics to predict the next value based on the already processed values. This predicted value is subtracted from the real value and only the delta is stored. The better the prediction, the closer to 0 is the delta. The prediction also calculates the error-probability of the prediction and uses this to select the number of bits which are needed to store the value. If the delta is larger than expected the so called BigBin encoding is used. The format to store the delta is optimized for values close to 0, so that many deltas are stored with only 1 or 2 bits. A 2nd trick to save space is to use run length encoding for repeated values. So, the tricky part is to find out exactly how the prediction algo by Garmin works. Frank has done really great work here. For those who want to try the display tool: You have to compile on your own and you also need an installed mgkmap.jar. If that is available a command like this (replace d:\... with your paths) should produce a large txt file: java -ea -Xmx1000m -cp d:\display\dist\display.jar;d:\mkgmap\dist\mkgmap.jar test.display.DemDisplay test.DEM > dem_decode.txt I am now searching for DEM files produced by BuildDEMFile which are not properly processed by display tool. When it fails you'll see a traceback like this Exception in thread "main" java.lang.RuntimeException: did not read all avalaible bytes at test.display.DemDisplay$DemTile.decodeBitStreamData(DemDisplay.java:532) at test.display.DemDisplay$DemSection.decodeBitstream(DemDisplay.java:831) at test.display.DemDisplay.readFileHeader(DemDisplay.java:887) at test.display.DemDisplay.print(DemDisplay.java:845) at test.display.CommonDisplay.display(CommonDisplay.java:171) at test.display.CommonDisplay.display(CommonDisplay.java:196) at test.display.DemDisplay.main(DemDisplay.java:903) Gerd

5 21

regular expression help needed
by lig fietser 06 Dec '17

06 Dec '17

Hi, I have noticed that this notation seems not to work anymore (I'm sure it worked before): barrier=* & barrier ~ '(lift_gate | toll_booth | border_control | swing_gate | log)' [0x3201 resolution 23] I have a lot like those lines in my style sheets and I dont know why pois like barrier=swing_gate is not catched by the line above anymore? When I add barrier=swing_gate [0x3201 resolution 23] it works as expected.

1 1

Re: [mkgmap-dev] mkgmap-dev Digest, Vol 113, Issue 3
by David 03 Dec '17

03 Dec '17

It is great thing that you progress on this topic. I am waiting for the result for my next map release. Congratulations to F. Stinner and you too. David > Le 3 déc. 2017 à 13:00, mkgmap-dev-request(a)lists.mkgmap.org.uk a écrit : > > Send mkgmap-dev mailing list submissions to > mkgmap-dev(a)lists.mkgmap.org.uk > > To subscribe or unsubscribe via the World Wide Web, visit > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > or, via email, send a message with subject or body 'help' to > mkgmap-dev-request(a)lists.mkgmap.org.uk > > You can reach the person managing the list at > mkgmap-dev-owner(a)lists.mkgmap.org.uk > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of mkgmap-dev digest..." > Today's Topics: > > 1. DEM File Format and mkgmap (Gerd Petermann) > <mime-attachment> > _______________________________________________ > mkgmap-dev mailing list > mkgmap-dev(a)lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

1 0

Re: [mkgmap-dev] gmtool and TDB
by Frank Stinner 01 Dec '17

01 Dec '17

Hi, i hope, the error message was "unbekanntes Argument: new.tdb". It's a stupid error from me in the description: please change from "--mapsource=new.tdb; ..." to "--mapsource=tdb:new.tdb; ...". The dot after -i means the actual working directory. If you start gmtool in directory "g:\mkgmap\Openfietsmap\versions\test\Openfietsmap (BNL).gmap\Product1" this is ok. Then you also don't need the full path for your old TDB. -o defined the output path, in this case only for the new TDB. Frank

2 1

gmtool and TDB
by Frank Stinner 01 Dec '17

01 Dec '17

Hi all, i have gmtool a little bit extended in the hope of simpler creation of TDB's. The main point is, that we have to register a old TDB as input file BEFORE all other files. Then take gmtool all possible properties from the old file and remove or complete the old subfilelist. If we only have additional DEM's can we do: gmtool -i old.tdb -i . --mapsource=new.tdb;noov;notyp;nomdx;nomdr;noinst --hasdem=1 -o . This should be enough for a map with IMG's. If we have a gmap-style map, we need the subfiles in the subdirectorys and we can do: gmtool -i old.tdb -i . --withsubdirs --mapsource=new.tdb;noov;notyp;nomdx;nomdr;noinst --hasdem=1 -o . Hope it worked. Frank

2 1

Re: [mkgmap-dev] too many restrictions
by Frank Stinner 30 Nov '17

30 Nov '17

Hi Gerd, my test-style was only a strong simplified style. > The lines file you posted ignores all lies with line_minorstreet=* If you are interesting for the original style: http://files.mkgmap.org.uk/download/360/style.zip In inc\lines_... are a lot of roads, path and so on. > ... ways with empty tag values, e.g. way 148796438 has line_path3="" That means a way with type "line_path3" and without a name. The rule for mkgmap is defined in inc\inc\lines_smallway and mkgmap should take "type 0x11406 level 2". Frank

2 1