March 2021 - mkgmap-dev - The mkgmap lists

Re: [mkgmap-dev] Embedding raster maps
by Oliver Eichler 25 Jan '22

25 Jan '22

Hi Steve, thanks for the file. On my device nothing is visible at all. But that's kind of an information, too. There is no way to circle down bugs by simply skipping the extended part. Probably it is still some missing encoding. In the Isle of Man map (and others) there is much more information in the RGN header. Note the values in the "filling bytes". And there seems to be an additional section after the extended point2 section. I called it offset1/length2. --- RGN header --- size 007D type 47 41 52 4D 49 4E 20 52 47 4E | GARMIN RGN byte0x0000000C 01 flag 00 year 2010 month 7 day 21 hour 9 minute 53 second 6 offset1 0000D54E length1 00000000 offset_polyg2 0000D54E length_polyg2 00001A1F byte0x00000025_0x00000038 02 00 00 00 00 00 00 00 FF 00 00 20 FD FC 03 00 00 00 00 00 offset_polyl2 0000EF6D length_polyl2 00000000 byte0x00000041_0x00000054 00 00 00 00 00 00 00 00 3F 00 00 20 FD 0F 00 00 00 00 00 00 offset_point2 0000EF6D length_point2 00000000 byte0x0000005D_0x00000070 00 00 00 00 00 00 00 00 FF 07 00 20 3F F7 3F 00 00 00 00 00 offset2 0000EF6D length2 0000020B unknown 000000E3 This section has a variable length in different sub-files. The unknown value E3 can be E5, too. It doesn't seem to be a record size. I had a look at several sections. They are not completely random, thus there seem to be no "compression". However the patterns do not ring a bell for me. I copied a couple of them into a file and loaded it into my cloud share: https://www.magentacloud.de/share/s429h1z-r6 Maybe one of you has an idea what it's about. I also uploaded my current version of the test map with a single raster map tile. It does not show. Top left coordinate is N46° 57.768 E011° 47.484. Sadly I am stuck here and I am running out of quality time to concentrate on the topic. I also have to update QMapShack's decoding. While reading the mkgmap source code I noticed a lot of new stuff that is not covered by the decoder. Cheers Oliver

4 6

Please try branch check-precomp-sea
by Gerd Petermann 04 May '21

04 May '21

Hi all, I've implemented some basic checks to detect large flooded areas in the precomp-sea data in this new branch. A link to the latest binary (r4579 as of now) is a the bottom of the normal download page : http://www.mkgmap.org.uk/download/mkgmap.html - If PrecompSeaGenerator is used to calculate the sea data and the index check fails a warning is printed and the program ends with rc -1. - If mkgmap is used with a "dubious" sea.zip one or more messages about the problematic areas are written and the program stops with an ExitException unless option --x-check-precomp-sea=0 is given before the first input file. The index test is normally performed once for the whole planet, so it doesn't care about the actual tile boundaries. If e.g. only a part of Australia is flooded and you create a map for the UK you'll still see the errors. BTW: The current code (also in trunk version) doesn't support to use --precomp-sea with different values. Results are rather unpredictable. Gerd

4 4

tile takes very long time to generate
by 7770 29 Apr '21

29 Apr '21

Hi. I observe a situation that one of the very first tiles (often the first or the third) that mkgmap generates takes 20 - 30 minutes to generate, wheres the others take about 30 seconds each. The map data produced by splitter is a total of around 700 files with splitter option --max-nodes=1275000. sea and bouds are used for mkgmap. At first i thought i am running low on memory but changing to max-jobs=1 (instead of the possible max of 2) did not make any change. Is mkgmap doing something specific in the beginning that can explain this long time generating one of the very first tiles? I can provide more details, let me know how i can best collect those details in case i need to set some parameters to have mkgmap to say more about what it is doing. Regards Karl

6 40

logging improvements
by Mike Baggaley 07 Apr '21

07 Apr '21

Dear all, Please find attached a first stab at improving the way error and other messages are written, which at present is an ad hoc mixture of writing to stderr, stdout and using Java logging, with no apparent logical structure. Writing to stderr or stdout is not thread safe and causes problems when max_jobs is greater than 1. When stderr is redirected to a file and threads simultaneously attempt to write to it, you end up with extra numbered log files containing a few lines that you would expect to be in the named log file. As mkgmap supports a configuration file that defines how messages should be logged, it doesn't make much sense writing a significant number of messages to stderr which is not under logging control. There are several diagnostic options that write their output using warning as the severity, but as this is not enabled by default in the logging, they appear to do nothing unless a logging configuration file is also used, which is not intuitive. The --check-routing-island-len option seems somewhat confused as it attempts to combine both reporting and fixing depending on the value supplied. It also requires info level of logging which would include a lot of other not necessarily wanted messages. If you are using it to fix, you don't necessarily also want any messages. In my view the way messages should be handled is as follows: If the message is an informational message about the program itself and is unrelated to processing any data then write it to stdout - this would apply to the output of --version and --help. If the message is an error message about an invalid parameter in the command line, write it to stderr. Once the command line has been processed sufficiently that a file is to be processed then write messages using logging. If a diagnostic option is enabled, the program should automatically log the requested diagnostic messages. This would apply to --check-roundabouts, --check-roundabout-flares, --report-similar-arcs, --check-routing-island-len and --report-dead-ends. The attached patch enhances the existing logging interface by providing the following: A mechanism for writing messages without any formatting getting added and that always get written whatever level of logging is enabled. A mechanism for writing stack trace information to the log file (I don't believe stack traces really have any place in released code but have not attempted to replace any of these). Mostly directs messages as per my above proposal, though there is the occasional place where the best option is not quite obvious to me, as I think some of the code is not part of the main mkgmap program (there are several main functions and some of the code is for testing purposes). Only logs the start and finish times if files are being processed. Outputs requested diagnostic messages regardless of enabled logging levels Changes --report-dead-ends to be off by default, with the value of 1 being the default if --report-dead-ends is specified without a value. Filters duplicated --report-similar-arcs messages and ignores synthesised arcs so that it can be used together with --make-opposite-cycleways. Deprecates --check-routing-island-len, replacing it with --report-routing-islands to handle reporting and --max-routing-island-len to handle fixing. I expect there will be the odd place where others may disagree with the logging level for a message, but no doubt we can come to a consensus. The patch touches quite a few source files, but there are only a few changes that are anything more than altering the message outputting mechanism. There is no change to any map outputs. Cheers, Mike

2 14

Behavior of overview options
by Carlos Dávila 03 Apr '21

03 Apr '21

According to options file, --overview-mapname and --overview-mapnumber parameters should only have an effect "If --tdbfile is enabled...", but they seem to be always taken into account regardless of --tdbfile, as can be checked with command: java -jar mkgmap.jar --gmapsupp --overview-mapname=test --overview-mapnumber=0000001 test.osm Additionally, overview files (osmmap.img, osmmap.tdb and ovm_XXX.img) are created when only --gmapsupp is passed to mkgmap (no --tdb-file), as in the command below: java -jar mkgmap.jar --gmapsupp test.osm I think if you just want to create a gmapsupp file, overview files are not needed at all and they should not be created.

2 3

rules test
by Mike Baggaley 30 Mar '21

30 Mar '21

Hi Gerd, I have been having difficulties with the mkgmap test code which uses the discontinued javax.xml.bind.DatatypeConverter for a single function printHexBinary. Despite downloading a copy of the library and telling Eclipse to use it, it doesn't seem to want to work. I found a replacement bit of code on the Internet, but as I can't compare the two outputs I can't confirm its output is identical. Can you take a look at the attached patch which does away with the dependency on javax and allows the tests to run under the latest version of Eclipse? Thanks, Mike

3 2

style and speed of ferries
by 7770 29 Mar '21

29 Mar '21

Hi. While doing automotive routing on a GPSMAP 66st i found it had very difficult to calculate routes taking a ferry if the land masses were connected. This occurs over the Baltic sea and over the Mediterranean sea, etc. Very often, even if the most direct route is to take a ferry, the GPS unit did not even try. I therefor checked the default style and raised the travel speed of ferries (both car and passenger) one step in the style. I would like to share it in case anyone finds it helpful. In the lines file. From: route=ferry & (motorcar=no | motor_vehicle=no) [0x1b road_class=0 road_speed=0 resolution 23] route=ferry [0x1a road_class=3 road_speed=0 resolution 19] To: route=ferry & (motorcar=no | motor_vehicle=no) [0x1b road_class=0 road_speed=1 resolution 23] route=ferry [0x1a road_class=3 road_speed=1 resolution 19] This makes it easier for the unit to route along a ferry at least if shortest way routing is selected. Shortest time can still make the unit to try to drive around very large waters. I did not want to set the speed higher as road_speed=1 seems to make rather okay time calculation for ferry-trips. Regards Karl

1 0

line/polygon filters fix
by Ticker Berkin 29 Mar '21

29 Mar '21

Hi Gerd I was trying to diagnose a problem with a repeating points in polylines as reported by GPSMapEdit and found a problem in RemoveObsoletePointsFilter where it duplicates a point. Also in this and/or RoundCoordsFilter I've made some changes: 1/ stop the chain when polygons get too small 2/ keep polygons closed with the same first/last point 3/ slight logic tidy-up 4/ add a couple of debug lines to be consistent Patch attached. Actually these didn't make any difference to the repeating points problem. This happens when there are enough unused bits in the last byte of the polyline bitsteam to represent an extra point. I can't see any good way of stopping this. Ticker

2 12

Commit r4620: inspired by filters_v2.patch from Ticker Berkin:
by svn commit 28 Mar '21

28 Mar '21

Version mkgmap-r4620 was committed by gerd on Sun, 28 Mar 2021 inspired by filters_v2.patch from Ticker Berkin: - add missing continue statement - remove dead code - add debug messages for special cases - improve formatting http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4620

1 0

Commit r4619: reformatting and minor code improvements, no functional change (inspired by filters_v2.patch from Ticker Berkin)
by svn commit 28 Mar '21

28 Mar '21

Version mkgmap-r4619 was committed by gerd on Sun, 28 Mar 2021 reformatting and minor code improvements, no functional change (inspired by filters_v2.patch from Ticker Berkin) http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4619

1 0