May 2020 - mkgmap-dev - The mkgmap lists

Commit r4493: fix some sonarlint issues, simplify code (no change in output expected)
by svn commit 21 May '20

21 May '20

Version mkgmap-r4493 was committed by gerd on Thu, 21 May 2020 fix some sonarlint issues, simplify code (no change in output expected) http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4493

1 0

New User: Inconsistent results - help!
by Andrew Morris 18 May '20

18 May '20

(Sorry for the possible repeat, the previous message was sent from the wrong e-mail address) Recently I updated my GPS unit and decided to also update my maps (2011). After looking at Garmin's prices I decided to take a look at OSM, and things have come a long way since I looked last (2012?). I did some reading and headed down the road. Things seem to be working, but I'm getting inconsistent results - aka the gmapsupp.img file is not the same each time I create it. (Note: My Java isn't the latest-greatest, but it should be sufficient for this process: java version "1.8.0_202") Here's what I've done: 1) Downloaded the latest versions of mkgmap (r4486) and splitter (r597). I also downloaded and compiled osmconvert and osmfilter. I placed them in their own directory: (I'm a sysadmin by trade, this is what I do...) ls -l /opt/mkgmap/ /opt/mkgmap/lib/ /opt/mkgmap/bin/ /opt/mkgmap/: -rw-rw-r-- 1 bin bin 17989 May 6 02:37 LICENCE -rw-rw-r-- 1 bin bin 2784 May 6 02:37 README drwxr-xr-x 2 bin bin 4096 May 7 13:58 bin drwxrwxr-x 3 bin bin 4096 May 6 02:38 doc drwxrwxr-x 6 bin bin 4096 May 6 02:37 examples drwxrwxr-x 2 bin bin 4096 May 6 02:37 lib -rw-rw-r-- 1 bin bin 2350418 May 6 02:37 mkgmap.jar -rw-rw-r-- 1 bin bin 413879 Jan 30 07:11 splitter.jar /opt/mkgmap/bin/: -rwxr-xr-x 1 bin bin 182128 May 7 13:58 osmconvert -rwxr-xr-x 1 bin bin 100640 May 7 13:58 osmfilter /opt/mkgmap/lib/: -rw-rw-r-- 1 bin bin 475284 May 6 02:37 fastutil-6.5.15-mkg.1b.jar -rw-rw-r-- 1 bin bin 132654 May 6 02:37 osmpbf-1.3.3.jar -rw-rw-r-- 1 bin bin 533455 May 6 02:37 protobuf-java-2.5.0.jar -rw-rw-r-- 1 bin bin 120069 Jan 30 07:11 xpp3-1.1.4c.jar 2) I downloaded an OSM extract from Geofabrik, and using splitter carved up my canada-latest.osm.pbf into pieces. As per the wiki I used osmconvert and osmfilter to build the bounds data. These steps are consistent and repeatable. Here are the steps I've taken just in case they are consistently wrong... osmconvert canada-latest.osm.pbf -o=canada-latest.o5m osmfilter canada-latest.o5m \ --keep-nodes= \ --keep-ways-relations="boundary=administrative =postal_code postal_code=" \ -o=canada-latest-boundries.o5m mkdir bounds java -cp /opt/mkgmap/mkgmap.jar \ uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryPreprocessor \ canada-latest-boundries.o5m bounds mkdir work ( cd work ; java -jar /opt/mkgmap/splitter.jar ../canada-latest.osm.pbf ) The final step (afaik) is to combine the split files and the bounds information into a single gmapsupp.img file. ( cd work ; java -jar /opt/mkgmap/mkgmap.jar \ --route --add-pois-to-areas --bounds=../bounds \ --series-name='Canada OSM 2020' \ --family-name='Canada OSM 2020' \ --area-name='Canada OSM 2020' \ --country-name='Canada' --country-abbr='CA' \ --description='OpenStreetMap data for Garmin (2020)' \ --index --gmapsupp 6324*.osm.pbf ) This always runs to completion with MapFailedExceptions and ExitExceptions returning 0. Approximate time 19 minutes (4 core, 32G). I am assuming that the series-name, family-name, etc are just different tags used by different Garmin units. I haven't tried uploading it to my GPS unit yet as I'm concerned about the inconsistencies. Here are the results from multiple runs (osm/bounds/split data is all identical): 1260027904 May 7 16:54 gmapsupp.img 1260027904 May 7 17:18 gmapsupp2.img 1259929600 May 14 10:49 gmapsupp3.img 1259929600 May 14 11:44 gmapsupp4.img md5sum -b *.img 34053990aecc68a257e95115b2bb3f45 *gmapsupp.img 91e3d3e9f0012672da65dead8c752dd9 *gmapsupp2.img 821234ad26e17d0135bf613cf9faf161 *gmapsupp3.img abf39cef1430c4cbd74795fa11b571b8 *gmapsupp4.img Is this normal? If so why? Thank you for any help you can provide!

4 4

new feature request, New style command 'goto'
by Joris Bo 16 May '20

16 May '20

Hello, I finished a fancy but complex 'access evaluator' together with unittests prior to set the { mkgmap:car/truck/bicycle=yes/no/toll } It was not easy to keep it maintainable. The complexity makes it more difficult to keep in mind the impact of existing sections. The current mkgmap technique is processing line by line and first stops after a match is found resulting in an object to be visible in the map. Even if the match is a 1000 lines down the road (and even when I know its useless to try 999 of them) it will process them all, enabling the chance of a false positive (in between). I was wondering if it would be difficult to implement a new feature called : "goto' which immediately jumps to a certain section and continues from there. It is like the batch command 'goto, gosub'. Today for example I need to exclude rule after rule the possible existence of certain tags which are tested later on, to avoid an unexpected early match. It would be easier to just skip these parts and jump to the 'right section'. Because the software today also includes rules on the fly, and also uses 'if () then ... end' exclusions and also jumps to the finalizer if a match is found, it maybe just as simple as jump to another section? This will help a lot. It even implements the possibility of subroutines which I now do sequential bij calling the "include file" mechanism 9 times in a row for every transport type. Just a thought, it also works without this, but it would help a lot spending less debugging time. Especially after re-arranging sections which today for sure will end up with unexpected results. For example: include 'inc/myaccessrules'; myvariable:notransporttypeallowedatall = true {goto mynonroutablehighwaysection} <myhighwaysection> #don't bother the private and non-routable, it is already managed elsewhere building = * {goto mybuildingsection} natural = * {goto mynaturalsection} <myshopsection> shop = * & building !=* & natural !=* & amenity!=restaurant {name '${name}' | '${ housename}'} [0x0400 resolution 12-22 continue] <mybuildingsection> building = * {name '${name}' | '${addr:housename}'} [0x05 resolution 12-22 continue] <finalize> #reserved section name Kind reagards Jorisbo(a)hotmail.com<mailto:Jorisbo@hotmail.com>

1 0

Missing osm tags in the echo output
by Joris Bo 15 May '20

15 May '20

Hello, When debugging my map I encountered missing tags in the echotags output which are available at openstreetmap.org And also in the geofabric download because after assigning the tag to a variable in my style and after that call the echotags statement they do appear in the output. So I was thinking that mkgmap will only echo a part of the tags and not just 'all'? Can i do something about this behavior? Kind regards Joris Met vriendelijke groet, Joris Bovens +31 6 53233898 jorisbo(a)hotmail.com<mailto:jorisbo@hotmail.com>

4 4

nearby-POI: What to do with several POI arranged on a circle?
by Gerd Petermann 08 May '20

08 May '20

Hi all, while doing further tests I've found this edge case (see attached file) Draw a circle with a diameter of ~56m and tag all nodes amenity=bank. The default style has amenity=bank [0x2f06 resolution 24] With --nearby-poi-rules=0x2f06:30:merge-at-mid-point one might expect a single POI in the middle of the circle, but the result is different: Since there is no POI near the center of the circle the current algo finds 4 groups, all close to the edge of the circle. I think for merge-at-mid-point I need a different algo. I'm looking at https://en.wikipedia.org/wiki/Cluster_analysis now. Gerd

6 10

Open question reg. new nearby-poi options
by Gerd Petermann 07 May '20

07 May '20

Hi all, some days ago I expressed my concerns that the new option might have a bad influence on routing or the dead-end check. I was mistaken, there was and is no effect on routing or the results of the dead-end checks because the option only effects the POI, which is a different data structure. Sorry for the confusion. Gerd -- Sent from: http://gis.19327.n8.nabble.com/Mkgmap-Development-f5324443.html

2 6

nearby-POI branch: please ignore r4489 and r4490
by Gerd Petermann 07 May '20

07 May '20

Hi all, sorry, see http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4491 Gerd

1 0

new branch nearby-POI
by Gerd Petermann 06 May '20

06 May '20

Hi all, it seems I underestimated the complexity of the new options implemented by Mikes Patch. Some errors as well as new interesting ideas came up so I decided to revert trunk (r4486 is like r4484) and and create a new branch. So, please check the bottom on http://www.mkgmap.org.uk/download/mkgmap.html for new versions of the nearby-POI branch. I've committed my latest work on the code, but documentation is not up-to-date. Unfortunately the generated docu at http://www.mkgmap.org.uk/doc/options is that for the branch. Today I don't have more time for this, so please try http://www.mkgmap.org.uk/download/mkgmap-nearby-POI-r4490.zip Please only report problems when you can reproduce them with r4490 or later. On the TODO list: - try to understand the new ideas - unit tests - docu Gerd

1 0

Commit r4486: revert changes r4485 nearby-POI handling needs more thinking
by svn commit 06 May '20

06 May '20

Version mkgmap-r4486 was committed by gerd on Wed, 06 May 2020 revert changes r4485 nearby-POI handling needs more thinking http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4486

1 0

Re: [mkgmap-dev] nearby POIs
by Gerd Petermann 06 May '20

06 May '20

Hi all, thanks for testing. I've just noticed that the jar file contained a modified points file, but I guess you all used your own style files. I thought about the possible effects on routing and found none. The new algo works like this: 1) Collect all generated POI and the corresponding nodes before adding them to the map 2) after all style rules were processed and WrongAngleFixer has moved points (and POI) to fix visible wrong angles the POI are deduplicated. This means they are - grouped by type and name - each group is processed so that + the first matching rule is applied using the given radius: Iterate as long as there are multiple POI * for each POI, build a group of all nearby POI * select the biggest group * calculate the center of that group * select the POI that is closest to the center * apply the action to the other POI of the group, maybe move the selected POI to the middle * remove all POI handled in this group Known problems: - The new implementation will not work well for the /all case because a group with different names is not detected as such - maybe identical POI are not removed with the delete-name case - Maybe there is an unwanted effect on rendering when WrongAngleFixer moves a point and the corresponding POI is not moved. This already happens sometimes and should be no big problem. If I find no solution for the first 2 problems today I'll revert trunk version and create a branch for further development. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces(a)lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap(a)jagit.co.uk> Gesendet: Dienstag, 5. Mai 2020 12:03 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] nearby POIs Hi Gerd That got rid of my multiple benches. Just looking at the log for the few other POI it also got rid of. It would be useful for the log message to include the distance between the POIs Ticker On Tue, 2020-05-05 at 09:19 +0000, Gerd Petermann wrote: > Hi all, > > attached is a patch that seems to work. I've kept the logging as > memory is not such a big problem. > I've not yet tested special cases like duplicated barrier nodes or a > group of nodes where only some are within the tile boundary. > > Please check if your problems are solved with this version, a binary > is here: > http://files.mkgmap.org.uk/download/469/mkgmap.jar > > Gerd > > ________________________________________ > Von: mkgmap-dev <mkgmap-dev-bounces(a)lists.mkgmap.org.uk> im Auftrag > von Joris Bo <jorisbo(a)hotmail.com> > Gesendet: Dienstag, 5. Mai 2020 08:32 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] nearby POIs > > Hi, > > > > I have been testing the nearby poi rules as well with 4485 > > A while I struggled thinking misconfiguration because I see nothing > happening but then after extending the ranges, an occasional poi > disappears. (toilet 0x2a0d, busstop 0x2f17, custom lift-gate 0x6905) > > > > On the cmd line > > --nearby-poi-rules=*/unnamed:100:delete-poi,*/named:100:delete-poi > > > > On my densed big city area (Antwerpen) In that case I expect maybe a > couple of thousand benches, trees and bicycle parkings disappear > which do not. > > > > Can I help testing better by increasing some logging? How could I do > that? > > > > Kind regards, joris > > > > Silo’s (0x2a19) don’t disappear / N51° 11.992' E4° 24.916' > > An occasional lift gate (0x6905) does disappear > > [cid:image001.jpg@01D622B7.BA7DE260] > > > > Bicycle stands (0x2f18): don’t disappear > > N51° 13.685' E4° 24.281' > > [cid:image002.jpg@01D622B7.BA7DE260] > > > > > > -----Oorspronkelijk bericht----- > Van: mkgmap-dev <mkgmap-dev-bounces(a)lists.mkgmap.org.uk> Namens > Ticker Berkin > Verzonden: maandag 4 mei 2020 16:13 > Aan: mkgmap development <mkgmap-dev(a)lists.mkgmap.org.uk> > Onderwerp: Re: [mkgmap-dev] nearby POIs > > > > Hi Mike > > > > No, I just have --nearby-poi-rules=0x6605:30 > > > > Gerd has explained why I see a message from NearbyPoiHandler about a > POI for that wasn't 0x6605 - the logging of the removal of an exact > duplicate has moved from StyledConverter > > > > Ticker > > > > On Mon, 2020-05-04 at 14:35 +0100, Mike Baggaley wrote: > > > Hi Ticker, have you got a --nearby-poi-rules-config option as well? > > > > > > Cheers, > > > Mike > > > > > > -----Original Message----- > > > From: Ticker Berkin [mailto:rwb-mkgmap@jagit.co.uk] > > > Sent: 04 May 2020 11:33 > > > To: Development list for mkgmap <mkgmap-dev(a)lists.mkgmap.org.uk<mai > > lto:mkgmap-dev@lists.mkgmap.org.uk>> > > > Subject: Re: [mkgmap-dev] nearby POIs > > > > > > OK, looking up the # as a way instead gives a polygon that matches > > the > > > description and would generate a typ code as per the message > > > > > > I was more worried about it logging messages about POI with typ > > codes > > > that I hadn't mentioned in rules and not saying anything about the > > one > > > I had. > > > > > > I get the same behaviour with just --nearby-poi-rules=0x6605:30 > > > > > > Ticker > > > > > > On Mon, 2020-05-04 at 10:14 +0000, Gerd Petermann wrote: > > > > Hi Ticker, > > > > > > > > the new option doesn't work well, Mike and I are working on it. > > > See > > > > http://gis.19327.n8.nabble.com/nearby-POIs-tp5962630p5964759.html > > > > The log messages are for POIs generated from areas, so the > > > > "generated from" ids are either ways or multipolygon relations. > > > > > > > > Gerd > > > > > > > > > > > > ________________________________________ > > > > Von: mkgmap-dev <mkgmap-dev-bounces(a)lists.mkgmap.org.uk<mailto: > > > mkgmap-dev-bounces(a)lists.mkgmap.org.uk>> im Auftrag > > > > von Ticker Berkin <rwb-mkgmap(a)jagit.co.uk<mailto: > > > rwb-mkgmap(a)jagit.co.uk>> > > > > Gesendet: Montag, 4. Mai 2020 12:04 > > > > An: mkgmap development > > > > Betreff: [mkgmap-dev] nearby POIs > > > > > > > > Hi Mike & Gerd > > > > > > > > I'm trying --nearby-poi-rules=0x6605:30:merge-at-mid-point > > > > > > > > with, in points: > > > > > > > > amenity=bench [0x6605 resolution 24 continue] > > > > > > > > and the resultant map is showing benches close to each other (< > > > 10 > > > > m) > > > > > > > > and, in the log file I'm getting messages like: > > > > > > > > INFO: uk.me.parabola.mkgmap.osmstyle.NearbyPoiHandler > > > > ../mapHants/74210001.osm.pbf: Removed duplicate POI with type > > > > 0x2d02 > > > > The Dungeon for element generated from 159255400 > > > > INFO: uk.me.parabola.mkgmap.osmstyle.NearbyPoiHandler > > > > ../mapHants/74210001.osm.pbf: Removed duplicate POI with type > > > > 0x2a07 > > > > Wah Ying for element generated from 468906008 > > > > > > > > The "generated from" number doesn't seem to correspond to a > > > relevant > > > > OSM node and the message doesn't have the trailing "and moved..." > > > > that > > > > I would have expected. > > > > > > > > Ticker > > > > _______________________________________________ > > > > mkgmap-dev mailing list > > > > mkgmap-dev(a)lists.mkgmap.org.uk<mailto: > > > mkgmap-dev(a)lists.mkgmap.org.uk> > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > > > > _______________________________________________ > > > > mkgmap-dev mailing list > > > > mkgmap-dev(a)lists.mkgmap.org.uk<mailto: > > > mkgmap-dev(a)lists.mkgmap.org.uk> > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > > > > > > > > _______________________________________________ > > mkgmap-dev mailing list > > mkgmap-dev(a)lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > _______________________________________________ > mkgmap-dev mailing list > mkgmap-dev(a)lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev(a)lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

3 3