May 2009 - mkgmap-dev - The mkgmap lists

R: [mkgmap-dev] [PATCH v1] - merge nodes to remove evil short arcs
by Marco Certelli 27 May '09

27 May '09

Mark, I'd like to help but I've no way to build mkgmap. Could you please send me a compiled mkgmap.jar to test the patch? Please, don't give up in searching for the BUG. I'm reading the code and I suspect the error is in the NET/NOD info calculations. In fact the issue appears when the 2 collapsing nodes has >= 3 arcs. Nodes with >= 3 arcs are "route nodes" (nodes were routing decision has to be taken) and a NOD record is to be calculated for each one. All NOD records are connected each other with pointers to build the routing network. If two connected routing nodes with 3 arcs each collapses into one with 4 arcs, I guess that all pointers structure shall be recalculated to take care of the missing routing node, and the fact the new node has now 4 arcs instead of 3 implies the NOD record is different and has to be updated. Of course my are just assumptions. Do you know who wrote the parts of the code that calculate the routing structures (src\uk\me\parabola\imgfmt\app\net\...) and how to contact them? In some file I read Robert Vollmert, Steve Ratcliffe,... Ciao, Marco. --- Mer 27/5/09, Mark Burton <markb(a)ordern.com> ha scritto: > Da: Mark Burton <markb(a)ordern.com> > Oggetto: [mkgmap-dev] [PATCH v1] - merge nodes to remove evil short arcs > A: "Development list for mkgmap" <mkgmap-dev(a)lists.mkgmap.org.uk> > Data: Mercoledì 27 maggio 2009, 11:12 > > The attached patch is a first stab at a solution for this > problem. > > Please test. > > It is implemented in an iterative fashion and it should > terminate after > a few passes. If 10 passes are executed it will give up, > please report > if you see it doing that. > > It would be nice if the diagnostics reported the OSM id of > the nodes > rather than their coordinates but I haven't worked out the > best way of > achieving that yet. > > Cheers, > > Mark > > -----Segue allegato----- > > _______________________________________________ > mkgmap-dev mailing list > mkgmap-dev(a)lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

2 1

[PATCH v1] - merge nodes to remove evil short arcs
by Mark Burton 27 May '09

27 May '09

The attached patch is a first stab at a solution for this problem. Please test. It is implemented in an iterative fashion and it should terminate after a few passes. If 10 passes are executed it will give up, please report if you see it doing that. It would be nice if the diagnostics reported the OSM id of the nodes rather than their coordinates but I haven't worked out the best way of achieving that yet. Cheers, Mark

1 0

Re: [mkgmap-dev] creating a fool-proof mapsource installer across different versions
by Marco Certelli 27 May '09

27 May '09

No, just keep the --family-id=639 and eliminate only the --product-id=xxx (elmiminating the product-id, the default is 1) Then the .reg would be: REGEDIT4 [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\OSM_PHIL] "ID"=hex:7f,02 [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\OSM_PHIL\1] "LOC"="C:\\OSM\\" "BMAP"="C:\\OSM\\40000001.img" "TDB"="C:\\OSM\\40000001.tdb" Should work. If you want to add a personal typ file, you must put the family-id,product-id in the typ file (for you, it would be 639,1). see http://ati.land.cz/gps/typdecomp/editor.cgi Remember you cannot contemporary have 2 maps installed with the same family-id or product-id. Ciao. --- Mer 27/5/09, maning sambale <emmanuel.sambale(a)gmail.com> ha scritto: > Da: maning sambale <emmanuel.sambale(a)gmail.com> > Oggetto: Re: [mkgmap-dev] creating a fool-proof mapsource installer across different versions > A: "Development list for mkgmap" <mkgmap-dev(a)lists.mkgmap.org.uk> > Data: Mercoledì 27 maggio 2009, 05:21 > So it should be: > mkgmap syntax: > java -Xmx512m -jar dist/mkgmap.jar > --code-page=1252 --tdbfile \ > --gmapsupp --latin1 --country-abbr=PHI > --country-name=PHILIPPINES \ > --road-name-pois --add-pois-to-areas > --no-poi-address --family-name="OSM_PHIL" > --overview-mapname=40000001 \ > --route --description="OSM_PHILIPPINES" > /path/to/your/osm/file/philippines.osm > > omitting family-id and product-id > > registry key should be: > REGEDIT4 > > [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\40000001] > "ID"=hex:7f,02 > > [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\OSM_PHIL] > [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\OSM_PHIL\1] > "LOC"="C:\\OSM\\" > "BMAP"="C:\\OSM\\40000001.img" > "TDB"="C:\\OSM\\40000001.tdb" > > I'm a bit perplexed with the > [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\40000001] > "ID"=hex:7f,02 > > > > On Tue, May 26, 2009 at 9:36 PM, Marco Certelli <marco_certelli(a)yahoo.it> > wrote: > > > > It might be: > > > > --family-name="OSM_PHIL" > > > > then > > > > > [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\OSM_PHIL] > > > [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\OSM_PHIL\1] > > > > Ciao. > > > > > > --- Mar 26/5/09, Lambertus <osm(a)na1400.info> > ha scritto: > > > >> Da: Lambertus <osm(a)na1400.info> > >> Oggetto: Re: [mkgmap-dev] creating a fool-proof > mapsource installer across different versions > >> A: "Development list for mkgmap" <mkgmap-dev(a)lists.mkgmap.org.uk> > >> Data: Martedì 26 maggio 2009, 15:25 > >> Strange, I have about 200 people > >> downloading maps per day. Most of them will use > the > >> MapSource installer which uses the product-id to > register > >> the map and I never receive complaints about > MapSource > >> crashes. > >> > >> Apollinaris Schoell wrote: > >> > Don't use --product-id > >> > this let's Mapsource crash most of the time > or you > >> can't even start a map after install > >> > > >> > > >> > On 26 May 2009, at 14:05 , maning sambale > wrote: > >> > > >> >> Hi, > >> >> > >> >> Somewhat off-topic with mkgmap but > closely > >> related, I have been > >> >> receiving numerous reports of mapsource > install > >> error of my OSM-PH > >> >> garmin maps > >> >> I create mapsource installer using > winrar's sfx > >> version, for details: > >> >> http://epsg4253.wordpress.com/2009/04/13/diy-garmin-routable-map-from-opens… > >> > >> >> > >> >> My mapsource version 6.11 and it > installs > >> OK. Problems persist on > >> >> other versions like 6.13 and mostly > 6.15.6. > >> >> > >> >> Any advice in creating fool-proof install > for > >> MapSource? > >> >> > >> >> > >> >> My registry key is like this: > >> >> REGEDIT4 > >> >> > >> > [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\40000001] > >> >> "ID"=hex:7f,02 > >> >> > >> > [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\40000001\678] > >> >> "LOC"="C:\\OSM\\" > >> >> "BMAP"="C:\\OSM\\40000001.img" > >> >> "TDB"="C:\\OSM\\40000001.tdb" > >> >> > >> >> mkgmap syntax: > >> >> java -Xmx512m -jar dist/mkgmap.jar > >> --code-page=1252 --tdbfile \ > >> >> --gmapsupp --latin1 --country-abbr=PHI > >> --country-name=PHILIPPINES \ > >> >> --road-name-pois --add-pois-to-areas > >> --no-poi-address --family-id=639 \ > >> >> --family-name="OSM_PHIL" > --product-id=678 > >> --overview-mapname=40000001 \ > >> >> --route --description="OSM_PHILIPPINES" > >> /path/to/your/osm/file/philippines.osm > >> >> - > >> >> - > >> >> cheers, > >> >> maning > >> >> > >> > ------------------------------------------------------ > >> >> "Freedom is still the most radical idea > of all" > >> -N.Branden > >> >> wiki: http://esambale.wikispaces.com/ > >> >> blog: http://epsg4253.wordpress.com/ > >> >> > >> > ------------------------------------------------------ > >> >> > _______________________________________________ > >> >> mkgmap-dev mailing list > >> >> mkgmap-dev(a)lists.mkgmap.org.uk > >> >> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > >> > > >> > > _______________________________________________ > >> > mkgmap-dev mailing list > >> > mkgmap-dev(a)lists.mkgmap.org.uk > >> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > >> > >> _______________________________________________ > >> mkgmap-dev mailing list > >> mkgmap-dev(a)lists.mkgmap.org.uk > >> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > >> > > > > > > > > _______________________________________________ > > mkgmap-dev mailing list > > mkgmap-dev(a)lists.mkgmap.org.uk > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > > > > > > -- > cheers, > maning > ------------------------------------------------------ > "Freedom is still the most radical idea of all" -N.Branden > wiki: http://esambale.wikispaces.com/ > blog: http://epsg4253.wordpress.com/ > ------------------------------------------------------ > _______________________________________________ > mkgmap-dev mailing list > mkgmap-dev(a)lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev >

2 1

Re: [mkgmap-dev] creating a fool-proof mapsource installer across different versions
by Marco Certelli 27 May '09

27 May '09

It might be: --family-name="OSM_PHIL" then [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\OSM_PHIL] [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\OSM_PHIL\1] Ciao. --- Mar 26/5/09, Lambertus <osm(a)na1400.info> ha scritto: > Da: Lambertus <osm(a)na1400.info> > Oggetto: Re: [mkgmap-dev] creating a fool-proof mapsource installer across different versions > A: "Development list for mkgmap" <mkgmap-dev(a)lists.mkgmap.org.uk> > Data: Martedì 26 maggio 2009, 15:25 > Strange, I have about 200 people > downloading maps per day. Most of them will use the > MapSource installer which uses the product-id to register > the map and I never receive complaints about MapSource > crashes. > > Apollinaris Schoell wrote: > > Don't use --product-id > > this let's Mapsource crash most of the time or you > can't even start a map after install > > > > > > On 26 May 2009, at 14:05 , maning sambale wrote: > > > >> Hi, > >> > >> Somewhat off-topic with mkgmap but closely > related, I have been > >> receiving numerous reports of mapsource install > error of my OSM-PH > >> garmin maps > >> I create mapsource installer using winrar's sfx > version, for details: > >> http://epsg4253.wordpress.com/2009/04/13/diy-garmin-routable-map-from-opens… > > >> > >> My mapsource version 6.11 and it installs > OK. Problems persist on > >> other versions like 6.13 and mostly 6.15.6. > >> > >> Any advice in creating fool-proof install for > MapSource? > >> > >> > >> My registry key is like this: > >> REGEDIT4 > >> > [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\40000001] > >> "ID"=hex:7f,02 > >> > [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\40000001\678] > >> "LOC"="C:\\OSM\\" > >> "BMAP"="C:\\OSM\\40000001.img" > >> "TDB"="C:\\OSM\\40000001.tdb" > >> > >> mkgmap syntax: > >> java -Xmx512m -jar dist/mkgmap.jar > --code-page=1252 --tdbfile \ > >> --gmapsupp --latin1 --country-abbr=PHI > --country-name=PHILIPPINES \ > >> --road-name-pois --add-pois-to-areas > --no-poi-address --family-id=639 \ > >> --family-name="OSM_PHIL" --product-id=678 > --overview-mapname=40000001 \ > >> --route --description="OSM_PHILIPPINES" > /path/to/your/osm/file/philippines.osm > >> - > >> - > >> cheers, > >> maning > >> > ------------------------------------------------------ > >> "Freedom is still the most radical idea of all" > -N.Branden > >> wiki: http://esambale.wikispaces.com/ > >> blog: http://epsg4253.wordpress.com/ > >> > ------------------------------------------------------ > >> _______________________________________________ > >> mkgmap-dev mailing list > >> mkgmap-dev(a)lists.mkgmap.org.uk > >> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > > > > _______________________________________________ > > mkgmap-dev mailing list > > mkgmap-dev(a)lists.mkgmap.org.uk > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > > _______________________________________________ > mkgmap-dev mailing list > mkgmap-dev(a)lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev >

2 1

Re: [mkgmap-dev] (almost) duplicated node issue
by Marco Certelli 26 May '09

26 May '09

Hi Mark, I said that a tiny map just made of 6 nodes and 5 arcs reproduce the error. Aren't you able to run in some debug mode mkgmap to see what goes wrong? Or just check the IMG result to catch the fault. You find the tiny OSM map attached. And anyway if you do not find the bug and believe it is right to proceed for the arc deletion, please check the number of arcs of the two nodes before collapsing nodes and deleting the arc: both nodes must have >=3 arcs (including the common short one) for the problem to arise. Should be feasible to check. Ciao, Marco.

1 0

Re: [mkgmap-dev] (almost) duplicated node issue
by Marco Certelli 26 May '09

26 May '09

Well, in the effort to bound the (almost) duplicated node issue, I've made many tests. I've built the simplest map that reproduce the error: 6 nodes and 5 arcs. You can build one by yourself, compile with mkgmap, and it always crashes mapsource routing. The interesting things is that the routing is broken if, and only if, the two collapsing nodes are connected by an arc and are also connected to (at least) 2 others arcs each one. So if you just have a highway with two so close nodes that will collapse during encoding, no routing issue will arise. I attached a png that shows the topology that triggers the problem. For your info, if the nodes distance is 3.4 meters or more, there is no possibility of collapsing in a level 24 map. If they are 1 meter apart, there is 50% possibility of collapsing. More or less. Ciao.

2 1

creating a fool-proof mapsource installer across different versions
by maning sambale 26 May '09

26 May '09

Hi, Somewhat off-topic with mkgmap but closely related, I have been receiving numerous reports of mapsource install error of my OSM-PH garmin maps I create mapsource installer using winrar's sfx version, for details: http://epsg4253.wordpress.com/2009/04/13/diy-garmin-routable-map-from-opens… My mapsource version 6.11 and it installs OK. Problems persist on other versions like 6.13 and mostly 6.15.6. Any advice in creating fool-proof install for MapSource? My registry key is like this: REGEDIT4 [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\40000001] "ID"=hex:7f,02 [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\40000001\678] "LOC"="C:\\OSM\\" "BMAP"="C:\\OSM\\40000001.img" "TDB"="C:\\OSM\\40000001.tdb" mkgmap syntax: java -Xmx512m -jar dist/mkgmap.jar --code-page=1252 --tdbfile \ --gmapsupp --latin1 --country-abbr=PHI --country-name=PHILIPPINES \ --road-name-pois --add-pois-to-areas --no-poi-address --family-id=639 \ --family-name="OSM_PHIL" --product-id=678 --overview-mapname=40000001 \ --route --description="OSM_PHILIPPINES" /path/to/your/osm/file/philippines.osm - - cheers, maning ------------------------------------------------------ "Freedom is still the most radical idea of all" -N.Branden wiki: http://esambale.wikispaces.com/ blog: http://epsg4253.wordpress.com/ ------------------------------------------------------

3 4

Re: [mkgmap-dev] (almost) duplicated node issue
by Marco Certelli 26 May '09

26 May '09

Wow... things get tough... I've three things to say in a single mail: First: I think the encoding is just a 24 bits rounding of the coordinates. You should know at the first run all the nodes that collapses in the same IMG point. But I understand when you say that is complicated to find all the collapsed nodes in a million nodes map. Well, after encoding you should order the nodes per latitude and then order the nodes with the same latitude per longitude. Then all collapsed nodes are in a row. (quicksort is able to sort a vector "in place" without memory wasting: in C I remember I ordered a vector of 1 million elements in few seconds). Second: But maybe the problem is somewere else, or there is a simpler solution then forcing a collapse of nodes: If the IMG specification accepts 2 (or more) independent nodes with the same coordinates (after encoding), maybe it is the routing info that should be encoded in some "smart" way (to be discovered...) that avoids routing engine hangs, infinite loops or errors. Or it could be just a "simple" mkgmap bug: the routing info of the two close nodes does not relates correctly to the two nodes individually. It looks strange to me the fact that the problem in one point creates routing errors rather far in the map. Like a mis-alignemt of routing info vectors vs. node vectors. Third: I made this further experiment you can reproduce: (1) compiled the osm file I've attached some mail ago: Routing error (2) I moved one node of the 2 almost duplicated 10 meters away, compiled the osm file again: NO ROUTING ERROR since the 2 compiled IMG do contain the same nodes, it should be easy for people who know IMG encoding and routing info encoding to discover the difference between the two IMG and understand what happened (a info vector shift, a missing info, a wrong info). Of course if you have a java debugger environment you can check variables before the IMG is even created (I think it's easier) Hope this help. Ciao, Marco. --- Lun 25/5/09, gypakk(a)gmx.eu <gypakk(a)gmx.eu> ha scritto: > Da: gypakk(a)gmx.eu <gypakk(a)gmx.eu> > Oggetto: Re: [mkgmap-dev] (almost) duplicated node issue > A: "Development list for mkgmap" <mkgmap-dev(a)lists.mkgmap.org.uk> > Data: Lunedì 25 maggio 2009, 22:34 > Just a few thoughts... > > 1. Why not giving a special mark to such nodes which have > been generated by a clipping process? Maybe there is own tag > for this? Then you would know that merging is not > appropriate. > > 2. And... please be careful when merging nodes which are > parts of ways with different layers. These could represent > separate floors of a public lift, e.g. > > 3. Is there special a tag for marking those nodes which > must not be merged although they have the same coordinates? > This tag could be used to prevent automatic merging. > > > > -------- Original-Nachricht -------- > > Datum: Mon, 25 May 2009 21:12:07 +0100 > > Von: Mark Burton <markb(a)ordern.com> > > An: mkgmap-dev(a)lists.mkgmap.org.uk > > Betreff: Re: [mkgmap-dev] (almost) duplicated node > issue > > > > > Hi Marco, > > > > > I want to say: duplicated nodes and almost > duplicated nodes are map > > errors (expecially if they describe an highway). We do > not know wich was the > > right intention of the mapper (make routing > possible/impossible). > > > > > > I think that the best we can do when mkgmap finds > 2 nodes that encode in > > the same IMG point (so in both cases: duplicated nodes > or > > almost-duplicated nodes) is to collapse the 2 nodes in > one node exacly as JOSM validator > > would do if they were duplicated. > > > > > > Does it sound good? > > > > > > Just my proposal... > > > > I should think that it would be reasonably easy to > merge close nodes > > using a first pass before any ways are processed. > > > > However, tricky issues remain. Like, how do you cope > with the situation > > of a node being generated by the clipping process and > that node is > > within the critical distance of an existing node on > the same way? > > > > Cheers, > > > > Mark > > _______________________________________________ > > mkgmap-dev mailing list > > mkgmap-dev(a)lists.mkgmap.org.uk > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > _______________________________________________ > mkgmap-dev mailing list > mkgmap-dev(a)lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev >

3 2

Re: [mkgmap-dev] (almost) duplicated node issue
by Marco Certelli 26 May '09

26 May '09

Mark, the easiest patch could be to avoid routing towards the short arc. If, after encoding, an arc is zero length, no routing should be allowd ito that arc (as the arc weren't there...) what do yo think about it? Ciao. --- Mar 26/5/09, Mark Burton <markb(a)ordern.com> ha scritto: > Da: Mark Burton <markb(a)ordern.com> > Oggetto: Re: [mkgmap-dev] (almost) duplicated node issue > A: mkgmap-dev(a)lists.mkgmap.org.uk > Data: Martedì 26 maggio 2009, 09:17 > > I may be wrong here, but I don't believe there is a problem > with 2 > nodes being close together unless they are connected by a > single arc. > > If I am correct, then we "only" have to worry about making > sure that we > don't end up with any arcs that are too short. > > As I mentioned previously, we can carry out an early pass > on the data > and merge nodes to remove short arcs but that still doesn't > help when > the ways are subsequently clipped as the clipping creates > new > nodes which could create new short arcs. It's true that > this > is quite unlikely to happen so it's probably a good idea to > fix up the > bulk of the map and then think about some way around the > latter issue. > > Perhaps it would be better if we did the clipping earlier > but that > would require a big rewrite so I am not keen to go down > that route (ho > ho) > > Cheers, > > Mark > _______________________________________________ > mkgmap-dev mailing list > mkgmap-dev(a)lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev >

1 0

(almost) duplicated node issue
by Marco Certelli 25 May '09

25 May '09

Well, when a node is duplicated (I think, latitudes and longitudes matches up to the last digit), JOSM validator detects it as a map error and offers the possibility to automatically correct the issue. I think there are two cases: (1) the 2 duplicated nodes are not connected by any arc. Well, if JOSM collapses the two nodes in one, it connects each other arcs that weren't connected before. (2) the 2 duplicated nodes are connected by an arc (a zero length arc). Well, if JOSM collapses the two nodes in one, I guess it just deletes the zero length arc. But if the deleted arc was one way, the deletion makes possible routes that weren't possible before I want to say: duplicated nodes and almost duplicated nodes are map errors (expecially if they describe an highway). We do not know wich was the right intention of the mapper (make routing possible/impossible). I think that the best we can do when mkgmap finds 2 nodes that encode in the same IMG point (so in both cases: duplicated nodes or almost-duplicated nodes) is to collapse the 2 nodes in one node exacly as JOSM validator would do if they were duplicated. Does it sound good? Just my proposal... --- Lun 25/5/09, Mark Burton <markb(a)ordern.com> ha scritto: > Da: Mark Burton <markb(a)ordern.com> > Oggetto: Re: R: [mkgmap-dev] (almost) duplicated node issue > A: mkgmap-dev(a)lists.mkgmap.org.uk > Data: Lunedì 25 maggio 2009, 16:26 > > Hi Johann, > > > Would it be an idea to enter a slightly longer > distance for the arc in > > the routing db? > > I tried that, it didn't help. > > > I really dont like such things, as they are only are > just workarounds, > > but it seems the easiest solution until the correct > encoding is known. > > I don't think the issue is the correctness of the encoding, > it's more > to do with the fact that the OSM map can contain nodes that > are closer > together than the Garmin can accept. The thorny issue is > how to munge > the data so that the GPS (or mapsource) doesn't barf but > the map is > still usable (connections not broken, etc.) > > One trivial fix is to delete any way that would otherwise > cause a > problem. That should stop the garminware from barfing but > would not be > ideal from the user's point of view because it would break > the routing. > > Cheers, > > Mark > _______________________________________________ > mkgmap-dev mailing list > mkgmap-dev(a)lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev >

3 2