May 2009 - mkgmap-dev - The mkgmap lists

some comments on mkgmap default style
by maning sambale 06 Feb '10

06 Feb '10

Hi, some observation about the default style: 1. Coastlines are shown as thin blue lines, is it possible to render coastline as polygon, where sea areas are shaded as blue. A work around I did before is to manually edit all sea-areas and then add a natural=water tag (bad data edit) . 2. Administrative boundaries clutter the map. They appear like roads and can be confusing while driving. Suggestion is to make the admin boundaries thinner. Hope it helps improving the default stylesheet. -- cheers, maning ------------------------------------------------------ "Freedom is still the most radical idea of all" -N.Branden wiki: http://esambale.wikispaces.com/ blog: http://epsg4253.wordpress.com/ ------------------------------------------------------

5 8

Re: [mkgmap-dev] creating a fool-proof mapsource installer across different versions
by Marco Certelli 19 Jun '09

19 Jun '09

Well, try before enjoing. Actually I take a slighly different approach to mapsoruce install. I use mkgmap with something like: --overview-mapname=40000000 --mapname=40000001 and then in the registry ... "BMAP"="C:\\OSM\\40000000.img" "TDB"="C:\\OSM\\40000000.tdb" ... Try it also. Have a nice day, Marco. --- Mer 27/5/09, maning sambale <emmanuel.sambale(a)gmail.com> ha scritto: > Da: maning sambale <emmanuel.sambale(a)gmail.com> > Oggetto: Re: [mkgmap-dev] creating a fool-proof mapsource installer across different versions > A: "Development list for mkgmap" <mkgmap-dev(a)lists.mkgmap.org.uk> > Data: Mercoledì 27 maggio 2009, 10:28 > Marko, > > You made my day! Will test this tonight. Thanks > thanks! > > On Wed, May 27, 2009 at 3:57 PM, Marco Certelli <marco_certelli(a)yahoo.it> > wrote: > > > > No, just keep the --family-id=639 and eliminate only > the --product-id=xxx (elmiminating the product-id, the > default is 1) > > > > Then the .reg would be: > > > > REGEDIT4 > > > > > [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\OSM_PHIL] > > "ID"=hex:7f,02 > > > > > [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\OSM_PHIL\1] > > "LOC"="C:\\OSM\\" > > "BMAP"="C:\\OSM\\40000001.img" > > "TDB"="C:\\OSM\\40000001.tdb" > > > > Should work. > > > > If you want to add a personal typ file, you must put > the family-id,product-id in the typ file (for you, it would > be 639,1). see http://ati.land.cz/gps/typdecomp/editor.cgi > > > > Remember you cannot contemporary have 2 maps installed > with the same family-id or product-id. > > > > Ciao. > > > > > > --- Mer 27/5/09, maning sambale <emmanuel.sambale(a)gmail.com> > ha scritto: > > > >> Da: maning sambale <emmanuel.sambale(a)gmail.com> > >> Oggetto: Re: [mkgmap-dev] creating a fool-proof > mapsource installer across different versions > >> A: "Development list for mkgmap" <mkgmap-dev(a)lists.mkgmap.org.uk> > >> Data: Mercoledì 27 maggio 2009, 05:21 > >> So it should be: > >> mkgmap syntax: > >> java -Xmx512m -jar dist/mkgmap.jar > >> --code-page=1252 --tdbfile \ > >> --gmapsupp --latin1 --country-abbr=PHI > >> --country-name=PHILIPPINES \ > >> --road-name-pois --add-pois-to-areas > >> --no-poi-address --family-name="OSM_PHIL" > >> --overview-mapname=40000001 \ > >> --route --description="OSM_PHILIPPINES" > >> /path/to/your/osm/file/philippines.osm > >> > >> omitting family-id and product-id > >> > >> registry key should be: > >> REGEDIT4 > >> > >> > [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\40000001] > >> "ID"=hex:7f,02 > >> > >> > [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\OSM_PHIL] > >> > [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\OSM_PHIL\1] > >> "LOC"="C:\\OSM\\" > >> "BMAP"="C:\\OSM\\40000001.img" > >> "TDB"="C:\\OSM\\40000001.tdb" > >> > >> I'm a bit perplexed with the > >> > [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\40000001] > >> "ID"=hex:7f,02 > >> > >> > >> > >> On Tue, May 26, 2009 at 9:36 PM, Marco Certelli > <marco_certelli(a)yahoo.it> > >> wrote: > >> > > >> > It might be: > >> > > >> > --family-name="OSM_PHIL" > >> > > >> > then > >> > > >> > > >> > [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\OSM_PHIL] > >> > > >> > [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\OSM_PHIL\1] > >> > > >> > Ciao. > >> > > >> > > >> > --- Mar 26/5/09, Lambertus <osm(a)na1400.info> > >> ha scritto: > >> > > >> >> Da: Lambertus <osm(a)na1400.info> > >> >> Oggetto: Re: [mkgmap-dev] creating a > fool-proof > >> mapsource installer across different versions > >> >> A: "Development list for mkgmap" <mkgmap-dev(a)lists.mkgmap.org.uk> > >> >> Data: Martedì 26 maggio 2009, 15:25 > >> >> Strange, I have about 200 people > >> >> downloading maps per day. Most of them > will use > >> the > >> >> MapSource installer which uses the > product-id to > >> register > >> >> the map and I never receive complaints > about > >> MapSource > >> >> crashes. > >> >> > >> >> Apollinaris Schoell wrote: > >> >> > Don't use --product-id > >> >> > this let's Mapsource crash most of > the time > >> or you > >> >> can't even start a map after install > >> >> > > >> >> > > >> >> > On 26 May 2009, at 14:05 , maning > sambale > >> wrote: > >> >> > > >> >> >> Hi, > >> >> >> > >> >> >> Somewhat off-topic with mkgmap > but > >> closely > >> >> related, I have been > >> >> >> receiving numerous reports of > mapsource > >> install > >> >> error of my OSM-PH > >> >> >> garmin maps > >> >> >> I create mapsource installer > using > >> winrar's sfx > >> >> version, for details: > >> >> >> http://epsg4253.wordpress.com/2009/04/13/diy-garmin-routable-map-from-opens… > >> >> > >> >> >> > >> >> >> My mapsource version 6.11 and > it > >> installs > >> >> OK. Problems persist on > >> >> >> other versions like 6.13 and > mostly > >> 6.15.6. > >> >> >> > >> >> >> Any advice in creating > fool-proof install > >> for > >> >> MapSource? > >> >> >> > >> >> >> > >> >> >> My registry key is like this: > >> >> >> REGEDIT4 > >> >> >> > >> >> > >> > [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\40000001] > >> >> >> "ID"=hex:7f,02 > >> >> >> > >> >> > >> > [HKEY_LOCAL_MACHINE\SOFTWARE\Garmin\MapSource\Families\40000001\678] > >> >> >> "LOC"="C:\\OSM\\" > >> >> >> "BMAP"="C:\\OSM\\40000001.img" > >> >> >> "TDB"="C:\\OSM\\40000001.tdb" > >> >> >> > >> >> >> mkgmap syntax: > >> >> >> java -Xmx512m -jar > dist/mkgmap.jar > >> >> --code-page=1252 --tdbfile \ > >> >> >> --gmapsupp --latin1 > --country-abbr=PHI > >> >> --country-name=PHILIPPINES \ > >> >> >> --road-name-pois > --add-pois-to-areas > >> >> --no-poi-address --family-id=639 \ > >> >> >> --family-name="OSM_PHIL" > >> --product-id=678 > >> >> --overview-mapname=40000001 \ > >> >> >> --route > --description="OSM_PHILIPPINES" > >> >> /path/to/your/osm/file/philippines.osm > >> >> >> - > >> >> >> - > >> >> >> cheers, > >> >> >> maning > >> >> >> > >> >> > >> > ------------------------------------------------------ > >> >> >> "Freedom is still the most > radical idea > >> of all" > >> >> -N.Branden > >> >> >> wiki: http://esambale.wikispaces.com/ > >> >> >> blog: http://epsg4253.wordpress.com/ > >> >> >> > >> >> > >> > ------------------------------------------------------ > >> >> >> > >> _______________________________________________ > >> >> >> mkgmap-dev mailing list > >> >> >> mkgmap-dev(a)lists.mkgmap.org.uk > >> >> >> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > >> >> > > >> >> > > >> _______________________________________________ > >> >> > mkgmap-dev mailing list > >> >> > mkgmap-dev(a)lists.mkgmap.org.uk > >> >> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > >> >> > >> >> > _______________________________________________ > >> >> mkgmap-dev mailing list > >> >> mkgmap-dev(a)lists.mkgmap.org.uk > >> >> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > >> >> > >> > > >> > > >> > > >> > > _______________________________________________ > >> > mkgmap-dev mailing list > >> > mkgmap-dev(a)lists.mkgmap.org.uk > >> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > >> > > >> > >> > >> > >> -- > >> cheers, > >> maning > >> > ------------------------------------------------------ > >> "Freedom is still the most radical idea of all" > -N.Branden > >> wiki: http://esambale.wikispaces.com/ > >> blog: http://epsg4253.wordpress.com/ > >> > ------------------------------------------------------ > >> _______________________________________________ > >> mkgmap-dev mailing list > >> mkgmap-dev(a)lists.mkgmap.org.uk > >> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > >> > > > > > > > > _______________________________________________ > > mkgmap-dev mailing list > > mkgmap-dev(a)lists.mkgmap.org.uk > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > > > > > > -- > cheers, > maning > ------------------------------------------------------ > "Freedom is still the most radical idea of all" -N.Branden > wiki: http://esambale.wikispaces.com/ > blog: http://epsg4253.wordpress.com/ > ------------------------------------------------------ > _______________________________________________ > mkgmap-dev mailing list > mkgmap-dev(a)lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev >

3 2

Either mkgmap error or splitter error
by Ralf Reimann 05 Jun '09

05 Jun '09

Great work of you in development of this software. But I have a problem in creating a map of the Netherlands. Mkgmap reports an error, because the map is too large. So I used splitter with the --max-nodes option and decreased the number down to 1000000, but now I get an error in splitter (Relation in too many areas). Ignoring this error will cause a problem in Mapsource, zooming in and out doesn't work correct (no screen updating) and some tiles are completely empty. In the forum I couldn't get an answer for this problem. My only solution is to delete all tracks POIs and paths in mkgmap style-file to reduce the osm-data, but in this way I lose a lot of information, what I don't want to do. Can you tell me what I have to do for creating an error free map with all available osm data? Greetings, Ralf

11 14

[PATCH] Alpha code for pedestrian routing in plazas
by Clinton Gladstone 01 Jun '09

01 Jun '09

Hello, It was previously mentioned on this list that pedestrian routing did not work when encountering pedestrian plazas and squares (highway=pedestrian & area=yes). Attached is an alpha patch inspired by (read: shamelessly copied from) Mark's cycleway=opposite addition. The coding adds a pedestrian way around the perimeter of the polygon. Routing appears to work in Mapsource, but I have not yet hat a chance to test this on a device. Note that Mark's cycleway=opposite code must be present for this patch to compile and work. Your comments would be appreciated. Cheers.

3 5

Problem with --road-name-pois
by Felix Hartmann 31 May '09

31 May '09

I think this has not yet been mentioned on the mailing-list: There seems to be some code missing in the road-name-pois. I changed the ID to several values that if used inside the Points file, would be findable with the search function. If however I use the same ID for the -road-name-pois option, then I can only find the roadname via the all points of interstest, via the category, but not via the subcategory. For Mapsource this makes no great difference, but on my Vista HCx it means that if I can't search via subcategory, the maximum distance in cities is around 10km to find a streetname by using category/find nearest containing..... If I could go via category/select subcategory/find nearest containing, the actual distance could be bigger, so the function would be more useful. (Well a real search index which supports searching for name and city and not just name would be much much better, so I don't know if anyone of you is interested in bugcoding this).

2 1

WARNING (MapArea): ycell was -1... (do I care?)
by Mark Burton 31 May '09

31 May '09

Hey Steve, I get loads of warnings like this when I build maps, should I be worried? If not, why don't we drop the message to info to reduce the noise level? Cheers, Mark

2 1

Need Help with osm5XmlHandler File
by Felix Hartmann 31 May '09

31 May '09

Hello, I wanted to patch /trunk/src/uk/me/parabola/mkgmap/reader/osm/xml/Osm5XmlHandler.java to map oneway attributes to 'highway=oneway1' adapting the code that was used for the cycleway opposite patch. Attached is what I added but it does not work. Can someone find the mistake or my understanding error? (I would like to do this for several attributes like tracktype, mtb:scale or oneway, currently I render them in different mkgmap runs, and add it together as a layer) I would prefer to have everything in one layer however to save compiling time. I would be fine editing all in this file, or maybe even better would be to have mkgmap call lines0, lines1, lines2, lines3 file and subsequent in one run and put them all into the same .img. I can't use the overlays file, because it's too limited for what I wan't to achieve because I can't possibly list all possible combinations. Find attached the patch I tried to use, please someone help me why it doesn't work (I added highway=oneway1 and oneway1=* in the lines stylefile to search for but id doesn't find them, nor displays them in the map). Index: Osm5XmlHandler.java =================================================================== --- Osm5XmlHandler.java (revision 1052) +++ Osm5XmlHandler.java (working copy) @@ -72,7 +72,8 @@ private static final int MODE_BOUNDS = 5; private static final long CYCLEWAY_ID_OFFSET = 0x10000000; - + private static final long ONEWAY_ID_OFFSET = 0x20000000; + private Node currentNode; private Way currentWay; private Relation currentRelation; @@ -84,7 +85,8 @@ private Runnable endTask; private long nextFakeId = 1; - + + private final boolean makeOnewayOverlay; private final boolean makeOppositeCycleways; private final boolean ignoreBounds; private final boolean ignoreTurnRestrictions; @@ -93,6 +95,7 @@ private final String frigRoundabouts; public Osm5XmlHandler(EnhancedProperties props) { + makeOnewayOverlay = props.getProperty("make-oneway-overlay", false); makeOppositeCycleways = props.getProperty("make-opposite-cycleways", false); ignoreBounds = props.getProperty("ignore-osm-bounds", false); routing = props.containsKey("route"); @@ -314,7 +317,26 @@ log.info("Making opposite cycleway '" + cycleWay.getTag("name") + "'"); } } - } + + + if(makeOnewayOverlay && currentWay.isBoolTag("oneway")) { + String onewayTag = currentWay.getTag("oneway1"); + // what we want is an arrow pointing into the direction + // of the onway attribute, therefore we make a copy of all + // oneway ways except oneway=-1 which we will have to give it's + // own ruleset. + long oneway1Id = currentWay.getId() + ONEWAY_ID_OFFSET; + Way oneway1 = new Way(oneway1Id); + wayMap.put(oneway1Id, oneway1); + // I hope that I made no errors above on copying the code + oneway1.copyTags(currentWay); + log.info("Making Oneway Overlay '"); + } + } + + + + if("motorway".equals(highway) || "trunk".equals(highway)) motorways.add(currentWay);

3 3

[PATCH v12] make maps in parallel
by Mark Burton 31 May '09

31 May '09

v12 Great stuff, I found out why I was getting occasional differences in the output files. It was another occurrence of a non-deterministic iteration order of a HashMap but this one was real hard to understand because it only had an effect if the OSM map data had a badly formed restriction relation in it. So now that I understand what was going on, it's cool. And yes, the re-ordering was completely harmless given that the OSM data wasn't valid anyway (restriction with multiple "to" ways"). I intend to commit this stuff to trunk by the end of this week unless any -ve reports come in. ---------- v11 Patch applies to r1052. This seems to me to be well behaved. However, I am still detecting some changes in the NOD data between runs but, at this time, have no evidence that the resulting maps are broken in any way. Before committing this patch to trunk, I would like to find the cause of the reordering but it's proving elusive. I have checked all the obvious stuff like non-deterministic iteration order and shared (static) data and I believe that's all good. Anyway, in the meantime, please use this patch if you have a multi-core machine (especially if > 2 cores), specify --max-jobs and report any notable findings. --------- Changed default number of threads to be 1. If you specify --max-jobs without a value, you get one thread per core. --max-jobs=N means use N threads. With regard to comparing the output with known good maps to see if the parallel processing is corrupting anything, one problem is that the files contain timestamps. I have test code that zeros the time stamps and have been able to compare the output from different runs. What I have seen is that sometimes there are differences that appear to be due to the order in which the labels are written to the output file. If only the order is changing that is harmless but it would be nice to understand how it's happening (I have a theory about this, yet to be proven). --------- Now preserves order in which files are combined (thanks Steve for the tweak). --------- Now serialises reading of style files and map source to avoid reentrancy issue in GType. Reworked top-level loop that waits for the parallel jobs to complete. Appears to use a lot less CPU and could possibly influence the weird problems some were reporting on Windows/Mac - please retest with this version. Steve, I haven't incorporated your changed options handling stuff yet but will do in the future if (a) you don't commit it separately and (b) we can fix the reliability issues with this parallelisation code. --------- Now respects --num-jobs again (broken in last patch). --------- Now reports exceptions in the worker threads. --------- Here's a better fix than last night's effort for the problem where the mapname and description for each job were getting clobbered due to the way that the command args are processed. Each job now gets a "snapshot" of the command args so it doesn't matter if they subsequently get changed. --------- Whoops! fixed a bad bug whereby each map was being output to the same file. Not sure if the fix is very elegant but at least it's not being silly any more. Now limits the default value of max-jobs to 4 no matter how many cores you have as further testing shows that having more threads just burns CPU cycles but doesn't actually finish any quicker. I guess the memory system is limiting the performance and the CPUs are spinning waiting for access. Now showing a real speedup of around 240% (my earlier higher claim was based on CPU usage and I now realise that was erroneous, sorry). -------- Now defaults to creating a thread per core so without doing anything you should see a speedup on a SMP box when processing multiple maps. You can use --max-jobs=N to limit the concurrency - you may want to specify that if you can't increase the VM size to what is required. However, it occurs to me that if you can afford a box with more than 2 cores, then you can probably afford a reasonable amount of memory (otherwise, what's the point in having more cores?) Added help blurb. -------- OK, let it not be said that I don't listen to others! The attached patch provides support for making maps in parallel. By default, the behaviour is the same as before but if you specify --num-threads=N where N is greater than 1, it will process N maps at the same time and then combine the results (if required). Don't forget to increase the heap size appropriately. A quick test on the big box shows good speedup - specifying --num-threads=4 and 2GB VM size. I was seeing better than 380% utilisation with 8 cores in use. I suspect the performance limitation here will be VM size and memory system bandwidth. BTW - I don't think num-threads is actually the best name for the option, so please suggest alternatives. Cheers, Mark

1 0

Commit: r1052: Split the CommandArgs class into two.
by svn commit 31 May '09

31 May '09

Version 1052 was commited by steve on 2009-05-31 17:25:20 +0100 (Sun, 31 May 2009) Split the CommandArgs class into two. One class now reads the arguments and the other represents them. A copy of the CommandsArgs is passed so that it doesn't need to be copied.

1 0

[PATCH v11] make maps in parallel
by Mark Burton 31 May '09

31 May '09

v11 Patch applies to r1052. This seems to me to be well behaved. However, I am still detecting some changes in the NOD data between runs but, at this time, have no evidence that the resulting maps are broken in any way. Before committing this patch to trunk, I would like to find the cause of the reordering but it's proving elusive. I have checked all the obvious stuff like non-deterministic iteration order and shared (static) data and I believe that's all good. Anyway, in the meantime, please use this patch if you have a multi-core machine (especially if > 2 cores), specify --max-jobs and report any notable findings. --------- Changed default number of threads to be 1. If you specify --max-jobs without a value, you get one thread per core. --max-jobs=N means use N threads. With regard to comparing the output with known good maps to see if the parallel processing is corrupting anything, one problem is that the files contain timestamps. I have test code that zeros the time stamps and have been able to compare the output from different runs. What I have seen is that sometimes there are differences that appear to be due to the order in which the labels are written to the output file. If only the order is changing that is harmless but it would be nice to understand how it's happening (I have a theory about this, yet to be proven). --------- Now preserves order in which files are combined (thanks Steve for the tweak). --------- Now serialises reading of style files and map source to avoid reentrancy issue in GType. Reworked top-level loop that waits for the parallel jobs to complete. Appears to use a lot less CPU and could possibly influence the weird problems some were reporting on Windows/Mac - please retest with this version. Steve, I haven't incorporated your changed options handling stuff yet but will do in the future if (a) you don't commit it separately and (b) we can fix the reliability issues with this parallelisation code. --------- Now respects --num-jobs again (broken in last patch). --------- Now reports exceptions in the worker threads. --------- Here's a better fix than last night's effort for the problem where the mapname and description for each job were getting clobbered due to the way that the command args are processed. Each job now gets a "snapshot" of the command args so it doesn't matter if they subsequently get changed. --------- Whoops! fixed a bad bug whereby each map was being output to the same file. Not sure if the fix is very elegant but at least it's not being silly any more. Now limits the default value of max-jobs to 4 no matter how many cores you have as further testing shows that having more threads just burns CPU cycles but doesn't actually finish any quicker. I guess the memory system is limiting the performance and the CPUs are spinning waiting for access. Now showing a real speedup of around 240% (my earlier higher claim was based on CPU usage and I now realise that was erroneous, sorry). -------- Now defaults to creating a thread per core so without doing anything you should see a speedup on a SMP box when processing multiple maps. You can use --max-jobs=N to limit the concurrency - you may want to specify that if you can't increase the VM size to what is required. However, it occurs to me that if you can afford a box with more than 2 cores, then you can probably afford a reasonable amount of memory (otherwise, what's the point in having more cores?) Added help blurb. -------- OK, let it not be said that I don't listen to others! The attached patch provides support for making maps in parallel. By default, the behaviour is the same as before but if you specify --num-threads=N where N is greater than 1, it will process N maps at the same time and then combine the results (if required). Don't forget to increase the heap size appropriately. A quick test on the big box shows good speedup - specifying --num-threads=4 and 2GB VM size. I was seeing better than 380% utilisation with 8 cores in use. I suspect the performance limitation here will be VM size and memory system bandwidth. BTW - I don't think num-threads is actually the best name for the option, so please suggest alternatives. Cheers, Mark

1 0