Re: [mkgmap-dev] Commit: r3081: merge from the high-prec-coord branch.
well, I'm (the server) is definitely not running out of memory. New bounds are fine, it's only bout sea. here's the options I use (no style-file given - seems not to depend on the style): c:\OpenMTBMap\maps>start /low /b /wait java -jar -Xms6000M -Xmx10300M c:\openmtbmap\mkgmap.jar --max-jobs=8 --generate-sea --latin1 --precomp-sea=c:\openmtbmap\maps\sea.zip --nsis --index --levels="0:24, 1:23, 2:22, 3:21, 4:20, 5:19, 6:18" --overview-levels="7:17, 8:16, 9:15, 10:14, 11:13, 12:12" --adjust-turn-headings --add-pois-to-areas --reduce-point-density=3.4 --reduce-point-density-polygon=6 --housenumbers --link-pois-to-ways --ignore-turn-rest rictions --polygon-size-limits="24:16, 23:14, 22:12, 21:11, 20:10, 19:9, 18:8, 17:7, 16:6, 15:5, 14:4, 13:3, 12:2, 11:0, 10:0" --description=openmtbmap_at --show-profiles=1 --location-autofill=bounds,is_in,nearest --bounds=c:\openmtbmap\maps\bounds .zip --route --country-abbr=at --country-name=austria --mapname=63650000 --family-id=6365 --product-id=1 --series-name=openmtbmap_austria_06.03.2014 --family-name=mtbmap_at_06.03.2014 --tdbfile --overview-mapname=mapsetc --keep-going --area-name="aus tria_06.03.2014_openmtbmap.org" -c c:\openmtbmap\maps\template.austria 7*.img 1>NUL My austria.osm.pbf is a bit older... On 06.03.2014 13:38, Gerd Petermann wrote:
Hi Felix,
I try to reproduce the problem. It might be related to higher memory consumption.
Gerd
------------------------------------------------------------------------ Date: Thu, 6 Mar 2014 13:36:06 +0100 From: extremecarver@gmail.com To: mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] Commit: r3081: merge from the high-prec-coord branch.
Well there is Lake Neusiedl - which is entered as sea, Bodensee and some others maybe too? OSM gotten so complex that even some lakes need to be entered as sea (and it makes sense because otherwise lakes on country boundary would be empty or overflowing...). On 06.03.2014 13:33, Enrico Liboni wrote:
interesting... sea in Austria, that's maybe the issue
On Thu, Mar 6, 2014 at 1:26 PM, Felix Hartmann <extremecarver@gmail.com <mailto:extremecarver@gmail.com>> wrote:
Oh well - there seems to be some problem with the new sea file:
java.lang.NullPointerException at uk.me.parabola.mkgmap.reader.osm.SeaGenerator$PrecompData.access$100(SeaGenerator.java:1486) at uk.me.parabola.mkgmap.reader.osm.SeaGenerator.addPrecompSea(SeaGenerator.java:616) at uk.me.parabola.mkgmap.reader.osm.SeaGenerator.end(SeaGenerator.java:847) at uk.me.parabola.mkgmap.reader.osm.OsmReadingHooksChain.end(OsmReadingHooksChain.java:79) at uk.me.parabola.mkgmap.reader.osm.bin.OsmBinMapDataSource.load(OsmBinMapDataSource.java:63) at uk.me.parabola.mkgmap.reader.osm.OsmMapDataSource.load(OsmMapDataSource.java:127) at uk.me.parabola.mkgmap.main.MapMaker.loadFromFile(MapMaker.java:167) at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:63) at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:220) at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:216) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)
nothing else changed besides using new instead of old sea file. splitter/mkgmap both trunk and latest svn version. (on compiling austria).
On 06.03.2014 13:02, Thorsten Kukuk wrote:
On Thu, Mar 06, Felix Hartmann wrote:
Are there precompiled bounds and sea to download already somewhere? (the ones on mkgmap.org.uk <http://mkgmap.org.uk> are rather old, and the ones on pleiades are about the same size - hence same old format I assume).
The size did grow up from 418MB to 624MB for the bounds.
Thorsten
-- keep on biking and discovering new trails
Felix openmtbmap.org <http://openmtbmap.org> & www.velomap.org <http://www.velomap.org>
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk <mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk <mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- keep on biking and discovering new trails
Felix openmtbmap.org &www.velomap.org <http://www.velomap.org>
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- keep on biking and discovering new trails Felix openmtbmap.org & www.velomap.org
Hi Felix, yes, I can reproduce the problem. The program fails to read the index in the file in the zip. The file works okay when you unzip the content. I'm trying to find out what goes wrong. Gerd Date: Thu, 6 Mar 2014 13:46:05 +0100 From: extremecarver@gmail.com To: gpetermann_muenchen@hotmail.com; mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] Commit: r3081: merge from the high-prec-coord branch. well, I'm (the server) is definitely not running out of memory. New bounds are fine, it's only bout sea. here's the options I use (no style-file given - seems not to depend on the style): c:\OpenMTBMap\maps>start /low /b /wait java -jar -Xms6000M -Xmx10300M c:\openmtbmap\mkgmap.jar --max-jobs=8 --generate-sea --latin1 --precomp-sea=c:\openmtbmap\maps\sea.zip --nsis --index --levels="0:24, 1:23, 2:22, 3:21, 4:20, 5:19, 6:18" --overview-levels="7:17, 8:16, 9:15, 10:14, 11:13, 12:12" --adjust-turn-headings --add-pois-to-areas --reduce-point-density=3.4 --reduce-point-density-polygon=6 --housenumbers --link-pois-to-ways --ignore-turn-rest rictions --polygon-size-limits="24:16, 23:14, 22:12, 21:11, 20:10, 19:9, 18:8, 17:7, 16:6, 15:5, 14:4, 13:3, 12:2, 11:0, 10:0" --description=openmtbmap_at --show-profiles=1 --location-autofill=bounds,is_in,nearest --bounds=c:\openmtbmap\maps\bounds .zip --route --country-abbr=at --country-name=austria --mapname=63650000 --family-id=6365 --product-id=1 --series-name=openmtbmap_austria_06.03.2014 --family-name=mtbmap_at_06.03.2014 --tdbfile --overview-mapname=mapsetc --keep-going --area-name="aus tria_06.03.2014_openmtbmap.org" -c c:\openmtbmap\maps\template.austria 7*.img 1>NUL My austria.osm.pbf is a bit older... On 06.03.2014 13:38, Gerd Petermann wrote: Hi Felix, I try to reproduce the problem. It might be related to higher memory consumption. Gerd Date: Thu, 6 Mar 2014 13:36:06 +0100 From: extremecarver@gmail.com To: mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] Commit: r3081: merge from the high-prec-coord branch. Well there is Lake Neusiedl - which is entered as sea, Bodensee and some others maybe too? OSM gotten so complex that even some lakes need to be entered as sea (and it makes sense because otherwise lakes on country boundary would be empty or overflowing...). On 06.03.2014 13:33, Enrico Liboni wrote: interesting... sea in Austria, that's maybe the issue On Thu, Mar 6, 2014 at 1:26 PM, Felix Hartmann <extremecarver@gmail.com> wrote: Oh well - there seems to be some problem with the new sea file: java.lang.NullPointerException at uk.me.parabola.mkgmap.reader.osm.SeaGenerator$PrecompData.access$100(SeaGenerator.java:1486) at uk.me.parabola.mkgmap.reader.osm.SeaGenerator.addPrecompSea(SeaGenerator.java:616) at uk.me.parabola.mkgmap.reader.osm.SeaGenerator.end(SeaGenerator.java:847) at uk.me.parabola.mkgmap.reader.osm.OsmReadingHooksChain.end(OsmReadingHooksChain.java:79) at uk.me.parabola.mkgmap.reader.osm.bin.OsmBinMapDataSource.load(OsmBinMapDataSource.java:63) at uk.me.parabola.mkgmap.reader.osm.OsmMapDataSource.load(OsmMapDataSource.java:127) at uk.me.parabola.mkgmap.main.MapMaker.loadFromFile(MapMaker.java:167) at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:63) at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:220) at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:216) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) nothing else changed besides using new instead of old sea file. splitter/mkgmap both trunk and latest svn version. (on compiling austria). On 06.03.2014 13:02, Thorsten Kukuk wrote: On Thu, Mar 06, Felix Hartmann wrote: Are there precompiled bounds and sea to download already somewhere? (the ones on mkgmap.org.uk are rather old, and the ones on pleiades are about the same size - hence same old format I assume). http://osm.thkukuk.de/data/ The size did grow up from 418MB to 624MB for the bounds. Thorsten -- keep on biking and discovering new trails Felix openmtbmap.org & www.velomap.org _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev -- keep on biking and discovering new trails Felix openmtbmap.org & www.velomap.org _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev -- keep on biking and discovering new trails Felix openmtbmap.org & www.velomap.org
okay great. Wonder everyone else unpacks... (It seemed to me there is no time penalty to using the sea file zipped - or is there?) On 06.03.2014 13:58, Gerd Petermann wrote:
Hi Felix,
yes, I can reproduce the problem. The program fails to read the index in the file in the zip. The file works okay when you unzip the content. I'm trying to find out what goes wrong.
Gerd
------------------------------------------------------------------------ Date: Thu, 6 Mar 2014 13:46:05 +0100 From: extremecarver@gmail.com To: gpetermann_muenchen@hotmail.com; mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] Commit: r3081: merge from the high-prec-coord branch.
well, I'm (the server) is definitely not running out of memory. New bounds are fine, it's only bout sea.
here's the options I use (no style-file given - seems not to depend on the style): c:\OpenMTBMap\maps>start /low /b /wait java -jar -Xms6000M -Xmx10300M c:\openmtbmap\mkgmap.jar --max-jobs=8 --generate-sea --latin1 --precomp-sea=c:\openmtbmap\maps\sea.zip --nsis --index --levels="0:24, 1:23, 2:22, 3:21, 4:20, 5:19, 6:18" --overview-levels="7:17, 8:16, 9:15, 10:14, 11:13, 12:12" --adjust-turn-headings --add-pois-to-areas --reduce-point-density=3.4 --reduce-point-density-polygon=6 --housenumbers --link-pois-to-ways --ignore-turn-rest rictions --polygon-size-limits="24:16, 23:14, 22:12, 21:11, 20:10, 19:9, 18:8, 17:7, 16:6, 15:5, 14:4, 13:3, 12:2, 11:0, 10:0" --description=openmtbmap_at --show-profiles=1 --location-autofill=bounds,is_in,nearest --bounds=c:\openmtbmap\maps\bounds .zip --route --country-abbr=at --country-name=austria --mapname=63650000 --family-id=6365 --product-id=1 --series-name=openmtbmap_austria_06.03.2014 --family-name=mtbmap_at_06.03.2014 --tdbfile --overview-mapname=mapsetc --keep-going --area-name="aus tria_06.03.2014_openmtbmap.org" -c c:\openmtbmap\maps\template.austria 7*.img 1>NUL
My austria.osm.pbf is a bit older...
On 06.03.2014 13:38, Gerd Petermann wrote:
Hi Felix,
I try to reproduce the problem. It might be related to higher memory consumption.
Gerd
------------------------------------------------------------------------ Date: Thu, 6 Mar 2014 13:36:06 +0100 From: extremecarver@gmail.com <mailto:extremecarver@gmail.com> To: mkgmap-dev@lists.mkgmap.org.uk <mailto:mkgmap-dev@lists.mkgmap.org.uk> Subject: Re: [mkgmap-dev] Commit: r3081: merge from the high-prec-coord branch.
Well there is Lake Neusiedl - which is entered as sea, Bodensee and some others maybe too? OSM gotten so complex that even some lakes need to be entered as sea (and it makes sense because otherwise lakes on country boundary would be empty or overflowing...). On 06.03.2014 13:33, Enrico Liboni wrote:
interesting... sea in Austria, that's maybe the issue
On Thu, Mar 6, 2014 at 1:26 PM, Felix Hartmann <extremecarver@gmail.com <mailto:extremecarver@gmail.com>> wrote:
Oh well - there seems to be some problem with the new sea file:
java.lang.NullPointerException at uk.me.parabola.mkgmap.reader.osm.SeaGenerator$PrecompData.access$100(SeaGenerator.java:1486) at uk.me.parabola.mkgmap.reader.osm.SeaGenerator.addPrecompSea(SeaGenerator.java:616) at uk.me.parabola.mkgmap.reader.osm.SeaGenerator.end(SeaGenerator.java:847) at uk.me.parabola.mkgmap.reader.osm.OsmReadingHooksChain.end(OsmReadingHooksChain.java:79) at uk.me.parabola.mkgmap.reader.osm.bin.OsmBinMapDataSource.load(OsmBinMapDataSource.java:63) at uk.me.parabola.mkgmap.reader.osm.OsmMapDataSource.load(OsmMapDataSource.java:127) at uk.me.parabola.mkgmap.main.MapMaker.loadFromFile(MapMaker.java:167) at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:63) at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:220) at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:216) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)
nothing else changed besides using new instead of old sea file. splitter/mkgmap both trunk and latest svn version. (on compiling austria).
On 06.03.2014 13:02, Thorsten Kukuk wrote:
On Thu, Mar 06, Felix Hartmann wrote:
Are there precompiled bounds and sea to download already somewhere? (the ones on mkgmap.org.uk <http://mkgmap.org.uk> are rather old, and the ones on pleiades are about the same size - hence same old format I assume).
The size did grow up from 418MB to 624MB for the bounds.
Thorsten
-- keep on biking and discovering new trails
Felix openmtbmap.org <http://openmtbmap.org> & www.velomap.org <http://www.velomap.org>
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk <mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk <mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- keep on biking and discovering new trails
Felix openmtbmap.org &www.velomap.org <http://www.velomap.org>
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk <mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- keep on biking and discovering new trails
Felix openmtbmap.org &www.velomap.org <http://www.velomap.org>
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- keep on biking and discovering new trails Felix openmtbmap.org & www.velomap.org
On Thu, Mar 06, Felix Hartmann wrote:
okay great. Wonder everyone else unpacks... (It seemed to me there is no time penalty to using the sea file zipped - or is there?)
Me ;) Or better, I never pack it for my own usage, the ziping was only done because people asked for provind the bounds and sea files ... So it could be that there is still something wrong from generating the zip archive from the directory. Thorsten -- Thorsten Kukuk, Senior Architect SLES & Common Code Base SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg)
yes - unpacking and repacking it myself (using 7-zip program with zip standard settings) and no more problems. So you definitely got some problems with your archive command/program. On 06.03.2014 14:17, Thorsten Kukuk wrote:
On Thu, Mar 06, Felix Hartmann wrote:
okay great. Wonder everyone else unpacks... (It seemed to me there is no time penalty to using the sea file zipped - or is there?) Me ;) Or better, I never pack it for my own usage, the ziping was only done because people asked for provind the bounds and sea files ...
So it could be that there is still something wrong from generating the zip archive from the directory.
Thorsten
-- keep on biking and discovering new trails Felix openmtbmap.org & www.velomap.org
On Thu, Mar 06, Felix Hartmann wrote:
yes - unpacking and repacking it myself (using 7-zip program with zip standard settings) and no more problems. So you definitely got some problems with your archive command/program.
I doubt. It's the official reference implementation for zip. Thorsten -- Thorsten Kukuk, Senior Architect SLES & Common Code Base SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg)
I've just committed a change that allows to read your file. I have no idea why the old code doesn't work with it, the archive looks good. Gerd
Date: Thu, 6 Mar 2014 14:27:40 +0100 From: kukuk@suse.de To: mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] Commit: r3081: merge from the high-prec-coord branch.
On Thu, Mar 06, Felix Hartmann wrote:
yes - unpacking and repacking it myself (using 7-zip program with zip standard settings) and no more problems. So you definitely got some problems with your archive command/program.
I doubt. It's the official reference implementation for zip.
Thorsten
-- Thorsten Kukuk, Senior Architect SLES & Common Code Base SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
On 06/03/14 13:29, Gerd Petermann wrote:
I've just committed a change that allows to read your file. I have no idea why the old code doesn't work with it, the archive looks good.
Thorsten's file doesn't have a separate entry for 'sea/' and it looks like the old code assumed that the files did not have a 'sea/' prefix if it isn't present. ..Steve
Hi Thorsten, I compared your file with one from WanMil with a hex editor. In WanMils file I see an entry "sea/". This is missing in your file, and that's why r3081 failed. I don't know what you have to do to create this entry. Gerd
Date: Thu, 6 Mar 2014 14:27:40 +0100 From: kukuk@suse.de To: mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] Commit: r3081: merge from the high-prec-coord branch.
On Thu, Mar 06, Felix Hartmann wrote:
yes - unpacking and repacking it myself (using 7-zip program with zip standard settings) and no more problems. So you definitely got some problems with your archive command/program.
I doubt. It's the official reference implementation for zip.
Thorsten
-- Thorsten Kukuk, Senior Architect SLES & Common Code Base SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
On Thu, Mar 06, Gerd Petermann wrote:
Hi Thorsten,
I compared your file with one from WanMil with a hex editor. In WanMils file I see an entry "sea/". This is missing in your file, and that's why r3081 failed. I don't know what you have to do to create this entry.
The difference I see with help of "zipinfo": Wanmil is using zip format version 2.0 on a fat filesystem, I'm using zip format version 3.0 on unix filesystem. I haven't found an option to create a zip version 2.0 archive. Thorsten -- Thorsten Kukuk, Senior Architect SLES & Common Code Base SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg)
@Thorsten - you could try from command line by using the java jar command: jar cMf sea.zip sea/* not sure about the zip version but this should create a separate entry for sea/ directory as well - Enrico On Thu, Mar 6, 2014 at 4:52 PM, Thorsten Kukuk <kukuk@suse.de> wrote:
On Thu, Mar 06, Gerd Petermann wrote:
Hi Thorsten,
I compared your file with one from WanMil with a hex editor. In WanMils file I see an entry "sea/". This is missing in your file, and that's why r3081 failed. I don't know what you have to do to create this entry.
The difference I see with help of "zipinfo": Wanmil is using zip format version 2.0 on a fat filesystem, I'm using zip format version 3.0 on unix filesystem.
I haven't found an option to create a zip version 2.0 archive.
Thorsten
-- Thorsten Kukuk, Senior Architect SLES & Common Code Base SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
participants (5)
-
Enrico Liboni -
Felix Hartmann -
Gerd Petermann -
Steve Ratcliffe -
Thorsten Kukuk