memory requirements for generating planet bounds
as per http://wiki.openstreetmap.org/wiki/Mkgmap/help/options#Using_preprocessed_bo... , i tried to generate bounds myself from a planet boundary extract (prepared with osmconvert/osmfilter like in the instructions on the same page). java -cp mkgmap.jar uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryPreprocessor <inputfile> <boundsdirectory> that run out of the heap space of 2300 megs. before i try experimenting further, how much heap space is required for creating planet bounds from boundary extract ? :) -- Rich
Hi, On Thu, Feb 14, Rich wrote:
as per http://wiki.openstreetmap.org/wiki/Mkgmap/help/options#Using_preprocessed_bo... , i tried to generate bounds myself from a planet boundary extract (prepared with osmconvert/osmfilter like in the instructions on the same page).
java -cp mkgmap.jar uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryPreprocessor <inputfile> <boundsdirectory>
that run out of the heap space of 2300 megs. before i try experimenting further, how much heap space is required for creating planet bounds from boundary extract ? :)
You need at least 8GB of free RAM. And I would suggest Sun/Oracle Java, not IBM Java or openjdk. Thorsten -- Thorsten Kukuk, Project Manager/Release Manager SLES SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg)
On 2013-02-14 17:25, Thorsten Kukuk wrote:
Hi,
On Thu, Feb 14, Rich wrote:
as per http://wiki.openstreetmap.org/wiki/Mkgmap/help/options#Using_preprocessed_bo... , i tried to generate bounds myself from a planet boundary extract (prepared with osmconvert/osmfilter like in the instructions on the same page).
java -cp mkgmap.jar uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryPreprocessor <inputfile> <boundsdirectory>
that run out of the heap space of 2300 megs. before i try experimenting further, how much heap space is required for creating planet bounds from boundary extract ? :)
You need at least 8GB of free RAM. And I would suggest Sun/Oracle Java,
thanks. so that means i won't do that on this laptop with 4gb of ram ;)
not IBM Java or openjdk.
...that does sound a bit strange, given that a certain distribution i use does not have sun java in official repos anymore ;)
Thorsten -- Rich
On Thu, Feb 14, Rich wrote:
...that does sound a bit strange, given that a certain distribution i use does not have sun java in official repos anymore ;)
That's because of the license Oracle use for this, which forbids redistribution if you don't have corresponding legal contracts. Thorsten -- Thorsten Kukuk, Project Manager/Release Manager SLES SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg)
On 2013-02-14 17:33, Thorsten Kukuk wrote:
On Thu, Feb 14, Rich wrote:
...that does sound a bit strange, given that a certain distribution i use does not have sun java in official repos anymore ;)
That's because of the license Oracle use for this, which forbids redistribution if you don't have corresponding legal contracts.
sure, that's understandable. but why would sun java be better for this task ?
Thorsten -- Rich
On Thu, Feb 14, Rich wrote:
On 2013-02-14 17:33, Thorsten Kukuk wrote:
On Thu, Feb 14, Rich wrote:
...that does sound a bit strange, given that a certain distribution i use does not have sun java in official repos anymore ;)
That's because of the license Oracle use for this, which forbids redistribution if you don't have corresponding legal contracts.
sure, that's understandable. but why would sun java be better for this task ?
IBM Java has problems with memory management, openjdk has, at least at my last try, still some compatibility issues. Thorsten -- Thorsten Kukuk, Project Manager/Release Manager SLES SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg)
Hi Thorsten, this reminds me on the osmid_v1.patch using fastutil.jar It is esp. usefull in this case because the memory requirements are caused by the huge number of coords. @Steve: Do you still plan to add it ? Gerd Thorsten Kukuk wrote
Hi,
On Thu, Feb 14, Rich wrote:
as per http://wiki.openstreetmap.org/wiki/Mkgmap/help/options#Using_preprocessed_bo... , i tried to generate bounds myself from a planet boundary extract (prepared with osmconvert/osmfilter like in the instructions on the same page).
java -cp mkgmap.jar uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryPreprocessor
<inputfile>
<boundsdirectory>
that run out of the heap space of 2300 megs. before i try experimenting further, how much heap space is required for creating planet bounds from boundary extract ? :)
You need at least 8GB of free RAM. And I would suggest Sun/Oracle Java, not IBM Java or openjdk.
Thorsten
-- Thorsten Kukuk, Project Manager/Release Manager SLES SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) _______________________________________________ mkgmap-dev mailing list
mkgmap-dev@.org
-- View this message in context: http://gis.19327.n5.nabble.com/memory-requirements-for-generating-planet-bou... Sent from the Mkgmap Development mailing list archive at Nabble.com.
There is also another way to generate the bounds. It is possible to cut the planet into pieces. Create the bounds for the pieces and merge the bounds afterwards. This is how I create bounds. I can create the bounds with cutting the planet into 3 pieces on a laptop with 4GB. You can merge bounds with the following command: java -cp mkgmap.jar uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryMerger <boundsdir1> <boundsdir2> <mergeddir> The command merges the bounds from boundsdir1 and boundsdir2 and put the merged bounds to mergedir. I think it would be easier if the BoundaryPreparer accepts multiple input files and does the merging automatically. Put it on the TODO... WanMil
Hi Thorsten,
this reminds me on the osmid_v1.patch using fastutil.jar It is esp. usefull in this case because the memory requirements are caused by the huge number of coords.
@Steve: Do you still plan to add it ?
Gerd
Thorsten Kukuk wrote
Hi,
On Thu, Feb 14, Rich wrote:
as per http://wiki.openstreetmap.org/wiki/Mkgmap/help/options#Using_preprocessed_bo... , i tried to generate bounds myself from a planet boundary extract (prepared with osmconvert/osmfilter like in the instructions on the same page).
java -cp mkgmap.jar uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryPreprocessor
<inputfile>
<boundsdirectory>
that run out of the heap space of 2300 megs. before i try experimenting further, how much heap space is required for creating planet bounds from boundary extract ? :)
You need at least 8GB of free RAM. And I would suggest Sun/Oracle Java, not IBM Java or openjdk.
Thorsten
-- Thorsten Kukuk, Project Manager/Release Manager SLES SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) _______________________________________________ mkgmap-dev mailing list
mkgmap-dev@.org
-- View this message in context: http://gis.19327.n5.nabble.com/memory-requirements-for-generating-planet-bou... Sent from the Mkgmap Development mailing list archive at Nabble.com. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Steve, fine, it is probably really needed to have the same version, else we'll risk trouble in the sea precompiler which loads splitter which may require fastutil.jar I'll verify that the corresponding patch works and commit it tomorrow. Gerd
Date: Tue, 19 Feb 2013 20:42:31 +0000 From: steve@parabola.me.uk To: mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] memory requirements for generating planet bounds
Hi
@Steve: Do you still plan to add it ?
Added. I do realise that it would be nice to have the same jar working on both mkgmap and splitter. I'll fix that once mkgmap uses it - sorry I've forgotten the details :(
..Steve
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
On 19/02/13 20:52, Gerd Petermann wrote:
fine, it is probably really needed to have the same version, else we'll risk trouble in the sea precompiler which loads splitter which may require fastutil.jar I'll verify that the corresponding patch works and commit it tomorrow.
OK, new version has been created and installed and the ivy file modified to match. I've set up a project to create the cut down jar automatically, so it will be very easy to update in the future. ..Steve
Hi Gerd, On Thu, Feb 14, GerdP wrote:
this reminds me on the osmid_v1.patch using fastutil.jar It is esp. usefull in this case because the memory requirements are caused by the huge number of coords.
That patch is really making a huge difference. Thanks a lot for that! Thorsten -- Thorsten Kukuk, Project Manager/Release Manager SLES SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg)
participants (6)
-
Gerd Petermann -
GerdP
-
Rich -
Steve Ratcliffe -
Thorsten Kukuk -
WanMil