One possible workaround would be for the splitter to make a pass to determine the min and max node IDs (as a 64 bit long), then use the min ID as an adjustment to rebase all the node IDs starting from zero. This would still fail if (max - min) was greater than 2^31. A performance improvement would be to allow the min node ID to be specified on the command line (assuming the user knew it) or remembered from a previous pass.
On Wed, 27 Jul 2011 19:39:38 +0200, WanMil wrote:
It is a regression if r161 works and r179 not so in this case the problem should be tracked down.
This is a known limitation of the current splitter. It can handle node IDs only up to something less than 2e09 because Java allows only 31 bit numbers as index for an array.
Even if this could be solved, you would need huge amounts of memory to process data with node IDs starting at 2e09 (like the srtm data), because the amount of memory the splitter needs primilary depends on the max. node ID (and not on the number of nodes) in the data. If there are node IDs larger than about 1.7e09 the splitter needs even more memory than a 32 bit JVM can provide.
Scott has explained this in http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2011q1/010598.html and http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2011q1/010629.html
Michael
Hi WanMil,
the easiest way for me would be 'splitter-r161'. Do you know, where to get?
BTW, the file with the srtm data is about 4,2 GB and has over 45 million nodes.
Josef
Am 26.07.2011 21:43, schrieb WanMil:
Hi Josef,
I have googled for "splitter mkgmap IndexOutOfBoundsException" and found a similar problem description: http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2011q1/010627.html
Maybe that helps?
WanMil
No hints?
Am 21.07.2011 08:36, schrieb Josef Latt:
Hi,
I get srtm data with 'mono Srtm2Osm.exe -bounds1 47.2 5.8 55.2 15.1 -step 20 -cat 100 20 -large -corrxy 0.0005 0.0005 -o srtm.osm'
Splitting (splitter-r179) the data with 'java -ea -Xmx2048M -jar /home/jolatt/OSM/splitter/splitter.jar --mapid=21000001 --mixed=yes --max-nodes=5000000 srtm.osm' into tiles I get the following error:
**** WARNING: the default output type has changed to pbf, use --output=xml for .osm.gz files
Elapsed time: 0s Memory: Current 59MB (1MB used, 58MB free) Max 1820MB Time started: Thu Jul 21 08:22:09 CEST 2011 Map is being split for resolution 13: - area boundaries are aligned to 0x800 map units - areas are multiples of 0x1000 map units wide and high Processing srtm.osm in 1 file Time: Thu Jul 21 08:22:09 CEST 2011 Exact map coverage is (53.99924039840698,5.800073146820068) to (55.20007610321045,8.0000638961792) Trimmed and rounded map coverage is (54.0087890625,6.9873046875) to (55.2392578125,8.0419921875) Splitting nodes into areas containing a maximum of 5.000.000 nodes each... Area (54.0087890625,6.9873046875) to (55.2392578125,8.0419921875) contains 2.629 nodes. DONE! 1 areas: Area 21000001 covers (0x266800,0x4f800) to (0x274800,0x5b800) Writing out split osm files Thu Jul 21 08:22:09 CEST 2011 Processing 1 areas in a single pass (54.0087890625,6.9873046875) to (55.2392578125,8.0419921875) Starting pass 1 of 1, processing 1 areas (21000001 to 21000001) Making SparseMultiMap Making SparseMultiMap Processing srtm.osm Exception in thread "main" java.lang.IndexOutOfBoundsException: Index (31250001) is greater than or equal to list size (31250001) at it.unimi.dsi.fastutil.objects.ObjectArrayList.get(ObjectArrayList .java:258) at uk.me.parabola.splitter.SparseInt2ShortMapInline.put(SparseInt2Sh ortMapInline.java:128) at uk.me.parabola.splitter.SparseInt2ShortMultiMap$Inner.put(SparseI nt2ShortMultiMap.java:81) at uk.me.parabola.splitter.SparseInt2ShortMultiMap.put(SparseInt2Sho rtMultiMap.java:31) at uk.me.parabola.splitter.SplitProcessor.writeNode(SplitProcessor.j ava:209) at uk.me.parabola.splitter.SplitProcessor.processNode(SplitProcessor .java:118) at uk.me.parabola.splitter.OSMParser.endElement(OSMParser.java:243) at uk.me.parabola.splitter.AbstractXppParser.parse(AbstractXppParser .java:57) at uk.me.parabola.splitter.Main.processMap(Main.java:412) at uk.me.parabola.splitter.Main.writeAreas(Main.java:368) at uk.me.parabola.splitter.Main.split(Main.java:190) at uk.me.parabola.splitter.Main.start(Main.java:118) at uk.me.parabola.splitter.Main.main(Main.java:107) I would try an older version of splitter, but where to download, or is there another idea?
Greetings Josef Latt
mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev