New splitter version available

5 Aug 2009

      I've checked in a new version of the tile splitter. It's not yet available 
for download on the splitter homepage, but until then you can either check 
it out of Subversion at http://svn.mkgmap.org.uk/splitter/trunk or download 
a prebuilt version from http://redyeti.net/splitter.jar

Here is what has changed:

- Reduced memory requirements, especially during the generation of areas.list
- About 10% faster overall
- Uses a slightly faster and leaner XML parser (XPP instead of SAX)
- Removed the 4-area limit for relations
- Improved the log messages to give more information about what is happening 
during the split
- Can handle an arbitrary number of areas. Note however that each multiple 
of 255 areas requires a separate pass over the XML. This means a large number 
of areas will slow down the splitting due to the additional passes. This 
is still a lot better than the previous behaviour which was for the splitter 
to silently put elements into the wrong areas.

It should be possible to split the whole planet file with far less memory 
now. The amount of memory required is roughly 2.5GB(?) for the initial areas.list 
generation, and then roughly proportional to (--max-nodes * 255). The smaller 
you make --max-nodes, the less memory the split will require but also the 
longer it will take (since a full parsing run over the planet file is required 
for each multiple of 255 areas).

I've just kicked off the following before I go to bed, will be interesting 
to see if it manages to complete by the morning without running out of memory:

java -Xmx4000m -jar splitter.jar --mapid=1 --max-nodes=1200000 planet-090715.osm

Enjoy!
Chris

Chris Miller

Clinton Gladstone

Chris Miller

Clinton Gladstone

tags

participants (2)