On 2011-05-04 11:51, Steve Ratcliffe wrote:
***** Full GC ***** Elapsed time: 18m 0s Memory: Current 135MB (106MB used, 29MB free) Max 6666MB
The 'Full GC' line is not a problem, it is a message printed by splitter every so often before it attempts to force a garbage collect. This is not usually a useful thing to do, but it can help make the printed memory usages more accurate, I suppose.
Thanks for clearing that up Steve. The newer splitter apparently needs only a fraction of the memory it needed before. Great work devs!
Splitter can still read .osm.gz from stdin as long as there is a --split-file option, and as long as it doesn't have to use more than one pass through the data.
Alright. I don't think splitter is able to only use one pass on a planet file so it's another reason to move to the pbf format.
You should just provide no file name on the command line though. (ie do not give /dev/stdin) However /dev/stdin should work as long as the input is uncompressed.
Thanks, the /dev/stdin ended up there because of other problems I had and it worked. I can't confirm or deny that it works without it ;-)
Previously there was a --cache option which would copy and convert the input to a cache file, which would then be re-read if --split-file was not given or if more than one pass was needed.
Since the newer pbf format is as or more efficient than the cache file format and osmosis is capable of producing it directly it seems best to just use osmosis to write it. Having splitter convert to pbf will not save any disk space and will probably be slower than overall than osmosis producing it directly.
Again: thanks. It's clear I'll better migrate to pbf.