Hi, I encounter the following error when I run splitter against the french osm data : Elapsed time: 12m 0s Memory: Current 793MB (605MB used, 188MB free) Max 796MB 40 000 000 nodes processed... 42 500 000 nodes processed... 45 000 000 nodes processed... Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at uk.me.parabola.splitter.IntIntMap.ensureSpace(IntIntMap.java:118) at uk.me.parabola.splitter.IntIntMap.put(IntIntMap.java:71) at uk.me.parabola.splitter.SplitIntMap.put(SplitIntMap.java:49) at uk.me.parabola.splitter.SplitProcessor.writeNode(SplitProcessor.java:246) at uk.me.parabola.splitter.SplitProcessor.endNode(SplitProcessor.java:177) at uk.me.parabola.splitter.BinaryMapLoader.processNodes(BinaryMapLoader.java:96) at uk.me.parabola.splitter.BinaryMapLoader.load(BinaryMapLoader.java:78) at uk.me.parabola.splitter.Main.processMap(Main.java:409) at uk.me.parabola.splitter.Main.writeAreas(Main.java:394) at uk.me.parabola.splitter.Main.split(Main.java:213) at uk.me.parabola.splitter.Main.start(Main.java:118) at uk.me.parabola.splitter.Main.main(Main.java:107) Elapsed time: 14m 0s Memory: Current 808MB (647MB used, 161MB free) Max 808MB Elapsed time: 16m 0s Memory: Current 808MB (647MB used, 161MB free) Max 808MB The command line parameters are : java -Xmx896m -jar $FILEDIR/splitter.jar --cache=$cache --geonames-file=$geonames_file --max-nodes="$max_nodes" --description="$description_splitter" --mapid=$mapid $CARTESDIR/$geofabrik_map I do not have more memory on that laptop. Is there a way I could change some parameters to succeed. Thanks. Francois --
Hi Francois, How many areas is the map being split into, and what value do you pass in for --max-nodes? The higher the values of --max-nodes and --max-areas, the more memory required to perform the split. Chris
The command line parameters are : java -Xmx896m -jar $FILEDIR/splitter.jar --cache=$cache --geonames-file=$geonames_file --max-nodes="$max_nodes" --description="$description_splitter" --mapid=$mapid $CARTESDIR/$geofabrik_map I do not have more memory on that laptop. Is there a way I could change some parameters to succeed. Thanks. Francois
Le 11/07/2010 11:59, Chris Miller a écrit : Hi Chris,
How many areas is the map being split into, and what value do you pass in for --max-nodes? The higher the values of --max-nodes and --max-areas, the more memory required to perform the split.
Oups, I thought I wrote those informations. These are they : cache=$CARTESDIR description_splitter="Carte Geographique OSM" geonames_file="cities1000.zip" mapid="66660001" max_areas="255" max_nodes="1000000" overlap="2000" split_file="areas.list"
The command line parameters are : java -Xmx896m -jar $FILEDIR/splitter.jar --cache=$cache --geonames-file=$geonames_file --max-nodes="$max_nodes" --description="$description_splitter" --mapid=$mapid $CARTESDIR/$geofabrik_map I do not have more memory on that laptop. Is there a way I could change some parameters to succeed. Thanks. Francois
Thanks. Francois --
Hi Francois, OK, how many areas do you have in areas.list? Note that the --max-nodes value is only used in the first stage of the split to determine how the map should be subdivided, but since you're specifying an areas.list file the first stage of the split isn't required and hence --max-nodes won't get used (which isn't a problem, I'm just explaining how it works). I'd suggest you try setting --max-areas to a value less than the number of areas in your areas.list. That will cause the split to run in multiple passes, which takes longer but also requires less memory. For example, if you have 150 areas, setting --max-areas=50 will cause the splitter to run 3 passes, using about 1/3 much memory for each pass. Chris
Le 11/07/2010 11:59, Chris Miller a écrit :
Hi Chris,
How many areas is the map being split into, and what value do you pass in for --max-nodes? The higher the values of --max-nodes and --max-areas, the more memory required to perform the split.
Oups, I thought I wrote those informations. These are they : cache=$CARTESDIR description_splitter="Carte Geographique OSM" geonames_file="cities1000.zip" mapid="66660001" max_areas="255" max_nodes="1000000" overlap="2000" split_file="areas.list"
The command line parameters are : java -Xmx896m -jar $FILEDIR/splitter.jar --cache=$cache --geonames-file=$geonames_file --max-nodes="$max_nodes" --description="$description_splitter" --mapid=$mapid $CARTESDIR/$geofabrik_map I do not have more memory on that laptop. Is there a way I could change some parameters to succeed. Thanks. Francois Thanks. Francois
The size of france has increased considerably in the last weeks. I've had to increase -Xmx from 1200M to 2000M in a question of few days. I'm currently splitting france.osm.bz2 with -Xmx2000M and --max-nodes=1600000 El 11/07/10 12:33, Chris Miller escribió:
Hi Francois,
OK, how many areas do you have in areas.list? Note that the --max-nodes value is only used in the first stage of the split to determine how the map should be subdivided, but since you're specifying an areas.list file the first stage of the split isn't required and hence --max-nodes won't get used (which isn't a problem, I'm just explaining how it works).
I'd suggest you try setting --max-areas to a value less than the number of areas in your areas.list. That will cause the split to run in multiple passes, which takes longer but also requires less memory. For example, if you have 150 areas, setting --max-areas=50 will cause the splitter to run 3 passes, using about 1/3 much memory for each pass.
Chris
Le 11/07/2010 11:59, Chris Miller a écrit :
Hi Chris,
How many areas is the map being split into, and what value do you pass in for --max-nodes? The higher the values of --max-nodes and --max-areas, the more memory required to perform the split.
Oups, I thought I wrote those informations. These are they : cache=$CARTESDIR description_splitter="Carte Geographique OSM" geonames_file="cities1000.zip" mapid="66660001" max_areas="255" max_nodes="1000000" overlap="2000" split_file="areas.list"
The command line parameters are : java -Xmx896m -jar $FILEDIR/splitter.jar --cache=$cache --geonames-file=$geonames_file --max-nodes="$max_nodes" --description="$description_splitter" --mapid=$mapid $CARTESDIR/$geofabrik_map I do not have more memory on that laptop. Is there a way I could change some parameters to succeed. Thanks. Francois
Le 11/07/2010 12:33, Chris Miller a écrit : Hi Chris,
I'd suggest you try setting --max-areas to a value less than the number of areas in your areas.list. That will cause the split to run in multiple passes, which takes longer but also requires less memory. For example, if you have 150 areas, setting --max-areas=50 will cause the splitter to run 3 passes, using about 1/3 much memory for each pass.
OK, I did what you said, and it worked like a charm. Thank you. Francois --
participants (3)
-
Carlos Dávila -
Chris Miller -
frmas