On 29.08.2011 20:52, WanMil wrote:
1. I noticed that (at least for my style-file) 7500MB of RAM available
does not suffice to compile a map of Europe including index! However if
I recreate the index in another go, then it is no problem.

How comes? The index creation seems to be completly independent of the
maptiles creation. Is mkgmap leaving stuff inside the memory, that is in
reality not needed anymore?

I think you have listed your style and your mkgmap parameters on your website. Can you give a link to them? That makes it easier to find the problem.

Subversion


And here is the specific options on full creation (crashes low on java heap space, passes if Xmx11000M is given, but I only have 12GB RAM on a testmachine on my build sever there is only about 7800M before it seriously slows down -- this is running against Europe extract from geofabrik):
c:\OpenMTBMap\maps>start /low /b /wait java -ea -jar -Xmx7500M c:\openmtbmap\mkgmap.jar --max-jobs=4 --generate-sea=extend-sea-sectors,close-gaps=6000,floodblocker,fbgap=60,fbthres=200,fbratio=0.6 --latin1 "--style-file=c:\openmtbmap\new4"  --reduce-point-density=4 --nsis --index --transparent --adjust-turn-headings --add-pois-to-areas --ignore-maxspeeds --x-reduce-point-density-polygon=8 --link-pois-to-ways --ignore-turn-restrictions --min-size-polygon=15 --remove-short-arcs=4 --description=openmtbmap_eu --merge-l
ines --location-autofill=1 --route --country-abbr=eu --country-name=europe  --mapname=65500000 --family-id=6550 --product-id=1 --series-name=openmtbmap_europe_29.08.2011 --family-name=mtbmap_eu_29.08.2011 --tdbfile --overview-mapname=mapset --keep-going --area-name="europe_29.08.2011_openmtbmap.org"  -c c:\openmtbmap\maps\template.europe  1>NUL

versus the options for index only creation (works well, I can even reduce Xmx to about 4500M):
c:\OpenMTBMap\maps>start /low /b /wait java -ea -jar -Xmx7500M c:\openmtbmap\mkgmap.jar --nsis --index --transparent --description=openmtbmap_eu --location-autofill=1 --country-abbr=eu --country-name=europe  --mapname=65500000 --family-id=6550 --product-id=1 --series-name=openmtbmap_europe_29.08.2011 --family-name=mtbmap_eu_29.08.2011
--tdbfile --overview-mapname=mapset --keep-going --area-name="europe_29.08.2011_openmtbmap.org" 6550*.img

And important: Do you use any patches?
yes listed here:
http://svn.origo.ethz.ch/wsvn/openmtbmap/mkgmap_patches.patch

but they shouldn't affect the index creation. Maybe it's again an issue of assigning names like ft to unnamed footways. However there must still be some memory leak if creating the index standalone it needs only about 50% of memory. Actually it would be really great if the index creation could be lighter on memory, maybe using a temporary file for buffer?

WanMil



2. Also as running out of memory primarily happens on index creation, it
would be great if mkgmap writes out the index under a temporary name,
and once it has passed correctly, renames the file. Else it is not
possible to easily assess in a bash/batch script whether mkgmap
completed the map correctly or not (and hence one uploads broken maps if
everything automated). Or at least I don't see any way to find out
whether it passed correctly or not.