Europe max-nodes 900.000 (resulting in 975 tiles, extract is about 4
weeks old already - hence 4 passes down to 1 pass )
old splitter: 52min 38 seconds (3158 seconds)
splitter_patched: 29min 45 seconds (1785 seconds)
splitter patched with optimize-mem: 30min 41seconds (1841 seconds)
So I would support optimize-mem to stay off by default. It seems to
produce worse results the more data it has to handle. The general
improvement on splitting time is however great, especially on bigger
files (mind mkgmap itself needs about 5x as much time as the patched
splitter). Also the estimates taken by optimize-mem seem to be plain
underestimated (end of the output):
MAP occupancy: 56'000'000, number of area dictionary entries:
10530
Chunk map details: used 1'447'751 (~ 4418%) of the estimated
32'768 entries.
56'000'000 ways processed... id=111312902
MAP occupancy: 57'000'000, number of area dictionary entries:
10596
Chunk map details: used 1'469'838 (~ 4486%) of the estimated
32'768 entries.
57'000'000 ways processed... id=112919265
MAP occupancy: 58'000'000, number of area dictionary entries:
10633
Chunk map details: used 1'493'839 (~ 4559%) of the estimated
32'768 entries.
58'000'000 ways processed... id=114728311
MAP occupancy: 59'000'000, number of area dictionary entries:
10660
Chunk map details: used 1'516'986 (~ 4629%) of the estimated
32'768 entries.
59'000'000 ways processed... id=116413634
MAP occupancy: 60'000'000, number of area dictionary entries:
10700
Chunk map details: used 1'541'052 (~ 4703%) of the estimated
32'768 entries.
60'000'000 ways processed... id=118147644
MAP occupancy: 61'000'000, number of area dictionary entries:
10750
Chunk map details: used 1'564'456 (~ 4774%) of the estimated
32'768 entries.
61'000'000 ways processed... id=119857968
MAP occupancy: 62'000'000, number of area dictionary entries:
10789
Chunk map details: used 1'589'073 (~ 4849%) of the estimated
32'768 entries.
62'000'000 ways processed... id=121660891
MAP occupancy: 63'000'000, number of area dictionary entries:
10828
Chunk map details: used 1'610'303 (~ 4914%) of the estimated
32'768 entries.
63'000'000 ways processed... id=123132157
MAP occupancy: 64'000'000, number of area dictionary entries:
10838
Chunk map details: used 1'632'875 (~ 4983%) of the estimated
32'768 entries.
64'000'000 ways processed... id=124780404
MAP occupancy: 65'000'000, number of area dictionary entries:
10847
Chunk map details: used 1'656'390 (~ 5055%) of the estimated
32'768 entries.
65'000'000 ways processed... id=126458865
MAP occupancy: 66'000'000, number of area dictionary entries:
10907
Chunk map details: used 1'679'991 (~ 5127%) of the estimated
32'768 entries.
66'000'000 ways processed... id=128180713
MAP occupancy: 67'000'000, number of area dictionary entries:
11004
Chunk map details: used 1'702'474 (~ 5196%) of the estimated
32'768 entries.
67'000'000 ways processed... id=129790909
MAP occupancy: 68'000'000, number of area dictionary entries:
11036
Chunk map details: used 1'724'908 (~ 5264%) of the estimated
32'768 entries.
68'000'000 ways processed... id=131429900
Writing relations Thu Nov 17 19:22:19 CET 2011
Elapsed time: 30m 6s Memory: Current 5574MB (4285MB used,
1289MB free) Max 6044MB
100'000 relations processed... id=190988
200'000 relations processed... id=349055
300'000 relations processed... id=485393
400'000 relations processed... id=952259
500'000 relations processed... id=1198900
600'000 relations processed... id=1376675
700'000 relations processed... id=1573951
800'000 relations processed... id=1743479
***********************************************************
Final statistics
***********************************************************
coords occupancy
MAP occupancy: 579'162'124, number of area dictionary entries:
6248
Chunk map details: used 15'886'784 (~ 48483%) of the estimated
32'768 entries.
ways occupancy
MAP occupancy: 68'744'281, number of area dictionary entries:
11058
Chunk map details: used 1'741'394 (~ 5314%) of the estimated
32'768 entries.
Thread worker-1 has finished
Thread worker-0 has finished
Thread worker-2 has finished
Time finished: Thu Nov 17 19:22:55 CET 2011
Total time taken: 1841s
On 17.11.2011 16:48, Felix Hartmann wrote:
some
more speed observations:
country germany.pbf from Geofabrik
splitter_patched with optimize-mem: 3 min 6 seconds
splitter_patched without optimize-mem: 3min 2 seconds
old splitter: 3 min 42 seconds
I am currently running against europe.pbf, will post the results
later. But it seems that the patched mkgmap is consistently a bit
faster. I think optimize-mem is better of by default though (most
people using mkgmap/splitter have powerful computers anyhow).
I'm interested if the 1024 max areas vs the old 255 and subsequent
less runs make for a bigger speedup than on small(er) extracts.