Europe max-nodes 900.000 (resulting in 975 tiles, extract is about 4 weeks old already - hence 4 passes down to 1 pass ) old splitter: 52min 38 seconds (3158 seconds) splitter_patched: 29min 45 seconds (1785 seconds) splitter patched with optimize-mem: 30min 41seconds (1841 seconds) So I would support optimize-mem to stay off by default. It seems to produce worse results the more data it has to handle. The general improvement on splitting time is however great, especially on bigger files (mind mkgmap itself needs about 5x as much time as the patched splitter). Also the estimates taken by optimize-mem seem to be plain underestimated (end of the output): / MAP occupancy: 56'000'000, number of area dictionary entries: 10530 Chunk map details: used 1'447'751 (~ 4418%) of the estimated 32'768 entries. 56'000'000 ways processed... id=111312902 MAP occupancy: 57'000'000, number of area dictionary entries: 10596 Chunk map details: used 1'469'838 (~ 4486%) of the estimated 32'768 entries. 57'000'000 ways processed... id=112919265 MAP occupancy: 58'000'000, number of area dictionary entries: 10633 Chunk map details: used 1'493'839 (~ 4559%) of the estimated 32'768 entries. 58'000'000 ways processed... id=114728311 MAP occupancy: 59'000'000, number of area dictionary entries: 10660 Chunk map details: used 1'516'986 (~ 4629%) of the estimated 32'768 entries. 59'000'000 ways processed... id=116413634 MAP occupancy: 60'000'000, number of area dictionary entries: 10700 Chunk map details: used 1'541'052 (~ 4703%) of the estimated 32'768 entries. 60'000'000 ways processed... id=118147644 MAP occupancy: 61'000'000, number of area dictionary entries: 10750 Chunk map details: used 1'564'456 (~ 4774%) of the estimated 32'768 entries. 61'000'000 ways processed... id=119857968 MAP occupancy: 62'000'000, number of area dictionary entries: 10789 Chunk map details: used 1'589'073 (~ 4849%) of the estimated 32'768 entries. 62'000'000 ways processed... id=121660891 MAP occupancy: 63'000'000, number of area dictionary entries: 10828 Chunk map details: used 1'610'303 (~ 4914%) of the estimated 32'768 entries. 63'000'000 ways processed... id=123132157 MAP occupancy: 64'000'000, number of area dictionary entries: 10838 Chunk map details: used 1'632'875 (~ 4983%) of the estimated 32'768 entries. 64'000'000 ways processed... id=124780404 MAP occupancy: 65'000'000, number of area dictionary entries: 10847 Chunk map details: used 1'656'390 (~ 5055%) of the estimated 32'768 entries. 65'000'000 ways processed... id=126458865 MAP occupancy: 66'000'000, number of area dictionary entries: 10907 Chunk map details: used 1'679'991 (~ 5127%) of the estimated 32'768 entries. 66'000'000 ways processed... id=128180713 MAP occupancy: 67'000'000, number of area dictionary entries: 11004 Chunk map details: used 1'702'474 (~ 5196%) of the estimated 32'768 entries. 67'000'000 ways processed... id=129790909 MAP occupancy: 68'000'000, number of area dictionary entries: 11036 Chunk map details: used 1'724'908 (~ 5264%) of the estimated 32'768 entries. 68'000'000 ways processed... id=131429900 Writing relations Thu Nov 17 19:22:19 CET 2011 Elapsed time: 30m 6s Memory: Current 5574MB (4285MB used, 1289MB free) Max 6044MB 100'000 relations processed... id=190988 200'000 relations processed... id=349055 300'000 relations processed... id=485393 400'000 relations processed... id=952259 500'000 relations processed... id=1198900 600'000 relations processed... id=1376675 700'000 relations processed... id=1573951 800'000 relations processed... id=1743479 *********************************************************** Final statistics *********************************************************** coords occupancy MAP occupancy: 579'162'124, number of area dictionary entries: 6248 Chunk map details: used 15'886'784 (~ 48483%) of the estimated 32'768 entries. ways occupancy MAP occupancy: 68'744'281, number of area dictionary entries: 11058 Chunk map details: used 1'741'394 (~ 5314%) of the estimated 32'768 entries. Thread worker-1 has finished Thread worker-0 has finished Thread worker-2 has finished Time finished: Thu Nov 17 19:22:55 CET 2011 Total time taken: 1841s / On 17.11.2011 16:48, Felix Hartmann wrote:
some more speed observations: country germany.pbf from Geofabrik
splitter_patched with optimize-mem: 3 min 6 seconds splitter_patched without optimize-mem: 3min 2 seconds old splitter: 3 min 42 seconds
I am currently running against europe.pbf, will post the results later. But it seems that the patched mkgmap is consistently a bit faster. I think optimize-mem is better of by default though (most people using mkgmap/splitter have powerful computers anyhow). I'm interested if the 1024 max areas vs the old 255 and subsequent less runs make for a bigger speedup than on small(er) extracts.