Automatic block-size setting
Hi The branch auto-block contains changes to automatically find a suitable value for the --block-size option. Previously the default block size was almost always OK, but with the DEM changes the tiles are larger and a larger block size is sometimes required. All kinds of .img file (tiles, overview, gmapsupp and mdr) now work the same way. Gmapsupp.img files have always selected an appropriate block size, so there should not be much change there. Regular tiles may be a little bit smaller (~1%) as a larger block size sometime results in a file that is a bit smaller and if they are large enough to require a larger block size that will happen automatically. The osmmap_mdr.img files will typically be a lot smaller as previously they had a fixed block size of 16M and wasted a lot of space. Now the correctly block size will be selected and the file will be as small as possible. Currently the --block-size option still exists, but it just determines the smallest block size that will be used - if it is not large enough it will be increased. I could remove the option altogether or make it override the automatically determined value. ..Steve
Hi Steve, I think best would be to ignore the --block-size option, at least for a for a while like we do with --remove-short-arcs. This makes it easier to compare results of two different binaries. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Steve Ratcliffe <steve@parabola.me.uk> Gesendet: Mittwoch, 17. Januar 2018 23:41:08 An: mkgmap development Betreff: [mkgmap-dev] Automatic block-size setting Hi The branch auto-block contains changes to automatically find a suitable value for the --block-size option. Previously the default block size was almost always OK, but with the DEM changes the tiles are larger and a larger block size is sometimes required. All kinds of .img file (tiles, overview, gmapsupp and mdr) now work the same way. Gmapsupp.img files have always selected an appropriate block size, so there should not be much change there. Regular tiles may be a little bit smaller (~1%) as a larger block size sometime results in a file that is a bit smaller and if they are large enough to require a larger block size that will happen automatically. The osmmap_mdr.img files will typically be a lot smaller as previously they had a fixed block size of 16M and wasted a lot of space. Now the correctly block size will be selected and the file will be as small as possible. Currently the --block-size option still exists, but it just determines the smallest block size that will be used - if it is not large enough it will be increased. I could remove the option altogether or make it override the automatically determined value. ..Steve _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Steve, I have compiled several maps with r4070 without even noticing, that it include your new code. I think it means, that you have done a good work, thanks! I have looked at block sizes and all is reasonable. Tiles are usually 2k or 4k, ovm_* are 512 or 1k, MDR are 8k or bigger. I guess you have set some preference for bigger blocks in case of MDR. -- Best regards, Andrzej
Hi Andrzej
I have looked at block sizes and all is reasonable. Tiles are usually 2k or 4k, ovm_* are 512 or 1k, MDR are 8k or bigger. I guess you have set some preference for bigger blocks in case of MDR.
I read your post where you noted that often bigger block sizes lead to a smaller file, so I made it calculate the whole file size and then pick the smallest one. Unless there is a bug, there should not be any special preference for larger blocks in the MDR case, I just did a run with trace information and I got: INFO: bs=512, whole size=47126016, hb=386, fb=91657, blocks=92043 INFO: bs=1024, whole size=47029248, hb=98, fb=45829, blocks=45927 INFO: bs=2048, whole size=46981120, hb=25, fb=22915, blocks=22940 INFO: bs=4096, whole size=46960640, hb=7, fb=11458, blocks=11465 INFO: bs=8192, whole size=46956544, hb=2, fb=5730, blocks=5732 INFO: bs=16384, whole size=46972928, hb=1, fb=2866, blocks=2867 [ hb = number of header blocks, fb = number of file blocks ] So the block size = 8192 does seem to give the lowest file size; only by 4k bytes. I was a bit suprised, but the number of header blocks needed goes down by a quarter when the block size is doubled, and that makes up for the wastage of bigger blocks. ..Steve
Hi Steve, it was my first impression, that MDR got only big blocks, but you are right, there are all sizes used. Now that I have looked more carefully, I see some inconsistency. MDR img has always the same structure, I would suspect, that block size increase with file size, but it is not true. See list of my files sorted by img size: File: SAmTopo_mdr.img, length 176291840 fat: 400h - 600h - 6200h, block 16384 File: SAmOTBR_mdr.img, length 113917952 fat: 400h - 600h - 7C00h, block 8192 File: AfricaHR_mdr.img, length 56901632 fat: 400h - 600h - 2600h, block 16384 File: SAmOTS_mdr.img, length 35307520 fat: 400h - 600h - 2C00h, block 8192 File: SAmOTN_mdr.img, length 18628608 fat: 400h - 600h - 1C00h, block 8192 File: SAmOTEC_mdr.img, length 17248256 fat: 400h - 600h - 2C00h, block 4096 File: AfricaN_mdr.img, length 13844480 fat: 400h - 600h - 1800h, block 8192 File: AfricaS_mdr.img, length 11411456 fat: 400h - 600h - 2000h, block 4096 File: AfricaW_mdr.img, length 8376320 fat: 400h - 600h - 1A00h, block 4096 File: AfricaC_mdr.img, length 7897088 fat: 400h - 600h - 1A00h, block 4096 File: AfricaE_mdr.img, length 6871040 fat: 400h - 600h - 2400h, block 2048 File: AfricaSE_mdr.img, length 4153344 fat: 400h - 600h - 1A00h, block 2048 File: AfricaSW_mdr.img, length 3403776 fat: 400h - 600h - 1000h, block 4096 File: AfricaNE_mdr.img, length 2904064 fat: 400h - 600h - E00h, block 4096 Probably something unimportant, but a bit surprising. -- Best regards, Andrzej
participants (3)
-
Andrzej Popowski -
Gerd Petermann -
Steve Ratcliffe