As far a I know, mkgmap is able to produce unlimited size gmapsupp files, but SD cards and internal device memories have a limit of 4 GB in a single file. To overcome that limit, recent versions of Garmin City Navigator maps are split into several gmapsupp that work as a single one in devices which include them. Would it be possible to add an option to mkgmap, so that it automatically splits gmapsupp when you process a map that is larger than 4 GB?
Hi Carlos, see http://gis.19327.n8.nabble.com/Split-gmapsupp-img-files-over-4GB-tc5967656.h... I tried to code this at that time but got stuck because I didn't have the time to create such a large map to find out where MapInstall places the other files (e.g. MDR, TYP and if any other file or flag is needed. Maybe I'll give it another try with a download of Europe made in 2019. Splitter will stress my PC for quite a while ... Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Carlos Dávila <carlos@alternativaslibres.org> Gesendet: Sonntag, 18. Oktober 2020 09:14 An: Development list for mkgmap Betreff: [mkgmap-dev] Split gmapsupp.img As far a I know, mkgmap is able to produce unlimited size gmapsupp files, but SD cards and internal device memories have a limit of 4 GB in a single file. To overcome that limit, recent versions of Garmin City Navigator maps are split into several gmapsupp that work as a single one in devices which include them. Would it be possible to add an option to mkgmap, so that it automatically splits gmapsupp when you process a map that is larger than 4 GB? _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
I'll have a look at that thread later. I can process large files in my server if needed. El 18/10/20 a las 9:50, Gerd Petermann escribió:
Hi Carlos,
see http://gis.19327.n8.nabble.com/Split-gmapsupp-img-files-over-4GB-tc5967656.h...
I tried to code this at that time but got stuck because I didn't have the time to create such a large map to find out where MapInstall places the other files (e.g. MDR, TYP and if any other file or flag is needed. Maybe I'll give it another try with a download of Europe made in 2019. Splitter will stress my PC for quite a while ...
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Carlos Dávila <carlos@alternativaslibres.org> Gesendet: Sonntag, 18. Oktober 2020 09:14 An: Development list for mkgmap Betreff: [mkgmap-dev] Split gmapsupp.img
As far a I know, mkgmap is able to produce unlimited size gmapsupp files, but SD cards and internal device memories have a limit of 4 GB in a single file. To overcome that limit, recent versions of Garmin City Navigator maps are split into several gmapsupp that work as a single one in devices which include them. Would it be possible to add an option to mkgmap, so that it automatically splits gmapsupp when you process a map that is larger than 4 GB?
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
The other files have to be split as well I am pretty sure. Each map should also work independently. So it's just a matter of way take 3700MB for .IMG files to have enough overhead for the MDR/MDX. Except if no address search, then take 4050MB for split. On Sun, 18 Oct 2020, 11:16 Carlos Dávila <carlos@alternativaslibres.org> wrote:
I'll have a look at that thread later. I can process large files in my server if needed.
El 18/10/20 a las 9:50, Gerd Petermann escribió:
Hi Carlos,
see http://gis.19327.n8.nabble.com/Split-gmapsupp-img-files-over-4GB-tc5967656.h...
I tried to code this at that time but got stuck because I didn't have the time to create such a large map to find out where MapInstall places the other files (e.g. MDR, TYP and if any other file or flag is needed. Maybe I'll give it another try with a download of Europe made in 2019. Splitter will stress my PC for quite a while ...
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Carlos Dávila <carlos@alternativaslibres.org> Gesendet: Sonntag, 18. Oktober 2020 09:14 An: Development list for mkgmap Betreff: [mkgmap-dev] Split gmapsupp.img
As far a I know, mkgmap is able to produce unlimited size gmapsupp files, but SD cards and internal device memories have a limit of 4 GB in a single file. To overcome that limit, recent versions of Garmin City Navigator maps are split into several gmapsupp that work as a single one in devices which include them. Would it be possible to add an option to mkgmap, so that it automatically splits gmapsupp when you process a map that is larger than 4 GB?
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
The 4GB limit is a result of the fact that Garmin devices use FAT16B without LFS (only the very oldest) and FAT32 (all the rest) filesystems. Garmin devices do not support exFAT; though, devices with rewritable firmware could be made to support exFAT. The reason that FAT file systems are used are they are freely available and work on Linux, Windows and Apple operating systems. Please, see: https://en.wikipedia.org/wiki/File_Allocation_Table https://physics.nist.gov/cuu/Units/binary.html https://en.wikipedia.org/wiki/ExFAT There are several sizes of concern: cluster size, maximum file size, maximum number of files, and maximum file system size. Each of these sizes are powers of two or powers of 2 less one. Because of the powers of 2, SI binary prefixes will be used. Remember only one file can be stored in a cluster and a file may have more than one cluster. The space at the end of the last cluster of a file is wasted. FAT12 4KiB and 8KiB clusters, file size limited by volume size, 4068 files with 8 KiB clusters, 16MiB with 4KiB and 32MiB with 8 KiB clusters. FAT16B minimum file system sizes 8 MiB with 128, 32 MiB with 512*, and 256 MiB 4 KiB disk logical sector sizes, 2GiB-1 without LFS and 4GiB-1 with LFS, cluster sizes are powers of 2 from 512 to 32768, 65460 files with 32 KiB clusters, maximum file systems size is 65525 clusters. FAT32 32 MiB-4.5 KiB (with 65525 clusters and 512 byte sectors) and 256 MiB-36 KiB (with 65525 clusters and 4 KiB sectors), same filesize limits as FAT16B, cluster sizes that are powers of 2 from 512 to 2,097,152. exFAT maximum file size practical 128 PiB-1, structural limit 16 EiB-1, maximum volume size practical 128 PiB, maximum number of files per directory 2,796,202. * almost all non CHR disks are 512 byte sectored. People will tell you that you can not use a SD card larger than a certain size dependent on Garmin receiver model. This is not true. The actual requirement is that a Garmin receiver only looks that the first primary partition on a SD card. Simply, divide the SD card into two partitions (in Windows, the disk manager can do this), format that partition as the appropriate FAT format, and put the map or maps there. The second partition can be formatted as one desires and used as desired. This is a handy space in which to store other previously made maps or even a complete map building system. I have done this on Garmin receivers as old as Garmin StreetPilot 2610 with SD cards as large as 256 GB (the card uses some of the memory for its management purposes). On 10/18/2020 2:14 AM, Carlos Dávila wrote:
As far as I know, mkgmap is able to produce unlimited size gmapsupp files, but SD cards and internal device memories have a limit of 4 GB in a single file. To overcome that limit, recent versions of Garmin City Navigator maps are split into several gmapsupp that work as a single one in devices which include them. Would it be possible to add an option to mkgmap, so that it automatically splits gmapsupp when you process a map that is larger than 4 GB?
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Carlos, splitting could be useful, when you'd like to create img in one step. Other option is to install map on PC and then use MapInstall to create img. MapInstall does splitting for big maps. -- Best regards, Andrzej
El 19/10/20 a las 0:30, Andrzej Popowski escribió:
Hi Carlos,
splitting could be useful, when you'd like to create img in one step. Other option is to install map on PC and then use MapInstall to create img. MapInstall does splitting for big maps.
Hi Andrej I know I can use MapSource/MapInstall to send parts of a map to device, but many people have problems using this method, so I would like to have way to provide ready to use gmapsupp files of my maps
MapInstall cannot be scripted to create a gmapsupp.img (plus setting it up on linux is a pain). I'm pretty sure the <4GB gmapsupp part files - shouldn't be too difficult if mkgmap can associate the address index for the choses tiles only. Right now for scripting I can only see how to create gmapsupp part files without address search. Cutting Europe into smaller parts using osmcut or similar takes quite a lot of time, plus if you compile a full map of Europe anyhow already - would kinda double the work. And you would need to adapt the area rather often as osm data increases... Having mkgmap split the map into parts would be great therefore (and also put out some sort of parts.list similar to the areas.list from splitter would be even better). On Mon, 19 Oct 2020 at 17:15, Carlos Dávila <carlos@alternativaslibres.org> wrote:
El 19/10/20 a las 0:30, Andrzej Popowski escribió:
Hi Carlos,
splitting could be useful, when you'd like to create img in one step. Other option is to install map on PC and then use MapInstall to create img. MapInstall does splitting for big maps.
Hi Andrej
I know I can use MapSource/MapInstall to send parts of a map to device, but many people have problems using this method, so I would like to have way to provide ready to use gmapsupp files of my maps
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- Felix Hartman - Openmtbmap.org & VeloMap.org
Hi Felix, I also thought about this alternative. Sad news that it doesn't work :( Yesterday I managed to download a map for Europe, so I think I have now the data to play with this. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Felix Hartmann <extremecarver@gmail.com> Gesendet: Montag, 19. Oktober 2020 20:37 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Split gmapsupp.img MapInstall cannot be scripted to create a gmapsupp.img (plus setting it up on linux is a pain). I'm pretty sure the <4GB gmapsupp part files - shouldn't be too difficult if mkgmap can associate the address index for the choses tiles only. Right now for scripting I can only see how to create gmapsupp part files without address search. Cutting Europe into smaller parts using osmcut or similar takes quite a lot of time, plus if you compile a full map of Europe anyhow already - would kinda double the work. And you would need to adapt the area rather often as osm data increases... Having mkgmap split the map into parts would be great therefore (and also put out some sort of parts.list similar to the areas.list from splitter would be even better). On Mon, 19 Oct 2020 at 17:15, Carlos Dávila <carlos@alternativaslibres.org<mailto:carlos@alternativaslibres.org>> wrote: El 19/10/20 a las 0:30, Andrzej Popowski escribió:
Hi Carlos,
splitting could be useful, when you'd like to create img in one step. Other option is to install map on PC and then use MapInstall to create img. MapInstall does splitting for big maps.
Hi Andrej I know I can use MapSource/MapInstall to send parts of a map to device, but many people have problems using this method, so I would like to have way to provide ready to use gmapsupp files of my maps _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev -- Felix Hartman - Openmtbmap.org & VeloMap.org
Hi Felix,
Right now for scripting I can only see how to create gmapsupp part files without address search.
Doesn't mkgmap combine *.img with correct index? I mean you can get like 1000 of tiles after compiling a big area, but you can combine first 500 and next 500 separately. I'm doing something similar for my maps. I split and compile area like Africa in one go and then I run separate tasks to create parts of Africa, without recompiling tiles. Mkgmap calculates indexes and preview maps basing on provided list of img. What could be a problem with partial maps is that one country can be split into multiple parts. This can prohibit for correct address search, if your current position is in one part, but searched address in another. But I haven't tested that. -- Best regards, Andrzej
Hi all, I tried to install a mapset larger than 4GB on my USB device with MapInstall 4.2.3 but it refuses to do so. @Felix: Can you please describe in detail how you do it? Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Andrzej Popowski <popej@poczta.onet.pl> Gesendet: Dienstag, 20. Oktober 2020 17:14 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] Split gmapsupp.img Hi Felix,
Right now for scripting I can only see how to create gmapsupp part files without address search.
Doesn't mkgmap combine *.img with correct index? I mean you can get like 1000 of tiles after compiling a big area, but you can combine first 500 and next 500 separately. I'm doing something similar for my maps. I split and compile area like Africa in one go and then I run separate tasks to create parts of Africa, without recompiling tiles. Mkgmap calculates indexes and preview maps basing on provided list of img. What could be a problem with partial maps is that one country can be split into multiple parts. This can prohibit for correct address search, if your current position is in one part, but searched address in another. But I haven't tested that. -- Best regards, Andrzej _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Gerd, empty usb drive is treated as a SD card, where only single gmapsupp.img is supported. Try to copy \Garmin\GarminDevice.xml from a GPS to pendrive, then MapInstall will treat it as a real GPS. There is other possibility. Select an empty folder on your HDD, unpack there files from attached archive and execute vgps-install.bat. This will create some config for MapInstall, which will use your selected folder as a destination for maps. You can then remove installation files from the folder. -- Best regards, Andrzej
Hi Andrej, great thanks, that seems to work fine. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Andrzej Popowski <popej@poczta.onet.pl> Gesendet: Mittwoch, 21. Oktober 2020 15:40 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] Split gmapsupp.img Hi Gerd, empty usb drive is treated as a SD card, where only single gmapsupp.img is supported. Try to copy \Garmin\GarminDevice.xml from a GPS to pendrive, then MapInstall will treat it as a real GPS. There is other possibility. Select an empty folder on your HDD, unpack there files from attached archive and execute vgps-install.bat. This will create some config for MapInstall, which will use your selected folder as a destination for maps. You can then remove installation files from the folder. -- Best regards, Andrzej
Hi Felix and Carlos, sorry, I am a bit lazy here because it's no fun to wait many minutes or hours for a result while my machine is barely usable... My current assumption is this: You use splitter to split e.g. Europe or Asia or Africa. You either split with a given split-file or you let splitter calculate the tiles each time you are splitting. The resulting OSM files are compiled and give a set of tiles, e.g. from 47110001.img to 47113456.img, all located in the same directory. You want to compile one or more gmapsupp files for this set. A single gmapsupp would be created with e.g. java -Xmx4G -jar mkgmap.jar --gmapsupp --no-tdbfile --index --description=MyMap family-id=4711 [... other options] 4711*.img My.TYP but that would be too large. Let's assume the size of all the 4711*.img files is > 4 G and below 6G my simple approach would be something like this: set options=--gmapsupp --no-tdbfile --index --description=MyMap family-id=4711 [... other options] java -Xmx4G -jar mkgmap.jar %options% 47110*.img 47111*.img My.TYP ren gmapsupp.img gmapsupp1.img java -Xmx4G -jar mkgmap.jar %options% 47112*.img 47113*.img My.TYP ren gmapsupp.img gmapsupp2.img This creates 2 files containing all the tiles, each probably < 4G. Does this work? If yes, I have an idea how to change mkgmap. @Felix: My test with MapInstall produced three files for a selection of ~6G, each between 2G and 2.5G. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Mittwoch, 21. Oktober 2020 15:50 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Split gmapsupp.img Hi Andrej, great thanks, that seems to work fine. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Andrzej Popowski <popej@poczta.onet.pl> Gesendet: Mittwoch, 21. Oktober 2020 15:40 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] Split gmapsupp.img Hi Gerd, empty usb drive is treated as a SD card, where only single gmapsupp.img is supported. Try to copy \Garmin\GarminDevice.xml from a GPS to pendrive, then MapInstall will treat it as a real GPS. There is other possibility. Select an empty folder on your HDD, unpack there files from attached archive and execute vgps-install.bat. This will create some config for MapInstall, which will use your selected folder as a destination for maps. You can then remove installation files from the folder. -- Best regards, Andrzej _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Yes this would work. Of course if mkgmap could group by region instead of by map-id it would be better - because then you could maybe just load two of the gmapsupp.img or a single one instead of all. The simplest approach would be that mkgmap just takes from *0001 to 0xxx until 3600 or 3700MB filesize is reached - then add the mdr/mdx and put it into a gmapsup1.img, then gmapsup2.img for the next 3700MB. Or maybe mkgmap splitter could be changed to group the tile numbers a bit more intelligently? my approach would be 4 gmapsupp regions 12 34 9 gmapsupp regions 124 357 689 12 regions 1 2 4 7 3 5 810 6 91112 because splitter doesn't know how big a tile will be - give an average value as option - I have 7MB per tile on average. So to be safe use 3200/7=457 - meaning 457 tiles per gmapsupp region. but I'm not sure if such a raster is possible or maybe way too difficult to program. My europe map is now about 23GB in size. So I guess in 2 years it will not fit a single microSD anymore anyhow (because some space should be left for other data). Also of course a simple overview map about the regions then would be great. But then the effort gets bigger and bigger. So maybe forget about all of this and simply create gmapsupp by tile number until 3700MB. KISS Then for Europe one could create 4 regions with OSMcut - and create 4 part Europe each part overlapping 200km or so. Each part then would be usable or many years. I kinda feel like up to 12-16GB of mapdata is not slowing down devices too much. However right now with the 4GB limit it would be too many maps to compile. On Fri, 23 Oct 2020 at 10:39, Gerd Petermann < gpetermann_muenchen@hotmail.com> wrote:
Hi Felix and Carlos,
sorry, I am a bit lazy here because it's no fun to wait many minutes or hours for a result while my machine is barely usable...
My current assumption is this: You use splitter to split e.g. Europe or Asia or Africa. You either split with a given split-file or you let splitter calculate the tiles each time you are splitting. The resulting OSM files are compiled and give a set of tiles, e.g. from 47110001.img to 47113456.img, all located in the same directory. You want to compile one or more gmapsupp files for this set. A single gmapsupp would be created with e.g. java -Xmx4G -jar mkgmap.jar --gmapsupp --no-tdbfile --index --description=MyMap family-id=4711 [... other options] 4711*.img My.TYP
but that would be too large. Let's assume the size of all the 4711*.img files is > 4 G and below 6G my simple approach would be something like this: set options=--gmapsupp --no-tdbfile --index --description=MyMap family-id=4711 [... other options] java -Xmx4G -jar mkgmap.jar %options% 47110*.img 47111*.img My.TYP ren gmapsupp.img gmapsupp1.img java -Xmx4G -jar mkgmap.jar %options% 47112*.img 47113*.img My.TYP ren gmapsupp.img gmapsupp2.img
This creates 2 files containing all the tiles, each probably < 4G.
Does this work? If yes, I have an idea how to change mkgmap. @Felix: My test with MapInstall produced three files for a selection of ~6G, each between 2G and 2.5G.
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Mittwoch, 21. Oktober 2020 15:50 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Split gmapsupp.img
Hi Andrej,
great thanks, that seems to work fine.
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Andrzej Popowski <popej@poczta.onet.pl> Gesendet: Mittwoch, 21. Oktober 2020 15:40 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] Split gmapsupp.img
Hi Gerd,
empty usb drive is treated as a SD card, where only single gmapsupp.img is supported. Try to copy \Garmin\GarminDevice.xml from a GPS to pendrive, then MapInstall will treat it as a real GPS.
There is other possibility. Select an empty folder on your HDD, unpack there files from attached archive and execute vgps-install.bat. This will create some config for MapInstall, which will use your selected folder as a destination for maps. You can then remove installation files from the folder.
-- Best regards, Andrzej _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- Felix Hartman - Openmtbmap.org & VeloMap.org
participants (5)
-
Andrzej Popowski -
Carlos Dávila -
Felix Hartmann -
Gerd Petermann -
Randolph J. Herber