I fear that the Europe map became too big for mkgmap to put it into one mapset. I have now 2293 img tiles that I wanted to combine with mkgmap-r3590 The java memory is set to -Xmx8000m, the error message is as follows: SEVERE (MapFailedException): (thrown in BufferedImgFileWriter.ensureSize()) There is not enough room in a single garmin map for all the input data. The .osm file should be split into smaller pieces first. There is not enough room in a single garmin map for all the input data. The .osm file should be split into smaller pieces first. What does this mean? Is my index file too big? Is there another way to combine the img tiles to make a single mapset?
Hi Minko, yes, the message is probably produced while writing the index file. Presuming that this limit really exists there is probably no program that works. You may try if Basecamp is able to produce a gmapsupp for all tiles. If that doesn't work I assume that you have to reduce the number of items that are indexed, e.g. POI. Or maybe we can add code so that some POI are created but not indexed. If I got that right some devices would still show them. Gerd
Date: Wed, 13 May 2015 21:37:06 +0200 From: ligfietser@online.nl To: mkgmap-dev@lists.mkgmap.org.uk Subject: [mkgmap-dev] EU map too big?
I fear that the Europe map became too big for mkgmap to put it into one mapset. I have now 2293 img tiles that I wanted to combine with mkgmap-r3590 The java memory is set to -Xmx8000m, the error message is as follows:
SEVERE (MapFailedException): (thrown in BufferedImgFileWriter.ensureSize()) There is not enough room in a single garmin map for all the input data. The .osm file should be split into smaller pieces first. There is not enough room in a single garmin map for all the input data. The .osm file should be split into smaller pieces first.
What does this mean? Is my index file too big? Is there another way to combine the img tiles to make a single mapset? _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Gerd, No, I can't create a gmapsupp because the whole Europe map is 11 Gb and the maximum size is 4Gb for one gmapsupp. So I need to split my Europe map in several gmapsupp's but that's not a problem. Last year I could manage this with 3, but now I need at least 5. However, I've also seen Mkgmap producing gmapsupp files >5 Gb which doesnt make any sense, because Garmin dont support this. Problem is not the gmapsupp set, but the mapset for the pc / Basecamp. I have set x-split-name-index and also added housenumbers from the French address database. I guess it would help to throw them out. Last option is no big Europe map anymore or on without a search index... :-/ ----- Oorspronkelijk bericht -----
Van: "Gerd Petermann" <gpetermann_muenchen@hotmail.com> Aan: mkgmap-dev@lists.mkgmap.org.uk Verzonden: Woensdag 13 mei 2015 22:18:33 Onderwerp: Re: [mkgmap-dev] EU map too big?
Hi Minko,
yes, the message is probably produced while writing the index file. Presuming that this limit really exists there is probably no program that works. You may try if Basecamp is able to produce a gmapsupp for all tiles. If that doesn't work I assume that you have to reduce the number of items that are indexed, e.g. POI.
Or maybe we can add code so that some POI are created but not indexed. If I got that right some devices would still show them.
Gerd
Hi Minko, ah, ok, so you don't have the problem with the index files contained in the 5 parts. I think you should wait for a comment from Steve regarding the limit, a comment i svn says that there is no known limit, so maybe the hard coded value 0xffffff is not needed. Gerd ligfietser wrote
Hi Gerd, No, I can't create a gmapsupp because the whole Europe map is 11 Gb and the maximum size is 4Gb for one gmapsupp. So I need to split my Europe map in several gmapsupp's but that's not a problem. Last year I could manage this with 3, but now I need at least 5. However, I've also seen Mkgmap producing gmapsupp files >5 Gb which doesnt make any sense, because Garmin dont support this.
Problem is not the gmapsupp set, but the mapset for the pc / Basecamp. I have set x-split-name-index and also added housenumbers from the French address database. I guess it would help to throw them out. Last option is no big Europe map anymore or on without a search index... :-/
----- Oorspronkelijk bericht -----
Van: "Gerd Petermann" <
gpetermann_muenchen@
>
Aan:
mkgmap-dev@.org
Verzonden: Woensdag 13 mei 2015 22:18:33 Onderwerp: Re: [mkgmap-dev] EU map too big?
Hi Minko,
yes, the message is probably produced while writing the index file. Presuming that this limit really exists there is probably no program that works. You may try if Basecamp is able to produce a gmapsupp for all tiles. If that doesn't work I assume that you have to reduce the number of items that are indexed, e.g. POI.
Or maybe we can add code so that some POI are created but not indexed. If I got that right some devices would still show them.
Gerd
_______________________________________________ mkgmap-dev mailing list
mkgmap-dev@.org
-- View this message in context: http://gis.19327.n5.nabble.com/EU-map-too-big-tp5844506p5844513.html Sent from the Mkgmap Development mailing list archive at Nabble.com.
Hi Gerd, There was no problem when I skipped about 200 tiles (~1,5 Gb), the index was created. So the limit seems to be around 2000 tiles / 10 Gb of data. I will try to skip split-name-index, lets see if that will work...
Hi Minko, I fear I was wrong, since r1290 the index size should not be a problem. So, that leaves the overview map as a candidate. Gerd ligfietser wrote
Hi Gerd,
There was no problem when I skipped about 200 tiles (~1,5 Gb), the index was created. So the limit seems to be around 2000 tiles / 10 Gb of data. I will try to skip split-name-index, lets see if that will work... _______________________________________________ mkgmap-dev mailing list
mkgmap-dev@.org
-- View this message in context: http://gis.19327.n5.nabble.com/EU-map-too-big-tp5844506p5844541.html Sent from the Mkgmap Development mailing list archive at Nabble.com.
Hi, On Wed, May 13, Minko wrote:
I fear that the Europe map became too big for mkgmap to put it into one mapset. I have now 2293 img tiles that I wanted to combine with mkgmap-r3590 The java memory is set to -Xmx8000m, the error message is as follows:
Maybe you should write more in detail what exaclty you try to do? For one gmapsupp, whole europe is of course far too big. And with 2293 img tiles you have too much, too. The number of img tiles is limited to 2000 for older devices. Don't know where the limit is for newer devices, I think 5000 tiles. I have no problems creating the data for whole europe with about 1300 img tiles. But I use "--tdbfile" and not "--gmapsupp". Afterwards I can select the img tiles I need and write them with MapSource to a GPS device. Thorsten -- Thorsten Kukuk, Senior Architect SLES & Common Code Base SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany GF: Felix Imendörffer, Jane Smithard, Dilip Upmanyu, Graham Norton, HRB 21284 (AG Nürnberg)
Hi Thorsten, I'm trying to make one mapset of Europe for the pc, not for the device. For the device I will make 5 sets, each < 4gb For the older devices, there is no issue, it's limit is one gmapsupp set which is 4Gb max. So this is far less than those 2000 tiles. Would it help to make bigger tiles with the splitter? I now use max-nodes=1800000
On Wed, May 13, Minko wrote:
I'm trying to make one mapset of Europe for the pc, not for the device.
That's what I'm doing, too.
Would it help to make bigger tiles with the splitter? I now use max-nodes=1800000
I use 1200000, which creates about 1300 img tiles for me, and have no problems. Thorsten -- Thorsten Kukuk, Senior Architect SLES & Common Code Base SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany GF: Felix Imendörffer, Jane Smithard, Dilip Upmanyu, Graham Norton, HRB 21284 (AG Nürnberg)
The lower the number, it will produce more tiles, the more tiles will not harm the routing long distance? _______________________________________________________________________________________________________
Date: Wed, 13 May 2015 23:18:09 +0200 From: kukuk@suse.de To: mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] EU map too big?
On Wed, May 13, Minko wrote:
I'm trying to make one mapset of Europe for the pc, not for the device.
That's what I'm doing, too.
Would it help to make bigger tiles with the splitter? I now use max-nodes=1800000
I use 1200000, which creates about 1300 img tiles for me, and have no problems.
Thorsten
-- Thorsten Kukuk, Senior Architect SLES & Common Code Base SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany GF: Felix Imendörffer, Jane Smithard, Dilip Upmanyu, Graham Norton, HRB 21284 (AG Nürnberg) _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Minko, I've just noticed that the merge-lines option was ignored and disabled by default when the overview map was created. I've changed that with r3591. The effect is great, the overview map size is reduced by > 30% . I have to look into the shape merging process, it does not yet work with the overview map. Gerd
Date: Wed, 13 May 2015 21:37:06 +0200 From: ligfietser@online.nl To: mkgmap-dev@lists.mkgmap.org.uk Subject: [mkgmap-dev] EU map too big?
I fear that the Europe map became too big for mkgmap to put it into one mapset. I have now 2293 img tiles that I wanted to combine with mkgmap-r3590 The java memory is set to -Xmx8000m, the error message is as follows:
SEVERE (MapFailedException): (thrown in BufferedImgFileWriter.ensureSize()) There is not enough room in a single garmin map for all the input data. The .osm file should be split into smaller pieces first. There is not enough room in a single garmin map for all the input data. The .osm file should be split into smaller pieces first.
What does this mean? Is my index file too big? Is there another way to combine the img tiles to make a single mapset? _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi again, shapes turned out to be simple, with r3592 the overview map file size is again smaller, esp. when the map contains sea or forest polygons. The ratio depends on the style, the 30% below were for the default style which adds motorways. For a map of Ireland, I see now 98KB with r3592 and 142KB with r3590, both with "Openfietsmap full" style. Gerd From: gpetermann_muenchen@hotmail.com To: mkgmap-dev@lists.mkgmap.org.uk Date: Thu, 14 May 2015 07:23:08 +0200 Subject: Re: [mkgmap-dev] EU map too big? Hi Minko, I've just noticed that the merge-lines option was ignored and disabled by default when the overview map was created. I've changed that with r3591. The effect is great, the overview map size is reduced by > 30% . I have to look into the shape merging process, it does not yet work with the overview map. Gerd
Date: Wed, 13 May 2015 21:37:06 +0200 From: ligfietser@online.nl To: mkgmap-dev@lists.mkgmap.org.uk Subject: [mkgmap-dev] EU map too big?
I fear that the Europe map became too big for mkgmap to put it into one mapset. I have now 2293 img tiles that I wanted to combine with mkgmap-r3590 The java memory is set to -Xmx8000m, the error message is as follows:
SEVERE (MapFailedException): (thrown in BufferedImgFileWriter.ensureSize()) There is not enough room in a single garmin map for all the input data. The .osm file should be split into smaller pieces first. There is not enough room in a single garmin map for all the input data. The .osm file should be split into smaller pieces first.
What does this mean? Is my index file too big? Is there another way to combine the img tiles to make a single mapset? _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Thanks Gerd, My old ovm_* files were 236 MB. I havent created new img tiles yet with r3595. If I run mkgmap on the old img files the same error is thrown out so I assume I must compile the tiles again. I'll report it here later today because whole europe will take some time.
shapes turned out to be simple, with r3592 the overview map file size is again smaller, esp. when the map contains sea or forest polygons. The ratio depends on the style, the 30% below were for the default style which adds motorways. For a map of Ireland, I see now 98KB with r3592 and 142KB with r3590, both with "Openfietsmap full" style.
Gerd
Hi Minko, hmm, there is no need to create the ovm_* files again. Maybe the problem is not the overview map? Please double check that you use r3595. The limit that is reached is 16MB. Is the compiled overview map ~ that size? Gerd Date: Thu, 14 May 2015 09:55:00 +0200 From: ligfietser@online.nl To: mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] EU map too big? Thanks Gerd, My old ovm_* files were 236 MB. I havent created new img tiles yet with r3595. If I run mkgmap on the old img files the same error is thrown out so I assume I must compile the tiles again. I'll report it here later today because whole europe will take some time. shapes turned out to be simple, with r3592 the overview map file size is again smaller, esp. when the map contains sea or forest polygons. The ratio depends on the style, the 30% below were for the default style which adds motorways. For a map of Ireland, I see now 98KB with r3592 and 142KB with r3590, both with "Openfietsmap full" style. Gerd _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
That's the problem, I cannot even create an overview map. All I see are tmp files. If I discard the UK & Ireland (68 tiles only, 800MB) I can create the total map of Europe. The mdr size is 373 MB and img size 12 MB. I fear I'm hitting the limit of the index siz, I've also discarded the x-split-name-index option but it didnt help. BTW @Thorsten, I was counting ovm files too, so the total number of img is 1145 tiles im using splitter 321 does it make sense to split all tiles again with 421? ----- Oorspronkelijk bericht ----- Van: "Gerd Petermann" <gpetermann_muenchen@hotmail.com> Aan: mkgmap-dev@lists.mkgmap.org.uk Verzonden: Donderdag 14 mei 2015 10:03:28 Onderwerp: Re: [mkgmap-dev] EU map too big? Hi Minko, hmm, there is no need to create the ovm_* files again. Maybe the problem is not the overview map? Please double check that you use r3595. The limit that is reached is 16MB. Is the compiled overview map ~ that size? Gerd
Hi Minko, seems we first have to find out which file is causing the problem. I try to add some code so that a name is reported when the error occurs. 12MB with the new version probably means > 16MB with r3590 and older, so I think that was the problem, but you say that you still see the error... Gerd Date: Thu, 14 May 2015 10:34:13 +0200 From: ligfietser@online.nl To: mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] EU map too big? That's the problem, I cannot even create an overview map. All I see are tmp files. If I discard the UK & Ireland (68 tiles only, 800MB) I can create the total map of Europe. The mdr size is 373 MB and img size 12 MB. I fear I'm hitting the limit of the index siz, I've also discarded the x-split-name-index option but it didnt help. BTW@Thorsten, I was counting ovm files too, so the total number of img is 1145 tiles im using splitter 321 does it make sense to split all tiles again with 421? Van: "Gerd Petermann" <gpetermann_muenchen@hotmail.com> Aan: mkgmap-dev@lists.mkgmap.org.uk Verzonden: Donderdag 14 mei 2015 10:03:28 Onderwerp: Re: [mkgmap-dev] EU map too big? Hi Minko, hmm, there is no need to create the ovm_* files again. Maybe the problem is not the overview map? Please double check that you use r3595. The limit that is reached is 16MB. Is the compiled overview map ~ that size? Gerd _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
I'm trying to throw more UK files into the Europe map now. It is not one particular img, because yesterday I also succeeded when leaving a bit of Eastern Europe out of it. So I'm pretty sure there must be some limits. ----- Oorspronkelijk bericht -----
Van: "Gerd Petermann" <gpetermann_muenchen@hotmail.com> Aan: mkgmap-dev@lists.mkgmap.org.uk Verzonden: Donderdag 14 mei 2015 10:41:13 Onderwerp: Re: [mkgmap-dev] EU map too big?
Hi Minko,
seems we first have to find out which file is causing the problem. I try to add some code so that a name is reported when the error occurs.
12MB with the new version probably means > 16MB with r3590 and older, so I think that was the problem, but you say that you still see the error...
Gerd
Date: Thu, 14 May 2015 10:34:13 +0200 From: ligfietser@online.nl To: mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] EU map too big?
That's the problem, I cannot even create an overview map. All I see are tmp files. If I discard the UK & Ireland (68 tiles only, 800MB) I can create the total map of Europe. The mdr size is 373 MB and img size 12 MB. I fear I'm hitting the limit of the index siz, I've also discarded the x-split-name-index option but it didnt help.
BTW @Thorsten, I was counting ovm files too, so the total number of img is 1145 tiles
im using splitter 321 does it make sense to split all tiles again with 421?
Van: "Gerd Petermann" <gpetermann_muenchen@hotmail.com>
Aan: mkgmap-dev@lists.mkgmap.org.uk
Verzonden: Donderdag 14 mei 2015 10:03:28
Onderwerp: Re: [mkgmap-dev] EU map too big?
Hi Minko,
hmm, there is no need to create the ovm_* files again. Maybe the problem
is not the overview map?
Please double check that you use r3595.
The limit that is reached is 16MB. Is the compiled overview map ~ that size?
Gerd
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Minko, okay, I have to leave now for a few hours. Gerd Date: Thu, 14 May 2015 10:44:37 +0200 From: ligfietser@online.nl To: mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] EU map too big? I'm trying to throw more UK files into the Europe map now. It is not one particular img, because yesterday I also succeeded when leaving a bit of Eastern Europe out of it. So I'm pretty sure there must be some limits. Van: "Gerd Petermann" <gpetermann_muenchen@hotmail.com> Aan: mkgmap-dev@lists.mkgmap.org.uk Verzonden: Donderdag 14 mei 2015 10:41:13 Onderwerp: Re: [mkgmap-dev] EU map too big? Hi Minko, seems we first have to find out which file is causing the problem. I try to add some code so that a name is reported when the error occurs. 12MB with the new version probably means > 16MB with r3590 and older, so I think that was the problem, but you say that you still see the error... Gerd Date: Thu, 14 May 2015 10:34:13 +0200 From: ligfietser@online.nl To: mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] EU map too big? That's the problem, I cannot even create an overview map. All I see are tmp files. If I discard the UK & Ireland (68 tiles only, 800MB) I can create the total map of Europe. The mdr size is 373 MB and img size 12 MB. I fear I'm hitting the limit of the index siz, I've also discarded the x-split-name-index option but it didnt help. BTW@Thorsten, I was counting ovm files too, so the total number of img is 1145 tiles im using splitter 321 does it make sense to split all tiles again with 421? Van: "Gerd Petermann" <gpetermann_muenchen@hotmail.com> Aan: mkgmap-dev@lists.mkgmap.org.uk Verzonden: Donderdag 14 mei 2015 10:03:28 Onderwerp: Re: [mkgmap-dev] EU map too big? Hi Minko, hmm, there is no need to create the ovm_* files again. Maybe the problem is not the overview map? Please double check that you use r3595. The limit that is reached is 16MB. Is the compiled overview map ~ that size? Gerd _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Gerd, Probably something went wrong the first time. I now managed to produce a full map with all the tiles and mkgmap-r3595. Thanks again for the quick fixes!
Hi Minko, that' s good news :-) When you reach the limit again you may have to remove some details from the overview map, e.g. use level 19/20 instead of 18 or remove the "background" type 0x10100. I only see one potential improvement in the program: Many bytes are used to store sea / coastline polygons. If we ever manage to find a better algo to simplify them for the low Garmin resolutions we may be able to store better info with fewer bytes. The current combination of filters tends to produce self-intersecting ugly looking triangles which often contain spikes, which means they have redundant bytes. @Mike Baggaley: I think your pointInPolygon-v0.patch showed a completely new approach to simplify polygons using the Java Graphics2D.fillPolygon() method. Maybe this could be used as a new filter for shapes in MapBuilder. It seems to do exactly what we want. Gerd Date: Thu, 14 May 2015 11:17:20 +0200 From: ligfietser@online.nl To: mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] EU map too big? Hi Gerd, Probably something went wrong the first time. I now managed to produce a full map with all the tiles and mkgmap-r3595. Thanks again for the quick fixes! _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Thanks Gerd, I have now updated and uploaded the Europe map in case you'll make another bike trip ;-) http://www.openfietsmap.nl/downloads/europe BTW Is 16 Mb still the limit for the overview map?
Hi Minko, I don't know much about the reason for this 16MB limit. I found this in imgformat-1.0.pdf: "This definition implies that an IMG file can never have more than 65,535 blocks, but the use of large block sizes (2048, 4096, etc…) allow the IMG file to hold several gigabytes of data. The only practical limit on IMG file size seems to be the use of 32-bit integers for file offsets, which imposes a hard limit of 4 GB. Large sub-files, those with more than 240 blocks of data, are spanned across multiple FAT blocks with each sub-file part numbered in a 32-bit integer at offset 0x10. File size is only present in the first of these (part 0x0000)." 65535 * 256 ~ 16BG, so I assume that this is the reason. I see files which use a different blocksize, so maybe this limit makes no more sense. @Steve: Can you have a look at this, please? Gerd ligfietser wrote
Thanks Gerd, I have now updated and uploaded the Europe map in case you'll make another bike trip ;-) http://www.openfietsmap.nl/downloads/europe
BTW Is 16 Mb still the limit for the overview map?
_______________________________________________ mkgmap-dev mailing list
mkgmap-dev@.org
-- View this message in context: http://gis.19327.n5.nabble.com/EU-map-too-big-tp5844506p5844688.html Sent from the Mkgmap Development mailing list archive at Nabble.com.
Hi
"This definition implies that an IMG file can never have more than 65,535 blocks, but the use of large block sizes (2048, 4096, etc…) allow the IMG file to hold several gigabytes of data. The only practical limit on IMG file size seems to be the use of 32-bit integers for file offsets, which imposes a hard limit of 4 GB. Large sub-files, those with more than 240 blocks of data, are spanned across multiple FAT blocks with each sub-file part numbered in a 32-bit integer at offset 0x10. File size is only present in the first of these (part 0x0000)."
65535 * 256 ~ 16BG, so I assume that this is the reason. I see files which use a different blocksize, so maybe this limit makes no more sense.
@Steve: Can you have a look at this, please?
The img file as a whole can be very large by using a large block size as the quoted passage states. This is how we create the global index and the gmapsupp files. (We may not however automatically adjust the block size for regular tiles, but that could be fixed if it became a problem.) The bigger issue is the limits on the individual files (RGN etc) within the .img file. For example the subdivision pointers from TRE into RGN are 3 byte quantities and so this means that the max possible offset into RGN for the start of a subdivision is around 16M. It is possible that there are flags that allows you to extend this, but I don't know of any, nor have I seen any files that break this limit. ..Steve
Hi Steve, thanks for the clarification. I first thought that the 16MB limit is for the *.img file, but in fact it is for each subfile (RGN,LBL,...) within one *.img file. I played a bit with the limits and noticed that we don't even reach the 16 MB when the "too large" message appears because we use 128 KB steps to enlarge the buffer. FWIW : Attached is a small patch to fix that, it still uses the "1 KB GUARD_SIZE". (With the patch) I can safely create an *.img file that has ~ 18 MB, while the RGN file size is 0xfebfb9, which is still below the limit. Do you think it's okay to commit that? Gerd
Date: Fri, 15 May 2015 11:45:31 +0100 From: steve@parabola.me.uk To: mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] EU map too big?
Hi
"This definition implies that an IMG file can never have more than 65,535 blocks, but the use of large block sizes (2048, 4096, etc…) allow the IMG file to hold several gigabytes of data. The only practical limit on IMG file size seems to be the use of 32-bit integers for file offsets, which imposes a hard limit of 4 GB. Large sub-files, those with more than 240 blocks of data, are spanned across multiple FAT blocks with each sub-file part numbered in a 32-bit integer at offset 0x10. File size is only present in the first of these (part 0x0000)."
65535 * 256 ~ 16BG, so I assume that this is the reason. I see files which use a different blocksize, so maybe this limit makes no more sense.
@Steve: Can you have a look at this, please?
The img file as a whole can be very large by using a large block size as the quoted passage states. This is how we create the global index and the gmapsupp files.
(We may not however automatically adjust the block size for regular tiles, but that could be fixed if it became a problem.)
The bigger issue is the limits on the individual files (RGN etc) within the .img file. For example the subdivision pointers from TRE into RGN are 3 byte quantities and so this means that the max possible offset into RGN for the start of a subdivision is around 16M.
It is possible that there are flags that allows you to extend this, but I don't know of any, nor have I seen any files that break this limit.
..Steve _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
participants (6)
-
A. Carlos -
Gerd Petermann -
GerdP
-
Minko -
Steve Ratcliffe -
Thorsten Kukuk