Creating gmapsupp.img fails with: Cannot invoke "String.endsWith(String)" because the return value of "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null
Partly related to the other problem - if I run mkgmap on the splitted files - I get this error. All .img files seem to be fine, but the gmpsupp.img is created with 0 bit size. This is not happening every time I resplit tiles. In this case two tiles were joined by osmconvert - and splitter splits it into three tiles. Then all .img files are created correctly, but the mapset.tdb / mapset.img files are not created - and the gmapsupp.img has 0 size, the gmapi files is also not created. I guess this happens because splitter somehow made a melange of the two input files into three? I really think the only thing missing in resplitting the too big tiles from mkgmap is an appropriate keep-complete mode - that keeps each input tile complete - but does not join input tiles. So osmconvert merging the input tiles is a bad idea even though it mostly works. C:\openmtbmap\maps>start /belownormal /b /wait java -jar -XX:+AggressiveHeap -XX:StringTableSize=1000003 -Xms5000M -Xmx43000M C:\openmtbmap\mkgmap.jar --max-jobs=12 --order-by-decreasing-area --code-page=1251 "--style-file=C:\openmtbmap\buildings_style" --levels="0:24, 1:23, 2:22, 3:21" --ignore-osm-bounds --ignore-turn-restrictions --merge-lines --allow-reverse-merge --transparent --draw-priority=28 --add-pois-to-areas --simplify-polygons=23:4,22:7,21:8 --copyright-file="C:\openmtbmap\openmtbmap_svn\copyrightopm.txt" --license-file="C:\openmtbmap\openmtbmap_svn\licenseopm.txt" --add-boundary-nodes-at-admin-boundaries=2 --poi-excl-index=0x6405,0x4316,0x2f00 --ignore-fixme-values --polygon-size-limits="24:16, 23:14, 22:12, 21:11" --ignore-turn-restrictions --description=buildings_ru --country-abbr=ru --country-name=russia --mapname=85800000 --family-id=8580 --product-id=1 --series-name="buildings_russia_31.08.2021_NU_Local" --family-name="buildings_ru_31.08.2021_NU_Local" --tdbfile --gmapi --gmapsupp --overview-mapname=mapsetb --keep-going --area-name="russia_31.08.2021_NU_Local_buildings" -c D:\openmtbmap\maps\template.russiab buildru.typ [0.005s][warning][gc,ergo] NewSize was set larger than initial heap size, will use initial heap size. Mkgmap version 4806M Time started: Tue Aug 31 21:00:52 CEST 2021 SEVERE (Main): D:\openmtbmap\maps\85800012.o5m: input file 'D:\openmtbmap\maps\85800012.o5m' doesn't exist SEVERE (Main): D:\openmtbmap\maps\85800024.o5m: input file 'D:\openmtbmap\maps\85800024.o5m' doesn't exist Number of MapFailedExceptions: 0 Exception in thread "main" java.lang.NullPointerException: Cannot invoke "String.endsWith(String)" because the return value of "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:620) at uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:126) at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:146) at uk.me.parabola.mkgmap.main.Main.main(Main.java:117) -- Felix Hartman - Openmtbmap.org & VeloMap.org
Okay - this error can be solved by removing the "Input that do not exist from the template.args input file. - e.g. if 85610002.o5m does not exist need to remove the line input-file: 85610002.o5m from the template.args file. On Tue, 31 Aug 2021 at 21:11, Felix Hartmann <extremecarver@gmail.com> wrote:
Partly related to the other problem - if I run mkgmap on the splitted files - I get this error. All .img files seem to be fine, but the gmpsupp.img is created with 0 bit size.
This is not happening every time I resplit tiles. In this case two tiles were joined by osmconvert - and splitter splits it into three tiles. Then all .img files are created correctly, but the mapset.tdb / mapset.img files are not created - and the gmapsupp.img has 0 size, the gmapi files is also not created.
I guess this happens because splitter somehow made a melange of the two input files into three?
I really think the only thing missing in resplitting the too big tiles from mkgmap is an appropriate keep-complete mode - that keeps each input tile complete - but does not join input tiles. So osmconvert merging the input tiles is a bad idea even though it mostly works.
C:\openmtbmap\maps>start /belownormal /b /wait java -jar -XX:+AggressiveHeap -XX:StringTableSize=1000003 -Xms5000M -Xmx43000M C:\openmtbmap\mkgmap.jar --max-jobs=12 --order-by-decreasing-area --code-page=1251 "--style-file=C:\openmtbmap\buildings_style" --levels="0:24, 1:23, 2:22, 3:21" --ignore-osm-bounds --ignore-turn-restrictions --merge-lines --allow-reverse-merge --transparent --draw-priority=28 --add-pois-to-areas --simplify-polygons=23:4,22:7,21:8 --copyright-file="C:\openmtbmap\openmtbmap_svn\copyrightopm.txt" --license-file="C:\openmtbmap\openmtbmap_svn\licenseopm.txt" --add-boundary-nodes-at-admin-boundaries=2 --poi-excl-index=0x6405,0x4316,0x2f00 --ignore-fixme-values --polygon-size-limits="24:16, 23:14, 22:12, 21:11" --ignore-turn-restrictions --description=buildings_ru --country-abbr=ru --country-name=russia --mapname=85800000 --family-id=8580 --product-id=1 --series-name="buildings_russia_31.08.2021_NU_Local" --family-name="buildings_ru_31.08.2021_NU_Local" --tdbfile --gmapi --gmapsupp --overview-mapname=mapsetb --keep-going --area-name="russia_31.08.2021_NU_Local_buildings" -c D:\openmtbmap\maps\template.russiab buildru.typ [0.005s][warning][gc,ergo] NewSize was set larger than initial heap size, will use initial heap size. Mkgmap version 4806M Time started: Tue Aug 31 21:00:52 CEST 2021 SEVERE (Main): D:\openmtbmap\maps\85800012.o5m: input file 'D:\openmtbmap\maps\85800012.o5m' doesn't exist SEVERE (Main): D:\openmtbmap\maps\85800024.o5m: input file 'D:\openmtbmap\maps\85800024.o5m' doesn't exist Number of MapFailedExceptions: 0 Exception in thread "main" java.lang.NullPointerException: Cannot invoke "String.endsWith(String)" because the return value of "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:620) at uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:126) at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:146) at uk.me.parabola.mkgmap.main.Main.main(Main.java:117)
-- Felix Hartman - Openmtbmap.org & VeloMap.org
-- Felix Hartman - Openmtbmap.org & VeloMap.org
Hi Felix, I've added a null check with r4807. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Felix Hartmann <extremecarver@gmail.com> Gesendet: Donnerstag, 2. September 2021 01:45 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Creating gmapsupp.img fails with: Cannot invoke "String.endsWith(String)" because the return value of "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null Okay - this error can be solved by removing the "Input that do not exist from the template.args input file. - e.g. if 85610002.o5m does not exist need to remove the line input-file: 85610002.o5m from the template.args file. On Tue, 31 Aug 2021 at 21:11, Felix Hartmann <extremecarver@gmail.com<mailto:extremecarver@gmail.com>> wrote: Partly related to the other problem - if I run mkgmap on the splitted files - I get this error. All .img files seem to be fine, but the gmpsupp.img is created with 0 bit size. This is not happening every time I resplit tiles. In this case two tiles were joined by osmconvert - and splitter splits it into three tiles. Then all .img files are created correctly, but the mapset.tdb / mapset.img files are not created - and the gmapsupp.img has 0 size, the gmapi files is also not created. I guess this happens because splitter somehow made a melange of the two input files into three? I really think the only thing missing in resplitting the too big tiles from mkgmap is an appropriate keep-complete mode - that keeps each input tile complete - but does not join input tiles. So osmconvert merging the input tiles is a bad idea even though it mostly works. C:\openmtbmap\maps>start /belownormal /b /wait java -jar -XX:+AggressiveHeap -XX:StringTableSize=1000003 -Xms5000M -Xmx43000M C:\openmtbmap\mkgmap.jar --max-jobs=12 --order-by-decreasing-area --code-page=1251 "--style-file=C:\openmtbmap\buildings_style" --levels="0:24, 1:23, 2:22, 3:21" --ignore-osm-bounds --ignore-turn-restrictions --merge-lines --allow-reverse-merge --transparent --draw-priority=28 --add-pois-to-areas --simplify-polygons=23:4,22:7,21:8 --copyright-file="C:\openmtbmap\openmtbmap_svn\copyrightopm.txt" --license-file="C:\openmtbmap\openmtbmap_svn\licenseopm.txt" --add-boundary-nodes-at-admin-boundaries=2 --poi-excl-index=0x6405,0x4316,0x2f00 --ignore-fixme-values --polygon-size-limits="24:16, 23:14, 22:12, 21:11" --ignore-turn-restrictions --description=buildings_ru --country-abbr=ru --country-name=russia --mapname=85800000 --family-id=8580 --product-id=1 --series-name="buildings_russia_31.08.2021_NU_Local" --family-name="buildings_ru_31.08.2021_NU_Local" --tdbfile --gmapi --gmapsupp --overview-mapname=mapsetb --keep-going --area-name="russia_31.08.2021_NU_Local_buildings" -c D:\openmtbmap\maps\template.russiab buildru.typ [0.005s][warning][gc,ergo] NewSize was set larger than initial heap size, will use initial heap size. Mkgmap version 4806M Time started: Tue Aug 31 21:00:52 CEST 2021 SEVERE (Main): D:\openmtbmap\maps\85800012.o5m: input file 'D:\openmtbmap\maps\85800012.o5m' doesn't exist SEVERE (Main): D:\openmtbmap\maps\85800024.o5m: input file 'D:\openmtbmap\maps\85800024.o5m' doesn't exist Number of MapFailedExceptions: 0 Exception in thread "main" java.lang.NullPointerException: Cannot invoke "String.endsWith(String)" because the return value of "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:620) at uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:126) at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:146) at uk.me.parabola.mkgmap.main.Main.main(Main.java:117) -- Felix Hartman - Openmtbmap.org & VeloMap.org -- Felix Hartman - Openmtbmap.org & VeloMap.org
yeah that makes resplits easier. You do not need to go into the template.args file anymore and comment out missing tiles. So it's enough to just delete those input tiles that were too big. On Tue, 14 Sept 2021 at 11:46, Gerd Petermann < gpetermann_muenchen@hotmail.com> wrote:
Hi Felix,
I've added a null check with r4807.
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Felix Hartmann <extremecarver@gmail.com> Gesendet: Donnerstag, 2. September 2021 01:45 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Creating gmapsupp.img fails with: Cannot invoke "String.endsWith(String)" because the return value of "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null
Okay - this error can be solved by removing the "Input that do not exist from the template.args input file. - e.g. if 85610002.o5m does not exist need to remove the line input-file: 85610002.o5m from the template.args file.
On Tue, 31 Aug 2021 at 21:11, Felix Hartmann <extremecarver@gmail.com <mailto:extremecarver@gmail.com>> wrote: Partly related to the other problem - if I run mkgmap on the splitted files - I get this error. All .img files seem to be fine, but the gmpsupp.img is created with 0 bit size.
This is not happening every time I resplit tiles. In this case two tiles were joined by osmconvert - and splitter splits it into three tiles. Then all .img files are created correctly, but the mapset.tdb / mapset.img files are not created - and the gmapsupp.img has 0 size, the gmapi files is also not created.
I guess this happens because splitter somehow made a melange of the two input files into three?
I really think the only thing missing in resplitting the too big tiles from mkgmap is an appropriate keep-complete mode - that keeps each input tile complete - but does not join input tiles. So osmconvert merging the input tiles is a bad idea even though it mostly works.
C:\openmtbmap\maps>start /belownormal /b /wait java -jar -XX:+AggressiveHeap -XX:StringTableSize=1000003 -Xms5000M -Xmx43000M C:\openmtbmap\mkgmap.jar --max-jobs=12 --order-by-decreasing-area --code-page=1251 "--style-file=C:\openmtbmap\buildings_style" --levels="0:24, 1:23, 2:22, 3:21" --ignore-osm-bounds --ignore-turn-restrictions --merge-lines --allow-reverse-merge --transparent --draw-priority=28 --add-pois-to-areas --simplify-polygons=23:4,22:7,21:8 --copyright-file="C:\openmtbmap\openmtbmap_svn\copyrightopm.txt" --license-file="C:\openmtbmap\openmtbmap_svn\licenseopm.txt" --add-boundary-nodes-at-admin-boundaries=2 --poi-excl-index=0x6405,0x4316,0x2f00 --ignore-fixme-values --polygon-size-limits="24:16, 23:14, 22:12, 21:11" --ignore-turn-restrictions --description=buildings_ru --country-abbr=ru --country-name=russia --mapname=85800000 --family-id=8580 --product-id=1 --series-name="buildings_russia_31.08.2021_NU_Local" --family-name="buildings_ru_31.08.2021_NU_Local" --tdbfile --gmapi --gmapsupp --overview-mapname=mapsetb --keep-going --area-name="russia_31.08.2021_NU_Local_buildings" -c D:\openmtbmap\maps\template.russiab buildru.typ [0.005s][warning][gc,ergo] NewSize was set larger than initial heap size, will use initial heap size. Mkgmap version 4806M Time started: Tue Aug 31 21:00:52 CEST 2021 SEVERE (Main): D:\openmtbmap\maps\85800012.o5m: input file 'D:\openmtbmap\maps\85800012.o5m' doesn't exist SEVERE (Main): D:\openmtbmap\maps\85800024.o5m: input file 'D:\openmtbmap\maps\85800024.o5m' doesn't exist Number of MapFailedExceptions: 0 Exception in thread "main" java.lang.NullPointerException: Cannot invoke "String.endsWith(String)" because the return value of "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:620) at uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:126) at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:146) at uk.me.parabola.mkgmap.main.Main.main(Main.java:117)
-- Felix Hartman - Openmtbmap.org & VeloMap.org
-- Felix Hartman - Openmtbmap.org & VeloMap.org
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- Felix Hartman - Openmtbmap.org & VeloMap.org
Hi Felix, I still don't understand the concept of resplitting. Is it meant to fix problems until you find time to look at the server or should it work in long terms? If you resplit more and more tiles I don't see how it will improve the map or the resource usage. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Felix Hartmann <extremecarver@gmail.com> Gesendet: Dienstag, 14. September 2021 11:01 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Creating gmapsupp.img fails with: Cannot invoke "String.endsWith(String)" because the return value of "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null yeah that makes resplits easier. You do not need to go into the template.args file anymore and comment out missing tiles. So it's enough to just delete those input tiles that were too big. On Tue, 14 Sept 2021 at 11:46, Gerd Petermann <gpetermann_muenchen@hotmail.com<mailto:gpetermann_muenchen@hotmail.com>> wrote: Hi Felix, I've added a null check with r4807. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk>> im Auftrag von Felix Hartmann <extremecarver@gmail.com<mailto:extremecarver@gmail.com>> Gesendet: Donnerstag, 2. September 2021 01:45 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Creating gmapsupp.img fails with: Cannot invoke "String.endsWith(String)" because the return value of "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null Okay - this error can be solved by removing the "Input that do not exist from the template.args input file. - e.g. if 85610002.o5m does not exist need to remove the line input-file: 85610002.o5m from the template.args file. On Tue, 31 Aug 2021 at 21:11, Felix Hartmann <extremecarver@gmail.com<mailto:extremecarver@gmail.com><mailto:extremecarver@gmail.com<mailto:extremecarver@gmail.com>>> wrote: Partly related to the other problem - if I run mkgmap on the splitted files - I get this error. All .img files seem to be fine, but the gmpsupp.img is created with 0 bit size. This is not happening every time I resplit tiles. In this case two tiles were joined by osmconvert - and splitter splits it into three tiles. Then all .img files are created correctly, but the mapset.tdb / mapset.img files are not created - and the gmapsupp.img has 0 size, the gmapi files is also not created. I guess this happens because splitter somehow made a melange of the two input files into three? I really think the only thing missing in resplitting the too big tiles from mkgmap is an appropriate keep-complete mode - that keeps each input tile complete - but does not join input tiles. So osmconvert merging the input tiles is a bad idea even though it mostly works. C:\openmtbmap\maps>start /belownormal /b /wait java -jar -XX:+AggressiveHeap -XX:StringTableSize=1000003 -Xms5000M -Xmx43000M C:\openmtbmap\mkgmap.jar --max-jobs=12 --order-by-decreasing-area --code-page=1251 "--style-file=C:\openmtbmap\buildings_style" --levels="0:24, 1:23, 2:22, 3:21" --ignore-osm-bounds --ignore-turn-restrictions --merge-lines --allow-reverse-merge --transparent --draw-priority=28 --add-pois-to-areas --simplify-polygons=23:4,22:7,21:8 --copyright-file="C:\openmtbmap\openmtbmap_svn\copyrightopm.txt" --license-file="C:\openmtbmap\openmtbmap_svn\licenseopm.txt" --add-boundary-nodes-at-admin-boundaries=2 --poi-excl-index=0x6405,0x4316,0x2f00 --ignore-fixme-values --polygon-size-limits="24:16, 23:14, 22:12, 21:11" --ignore-turn-restrictions --description=buildings_ru --country-abbr=ru --country-name=russia --mapname=85800000 --family-id=8580 --product-id=1 --series-name="buildings_russia_31.08.2021_NU_Local" --family-name="buildings_ru_31.08.2021_NU_Local" --tdbfile --gmapi --gmapsupp --overview-mapname=mapsetb --keep-going --area-name="russia_31.08.2021_NU_Local_buildings" -c D:\openmtbmap\maps\template.russiab buildru.typ [0.005s][warning][gc,ergo] NewSize was set larger than initial heap size, will use initial heap size. Mkgmap version 4806M Time started: Tue Aug 31 21:00:52 CEST 2021 SEVERE (Main): D:\openmtbmap\maps\85800012.o5m: input file 'D:\openmtbmap\maps\85800012.o5m' doesn't exist SEVERE (Main): D:\openmtbmap\maps\85800024.o5m: input file 'D:\openmtbmap\maps\85800024.o5m' doesn't exist Number of MapFailedExceptions: 0 Exception in thread "main" java.lang.NullPointerException: Cannot invoke "String.endsWith(String)" because the return value of "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:620) at uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:126) at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:146) at uk.me.parabola.mkgmap.main.Main.main(Main.java:117) -- Felix Hartman - Openmtbmap.org & VeloMap.org -- Felix Hartman - Openmtbmap.org & VeloMap.org _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev -- Felix Hartman - Openmtbmap.org & VeloMap.org
Hi Gerd, The reason for this is - many garmin GPS devices have a max tile limit of 2048, 4096 or some newer devices 15k. Except those with 15k tiles limit - users may break the 2048 or 4096 tile limit on their device if the tiles are not very big. For some countries I can double the max-nodes and only get one tile that is too big. Then I just resplit that tile and compile the output again. However as those tiles are not static - and it doesn't happen all the time - I just prefer not to bother about that - and set a rather high --max-nodes value which will at some extracts lead to failed tiles. With 32GB sd Cards being virtually same price as 8GB or smaller - and users not reading instructions of not installing too many maps to slow down their device - they from time to time complain about missing maps on their device. It's very tedious to find out that the reason for this is the tile limit. because the maps all show up in the map setup list on the Garmin GPS device - just some will not display (I actually do not know which will not display - I think the newly added ones - it could be also the ones with the higher map-id or whatever). Therefore I try to maximise the --max-nodes value as good as possible. Only if my scripts can automatically resplit those failed tiles and run mkgamp.jar easily again without starting from scratch this approach makes sense. Else I have to fallback to much smaller max-nodes values. So actually the optimum is likely somewhere around 3% of tiles with resplit. If done well this will only result in maybe 5% longer compile times on your server - but half the number of tiles created. You could even write a script to analyze the splits - and each time no tile fails you increase the max-nodes value by X and each time more than 3% of tiles fail you decrease it again. Well a bit theoretic with the no more compile time - because if you have less tiles than threads on your render server - it takes more time to compile because mgkmap.jar cannot run so many threads in parallel. In general I noticed that over time as OSM grows there is more and more data in OSM which will not end up in my maps - so over time to slowly have to increase max-nodes else your .img tiles get smaller and smaller. I had been using the same --max-nodes values for years (with lower values if a country failed) and noticed I could double, sometimes tripple them without any failed tiles. On Tue, 14 Sept 2021 at 12:09, Gerd Petermann < gpetermann_muenchen@hotmail.com> wrote:
Hi Felix,
I still don't understand the concept of resplitting. Is it meant to fix problems until you find time to look at the server or should it work in long terms? If you resplit more and more tiles I don't see how it will improve the map or the resource usage.
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Felix Hartmann <extremecarver@gmail.com> Gesendet: Dienstag, 14. September 2021 11:01 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Creating gmapsupp.img fails with: Cannot invoke "String.endsWith(String)" because the return value of "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null
yeah that makes resplits easier. You do not need to go into the template.args file anymore and comment out missing tiles. So it's enough to just delete those input tiles that were too big.
On Tue, 14 Sept 2021 at 11:46, Gerd Petermann < gpetermann_muenchen@hotmail.com<mailto:gpetermann_muenchen@hotmail.com>> wrote: Hi Felix,
I've added a null check with r4807.
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk<mailto: mkgmap-dev-bounces@lists.mkgmap.org.uk>> im Auftrag von Felix Hartmann < extremecarver@gmail.com<mailto:extremecarver@gmail.com>> Gesendet: Donnerstag, 2. September 2021 01:45 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Creating gmapsupp.img fails with: Cannot invoke "String.endsWith(String)" because the return value of "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null
Okay - this error can be solved by removing the "Input that do not exist from the template.args input file. - e.g. if 85610002.o5m does not exist need to remove the line input-file: 85610002.o5m from the template.args file.
On Tue, 31 Aug 2021 at 21:11, Felix Hartmann <extremecarver@gmail.com <mailto:extremecarver@gmail.com><mailto:extremecarver@gmail.com<mailto: extremecarver@gmail.com>>> wrote: Partly related to the other problem - if I run mkgmap on the splitted files - I get this error. All .img files seem to be fine, but the gmpsupp.img is created with 0 bit size.
This is not happening every time I resplit tiles. In this case two tiles were joined by osmconvert - and splitter splits it into three tiles. Then all .img files are created correctly, but the mapset.tdb / mapset.img files are not created - and the gmapsupp.img has 0 size, the gmapi files is also not created.
I guess this happens because splitter somehow made a melange of the two input files into three?
I really think the only thing missing in resplitting the too big tiles from mkgmap is an appropriate keep-complete mode - that keeps each input tile complete - but does not join input tiles. So osmconvert merging the input tiles is a bad idea even though it mostly works.
C:\openmtbmap\maps>start /belownormal /b /wait java -jar -XX:+AggressiveHeap -XX:StringTableSize=1000003 -Xms5000M -Xmx43000M C:\openmtbmap\mkgmap.jar --max-jobs=12 --order-by-decreasing-area --code-page=1251 "--style-file=C:\openmtbmap\buildings_style" --levels="0:24, 1:23, 2:22, 3:21" --ignore-osm-bounds --ignore-turn-restrictions --merge-lines --allow-reverse-merge --transparent --draw-priority=28 --add-pois-to-areas --simplify-polygons=23:4,22:7,21:8 --copyright-file="C:\openmtbmap\openmtbmap_svn\copyrightopm.txt" --license-file="C:\openmtbmap\openmtbmap_svn\licenseopm.txt" --add-boundary-nodes-at-admin-boundaries=2 --poi-excl-index=0x6405,0x4316,0x2f00 --ignore-fixme-values --polygon-size-limits="24:16, 23:14, 22:12, 21:11" --ignore-turn-restrictions --description=buildings_ru --country-abbr=ru --country-name=russia --mapname=85800000 --family-id=8580 --product-id=1 --series-name="buildings_russia_31.08.2021_NU_Local" --family-name="buildings_ru_31.08.2021_NU_Local" --tdbfile --gmapi --gmapsupp --overview-mapname=mapsetb --keep-going --area-name="russia_31.08.2021_NU_Local_buildings" -c D:\openmtbmap\maps\template.russiab buildru.typ [0.005s][warning][gc,ergo] NewSize was set larger than initial heap size, will use initial heap size. Mkgmap version 4806M Time started: Tue Aug 31 21:00:52 CEST 2021 SEVERE (Main): D:\openmtbmap\maps\85800012.o5m: input file 'D:\openmtbmap\maps\85800012.o5m' doesn't exist SEVERE (Main): D:\openmtbmap\maps\85800024.o5m: input file 'D:\openmtbmap\maps\85800024.o5m' doesn't exist Number of MapFailedExceptions: 0 Exception in thread "main" java.lang.NullPointerException: Cannot invoke "String.endsWith(String)" because the return value of "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:620) at uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:126) at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:146) at uk.me.parabola.mkgmap.main.Main.main(Main.java:117)
-- Felix Hartman - Openmtbmap.org & VeloMap.org
-- Felix Hartman - Openmtbmap.org & VeloMap.org
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- Felix Hartman - Openmtbmap.org & VeloMap.org
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- Felix Hartman - Openmtbmap.org & VeloMap.org
Hi Felix, yes, the more exotic the OSM data gets the more likely it is that you can increase --max-nodes. Another reason might be the many optimizations during the last years which achieved smaller file sizes. Your approach probably works, but I doubt that you can solve the original problem that users install too many maps (tiles). You just postpone it a few years. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Felix Hartmann <extremecarver@gmail.com> Gesendet: Dienstag, 14. September 2021 12:52 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Creating gmapsupp.img fails with: Cannot invoke "String.endsWith(String)" because the return value of "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null Hi Gerd, The reason for this is - many garmin GPS devices have a max tile limit of 2048, 4096 or some newer devices 15k. Except those with 15k tiles limit - users may break the 2048 or 4096 tile limit on their device if the tiles are not very big. For some countries I can double the max-nodes and only get one tile that is too big. Then I just resplit that tile and compile the output again. However as those tiles are not static - and it doesn't happen all the time - I just prefer not to bother about that - and set a rather high --max-nodes value which will at some extracts lead to failed tiles. With 32GB sd Cards being virtually same price as 8GB or smaller - and users not reading instructions of not installing too many maps to slow down their device - they from time to time complain about missing maps on their device. It's very tedious to find out that the reason for this is the tile limit. because the maps all show up in the map setup list on the Garmin GPS device - just some will not display (I actually do not know which will not display - I think the newly added ones - it could be also the ones with the higher map-id or whatever). Therefore I try to maximise the --max-nodes value as good as possible. Only if my scripts can automatically resplit those failed tiles and run mkgamp.jar easily again without starting from scratch this approach makes sense. Else I have to fallback to much smaller max-nodes values. So actually the optimum is likely somewhere around 3% of tiles with resplit. If done well this will only result in maybe 5% longer compile times on your server - but half the number of tiles created. You could even write a script to analyze the splits - and each time no tile fails you increase the max-nodes value by X and each time more than 3% of tiles fail you decrease it again. Well a bit theoretic with the no more compile time - because if you have less tiles than threads on your render server - it takes more time to compile because mgkmap.jar cannot run so many threads in parallel. In general I noticed that over time as OSM grows there is more and more data in OSM which will not end up in my maps - so over time to slowly have to increase max-nodes else your .img tiles get smaller and smaller. I had been using the same --max-nodes values for years (with lower values if a country failed) and noticed I could double, sometimes tripple them without any failed tiles. On Tue, 14 Sept 2021 at 12:09, Gerd Petermann <gpetermann_muenchen@hotmail.com<mailto:gpetermann_muenchen@hotmail.com>> wrote: Hi Felix, I still don't understand the concept of resplitting. Is it meant to fix problems until you find time to look at the server or should it work in long terms? If you resplit more and more tiles I don't see how it will improve the map or the resource usage. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk>> im Auftrag von Felix Hartmann <extremecarver@gmail.com<mailto:extremecarver@gmail.com>> Gesendet: Dienstag, 14. September 2021 11:01 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Creating gmapsupp.img fails with: Cannot invoke "String.endsWith(String)" because the return value of "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null yeah that makes resplits easier. You do not need to go into the template.args file anymore and comment out missing tiles. So it's enough to just delete those input tiles that were too big. On Tue, 14 Sept 2021 at 11:46, Gerd Petermann <gpetermann_muenchen@hotmail.com<mailto:gpetermann_muenchen@hotmail.com><mailto:gpetermann_muenchen@hotmail.com<mailto:gpetermann_muenchen@hotmail.com>>> wrote: Hi Felix, I've added a null check with r4807. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk>>> im Auftrag von Felix Hartmann <extremecarver@gmail.com<mailto:extremecarver@gmail.com><mailto:extremecarver@gmail.com<mailto:extremecarver@gmail.com>>> Gesendet: Donnerstag, 2. September 2021 01:45 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Creating gmapsupp.img fails with: Cannot invoke "String.endsWith(String)" because the return value of "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null Okay - this error can be solved by removing the "Input that do not exist from the template.args input file. - e.g. if 85610002.o5m does not exist need to remove the line input-file: 85610002.o5m from the template.args file. On Tue, 31 Aug 2021 at 21:11, Felix Hartmann <extremecarver@gmail.com<mailto:extremecarver@gmail.com><mailto:extremecarver@gmail.com<mailto:extremecarver@gmail.com>><mailto:extremecarver@gmail.com<mailto:extremecarver@gmail.com><mailto:extremecarver@gmail.com<mailto:extremecarver@gmail.com>>>> wrote: Partly related to the other problem - if I run mkgmap on the splitted files - I get this error. All .img files seem to be fine, but the gmpsupp.img is created with 0 bit size. This is not happening every time I resplit tiles. In this case two tiles were joined by osmconvert - and splitter splits it into three tiles. Then all .img files are created correctly, but the mapset.tdb / mapset.img files are not created - and the gmapsupp.img has 0 size, the gmapi files is also not created. I guess this happens because splitter somehow made a melange of the two input files into three? I really think the only thing missing in resplitting the too big tiles from mkgmap is an appropriate keep-complete mode - that keeps each input tile complete - but does not join input tiles. So osmconvert merging the input tiles is a bad idea even though it mostly works. C:\openmtbmap\maps>start /belownormal /b /wait java -jar -XX:+AggressiveHeap -XX:StringTableSize=1000003 -Xms5000M -Xmx43000M C:\openmtbmap\mkgmap.jar --max-jobs=12 --order-by-decreasing-area --code-page=1251 "--style-file=C:\openmtbmap\buildings_style" --levels="0:24, 1:23, 2:22, 3:21" --ignore-osm-bounds --ignore-turn-restrictions --merge-lines --allow-reverse-merge --transparent --draw-priority=28 --add-pois-to-areas --simplify-polygons=23:4,22:7,21:8 --copyright-file="C:\openmtbmap\openmtbmap_svn\copyrightopm.txt" --license-file="C:\openmtbmap\openmtbmap_svn\licenseopm.txt" --add-boundary-nodes-at-admin-boundaries=2 --poi-excl-index=0x6405,0x4316,0x2f00 --ignore-fixme-values --polygon-size-limits="24:16, 23:14, 22:12, 21:11" --ignore-turn-restrictions --description=buildings_ru --country-abbr=ru --country-name=russia --mapname=85800000 --family-id=8580 --product-id=1 --series-name="buildings_russia_31.08.2021_NU_Local" --family-name="buildings_ru_31.08.2021_NU_Local" --tdbfile --gmapi --gmapsupp --overview-mapname=mapsetb --keep-going --area-name="russia_31.08.2021_NU_Local_buildings" -c D:\openmtbmap\maps\template.russiab buildru.typ [0.005s][warning][gc,ergo] NewSize was set larger than initial heap size, will use initial heap size. Mkgmap version 4806M Time started: Tue Aug 31 21:00:52 CEST 2021 SEVERE (Main): D:\openmtbmap\maps\85800012.o5m: input file 'D:\openmtbmap\maps\85800012.o5m' doesn't exist SEVERE (Main): D:\openmtbmap\maps\85800024.o5m: input file 'D:\openmtbmap\maps\85800024.o5m' doesn't exist Number of MapFailedExceptions: 0 Exception in thread "main" java.lang.NullPointerException: Cannot invoke "String.endsWith(String)" because the return value of "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:620) at uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:126) at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:146) at uk.me.parabola.mkgmap.main.Main.main(Main.java:117) -- Felix Hartman - Openmtbmap.org & VeloMap.org -- Felix Hartman - Openmtbmap.org & VeloMap.org _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk><mailto:mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk>> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev -- Felix Hartman - Openmtbmap.org & VeloMap.org _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev -- Felix Hartman - Openmtbmap.org & VeloMap.org
Well as the new Garmin devices have 15k tiles and still fat32 the problem will disappear for them at least. If you manage to get every tile on average to 10MB those with 4096 tiles are fine too. Those with 2048 tiles at least postponed a bit...this and same FID is like the most annoying problem that users face, because both are very hard to identify.... On Tue, 14 Sep 2021, 16:28 Gerd Petermann <gpetermann_muenchen@hotmail.com> wrote:
Hi Felix,
yes, the more exotic the OSM data gets the more likely it is that you can increase --max-nodes. Another reason might be the many optimizations during the last years which achieved smaller file sizes.
Your approach probably works, but I doubt that you can solve the original problem that users install too many maps (tiles). You just postpone it a few years.
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Felix Hartmann <extremecarver@gmail.com> Gesendet: Dienstag, 14. September 2021 12:52 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Creating gmapsupp.img fails with: Cannot invoke "String.endsWith(String)" because the return value of "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null
Hi Gerd,
The reason for this is - many garmin GPS devices have a max tile limit of 2048, 4096 or some newer devices 15k. Except those with 15k tiles limit - users may break the 2048 or 4096 tile limit on their device if the tiles are not very big. For some countries I can double the max-nodes and only get one tile that is too big. Then I just resplit that tile and compile the output again. However as those tiles are not static - and it doesn't happen all the time - I just prefer not to bother about that - and set a rather high --max-nodes value which will at some extracts lead to failed tiles.
With 32GB sd Cards being virtually same price as 8GB or smaller - and users not reading instructions of not installing too many maps to slow down their device - they from time to time complain about missing maps on their device. It's very tedious to find out that the reason for this is the tile limit. because the maps all show up in the map setup list on the Garmin GPS device - just some will not display (I actually do not know which will not display - I think the newly added ones - it could be also the ones with the higher map-id or whatever). Therefore I try to maximise the --max-nodes value as good as possible. Only if my scripts can automatically resplit those failed tiles and run mkgamp.jar easily again without starting from scratch this approach makes sense. Else I have to fallback to much smaller max-nodes values.
So actually the optimum is likely somewhere around 3% of tiles with resplit. If done well this will only result in maybe 5% longer compile times on your server - but half the number of tiles created. You could even write a script to analyze the splits - and each time no tile fails you increase the max-nodes value by X and each time more than 3% of tiles fail you decrease it again.
Well a bit theoretic with the no more compile time - because if you have less tiles than threads on your render server - it takes more time to compile because mgkmap.jar cannot run so many threads in parallel.
In general I noticed that over time as OSM grows there is more and more data in OSM which will not end up in my maps - so over time to slowly have to increase max-nodes else your .img tiles get smaller and smaller. I had been using the same --max-nodes values for years (with lower values if a country failed) and noticed I could double, sometimes tripple them without any failed tiles.
On Tue, 14 Sept 2021 at 12:09, Gerd Petermann < gpetermann_muenchen@hotmail.com<mailto:gpetermann_muenchen@hotmail.com>> wrote: Hi Felix,
I still don't understand the concept of resplitting. Is it meant to fix problems until you find time to look at the server or should it work in long terms? If you resplit more and more tiles I don't see how it will improve the map or the resource usage.
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk<mailto: mkgmap-dev-bounces@lists.mkgmap.org.uk>> im Auftrag von Felix Hartmann < extremecarver@gmail.com<mailto:extremecarver@gmail.com>> Gesendet: Dienstag, 14. September 2021 11:01 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Creating gmapsupp.img fails with: Cannot invoke "String.endsWith(String)" because the return value of "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null
yeah that makes resplits easier. You do not need to go into the template.args file anymore and comment out missing tiles. So it's enough to just delete those input tiles that were too big.
On Tue, 14 Sept 2021 at 11:46, Gerd Petermann < gpetermann_muenchen@hotmail.com<mailto:gpetermann_muenchen@hotmail.com
<mailto:gpetermann_muenchen@hotmail.com<mailto: gpetermann_muenchen@hotmail.com>>> wrote: Hi Felix,
I've added a null check with r4807.
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk<mailto: mkgmap-dev-bounces@lists.mkgmap.org.uk><mailto: mkgmap-dev-bounces@lists.mkgmap.org.uk<mailto: mkgmap-dev-bounces@lists.mkgmap.org.uk>>> im Auftrag von Felix Hartmann < extremecarver@gmail.com<mailto:extremecarver@gmail.com><mailto: extremecarver@gmail.com<mailto:extremecarver@gmail.com>>> Gesendet: Donnerstag, 2. September 2021 01:45 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Creating gmapsupp.img fails with: Cannot invoke "String.endsWith(String)" because the return value of "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null
Okay - this error can be solved by removing the "Input that do not exist from the template.args input file. - e.g. if 85610002.o5m does not exist need to remove the line input-file: 85610002.o5m from the template.args file.
On Tue, 31 Aug 2021 at 21:11, Felix Hartmann <extremecarver@gmail.com <mailto:extremecarver@gmail.com><mailto:extremecarver@gmail.com<mailto: extremecarver@gmail.com>><mailto:extremecarver@gmail.com<mailto: extremecarver@gmail.com><mailto:extremecarver@gmail.com<mailto: extremecarver@gmail.com>>>> wrote: Partly related to the other problem - if I run mkgmap on the splitted files - I get this error. All .img files seem to be fine, but the gmpsupp.img is created with 0 bit size.
This is not happening every time I resplit tiles. In this case two tiles were joined by osmconvert - and splitter splits it into three tiles. Then all .img files are created correctly, but the mapset.tdb / mapset.img files are not created - and the gmapsupp.img has 0 size, the gmapi files is also not created.
I guess this happens because splitter somehow made a melange of the two input files into three?
I really think the only thing missing in resplitting the too big tiles from mkgmap is an appropriate keep-complete mode - that keeps each input tile complete - but does not join input tiles. So osmconvert merging the input tiles is a bad idea even though it mostly works.
C:\openmtbmap\maps>start /belownormal /b /wait java -jar -XX:+AggressiveHeap -XX:StringTableSize=1000003 -Xms5000M -Xmx43000M C:\openmtbmap\mkgmap.jar --max-jobs=12 --order-by-decreasing-area --code-page=1251 "--style-file=C:\openmtbmap\buildings_style" --levels="0:24, 1:23, 2:22, 3:21" --ignore-osm-bounds --ignore-turn-restrictions --merge-lines --allow-reverse-merge --transparent --draw-priority=28 --add-pois-to-areas --simplify-polygons=23:4,22:7,21:8 --copyright-file="C:\openmtbmap\openmtbmap_svn\copyrightopm.txt" --license-file="C:\openmtbmap\openmtbmap_svn\licenseopm.txt" --add-boundary-nodes-at-admin-boundaries=2 --poi-excl-index=0x6405,0x4316,0x2f00 --ignore-fixme-values --polygon-size-limits="24:16, 23:14, 22:12, 21:11" --ignore-turn-restrictions --description=buildings_ru --country-abbr=ru --country-name=russia --mapname=85800000 --family-id=8580 --product-id=1 --series-name="buildings_russia_31.08.2021_NU_Local" --family-name="buildings_ru_31.08.2021_NU_Local" --tdbfile --gmapi --gmapsupp --overview-mapname=mapsetb --keep-going --area-name="russia_31.08.2021_NU_Local_buildings" -c D:\openmtbmap\maps\template.russiab buildru.typ [0.005s][warning][gc,ergo] NewSize was set larger than initial heap size, will use initial heap size. Mkgmap version 4806M Time started: Tue Aug 31 21:00:52 CEST 2021 SEVERE (Main): D:\openmtbmap\maps\85800012.o5m: input file 'D:\openmtbmap\maps\85800012.o5m' doesn't exist SEVERE (Main): D:\openmtbmap\maps\85800024.o5m: input file 'D:\openmtbmap\maps\85800024.o5m' doesn't exist Number of MapFailedExceptions: 0 Exception in thread "main" java.lang.NullPointerException: Cannot invoke "String.endsWith(String)" because the return value of "uk.me.parabola.mkgmap.main.Main$FilenameTask.getFilename()" is null at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:620) at uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:126) at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:146) at uk.me.parabola.mkgmap.main.Main.main(Main.java:117)
-- Felix Hartman - Openmtbmap.org & VeloMap.org
-- Felix Hartman - Openmtbmap.org & VeloMap.org
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk
<mailto:mkgmap-dev@lists.mkgmap.org.uk<mailto: mkgmap-dev@lists.mkgmap.org.uk>> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- Felix Hartman - Openmtbmap.org & VeloMap.org
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- Felix Hartman - Openmtbmap.org & VeloMap.org
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
participants (2)
-
Felix Hartmann -
Gerd Petermann