SeeGenerator error with r1752

Using r1752 I get the error below if I call generate-sea=multipolygon,extend-sea-sectors java.lang.NullPointerException at uk.me.parabola.mkgmap.reader.osm.SeaGenerator.end(SeaGenerator.java:432) at uk.me.parabola.mkgmap.reader.osm.OsmReadingHooksChain.end(OsmReadingHooksChain.java:78) at uk.me.parabola.mkgmap.reader.osm.bin.OsmBinMapDataSource.load(OsmBinMapDataSource.java:68) at uk.me.parabola.mkgmap.main.MapMaker.loadFromFile(MapMaker.java:145) at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:56) at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:220) at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:217) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Using generate-sea=polygons,extend-sea-sectors works fine as usual

I'm geting trouble with following options as well: --generate-sea=multipolygon,extend-sea-sectors,close-gaps=2000 r1751 did not have the problems and they appeared in r1752. --- On 12/23/2010 06:36 PM, Carlos Dávila wrote:
Using r1752 I get the error below if I call generate-sea=multipolygon,extend-sea-sectors java.lang.NullPointerException at uk.me.parabola.mkgmap.reader.osm.SeaGenerator.end(SeaGenerator.java:432) at uk.me.parabola.mkgmap.reader.osm.OsmReadingHooksChain.end(OsmReadingHooksChain.java:78) at uk.me.parabola.mkgmap.reader.osm.bin.OsmBinMapDataSource.load(OsmBinMapDataSource.java:68) at uk.me.parabola.mkgmap.main.MapMaker.loadFromFile(MapMaker.java:145) at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:56) at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:220) at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:217) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662)
Using generate-sea=polygons,extend-sea-sectors works fine as usual _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

On Thu, Dec 23, 2010 at 07:14:54PM +0200, Harri wrote:
I'm geting trouble with following options as well: --generate-sea=multipolygon,extend-sea-sectors,close-gaps=2000
r1751 did not have the problems and they appeared in r1752.
I did not upgrade yet, still at r1728. My scripts at http://www.polkupyoraily.net/osm/ use generate-sea=multipolygon for all tiles except one, which is generate-sea=multipolygon,extend-sea-sectors. I guess I should retry today's extract with r1752 and report if it works. Marko

On Thu, Dec 23, 2010 at 08:14:27PM +0200, Marko Mäkelä wrote:
On Thu, Dec 23, 2010 at 07:14:54PM +0200, Harri wrote:
I'm geting trouble with following options as well: --generate-sea=multipolygon,extend-sea-sectors,close-gaps=2000
r1751 did not have the problems and they appeared in r1752.
I did not upgrade yet, still at r1728. My scripts at http://www.polkupyoraily.net/osm/ use generate-sea=multipolygon for all tiles except one, which is generate-sea=multipolygon,extend-sea-sectors. I guess I should retry today's extract with r1752 and report if it works.
It does not work. This command from my osm2img.sh script (simplified) crashes: java -ea -Dlog.config=logging.properties -jar mkgmap.jar -c mkgmap.args The end of the mkgmap.log.0 looks comparable to the mkgmap.log.1 from a previous run with r1728. After stripping time stamps and sorting the lines, the files are almost equal. The differences do not seem related to generate-sea (except the different IDs of generated polygons). The last map tile (63240006) uses --generate-sea=multipolygon,extend-sea-sectors. All other tiles use just --generate-sea=multipolygon. Here is the stack trace: java.lang.NullPointerException at uk.me.parabola.mkgmap.reader.osm.SeaGenerator.end(SeaGenerator.java:432) at uk.me.parabola.mkgmap.reader.osm.OsmReadingHooksChain.end(OsmReadingHooksChain.java:78) at uk.me.parabola.mkgmap.reader.osm.xml.Osm5MapDataSource.load(Osm5MapDataSource.java:73) at uk.me.parabola.mkgmap.main.MapMaker.loadFromFile(MapMaker.java:145) at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:56) at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:220) at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:217) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) This seems to point to this line: coastRel.setFloodBlockerRules(fbRules.getWayRules()); Should it not skip this when fbRules == null, or should empty flood-blocking rules be assigned an object? Best regards, Marko

Thanks for the report! It is fixed with r1753. WanMil
Using r1752 I get the error below if I call generate-sea=multipolygon,extend-sea-sectors java.lang.NullPointerException at uk.me.parabola.mkgmap.reader.osm.SeaGenerator.end(SeaGenerator.java:432) at uk.me.parabola.mkgmap.reader.osm.OsmReadingHooksChain.end(OsmReadingHooksChain.java:78) at uk.me.parabola.mkgmap.reader.osm.bin.OsmBinMapDataSource.load(OsmBinMapDataSource.java:68) at uk.me.parabola.mkgmap.main.MapMaker.loadFromFile(MapMaker.java:145) at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:56) at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:220) at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:217) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662)
Using generate-sea=polygons,extend-sea-sectors works fine as usual _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

On Thu, Dec 23, 2010 at 09:24:08PM +0100, WanMil wrote:
Thanks for the report! It is fixed with r1753.
Yes, I can verify that r1753 does not crash any more. Something strange is still happening: -rw-r--r-- 1 marko marko 65486848 23.12. 22:26 gmapsupp.img -rw-r--r-- 1 marko marko 69042176 23.12. 02:54 gmapsupp.img.1728 Why did r1728 generate a much larger file than r1753, on the same data? Marko

On Thu, Dec 23, 2010 at 09:24:08PM +0100, WanMil wrote:
Thanks for the report! It is fixed with r1753.
Yes, I can verify that r1753 does not crash any more.
Something strange is still happening:
-rw-r--r-- 1 marko marko 65486848 23.12. 22:26 gmapsupp.img -rw-r--r-- 1 marko marko 69042176 23.12. 02:54 gmapsupp.img.1728
Why did r1728 generate a much larger file than r1753, on the same data?
Marko
That doesn't sound good.... I have looked through the changelog: 1. A synchronization in the MapMaker class was removed. Do you use more than 1 thread? 2. The procedure how the list of relevant tags that are read from the OSM file is created was changed a bit. Maybe this introduced a bug so that some things are missing now. Can you do a quick check, if you are missing some POIs or highways or polygons in the r1753 map? 3. The procedure how the mulitpolygon processing removes tags from the original ways was changed. Either this introduced or fixed a bug. ?!? Yeah, it is possible that the old algorithm missed some tag removals on lines and polygons so that the number of objects was too large in the maps. It would be great if we have a tool that reads the IMG-file and creates a statistic of how many objects are contained in the file. Do we have that already?! I think I remember a thread about that but I cannot find it. WanMil

I once supplied statistics of number of objects in an IMG-File. I used GPSMapEdit. www.geopainting.com Under map properties choose Statistics. Markus_g -----Original Message----- From: mkgmap-dev-bounces@lists.mkgmap.org.uk [mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk] On Behalf Of WanMil Sent: Friday, 24 December 2010 8:35 AM To: Development list for mkgmap Subject: Re: [mkgmap-dev] SeeGenerator error with r1752
On Thu, Dec 23, 2010 at 09:24:08PM +0100, WanMil wrote:
Thanks for the report! It is fixed with r1753.
Yes, I can verify that r1753 does not crash any more.
Something strange is still happening:
-rw-r--r-- 1 marko marko 65486848 23.12. 22:26 gmapsupp.img -rw-r--r-- 1 marko marko 69042176 23.12. 02:54 gmapsupp.img.1728
Why did r1728 generate a much larger file than r1753, on the same data?
Marko
That doesn't sound good.... I have looked through the changelog: 1. A synchronization in the MapMaker class was removed. Do you use more than 1 thread? 2. The procedure how the list of relevant tags that are read from the OSM file is created was changed a bit. Maybe this introduced a bug so that some things are missing now. Can you do a quick check, if you are missing some POIs or highways or polygons in the r1753 map? 3. The procedure how the mulitpolygon processing removes tags from the original ways was changed. Either this introduced or fixed a bug. ?!? Yeah, it is possible that the old algorithm missed some tag removals on lines and polygons so that the number of objects was too large in the maps. It would be great if we have a tool that reads the IMG-file and creates a statistic of how many objects are contained in the file. Do we have that already?! I think I remember a thread about that but I cannot find it. WanMil _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi WanMil,
1. A synchronization in the MapMaker class was removed. Do you use more than 1 thread?
Yes, I use --max-jobs, with 2 threads (Intel Core Duo L2400). The log from r1728 seemed to contain more warnings. Could the new mkgmap be silently dropping some features?
2. The procedure how the list of relevant tags that are read from the OSM file is created was changed a bit. Maybe this introduced a bug so that some things are missing now. Can you do a quick check, if you are missing some POIs or highways or polygons in the r1753 map?
Here are the most recent log files, all on 6 osm.gz tiles split from yesterday's finland.osm.pbf: -rw-r--r-- 1 marko marko 3855193 23.12. 22:24 mkgmap.log.0 r1753 -rw-r--r-- 1 marko marko 19309 23.12. 22:06 mkgmap.log.1 r1752, NPE -rw-r--r-- 1 marko marko 322567 23.12. 10:55 mkgmap.log.2 r1728 In r1753, the growth could be due to an error in generate-sea. Here is an example: 2010/12/23 22:22:11 WARNING (MultiPolygonRelation): 63240001.osm.gz: Multipolygon http://www.openstreetmap.org/browse/relation/4611686018427403589 contains errors. 2010/12/23 22:22:11 WARNING (MultiPolygonRelation): 63240001.osm.gz: Some polygons are intersecting. This is not allowed in multipolygons. 2010/12/23 22:22:11 WARNING (MultiPolygonRelation): 63240001.osm.gz: - http://www.openstreetmap.org/browse/way/4611686018427403617 ... The NullPointerException in r1752 seemed to occur very late in the processing, after writing everything to the log. The shrinkage from r1728 to r1752 log output is due to stuff like this that r1752 no longer complains about: 2010/12/23 10:51:12 WARNING (RoadDef): 63240001.osm.gz: (^E4/7 Meritullintori, http://www.openstreetmap.org/browse/way/30287920) discarding extra label (already have 4) 2010/12/23 10:51:12 WARNING (RoadDef): 63240001.osm.gz: (^E4/7 Meritullintori, http://www.openstreetmap.org/browse/way/35148621) discarding extra label (already have 4) The way names look like highway=trunk or highway=primary, and presumably they all belong to a route relation, such as http://www.openstreetmap.org/browse/relation/67630 (European Route 75).
3. The procedure how the mulitpolygon processing removes tags from the original ways was changed. Either this introduced or fixed a bug. ?!?
The above ways do not belong to a multipolygon, only to a route relation and a turn restriction relation.
Yeah, it is possible that the old algorithm missed some tag removals on lines and polygons so that the number of objects was too large in the maps.
Could you have improved this 'discarding extra label' too?
It would be great if we have a tool that reads the IMG-file and creates a statistic of how many objects are contained in the file. Do we have that already?! I think I remember a thread about that but I cannot find it.
IIRC, there is something in the mkgmap svn repository, presumably written by Steve in the very early stages of mkgmap. I have never used it. There could also be something on SourceForge(t), in a Garmin reverse-engineering project. Marko

It would be great if we have a tool that reads the IMG-file and creates a statistic of how many objects are contained in the file. Do we have that already?! I think I remember a thread about that but I cannot find it.
IIRC, there is something in the mkgmap svn repository, presumably written by Steve in the very early stages of mkgmap. I have never used it. There could also be something on SourceForge(t), in a Garmin reverse-engineering project.
In the svn repository are the 'display' tools. This tools have been written for displaying the contents of a img in more or less clear text. I don't remember if there is a tool which counts the objects, but may be well the case. http://svn.parabola.me.uk/display/trunk/
participants (6)
-
Carlos Dávila
-
Harri
-
Johann Gail
-
Marko Mäkelä
-
Markus_g
-
WanMil