Re: [mkgmap-dev] mkgmap gets stuck on Kosovo
On 05.03.2010 22:03, Felix Hartmann wrote:
On 05.03.2010 22:00, Mark Burton wrote:
Felix,
The very first time it got stuck about 6 hours. Normal style finishes in 30 seconds. Actually I get stuck too now on Europe....
There must be some serious bug or new kind of data people are writing in OSM. In your style, do you generate multiple routable ways for each road?
Mark Not for each, but for many (basically if any extended attribute like a route, mtb:scale, incline, oneway, .... exist).
You can download my style either as 7zip (lzma) from here: http://svn.origo.ethz.ch/dump/openmtbmap/latest.tar.gz Look at it here: http://svn.origo.ethz.ch/wsvn/openmtbmap/ or checkout from SVN: https://svn.origo.ethz.ch/openmtbmap/
Any news on solving this bug. I get stuck now on more and more countries. For my style, this is getting really serious trouble (basically unable to compile 1/10 countries, meaning I can't update my maps anymore as it gets stuck)
Hello Felix,
Any news on solving this bug. I get stuck now on more and more countries. For my style, this is getting really serious trouble (basically unable to compile 1/10 countries, meaning I can't update my maps anymore as it gets stuck)
I have looked closely at the code where your stack trace indicates it's getting stuck and I cannot see what could be causing the problem. As I said before, that's not new code so I find it hard to believe that is what's causing the hang up. I think at this point, you need to find which version of mkgmap introduces the problem (or even, which version of your style introduces the problem). So, bisect mkgmap, starting at a known good revision and find exactly the revision that goes wrong. Tedious, I know but it will tell us what we want to know. Mark
Felix, I have grabbed your style stuff from the SVN site and using that style it takes less than 2 min to process kosovo.osm.gz. Here's the options I used: java -Xmx2000m -Dlog.config=/home/markb/OSM/logging.properties -ea -jar /home/markb/OSM/mkgmap.jar --adjust-turn-headings --country-abbr=GBR --country-name=GBR --check-roundabouts --check-roundabout-flares --description=A fine map --draw-priority=28 --family-id=909 --family-name=Burto Maps --gmapsupp --generate-sea=polygons,no-sea-sectors,close-gaps=2000 --ignore-maxspeeds --index --link-pois-to-ways --max-jobs=1 --nsis --product-id=1 --region-abbr=UK --remove-bogus-nodes --report-dead-ends=2 --region-name=UK --route --remove-short-arcs --series-name=OSM map --style-file=/home/markb/OSM/openmtbmap --tdbfile kosovo.osm.gz I checked the output with mapedit and mapsource and it's definitely using your style. Mark
On 12.03.2010 20:39, Mark Burton wrote:
Felix,
I have grabbed your style stuff from the SVN site and using that style it takes less than 2 min to process kosovo.osm.gz. Here's the options I used:
java -Xmx2000m -Dlog.config=/home/markb/OSM/logging.properties -ea -jar /home/markb/OSM/mkgmap.jar --adjust-turn-headings --country-abbr=GBR --country-name=GBR --check-roundabouts --check-roundabout-flares --description=A fine map --draw-priority=28 --family-id=909 --family-name=Burto Maps --gmapsupp --generate-sea=polygons,no-sea-sectors,close-gaps=2000 --ignore-maxspeeds --index --link-pois-to-ways --max-jobs=1 --nsis --product-id=1 --region-abbr=UK --remove-bogus-nodes --report-dead-ends=2 --region-name=UK --route --remove-short-arcs --series-name=OSM map --style-file=/home/markb/OSM/openmtbmap --tdbfile kosovo.osm.gz
I checked the output with mapedit and mapsource and it's definitely using your style.
Ups, sorry. I don't understand why it ran through using the default style, but it is hanging on the templates.args file which seems to be the real culprit. If I run mkgmap with *.osm.gz for map input, it runs through fine. If however I use -c template.args then it gets stuck infinitely. I have attached the problem causing template.args to this mail. Maybe something is wrong with it??? (it is supposed to compile tile 63220000.osm.gz). It only happens on a very few countries. (I run splitter.jar with following commandline: java -Xmx4050m -Xmx4500M splitter.jar --mapid=63220000 --max-nodes=1300000 kosovo.osm) See tem
Mark _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Templates.arg attached. this time On 12.03.2010 20:39, Mark Burton wrote:
Felix,
I have grabbed your style stuff from the SVN site and using that style it takes less than 2 min to process kosovo.osm.gz. Here's the options I used:
java -Xmx2000m -Dlog.config=/home/markb/OSM/logging.properties -ea -jar /home/markb/OSM/mkgmap.jar --adjust-turn-headings --country-abbr=GBR --country-name=GBR --check-roundabouts --check-roundabout-flares --description=A fine map --draw-priority=28 --family-id=909 --family-name=Burto Maps --gmapsupp --generate-sea=polygons,no-sea-sectors,close-gaps=2000 --ignore-maxspeeds --index --link-pois-to-ways --max-jobs=1 --nsis --product-id=1 --region-abbr=UK --remove-bogus-nodes --report-dead-ends=2 --region-name=UK --route --remove-short-arcs --series-name=OSM map --style-file=/home/markb/OSM/openmtbmap --tdbfile kosovo.osm.gz
I checked the output with mapedit and mapsource and it's definitely using your style.
Ups, sorry. I don't understand why it ran through using the default style, but it is hanging on the templates.args file which seems to be the real culprit. If I run mkgmap with *.osm.gz for map input, it runs through fine. If however I use -c template.args then it gets stuck infinitely. I have attached the problem causing template.args to this mail. Maybe something is wrong with it??? (it is supposed to compile tile 63220000.osm.gz). It only happens on a very few countries. (I run splitter.jar with following commandline: java -Xmx4050m -Xmx4500M splitter.jar --mapid=63220000 --max-nodes=1300000 kosovo.osm)
Mark _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Felix,
Ups, sorry. I don't understand why it ran through using the default style, but it is hanging on the templates.args file which seems to be the real culprit. If I run mkgmap with *.osm.gz for map input, it runs through fine. If however I use -c template.args then it gets stuck infinitely.
I have attached the problem causing template.args to this mail. Maybe something is wrong with it??? (it is supposed to compile tile 63220000.osm.gz). It only happens on a very few countries. (I run splitter.jar with following commandline: java -Xmx4050m -Xmx4500M splitter.jar --mapid=63220000 --max-nodes=1300000 kosovo.osm)
I tried your template.args file, it didn't get stuck, finished in less than 2 mins. Mark
On 14.03.2010 23:18, Mark Burton wrote:
Felix,
Ups, sorry. I don't understand why it ran through using the default style, but it is hanging on the templates.args file which seems to be the real culprit. If I run mkgmap with *.osm.gz for map input, it runs through fine. If however I use -c template.args then it gets stuck infinitely.
I have attached the problem causing template.args to this mail. Maybe something is wrong with it??? (it is supposed to compile tile 63220000.osm.gz). It only happens on a very few countries. (I run splitter.jar with following commandline: java -Xmx4050m -Xmx4500M splitter.jar --mapid=63220000 --max-nodes=1300000 kosovo.osm)
I tried your template.args file, it didn't get stuck, finished in less than 2 mins.
Sorry, I had a syntax error here that caused mkgmap to pass when not using template.args (and outputting a 0kb map). It's using location-autofill=2 and my style-file which will cause mkgmap to get stuck on kosovo (as well as on some more countries like recently Slovakia).
Mark _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Chill Felix,
Sorry, I had a syntax error here that caused mkgmap to pass when not using template.args (and outputting a 0kb map). It's using location-autofill=2 and my style-file which will cause mkgmap to get stuck on kosovo (as well as on some more countries like recently Slovakia).
I am looking at that now.
Felix, The problem is triggered by the fact that in your style file you give anonymous roads of the same type, the same name i.e. 'rd', 'trk', 'ucl', etc. So the map ends up containing lot's of roads with the same names. The kosovo map contains a huge number of anonymous roads. The code that is getting stuck is trying to find all of the roads that have the same name and are associated with the same city and are connected to each other. Perhaps, and this is a guess, the autofill=2 option increases the number of roads that have the same city such that it becomes computationally, very expensive to work out which roads are connected to each other. It may well finish if you wait long enough! Mark
On 14.03.2010 23:49, Mark Burton wrote:
Felix,
The problem is triggered by the fact that in your style file you give anonymous roads of the same type, the same name i.e. 'rd', 'trk', 'ucl', etc. So the map ends up containing lot's of roads with the same names. The kosovo map contains a huge number of anonymous roads.
The code that is getting stuck is trying to find all of the roads that have the same name and are associated with the same city and are connected to each other.
Perhaps, and this is a guess, the autofill=2 option increases the number of roads that have the same city such that it becomes computationally, very expensive to work out which roads are connected to each other. It may well finish if you wait long enough!
Well is 12 hours too short????? I think I had even left mkgmap working for well over 24 hours on Kosovo once (using autofill=1 it runs through in about 20 seconds). It seems strange to me that most countries still do work, and Slovakia from geofabrik worked 1 week ago, but not currently. On the other hand a country that blocked last weak, compiled with current data. I do suppose there is some bigger bug, but can live with autofill=1....
Mark _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
I do suppose there is some bigger bug, but can live with autofill=1....
Hey look, why should I spend my time trying to solve problems for people who can't even be bothered to say thanks for the effort? Felix, you are a heartless tyrant. Go to the end of the queue.
On 15.03.2010 00:30, Mark Burton wrote:
I do suppose there is some bigger bug, but can live with autofill=1....
Hey look, why should I spend my time trying to solve problems for people who can't even be bothered to say thanks for the effort?
Felix, you are a heartless tyrant. Go to the end of the queue. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Sorry, for not saying thanks, but I do hope that you know that I highly appreciate all of your (and other mkgmap Coders) work and hope that my error reporting helps push mkgmap overall forward. I am simply far too much used to simply only write about relevant stuff forgetting saying thanks. I exchanged set name with set asdf and the map compiled clean withing 28 seconds. So yes it's my setting of default names for unnamed streets that is causing the error. I had my PC running 36 hours on Kosovo and it did not finish, so I think something did overflow but the error may be elsewhere. It's not very important to me anyhow (now that I know that I can avoid it using autofill=1 - as before it was really really annoying when running map updates to discover that it had stopped after few time, and I had to block my desktop for some more hours/days to push the updates through), I simply wanted to tell what's happening.
Sorry, I was wrong again. Its the --location-autofill=2 option in combination to my style-file. If I use --location-autofill=1 there is no problem. So clearly there is a bug in --location-autofill=2, previously I always used --location-autofill=1 therefore I had not problems with mkgmap getting stuck.
participants (2)
-
Felix Hartmann -
Mark Burton