[PATCH V1] Idea for minor speed improvement for boundary preprocessing
Hi WanMil, this was a very good hint :-) Time for african boundaries was 250 secs with /branch/performance/r2225, 267 secs with trunk, and with this patch it is now 152 secs - use simple quadtree in BoundarySaver.splitArea() - add try/catch in BoundaryPreparer.run() to allow e.g. usage of assert. Without that, an assertion was handled by the threading routines and caused program stop without any message. Please double check this, I am still not so experienced with try+catch and wrong placed Ciao, Gerd http://gis.19327.n5.nabble.com/file/n5512906/splitArea_v1.patch splitArea_v1.patch WanMil wrote
Hi Gerd,
boundary preprocessing takes quite a lot of time. That's not a big problem because it's performed only by some people who publish their results.
But I think some speed improvements would not be bad anyhow?!
While preprocessing asia and america I go an idea. The area of a boundary is splitted into the raster. But each time the complete area is intersected to the raster. In case an area is big and/or complex (boundary of russia, USA, canada etc.) this takes a long time. It would be possible first to cut the area into smaller pieces (columns or rows of the raster) and then do the final cut to the raster. Maybe this saves time because the column or row areas should be less complex.
What do you think?
WanMil _______________________________________________ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- View this message in context: http://gis.19327.n5.nabble.com/PATCH-V1-Idea-for-minor-speed-improvement-for... Sent from the Mkgmap Development mailing list archive at Nabble.com.
Hi Gerd, that's great!
Hi WanMil,
this was a very good hint :-)
Time for african boundaries was 250 secs with /branch/performance/r2225, 267 secs with trunk, and with this patch it is now 152 secs
wow, that's more improvement than I expected.
- use simple quadtree in BoundarySaver.splitArea()
You are a quadtree guy. In this case I thinks it's more irritating and complex to split the area into four subareas. I've changed that to split into two subareas and the code is now much easier to read (I think so..?!?) and performs in similar speed. WanMil
- add try/catch in BoundaryPreparer.run() to allow e.g. usage of assert. Without that, an assertion was handled by the threading routines and caused program stop without any message.
Please double check this, I am still not so experienced with try+catch and wrong placed
Ciao, Gerd
http://gis.19327.n5.nabble.com/file/n5512906/splitArea_v1.patch splitArea_v1.patch
WanMil wrote
Hi Gerd,
boundary preprocessing takes quite a lot of time. That's not a big problem because it's performed only by some people who publish their results.
But I think some speed improvements would not be bad anyhow?!
While preprocessing asia and america I go an idea. The area of a boundary is splitted into the raster. But each time the complete area is intersected to the raster. In case an area is big and/or complex (boundary of russia, USA, canada etc.) this takes a long time. It would be possible first to cut the area into smaller pieces (columns or rows of the raster) and then do the final cut to the raster. Maybe this saves time because the column or row areas should be less complex.
What do you think?
WanMil _______________________________________________ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- View this message in context: http://gis.19327.n5.nabble.com/PATCH-V1-Idea-for-minor-speed-improvement-for... Sent from the Mkgmap Development mailing list archive at Nabble.com. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi WanMil, yes, your solution is better. I wanted to do it in the same way, but failed to find the correct tile sizes for areas that cover e.g. 7 split rectangles (the middle is not on the grid) two small points: 1) the variable names midLon and midLat are a bit missleading, since you don't split in the middle, but near the middle (which perfectly solves my problem described above) 2) You removed my added try/catch from BoundaryPreparer.run() I hope you found a better solution? I needed it there to make assert work in e.g. splitArea() Gerd WanMil wrote
Hi Gerd,
that's great!
Hi WanMil,
this was a very good hint :-)
Time for african boundaries was 250 secs with /branch/performance/r2225, 267 secs with trunk, and with this patch it is now 152 secs
wow, that's more improvement than I expected.
- use simple quadtree in BoundarySaver.splitArea()
You are a quadtree guy. In this case I thinks it's more irritating and complex to split the area into four subareas. I've changed that to split into two subareas and the code is now much easier to read (I think so..?!?) and performs in similar speed.
WanMil
- add try/catch in BoundaryPreparer.run() to allow e.g. usage of assert. Without that, an assertion was handled by the threading routines and caused program stop without any message.
Please double check this, I am still not so experienced with try+catch and wrong placed
Ciao, Gerd
http://gis.19327.n5.nabble.com/file/n5512906/splitArea_v1.patch splitArea_v1.patch
WanMil wrote
Hi Gerd,
boundary preprocessing takes quite a lot of time. That's not a big problem because it's performed only by some people who publish their results.
But I think some speed improvements would not be bad anyhow?!
While preprocessing asia and america I go an idea. The area of a boundary is splitted into the raster. But each time the complete area is intersected to the raster. In case an area is big and/or complex (boundary of russia, USA, canada etc.) this takes a long time. It would be possible first to cut the area into smaller pieces (columns or rows of the raster) and then do the final cut to the raster. Maybe this saves time because the column or row areas should be less complex.
What do you think?
WanMil _______________________________________________ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- View this message in context: http://gis.19327.n5.nabble.com/PATCH-V1-Idea-for-minor-speed-improvement-for... Sent from the Mkgmap Development mailing list archive at Nabble.com. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- View this message in context: http://gis.19327.n5.nabble.com/PATCH-V1-Idea-for-minor-speed-improvement-for... Sent from the Mkgmap Development mailing list archive at Nabble.com.
Hi WanMil,
yes, your solution is better. I wanted to do it in the same way, but failed to find the correct tile sizes for areas that cover e.g. 7 split rectangles (the middle is not on the grid)
two small points: 1) the variable names midLon and midLat are a bit missleading, since you don't split in the middle, but near the middle (which perfectly solves my problem described above)
2) You removed my added try/catch from BoundaryPreparer.run() I hope you found a better solution? I needed it there to make assert work in e.g. splitArea()
An assertion is a check for something that should never be true. Only in case there is a bug this might become true. By default assertions are not checked only if you add the java paramerter -ea. If you want to perform a runtime check you should throw an Exception. WanMil
Gerd
WanMil wrote
Hi Gerd,
that's great!
Hi WanMil,
this was a very good hint :-)
Time for african boundaries was 250 secs with /branch/performance/r2225, 267 secs with trunk, and with this patch it is now 152 secs
wow, that's more improvement than I expected.
- use simple quadtree in BoundarySaver.splitArea()
You are a quadtree guy. In this case I thinks it's more irritating and complex to split the area into four subareas. I've changed that to split into two subareas and the code is now much easier to read (I think so..?!?) and performs in similar speed.
WanMil
- add try/catch in BoundaryPreparer.run() to allow e.g. usage of assert. Without that, an assertion was handled by the threading routines and caused program stop without any message.
Please double check this, I am still not so experienced with try+catch and wrong placed
Ciao, Gerd
http://gis.19327.n5.nabble.com/file/n5512906/splitArea_v1.patch splitArea_v1.patch
WanMil wrote
Hi Gerd,
boundary preprocessing takes quite a lot of time. That's not a big problem because it's performed only by some people who publish their results.
But I think some speed improvements would not be bad anyhow?!
While preprocessing asia and america I go an idea. The area of a boundary is splitted into the raster. But each time the complete area is intersected to the raster. In case an area is big and/or complex (boundary of russia, USA, canada etc.) this takes a long time. It would be possible first to cut the area into smaller pieces (columns or rows of the raster) and then do the final cut to the raster. Maybe this saves time because the column or row areas should be less complex.
What do you think?
WanMil _______________________________________________ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- View this message in context: http://gis.19327.n5.nabble.com/PATCH-V1-Idea-for-minor-speed-improvement-for... Sent from the Mkgmap Development mailing list archive at Nabble.com. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- View this message in context: http://gis.19327.n5.nabble.com/PATCH-V1-Idea-for-minor-speed-improvement-for... Sent from the Mkgmap Development mailing list archive at Nabble.com. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi WanMil, but if I code an assert statement and execute the program with -ea, I surely want to see the assertion that happened. Instead, the program stopped without any message. Gerd WanMil wrote
Hi WanMil,
yes, your solution is better. I wanted to do it in the same way, but failed to find the correct tile sizes for areas that cover e.g. 7 split rectangles (the middle is not on the grid)
two small points: 1) the variable names midLon and midLat are a bit missleading, since you don't split in the middle, but near the middle (which perfectly solves my problem described above)
2) You removed my added try/catch from BoundaryPreparer.run() I hope you found a better solution? I needed it there to make assert work in e.g. splitArea()
An assertion is a check for something that should never be true. Only in case there is a bug this might become true. By default assertions are not checked only if you add the java paramerter -ea. If you want to perform a runtime check you should throw an Exception.
WanMil
Gerd
WanMil wrote
Hi Gerd,
that's great!
Hi WanMil,
this was a very good hint :-)
Time for african boundaries was 250 secs with /branch/performance/r2225, 267 secs with trunk, and with this patch it is now 152 secs
wow, that's more improvement than I expected.
- use simple quadtree in BoundarySaver.splitArea()
You are a quadtree guy. In this case I thinks it's more irritating and complex to split the area into four subareas. I've changed that to split into two subareas and the code is now much easier to read (I think so..?!?) and performs in similar speed.
WanMil
- add try/catch in BoundaryPreparer.run() to allow e.g. usage of assert. Without that, an assertion was handled by the threading routines and caused program stop without any message.
Please double check this, I am still not so experienced with try+catch and wrong placed
Ciao, Gerd
http://gis.19327.n5.nabble.com/file/n5512906/splitArea_v1.patch splitArea_v1.patch
WanMil wrote
Hi Gerd,
boundary preprocessing takes quite a lot of time. That's not a big problem because it's performed only by some people who publish their results.
But I think some speed improvements would not be bad anyhow?!
While preprocessing asia and america I go an idea. The area of a boundary is splitted into the raster. But each time the complete area is intersected to the raster. In case an area is big and/or complex (boundary of russia, USA, canada etc.) this takes a long time. It would be possible first to cut the area into smaller pieces (columns or rows of the raster) and then do the final cut to the raster. Maybe this saves time because the column or row areas should be less complex.
What do you think?
WanMil _______________________________________________ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- View this message in context: http://gis.19327.n5.nabble.com/PATCH-V1-Idea-for-minor-speed-improvement-for... Sent from the Mkgmap Development mailing list archive at Nabble.com. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- View this message in context: http://gis.19327.n5.nabble.com/PATCH-V1-Idea-for-minor-speed-improvement-for... Sent from the Mkgmap Development mailing list archive at Nabble.com. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- View this message in context: http://gis.19327.n5.nabble.com/PATCH-V1-Idea-for-minor-speed-improvement-for... Sent from the Mkgmap Development mailing list archive at Nabble.com.
participants (2)
-
GerdP -
WanMil