reduce_peak_mem_v1.patch
Hi, attached is another patch to reduce peak memory usage in mkgmap. - reduce initial size of typically ArrayList instances which typically contain few entries - free way after it was converted in ElementSaver - free the precomp-sea index in SeaGenerator after usage (the related HashMap requires ~17Mb) for each job. I do not yet understand why this is index is a ThreadLocal. I guess it was planned to have a single thread just for the index? I think that would be good, but I don't want to change that part now because it will probably be changed in the mp_cut branch. Ciao, Gerd -- View this message in context: http://gis.19327.n5.nabble.com/reduce-peak-mem-v1-patch-tp5743803.html Sent from the Mkgmap Development mailing list archive at Nabble.com.
Hi, it turned out that the index in the SeaGenerator should NOT be deleted, because it us reused for each job (which also explains what ThreadLocal is good for ;-) @WanMil: It would be quite easy to reduce the memory needs for this index. I assume ~ 1MB would be enough if we just store the information whether the grid element contains land, see, or both, and calculate the file name instead of reading it from the index. Are you still working on the mp_cut branch or do you wait for something from me? I assume that the precomp-sea handling will change when the mp cut algo changes? Gerd
Date: Thu, 10 Jan 2013 09:31:57 -0800 From: gpetermann_muenchen@hotmail.com To: mkgmap-dev@lists.mkgmap.org.uk Subject: [mkgmap-dev] reduce_peak_mem_v1.patch
Hi,
attached is another patch to reduce peak memory usage in mkgmap. - reduce initial size of typically ArrayList instances which typically contain few entries - free way after it was converted in ElementSaver - free the precomp-sea index in SeaGenerator after usage (the related HashMap requires ~17Mb) for each job. I do not yet understand why this is index is a ThreadLocal. I guess it was planned to have a single thread just for the index? I think that would be good, but I don't want to change that part now because it will probably be changed in the mp_cut branch.
Ciao, Gerd
-- View this message in context: http://gis.19327.n5.nabble.com/reduce-peak-mem-v1-patch-tp5743803.html Sent from the Mkgmap Development mailing list archive at Nabble.com. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Are you still working on the mp_cut branch or do you wait for something from me? I assume that the precomp-sea handling will change when the mp cut algo changes?
Yes. I have found some problems around tagging in mps that need to be fixed for the mp_cut branch but I am working on it. I am not waiting for you. Feel free to create you own CutAlgorithm (the interface probably will change a bit but that will be no big problem). The precomp-sea handling probably will not change now. The only change is that data for multiple resolutions will be created. But that's only a task for a different cut algorithm. The other code need not be changed for that at first. Once we know that it improves things there will be lots of things that might be optimized which maybe also requires some changes in precomp-sea handling. WanMil
Hi WanMil,
Are you still working on the mp_cut branch or do you wait for something from me? I assume that the precomp-sea handling will change when the mp cut algo changes?
Yes. I have found some problems around tagging in mps that need to be fixed for the mp_cut branch but I am working on it. I am not waiting for you. Feel free to create you own CutAlgorithm (the interface probably will change a bit but that will be no big problem).
OK, I am sure this is a complex change. Up to now I have no real new algorithm because I am not sure if the new algo should just try to avoid thin areas or if it should try to avoid small areas at all. Another point: I think you wrote that garmin format doesn't support holes in areas, but I can't imagine what problem one gets when we just overlay one polygon with one or more other polygons. I think this is by far the better solution compared to the complex algo in cutOutInnerPolygons(). I wanted to try it with --generate-sea=polygons, but I am not experienced regarding styles and TYP files. My problem: I did not find out what changes are needed in the style and what TYP file one needs to try it. It would be great if the wiki could show all needed steps. Ciao, Gerd
Hi WanMil,
Are you still working on the mp_cut branch or do you wait for something from me? I assume that the precomp-sea handling will change when the mp cut algo changes?
Yes. I have found some problems around tagging in mps that need to be fixed for the mp_cut branch but I am working on it. I am not waiting for you. Feel free to create you own CutAlgorithm (the interface probably will change a bit but that will be no big problem).
OK, I am sure this is a complex change. Up to now I have no real new algorithm because I am not sure if the new algo should just try to avoid thin areas or if it should try to avoid small areas at all.
Another point: I think you wrote that garmin format doesn't support holes in areas, but I can't imagine what problem one gets when we just overlay one polygon with one or more other polygons.
One little thing is obvious: When selecting a point on the land area Garmin shows you all stuff at this point and that's: Land Sea Another thing is that the order of how data is drawn cannot be defined (as far as I know...). So if you have overlapping polygons it's a kind of luck which polygon is on top of the other.
I think this is by far the better solution compared to the complex algo in cutOutInnerPolygons(). I wanted to try it with --generate-sea=polygons, but I am not experienced regarding styles and TYP files. My problem: I did not find out what changes are needed in the style and what TYP file one needs to try it. It would be great if the wiki could show all needed steps.
I think the only thing you have to do is to add a rule for natural=sea and natural=land in your polygons file. WanMil
Ciao, Gerd
Regarding polygon draw order. Draw level can be defined in TYP file if one is used. Draw level of sea should be < that of land. Geoff. WanMil <wmgcnfg@web.de> wrote:
Hi WanMil,
Are you still working on the mp_cut branch or do you wait for something from me? I assume that the precomp-sea handling will change when the mp cut algo changes?
Yes. I have found some problems around tagging in mps that need to be fixed for the mp_cut branch but I am working on it. I am not waiting for you. Feel free to create you own CutAlgorithm (the interface probably will change a bit but that will be no big problem).
OK, I am sure this is a complex change. Up to now I have no real new algorithm because I am not sure if the new algo should just try to avoid thin areas or if it should try to avoid small areas at all.
Another point: I think you wrote that garmin format doesn't support holes in areas, but I can't imagine what problem one gets when we just overlay one polygon with one or more other polygons.
One little thing is obvious: When selecting a point on the land area Garmin shows you all stuff at this point and that's: Land Sea
Another thing is that the order of how data is drawn cannot be defined (as far as I know...). So if you have overlapping polygons it's a kind of luck which polygon is on top of the other.
I think this is by far the better solution compared to the complex algo in cutOutInnerPolygons(). I wanted to try it with --generate-sea=polygons, but I am not experienced regarding styles and TYP files. My problem: I did not find out what changes are needed in the style and what TYP file one needs to try it. It would be great if the wiki could show all needed steps.
I think the only thing you have to do is to add a rule for natural=sea and natural=land in your polygons file.
WanMil
Ciao, Gerd
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
On 12.01.2013 17:26, Geoff Sherlock wrote:
Regarding polygon draw order. Draw level can be defined in TYP file if one is used. Draw level of sea should be < that of land.
Geoff. Yes they can be changed. But you only have about ~ 10 levels AFAIK. So for some things (but only the most important, like sea, land, forest, residential, any others?) - we could say that we put them into even every order of 4 levels (and use for each one type - so that makes 16 types in the typfile. Then we would say - depending on order, mkgmap chooses one of the types. From level 5 upwards come all other types - but they won't be ordered by mkgmap.
So for a small number of objects, we could indeed use the typfile in conjunction with mkgmap to choose the type. In general this is no good solution....
participants (5)
-
Felix Hartmann -
Geoff Sherlock -
Gerd Petermann -
GerdP
-
WanMil