Patch to reduce memory usage by interning strings.

On 31/03/10 14:56, Scott A Crosby wrote:
I noticed that mkgmap does not intern any strings. In particular, this
This is true. There is a pending patch that deals with excessive memory use in a slightly different way which is on the style-speed branch, with a pre-built one available at the bottom of the mkgmap download page. I drops all tags that are not used by the current style and as an extra feature it ensures that there is only copy of all the strings on the key side. Could you try it out please. It may be worth also interning the values but when I was looking at it there was much less benefit for the maps I was looking at (but it might vary with the input). I'd be happy to apply a patch that interned the values too if there was a decent memory saving without a significant performance drop. I think the point that Chris brought up applies mostly in cases where there is a lot of contended access between threads. If that is the case then it won't matter much for us as reading the input files is currently single threaded. Kristen Eisenberg Billige Flüge Marketing GmbH Emanuelstr. 3, 10317 Berlin Deutschland Telefon: +49 (33) 5310967 Email: utebachmeier at gmail.com Site: http://flug.airego.de - Billige Flüge vergleichen

On 20/10/11 19:57, Kristen Eisenberg wrote:
On 31/03/10 14:56, Scott A Crosby wrote:
I noticed that mkgmap does not intern any strings. In particular, this
This is true. There is a pending patch that deals with excessive memory use in a slightly different way which is on the style-speed branch, with a pre-built one available at the bottom of the mkgmap download page.
That branch was merged to trunk quite some time ago. There was also a change to intern the string tag values. So now there are no duplicates in either the tag keys or values... Unless there is a new bug? ..Steve
participants (2)
-
Kristen Eisenberg
-
Steve Ratcliffe