echo improvements for elements with fake ids
The attached patch improves the echo and echotags messages by including the id of the original element when a fake id is generated by mkgmap, thus enabling you to find it in OSM. Please try, and commit if you are happy with the change. Regards, Mike
Hi Mike, I also think that we should try to show the original id when possible. I did not try it, but I think it would be less memory consuming to add a tag with the original id, as most elements do not have a fake id. Another problem here is that we have methods which combine different OSM elements, e.g. RoadMerger, which is not using a fake id but simply changing one way element and removes another. The tag would allow to combine the ids. What do you think? Gerd Mike Baggaley wrote
The attached patch improves the echo and echotags messages by including the id of the original element when a fake id is generated by mkgmap, thus enabling you to find it in OSM.
Please try, and commit if you are happy with the change.
Regards, Mike
_______________________________________________ mkgmap-dev mailing list
mkgmap-dev@.org
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
fakeid.patch (12K) <http://gis.19327.n5.nabble.com/attachment/5830557/0/fakeid.patch>;
-- View this message in context: http://gis.19327.n5.nabble.com/echo-improvements-for-elements-with-fake-ids-... Sent from the Mkgmap Development mailing list archive at Nabble.com.
@Mike - Wonderful idea. I'm a complete newbie here but have been using mkgmap for a while now with good results. I was just asking about the echotags feature on the OSM Garmin list, and more specifically what the long number was that precedes the message. Now I read that it will be made to do some useful work for OSMers. How long before this patch makes it into the released application do you think? Cheers, Dave On Mon, Jan 19, 2015 at 6:55 PM, Mike Baggaley <mike@tvage.co.uk> wrote:
The attached patch improves the echo and echotags messages by including the id of the original element when a fake id is generated by mkgmap, thus enabling you to find it in OSM.
Please try, and commit if you are happy with the change.
Regards, Mike
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- Dave Swarthout Homer, Alaska Chiang Mai, Thailand Travel Blog at http://dswarthout.blogspot.com
Hi Mike, I've committed the patch as is. The memory impact is not that big as we have typically less than 500.000 instances of Element in one tile, so 8 more bytes * 500.000 ~ 4MB more heap. Gerd From: mike@tvage.co.uk To: mkgmap-dev@lists.mkgmap.org.uk Date: Mon, 19 Jan 2015 11:55:00 +0000 Subject: [mkgmap-dev] echo improvements for elements with fake ids The attached patch improves the echo and echotags messages by including the id of the original element when a fake id is generated by mkgmap, thus enabling you to find it in OSM. Please try, and commit if you are happy with the change. Regards, Mike _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
HI Gerd, thanks for that - I was scratching my head trying to think of a way that would use less memory. If using the tags, each tag would have used more than 8 bytes, although they would have only been on faked elements. The tag iterators would also have processed them (unless special code was included to exclude them). I did wonder whether we could store both the original Id and the fake Id in the 64 bit number with perhaps 40 bits for the OSM Id and 24 bits for the fake Id, but this might not be enough space (taking into account the future). Mike From: Gerd Petermann [mailto:gpetermann_muenchen@hotmail.com] Sent: 19 January 2015 20:29 To: mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] echo improvements for elements with fake ids Hi Mike, I've committed the patch as is. The memory impact is not that big as we have typically less than 500.000 instances of Element in one tile, so 8 more bytes * 500.000 ~ 4MB more heap. Gerd From: mike@tvage.co.uk <mailto:mike@tvage.co.uk> To: mkgmap-dev@lists.mkgmap.org.uk <mailto:mkgmap-dev@lists.mkgmap.org.uk> Date: Mon, 19 Jan 2015 11:55:00 +0000 Subject: [mkgmap-dev] echo improvements for elements with fake ids The attached patch improves the echo and echotags messages by including the id of the original element when a fake id is generated by mkgmap, thus enabling you to find it in OSM. Please try, and commit if you are happy with the change. Regards, Mike _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk <mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Mike, that's a good idea to tackle the fake id problem. I didn't have time to test it but after a short look over the committed patch I think there are a few things to be improved: First of all small stylistic things: fields should start with lower case letters. So OriginalId in Element should be originalId. It would be good if you add (javadoc) comments to your source code :-) It seems that the ways created by Multipolygon relations use the relation id as original id. I am not sure if that's the best choice. It's not easy to decide which id should be chosen when ways are merged. When using the relation id it should be clear in the output that this id is the relation id and not a way id. Otherwise users will search for completely wrong ways. WanMil
HI Gerd, thanks for that – I was scratching my head trying to think of a way that would use less memory. If using the tags, each tag would have used more than 8 bytes, although they would have only been on faked elements. The tag iterators would also have processed them (unless special code was included to exclude them). I did wonder whether we could store both the original Id and the fake Id in the 64 bit number with perhaps 40 bits for the OSM Id and 24 bits for the fake Id, but this might not be enough space (taking into account the future).
Mike
*From:*Gerd Petermann [mailto:gpetermann_muenchen@hotmail.com] *Sent:* 19 January 2015 20:29 *To:* mkgmap-dev@lists.mkgmap.org.uk *Subject:* Re: [mkgmap-dev] echo improvements for elements with fake ids
Hi Mike,
I've committed the patch as is. The memory impact is not that big as we have typically less than 500.000 instances of Element in one tile, so 8 more bytes * 500.000 ~ 4MB more heap.
Gerd
From: mike@tvage.co.uk <mailto:mike@tvage.co.uk> To: mkgmap-dev@lists.mkgmap.org.uk <mailto:mkgmap-dev@lists.mkgmap.org.uk> Date: Mon, 19 Jan 2015 11:55:00 +0000 Subject: [mkgmap-dev] echo improvements for elements with fake ids
The attached patch improves the echo and echotags messages by including the id of the original element when a fake id is generated by mkgmap, thus enabling you to find it in OSM.
Please try, and commit if you are happy with the change.
Regards, Mike
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk <mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi WanMil, I did notice after posting the code that the other class members started with a lower case letter. No problem with following that convention (developments I have been involved in have always used upper case first letter for class variables, to distinguish them from local variables). I come from C++/C# background, so have not heard of Javadoc comments before. I will have a look at how they work. It seems to me to be better to use the relation Id (if it is available) if a fake way is being built from a relation, rather than the Id of one way within the relation. I gather that echo/echotags can be used on points, lines and relations, and that the original id could be a line or relation (I haven't noticed any code generating fake points from points, but could have missed it). I did think about storing a reference to the original Element instead of its id, but didn't think that improving a diagnostic message warranted that much, just to add the word relation or way. Adding a variable to indicate the sort of original element would use another 8 bytes per Element (unless we used the significant bits within originalId). The ways/relations Ids I used for testing were all unique - when looking up the Id in OSM either the way or relation did not exist with that number. I don't know whether this is always be the case. Please feel free to improve the algorithm. Regards, Mike -----Original Message----- From: WanMil [mailto:wmgcnfg@web.de] Sent: 20 January 2015 22:24 To: Development list for mkgmap Subject: Re: [mkgmap-dev] echo improvements for elements with fake ids Hi Mike, that's a good idea to tackle the fake id problem. I didn't have time to test it but after a short look over the committed patch I think there are a few things to be improved: First of all small stylistic things: fields should start with lower case letters. So OriginalId in Element should be originalId. It would be good if you add (javadoc) comments to your source code :-) It seems that the ways created by Multipolygon relations use the relation id as original id. I am not sure if that's the best choice. It's not easy to decide which id should be chosen when ways are merged. When using the relation id it should be clear in the output that this id is the relation id and not a way id. Otherwise users will search for completely wrong ways. WanMil
HI Gerd, thanks for that - I was scratching my head trying to think of a way that would use less memory. If using the tags, each tag would have used more than 8 bytes, although they would have only been on faked elements. The tag iterators would also have processed them (unless special code was included to exclude them). I did wonder whether we could store both the original Id and the fake Id in the 64 bit number with perhaps 40 bits for the OSM Id and 24 bits for the fake Id, but this might not be enough space (taking into account the future).
Mike
*From:*Gerd Petermann [mailto:gpetermann_muenchen@hotmail.com] *Sent:* 19 January 2015 20:29 *To:* mkgmap-dev@lists.mkgmap.org.uk *Subject:* Re: [mkgmap-dev] echo improvements for elements with fake ids
Hi Mike,
I've committed the patch as is. The memory impact is not that big as we have typically less than 500.000 instances of Element in one tile, so 8 more bytes * 500.000 ~ 4MB more heap.
Gerd
From: mike@tvage.co.uk <mailto:mike@tvage.co.uk> To: mkgmap-dev@lists.mkgmap.org.uk <mailto:mkgmap-dev@lists.mkgmap.org.uk> Date: Mon, 19 Jan 2015 11:55:00 +0000 Subject: [mkgmap-dev] echo improvements for elements with fake ids
The attached patch improves the echo and echotags messages by including the id of the original element when a fake id is generated by mkgmap, thus enabling you to find it in OSM.
Please try, and commit if you are happy with the change.
Regards, Mike
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk <mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Gerd, did you measure the impact on the BoundaryPreprocessor and the PrecompiledSeaGenerator? I guess they might load much more than 500k instances and I think it would be ok to disable the original id tracking when they are running. On the other hand can you roughly estimate the size of an additional tag? Is it the additional String instance? I am thinking how to fix the thing that the original id might be derived from a different element type. A tag would be easy to implement without big impacts. WanMil
Hi Mike,
I've committed the patch as is. The memory impact is not that big as we have typically less than 500.000 instances of Element in one tile, so 8 more bytes * 500.000 ~ 4MB more heap.
Gerd
From: mike@tvage.co.uk To: mkgmap-dev@lists.mkgmap.org.uk Date: Mon, 19 Jan 2015 11:55:00 +0000 Subject: [mkgmap-dev] echo improvements for elements with fake ids
The attached patch improves the echo and echotags messages by including the id of the original element when a fake id is generated by mkgmap, thus enabling you to find it in OSM.
Please try, and commit if you are happy with the change.
Regards, Mike
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi WanMil, good catch. I only looked at the normal processing. Reg. memory: I think one additional String instance + In some cases the additional tag will increase the tag arrays, hard to say how often that will happen. Gerd WanMil wrote
Hi Gerd,
did you measure the impact on the BoundaryPreprocessor and the PrecompiledSeaGenerator? I guess they might load much more than 500k instances and I think it would be ok to disable the original id tracking when they are running.
On the other hand can you roughly estimate the size of an additional tag? Is it the additional String instance?
I am thinking how to fix the thing that the original id might be derived from a different element type. A tag would be easy to implement without big impacts.
WanMil
Hi Mike,
I've committed the patch as is. The memory impact is not that big as we have typically less than 500.000 instances of Element in one tile, so 8 more bytes * 500.000 ~ 4MB more heap.
Gerd
From:
mike@.co
To:
mkgmap-dev@.org
Date: Mon, 19 Jan 2015 11:55:00 +0000 Subject: [mkgmap-dev] echo improvements for elements with fake ids
The attached patch improves the echo and echotags messages by including the id of the original element when a fake id is generated by mkgmap, thus enabling you to find it in OSM.
Please try, and commit if you are happy with the change.
Regards, Mike
_______________________________________________ mkgmap-dev mailing list
mkgmap-dev@.org
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list
mkgmap-dev@.org
_______________________________________________ mkgmap-dev mailing list
mkgmap-dev@.org
-- View this message in context: http://gis.19327.n5.nabble.com/echo-improvements-for-elements-with-fake-ids-... Sent from the Mkgmap Development mailing list archive at Nabble.com.
Hi WanMil, probably no problem here. My results: PrecompiledSeaGenerator: no significant effect BoundaryPreprocessor : very small effect. I think in both cases the Element instances are not kept for long, means, they are converted into something else which is not based on Element. The number of Element instances is not very high compared to Coord instances and / or the doubles used in java areas. Gerd
Date: Wed, 21 Jan 2015 21:16:54 +0100 From: wmgcnfg@web.de To: mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] echo improvements for elements with fake ids
Hi Gerd,
did you measure the impact on the BoundaryPreprocessor and the PrecompiledSeaGenerator? I guess they might load much more than 500k instances and I think it would be ok to disable the original id tracking when they are running.
On the other hand can you roughly estimate the size of an additional tag? Is it the additional String instance?
I am thinking how to fix the thing that the original id might be derived from a different element type. A tag would be easy to implement without big impacts.
WanMil
Hi Mike,
I've committed the patch as is. The memory impact is not that big as we have typically less than 500.000 instances of Element in one tile, so 8 more bytes * 500.000 ~ 4MB more heap.
Gerd
From: mike@tvage.co.uk To: mkgmap-dev@lists.mkgmap.org.uk Date: Mon, 19 Jan 2015 11:55:00 +0000 Subject: [mkgmap-dev] echo improvements for elements with fake ids
The attached patch improves the echo and echotags messages by including the id of the original element when a fake id is generated by mkgmap, thus enabling you to find it in OSM.
Please try, and commit if you are happy with the change.
Regards, Mike
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Gerd, sounds good! I was wrong that each Coord instance would be an Element but of course that's wrong... WanMil
Hi WanMil,
probably no problem here. My results: PrecompiledSeaGenerator: no significant effect BoundaryPreprocessor : very small effect. I think in both cases the Element instances are not kept for long, means, they are converted into something else which is not based on Element. The number of Element instances is not very high compared to Coord instances and / or the doubles used in java areas.
Gerd
Date: Wed, 21 Jan 2015 21:16:54 +0100 From: wmgcnfg@web.de To: mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] echo improvements for elements with fake ids
Hi Gerd,
did you measure the impact on the BoundaryPreprocessor and the PrecompiledSeaGenerator? I guess they might load much more than 500k instances and I think it would be ok to disable the original id tracking when they are running.
On the other hand can you roughly estimate the size of an additional tag? Is it the additional String instance?
I am thinking how to fix the thing that the original id might be derived from a different element type. A tag would be easy to implement without big impacts.
WanMil
Hi Mike,
I've committed the patch as is. The memory impact is not that big as we have typically less than 500.000 instances of Element in one tile, so 8 more bytes * 500.000 ~ 4MB more heap.
Gerd
From: mike@tvage.co.uk To: mkgmap-dev@lists.mkgmap.org.uk Date: Mon, 19 Jan 2015 11:55:00 +0000 Subject: [mkgmap-dev] echo improvements for elements with fake ids
The attached patch improves the echo and echotags messages by including the id of the original element when a fake id is generated by mkgmap, thus enabling you to find it in OSM.
Please try, and commit if you are happy with the change.
Regards, Mike
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
participants (5)
-
Dave Swarthout -
Gerd Petermann -
GerdP
-
Mike Baggaley -
WanMil