Commit: r1235: Fix hang on continue action.

Version 1235 was commited by steve on 2009-09-27 16:42:08 +0100 (Sun, 27 Sep 2009) BRANCH: style Fix hang on continue action.

svn commit wrote:
Version 1235 was commited by steve on 2009-09-27 16:42:08 +0100 (Sun, 27 Sep 2009) BRANCH: style
Fix hang on continue action.
Thanks for your work, sadly though for me at the moment the style branch is useless because continue on the strictly ordered style rules has no effect. (I need to have the rules running not only in order, but also if continue given, several rules on the same object. (actually I don't even need to really have then run in order, but would really need to have every rule run on each object, even if another rule matched already) Plus, it runs about 10 times slower than the trunk branch (plus continue patch, plus make ways out of relations patch) with my style-files - I hope that this can be optimized. (Austria.osm.bz2 from Geofabrik takes 45 minutes with the style-branch and still needs to calculate 1 out of 6 tiles with my style-file unadapted, while it takes 6:35 with the trunk and continue patch (plus a few more patches all however with little impact on calculation time). 5 times slower is really too slow (and it's definitely not swapping memory to disk, plenty free) - this might be however because the max-jobs seems to be broken (though it does not get the tiles out in input order but cpu load is only 1 instead of 3). The style which I adapted for strict rules ordering took 12 minutes (but here with no continue on the rules working, it's useless).
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

On 27/09/09 21:28, Felix Hartmann wrote:
Thanks for your work, sadly though for me at the moment the style branch is useless because continue on the strictly ordered style rules has no effect. (I need to have the rules running not only in order, but also if continue given, several rules on the same object. (actually I don't even
Thanks for testing it. I was trying out your style file and it didn't appear to be working for me either but I couldn't be sure, since I don't really know what it should be like. Obviously it is a bug if multiple lines are not being created.
Plus, it runs about 10 times slower than the trunk branch (plus continue
I've not really looked at the performance yet, but there is no reason why it should be any slower, so I'm sure that is fixable. ..Steve

Steve Ratcliffe wrote:
On 27/09/09 21:28, Felix Hartmann wrote:
Thanks for your work, sadly though for me at the moment the style branch is useless because continue on the strictly ordered style rules has no effect. (I need to have the rules running not only in order, but also if continue given, several rules on the same object. (actually I don't even
Thanks for testing it. I was trying out your style file and it didn't appear to be working for me either but I couldn't be sure, since I don't really know what it should be like.
Obviously it is a bug if multiple lines are not being created.
Multiple lines ARE created. but what does not work is rules like the following: highway=primary { name '${name} pri' | 'primary' } continue highway=* { name '${name} (${ref})' | '${ref}' | '${name}' } continue if a highway=primary has a reference set this case will result in the reference being dropped. I would need it to be run without the reference being dropped. Actually this concept of continue is not ideal for such rules, best would be a command to start again from continue on all lines: i.e. I have a: highway=path; route=mtb; mtb:scale=0: mtb:scale:uphill=0 mtb:scale=0 & mtb:scale:uphill=0 & route=mtb { name 'mtbrt00 ${name}' | 'mtbrt00' } continue mtb:scale:uphill=0 & route=mtb { name 'mtbrt.0 ${name}' | 'mtbrt.0' } continue mtb:scale=0 & route=mtb { name 'mtbrt0. ${name}' | 'mtbrt0.' } continue highway=path { name '${name} pth' | 'pth' } continue would make all three rules being enacted which I do not want, I only want to have further rules still run again on the same objects (if not yet written out to 0x* without continue). best would be to have the continue tag working as a new pass, so on each continue tag, all objects (that are not yet written out to 0x* without continue) will be parsed again. so the resulting way would be named: name="mtbrt00mtbrt.0mtbrt0.{name}pth" I wan't the result however to be "mtbrt00 {name} pth" ideal would be to have inside the style-file the following: mtb:scale=0 & mtb:scale:uphill=0 & route=mtb { name 'mtbrt00 ${name}' | 'mtbrt00' } parseagain mtb:scale:uphill=0 & route=mtb { name 'mtbrt.0 ${name}' | 'mtbrt.0' } mtb:scale=0 & route=mtb { name 'mtbrt0. ${name}' | 'mtbrt0.' } highway=path { name '${name} pth' | 'pth' } parseagain to reach my above wanted result. *Objects* should only be excluded after parsing through lines with 0x* but without continue. So I could start my style-file with boundaries with continue (before any rules are run) then things I don't want to be considered for the main rules block like: highway=motorway [0x1e resolution 16] highway=motorway_link [0x28 resolution 20] highway=motorway_junction [0x28 resolution 20] highway=trunk [0x28 resolution 18] highway=trunk_link [0x28 resolution 20] highway=* & motorroad=yes [0x28 resolution 18] highway=* & motoroad=yes [0x28 resolution 18] Rules (run against everything that is still, in, but motorways are already out, the same for say railway=* & tunnel=* - objects which I don't need the main rules block to be run against so that it has speed improvements) highway=* [0x* resolution * continue] *Actually* it would be great if we could define a type, say 0x999999 which drops items out from further processing because we don't want them to be displayed at all in our map. I.e. I don't want my maps to include waterway=* & tunnel=yes, but I want to display all tunnel=yes with type 0x*. To speed up the processing any object which is on type 0x99999 is dropped from further processing and not put into the map. This would avoid a lot of != use and could speed up mkgmap because you can quickly exclude objects from the database that the rules and following lines have to be run against.
Plus, it runs about 10 times slower than the trunk branch (plus continue
I've not really looked at the performance yet, but there is no reason why it should be any slower, so I'm sure that is fixable.
..Steve _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Moin, I am not sure, I understand plans completely. Felix Hartmann schrieb:
I have a: highway=path; route=mtb; mtb:scale=0: mtb:scale:uphill=0
mtb:scale=0 & mtb:scale:uphill=0 & route=mtb { name 'mtbrt00 ${name}' | 'mtbrt00' } continue mtb:scale:uphill=0 & route=mtb { name 'mtbrt.0 ${name}' | 'mtbrt.0' } continue mtb:scale=0 & route=mtb { name 'mtbrt0. ${name}' | 'mtbrt0.' } continue
highway=path { name '${name} pth' | 'pth' } continue
would make all three rules being enacted which I do not want, I only want to have further rules still run again on the same objects (if not yet written out to 0x* without continue).
So in this case, you only want to have the first of your action rules beeing carried out, the next two action rules shall be left out, and than the conversion shall contiue with last rule and carrie on with all following rules? I do not see any usefull syntax right now, to enable such complex style-rules control. But I think you can reach your goal quite easily, when you introduce some auxilliary flags: mkgmap_mtbrt_name_fixed!=yes & mtb:scale=0 & mtb:scale:uphill=0 & route=mtb { name 'mtbrt00 ${name}' | 'mtbrt00' ; mkgmap_mtbrt_name_fixed=yes } continue mkgmap_mtbrt_name_fixed!=yes & mtb:scale:uphill=0 & route=mtb { name 'mtbrt.0 ${name}' | 'mtbrt.0' ; mkgmap_mtbrt_name_fixed=yes } continue mkgmap_mtbrt_name_fixed!=yes & mtb:scale=0 & route=mtb { name 'mtbrt0. ${name}' | 'mtbrt0.' ; mkgmap_mtbrt_name_fixed=yes } continue
*Actually* it would be great if we could define a type, say 0x999999 which drops items out from further processing because we don't want them to be displayed at all in our map. I.e. I don't want my maps to include waterway=* & tunnel=yes, but I want to display all tunnel=yes with type 0x*. To speed up the processing any object which is on type 0x99999 is dropped from further processing and not put into the map. This would avoid a lot of != use and could speed up mkgmap because you can quickly exclude objects from the database that the rules and following lines have to be run against.
I think the 0x999999 is quite a bad hack. But some kind of stop command might be really usefull, not only for performance gains but also for making the style files less complicated. But what about a style rule without the continue? Wouldn't this stop the processing. I haven't look at the latest work from Steve yet, but in your above eyample, you use the continue statement on rules with an action part only. If you insert a rule like: waterway=* & tunnel=yes {set ignored=yes} without any continue, shouldn't this stop the processing of the item? Gruss Torsten

Torsten Leistikow wrote:
Moin,
I am not sure, I understand plans completely.
Felix Hartmann schrieb:
I have a: highway=path; route=mtb; mtb:scale=0: mtb:scale:uphill=0
mtb:scale=0 & mtb:scale:uphill=0 & route=mtb { name 'mtbrt00 ${name}' | 'mtbrt00' } continue mtb:scale:uphill=0 & route=mtb { name 'mtbrt.0 ${name}' | 'mtbrt.0' } continue mtb:scale=0 & route=mtb { name 'mtbrt0. ${name}' | 'mtbrt0.' } continue
highway=path { name '${name} pth' | 'pth' } continue
would make all three rules being enacted which I do not want, I only want to have further rules still run again on the same objects (if not yet written out to 0x* without continue).
So in this case, you only want to have the first of your action rules beeing carried out, the next two action rules shall be left out, and than the conversion shall contiue with last rule and carrie on with all following rules?
No, not the first. I want that it stops after the first matching. So if the first matches, 2 and 3 are dropped. if 2 matches 3 is dropped.
I do not see any usefull syntax right now, to enable such complex style-rules control. But I think you can reach your goal quite easily, when you introduce some auxilliary flags:
mkgmap_mtbrt_name_fixed!=yes & mtb:scale=0 & mtb:scale:uphill=0 & route=mtb { name 'mtbrt00 ${name}' | 'mtbrt00' ; mkgmap_mtbrt_name_fixed=yes } continue
mkgmap_mtbrt_name_fixed!=yes & mtb:scale:uphill=0 & route=mtb { name 'mtbrt.0 ${name}' | 'mtbrt.0' ; mkgmap_mtbrt_name_fixed=yes } continue
mkgmap_mtbrt_name_fixed!=yes & mtb:scale=0 & route=mtb { name 'mtbrt0. ${name}' | 'mtbrt0.' ; mkgmap_mtbrt_name_fixed=yes } continue
okay, I see that point. makes it a bit more complicated.
*Actually* it would be great if we could define a type, say 0x999999 which drops items out from further processing because we don't want them to be displayed at all in our map. I.e. I don't want my maps to include waterway=* & tunnel=yes, but I want to display all tunnel=yes with type 0x*. To speed up the processing any object which is on type 0x99999 is dropped from further processing and not put into the map. This would avoid a lot of != use and could speed up mkgmap because you can quickly exclude objects from the database that the rules and following lines have to be run against.
I think the 0x999999 is quite a bad hack. But some kind of stop command might be really usefull, not only for performance gains but also for making the style files less complicated.
I did not mean to actually put 0x999999 into the final map. I just used it because 0x999999 can never exist. Of course stop as command would work too.
But what about a style rule without the continue? Wouldn't this stop the processing. I haven't look at the latest work from Steve yet, but in your above eyample, you use the continue statement on rules with an action part only. If you insert a rule like: waterway=* & tunnel=yes {set ignored=yes} without any continue, shouldn't this stop the processing of the item?
Currently the continue as Steve implemented it does not work at all for action rules - they don't seem to have an effect. But yes to answer the question, a stop command should stop the processing of the item. So stop and continue together are impossible. Without stop command or ignored=yes in a simple action rule, they will be taken into account in 0x* rules - that is what I don't want.
Gruss Torsten
Hope to have made clear what I think would be important. Actually the strict order is the least important for me.
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Felix Hartmann schrieb:
No, not the first. I want that it stops after the first matching. So if the first matches, 2 and 3 are dropped. if 2 matches 3 is dropped.
With the "first rule matching", I was referring to your example, where the first rule was already matching.
Hope to have made clear what I think would be important. Actually the strict order is the least important for me.
If you extend your styles with such auxiliary flags, it can only work, if the execution order for the rules is guaranteed. And I do not see any possibility, how you can implement your naming scheme without a strict ordering. Gruss Torsten

Moin, I tried out the latest version of the style-branch with the attached style on a small example map, and the result looked quite good. I just noted one problem: If you have a combined action and conversion rule with a continue statement, then the action part is not performed for the following conversion rules. Example: An element has the following tags: highway=crossing and crossing=zebra_crossing Now with the following two rules highway=crossing & crossing=zebra_crossing {set highway=deleted_crossing} [0x4004 resolution 24 continue] and highway=crossing [0x610f resolution 24 continue] two elements are created on the map (0x610f and 0x4004). My idea of the continue-statement would mean, that only the first element should be created, because afterwards the second rule isn't matching anymore. Gruss Torsten

Hi
highway=crossing& crossing=zebra_crossing {set highway=deleted_crossing} [0x4004 resolution 24 continue]
highway=crossing [0x610f resolution 24 continue]
two elements are created on the map (0x610f and 0x4004).
My idea of the continue-statement would mean, that only the first element should be created, because afterwards the second rule isn't matching anymore.
Yes that is what should happen, so it is a bug. Previously changing a tag that was later matched did not work reliably, but I added code to make it work as part of strict ordering. Or so I thought! ..Steve
participants (4)
-
Felix Hartmann
-
Steve Ratcliffe
-
svn commit
-
Torsten Leistikow