Hi Every now and again, there is a bug report on a style rule that gives an error when it shouldn't. The most recent from Mike Baggaley, included a case where the rule was accepted but the result was incorrect. I wrote a little program to generate random rules and test them using the style tester. This showed that there were many rules that should be accepted that are not and worse there are cases where the wrong result is given. Worst of all this simple rule: a>=1 & (a=1 | b~2) [0x2] (and many others) crash mkgmap with a StackOverflowError. I'm surprised that no-one has found that. Perhaps they have and it just hasn't been reported. An example of a rule that is accepted but doesn't work is: (a!=1 | b~2) & a<2 & a>=2 [0x2] This matches (with trunk) with a way with tags a=2 and b=2 but it should not. So not very complex rules fail. There are also vast number of rules that involve using not (!) such as !(highway=service & ...) that are not accepted or do not work. Probably not many styles have many such rules though. So I re-wrote and simplified the code and the result is on the expr branch. There is also a new feature that was needed to help make it work. There is now a not regex match operator !~. It is probably not all that useful but it is now availabl. name !~ 'Main.*' is exactly the same as !(name ~ 'Main.*') The latest branch build can be downloaded from: http://www.mkgmap.org.uk/download/mkgmap-expr-latest.zip It is important to test it, as although I believe that the branch is now correct, some styles may have been relying on previous bugs. ..Steve
I ran into this bug many years ago - but back then I either assumed or got told that the first rule must not be != but has to be plain =. However I think it simply did not work instead of throwing a stack overflow. On 7 August 2017 at 22:43, Steve Ratcliffe <steve@parabola.me.uk> wrote:
Hi
Every now and again, there is a bug report on a style rule that gives an error when it shouldn't. The most recent from Mike Baggaley, included a case where the rule was accepted but the result was incorrect.
I wrote a little program to generate random rules and test them using the style tester.
This showed that there were many rules that should be accepted that are not and worse there are cases where the wrong result is given.
Worst of all this simple rule:
a>=1 & (a=1 | b~2) [0x2]
(and many others) crash mkgmap with a StackOverflowError. I'm surprised that no-one has found that. Perhaps they have and it just hasn't been reported.
An example of a rule that is accepted but doesn't work is:
(a!=1 | b~2) & a<2 & a>=2 [0x2]
This matches (with trunk) with a way with tags a=2 and b=2 but it should not.
So not very complex rules fail. There are also vast number of rules that involve using not (!) such as !(highway=service & ...) that are not accepted or do not work. Probably not many styles have many such rules though.
So I re-wrote and simplified the code and the result is on the expr branch.
There is also a new feature that was needed to help make it work. There is now a not regex match operator !~. It is probably not all that useful but it is now availabl.
name !~ 'Main.*'
is exactly the same as
!(name ~ 'Main.*')
The latest branch build can be downloaded from:
http://www.mkgmap.org.uk/download/mkgmap-expr-latest.zip
It is important to test it, as although I believe that the branch is now correct, some styles may have been relying on previous bugs.
..Steve _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- Felix Hartman - Openmtbmap.org & VeloMap.org Schusterbergweg 32/8 6020 Innsbruck Austria - Österreich
Hi Steve, I've tried it with a slightly modified version of Minkos OFM lite style and got a NPE. My command: java -jar c:\temp\mkgmap-expr-r3990\mkgmap.jar --style-file=c:\mystyles\ofm_lite --check-styles Style see http://files.mkgmap.org.uk/download/355/ofm_lite.zip I've also noted an experimental line in the default style lines file: !!(aeroway=runway & highway!=*) & foo=bar [0x1] Gerd java.lang.NullPointerException at uk.me.parabola.mkgmap.osmstyle.eval.NotExistsOp.toString(NotExistsOp.java:39) at uk.me.parabola.mkgmap.osmstyle.eval.AbstractBinaryOp.toString(AbstractBinaryOp.java:54) at uk.me.parabola.mkgmap.osmstyle.eval.AbstractBinaryOp.toString(AbstractBinaryOp.java:65) at java.util.Formatter$FormatSpecifier.printString(Unknown Source) at java.util.Formatter$FormatSpecifier.print(Unknown Source) at java.util.Formatter.format(Unknown Source) at java.util.Formatter.format(Unknown Source) at java.lang.String.format(Unknown Source) at uk.me.parabola.mkgmap.osmstyle.ExpressionArranger.fmtExpr(ExpressionArranger.java:560) at uk.me.parabola.mkgmap.osmstyle.ExpressionArranger.arrange(ExpressionArranger.java:73) at uk.me.parabola.mkgmap.osmstyle.RuleFileReader.loadFile(RuleFileReader.java:125) at uk.me.parabola.mkgmap.osmstyle.RuleFileReader.load(RuleFileReader.java:95) at uk.me.parabola.mkgmap.osmstyle.StyleImpl.readRules(StyleImpl.java:301) at uk.me.parabola.mkgmap.osmstyle.StyleImpl.<init>(StyleImpl.java:155) at uk.me.parabola.mkgmap.osmstyle.StyleImpl.readStyle(StyleImpl.java:545) at uk.me.parabola.mkgmap.reader.osm.OsmMapDataSource.createConverter(OsmMapDataSource.java:333) at uk.me.parabola.mkgmap.reader.osm.OsmMapDataSource.setupHandler(OsmMapDataSource.java:226) at uk.me.parabola.mkgmap.reader.osm.OsmMapDataSource.parse(OsmMapDataSource.java:168) at uk.me.parabola.mkgmap.reader.osm.OsmMapDataSource.load(OsmMapDataSource.java:150) at uk.me.parabola.mkgmap.main.MapMaker.loadFromFile(MapMaker.java:154) at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:52) at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:263) at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:259) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Steve Ratcliffe <steve@parabola.me.uk> Gesendet: Montag, 7. August 2017 22:43:55 An: mkgmap development Betreff: [mkgmap-dev] New style branch Hi Every now and again, there is a bug report on a style rule that gives an error when it shouldn't. The most recent from Mike Baggaley, included a case where the rule was accepted but the result was incorrect. I wrote a little program to generate random rules and test them using the style tester. This showed that there were many rules that should be accepted that are not and worse there are cases where the wrong result is given. Worst of all this simple rule: a>=1 & (a=1 | b~2) [0x2] (and many others) crash mkgmap with a StackOverflowError. I'm surprised that no-one has found that. Perhaps they have and it just hasn't been reported. An example of a rule that is accepted but doesn't work is: (a!=1 | b~2) & a<2 & a>=2 [0x2] This matches (with trunk) with a way with tags a=2 and b=2 but it should not. So not very complex rules fail. There are also vast number of rules that involve using not (!) such as !(highway=service & ...) that are not accepted or do not work. Probably not many styles have many such rules though. So I re-wrote and simplified the code and the result is on the expr branch. There is also a new feature that was needed to help make it work. There is now a not regex match operator !~. It is probably not all that useful but it is now availabl. name !~ 'Main.*' is exactly the same as !(name ~ 'Main.*') The latest branch build can be downloaded from: http://www.mkgmap.org.uk/download/mkgmap-expr-latest.zip It is important to test it, as although I believe that the branch is now correct, some styles may have been relying on previous bugs. ..Steve _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Steve, one more small problem: For the attached style the check-styles option says: e:\ld>java -jar c:\temp\mkgmap-expr-r3991\mkgmap.jar --style-file=c:\mystyles\mini --check-styles Time started: Fri Aug 11 08:09:08 CEST 2017 Found one style in c:\mystyles\mini Error in style: Error: (polygons:9): Invalid rule expression: $building!=no & $building!=public & $building!=school & $building!=church & $building!=cathredral & $building!=chapel & $building!=proposed & $mkgmap:residential!=* ... Would it be possible to report the 1st line of the rule (polygons:1) instead of the empty line after the wrong rule? In this case the author first thought that this is related to an eol/ eof problem because the rule also appeared at the end of the file. ciao, Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Steve Ratcliffe <steve@parabola.me.uk> Gesendet: Mittwoch, 9. August 2017 12:58:43 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] New style branch Hi Gerd,
I've tried it with a slightly modified version of Minkos OFM lite style and got a NPE.
Thanks for testing. My random rules were not including any exists nodes so didn't catch that one :-( I've removed the experimental line from the lines file. Cheers Steve _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
participants (3)
-
Felix Hartmann -
Gerd Petermann -
Steve Ratcliffe