diff --git a/NEWS.rst b/NEWS.rst index 5b2d54c..35155f2 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -47,6 +47,7 @@ New Features keyword arguments * Added a command-line option `-E` per CPython * `while` and `for` are allowed to have empty bodies +* Added a new module ``hy.model_patterns`` Bug Fixes ------------------------------ diff --git a/docs/language/index.rst b/docs/language/index.rst index 1a73bd2..abfe290 100644 --- a/docs/language/index.rst +++ b/docs/language/index.rst @@ -12,4 +12,5 @@ Contents: syntax api core + model_patterns internals diff --git a/docs/language/model_patterns.rst b/docs/language/model_patterns.rst new file mode 100644 index 0000000..a099850 --- /dev/null +++ b/docs/language/model_patterns.rst @@ -0,0 +1,113 @@ +============== +Model Patterns +============== + +The module ``hy.model-patterns`` provides a library of parser combinators for +parsing complex trees of Hy models. Model patterns exist mostly to help +implement the compiler, but they can also be useful for writing macros. + +A motivating example +-------------------- + +The kind of problem that model patterns are suited for is the following. +Suppose you want to validate and extract the components of a form like: + +.. code-block:: clj + + (setv form '(try + (foo1) + (foo2) + (except [EType1] + (foo3)) + (except [e EType2] + (foo4) + (foo5)) + (except [] + (foo6)) + (finally + (foo7) + (foo8)))) + +You could do this with loops and indexing, but it would take a lot of code and +be error-prone. Model patterns concisely express the general form of an +expression to be matched, like what a regular expression does for text. Here's +a pattern for a ``try`` form of the above kind: + +.. code-block:: clj + + (import [funcparserlib.parser [maybe many]]) + (import [hy.model-patterns [*]]) + (setv parser (whole [ + (sym "try") + (many (notpexpr "except" "else" "finally")) + (many (pexpr + (sym "except") + (| (brackets) (brackets FORM) (brackets SYM FORM)) + (many FORM))) + (maybe (dolike "else")) + (maybe (dolike "finally"))])) + +You can run the parser with ``(.parse parser form)``. The result is: + +.. code-block:: clj + + (, + ['(foo1) '(foo2)] + [ + '([EType1] [(foo3)]) + '([e EType2] [(foo4) (foo5)]) + '([] [(foo6)])] + None + '((foo7) (foo8))) + +which is conveniently utilized with an assignment such as ``(setv [body +except-clauses else-part finally-part] result)``. Notice that ``else-part`` +will be set to ``None`` because there is no ``else`` clause in the original +form. + +Usage +----- + +Model patterns are implemented as funcparserlib_ parser combinators. We won't +reproduce funcparserlib's own documentation, but here are some important +built-in parsers: + +- ``(+ ...)`` matches its arguments in sequence. +- ``(| ...)`` matches any one of its arguments. +- ``(>> parser function)`` matches ``parser``, then feeds the result through + ``function`` to change the value that's produced on a successful parse. +- ``(skip parser)`` matches ``parser``, but doesn't add it to the produced + value. +- ``(maybe parser)`` matches ``parser`` if possible. Otherwise, it produces + the value ``None``. +- ``(some function)`` takes a predicate ``function`` and matches a form if it + satisfies the predicate. + +The best reference for Hy's parsers is the docstrings (use ``(help +hy.model-patterns)``), but again, here are some of the more important ones: + +- ``FORM`` matches anything. +- ``SYM`` matches any symbol. +- ``(sym "foo")`` or ``(sym ":foo")`` matches and discards (per ``skip``) the + named symbol or keyword. +- ``(brackets ...)`` matches the arguments in square brackets. +- ``(pexpr ...)`` matches the arguments in parentheses. + +Here's how you could write a simple macro using model patterns: + +.. code-block:: clj + + (defmacro pairs [&rest args] + (import [funcparserlib.parser [many]]) + (import [hy.model-patterns [whole SYM FORM]]) + (setv [args] (->> args (.parse (whole [ + (many (+ SYM FORM))])))) + `[~@(->> args (map (fn [x] + (, (name (get x 0)) (get x 1)))))]) + + (print (pairs a 1 b 2 c 3)) + ; => [["a" 1] ["b" 2] ["c" 3]] + +A failed parse will raise ``funcparserlib.parser.NoParseError``. + +.. _funcparserlib: https://github.com/vlasovskikh/funcparserlib diff --git a/tests/native_tests/model_patterns.hy b/tests/native_tests/model_patterns.hy new file mode 100644 index 0000000..0e6c316 --- /dev/null +++ b/tests/native_tests/model_patterns.hy @@ -0,0 +1,70 @@ +;; Copyright 2018 the authors. +;; This file is part of Hy, which is free software licensed under the Expat +;; license. See the LICENSE. + +(defmacro do-until [&rest args] + (import + [hy.model-patterns [whole FORM notpexpr dolike]] + [funcparserlib.parser [many]]) + (setv [body condition] (->> args (.parse (whole + [(many (notpexpr "until")) (dolike "until")])))) + (setv g (gensym)) + `(do + (setv ~g True) + (while (or ~g (not (do ~@condition))) + ~@body + (setv ~g False)))) + +(defn test-do-until [] + (setv n 0 s "") + (do-until + (+= s "x") + (until (+= n 1) (>= n 3))) + (assert (= s "xxx")) + (do-until + (+= s "x") + (until (+= n 1) (>= n 3))) + (assert (= s "xxxx"))) + +(defmacro loop [&rest args] + (import + [hy.model-patterns [whole FORM sym SYM]] + [funcparserlib.parser [many]]) + (setv [loopers body] (->> args (.parse (whole [ + (many (| + (>> (+ (sym "while") FORM) (fn [x] [x])) + (+ (sym "for") SYM (sym "in") FORM) + (+ (sym "for") SYM (sym "from") FORM (sym "to") FORM))) + (sym "do") + (many FORM)])))) + (defn f [loopers] + (setv [head tail] [(first loopers) (cut loopers 1)]) + (print head) + (cond + [(none? head) + `(do ~@body)] + [(= (len head) 1) + `(while ~@head ~(f tail))] + [(= (len head) 2) + `(for [~@head] ~(f tail))] + [True ; (= (len head) 3) + (setv [sym from to] head) + `(for [~sym (range ~from (inc ~to))] ~(f tail))])) + (f loopers)) + +(defn test-loop [] + + (setv l []) + (loop + for x in "abc" + do (.append l x)) + (assert (= l ["a" "b" "c"])) + + (setv l [] k 2) + (loop + while (> k 0) + for n from 1 to 3 + for p in [k n (* 10 n)] + do (.append l p) (-= k 1)) + (print l) + (assert (= l [2 1 10 -1 2 20 -4 3 30])))