From da754c0e5d4b7e6a2f34b0345c6c694b27319cea Mon Sep 17 00:00:00 2001 From: Kodi Arfer Date: Tue, 12 Jun 2018 11:13:06 -0700 Subject: [PATCH] Update NEWS and docs for the new comprehensions --- NEWS.rst | 6 + docs/extra/reserved.rst | 2 +- docs/language/api.rst | 255 +++++++++++++++++++++------------------- docs/tutorial.rst | 17 ++- 4 files changed, 149 insertions(+), 131 deletions(-) diff --git a/NEWS.rst b/NEWS.rst index 35155f2..5ca0d90 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -14,6 +14,7 @@ Removals * Macros `ap-pipe` and `ap-compose` have been removed. Anaphoric macros do not work well with point-free style programming, in which case both threading macros and `comp` are more adequate. +* `for/a` has been removed. Use `(for [:async ...] ...)` instead. Other Breaking Changes ------------------------------ @@ -30,6 +31,10 @@ Other Breaking Changes * Non-shadow unary `=`, `is`, `<`, etc. now evaluate their argument instead of ignoring it. This change increases consistency a bit and makes accidental unary uses easier to notice. +* `list-comp`, `set-comp`, `dict-comp`, and `genexpr` have been replaced + by `lfor`, `sfor`, `dfor`, and `gfor`, respectively, which use a new + syntax and have additional features. All Python comprehensions can now + be written in Hy. * `hy-repr` uses registered functions instead of methods * `HyKeyword` no longer inherits from the string type and has been made into its own object type. @@ -47,6 +52,7 @@ New Features keyword arguments * Added a command-line option `-E` per CPython * `while` and `for` are allowed to have empty bodies +* `for` supports the various new clause types offered by `lfor` * Added a new module ``hy.model_patterns`` Bug Fixes diff --git a/docs/extra/reserved.rst b/docs/extra/reserved.rst index 26f853f..9ef4639 100644 --- a/docs/extra/reserved.rst +++ b/docs/extra/reserved.rst @@ -10,7 +10,7 @@ Usage: ``(names)`` This function can be used to get a list (actually, a ``frozenset``) of the names of Hy's built-in functions, macros, and special forms. The output also includes all Python reserved words. All names are in unmangled form -(e.g., ``list-comp`` rather than ``list_comp``). +(e.g., ``not-in`` rather than ``not_in``). .. code-block:: hy diff --git a/docs/language/api.rst b/docs/language/api.rst index 73066ee..68c9280 100644 --- a/docs/language/api.rst +++ b/docs/language/api.rst @@ -321,48 +321,17 @@ is only called on every other value in the list. (side-effect2 x)) -dict-comp ---------- - -``dict-comp`` is used to create dictionaries. It takes three or four parameters. -The first two parameters are for controlling the return value (key-value pair) -while the third is used to select items from a sequence. The fourth and optional -parameter can be used to filter out some of the items in the sequence based on a -conditional expression. - -.. code-block:: hy - - => (dict-comp x (* x 2) [x (range 10)] (odd? x)) - {1: 2, 3: 6, 9: 18, 5: 10, 7: 14} - - do ---------- -``do`` is used to evaluate each of its arguments and return the -last one. Return values from every other than the last argument are discarded. -It can be used in ``list-comp`` to perform more complex logic as shown in one -of the following examples. +``do`` (called ``progn`` in some Lisps) takes any number of forms, +evaluates them, and returns the value of the last one, or ``None`` if no +forms were provided. -Some example usage: +:: -.. code-block:: clj - - => (if True - ... (do (print "Side effects rock!") - ... (print "Yeah, really!"))) - Side effects rock! - Yeah, really! - - ;; assuming that (side-effect) is a function that we want to call for each - ;; and every value in the list, but whose return value we do not care about - => (list-comp (do (side-effect x) - ... (if (< x 5) (* 2 x) - ... (* 4 x))) - ... (x (range 10))) - [0, 2, 4, 6, 8, 20, 24, 28, 32, 36] - -``do`` can accept any number of arguments, from 1 to n. + => (+ 1 (do (setv x (+ 1 1)) x)) + 3 doc / #doc @@ -400,6 +369,20 @@ Gets help for macros or tag macros, respectively. Gets help for a tag macro function available in this module. +dfor +---- + +``dfor`` creates a :ref:`dictionary comprehension `. Its syntax +is the same as that of `lfor`_ except that the final value form must be +a literal list of two elements, the first of which becomes each key and +the second of which becomes each value. + +.. code-block:: hy + + => (dfor x (range 5) [x (* x 10)]) + {0: 0, 1: 10, 2: 20, 3: 30, 4: 40} + + setv ---- @@ -524,8 +507,8 @@ Parameters may have the following keywords in front of them: .. code-block:: clj => (defn zig-zag-sum [&rest numbers] - (setv odd-numbers (list-comp x [x numbers] (odd? x)) - even-numbers (list-comp x [x numbers] (even? x))) + (setv odd-numbers (lfor x numbers :if (odd? x) x) + even-numbers (lfor x numbers :if (even? x) x)) (- (sum odd-numbers) (sum even-numbers))) => (zig-zag-sum) @@ -850,24 +833,39 @@ raising an exception. for --- -``for`` is used to call a function for each element in a list or vector. -The results of each call are discarded and the ``for`` expression returns -``None`` instead. The example code iterates over *collection* and for each -*element* in *collection* calls the ``side-effect`` function with *element* -as its argument: +``for`` is used to evaluate some forms for each element in an iterable +object, such as a list. The return values of the forms are discarded and +the ``for`` form returns ``None``. -.. code-block:: clj +:: - ;; assuming that (side-effect) is a function that takes a single parameter - (for [element collection] (side-effect element)) + => (for [x [1 2 3]] + ... (print "iterating") + ... (print x)) + iterating + 1 + iterating + 2 + iterating + 3 - ;; for can have an optional else block - (for [element collection] (side-effect element) - (else (side-effect-2))) +In its square-bracketed first argument, ``for`` allows the same types of +clauses as lfor_. -The optional ``else`` block is only executed if the ``for`` loop terminates -normally. If the execution is halted with ``break``, the ``else`` block does -not execute. +:: + + => (for [x [1 2 3] :if (!= x 2) y [7 8]] + ... (print x y)) + 1 7 + 1 8 + 3 7 + 3 8 + +Furthermore, the last argument of ``for`` can be an ``(else …)`` form. +This form is executed after the last iteration of the ``for``\'s +outermost iteration clause, but only if that outermost loop terminates +normally. If it's jumped out of with e.g. ``break``, the ``else`` is +ignored. .. code-block:: clj @@ -888,43 +886,6 @@ not execute. loop finished -for/a ------ - -``for/a`` behaves like ``for`` but is used to call a function for each -element generated by an asynchronous generator expression. The results -of each call are discarded and the ``for/a`` expression returns -``None`` instead. - -.. code-block:: clj - - ;; assuming that (side-effect) is a function that takes a single parameter - (for/a [element (agen)] (side-effect element)) - - ;; for/a can have an optional else block - (for/a [element (agen)] (side-effect element) - (else (side-effect-2))) - - -genexpr -------- - -``genexpr`` is used to create generator expressions. It takes two or three -parameters. The first parameter is the expression controlling the return value, -while the second is used to select items from a list. The third and optional -parameter can be used to filter out some of the items in the list based on a -conditional expression. ``genexpr`` is similar to ``list-comp``, except it -returns an iterable that evaluates values one by one instead of evaluating them -immediately. - -.. code-block:: hy - - => (setv collection (range 10)) - => (setv filtered (genexpr x [x collection] (even? x))) - => (list filtered) - [0, 2, 4, 6, 8] - - .. _gensym: gensym @@ -977,6 +938,24 @@ successive elements in a nested structure. Example usage: index that is out of bounds. +gfor +---- + +``gfor`` creates a :ref:`generator expression `. Its syntax +is the same as that of `lfor`_. The difference is that ``gfor`` returns +an iterator, which evaluates and yields values one at a time. + +:: + + => (setv accum []) + => (list (take-while + ... (fn [x] (< x 5)) + ... (gfor x (count) :do (.append accum x) x))) + [0, 1, 2, 3, 4] + => accum + [0, 1, 2, 3, 4, 5] + + global ------ @@ -1190,27 +1169,69 @@ last 6 -list-comp ---------- +lfor +---- -``list-comp`` performs list comprehensions. It takes two or three parameters. -The first parameter is the expression controlling the return value, while -the second is used to select items from a list. The third and optional -parameter can be used to filter out some of the items in the list based on a -conditional expression. Some examples: +The comprehension forms ``lfor``, `sfor`_, `dfor`_, `gfor`_, and `for`_ +are used to produce various kinds of loops, including Python-style +:ref:`comprehensions `. ``lfor`` in particular +creates a list comprehension. A simple use of ``lfor`` is:: -.. code-block:: clj - - => (setv collection (range 10)) - => (list-comp x [x collection]) - [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] - - => (list-comp (* x 2) [x collection]) - [0, 2, 4, 6, 8, 10, 12, 14, 16, 18] - - => (list-comp (* x 2) [x collection] (< x 5)) + => (lfor x (range 5) (* 2 x)) [0, 2, 4, 6, 8] +``x`` is the name of a new variable, which is bound to each element of +``(range 5)``. Each such element in turn is used to evaluate the value +form ``(* 2 x)``, and the results are accumulated into a list. + +Here's a more complex example:: + + => (lfor + ... x (range 3) + ... y (range 3) + ... :if (!= x y) + ... :setv total (+ x y) + ... [x y total]) + [[0, 1, 1], [0, 2, 2], [1, 0, 1], [1, 2, 3], [2, 0, 2], [2, 1, 3]] + +When there are several iteration clauses (here, the pairs of forms ``x +(range 3)`` and ``y (range 3)``), the result works like a nested loop or +Cartesian product: all combinations are considered in lexicographic +order. + +The general form of ``lfor`` is:: + + (lfor CLAUSES VALUE) + +where the ``VALUE`` is an arbitrary form that is evaluated to produce +each element of the result list, and ``CLAUSES`` is any number of +clauses. There are several types of clauses: + +- Iteration clauses, which look like ``LVALUE ITERABLE``. The ``LVALUE`` + is usually just a symbol, but could be something more complicated, + like ``[x y]``. +- ``:async LVALUE ITERABLE``, which is an + :ref:`asynchronous ` form of iteration clause. +- ``:do FORM``, which simply evaluates the ``FORM``. If you use + ``(continue)`` or ``(break)`` here, they will apply to the innermost + iteration clause before the ``:do``. +- ``:setv LVALUE RVALUE``, which is equivalent to ``:do (setv LVALUE + RVALUE)``. +- ``:if CONDITION``, which is equivalent to ``:do (unless CONDITION + (continue))``. + +For ``lfor``, ``sfor``, ``gfor``, and ``dfor``, variables are scoped as +if the comprehension form were its own function, so variables defined by +an iteration clause or ``:setv`` are not visible outside the form. In +fact, these forms are implemented as generator functions whenever they +contain Python statements, with the attendant consequences for calling +``return``. By contrast, ``for`` shares the caller's scope. + +.. note:: An exception to the above scoping rules occurs on Python 2 for + ``lfor`` specifically (and not ``sfor``, ``gfor``, or ``dfor``) when + Hy can implement the ``lfor`` as a Python list comprehension. Then, + variables will leak to the surrounding scope. + nonlocal -------- @@ -1465,20 +1486,12 @@ the end of a function, put ``None`` there yourself. => (print (f 4)) None -set-comp --------- -``set-comp`` is used to create sets. It takes two or three parameters. -The first parameter is for controlling the return value, while the second is -used to select items from a sequence. The third and optional parameter can be -used to filter out some of the items in the sequence based on a conditional -expression. +sfor +---- -.. code-block:: hy - - => (setv data [1 2 3 4 5 2 3 4 5 3 4 5]) - => (set-comp x [x data] (odd? x)) - {1, 3, 5} +``sfor`` creates a set comprehension. ``(sfor CLAUSES VALUE)`` is +equivalent to ``(set (lfor CLAUSES VALUE))``. See `lfor`_. cut @@ -1944,13 +1957,13 @@ infinite series without consuming infinite amount of memory. => (multiply (range 5) (range 5)) - => (list-comp value [value (multiply (range 10) (range 10))]) + => (list (multiply (range 10) (range 10))) [0, 1, 4, 9, 16, 25, 36, 49, 64, 81] => (import random) => (defn random-numbers [low high] ... (while True (yield (.randint random low high)))) - => (list-comp x [x (take 15 (random-numbers 1 50))]) + => (list (take 15 (random-numbers 1 50))) [7, 41, 6, 22, 32, 17, 5, 38, 18, 38, 17, 14, 23, 23, 19] diff --git a/docs/tutorial.rst b/docs/tutorial.rst index b85bf3a..70522e6 100644 --- a/docs/tutorial.rst +++ b/docs/tutorial.rst @@ -378,21 +378,20 @@ In Hy, you could do these like: .. code-block:: clj (setv odds-squared - (list-comp - (pow num 2) - (num (range 100)) - (= (% num 2) 1))) - + (lfor + num (range 100) + :if (= (% num 2) 1) + (pow num 2))) .. code-block:: clj ; And, an example stolen shamelessly from a Clojure page: ; Let's list all the blocks of a Chessboard: - (list-comp - (, x y) - (x (range 8) - y "ABCDEFGH")) + (lfor + x (range 8) + y "ABCDEFGH" + (, x y)) ; [(0, 'A'), (0, 'B'), (0, 'C'), (0, 'D'), (0, 'E'), (0, 'F'), (0, 'G'), (0, 'H'), ; (1, 'A'), (1, 'B'), (1, 'C'), (1, 'D'), (1, 'E'), (1, 'F'), (1, 'G'), (1, 'H'),