Merge pull request #1626 from Kodiologist/newcomp3
Fancier `for` and comprehensions
This commit is contained in:
Kodi Arfer
GitHub
commit
f22195dfbc
6
NEWS.rst
6
NEWS.rst
@ -14,6 +14,7 @@ Removals
|
||||
* Macros `ap-pipe` and `ap-compose` have been removed.
|
||||
Anaphoric macros do not work well with point-free style programming,
|
||||
in which case both threading macros and `comp` are more adequate.
|
||||
* `for/a` has been removed. Use `(for [:async ...] ...)` instead.
|
||||
|
||||
Other Breaking Changes
|
||||
------------------------------
|
||||
@ -30,6 +31,10 @@ Other Breaking Changes
|
||||
* Non-shadow unary `=`, `is`, `<`, etc. now evaluate their argument
|
||||
instead of ignoring it. This change increases consistency a bit
|
||||
and makes accidental unary uses easier to notice.
|
||||
* `list-comp`, `set-comp`, `dict-comp`, and `genexpr` have been replaced
|
||||
by `lfor`, `sfor`, `dfor`, and `gfor`, respectively, which use a new
|
||||
syntax and have additional features. All Python comprehensions can now
|
||||
be written in Hy.
|
||||
* `hy-repr` uses registered functions instead of methods
|
||||
* `HyKeyword` no longer inherits from the string type and has been
|
||||
made into its own object type.
|
||||
@ -47,6 +52,7 @@ New Features
|
||||
keyword arguments
|
||||
* Added a command-line option `-E` per CPython
|
||||
* `while` and `for` are allowed to have empty bodies
|
||||
* `for` supports the various new clause types offered by `lfor`
|
||||
* Added a new module ``hy.model_patterns``
|
||||
|
||||
Bug Fixes
|
||||
|
||||
@ -10,7 +10,7 @@ Usage: ``(names)``
|
||||
This function can be used to get a list (actually, a ``frozenset``) of the
|
||||
names of Hy's built-in functions, macros, and special forms. The output
|
||||
also includes all Python reserved words. All names are in unmangled form
|
||||
(e.g., ``list-comp`` rather than ``list_comp``).
|
||||
(e.g., ``not-in`` rather than ``not_in``).
|
||||
|
||||
.. code-block:: hy
|
||||
|
||||
|
||||
@ -321,48 +321,17 @@ is only called on every other value in the list.
|
||||
(side-effect2 x))
|
||||
|
||||
|
||||
dict-comp
|
||||
---------
|
||||
|
||||
``dict-comp`` is used to create dictionaries. It takes three or four parameters.
|
||||
The first two parameters are for controlling the return value (key-value pair)
|
||||
while the third is used to select items from a sequence. The fourth and optional
|
||||
parameter can be used to filter out some of the items in the sequence based on a
|
||||
conditional expression.
|
||||
|
||||
.. code-block:: hy
|
||||
|
||||
=> (dict-comp x (* x 2) [x (range 10)] (odd? x))
|
||||
{1: 2, 3: 6, 9: 18, 5: 10, 7: 14}
|
||||
|
||||
|
||||
do
|
||||
----------
|
||||
|
||||
``do`` is used to evaluate each of its arguments and return the
|
||||
last one. Return values from every other than the last argument are discarded.
|
||||
It can be used in ``list-comp`` to perform more complex logic as shown in one
|
||||
of the following examples.
|
||||
``do`` (called ``progn`` in some Lisps) takes any number of forms,
|
||||
evaluates them, and returns the value of the last one, or ``None`` if no
|
||||
forms were provided.
|
||||
|
||||
Some example usage:
|
||||
::
|
||||
|
||||
.. code-block:: clj
|
||||
|
||||
=> (if True
|
||||
... (do (print "Side effects rock!")
|
||||
... (print "Yeah, really!")))
|
||||
Side effects rock!
|
||||
Yeah, really!
|
||||
|
||||
;; assuming that (side-effect) is a function that we want to call for each
|
||||
;; and every value in the list, but whose return value we do not care about
|
||||
=> (list-comp (do (side-effect x)
|
||||
... (if (< x 5) (* 2 x)
|
||||
... (* 4 x)))
|
||||
... (x (range 10)))
|
||||
[0, 2, 4, 6, 8, 20, 24, 28, 32, 36]
|
||||
|
||||
``do`` can accept any number of arguments, from 1 to n.
|
||||
=> (+ 1 (do (setv x (+ 1 1)) x))
|
||||
3
|
||||
|
||||
|
||||
doc / #doc
|
||||
@ -400,6 +369,20 @@ Gets help for macros or tag macros, respectively.
|
||||
Gets help for a tag macro function available in this module.
|
||||
|
||||
|
||||
dfor
|
||||
----
|
||||
|
||||
``dfor`` creates a :ref:`dictionary comprehension <py:dict>`. Its syntax
|
||||
is the same as that of `lfor`_ except that the final value form must be
|
||||
a literal list of two elements, the first of which becomes each key and
|
||||
the second of which becomes each value.
|
||||
|
||||
.. code-block:: hy
|
||||
|
||||
=> (dfor x (range 5) [x (* x 10)])
|
||||
{0: 0, 1: 10, 2: 20, 3: 30, 4: 40}
|
||||
|
||||
|
||||
setv
|
||||
----
|
||||
|
||||
@ -524,8 +507,8 @@ Parameters may have the following keywords in front of them:
|
||||
.. code-block:: clj
|
||||
|
||||
=> (defn zig-zag-sum [&rest numbers]
|
||||
(setv odd-numbers (list-comp x [x numbers] (odd? x))
|
||||
even-numbers (list-comp x [x numbers] (even? x)))
|
||||
(setv odd-numbers (lfor x numbers :if (odd? x) x)
|
||||
even-numbers (lfor x numbers :if (even? x) x))
|
||||
(- (sum odd-numbers) (sum even-numbers)))
|
||||
|
||||
=> (zig-zag-sum)
|
||||
@ -850,24 +833,39 @@ raising an exception.
|
||||
for
|
||||
---
|
||||
|
||||
``for`` is used to call a function for each element in a list or vector.
|
||||
The results of each call are discarded and the ``for`` expression returns
|
||||
``None`` instead. The example code iterates over *collection* and for each
|
||||
*element* in *collection* calls the ``side-effect`` function with *element*
|
||||
as its argument:
|
||||
``for`` is used to evaluate some forms for each element in an iterable
|
||||
object, such as a list. The return values of the forms are discarded and
|
||||
the ``for`` form returns ``None``.
|
||||
|
||||
.. code-block:: clj
|
||||
::
|
||||
|
||||
;; assuming that (side-effect) is a function that takes a single parameter
|
||||
(for [element collection] (side-effect element))
|
||||
=> (for [x [1 2 3]]
|
||||
... (print "iterating")
|
||||
... (print x))
|
||||
iterating
|
||||
1
|
||||
iterating
|
||||
2
|
||||
iterating
|
||||
3
|
||||
|
||||
;; for can have an optional else block
|
||||
(for [element collection] (side-effect element)
|
||||
(else (side-effect-2)))
|
||||
In its square-bracketed first argument, ``for`` allows the same types of
|
||||
clauses as lfor_.
|
||||
|
||||
The optional ``else`` block is only executed if the ``for`` loop terminates
|
||||
normally. If the execution is halted with ``break``, the ``else`` block does
|
||||
not execute.
|
||||
::
|
||||
|
||||
=> (for [x [1 2 3] :if (!= x 2) y [7 8]]
|
||||
... (print x y))
|
||||
1 7
|
||||
1 8
|
||||
3 7
|
||||
3 8
|
||||
|
||||
Furthermore, the last argument of ``for`` can be an ``(else …)`` form.
|
||||
This form is executed after the last iteration of the ``for``\'s
|
||||
outermost iteration clause, but only if that outermost loop terminates
|
||||
normally. If it's jumped out of with e.g. ``break``, the ``else`` is
|
||||
ignored.
|
||||
|
||||
.. code-block:: clj
|
||||
|
||||
@ -888,43 +886,6 @@ not execute.
|
||||
loop finished
|
||||
|
||||
|
||||
for/a
|
||||
-----
|
||||
|
||||
``for/a`` behaves like ``for`` but is used to call a function for each
|
||||
element generated by an asynchronous generator expression. The results
|
||||
of each call are discarded and the ``for/a`` expression returns
|
||||
``None`` instead.
|
||||
|
||||
.. code-block:: clj
|
||||
|
||||
;; assuming that (side-effect) is a function that takes a single parameter
|
||||
(for/a [element (agen)] (side-effect element))
|
||||
|
||||
;; for/a can have an optional else block
|
||||
(for/a [element (agen)] (side-effect element)
|
||||
(else (side-effect-2)))
|
||||
|
||||
|
||||
genexpr
|
||||
-------
|
||||
|
||||
``genexpr`` is used to create generator expressions. It takes two or three
|
||||
parameters. The first parameter is the expression controlling the return value,
|
||||
while the second is used to select items from a list. The third and optional
|
||||
parameter can be used to filter out some of the items in the list based on a
|
||||
conditional expression. ``genexpr`` is similar to ``list-comp``, except it
|
||||
returns an iterable that evaluates values one by one instead of evaluating them
|
||||
immediately.
|
||||
|
||||
.. code-block:: hy
|
||||
|
||||
=> (setv collection (range 10))
|
||||
=> (setv filtered (genexpr x [x collection] (even? x)))
|
||||
=> (list filtered)
|
||||
[0, 2, 4, 6, 8]
|
||||
|
||||
|
||||
.. _gensym:
|
||||
|
||||
gensym
|
||||
@ -977,6 +938,24 @@ successive elements in a nested structure. Example usage:
|
||||
index that is out of bounds.
|
||||
|
||||
|
||||
gfor
|
||||
----
|
||||
|
||||
``gfor`` creates a :ref:`generator expression <py:genexpr>`. Its syntax
|
||||
is the same as that of `lfor`_. The difference is that ``gfor`` returns
|
||||
an iterator, which evaluates and yields values one at a time.
|
||||
|
||||
::
|
||||
|
||||
=> (setv accum [])
|
||||
=> (list (take-while
|
||||
... (fn [x] (< x 5))
|
||||
... (gfor x (count) :do (.append accum x) x)))
|
||||
[0, 1, 2, 3, 4]
|
||||
=> accum
|
||||
[0, 1, 2, 3, 4, 5]
|
||||
|
||||
|
||||
global
|
||||
------
|
||||
|
||||
@ -1190,27 +1169,69 @@ last
|
||||
6
|
||||
|
||||
|
||||
list-comp
|
||||
---------
|
||||
lfor
|
||||
----
|
||||
|
||||
``list-comp`` performs list comprehensions. It takes two or three parameters.
|
||||
The first parameter is the expression controlling the return value, while
|
||||
the second is used to select items from a list. The third and optional
|
||||
parameter can be used to filter out some of the items in the list based on a
|
||||
conditional expression. Some examples:
|
||||
The comprehension forms ``lfor``, `sfor`_, `dfor`_, `gfor`_, and `for`_
|
||||
are used to produce various kinds of loops, including Python-style
|
||||
:ref:`comprehensions <py:comprehensions>`. ``lfor`` in particular
|
||||
creates a list comprehension. A simple use of ``lfor`` is::
|
||||
|
||||
.. code-block:: clj
|
||||
|
||||
=> (setv collection (range 10))
|
||||
=> (list-comp x [x collection])
|
||||
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
|
||||
|
||||
=> (list-comp (* x 2) [x collection])
|
||||
[0, 2, 4, 6, 8, 10, 12, 14, 16, 18]
|
||||
|
||||
=> (list-comp (* x 2) [x collection] (< x 5))
|
||||
=> (lfor x (range 5) (* 2 x))
|
||||
[0, 2, 4, 6, 8]
|
||||
|
||||
``x`` is the name of a new variable, which is bound to each element of
|
||||
``(range 5)``. Each such element in turn is used to evaluate the value
|
||||
form ``(* 2 x)``, and the results are accumulated into a list.
|
||||
|
||||
Here's a more complex example::
|
||||
|
||||
=> (lfor
|
||||
... x (range 3)
|
||||
... y (range 3)
|
||||
... :if (!= x y)
|
||||
... :setv total (+ x y)
|
||||
... [x y total])
|
||||
[[0, 1, 1], [0, 2, 2], [1, 0, 1], [1, 2, 3], [2, 0, 2], [2, 1, 3]]
|
||||
|
||||
When there are several iteration clauses (here, the pairs of forms ``x
|
||||
(range 3)`` and ``y (range 3)``), the result works like a nested loop or
|
||||
Cartesian product: all combinations are considered in lexicographic
|
||||
order.
|
||||
|
||||
The general form of ``lfor`` is::
|
||||
|
||||
(lfor CLAUSES VALUE)
|
||||
|
||||
where the ``VALUE`` is an arbitrary form that is evaluated to produce
|
||||
each element of the result list, and ``CLAUSES`` is any number of
|
||||
clauses. There are several types of clauses:
|
||||
|
||||
- Iteration clauses, which look like ``LVALUE ITERABLE``. The ``LVALUE``
|
||||
is usually just a symbol, but could be something more complicated,
|
||||
like ``[x y]``.
|
||||
- ``:async LVALUE ITERABLE``, which is an
|
||||
:ref:`asynchronous <py:async for>` form of iteration clause.
|
||||
- ``:do FORM``, which simply evaluates the ``FORM``. If you use
|
||||
``(continue)`` or ``(break)`` here, they will apply to the innermost
|
||||
iteration clause before the ``:do``.
|
||||
- ``:setv LVALUE RVALUE``, which is equivalent to ``:do (setv LVALUE
|
||||
RVALUE)``.
|
||||
- ``:if CONDITION``, which is equivalent to ``:do (unless CONDITION
|
||||
(continue))``.
|
||||
|
||||
For ``lfor``, ``sfor``, ``gfor``, and ``dfor``, variables are scoped as
|
||||
if the comprehension form were its own function, so variables defined by
|
||||
an iteration clause or ``:setv`` are not visible outside the form. In
|
||||
fact, these forms are implemented as generator functions whenever they
|
||||
contain Python statements, with the attendant consequences for calling
|
||||
``return``. By contrast, ``for`` shares the caller's scope.
|
||||
|
||||
.. note:: An exception to the above scoping rules occurs on Python 2 for
|
||||
``lfor`` specifically (and not ``sfor``, ``gfor``, or ``dfor``) when
|
||||
Hy can implement the ``lfor`` as a Python list comprehension. Then,
|
||||
variables will leak to the surrounding scope.
|
||||
|
||||
|
||||
nonlocal
|
||||
--------
|
||||
@ -1465,20 +1486,12 @@ the end of a function, put ``None`` there yourself.
|
||||
=> (print (f 4))
|
||||
None
|
||||
|
||||
set-comp
|
||||
--------
|
||||
|
||||
``set-comp`` is used to create sets. It takes two or three parameters.
|
||||
The first parameter is for controlling the return value, while the second is
|
||||
used to select items from a sequence. The third and optional parameter can be
|
||||
used to filter out some of the items in the sequence based on a conditional
|
||||
expression.
|
||||
sfor
|
||||
----
|
||||
|
||||
.. code-block:: hy
|
||||
|
||||
=> (setv data [1 2 3 4 5 2 3 4 5 3 4 5])
|
||||
=> (set-comp x [x data] (odd? x))
|
||||
{1, 3, 5}
|
||||
``sfor`` creates a set comprehension. ``(sfor CLAUSES VALUE)`` is
|
||||
equivalent to ``(set (lfor CLAUSES VALUE))``. See `lfor`_.
|
||||
|
||||
|
||||
cut
|
||||
@ -1944,13 +1957,13 @@ infinite series without consuming infinite amount of memory.
|
||||
=> (multiply (range 5) (range 5))
|
||||
<generator object multiply at 0x978d8ec>
|
||||
|
||||
=> (list-comp value [value (multiply (range 10) (range 10))])
|
||||
=> (list (multiply (range 10) (range 10)))
|
||||
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
|
||||
|
||||
=> (import random)
|
||||
=> (defn random-numbers [low high]
|
||||
... (while True (yield (.randint random low high))))
|
||||
=> (list-comp x [x (take 15 (random-numbers 1 50))])
|
||||
=> (list (take 15 (random-numbers 1 50)))
|
||||
[7, 41, 6, 22, 32, 17, 5, 38, 18, 38, 17, 14, 23, 23, 19]
|
||||
|
||||
|
||||
|
||||
@ -378,21 +378,20 @@ In Hy, you could do these like:
|
||||
.. code-block:: clj
|
||||
|
||||
(setv odds-squared
|
||||
(list-comp
|
||||
(pow num 2)
|
||||
(num (range 100))
|
||||
(= (% num 2) 1)))
|
||||
|
||||
(lfor
|
||||
num (range 100)
|
||||
:if (= (% num 2) 1)
|
||||
(pow num 2)))
|
||||
|
||||
.. code-block:: clj
|
||||
|
||||
; And, an example stolen shamelessly from a Clojure page:
|
||||
; Let's list all the blocks of a Chessboard:
|
||||
|
||||
(list-comp
|
||||
(, x y)
|
||||
(x (range 8)
|
||||
y "ABCDEFGH"))
|
||||
(lfor
|
||||
x (range 8)
|
||||
y "ABCDEFGH"
|
||||
(, x y))
|
||||
|
||||
; [(0, 'A'), (0, 'B'), (0, 'C'), (0, 'D'), (0, 'E'), (0, 'F'), (0, 'G'), (0, 'H'),
|
||||
; (1, 'A'), (1, 'B'), (1, 'C'), (1, 'D'), (1, 'E'), (1, 'F'), (1, 'G'), (1, 'H'),
|
||||
|
||||
224
hy/compiler.py
224
hy/compiler.py
@ -7,7 +7,7 @@ from hy.models import (HyObject, HyExpression, HyKeyword, HyInteger, HyComplex,
|
||||
HyString, HyBytes, HySymbol, HyFloat, HyList, HySet,
|
||||
HyDict, HySequence, wrap_value)
|
||||
from hy.model_patterns import (FORM, SYM, STR, sym, brackets, whole, notpexpr,
|
||||
dolike, pexpr, times)
|
||||
dolike, pexpr, times, Tag, tag)
|
||||
from funcparserlib.parser import some, many, oneplus, maybe, NoParseError
|
||||
from hy.errors import HyCompileError, HyTypeError
|
||||
|
||||
@ -111,13 +111,13 @@ def builds_model(*model_types):
|
||||
# ast.Foo(..., lineno=x.lineno, col_offset=x.col_offset)
|
||||
class Asty(object):
|
||||
def __getattr__(self, name):
|
||||
setattr(Asty, name, lambda self, x, **kwargs: getattr(ast, name)(
|
||||
setattr(Asty, name, staticmethod(lambda x, **kwargs: getattr(ast, name)(
|
||||
lineno=getattr(
|
||||
x, 'start_line', getattr(x, 'lineno', None)),
|
||||
col_offset=getattr(
|
||||
x, 'start_column', getattr(x, 'col_offset', None)),
|
||||
**kwargs))
|
||||
return getattr(self, name)
|
||||
**kwargs)))
|
||||
return getattr(Asty, name)
|
||||
asty = Asty()
|
||||
|
||||
|
||||
@ -180,6 +180,22 @@ class Result(object):
|
||||
self.__used_expr = False
|
||||
self._expr = value
|
||||
|
||||
@property
|
||||
def lineno(self):
|
||||
if self._expr is not None:
|
||||
return self._expr.lineno
|
||||
if self.stmts:
|
||||
return self.stmts[-1].lineno
|
||||
return None
|
||||
|
||||
@property
|
||||
def col_offset(self):
|
||||
if self._expr is not None:
|
||||
return self._expr.col_offset
|
||||
if self.stmts:
|
||||
return self.stmts[-1].col_offset
|
||||
return None
|
||||
|
||||
def add_imports(self, mod, imports):
|
||||
"""Autoimport `imports` from `mod`"""
|
||||
self.imports[mod].update(imports)
|
||||
@ -1004,40 +1020,150 @@ class HyASTCompiler(object):
|
||||
|
||||
return gen_res + cond, gen
|
||||
|
||||
@special(["list-comp", "set-comp", "genexpr"], [FORM, FORM, maybe(FORM)])
|
||||
def compile_comprehension(self, expr, form, expression, gen, cond):
|
||||
# (list-comp expr [target iter] cond?)
|
||||
_loopers = many(
|
||||
tag('setv', sym(":setv") + FORM + FORM) |
|
||||
tag('if', sym(":if") + FORM) |
|
||||
tag('do', sym(":do") + FORM) |
|
||||
tag('afor', sym(":async") + FORM + FORM) |
|
||||
tag('for', FORM + FORM))
|
||||
@special(["for"], [brackets(_loopers),
|
||||
many(notpexpr("else")) + maybe(dolike("else"))])
|
||||
@special(["lfor", "sfor", "gfor"], [_loopers, FORM])
|
||||
@special(["dfor"], [_loopers, brackets(FORM, FORM)])
|
||||
def compile_comprehension(self, expr, root, parts, final):
|
||||
root = unmangle(ast_str(root))
|
||||
node_class = {
|
||||
"for": asty.For,
|
||||
"lfor": asty.ListComp,
|
||||
"dfor": asty.DictComp,
|
||||
"sfor": asty.SetComp,
|
||||
"gfor": asty.GeneratorExp}[root]
|
||||
is_for = root == "for"
|
||||
|
||||
if not isinstance(gen, HyList):
|
||||
raise HyTypeError(gen, "Generator expression must be a list.")
|
||||
orel = []
|
||||
if is_for:
|
||||
# Get the `else`.
|
||||
body, else_expr = final
|
||||
if else_expr is not None:
|
||||
orel.append(self._compile_branch(else_expr))
|
||||
orel[0] += orel[0].expr_as_stmt()
|
||||
else:
|
||||
# Get the final value (and for dictionary
|
||||
# comprehensions, the final key).
|
||||
if node_class is asty.DictComp:
|
||||
key, elt = map(self.compile, final)
|
||||
else:
|
||||
key = None
|
||||
elt = self.compile(final)
|
||||
|
||||
gen_res, gen = self._compile_generator_iterables(
|
||||
[gen] + ([] if cond is None else [cond]))
|
||||
# Compile the parts.
|
||||
if is_for:
|
||||
parts = parts[0]
|
||||
if not parts:
|
||||
return Result(expr=ast.parse({
|
||||
asty.For: "None",
|
||||
asty.ListComp: "[]",
|
||||
asty.DictComp: "{}",
|
||||
asty.SetComp: "{1}.__class__()",
|
||||
asty.GeneratorExp: "(_ for _ in [])"}[node_class]).body[0].value)
|
||||
parts = [
|
||||
Tag(p.tag, self.compile(p.value) if p.tag in ["if", "do"] else [
|
||||
self._storeize(p.value[0], self.compile(p.value[0])),
|
||||
self.compile(p.value[1])])
|
||||
for p in parts]
|
||||
|
||||
if len(gen) == 0:
|
||||
raise HyTypeError(expr, "Generator expression cannot be empty.")
|
||||
# Produce a result.
|
||||
if (is_for or elt.stmts or (key is not None and key.stmts) or
|
||||
any(p.tag == 'do' or (p.value[1].stmts if p.tag in ("for", "afor", "setv") else p.value.stmts)
|
||||
for p in parts)):
|
||||
# The desired comprehension can't be expressed as a
|
||||
# real Python comprehension. We'll write it as a nested
|
||||
# loop in a function instead.
|
||||
contains_yield = []
|
||||
def f(parts):
|
||||
# This function is called recursively to construct
|
||||
# the nested loop.
|
||||
if not parts:
|
||||
if is_for:
|
||||
if body:
|
||||
bd = self._compile_branch(body)
|
||||
if bd.contains_yield:
|
||||
contains_yield.append(True)
|
||||
return bd + bd.expr_as_stmt()
|
||||
return Result(stmts=[asty.Pass(expr)])
|
||||
if node_class is asty.DictComp:
|
||||
ret = key + elt
|
||||
val = asty.Tuple(
|
||||
key, ctx=ast.Load(),
|
||||
elts=[key.force_expr, elt.force_expr])
|
||||
else:
|
||||
ret = elt
|
||||
val = elt.force_expr
|
||||
return ret + asty.Expr(
|
||||
elt, value=asty.Yield(elt, value=val))
|
||||
(tagname, v), parts = parts[0], parts[1:]
|
||||
if tagname in ("for", "afor"):
|
||||
orelse = orel and orel.pop().stmts
|
||||
node = asty.AsyncFor if tagname == "afor" else asty.For
|
||||
return v[1] + node(
|
||||
v[1], target=v[0], iter=v[1].force_expr, body=f(parts).stmts,
|
||||
orelse=orelse)
|
||||
elif tagname == "setv":
|
||||
return v[1] + asty.Assign(
|
||||
v[1], targets=[v[0]], value=v[1].force_expr) + f(parts)
|
||||
elif tagname == "if":
|
||||
return v + asty.If(
|
||||
v, test=v.force_expr, body=f(parts).stmts, orelse=[])
|
||||
elif tagname == "do":
|
||||
return v + v.expr_as_stmt() + f(parts)
|
||||
else:
|
||||
raise ValueError("can't happen")
|
||||
if is_for:
|
||||
ret = f(parts)
|
||||
ret.contains_yield = bool(contains_yield)
|
||||
return ret
|
||||
fname = self.get_anon_var()
|
||||
# Define the generator function.
|
||||
ret = Result() + asty.FunctionDef(
|
||||
expr,
|
||||
name=fname,
|
||||
args=ast.arguments(
|
||||
args=[], vararg=None, kwarg=None,
|
||||
kwonlyargs=[], kw_defaults=[], defaults=[]),
|
||||
body=f(parts).stmts,
|
||||
decorator_list=[])
|
||||
# Immediately call the new function. Unless the user asked
|
||||
# for a generator, wrap the call in `[].__class__(...)` or
|
||||
# `{}.__class__(...)` or `{1}.__class__(...)` to get the
|
||||
# right type. We don't want to just use e.g. `list(...)`
|
||||
# because the name `list` might be rebound.
|
||||
return ret + Result(expr=ast.parse(
|
||||
"{}({}())".format(
|
||||
{asty.ListComp: "[].__class__",
|
||||
asty.DictComp: "{}.__class__",
|
||||
asty.SetComp: "{1}.__class__",
|
||||
asty.GeneratorExp: ""}[node_class],
|
||||
fname)).body[0].value)
|
||||
|
||||
ret = self.compile(expression)
|
||||
node_class = (
|
||||
asty.ListComp if form == "list-comp" else
|
||||
asty.SetComp if form == "set-comp" else
|
||||
asty.GeneratorExp)
|
||||
return ret + gen_res + node_class(
|
||||
expr, elt=ret.force_expr, generators=gen)
|
||||
|
||||
@special("dict-comp", [FORM, FORM, FORM, maybe(FORM)])
|
||||
def compile_dict_comprehension(self, expr, root, key, value, gen, cond):
|
||||
key = self.compile(key)
|
||||
value = self.compile(value)
|
||||
|
||||
gen_res, gen = self._compile_generator_iterables(
|
||||
[gen] + ([] if cond is None else [cond]))
|
||||
|
||||
return key + value + gen_res + asty.DictComp(
|
||||
expr,
|
||||
key=key.force_expr,
|
||||
value=value.force_expr,
|
||||
generators=gen)
|
||||
# We can produce a real comprehension.
|
||||
generators = []
|
||||
for tagname, v in parts:
|
||||
if tagname in ("for", "afor"):
|
||||
generators.append(ast.comprehension(
|
||||
target=v[0], iter=v[1].expr, ifs=[],
|
||||
is_async=int(tagname == "afor")))
|
||||
elif tagname == "setv":
|
||||
generators.append(ast.comprehension(
|
||||
target=v[0],
|
||||
iter=asty.Tuple(v[1], elts=[v[1].expr], ctx=ast.Load()),
|
||||
ifs=[], is_async=0))
|
||||
elif tagname == "if":
|
||||
generators[-1].ifs.append(v.expr)
|
||||
else:
|
||||
raise ValueError("can't happen")
|
||||
if node_class is asty.DictComp:
|
||||
return asty.DictComp(expr, key=key.expr, value=elt.expr, generators=generators)
|
||||
return node_class(expr, elt=elt.expr, generators=generators)
|
||||
|
||||
@special(["not", "~"], [FORM])
|
||||
def compile_unary_operator(self, expr, root, arg):
|
||||
@ -1288,36 +1414,6 @@ class HyASTCompiler(object):
|
||||
|
||||
return result
|
||||
|
||||
@special(["for*", (PY35, "for/a*")],
|
||||
[brackets(FORM, FORM), many(notpexpr("else")), maybe(dolike("else"))])
|
||||
def compile_for_expression(self, expr, root, args, body, else_expr):
|
||||
target_name, iterable = args
|
||||
target = self._storeize(target_name, self.compile(target_name))
|
||||
|
||||
ret = Result()
|
||||
|
||||
orel = Result()
|
||||
if else_expr is not None:
|
||||
for else_body in else_expr:
|
||||
orel += self.compile(else_body)
|
||||
orel += orel.expr_as_stmt()
|
||||
|
||||
ret += self.compile(iterable)
|
||||
|
||||
body = self._compile_branch(body)
|
||||
body += body.expr_as_stmt()
|
||||
|
||||
node = asty.For if root == 'for*' else asty.AsyncFor
|
||||
ret += node(expr,
|
||||
target=target,
|
||||
iter=ret.force_expr,
|
||||
body=body.stmts or [asty.Pass(expr)],
|
||||
orelse=orel.stmts)
|
||||
|
||||
ret.contains_yield = body.contains_yield
|
||||
|
||||
return ret
|
||||
|
||||
@special(["while"], [FORM, many(notpexpr("else")), maybe(dolike("else"))])
|
||||
def compile_while_expression(self, expr, root, cond, body, else_expr):
|
||||
cond_compiled = self.compile(cond)
|
||||
|
||||
@ -25,9 +25,10 @@
|
||||
(setv -seen (set))
|
||||
(defn hy-repr [obj]
|
||||
(setv [f placeholder] (next
|
||||
(genexpr (get -registry t)
|
||||
[t (. (type obj) __mro__)]
|
||||
(in t -registry))
|
||||
(gfor
|
||||
t (. (type obj) __mro__)
|
||||
:if (in t -registry)
|
||||
(get -registry t))
|
||||
[-base-repr None]))
|
||||
|
||||
(global -quoting)
|
||||
@ -55,18 +56,18 @@
|
||||
; collections.namedtuple.)
|
||||
(.format "({} {})"
|
||||
(. (type x) __name__)
|
||||
(.join " " (genexpr (+ ":" k " " (hy-repr v)) [[k v] (zip x._fields x)])))
|
||||
(.join " " (gfor [k v] (zip x._fields x) (+ ":" k " " (hy-repr v)))))
|
||||
; Otherwise, print it as a regular tuple.
|
||||
(+ "(," (if x " " "") (-cat x) ")"))))
|
||||
(hy-repr-register dict :placeholder "{...}" (fn [x]
|
||||
(setv text (.join " " (genexpr
|
||||
(+ (hy-repr k) " " (hy-repr v))
|
||||
[[k v] (.items x)])))
|
||||
(setv text (.join " " (gfor
|
||||
[k v] (.items x)
|
||||
(+ (hy-repr k) " " (hy-repr v)))))
|
||||
(+ "{" text "}")))
|
||||
(hy-repr-register HyDict :placeholder "{...}" (fn [x]
|
||||
(setv text (.join " " (genexpr
|
||||
(+ (hy-repr k) " " (hy-repr v))
|
||||
[[k v] (partition x)])))
|
||||
(setv text (.join " " (gfor
|
||||
[k v] (partition x)
|
||||
(+ (hy-repr k) " " (hy-repr v)))))
|
||||
(if (% (len x) 2)
|
||||
(+= text (+ " " (hy-repr (get x -1)))))
|
||||
(+ "{" text "}")))
|
||||
@ -162,5 +163,8 @@
|
||||
; Call (.repr x) using the first class of x that doesn't inherit from
|
||||
; HyObject.
|
||||
(.__repr__
|
||||
(next (genexpr t [t (. (type x) __mro__)] (not (issubclass t HyObject))))
|
||||
(next (gfor
|
||||
t (. (type x) __mro__)
|
||||
:if (not (issubclass t HyObject))
|
||||
t))
|
||||
x))
|
||||
|
||||
@ -11,8 +11,8 @@
|
||||
--getitem-- (fn [self n]
|
||||
"get nth item of sequence"
|
||||
(if (hasattr n "start")
|
||||
(genexpr (get self x) [x (range n.start n.stop
|
||||
(or n.step 1))])
|
||||
(gfor x (range n.start n.stop (or n.step 1))
|
||||
(get self x))
|
||||
(do (when (neg? n)
|
||||
; Call (len) to force the whole
|
||||
; sequence to be evaluated.
|
||||
|
||||
@ -18,7 +18,7 @@
|
||||
(% "received a `%s' instead of a symbol for macro name"
|
||||
(. (type name)
|
||||
__name__)))))
|
||||
(for* [kw '[&kwonly &kwargs]]
|
||||
(for [kw '[&kwonly &kwargs]]
|
||||
(if* (in kw lambda-list)
|
||||
(raise (hy.errors.HyTypeError macro-name
|
||||
(% "macros cannot use %s"
|
||||
|
||||
@ -41,7 +41,7 @@
|
||||
fs (tuple rfs))
|
||||
(fn [&rest args &kwargs kwargs]
|
||||
(setv res (first-f #* args #** kwargs))
|
||||
(for* [f fs]
|
||||
(for [f fs]
|
||||
(setv res (f res)))
|
||||
res))))
|
||||
|
||||
@ -79,7 +79,7 @@ If the second argument `codegen` is true, generate python code instead."
|
||||
(defn distinct [coll]
|
||||
"Return a generator from the original collection `coll` with no duplicates."
|
||||
(setv seen (set) citer (iter coll))
|
||||
(for* [val citer]
|
||||
(for [val citer]
|
||||
(if (not-in val seen)
|
||||
(do
|
||||
(yield val)
|
||||
@ -159,7 +159,7 @@ Return series of accumulated sums (or other binary function results)."
|
||||
(setv it (iter iterable)
|
||||
total (next it))
|
||||
(yield total)
|
||||
(for* [element it]
|
||||
(for [element it]
|
||||
(setv total (func total element))
|
||||
(yield total)))
|
||||
|
||||
@ -193,7 +193,7 @@ Return series of accumulated sums (or other binary function results)."
|
||||
|
||||
(defn _flatten [coll result]
|
||||
(if (coll? coll)
|
||||
(do (for* [b coll]
|
||||
(do (for [b coll]
|
||||
(_flatten b result)))
|
||||
(.append result coll))
|
||||
result)
|
||||
@ -287,7 +287,7 @@ Return series of accumulated sums (or other binary function results)."
|
||||
"Return a function applying each `fs` to args, collecting results in a list."
|
||||
(setv fs (+ (, f) fs))
|
||||
(fn [&rest args &kwargs kwargs]
|
||||
(list-comp (f #* args #** kwargs) [f fs])))
|
||||
(lfor f fs (f #* args #** kwargs))))
|
||||
|
||||
(defn last [coll]
|
||||
"Return last item from `coll`."
|
||||
@ -352,8 +352,8 @@ with overlap."
|
||||
(setv
|
||||
step (or step n)
|
||||
coll-clones (tee coll n)
|
||||
slices (genexpr (islice (get coll-clones start) start None step)
|
||||
[start (range n)]))
|
||||
slices (gfor start (range n)
|
||||
(islice (get coll-clones start) start None step)))
|
||||
(if (is fillvalue -sentinel)
|
||||
(zip #* slices)
|
||||
(zip-longest #* slices :fillvalue fillvalue)))
|
||||
@ -402,9 +402,9 @@ Raises ValueError for (not (pos? n))."
|
||||
(if (not (pos? n))
|
||||
(raise (ValueError "n must be positive")))
|
||||
(setv citer (iter coll) skip (dec n))
|
||||
(for* [val citer]
|
||||
(for [val citer]
|
||||
(yield val)
|
||||
(for* [_ (range skip)]
|
||||
(for [_ (range skip)]
|
||||
(try
|
||||
(next citer)
|
||||
(except [StopIteration]
|
||||
|
||||
@ -38,9 +38,9 @@ be associated in pairs."
|
||||
`(setv ~@(+ (if other-kvs
|
||||
[c coll]
|
||||
[])
|
||||
#* (genexpr [`(get ~c ~k) v]
|
||||
[[k v] (partition (+ (, k1 v1)
|
||||
other-kvs))]))))
|
||||
#* (gfor [k v] (partition (+ (, k1 v1)
|
||||
other-kvs))
|
||||
[`(get ~c ~k) v]))))
|
||||
|
||||
|
||||
(defn _with [node args body]
|
||||
@ -101,50 +101,20 @@ used as the result."
|
||||
(setv root (check-branch branch))
|
||||
(setv latest-branch root)
|
||||
|
||||
(for* [branch branches]
|
||||
(for [branch branches]
|
||||
(setv cur-branch (check-branch branch))
|
||||
(.append latest-branch cur-branch)
|
||||
(setv latest-branch cur-branch))
|
||||
root)))
|
||||
|
||||
|
||||
(defn _for [node args body]
|
||||
(setv body (list body))
|
||||
(setv belse (if (and body (isinstance (get body -1) HyExpression) (= (get body -1 0) "else"))
|
||||
[(body.pop)]
|
||||
[]))
|
||||
(if
|
||||
(odd? (len args)) (macro-error args "`for' requires an even number of args.")
|
||||
(empty? args) `(do ~@body ~@belse)
|
||||
(= (len args) 2) `(~node [~@args] (do ~@body) ~@belse)
|
||||
(do
|
||||
(setv alist (cut args 0 None 2))
|
||||
`(~node [(, ~@alist) (genexpr (, ~@alist) [~@args])] (do ~@body) ~@belse))))
|
||||
|
||||
|
||||
(defmacro for [args &rest body]
|
||||
"Build a for-loop with `args` as a [element coll] bracket pair and run `body`.
|
||||
|
||||
Args may contain multiple pairs, in which case it executes a nested for-loop
|
||||
in order of the given pairs."
|
||||
(_for 'for* args body))
|
||||
|
||||
|
||||
(defmacro for/a [args &rest body]
|
||||
"Build a for/a-loop with `args` as a [element coll] bracket pair and run `body`.
|
||||
|
||||
Args may contain multiple pairs, in which case it executes a nested for/a-loop
|
||||
in order of the given pairs."
|
||||
(_for 'for/a* args body))
|
||||
|
||||
|
||||
(defmacro -> [head &rest args]
|
||||
"Thread `head` first through the `rest` of the forms.
|
||||
|
||||
The result of the first threaded form is inserted into the first position of
|
||||
the second form, the second result is inserted into the third form, and so on."
|
||||
(setv ret head)
|
||||
(for* [node args]
|
||||
(for [node args]
|
||||
(setv ret (if (isinstance node HyExpression)
|
||||
`(~(first node) ~ret ~@(rest node))
|
||||
`(~node ~ret))))
|
||||
@ -163,13 +133,14 @@ the second form, the second result is inserted into the third form, and so on."
|
||||
~@(map build-form expressions)
|
||||
~f))
|
||||
|
||||
|
||||
(defmacro ->> [head &rest args]
|
||||
"Thread `head` last through the `rest` of the forms.
|
||||
|
||||
The result of the first threaded form is inserted into the last position of
|
||||
the second form, the second result is inserted into the third form, and so on."
|
||||
(setv ret head)
|
||||
(for* [node args]
|
||||
(for [node args]
|
||||
(setv ret (if (isinstance node HyExpression)
|
||||
`(~@node ~ret)
|
||||
`(~node ~ret))))
|
||||
@ -210,7 +181,7 @@ the second form, the second result is inserted into the third form, and so on."
|
||||
(defmacro with-gensyms [args &rest body]
|
||||
"Execute `body` with `args` as bracket of names to gensym for use in macros."
|
||||
(setv syms [])
|
||||
(for* [arg args]
|
||||
(for [arg args]
|
||||
(.extend syms [arg `(gensym '~arg)]))
|
||||
`(do
|
||||
(setv ~@syms)
|
||||
@ -225,7 +196,7 @@ the second form, the second result is inserted into the third form, and so on."
|
||||
(.startswith x "g!")))
|
||||
(flatten body))))
|
||||
gensyms [])
|
||||
(for* [sym syms]
|
||||
(for [sym syms]
|
||||
(.extend gensyms [sym `(gensym ~(cut sym 2))]))
|
||||
`(defmacro ~name [~@args]
|
||||
(setv ~@gensyms)
|
||||
@ -235,8 +206,8 @@ the second form, the second result is inserted into the third form, and so on."
|
||||
"Like `defmacro/g!`, with automatic once-only evaluation for 'o!' params.
|
||||
|
||||
Such 'o!' params are available within `body` as the equivalent 'g!' symbol."
|
||||
(setv os (list-comp s [s args] (.startswith s "o!"))
|
||||
gs (list-comp (HySymbol (+ "g!" (cut s 2))) [s os]))
|
||||
(setv os (lfor s args :if (.startswith s "o!") s)
|
||||
gs (lfor s os (HySymbol (+ "g!" (cut s 2)))))
|
||||
`(defmacro/g! ~name ~args
|
||||
`(do (setv ~@(interleave ~gs ~os))
|
||||
~@~body)))
|
||||
|
||||
@ -98,8 +98,7 @@
|
||||
(defn comp-op [op a1 a-rest]
|
||||
"Helper for shadow comparison operators"
|
||||
(if a-rest
|
||||
(reduce (fn [x y] (and x y))
|
||||
(list-comp (op x y) [(, x y) (zip (+ (, a1) a-rest) a-rest)]))
|
||||
(and #* (gfor (, x y) (zip (+ (, a1) a-rest) a-rest) (op x y)))
|
||||
True))
|
||||
(defn < [a1 &rest a-rest]
|
||||
"Shadowed `<` operator perform lt comparison on `a1` by each `a-rest`."
|
||||
@ -161,7 +160,7 @@
|
||||
(defn get [coll key1 &rest keys]
|
||||
"Access item in `coll` indexed by `key1`, with optional `keys` nested-access."
|
||||
(setv coll (get coll key1))
|
||||
(for* [k keys]
|
||||
(for [k keys]
|
||||
(setv coll (get coll k)))
|
||||
coll)
|
||||
|
||||
|
||||
@ -109,18 +109,18 @@
|
||||
`%*` and `%**` name the `&rest` and `&kwargs` parameters, respectively.
|
||||
|
||||
Nesting of `#%` forms is not recommended."
|
||||
(setv %symbols (set-comp a
|
||||
[a (flatten [expr])]
|
||||
(and (symbol? a)
|
||||
(.startswith a '%))))
|
||||
(setv %symbols (sfor a (flatten [expr])
|
||||
:if (and (symbol? a)
|
||||
(.startswith a '%))
|
||||
a))
|
||||
`(fn [;; generate all %i symbols up to the maximum found in expr
|
||||
~@(genexpr (HySymbol (+ "%" (str i)))
|
||||
[i (range 1 (-> (list-comp (int (cut a 1))
|
||||
[a %symbols]
|
||||
(.isdigit (cut a 1)))
|
||||
(or (, 0))
|
||||
max
|
||||
inc))])
|
||||
~@(gfor i (range 1 (-> (lfor a %symbols
|
||||
:if (.isdigit (cut a 1))
|
||||
(int (cut a 1)))
|
||||
(or (, 0))
|
||||
max
|
||||
inc))
|
||||
(HySymbol (+ "%" (str i))))
|
||||
;; generate the &rest parameter only if '%* is present in expr
|
||||
~@(if (in '%* %symbols)
|
||||
'(&rest %*))
|
||||
|
||||
@ -9,6 +9,7 @@ from funcparserlib.parser import (
|
||||
some, skip, many, finished, a, Parser, NoParseError, State)
|
||||
from functools import reduce
|
||||
from itertools import repeat
|
||||
from collections import namedtuple
|
||||
from operator import add
|
||||
from math import isinf
|
||||
|
||||
@ -74,3 +75,11 @@ def times(lo, hi, parser):
|
||||
end = e.state.max
|
||||
return result, State(s.pos, end)
|
||||
return f
|
||||
|
||||
Tag = namedtuple('Tag', ['tag', 'value'])
|
||||
|
||||
def tag(tag_name, parser):
|
||||
"""Matches the given parser and produces a named tuple `(Tag tag value)`
|
||||
with `tag` set to the given tag name and `value` set to the parser's
|
||||
value."""
|
||||
return parser >> (lambda x: Tag(tag_name, x))
|
||||
|
||||
@ -531,9 +531,7 @@ def test_for_compile_error():
|
||||
can_compile("(fn [] (for)))")
|
||||
assert excinfo.value.message == "Ran into a RPAREN where it wasn't expected."
|
||||
|
||||
with pytest.raises(HyTypeError) as excinfo:
|
||||
can_compile("(fn [] (for [x] x))")
|
||||
assert excinfo.value.message == "`for' requires an even number of args."
|
||||
cant_compile("(fn [] (for [x] x))")
|
||||
|
||||
|
||||
def test_attribute_access():
|
||||
@ -556,14 +554,6 @@ def test_attribute_empty():
|
||||
cant_compile('[2].foo')
|
||||
|
||||
|
||||
def test_invalid_list_comprehension():
|
||||
"""Ensure that invalid list comprehensions do not break the compiler"""
|
||||
cant_compile("(genexpr x [])")
|
||||
cant_compile("(genexpr [x [1 2 3 4]] x)")
|
||||
cant_compile("(list-comp None [])")
|
||||
cant_compile("(list-comp [x [1 2 3]] x)")
|
||||
|
||||
|
||||
def test_bad_setv():
|
||||
"""Ensure setv handles error cases"""
|
||||
cant_compile("(setv (a b) [1 2])")
|
||||
|
||||
@ -84,5 +84,5 @@ def test_eval():
|
||||
assert eval_str('(.strip " fooooo ")') == 'fooooo'
|
||||
assert eval_str(
|
||||
'(if True "this is if true" "this is if false")') == "this is if true"
|
||||
assert eval_str('(list-comp (pow num 2) [num (range 100)] (= (% num 2) 1))') == [
|
||||
assert eval_str('(lfor num (range 100) :if (= (% num 2) 1) (pow num 2))') == [
|
||||
pow(num, 2) for num in range(100) if num % 2 == 1]
|
||||
|
||||
226
tests/native_tests/comprehensions.hy
Normal file
226
tests/native_tests/comprehensions.hy
Normal file
@ -0,0 +1,226 @@
|
||||
(import
|
||||
types
|
||||
pytest
|
||||
[hy._compat [PY3]])
|
||||
|
||||
|
||||
(defn test-comprehension-types []
|
||||
|
||||
; Forms that get compiled to real comprehensions
|
||||
(assert (is (type (lfor x "abc" x)) list))
|
||||
(assert (is (type (sfor x "abc" x)) set))
|
||||
(assert (is (type (dfor x "abc" [x x])) dict))
|
||||
(assert (is (type (gfor x "abc" x)) types.GeneratorType))
|
||||
|
||||
; Forms that get compiled to loops
|
||||
(assert (is (type (lfor x "abc" :do (setv y 1) x)) list))
|
||||
(assert (is (type (sfor x "abc" :do (setv y 1) x)) set))
|
||||
(assert (is (type (dfor x "abc" :do (setv y 1) [x x])) dict))
|
||||
(assert (is (type (gfor x "abc" :do (setv y 1) x)) types.GeneratorType)))
|
||||
|
||||
|
||||
#@ ((pytest.mark.parametrize "specialop" ["for" "lfor" "sfor" "gfor" "dfor"])
|
||||
(defn test-fors [specialop]
|
||||
|
||||
(setv cases [
|
||||
['(f x [] x)
|
||||
[]]
|
||||
['(f j [1 2 3] j)
|
||||
[1 2 3]]
|
||||
['(f x (range 3) (* x 2))
|
||||
[0 2 4]]
|
||||
['(f x (range 2) y (range 2) (, x y))
|
||||
[(, 0 0) (, 0 1) (, 1 0) (, 1 1)]]
|
||||
['(f (, x y) (.items {"1" 1 "2" 2}) (* y 2))
|
||||
[2 4]]
|
||||
['(f x (do (setv s "x") "ab") y (do (+= s "y") "def") (+ x y s))
|
||||
["adxy" "aexy" "afxy" "bdxyy" "bexyy" "bfxyy"]]
|
||||
['(f x (range 4) :if (% x 2) (* x 2))
|
||||
[2 6]]
|
||||
['(f x "abc" :setv y (.upper x) (+ x y))
|
||||
["aA" "bB" "cC"]]
|
||||
['(f x "abc" :do (setv y (.upper x)) (+ x y))
|
||||
["aA" "bB" "cC"]]
|
||||
['(f
|
||||
x (range 3)
|
||||
y (range 3)
|
||||
:if (> y x)
|
||||
z [7 8 9]
|
||||
:setv s (+ x y z)
|
||||
:if (!= z 8)
|
||||
(, x y z s))
|
||||
[(, 0 1 7 8) (, 0 1 9 10) (, 0 2 7 9) (, 0 2 9 11)
|
||||
(, 1 2 7 10) (, 1 2 9 12)]]
|
||||
['(f
|
||||
x [0 1]
|
||||
:setv l []
|
||||
y (range 4)
|
||||
:do (.append l (, x y))
|
||||
:if (>= y 2)
|
||||
z [7 8 9]
|
||||
:if (!= z 8)
|
||||
(, x y (tuple l) z))
|
||||
[(, 0 2 (, (, 0 0) (, 0 1) (, 0 2)) 7)
|
||||
(, 0 2 (, (, 0 0) (, 0 1) (, 0 2)) 9)
|
||||
(, 0 3 (, (, 0 0) (, 0 1) (, 0 2) (, 0 3)) 7)
|
||||
(, 0 3 (, (, 0 0) (, 0 1) (, 0 2) (, 0 3)) 9)
|
||||
(, 1 2 (, (, 1 0) (, 1 1) (, 1 2)) 7)
|
||||
(, 1 2 (, (, 1 0) (, 1 1) (, 1 2)) 9)
|
||||
(, 1 3 (, (, 1 0) (, 1 1) (, 1 2) (, 1 3)) 7)
|
||||
(, 1 3 (, (, 1 0) (, 1 1) (, 1 2) (, 1 3)) 9)]]
|
||||
|
||||
['(f x (range 4) :do (unless (% x 2) (continue)) (* x 2))
|
||||
[2 6]]
|
||||
['(f x (range 4) :setv p 9 :do (unless (% x 2) (continue)) (* x 2))
|
||||
[2 6]]
|
||||
['(f x (range 20) :do (when (= x 3) (break)) (* x 2))
|
||||
[0 2 4]]
|
||||
['(f x (range 20) :setv p 9 :do (when (= x 3) (break)) (* x 2))
|
||||
[0 2 4]]
|
||||
['(f x [4 5] y (range 20) :do (when (> y 1) (break)) z [8 9] (, x y z))
|
||||
[(, 4 0 8) (, 4 0 9) (, 4 1 8) (, 4 1 9)
|
||||
(, 5 0 8) (, 5 0 9) (, 5 1 8) (, 5 1 9)]]])
|
||||
|
||||
(for [[expr answer] cases]
|
||||
; Mutate the case as appropriate for the operator before
|
||||
; evaluating it.
|
||||
(setv expr (+ (HyExpression [(HySymbol specialop)]) (cut expr 1)))
|
||||
(when (= specialop "dfor")
|
||||
(setv expr (+ (cut expr 0 -1) `([~(get expr -1) 1]))))
|
||||
(when (= specialop "for")
|
||||
(setv expr `(do
|
||||
(setv out [])
|
||||
(for [~@(cut expr 1 -1)]
|
||||
(.append out ~(get expr -1)))
|
||||
out)))
|
||||
(setv result (eval expr))
|
||||
(when (= specialop "dfor")
|
||||
(setv result (.keys result)))
|
||||
(assert (= (sorted result) answer) (str expr)))))
|
||||
|
||||
|
||||
(defn test-fors-no-loopers []
|
||||
|
||||
(setv l [])
|
||||
(for [] (.append l 1))
|
||||
(assert (= l []))
|
||||
|
||||
(assert (= (lfor 1) []))
|
||||
(assert (= (sfor 1) #{}))
|
||||
(assert (= (list (gfor 1)) []))
|
||||
(assert (= (dfor [1 2]) {})))
|
||||
|
||||
|
||||
(defn test-raise-in-comp []
|
||||
(defclass E [Exception] [])
|
||||
(setv l [])
|
||||
(import pytest)
|
||||
(with [(pytest.raises E)]
|
||||
(lfor
|
||||
x (range 10)
|
||||
:do (.append l x)
|
||||
:do (when (= x 5)
|
||||
(raise (E)))
|
||||
x))
|
||||
(assert (= l [0 1 2 3 4 5])))
|
||||
|
||||
|
||||
(defn test-scoping []
|
||||
|
||||
(setv x 0)
|
||||
(for [x [1 2 3]])
|
||||
(assert (= x 3))
|
||||
|
||||
; An `lfor` that gets compiled to a real comprehension
|
||||
(setv x 0)
|
||||
(assert (= (lfor x [1 2 3] (inc x)) [2 3 4]))
|
||||
(assert (= x (if PY3 0 3)))
|
||||
; Python 2 list comprehensions leak their variables.
|
||||
|
||||
; An `lfor` that gets compiled to a loop
|
||||
(setv x 0 l [])
|
||||
(assert (= (lfor x [4 5 6] :do (.append l 1) (inc x)) [5 6 7]))
|
||||
(assert (= l [1 1 1]))
|
||||
(assert (= x 0))
|
||||
|
||||
; An `sfor` that gets compiled to a real comprehension
|
||||
(setv x 0)
|
||||
(assert (= (sfor x [1 2 3] (inc x)) #{2 3 4}))
|
||||
(assert (= x 0)))
|
||||
|
||||
|
||||
(defn test-for-loop []
|
||||
"NATIVE: test for loops"
|
||||
(setv count1 0 count2 0)
|
||||
(for [x [1 2 3 4 5]]
|
||||
(setv count1 (+ count1 x))
|
||||
(setv count2 (+ count2 x)))
|
||||
(assert (= count1 15))
|
||||
(assert (= count2 15))
|
||||
(setv count 0)
|
||||
(for [x [1 2 3 4 5]
|
||||
y [1 2 3 4 5]]
|
||||
(setv count (+ count x y))
|
||||
(else
|
||||
(+= count 1)))
|
||||
(assert (= count 151))
|
||||
|
||||
(setv count 0)
|
||||
; multiple statements in the else branch should work
|
||||
(for [x [1 2 3 4 5]
|
||||
y [1 2 3 4 5]]
|
||||
(setv count (+ count x y))
|
||||
(else
|
||||
(+= count 1)
|
||||
(+= count 10)))
|
||||
(assert (= count 161))
|
||||
|
||||
; don't be fooled by constructs that look like else
|
||||
(setv s "")
|
||||
(setv else True)
|
||||
(for [x "abcde"]
|
||||
(+= s x)
|
||||
[else (+= s "_")])
|
||||
(assert (= s "a_b_c_d_e_"))
|
||||
|
||||
(setv s "")
|
||||
(with [(pytest.raises TypeError)]
|
||||
(for [x "abcde"]
|
||||
(+= s x)
|
||||
("else" (+= s "z"))))
|
||||
(assert (= s "az"))
|
||||
|
||||
(assert (= (list ((fn [] (for [x [[1] [2 3]] y x] (yield y)))))
|
||||
(lfor x [[1] [2 3]] y x y)))
|
||||
(assert (= (list ((fn [] (for [x [[1] [2 3]] y x z (range 5)] (yield z)))))
|
||||
(lfor x [[1] [2 3]] y x z (range 5) z))))
|
||||
|
||||
|
||||
(defn test-nasty-for-nesting []
|
||||
"NATIVE: test nesting for loops harder"
|
||||
;; This test and feature is dedicated to @nedbat.
|
||||
|
||||
;; OK. This next test will ensure that we call the else branch exactly
|
||||
;; once.
|
||||
(setv flag 0)
|
||||
(for [x (range 2)
|
||||
y (range 2)]
|
||||
(+ 1 1)
|
||||
(else (setv flag (+ flag 2))))
|
||||
(assert (= flag 2)))
|
||||
|
||||
|
||||
(defn test-empty-for []
|
||||
|
||||
(setv l [])
|
||||
(defn f []
|
||||
(for [x (range 3)]
|
||||
(.append l "a")
|
||||
(yield x)))
|
||||
(for [x (f)])
|
||||
(assert (= l ["a" "a" "a"]))
|
||||
|
||||
(setv l [])
|
||||
(for [x (f)]
|
||||
(else (.append l "z")))
|
||||
(assert (= l ["a" "a" "a" "z"])))
|
||||
@ -144,11 +144,11 @@
|
||||
|
||||
(setv x {1 2 3 [4 5] 6 7})
|
||||
(setv (get x 3 1) x)
|
||||
(assert (in (hy-repr x) (list-comp
|
||||
(assert (in (hy-repr x) (lfor
|
||||
; The ordering of a dictionary isn't guaranteed, so we need
|
||||
; to check for all possible orderings.
|
||||
(+ "{" (.join " " p) "}")
|
||||
[p (permutations ["1 2" "3 [4 {...}]" "6 7"])]))))
|
||||
p (permutations ["1 2" "3 [4 {...}]" "6 7"])
|
||||
(+ "{" (.join " " p) "}")))))
|
||||
|
||||
(defn test-matchobject []
|
||||
(import re)
|
||||
|
||||
@ -46,7 +46,7 @@
|
||||
(assert (= (macroexpand-all '(with [a 1]))
|
||||
'(with* [a 1] (do))))
|
||||
(assert (= (macroexpand-all '(with [a 1 b 2 c 3] (for [d c] foo)))
|
||||
'(with* [a 1] (with* [b 2] (with* [c 3] (do (for* [d c] (do foo))))))))
|
||||
'(with* [a 1] (with* [b 2] (with* [c 3] (do (for [d c] foo)))))))
|
||||
(assert (= (macroexpand-all '(with [a 1]
|
||||
'(with [b 2])
|
||||
`(with [c 3]
|
||||
|
||||
@ -176,110 +176,6 @@
|
||||
(with [(pytest.raises TypeError)] ("when" 1 2))) ; A macro
|
||||
|
||||
|
||||
(defn test-for-loop []
|
||||
"NATIVE: test for loops"
|
||||
(setv count1 0 count2 0)
|
||||
(for [x [1 2 3 4 5]]
|
||||
(setv count1 (+ count1 x))
|
||||
(setv count2 (+ count2 x)))
|
||||
(assert (= count1 15))
|
||||
(assert (= count2 15))
|
||||
(setv count 0)
|
||||
(for [x [1 2 3 4 5]
|
||||
y [1 2 3 4 5]]
|
||||
(setv count (+ count x y))
|
||||
(else
|
||||
(+= count 1)))
|
||||
(assert (= count 151))
|
||||
|
||||
(setv count 0)
|
||||
; multiple statements in the else branch should work
|
||||
(for [x [1 2 3 4 5]
|
||||
y [1 2 3 4 5]]
|
||||
(setv count (+ count x y))
|
||||
(else
|
||||
(+= count 1)
|
||||
(+= count 10)))
|
||||
(assert (= count 161))
|
||||
|
||||
; don't be fooled by constructs that look like else
|
||||
(setv s "")
|
||||
(setv else True)
|
||||
(for [x "abcde"]
|
||||
(+= s x)
|
||||
[else (+= s "_")])
|
||||
(assert (= s "a_b_c_d_e_"))
|
||||
|
||||
(setv s "")
|
||||
(setv else True)
|
||||
(with [(pytest.raises TypeError)]
|
||||
(for [x "abcde"]
|
||||
(+= s x)
|
||||
("else" (+= s "z"))))
|
||||
(assert (= s "az"))
|
||||
|
||||
(assert (= (list ((fn [] (for [x [[1] [2 3]] y x] (yield y)))))
|
||||
(list-comp y [x [[1] [2 3]] y x])))
|
||||
(assert (= (list ((fn [] (for [x [[1] [2 3]] y x z (range 5)] (yield z)))))
|
||||
(list-comp z [x [[1] [2 3]] y x z (range 5)])))
|
||||
|
||||
(setv l [])
|
||||
(defn f []
|
||||
(for [x [4 9 2]]
|
||||
(.append l (* 10 x))
|
||||
(yield x)))
|
||||
(for [_ (f)])
|
||||
(assert (= l [40 90 20])))
|
||||
|
||||
|
||||
(defn test-nasty-for-nesting []
|
||||
"NATIVE: test nesting for loops harder"
|
||||
;; This test and feature is dedicated to @nedbat.
|
||||
|
||||
;; let's ensure empty iterating is an implicit do
|
||||
(setv t 0)
|
||||
(for [] (setv t 1))
|
||||
(assert (= t 1))
|
||||
|
||||
;; OK. This first test will ensure that the else is hooked up to the
|
||||
;; for when we break out of it.
|
||||
(for [x (range 2)
|
||||
y (range 2)]
|
||||
(break)
|
||||
(else (raise Exception)))
|
||||
|
||||
;; OK. This next test will ensure that the else is hooked up to the
|
||||
;; "inner" iteration
|
||||
(for [x (range 2)
|
||||
y (range 2)]
|
||||
(if (= y 1) (break))
|
||||
(else (raise Exception)))
|
||||
|
||||
;; OK. This next test will ensure that the else is hooked up to the
|
||||
;; "outer" iteration
|
||||
(for [x (range 2)
|
||||
y (range 2)]
|
||||
(if (= x 1) (break))
|
||||
(else (raise Exception)))
|
||||
|
||||
;; OK. This next test will ensure that we call the else branch exactly
|
||||
;; once.
|
||||
(setv flag 0)
|
||||
(for [x (range 2)
|
||||
y (range 2)]
|
||||
(+ 1 1)
|
||||
(else (setv flag (+ flag 2))))
|
||||
(assert (= flag 2))
|
||||
|
||||
(setv l [])
|
||||
(defn f []
|
||||
(for [x [4 9 2]]
|
||||
(.append l (* 10 x))
|
||||
(yield x)))
|
||||
(for [_ (f)])
|
||||
(assert (= l [40 90 20])))
|
||||
|
||||
|
||||
(defn test-while-loop []
|
||||
"NATIVE: test while loops?"
|
||||
(setv count 5)
|
||||
@ -934,64 +830,18 @@
|
||||
(defn test-for-else []
|
||||
"NATIVE: test for else"
|
||||
(setv x 0)
|
||||
(for* [a [1 2]]
|
||||
(for [a [1 2]]
|
||||
(setv x (+ x a))
|
||||
(else (setv x (+ x 50))))
|
||||
(assert (= x 53))
|
||||
|
||||
(setv x 0)
|
||||
(for* [a [1 2]]
|
||||
(for [a [1 2]]
|
||||
(setv x (+ x a))
|
||||
(else))
|
||||
(assert (= x 3)))
|
||||
|
||||
|
||||
(defn test-list-comprehensions []
|
||||
"NATIVE: test list comprehensions"
|
||||
(assert (= (list-comp (* x 2) [x (range 2)]) [0 2]))
|
||||
(assert (= (list-comp (* x 2) [x (range 4)] (% x 2)) [2 6]))
|
||||
(assert (= (sorted (list-comp (* y 2) [(, x y) (.items {"1" 1 "2" 2})]))
|
||||
[2 4]))
|
||||
(assert (= (list-comp (, x y) [x (range 2) y (range 2)])
|
||||
[(, 0 0) (, 0 1) (, 1 0) (, 1 1)]))
|
||||
(assert (= (list-comp j [j [1 2]]) [1 2])))
|
||||
|
||||
|
||||
(defn test-set-comprehensions []
|
||||
"NATIVE: test set comprehensions"
|
||||
(assert (instance? set (set-comp x [x (range 2)])))
|
||||
(assert (= (set-comp (* x 2) [x (range 2)]) (set [0 2])))
|
||||
(assert (= (set-comp (* x 2) [x (range 4)] (% x 2)) (set [2 6])))
|
||||
(assert (= (set-comp (* y 2) [(, x y) (.items {"1" 1 "2" 2})])
|
||||
(set [2 4])))
|
||||
(assert (= (set-comp (, x y) [x (range 2) y (range 2)])
|
||||
(set [(, 0 0) (, 0 1) (, 1 0) (, 1 1)])))
|
||||
(assert (= (set-comp j [j [1 2]]) (set [1 2]))))
|
||||
|
||||
|
||||
(defn test-dict-comprehensions []
|
||||
"NATIVE: test dict comprehensions"
|
||||
(assert (instance? dict (dict-comp x x [x (range 2)])))
|
||||
(assert (= (dict-comp x (* x 2) [x (range 2)]) {1 2 0 0}))
|
||||
(assert (= (dict-comp x (* x 2) [x (range 4)] (% x 2)) {3 6 1 2}))
|
||||
(assert (= (dict-comp x (* y 2) [(, x y) (.items {"1" 1 "2" 2})])
|
||||
{"2" 4 "1" 2}))
|
||||
(assert (= (dict-comp (, x y) (+ x y) [x (range 2) y (range 2)])
|
||||
{(, 0 0) 0 (, 1 0) 1 (, 0 1) 1 (, 1 1) 2})))
|
||||
|
||||
|
||||
(defn test-generator-expressions []
|
||||
"NATIVE: test generator expressions"
|
||||
(assert (not (instance? list (genexpr x [x (range 2)]))))
|
||||
(assert (= (list (genexpr (* x 2) [x (range 2)])) [0 2]))
|
||||
(assert (= (list (genexpr (* x 2) [x (range 4)] (% x 2))) [2 6]))
|
||||
(assert (= (list (sorted (genexpr (* y 2) [(, x y) (.items {"1" 1 "2" 2})])))
|
||||
[2 4]))
|
||||
(assert (= (list (genexpr (, x y) [x (range 2) y (range 2)]))
|
||||
[(, 0 0) (, 0 1) (, 1 0) (, 1 1)]))
|
||||
(assert (= (list (genexpr j [j [1 2]])) [1 2])))
|
||||
|
||||
|
||||
(defn test-defn-order []
|
||||
"NATIVE: test defn evaluation order"
|
||||
(setv acc [])
|
||||
|
||||
@ -101,7 +101,7 @@
|
||||
(defn test-midtree-yield-in-for []
|
||||
"NATIVE: test yielding in a for with a return"
|
||||
(defn kruft-in-for []
|
||||
(for* [i (range 5)]
|
||||
(for [i (range 5)]
|
||||
(yield i))
|
||||
(+ 1 2)))
|
||||
|
||||
@ -117,7 +117,7 @@
|
||||
(defn test-multi-yield []
|
||||
"NATIVE: testing multiple yields"
|
||||
(defn multi-yield []
|
||||
(for* [i (range 3)]
|
||||
(for [i (range 3)]
|
||||
(yield i))
|
||||
(yield "a")
|
||||
(yield "end"))
|
||||
|
||||
@ -13,7 +13,7 @@
|
||||
(.run_until_complete (get-event-loop) (coro)))
|
||||
|
||||
|
||||
(defn test-for/a []
|
||||
(defn test-for-async []
|
||||
(defn/a numbers []
|
||||
(for [i [1 2]]
|
||||
(yield i)))
|
||||
@ -21,11 +21,11 @@
|
||||
(run-coroutine
|
||||
(fn/a []
|
||||
(setv x 0)
|
||||
(for/a [a (numbers)]
|
||||
(for [:async a (numbers)]
|
||||
(setv x (+ x a)))
|
||||
(assert (= x 3)))))
|
||||
|
||||
(defn test-for/a-else []
|
||||
(defn test-for-async-else []
|
||||
(defn/a numbers []
|
||||
(for [i [1 2]]
|
||||
(yield i)))
|
||||
@ -33,7 +33,7 @@
|
||||
(run-coroutine
|
||||
(fn/a []
|
||||
(setv x 0)
|
||||
(for/a [a (numbers)]
|
||||
(for [:async a (numbers)]
|
||||
(setv x (+ x a))
|
||||
(else (setv x (+ x 50))))
|
||||
(assert (= x 53)))))
|
||||
|
||||
@ -49,7 +49,7 @@
|
||||
(defn test-yield-from []
|
||||
"NATIVE: testing yield from"
|
||||
(defn yield-from-test []
|
||||
(for* [i (range 3)]
|
||||
(for [i (range 3)]
|
||||
(yield i))
|
||||
(yield-from [1 2 3]))
|
||||
(assert (= (list (yield-from-test)) [0 1 2 1 2 3])))
|
||||
@ -63,7 +63,7 @@
|
||||
(yield 3)
|
||||
(assert 0))
|
||||
(defn yield-from-test []
|
||||
(for* [i (range 3)]
|
||||
(for [i (range 3)]
|
||||
(yield i))
|
||||
(try
|
||||
(yield-from (yield-from-subgenerator-test))
|
||||
|
||||
@ -110,7 +110,7 @@
|
||||
((wraps func)
|
||||
(fn [&rest args &kwargs kwargs]
|
||||
(func #* (map inc args)
|
||||
#** (dict-comp k (inc v) [[k v] (.items kwargs)])))))
|
||||
#** (dfor [k v] (.items kwargs) [k (inc v)])))))
|
||||
|
||||
#@(increment-arguments
|
||||
(defn foo [&rest args &kwargs kwargs]
|
||||
|
||||
@ -29,10 +29,10 @@ Call me Ishmael. Some years ago—never mind how long precisely—having little
|
||||
(setv myset #{4 5 6})
|
||||
(setv mydict {7 8 9 900 10 15})
|
||||
|
||||
(setv mylistcomp (list-comp x [x (range 10)] (% x 2)))
|
||||
(setv mysetcomp (set-comp x [x (range 5)] (not (% x 2))))
|
||||
(setv mydictcomp (dict-comp k (.upper k) [k "abcde"] (!= k "c")))
|
||||
(setv mygenexpr (genexpr x [x (cycle [1 2 3])] (!= x 2)))
|
||||
(setv mylistcomp (lfor x (range 10) :if (% x 2) x))
|
||||
(setv mysetcomp (sfor x (range 5) :if (not (% x 2)) x))
|
||||
(setv mydictcomp (dfor k "abcde" :if (!= k "c") [k (.upper k)]))
|
||||
(setv mygenexpr (gfor x (cycle [1 2 3]) :if (!= x 2) x))
|
||||
|
||||
(setv attr-ref str.upper)
|
||||
(setv subscript (get "hello" 2))
|
||||
|
||||
Loading…
Reference in New Issue
Block a user