Yaltik/hy
2
0
Fork 0
Code Pull Requests Releases Activity

Merge pull request #1626 from Kodiologist/newcomp3

Browse Source 
Fancier `for` and comprehensions
This commit is contained in:
Kodi Arfer 2018-06-16 14:16:41 -07:00 committed by GitHub
commit f22195dfbc
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
24 changed files with 616 additions and 453 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
NEWS.rst
View File

@ -14,6 +14,7 @@ Removals
* Macros `ap-pipe` and `ap-compose` have been removed. * Macros `ap-pipe` and `ap-compose` have been removed.
Anaphoric macros do not work well with point-free style programming, Anaphoric macros do not work well with point-free style programming,
in which case both threading macros and `comp` are more adequate. in which case both threading macros and `comp` are more adequate.
* `for/a` has been removed. Use `(for [:async ...] ...)` instead.
Other Breaking Changes Other Breaking Changes
------------------------------ ------------------------------
@ -30,6 +31,10 @@ Other Breaking Changes
* Non-shadow unary `=`, `is`, `<`, etc. now evaluate their argument * Non-shadow unary `=`, `is`, `<`, etc. now evaluate their argument
instead of ignoring it. This change increases consistency a bit instead of ignoring it. This change increases consistency a bit
and makes accidental unary uses easier to notice. and makes accidental unary uses easier to notice.
* `list-comp`, `set-comp`, `dict-comp`, and `genexpr` have been replaced
by `lfor`, `sfor`, `dfor`, and `gfor`, respectively, which use a new
syntax and have additional features. All Python comprehensions can now
be written in Hy.
* `hy-repr` uses registered functions instead of methods * `hy-repr` uses registered functions instead of methods
* `HyKeyword` no longer inherits from the string type and has been * `HyKeyword` no longer inherits from the string type and has been
made into its own object type. made into its own object type.
@ -47,6 +52,7 @@ New Features
keyword arguments keyword arguments
* Added a command-line option `-E` per CPython * Added a command-line option `-E` per CPython
* `while` and `for` are allowed to have empty bodies * `while` and `for` are allowed to have empty bodies
* `for` supports the various new clause types offered by `lfor`
* Added a new module ``hy.model_patterns`` * Added a new module ``hy.model_patterns``
Bug Fixes Bug Fixes

2
docs/extra/reserved.rst
View File

@ -10,7 +10,7 @@ Usage: ``(names)``
This function can be used to get a list (actually, a ``frozenset``) of the This function can be used to get a list (actually, a ``frozenset``) of the
names of Hy's built-in functions, macros, and special forms. The output names of Hy's built-in functions, macros, and special forms. The output
also includes all Python reserved words. All names are in unmangled form also includes all Python reserved words. All names are in unmangled form
(e.g., ``list-comp`` rather than ``list_comp``). (e.g., ``not-in`` rather than ``not_in``).
.. code-block:: hy .. code-block:: hy

255
docs/language/api.rst
View File

@ -321,48 +321,17 @@ is only called on every other value in the list.
(side-effect2 x)) (side-effect2 x))
dict-comp
---------
``dict-comp`` is used to create dictionaries. It takes three or four parameters.
The first two parameters are for controlling the return value (key-value pair)
while the third is used to select items from a sequence. The fourth and optional
parameter can be used to filter out some of the items in the sequence based on a
conditional expression.
.. code-block:: hy
=> (dict-comp x (* x 2) [x (range 10)] (odd? x))
{1: 2, 3: 6, 9: 18, 5: 10, 7: 14}
do do
---------- ----------
``do`` is used to evaluate each of its arguments and return the ``do`` (called ``progn`` in some Lisps) takes any number of forms,
last one. Return values from every other than the last argument are discarded. evaluates them, and returns the value of the last one, or ``None`` if no
It can be used in ``list-comp`` to perform more complex logic as shown in one forms were provided.
of the following examples.
Some example usage: ::
.. code-block:: clj => (+ 1 (do (setv x (+ 1 1)) x))
3
=> (if True
... (do (print "Side effects rock!")
... (print "Yeah, really!")))
Side effects rock!
Yeah, really!
;; assuming that (side-effect) is a function that we want to call for each
;; and every value in the list, but whose return value we do not care about
=> (list-comp (do (side-effect x)
... (if (< x 5) (* 2 x)
... (* 4 x)))
... (x (range 10)))
[0, 2, 4, 6, 8, 20, 24, 28, 32, 36]
``do`` can accept any number of arguments, from 1 to n.
doc / #doc doc / #doc
@ -400,6 +369,20 @@ Gets help for macros or tag macros, respectively.
Gets help for a tag macro function available in this module. Gets help for a tag macro function available in this module.
dfor
----
``dfor`` creates a :ref:`dictionary comprehension <py:dict>`. Its syntax
is the same as that of `lfor`_ except that the final value form must be
a literal list of two elements, the first of which becomes each key and
the second of which becomes each value.
.. code-block:: hy
=> (dfor x (range 5) [x (* x 10)])
{0: 0, 1: 10, 2: 20, 3: 30, 4: 40}
setv setv
---- ----
@ -524,8 +507,8 @@ Parameters may have the following keywords in front of them:
.. code-block:: clj .. code-block:: clj
=> (defn zig-zag-sum [&rest numbers] => (defn zig-zag-sum [&rest numbers]
(setv odd-numbers (list-comp x [x numbers] (odd? x)) (setv odd-numbers (lfor x numbers :if (odd? x) x)
even-numbers (list-comp x [x numbers] (even? x))) even-numbers (lfor x numbers :if (even? x) x))
(- (sum odd-numbers) (sum even-numbers))) (- (sum odd-numbers) (sum even-numbers)))
=> (zig-zag-sum) => (zig-zag-sum)
@ -850,24 +833,39 @@ raising an exception.
for for
--- ---
``for`` is used to call a function for each element in a list or vector. ``for`` is used to evaluate some forms for each element in an iterable
The results of each call are discarded and the ``for`` expression returns object, such as a list. The return values of the forms are discarded and
``None`` instead. The example code iterates over *collection* and for each the ``for`` form returns ``None``.
*element* in *collection* calls the ``side-effect`` function with *element*
as its argument:
.. code-block:: clj ::
;; assuming that (side-effect) is a function that takes a single parameter => (for [x [1 2 3]]
(for [element collection] (side-effect element)) ... (print "iterating")
... (print x))
iterating
1
iterating
2
iterating
3
;; for can have an optional else block In its square-bracketed first argument, ``for`` allows the same types of
(for [element collection] (side-effect element) clauses as lfor_.
(else (side-effect-2)))
The optional ``else`` block is only executed if the ``for`` loop terminates ::
normally. If the execution is halted with ``break``, the ``else`` block does
not execute. => (for [x [1 2 3] :if (!= x 2) y [7 8]]
... (print x y))
1 7
1 8
3 7
3 8
Furthermore, the last argument of ``for`` can be an ``(else …)`` form.
This form is executed after the last iteration of the ``for``\'s
outermost iteration clause, but only if that outermost loop terminates
normally. If it's jumped out of with e.g. ``break``, the ``else`` is
ignored.
.. code-block:: clj .. code-block:: clj
@ -888,43 +886,6 @@ not execute.
loop finished loop finished
for/a
-----
``for/a`` behaves like ``for`` but is used to call a function for each
element generated by an asynchronous generator expression. The results
of each call are discarded and the ``for/a`` expression returns
``None`` instead.
.. code-block:: clj
;; assuming that (side-effect) is a function that takes a single parameter
(for/a [element (agen)] (side-effect element))
;; for/a can have an optional else block
(for/a [element (agen)] (side-effect element)
(else (side-effect-2)))
genexpr
-------
``genexpr`` is used to create generator expressions. It takes two or three
parameters. The first parameter is the expression controlling the return value,
while the second is used to select items from a list. The third and optional
parameter can be used to filter out some of the items in the list based on a
conditional expression. ``genexpr`` is similar to ``list-comp``, except it
returns an iterable that evaluates values one by one instead of evaluating them
immediately.
.. code-block:: hy
=> (setv collection (range 10))
=> (setv filtered (genexpr x [x collection] (even? x)))
=> (list filtered)
[0, 2, 4, 6, 8]
.. _gensym: .. _gensym:
gensym gensym
@ -977,6 +938,24 @@ successive elements in a nested structure. Example usage:
index that is out of bounds. index that is out of bounds.
gfor
----
``gfor`` creates a :ref:`generator expression <py:genexpr>`. Its syntax
is the same as that of `lfor`_. The difference is that ``gfor`` returns
an iterator, which evaluates and yields values one at a time.
::
=> (setv accum [])
=> (list (take-while
... (fn [x] (< x 5))
... (gfor x (count) :do (.append accum x) x)))
[0, 1, 2, 3, 4]
=> accum
[0, 1, 2, 3, 4, 5]
global global
------ ------
@ -1190,27 +1169,69 @@ last
6 6
list-comp lfor
--------- ----
``list-comp`` performs list comprehensions. It takes two or three parameters. The comprehension forms ``lfor``, `sfor`_, `dfor`_, `gfor`_, and `for`_
The first parameter is the expression controlling the return value, while are used to produce various kinds of loops, including Python-style
the second is used to select items from a list. The third and optional :ref:`comprehensions <py:comprehensions>`. ``lfor`` in particular
parameter can be used to filter out some of the items in the list based on a creates a list comprehension. A simple use of ``lfor`` is::
conditional expression. Some examples:
.. code-block:: clj => (lfor x (range 5) (* 2 x))
=> (setv collection (range 10))
=> (list-comp x [x collection])
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
=> (list-comp (* x 2) [x collection])
[0, 2, 4, 6, 8, 10, 12, 14, 16, 18]
=> (list-comp (* x 2) [x collection] (< x 5))
[0, 2, 4, 6, 8] [0, 2, 4, 6, 8]
``x`` is the name of a new variable, which is bound to each element of
``(range 5)``. Each such element in turn is used to evaluate the value
form ``(* 2 x)``, and the results are accumulated into a list.
Here's a more complex example::
=> (lfor
... x (range 3)
... y (range 3)
... :if (!= x y)
... :setv total (+ x y)
... [x y total])
[[0, 1, 1], [0, 2, 2], [1, 0, 1], [1, 2, 3], [2, 0, 2], [2, 1, 3]]
When there are several iteration clauses (here, the pairs of forms ``x
(range 3)`` and ``y (range 3)``), the result works like a nested loop or
Cartesian product: all combinations are considered in lexicographic
order.
The general form of ``lfor`` is::
(lfor CLAUSES VALUE)
where the ``VALUE`` is an arbitrary form that is evaluated to produce
each element of the result list, and ``CLAUSES`` is any number of
clauses. There are several types of clauses:
- Iteration clauses, which look like ``LVALUE ITERABLE``. The ``LVALUE``
is usually just a symbol, but could be something more complicated,
like ``[x y]``.
- ``:async LVALUE ITERABLE``, which is an
:ref:`asynchronous <py:async for>` form of iteration clause.
- ``:do FORM``, which simply evaluates the ``FORM``. If you use
``(continue)`` or ``(break)`` here, they will apply to the innermost
iteration clause before the ``:do``.
- ``:setv LVALUE RVALUE``, which is equivalent to ``:do (setv LVALUE
RVALUE)``.
- ``:if CONDITION``, which is equivalent to ``:do (unless CONDITION
(continue))``.
For ``lfor``, ``sfor``, ``gfor``, and ``dfor``, variables are scoped as
if the comprehension form were its own function, so variables defined by
an iteration clause or ``:setv`` are not visible outside the form. In
fact, these forms are implemented as generator functions whenever they
contain Python statements, with the attendant consequences for calling
``return``. By contrast, ``for`` shares the caller's scope.
.. note:: An exception to the above scoping rules occurs on Python 2 for
``lfor`` specifically (and not ``sfor``, ``gfor``, or ``dfor``) when
Hy can implement the ``lfor`` as a Python list comprehension. Then,
variables will leak to the surrounding scope.
nonlocal nonlocal
-------- --------
@ -1465,20 +1486,12 @@ the end of a function, put ``None`` there yourself.
=> (print (f 4)) => (print (f 4))
None None
set-comp
--------
``set-comp`` is used to create sets. It takes two or three parameters. sfor
The first parameter is for controlling the return value, while the second is ----
used to select items from a sequence. The third and optional parameter can be
used to filter out some of the items in the sequence based on a conditional
expression.
.. code-block:: hy ``sfor`` creates a set comprehension. ``(sfor CLAUSES VALUE)`` is
equivalent to ``(set (lfor CLAUSES VALUE))``. See `lfor`_.
=> (setv data [1 2 3 4 5 2 3 4 5 3 4 5])
=> (set-comp x [x data] (odd? x))
{1, 3, 5}
cut cut
@ -1944,13 +1957,13 @@ infinite series without consuming infinite amount of memory.
=> (multiply (range 5) (range 5)) => (multiply (range 5) (range 5))
<generator object multiply at 0x978d8ec> <generator object multiply at 0x978d8ec>
=> (list-comp value [value (multiply (range 10) (range 10))]) => (list (multiply (range 10) (range 10)))
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81] [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
=> (import random) => (import random)
=> (defn random-numbers [low high] => (defn random-numbers [low high]
... (while True (yield (.randint random low high)))) ... (while True (yield (.randint random low high))))
=> (list-comp x [x (take 15 (random-numbers 1 50))]) => (list (take 15 (random-numbers 1 50)))
[7, 41, 6, 22, 32, 17, 5, 38, 18, 38, 17, 14, 23, 23, 19] [7, 41, 6, 22, 32, 17, 5, 38, 18, 38, 17, 14, 23, 23, 19]

17
docs/tutorial.rst
View File

@ -378,21 +378,20 @@ In Hy, you could do these like:
.. code-block:: clj .. code-block:: clj
(setv odds-squared (setv odds-squared
(list-comp (lfor
(pow num 2) num (range 100)
(num (range 100)) :if (= (% num 2) 1)
(= (% num 2) 1))) (pow num 2)))
.. code-block:: clj .. code-block:: clj
; And, an example stolen shamelessly from a Clojure page: ; And, an example stolen shamelessly from a Clojure page:
; Let's list all the blocks of a Chessboard: ; Let's list all the blocks of a Chessboard:
(list-comp (lfor
(, x y) x (range 8)
(x (range 8) y "ABCDEFGH"
y "ABCDEFGH")) (, x y))
; [(0, 'A'), (0, 'B'), (0, 'C'), (0, 'D'), (0, 'E'), (0, 'F'), (0, 'G'), (0, 'H'), ; [(0, 'A'), (0, 'B'), (0, 'C'), (0, 'D'), (0, 'E'), (0, 'F'), (0, 'G'), (0, 'H'),
; (1, 'A'), (1, 'B'), (1, 'C'), (1, 'D'), (1, 'E'), (1, 'F'), (1, 'G'), (1, 'H'), ; (1, 'A'), (1, 'B'), (1, 'C'), (1, 'D'), (1, 'E'), (1, 'F'), (1, 'G'), (1, 'H'),

224
hy/compiler.py
View File

@ -7,7 +7,7 @@ from hy.models import (HyObject, HyExpression, HyKeyword, HyInteger, HyComplex,
HyString, HyBytes, HySymbol, HyFloat, HyList, HySet, HyString, HyBytes, HySymbol, HyFloat, HyList, HySet,
HyDict, HySequence, wrap_value) HyDict, HySequence, wrap_value)
from hy.model_patterns import (FORM, SYM, STR, sym, brackets, whole, notpexpr, from hy.model_patterns import (FORM, SYM, STR, sym, brackets, whole, notpexpr,
dolike, pexpr, times) dolike, pexpr, times, Tag, tag)
from funcparserlib.parser import some, many, oneplus, maybe, NoParseError from funcparserlib.parser import some, many, oneplus, maybe, NoParseError
from hy.errors import HyCompileError, HyTypeError from hy.errors import HyCompileError, HyTypeError
@ -111,13 +111,13 @@ def builds_model(*model_types):
# ast.Foo(..., lineno=x.lineno, col_offset=x.col_offset) # ast.Foo(..., lineno=x.lineno, col_offset=x.col_offset)
class Asty(object): class Asty(object):
def __getattr__(self, name): def __getattr__(self, name):
setattr(Asty, name, lambda self, x, **kwargs: getattr(ast, name)( setattr(Asty, name, staticmethod(lambda x, **kwargs: getattr(ast, name)(
lineno=getattr( lineno=getattr(
x, 'start_line', getattr(x, 'lineno', None)), x, 'start_line', getattr(x, 'lineno', None)),
col_offset=getattr( col_offset=getattr(
x, 'start_column', getattr(x, 'col_offset', None)), x, 'start_column', getattr(x, 'col_offset', None)),
**kwargs)) **kwargs)))
return getattr(self, name) return getattr(Asty, name)
asty = Asty() asty = Asty()
@ -180,6 +180,22 @@ class Result(object):
self.__used_expr = False self.__used_expr = False
self._expr = value self._expr = value
@property
def lineno(self):
if self._expr is not None:
return self._expr.lineno
if self.stmts:
return self.stmts[-1].lineno
return None
@property
def col_offset(self):
if self._expr is not None:
return self._expr.col_offset
if self.stmts:
return self.stmts[-1].col_offset
return None
def add_imports(self, mod, imports): def add_imports(self, mod, imports):
"""Autoimport `imports` from `mod`""" """Autoimport `imports` from `mod`"""
self.imports[mod].update(imports) self.imports[mod].update(imports)
@ -1004,40 +1020,150 @@ class HyASTCompiler(object):
return gen_res + cond, gen return gen_res + cond, gen
@special(["list-comp", "set-comp", "genexpr"], [FORM, FORM, maybe(FORM)]) _loopers = many(
def compile_comprehension(self, expr, form, expression, gen, cond): tag('setv', sym(":setv") + FORM + FORM) |
# (list-comp expr [target iter] cond?) tag('if', sym(":if") + FORM) |
tag('do', sym(":do") + FORM) |
tag('afor', sym(":async") + FORM + FORM) |
tag('for', FORM + FORM))
@special(["for"], [brackets(_loopers),
many(notpexpr("else")) + maybe(dolike("else"))])
@special(["lfor", "sfor", "gfor"], [_loopers, FORM])
@special(["dfor"], [_loopers, brackets(FORM, FORM)])
def compile_comprehension(self, expr, root, parts, final):
root = unmangle(ast_str(root))
node_class = {
"for": asty.For,
"lfor": asty.ListComp,
"dfor": asty.DictComp,
"sfor": asty.SetComp,
"gfor": asty.GeneratorExp}[root]
is_for = root == "for"
if not isinstance(gen, HyList): orel = []
raise HyTypeError(gen, "Generator expression must be a list.") if is_for:
# Get the `else`.
body, else_expr = final
if else_expr is not None:
orel.append(self._compile_branch(else_expr))
orel[0] += orel[0].expr_as_stmt()
else:
# Get the final value (and for dictionary
# comprehensions, the final key).
if node_class is asty.DictComp:
key, elt = map(self.compile, final)
else:
key = None
elt = self.compile(final)
gen_res, gen = self._compile_generator_iterables( # Compile the parts.
[gen] + ([] if cond is None else [cond])) if is_for:
parts = parts[0]
if not parts:
return Result(expr=ast.parse({
asty.For: "None",
asty.ListComp: "[]",
asty.DictComp: "{}",
asty.SetComp: "{1}.__class__()",
asty.GeneratorExp: "(_ for _ in [])"}[node_class]).body[0].value)
parts = [
Tag(p.tag, self.compile(p.value) if p.tag in ["if", "do"] else [
self._storeize(p.value[0], self.compile(p.value[0])),
self.compile(p.value[1])])
for p in parts]
if len(gen) == 0: # Produce a result.
raise HyTypeError(expr, "Generator expression cannot be empty.") if (is_for or elt.stmts or (key is not None and key.stmts) or
any(p.tag == 'do' or (p.value[1].stmts if p.tag in ("for", "afor", "setv") else p.value.stmts)
for p in parts)):
# The desired comprehension can't be expressed as a
# real Python comprehension. We'll write it as a nested
# loop in a function instead.
contains_yield = []
def f(parts):
# This function is called recursively to construct
# the nested loop.
if not parts:
if is_for:
if body:
bd = self._compile_branch(body)
if bd.contains_yield:
contains_yield.append(True)
return bd + bd.expr_as_stmt()
return Result(stmts=[asty.Pass(expr)])
if node_class is asty.DictComp:
ret = key + elt
val = asty.Tuple(
key, ctx=ast.Load(),
elts=[key.force_expr, elt.force_expr])
else:
ret = elt
val = elt.force_expr
return ret + asty.Expr(
elt, value=asty.Yield(elt, value=val))
(tagname, v), parts = parts[0], parts[1:]
if tagname in ("for", "afor"):
orelse = orel and orel.pop().stmts
node = asty.AsyncFor if tagname == "afor" else asty.For
return v[1] + node(
v[1], target=v[0], iter=v[1].force_expr, body=f(parts).stmts,
orelse=orelse)
elif tagname == "setv":
return v[1] + asty.Assign(
v[1], targets=[v[0]], value=v[1].force_expr) + f(parts)
elif tagname == "if":
return v + asty.If(
v, test=v.force_expr, body=f(parts).stmts, orelse=[])
elif tagname == "do":
return v + v.expr_as_stmt() + f(parts)
else:
raise ValueError("can't happen")
if is_for:
ret = f(parts)
ret.contains_yield = bool(contains_yield)
return ret
fname = self.get_anon_var()
# Define the generator function.
ret = Result() + asty.FunctionDef(
expr,
name=fname,
args=ast.arguments(
args=[], vararg=None, kwarg=None,
kwonlyargs=[], kw_defaults=[], defaults=[]),
body=f(parts).stmts,
decorator_list=[])
# Immediately call the new function. Unless the user asked
# for a generator, wrap the call in `[].__class__(...)` or
# `{}.__class__(...)` or `{1}.__class__(...)` to get the
# right type. We don't want to just use e.g. `list(...)`
# because the name `list` might be rebound.
return ret + Result(expr=ast.parse(
"{}({}())".format(
{asty.ListComp: "[].__class__",
asty.DictComp: "{}.__class__",
asty.SetComp: "{1}.__class__",
asty.GeneratorExp: ""}[node_class],
fname)).body[0].value)
ret = self.compile(expression) # We can produce a real comprehension.
node_class = ( generators = []
asty.ListComp if form == "list-comp" else for tagname, v in parts:
asty.SetComp if form == "set-comp" else if tagname in ("for", "afor"):
asty.GeneratorExp) generators.append(ast.comprehension(
return ret + gen_res + node_class( target=v[0], iter=v[1].expr, ifs=[],
expr, elt=ret.force_expr, generators=gen) is_async=int(tagname == "afor")))
elif tagname == "setv":
@special("dict-comp", [FORM, FORM, FORM, maybe(FORM)]) generators.append(ast.comprehension(
def compile_dict_comprehension(self, expr, root, key, value, gen, cond): target=v[0],
key = self.compile(key) iter=asty.Tuple(v[1], elts=[v[1].expr], ctx=ast.Load()),
value = self.compile(value) ifs=[], is_async=0))
elif tagname == "if":
gen_res, gen = self._compile_generator_iterables( generators[-1].ifs.append(v.expr)
[gen] + ([] if cond is None else [cond])) else:
raise ValueError("can't happen")
return key + value + gen_res + asty.DictComp( if node_class is asty.DictComp:
expr, return asty.DictComp(expr, key=key.expr, value=elt.expr, generators=generators)
key=key.force_expr, return node_class(expr, elt=elt.expr, generators=generators)
value=value.force_expr,
generators=gen)
@special(["not", "~"], [FORM]) @special(["not", "~"], [FORM])
def compile_unary_operator(self, expr, root, arg): def compile_unary_operator(self, expr, root, arg):
@ -1288,36 +1414,6 @@ class HyASTCompiler(object):
return result return result
@special(["for*", (PY35, "for/a*")],
[brackets(FORM, FORM), many(notpexpr("else")), maybe(dolike("else"))])
def compile_for_expression(self, expr, root, args, body, else_expr):
target_name, iterable = args
target = self._storeize(target_name, self.compile(target_name))
ret = Result()
orel = Result()
if else_expr is not None:
for else_body in else_expr:
orel += self.compile(else_body)
orel += orel.expr_as_stmt()
ret += self.compile(iterable)
body = self._compile_branch(body)
body += body.expr_as_stmt()
node = asty.For if root == 'for*' else asty.AsyncFor
ret += node(expr,
target=target,
iter=ret.force_expr,
body=body.stmts or [asty.Pass(expr)],
orelse=orel.stmts)
ret.contains_yield = body.contains_yield
return ret
@special(["while"], [FORM, many(notpexpr("else")), maybe(dolike("else"))]) @special(["while"], [FORM, many(notpexpr("else")), maybe(dolike("else"))])
def compile_while_expression(self, expr, root, cond, body, else_expr): def compile_while_expression(self, expr, root, cond, body, else_expr):
cond_compiled = self.compile(cond) cond_compiled = self.compile(cond)

26
hy/contrib/hy_repr.hy
View File

@ -25,9 +25,10 @@
(setv -seen (set)) (setv -seen (set))
(defn hy-repr [obj] (defn hy-repr [obj]
(setv [f placeholder] (next (setv [f placeholder] (next
(genexpr (get -registry t) (gfor
[t (. (type obj) __mro__)] t (. (type obj) __mro__)
(in t -registry)) :if (in t -registry)
(get -registry t))
[-base-repr None])) [-base-repr None]))
(global -quoting) (global -quoting)
@ -55,18 +56,18 @@
; collections.namedtuple.) ; collections.namedtuple.)
(.format "({} {})" (.format "({} {})"
(. (type x) __name__) (. (type x) __name__)
(.join " " (genexpr (+ ":" k " " (hy-repr v)) [[k v] (zip x._fields x)]))) (.join " " (gfor [k v] (zip x._fields x) (+ ":" k " " (hy-repr v)))))
; Otherwise, print it as a regular tuple. ; Otherwise, print it as a regular tuple.
(+ "(," (if x " " "") (-cat x) ")")))) (+ "(," (if x " " "") (-cat x) ")"))))
(hy-repr-register dict :placeholder "{...}" (fn [x] (hy-repr-register dict :placeholder "{...}" (fn [x]
(setv text (.join " " (genexpr (setv text (.join " " (gfor
(+ (hy-repr k) " " (hy-repr v)) [k v] (.items x)
[[k v] (.items x)]))) (+ (hy-repr k) " " (hy-repr v)))))
(+ "{" text "}"))) (+ "{" text "}")))
(hy-repr-register HyDict :placeholder "{...}" (fn [x] (hy-repr-register HyDict :placeholder "{...}" (fn [x]
(setv text (.join " " (genexpr (setv text (.join " " (gfor
(+ (hy-repr k) " " (hy-repr v)) [k v] (partition x)
[[k v] (partition x)]))) (+ (hy-repr k) " " (hy-repr v)))))
(if (% (len x) 2) (if (% (len x) 2)
(+= text (+ " " (hy-repr (get x -1))))) (+= text (+ " " (hy-repr (get x -1)))))
(+ "{" text "}"))) (+ "{" text "}")))
@ -162,5 +163,8 @@
; Call (.repr x) using the first class of x that doesn't inherit from ; Call (.repr x) using the first class of x that doesn't inherit from
; HyObject. ; HyObject.
(.__repr__ (.__repr__
(next (genexpr t [t (. (type x) __mro__)] (not (issubclass t HyObject)))) (next (gfor
t (. (type x) __mro__)
:if (not (issubclass t HyObject))
t))
x)) x))

4
hy/contrib/sequences.hy
View File

@ -11,8 +11,8 @@
--getitem-- (fn [self n] --getitem-- (fn [self n]
"get nth item of sequence" "get nth item of sequence"
(if (hasattr n "start") (if (hasattr n "start")
(genexpr (get self x) [x (range n.start n.stop (gfor x (range n.start n.stop (or n.step 1))
(or n.step 1))]) (get self x))
(do (when (neg? n) (do (when (neg? n)
; Call (len) to force the whole ; Call (len) to force the whole
; sequence to be evaluated. ; sequence to be evaluated.

2
hy/core/bootstrap.hy
View File

@ -18,7 +18,7 @@
(% "received a `%s' instead of a symbol for macro name" (% "received a `%s' instead of a symbol for macro name"
(. (type name) (. (type name)
__name__))))) __name__)))))
(for* [kw '[&kwonly &kwargs]] (for [kw '[&kwonly &kwargs]]
(if* (in kw lambda-list) (if* (in kw lambda-list)
(raise (hy.errors.HyTypeError macro-name (raise (hy.errors.HyTypeError macro-name
(% "macros cannot use %s" (% "macros cannot use %s"

18
hy/core/language.hy
View File

@ -41,7 +41,7 @@
fs (tuple rfs)) fs (tuple rfs))
(fn [&rest args &kwargs kwargs] (fn [&rest args &kwargs kwargs]
(setv res (first-f #* args #** kwargs)) (setv res (first-f #* args #** kwargs))
(for* [f fs] (for [f fs]
(setv res (f res))) (setv res (f res)))
res)))) res))))
@ -79,7 +79,7 @@ If the second argument `codegen` is true, generate python code instead."
(defn distinct [coll] (defn distinct [coll]
"Return a generator from the original collection `coll` with no duplicates." "Return a generator from the original collection `coll` with no duplicates."
(setv seen (set) citer (iter coll)) (setv seen (set) citer (iter coll))
(for* [val citer] (for [val citer]
(if (not-in val seen) (if (not-in val seen)
(do (do
(yield val) (yield val)
@ -159,7 +159,7 @@ Return series of accumulated sums (or other binary function results)."
(setv it (iter iterable) (setv it (iter iterable)
total (next it)) total (next it))
(yield total) (yield total)
(for* [element it] (for [element it]
(setv total (func total element)) (setv total (func total element))
(yield total))) (yield total)))
@ -193,7 +193,7 @@ Return series of accumulated sums (or other binary function results)."
(defn _flatten [coll result] (defn _flatten [coll result]
(if (coll? coll) (if (coll? coll)
(do (for* [b coll] (do (for [b coll]
(_flatten b result))) (_flatten b result)))
(.append result coll)) (.append result coll))
result) result)
@ -287,7 +287,7 @@ Return series of accumulated sums (or other binary function results)."
"Return a function applying each `fs` to args, collecting results in a list." "Return a function applying each `fs` to args, collecting results in a list."
(setv fs (+ (, f) fs)) (setv fs (+ (, f) fs))
(fn [&rest args &kwargs kwargs] (fn [&rest args &kwargs kwargs]
(list-comp (f #* args #** kwargs) [f fs]))) (lfor f fs (f #* args #** kwargs))))
(defn last [coll] (defn last [coll]
"Return last item from `coll`." "Return last item from `coll`."
@ -352,8 +352,8 @@ with overlap."
(setv (setv
step (or step n) step (or step n)
coll-clones (tee coll n) coll-clones (tee coll n)
slices (genexpr (islice (get coll-clones start) start None step) slices (gfor start (range n)
[start (range n)])) (islice (get coll-clones start) start None step)))
(if (is fillvalue -sentinel) (if (is fillvalue -sentinel)
(zip #* slices) (zip #* slices)
(zip-longest #* slices :fillvalue fillvalue))) (zip-longest #* slices :fillvalue fillvalue)))
@ -402,9 +402,9 @@ Raises ValueError for (not (pos? n))."
(if (not (pos? n)) (if (not (pos? n))
(raise (ValueError "n must be positive"))) (raise (ValueError "n must be positive")))
(setv citer (iter coll) skip (dec n)) (setv citer (iter coll) skip (dec n))
(for* [val citer] (for [val citer]
(yield val) (yield val)
(for* [_ (range skip)] (for [_ (range skip)]
(try (try
(next citer) (next citer)
(except [StopIteration] (except [StopIteration]

51
hy/core/macros.hy
View File

@ -38,9 +38,9 @@ be associated in pairs."
`(setv ~@(+ (if other-kvs `(setv ~@(+ (if other-kvs
[c coll] [c coll]
[]) [])
#* (genexpr [`(get ~c ~k) v] #* (gfor [k v] (partition (+ (, k1 v1)
[[k v] (partition (+ (, k1 v1) other-kvs))
other-kvs))])))) [`(get ~c ~k) v]))))
(defn _with [node args body] (defn _with [node args body]
@ -101,50 +101,20 @@ used as the result."
(setv root (check-branch branch)) (setv root (check-branch branch))
(setv latest-branch root) (setv latest-branch root)
(for* [branch branches] (for [branch branches]
(setv cur-branch (check-branch branch)) (setv cur-branch (check-branch branch))
(.append latest-branch cur-branch) (.append latest-branch cur-branch)
(setv latest-branch cur-branch)) (setv latest-branch cur-branch))
root))) root)))
(defn _for [node args body]
(setv body (list body))
(setv belse (if (and body (isinstance (get body -1) HyExpression) (= (get body -1 0) "else"))
[(body.pop)]
[]))
(if
(odd? (len args)) (macro-error args "`for' requires an even number of args.")
(empty? args) `(do ~@body ~@belse)
(= (len args) 2) `(~node [~@args] (do ~@body) ~@belse)
(do
(setv alist (cut args 0 None 2))
`(~node [(, ~@alist) (genexpr (, ~@alist) [~@args])] (do ~@body) ~@belse))))
(defmacro for [args &rest body]
"Build a for-loop with `args` as a [element coll] bracket pair and run `body`.
Args may contain multiple pairs, in which case it executes a nested for-loop
in order of the given pairs."
(_for 'for* args body))
(defmacro for/a [args &rest body]
"Build a for/a-loop with `args` as a [element coll] bracket pair and run `body`.
Args may contain multiple pairs, in which case it executes a nested for/a-loop
in order of the given pairs."
(_for 'for/a* args body))
(defmacro -> [head &rest args] (defmacro -> [head &rest args]
"Thread `head` first through the `rest` of the forms. "Thread `head` first through the `rest` of the forms.
The result of the first threaded form is inserted into the first position of The result of the first threaded form is inserted into the first position of
the second form, the second result is inserted into the third form, and so on." the second form, the second result is inserted into the third form, and so on."
(setv ret head) (setv ret head)
(for* [node args] (for [node args]
(setv ret (if (isinstance node HyExpression) (setv ret (if (isinstance node HyExpression)
`(~(first node) ~ret ~@(rest node)) `(~(first node) ~ret ~@(rest node))
`(~node ~ret)))) `(~node ~ret))))
@ -163,13 +133,14 @@ the second form, the second result is inserted into the third form, and so on."
~@(map build-form expressions) ~@(map build-form expressions)
~f)) ~f))
(defmacro ->> [head &rest args] (defmacro ->> [head &rest args]
"Thread `head` last through the `rest` of the forms. "Thread `head` last through the `rest` of the forms.
The result of the first threaded form is inserted into the last position of The result of the first threaded form is inserted into the last position of
the second form, the second result is inserted into the third form, and so on." the second form, the second result is inserted into the third form, and so on."
(setv ret head) (setv ret head)
(for* [node args] (for [node args]
(setv ret (if (isinstance node HyExpression) (setv ret (if (isinstance node HyExpression)
`(~@node ~ret) `(~@node ~ret)
`(~node ~ret)))) `(~node ~ret))))
@ -210,7 +181,7 @@ the second form, the second result is inserted into the third form, and so on."
(defmacro with-gensyms [args &rest body] (defmacro with-gensyms [args &rest body]
"Execute `body` with `args` as bracket of names to gensym for use in macros." "Execute `body` with `args` as bracket of names to gensym for use in macros."
(setv syms []) (setv syms [])
(for* [arg args] (for [arg args]
(.extend syms [arg `(gensym '~arg)])) (.extend syms [arg `(gensym '~arg)]))
`(do `(do
(setv ~@syms) (setv ~@syms)
@ -225,7 +196,7 @@ the second form, the second result is inserted into the third form, and so on."
(.startswith x "g!"))) (.startswith x "g!")))
(flatten body)))) (flatten body))))
gensyms []) gensyms [])
(for* [sym syms] (for [sym syms]
(.extend gensyms [sym `(gensym ~(cut sym 2))])) (.extend gensyms [sym `(gensym ~(cut sym 2))]))
`(defmacro ~name [~@args] `(defmacro ~name [~@args]
(setv ~@gensyms) (setv ~@gensyms)
@ -235,8 +206,8 @@ the second form, the second result is inserted into the third form, and so on."
"Like `defmacro/g!`, with automatic once-only evaluation for 'o!' params. "Like `defmacro/g!`, with automatic once-only evaluation for 'o!' params.
Such 'o!' params are available within `body` as the equivalent 'g!' symbol." Such 'o!' params are available within `body` as the equivalent 'g!' symbol."
(setv os (list-comp s [s args] (.startswith s "o!")) (setv os (lfor s args :if (.startswith s "o!") s)
gs (list-comp (HySymbol (+ "g!" (cut s 2))) [s os])) gs (lfor s os (HySymbol (+ "g!" (cut s 2)))))
`(defmacro/g! ~name ~args `(defmacro/g! ~name ~args
`(do (setv ~@(interleave ~gs ~os)) `(do (setv ~@(interleave ~gs ~os))
~@~body))) ~@~body)))

5
hy/core/shadow.hy
View File

@ -98,8 +98,7 @@
(defn comp-op [op a1 a-rest] (defn comp-op [op a1 a-rest]
"Helper for shadow comparison operators" "Helper for shadow comparison operators"
(if a-rest (if a-rest
(reduce (fn [x y] (and x y)) (and #* (gfor (, x y) (zip (+ (, a1) a-rest) a-rest) (op x y)))
(list-comp (op x y) [(, x y) (zip (+ (, a1) a-rest) a-rest)]))
True)) True))
(defn < [a1 &rest a-rest] (defn < [a1 &rest a-rest]
"Shadowed `<` operator perform lt comparison on `a1` by each `a-rest`." "Shadowed `<` operator perform lt comparison on `a1` by each `a-rest`."
@ -161,7 +160,7 @@
(defn get [coll key1 &rest keys] (defn get [coll key1 &rest keys]
"Access item in `coll` indexed by `key1`, with optional `keys` nested-access." "Access item in `coll` indexed by `key1`, with optional `keys` nested-access."
(setv coll (get coll key1)) (setv coll (get coll key1))
(for* [k keys] (for [k keys]
(setv coll (get coll k))) (setv coll (get coll k)))
coll) coll)

22
hy/extra/anaphoric.hy
View File

@ -109,18 +109,18 @@
`%*` and `%**` name the `&rest` and `&kwargs` parameters, respectively. `%*` and `%**` name the `&rest` and `&kwargs` parameters, respectively.
Nesting of `#%` forms is not recommended." Nesting of `#%` forms is not recommended."
(setv %symbols (set-comp a (setv %symbols (sfor a (flatten [expr])
[a (flatten [expr])] :if (and (symbol? a)
(and (symbol? a) (.startswith a '%))
(.startswith a '%)))) a))
`(fn [;; generate all %i symbols up to the maximum found in expr `(fn [;; generate all %i symbols up to the maximum found in expr
~@(genexpr (HySymbol (+ "%" (str i))) ~@(gfor i (range 1 (-> (lfor a %symbols
[i (range 1 (-> (list-comp (int (cut a 1)) :if (.isdigit (cut a 1))
[a %symbols] (int (cut a 1)))
(.isdigit (cut a 1))) (or (, 0))
(or (, 0)) max
max inc))
inc))]) (HySymbol (+ "%" (str i))))
;; generate the &rest parameter only if '%* is present in expr ;; generate the &rest parameter only if '%* is present in expr
~@(if (in '%* %symbols) ~@(if (in '%* %symbols)
'(&rest %*)) '(&rest %*))

9
hy/model_patterns.py
View File

@ -9,6 +9,7 @@ from funcparserlib.parser import (
some, skip, many, finished, a, Parser, NoParseError, State) some, skip, many, finished, a, Parser, NoParseError, State)
from functools import reduce from functools import reduce
from itertools import repeat from itertools import repeat
from collections import namedtuple
from operator import add from operator import add
from math import isinf from math import isinf
@ -74,3 +75,11 @@ def times(lo, hi, parser):
end = e.state.max end = e.state.max
return result, State(s.pos, end) return result, State(s.pos, end)
return f return f
Tag = namedtuple('Tag', ['tag', 'value'])
def tag(tag_name, parser):
"""Matches the given parser and produces a named tuple `(Tag tag value)`
with `tag` set to the given tag name and `value` set to the parser's
value."""
return parser >> (lambda x: Tag(tag_name, x))

12
tests/compilers/test_ast.py
View File

@ -531,9 +531,7 @@ def test_for_compile_error():
can_compile("(fn [] (for)))") can_compile("(fn [] (for)))")
assert excinfo.value.message == "Ran into a RPAREN where it wasn't expected." assert excinfo.value.message == "Ran into a RPAREN where it wasn't expected."
with pytest.raises(HyTypeError) as excinfo: cant_compile("(fn [] (for [x] x))")
can_compile("(fn [] (for [x] x))")
assert excinfo.value.message == "`for' requires an even number of args."
def test_attribute_access(): def test_attribute_access():
@ -556,14 +554,6 @@ def test_attribute_empty():
cant_compile('[2].foo') cant_compile('[2].foo')
def test_invalid_list_comprehension():
"""Ensure that invalid list comprehensions do not break the compiler"""
cant_compile("(genexpr x [])")
cant_compile("(genexpr [x [1 2 3 4]] x)")
cant_compile("(list-comp None [])")
cant_compile("(list-comp [x [1 2 3]] x)")
def test_bad_setv(): def test_bad_setv():
"""Ensure setv handles error cases""" """Ensure setv handles error cases"""
cant_compile("(setv (a b) [1 2])") cant_compile("(setv (a b) [1 2])")

2
tests/importer/test_importer.py
View File

@ -84,5 +84,5 @@ def test_eval():
assert eval_str('(.strip " fooooo ")') == 'fooooo' assert eval_str('(.strip " fooooo ")') == 'fooooo'
assert eval_str( assert eval_str(
'(if True "this is if true" "this is if false")') == "this is if true" '(if True "this is if true" "this is if false")') == "this is if true"
assert eval_str('(list-comp (pow num 2) [num (range 100)] (= (% num 2) 1))') == [ assert eval_str('(lfor num (range 100) :if (= (% num 2) 1) (pow num 2))') == [
pow(num, 2) for num in range(100) if num % 2 == 1] pow(num, 2) for num in range(100) if num % 2 == 1]

226
tests/native_tests/comprehensions.hy Normal file
View File

@ -0,0 +1,226 @@
(import
types
pytest
[hy._compat [PY3]])
(defn test-comprehension-types []
; Forms that get compiled to real comprehensions
(assert (is (type (lfor x "abc" x)) list))
(assert (is (type (sfor x "abc" x)) set))
(assert (is (type (dfor x "abc" [x x])) dict))
(assert (is (type (gfor x "abc" x)) types.GeneratorType))
; Forms that get compiled to loops
(assert (is (type (lfor x "abc" :do (setv y 1) x)) list))
(assert (is (type (sfor x "abc" :do (setv y 1) x)) set))
(assert (is (type (dfor x "abc" :do (setv y 1) [x x])) dict))
(assert (is (type (gfor x "abc" :do (setv y 1) x)) types.GeneratorType)))
#@ ((pytest.mark.parametrize "specialop" ["for" "lfor" "sfor" "gfor" "dfor"])
(defn test-fors [specialop]
(setv cases [
['(f x [] x)
[]]
['(f j [1 2 3] j)
[1 2 3]]
['(f x (range 3) (* x 2))
[0 2 4]]
['(f x (range 2) y (range 2) (, x y))
[(, 0 0) (, 0 1) (, 1 0) (, 1 1)]]
['(f (, x y) (.items {"1" 1 "2" 2}) (* y 2))
[2 4]]
['(f x (do (setv s "x") "ab") y (do (+= s "y") "def") (+ x y s))
["adxy" "aexy" "afxy" "bdxyy" "bexyy" "bfxyy"]]
['(f x (range 4) :if (% x 2) (* x 2))
[2 6]]
['(f x "abc" :setv y (.upper x) (+ x y))
["aA" "bB" "cC"]]
['(f x "abc" :do (setv y (.upper x)) (+ x y))
["aA" "bB" "cC"]]
['(f
x (range 3)
y (range 3)
:if (> y x)
z [7 8 9]
:setv s (+ x y z)
:if (!= z 8)
(, x y z s))
[(, 0 1 7 8) (, 0 1 9 10) (, 0 2 7 9) (, 0 2 9 11)
(, 1 2 7 10) (, 1 2 9 12)]]
['(f
x [0 1]
:setv l []
y (range 4)
:do (.append l (, x y))
:if (>= y 2)
z [7 8 9]
:if (!= z 8)
(, x y (tuple l) z))
[(, 0 2 (, (, 0 0) (, 0 1) (, 0 2)) 7)
(, 0 2 (, (, 0 0) (, 0 1) (, 0 2)) 9)
(, 0 3 (, (, 0 0) (, 0 1) (, 0 2) (, 0 3)) 7)
(, 0 3 (, (, 0 0) (, 0 1) (, 0 2) (, 0 3)) 9)
(, 1 2 (, (, 1 0) (, 1 1) (, 1 2)) 7)
(, 1 2 (, (, 1 0) (, 1 1) (, 1 2)) 9)
(, 1 3 (, (, 1 0) (, 1 1) (, 1 2) (, 1 3)) 7)
(, 1 3 (, (, 1 0) (, 1 1) (, 1 2) (, 1 3)) 9)]]
['(f x (range 4) :do (unless (% x 2) (continue)) (* x 2))
[2 6]]
['(f x (range 4) :setv p 9 :do (unless (% x 2) (continue)) (* x 2))
[2 6]]
['(f x (range 20) :do (when (= x 3) (break)) (* x 2))
[0 2 4]]
['(f x (range 20) :setv p 9 :do (when (= x 3) (break)) (* x 2))
[0 2 4]]
['(f x [4 5] y (range 20) :do (when (> y 1) (break)) z [8 9] (, x y z))
[(, 4 0 8) (, 4 0 9) (, 4 1 8) (, 4 1 9)
(, 5 0 8) (, 5 0 9) (, 5 1 8) (, 5 1 9)]]])
(for [[expr answer] cases]
; Mutate the case as appropriate for the operator before
; evaluating it.
(setv expr (+ (HyExpression [(HySymbol specialop)]) (cut expr 1)))
(when (= specialop "dfor")
(setv expr (+ (cut expr 0 -1) `([~(get expr -1) 1]))))
(when (= specialop "for")
(setv expr `(do
(setv out [])
(for [~@(cut expr 1 -1)]
(.append out ~(get expr -1)))
out)))
(setv result (eval expr))
(when (= specialop "dfor")
(setv result (.keys result)))
(assert (= (sorted result) answer) (str expr)))))
(defn test-fors-no-loopers []
(setv l [])
(for [] (.append l 1))
(assert (= l []))
(assert (= (lfor 1) []))
(assert (= (sfor 1) #{}))
(assert (= (list (gfor 1)) []))
(assert (= (dfor [1 2]) {})))
(defn test-raise-in-comp []
(defclass E [Exception] [])
(setv l [])
(import pytest)
(with [(pytest.raises E)]
(lfor
x (range 10)
:do (.append l x)
:do (when (= x 5)
(raise (E)))
x))
(assert (= l [0 1 2 3 4 5])))
(defn test-scoping []
(setv x 0)
(for [x [1 2 3]])
(assert (= x 3))
; An `lfor` that gets compiled to a real comprehension
(setv x 0)
(assert (= (lfor x [1 2 3] (inc x)) [2 3 4]))
(assert (= x (if PY3 0 3)))
; Python 2 list comprehensions leak their variables.
; An `lfor` that gets compiled to a loop
(setv x 0 l [])
(assert (= (lfor x [4 5 6] :do (.append l 1) (inc x)) [5 6 7]))
(assert (= l [1 1 1]))
(assert (= x 0))
; An `sfor` that gets compiled to a real comprehension
(setv x 0)
(assert (= (sfor x [1 2 3] (inc x)) #{2 3 4}))
(assert (= x 0)))
(defn test-for-loop []
"NATIVE: test for loops"
(setv count1 0 count2 0)
(for [x [1 2 3 4 5]]
(setv count1 (+ count1 x))
(setv count2 (+ count2 x)))
(assert (= count1 15))
(assert (= count2 15))
(setv count 0)
(for [x [1 2 3 4 5]
y [1 2 3 4 5]]
(setv count (+ count x y))
(else
(+= count 1)))
(assert (= count 151))
(setv count 0)
; multiple statements in the else branch should work
(for [x [1 2 3 4 5]
y [1 2 3 4 5]]
(setv count (+ count x y))
(else
(+= count 1)
(+= count 10)))
(assert (= count 161))
; don't be fooled by constructs that look like else
(setv s "")
(setv else True)
(for [x "abcde"]
(+= s x)
[else (+= s "_")])
(assert (= s "a_b_c_d_e_"))
(setv s "")
(with [(pytest.raises TypeError)]
(for [x "abcde"]
(+= s x)
("else" (+= s "z"))))
(assert (= s "az"))
(assert (= (list ((fn [] (for [x [[1] [2 3]] y x] (yield y)))))
(lfor x [[1] [2 3]] y x y)))
(assert (= (list ((fn [] (for [x [[1] [2 3]] y x z (range 5)] (yield z)))))
(lfor x [[1] [2 3]] y x z (range 5) z))))
(defn test-nasty-for-nesting []
"NATIVE: test nesting for loops harder"
;; This test and feature is dedicated to @nedbat.
;; OK. This next test will ensure that we call the else branch exactly
;; once.
(setv flag 0)
(for [x (range 2)
y (range 2)]
(+ 1 1)
(else (setv flag (+ flag 2))))
(assert (= flag 2)))
(defn test-empty-for []
(setv l [])
(defn f []
(for [x (range 3)]
(.append l "a")
(yield x)))
(for [x (f)])
(assert (= l ["a" "a" "a"]))
(setv l [])
(for [x (f)]
(else (.append l "z")))
(assert (= l ["a" "a" "a" "z"])))

6
tests/native_tests/contrib/hy_repr.hy
View File

@ -144,11 +144,11 @@
(setv x {1 2 3 [4 5] 6 7}) (setv x {1 2 3 [4 5] 6 7})
(setv (get x 3 1) x) (setv (get x 3 1) x)
(assert (in (hy-repr x) (list-comp (assert (in (hy-repr x) (lfor
; The ordering of a dictionary isn't guaranteed, so we need ; The ordering of a dictionary isn't guaranteed, so we need
; to check for all possible orderings. ; to check for all possible orderings.
(+ "{" (.join " " p) "}") p (permutations ["1 2" "3 [4 {...}]" "6 7"])
[p (permutations ["1 2" "3 [4 {...}]" "6 7"])])))) (+ "{" (.join " " p) "}")))))
(defn test-matchobject [] (defn test-matchobject []
(import re) (import re)

2
tests/native_tests/contrib/walk.hy
View File

@ -46,7 +46,7 @@
(assert (= (macroexpand-all '(with [a 1])) (assert (= (macroexpand-all '(with [a 1]))
'(with* [a 1] (do)))) '(with* [a 1] (do))))
(assert (= (macroexpand-all '(with [a 1 b 2 c 3] (for [d c] foo))) (assert (= (macroexpand-all '(with [a 1 b 2 c 3] (for [d c] foo)))
'(with* [a 1] (with* [b 2] (with* [c 3] (do (for* [d c] (do foo)))))))) '(with* [a 1] (with* [b 2] (with* [c 3] (do (for [d c] foo)))))))
(assert (= (macroexpand-all '(with [a 1] (assert (= (macroexpand-all '(with [a 1]
'(with [b 2]) '(with [b 2])
`(with [c 3] `(with [c 3]

154
tests/native_tests/language.hy
View File

@ -176,110 +176,6 @@
(with [(pytest.raises TypeError)] ("when" 1 2))) ; A macro (with [(pytest.raises TypeError)] ("when" 1 2))) ; A macro
(defn test-for-loop []
"NATIVE: test for loops"
(setv count1 0 count2 0)
(for [x [1 2 3 4 5]]
(setv count1 (+ count1 x))
(setv count2 (+ count2 x)))
(assert (= count1 15))
(assert (= count2 15))
(setv count 0)
(for [x [1 2 3 4 5]
y [1 2 3 4 5]]
(setv count (+ count x y))
(else
(+= count 1)))
(assert (= count 151))
(setv count 0)
; multiple statements in the else branch should work
(for [x [1 2 3 4 5]
y [1 2 3 4 5]]
(setv count (+ count x y))
(else
(+= count 1)
(+= count 10)))
(assert (= count 161))
; don't be fooled by constructs that look like else
(setv s "")
(setv else True)
(for [x "abcde"]
(+= s x)
[else (+= s "_")])
(assert (= s "a_b_c_d_e_"))
(setv s "")
(setv else True)
(with [(pytest.raises TypeError)]
(for [x "abcde"]
(+= s x)
("else" (+= s "z"))))
(assert (= s "az"))
(assert (= (list ((fn [] (for [x [[1] [2 3]] y x] (yield y)))))
(list-comp y [x [[1] [2 3]] y x])))
(assert (= (list ((fn [] (for [x [[1] [2 3]] y x z (range 5)] (yield z)))))
(list-comp z [x [[1] [2 3]] y x z (range 5)])))
(setv l [])
(defn f []
(for [x [4 9 2]]
(.append l (* 10 x))
(yield x)))
(for [_ (f)])
(assert (= l [40 90 20])))
(defn test-nasty-for-nesting []
"NATIVE: test nesting for loops harder"
;; This test and feature is dedicated to @nedbat.
;; let's ensure empty iterating is an implicit do
(setv t 0)
(for [] (setv t 1))
(assert (= t 1))
;; OK. This first test will ensure that the else is hooked up to the
;; for when we break out of it.
(for [x (range 2)
y (range 2)]
(break)
(else (raise Exception)))
;; OK. This next test will ensure that the else is hooked up to the
;; "inner" iteration
(for [x (range 2)
y (range 2)]
(if (= y 1) (break))
(else (raise Exception)))
;; OK. This next test will ensure that the else is hooked up to the
;; "outer" iteration
(for [x (range 2)
y (range 2)]
(if (= x 1) (break))
(else (raise Exception)))
;; OK. This next test will ensure that we call the else branch exactly
;; once.
(setv flag 0)
(for [x (range 2)
y (range 2)]
(+ 1 1)
(else (setv flag (+ flag 2))))
(assert (= flag 2))
(setv l [])
(defn f []
(for [x [4 9 2]]
(.append l (* 10 x))
(yield x)))
(for [_ (f)])
(assert (= l [40 90 20])))
(defn test-while-loop [] (defn test-while-loop []
"NATIVE: test while loops?" "NATIVE: test while loops?"
(setv count 5) (setv count 5)
@ -934,64 +830,18 @@
(defn test-for-else [] (defn test-for-else []
"NATIVE: test for else" "NATIVE: test for else"
(setv x 0) (setv x 0)
(for* [a [1 2]] (for [a [1 2]]
(setv x (+ x a)) (setv x (+ x a))
(else (setv x (+ x 50)))) (else (setv x (+ x 50))))
(assert (= x 53)) (assert (= x 53))
(setv x 0) (setv x 0)
(for* [a [1 2]] (for [a [1 2]]
(setv x (+ x a)) (setv x (+ x a))
(else)) (else))
(assert (= x 3))) (assert (= x 3)))
(defn test-list-comprehensions []
"NATIVE: test list comprehensions"
(assert (= (list-comp (* x 2) [x (range 2)]) [0 2]))
(assert (= (list-comp (* x 2) [x (range 4)] (% x 2)) [2 6]))
(assert (= (sorted (list-comp (* y 2) [(, x y) (.items {"1" 1 "2" 2})]))
[2 4]))
(assert (= (list-comp (, x y) [x (range 2) y (range 2)])
[(, 0 0) (, 0 1) (, 1 0) (, 1 1)]))
(assert (= (list-comp j [j [1 2]]) [1 2])))
(defn test-set-comprehensions []
"NATIVE: test set comprehensions"
(assert (instance? set (set-comp x [x (range 2)])))
(assert (= (set-comp (* x 2) [x (range 2)]) (set [0 2])))
(assert (= (set-comp (* x 2) [x (range 4)] (% x 2)) (set [2 6])))
(assert (= (set-comp (* y 2) [(, x y) (.items {"1" 1 "2" 2})])
(set [2 4])))
(assert (= (set-comp (, x y) [x (range 2) y (range 2)])
(set [(, 0 0) (, 0 1) (, 1 0) (, 1 1)])))
(assert (= (set-comp j [j [1 2]]) (set [1 2]))))
(defn test-dict-comprehensions []
"NATIVE: test dict comprehensions"
(assert (instance? dict (dict-comp x x [x (range 2)])))
(assert (= (dict-comp x (* x 2) [x (range 2)]) {1 2 0 0}))
(assert (= (dict-comp x (* x 2) [x (range 4)] (% x 2)) {3 6 1 2}))
(assert (= (dict-comp x (* y 2) [(, x y) (.items {"1" 1 "2" 2})])
{"2" 4 "1" 2}))
(assert (= (dict-comp (, x y) (+ x y) [x (range 2) y (range 2)])
{(, 0 0) 0 (, 1 0) 1 (, 0 1) 1 (, 1 1) 2})))
(defn test-generator-expressions []
"NATIVE: test generator expressions"
(assert (not (instance? list (genexpr x [x (range 2)]))))
(assert (= (list (genexpr (* x 2) [x (range 2)])) [0 2]))
(assert (= (list (genexpr (* x 2) [x (range 4)] (% x 2))) [2 6]))
(assert (= (list (sorted (genexpr (* y 2) [(, x y) (.items {"1" 1 "2" 2})])))
[2 4]))
(assert (= (list (genexpr (, x y) [x (range 2) y (range 2)]))
[(, 0 0) (, 0 1) (, 1 0) (, 1 1)]))
(assert (= (list (genexpr j [j [1 2]])) [1 2])))
(defn test-defn-order [] (defn test-defn-order []
"NATIVE: test defn evaluation order" "NATIVE: test defn evaluation order"
(setv acc []) (setv acc [])

4
tests/native_tests/native_macros.hy
View File

@ -101,7 +101,7 @@
(defn test-midtree-yield-in-for [] (defn test-midtree-yield-in-for []
"NATIVE: test yielding in a for with a return" "NATIVE: test yielding in a for with a return"
(defn kruft-in-for [] (defn kruft-in-for []
(for* [i (range 5)] (for [i (range 5)]
(yield i)) (yield i))
(+ 1 2))) (+ 1 2)))
@ -117,7 +117,7 @@
(defn test-multi-yield [] (defn test-multi-yield []
"NATIVE: testing multiple yields" "NATIVE: testing multiple yields"
(defn multi-yield [] (defn multi-yield []
(for* [i (range 3)] (for [i (range 3)]
(yield i)) (yield i))
(yield "a") (yield "a")
(yield "end")) (yield "end"))

8
tests/native_tests/py36_only_tests.hy
View File

@ -13,7 +13,7 @@
(.run_until_complete (get-event-loop) (coro))) (.run_until_complete (get-event-loop) (coro)))
(defn test-for/a [] (defn test-for-async []
(defn/a numbers [] (defn/a numbers []
(for [i [1 2]] (for [i [1 2]]
(yield i))) (yield i)))
@ -21,11 +21,11 @@
(run-coroutine (run-coroutine
(fn/a [] (fn/a []
(setv x 0) (setv x 0)
(for/a [a (numbers)] (for [:async a (numbers)]
(setv x (+ x a))) (setv x (+ x a)))
(assert (= x 3))))) (assert (= x 3)))))
(defn test-for/a-else [] (defn test-for-async-else []
(defn/a numbers [] (defn/a numbers []
(for [i [1 2]] (for [i [1 2]]
(yield i))) (yield i)))
@ -33,7 +33,7 @@
(run-coroutine (run-coroutine
(fn/a [] (fn/a []
(setv x 0) (setv x 0)
(for/a [a (numbers)] (for [:async a (numbers)]
(setv x (+ x a)) (setv x (+ x a))
(else (setv x (+ x 50)))) (else (setv x (+ x 50))))
(assert (= x 53))))) (assert (= x 53)))))

4
tests/native_tests/py3_only_tests.hy
View File

@ -49,7 +49,7 @@
(defn test-yield-from [] (defn test-yield-from []
"NATIVE: testing yield from" "NATIVE: testing yield from"
(defn yield-from-test [] (defn yield-from-test []
(for* [i (range 3)] (for [i (range 3)]
(yield i)) (yield i))
(yield-from [1 2 3])) (yield-from [1 2 3]))
(assert (= (list (yield-from-test)) [0 1 2 1 2 3]))) (assert (= (list (yield-from-test)) [0 1 2 1 2 3])))
@ -63,7 +63,7 @@
(yield 3) (yield 3)
(assert 0)) (assert 0))
(defn yield-from-test [] (defn yield-from-test []
(for* [i (range 3)] (for [i (range 3)]
(yield i)) (yield i))
(try (try
(yield-from (yield-from-subgenerator-test)) (yield-from (yield-from-subgenerator-test))

2
tests/native_tests/tag_macros.hy
View File

@ -110,7 +110,7 @@
((wraps func) ((wraps func)
(fn [&rest args &kwargs kwargs] (fn [&rest args &kwargs kwargs]
(func #* (map inc args) (func #* (map inc args)
#** (dict-comp k (inc v) [[k v] (.items kwargs)]))))) #** (dfor [k v] (.items kwargs) [k (inc v)])))))
#@(increment-arguments #@(increment-arguments
(defn foo [&rest args &kwargs kwargs] (defn foo [&rest args &kwargs kwargs]

8
tests/resources/pydemo.hy
View File

@ -29,10 +29,10 @@ Call me Ishmael. Some years ago—never mind how long precisely—having little
(setv myset #{4 5 6}) (setv myset #{4 5 6})
(setv mydict {7 8 9 900 10 15}) (setv mydict {7 8 9 900 10 15})
(setv mylistcomp (list-comp x [x (range 10)] (% x 2))) (setv mylistcomp (lfor x (range 10) :if (% x 2) x))
(setv mysetcomp (set-comp x [x (range 5)] (not (% x 2)))) (setv mysetcomp (sfor x (range 5) :if (not (% x 2)) x))
(setv mydictcomp (dict-comp k (.upper k) [k "abcde"] (!= k "c"))) (setv mydictcomp (dfor k "abcde" :if (!= k "c") [k (.upper k)]))
(setv mygenexpr (genexpr x [x (cycle [1 2 3])] (!= x 2))) (setv mygenexpr (gfor x (cycle [1 2 3]) :if (!= x 2) x))
(setv attr-ref str.upper) (setv attr-ref str.upper)
(setv subscript (get "hello" 2)) (setv subscript (get "hello" 2))