Merge pull request #1886 from Kodiologist/empty-expr

Fixes for empty expressions

.travis.yml

@@ -2,12 +2,10 @@ sudo: false
 dist: xenial
 language: python
 python:
-  - "2.7"
   - "3.5"
   - "3.6"
   - "3.7"
-  - 3.8-dev
-  - pypy2.7-6.0
+  - "3.8"
   - pypy3.5-6.0
 install:
   - pip install -r requirements-travis.txt

AUTHORS

@@ -92,3 +92,10 @@
 * Brandon T. Willard <brandonwillard@gmail.com>
 * Andrew R. M. <nixy@nixy.moe>
 * Tristan de Cacqueray <tdecacqu@redhat.com>
+* Sören Tempel <soeren@soeren-tempel.net>
+* Noah Snelson <noah.snelson@protonmail.com>
+* Adam Porter <adam@alphapapa.net>
+* Gábor Lipták <gliptak@gmail.com>
+* Raymund MARTINEZ <zhaqenl@protonmail.com>
+* Zepeng Zhang <redraiment@gmail.com>
+* Joseph Egan <joseph.s.egan@gmail.com>

LICENSE

@@ -1,4 +1,5 @@
-Copyright 2019 the authors.
+Copyright 2020 the authors.
+Portions of setup.py, copyright 2016 Jason R Coombs <jaraco@jaraco.com>.
 
 Permission is hereby granted, free of charge, to any person obtaining a
 copy of this software and associated documentation files (the "Software"),

MANIFEST.in

@@ -2,3 +2,4 @@ include AUTHORS
 include LICENSE
 include NEWS.rst
 include README.md
+include fastentrypoints.py

NEWS.rst

@@ -1,5 +1,66 @@
 .. default-role:: code
 
+Unreleased
+==============================
+
+Other Breaking Changes
+------------------------------
+* `parse-args` is no longer implemented with `eval`; so e.g. you should
+  now say `:type int` instead of `:type 'int`.
+
+Bug Fixes
+------------------------------
+* Improved support for nesting anaphoric macros by only applying
+  symbol replacement where absolutely necessary.
+* Quoted f-strings are no longer evaluated prematurely.
+* Fixed a regression in the production of error messages for empty
+  expressions.
+
+0.18.0
+==============================
+
+Removals
+------------------------------
+* Python 2 is no longer supported.
+* Support for attribute lists in `defclass` has been removed. Use `setv`
+  and `defn` instead.
+* Literal keywords are no longer parsed differently in calls to functions
+  with certain names.
+* `hy.contrib.multi` has been removed. Use `cond` or the PyPI package
+  `multipledispatch` instead.
+
+Other Breaking Changes
+------------------------------
+* `HySequence` is now a subclass of `tuple` instead of `list`.
+  Thus, a `HyList` will never be equal to a `list`, and you can't
+  use `.append`, `.pop`, etc. on a `HyExpression` or `HyList`.
+
+New Features
+------------------------------
+* Added special forms `py` to `pys` that allow Hy programs to include
+  inline Python code.
+* Added a special form `cmp` for chained comparisons.
+* All augmented assignment operators (except `%=` and `^=`) now allow
+  more than two arguments.
+* Added support for function annotations (PEP 3107) and variable
+  annotations (PEP 526).
+* Added a function `parse-args` as a wrapper for Python's `argparse`.
+
+Bug Fixes
+------------------------------
+* Statements in the second argument of `assert` are now executed.
+* Fixed a bug that caused the condition of a `while` to be compiled
+  twice.
+* `in` and `not-in` now allow more than two arguments, as in Python.
+* `hy2py` can now handle format strings.
+* Fixed crashes from inaccessible history files.
+* Removed an accidental import from the internal Python module `test`.
+* Fixed a swarm of bugs in `hy.extra.anaphoric`.
+
+Misc. Improvements
+------------------------------
+* Replaced the dependency `clint` with `colorama`.
+
 0.17.0
 ==============================
 

README.md

@@ -1,38 +1,22 @@
 Hy
 ==
 
-[![Build Status](https://img.shields.io/travis/hylang/hy/master.svg)](https://travis-ci.org/hylang/hy)
 [![Version](https://img.shields.io/pypi/v/hy.svg)](https://pypi.python.org/pypi/hy)
 
 <a href="https://xkcd.com/224/"><img title="We lost the documentation on quantum mechanics. You'll have to decode the regexes yourself." alt="XKCD #224" src="https://raw.github.com/hylang/shyte/18f6925e08684b0e1f52b2cc2c803989cd62cd91/imgs/xkcd.png"></a>
 
-Lisp and Python should love each other. Let's make it happen. [Try it](http://try-hy.appspot.com/).
+Lisp and Python should love each other. Let's make it happen.
 
-Hylarious Hacks
----------------
+Hy is a Lisp dialect that's embedded in Python. Since Hy transforms its Lisp
+code into Python abstract syntax tree (AST) objects, you have the whole
+beautiful world of Python at your fingertips, in Lisp form.
 
-* [Django + Lisp](https://github.com/paultag/djlisp/tree/master/djlisp)
-* [Python `sh` Fun](https://twitter.com/paultag/status/314925996442796032)
-* [Hy IRC Bot](https://github.com/hylang/hygdrop)
-* [miniKanren in Hy](https://github.com/algernon/adderall)
+To install the latest stable release of Hy, just use the command `pip3 install
+--user hy`. Then you can start an interactive read-eval-print loop (REPL) with
+the command `hy`, or run a Hy program with `hy myprogram.hy`.
 
-OK, so, why?
-------------
-
-Well. Python is awesome. So awesome, that we have so many tools to alter the
-language in a *core* way, but we never use them.
-
-Why?
-
-Well, I wrote Hy to help people realize one thing about Python:
-
-It's really awesome.
-
-Oh, and lisps are neat.
-
-![Cuddles the Hacker](https://i.imgur.com/QbPMXTN.png)
-
-(fan art from the one and only [doctormo](http://doctormo.deviantart.com/art/Cuddles-the-Hacker-372184766))
+* [Why Hy?](http://docs.hylang.org/en/master/whyhy.html)
+* [Tutorial](http://docs.hylang.org/en/master/tutorial.html)
 
 Project
 -------
@@ -41,10 +25,14 @@ Project
 * Documentation:
   * stable, for use with the latest stable release: http://hylang.org/
   * master, for use with the latest revision on GitHub: http://docs.hylang.org/en/master
-* Quickstart: http://hylang.org/en/stable/quickstart.html
 * Bug reports: We have no bugs! Your bugs are your own! (https://github.com/hylang/hy/issues)
 * License: MIT (Expat)
 * [Hacking on Hy](http://docs.hylang.org/en/master/hacking.html)
   * [Contributor Guidelines](http://docs.hylang.org/en/master/hacking.html#contributor-guidelines)
   * [Code of Conduct](http://docs.hylang.org/en/master/hacking.html#contributor-code-of-conduct)
 * IRC: Join #hy on [freenode](https://webchat.freenode.net/)
+* [Stack Overflow: The [hy] tag](https://stackoverflow.com/questions/tagged/hy)
+
+![Cuddles the Hacker](https://i.imgur.com/QbPMXTN.png)
+
+(fan art from the one and only [doctormo](http://doctormo.deviantart.com/art/Cuddles-the-Hacker-372184766))

conftest.py

@@ -4,7 +4,7 @@ import importlib
 import py
 import pytest
 import hy
-from hy._compat import PY3, PY36, PY38
+from hy._compat import PY36, PY38
 
 NATIVE_TESTS = os.path.join("", "tests", "native_tests", "")
 
@@ -12,8 +12,7 @@ _fspath_pyimport = py.path.local.pyimport
 
 
 def pytest_ignore_collect(path, config):
-    return (("py3_only" in path.basename and not PY3) or
-            ("py36_only" in path.basename and not PY36) or
+    return (("py36_only" in path.basename and not PY36) or
             ("py38_only" in path.basename and not PY38) or None)
 
 

docs/conf.py

@@ -52,5 +52,4 @@ html_context = dict(
     hy_descriptive_version = hy_descriptive_version)
 
 intersphinx_mapping = dict(
-    py2 = ('https://docs.python.org/2/', None),
     py = ('https://docs.python.org/3/', None))

docs/contrib/hy_repr.rst

@@ -67,8 +67,8 @@ your choice to the keyword argument ``:placeholder`` of
 .. code-block:: hy
 
    (defclass Container [object]
-     [__init__ (fn [self value]
-       (setv self.value value))])
+     (defn __init__ (fn [self value]
+       (setv self.value value))))
    (hy-repr-register Container :placeholder "HY THERE" (fn [x]
      (+ "(Container " (hy-repr x.value) ")")))
    (setv container (Container 5))

docs/contrib/index.rst

@@ -12,7 +12,6 @@ Contents:
    :maxdepth: 3
 
    loop
-   multi
    profile
    sequences
    walk

docs/contrib/multi.rst

@@ -1,110 +0,0 @@
-========
-defmulti
-========
-
-defn
-----
-.. versionadded:: 0.10.0
-
-``defn`` lets you arity-overload a function by the given number of
-args and/or kwargs. This version of ``defn`` works with regular syntax and
-with the arity overloaded one. Inspired by Clojures take on ``defn``.
-
-.. code-block:: clj
-
-    => (require [hy.contrib.multi [defn]])
-    => (defn fun
-    ...  ([a] "a")
-    ...  ([a b] "a b")
-    ...  ([a b c] "a b c"))
-
-    => (fun 1)
-    "a"
-    => (fun 1 2)
-    "a b"
-    => (fun 1 2 3)
-    "a b c"
-
-    => (defn add [a b]
-    ...  (+ a b))
-    => (add 1 2)
-    3
-
-
-defmulti
---------
-
-.. versionadded:: 0.12.0
-
-``defmulti``, ``defmethod`` and ``default-method`` lets you define
-multimethods where a dispatching function is used to select between different
-implementations of the function. Inspired by Clojure's multimethod and based
-on the code by `Adam Bard`_.
-
-.. code-block:: clj
-
-    => (require [hy.contrib.multi [defmulti defmethod default-method]])
-    => (defmulti area [shape]
-    ...  "calculate area of a shape"
-    ...  (:type shape))
-
-    => (defmethod area "square" [square]
-    ...  (* (:width square)
-    ...     (:height square)))
-
-    => (defmethod area "circle" [circle]
-    ...  (* (** (:radius circle) 2)
-    ...     3.14))
-
-    => (default-method area [shape]
-    ...  0)
-
-    => (area {:type "circle" :radius 0.5})
-    0.785
-
-    => (area {:type "square" :width 2 :height 2})
-    4
-
-    => (area {:type "non-euclid rhomboid"})
-    0
-
-``defmulti`` is used to define the initial multimethod with name, signature
-and code that selects between different implementations. In the example,
-multimethod expects a single input that is type of dictionary and contains
-at least key :type. The value that corresponds to this key is returned and
-is used to selected between different implementations.
-
-``defmethod`` defines a possible implementation for multimethod. It works
-otherwise in the same way as ``defn``, but has an extra parameters
-for specifying multimethod and which calls are routed to this specific
-implementation. In the example, shapes with "square" as :type are routed to
-first function and shapes with "circle" as :type are routed to second
-function.
-
-``default-method`` specifies default implementation for multimethod that is
-called when no other implementation matches.
-
-Interfaces of multimethod and different implementation don't have to be
-exactly identical, as long as they're compatible enough. In practice this
-means that multimethod should accept the broadest range of parameters and
-different implementations can narrow them down.
-
-.. code-block:: clj
-
-    => (require [hy.contrib.multi [defmulti defmethod]])
-    => (defmulti fun [&rest args]
-    ...  (len args))
-
-    => (defmethod fun 1 [a]
-    ...  a)
-
-    => (defmethod fun 2 [a b]
-    ...  (+ a b))
-
-    => (fun 1)
-    1
-
-    => (fun 1 2)
-    3
-
-.. _Adam Bard: https://adambard.com/blog/implementing-multimethods-in-python/

docs/contrib/profile.rst

@@ -17,7 +17,7 @@ profile/calls
 --------------
 
 ``profile/calls`` allows you to create a call graph visualization.
-**Note:** You must have `Graphviz <http://www.graphviz.org/Home.php>`_
+**Note:** You must have `Graphviz <http://www.graphviz.org/>`_
 installed for this to work.
 
 

docs/contrib/walk.rst

@@ -206,6 +206,8 @@ Recursively performs all possible macroexpansions in form, using the ``require``
 Macros
 ======
 
+.. _let:
+
 let
 ---
 
@@ -238,7 +240,8 @@ The ``let`` macro takes two parameters: a list defining *variables*
 and the *body* which gets executed. *variables* is a vector of
 variable and value pairs.
 
-``let`` executes the variable assignments one-by-one, in the order written.
+Like the ``let*`` of many other Lisps, ``let`` executes the variable
+assignments one-by-one, in the order written::
 
 .. code-block:: hy
 
@@ -247,4 +250,8 @@ variable and value pairs.
     ...  (print x y))
     5 6
 
+Unlike them, however, each ``(let …)`` form uses only one
+namespace for all its assignments. Thus, ``(let [x 1  x (fn [] x)]
+(x))`` returns a function object, not 1 as you might expect.
+
 It is an error to use a let-bound name in a ``global`` or ``nonlocal`` form.

docs/coreteam.rst

@@ -1,16 +1,11 @@
-* `Julien Danjou <https://github.com/jd>`_
-* `Morten Linderud <https://github.com/Foxboron>`_
-* `J Kenneth King <https://github.com/agentultra>`_
-* `Gergely Nagy <https://github.com/algernon>`_
-* `Tuukka Turto <https://github.com/tuturto>`_
-* `Karen Rustad <https://github.com/aldeka>`_
-* `Abhishek L <https://github.com/theanalyst>`_
-* `Christopher Allan Webber <https://github.com/cwebber>`_
-* `Konrad Hinsen <https://github.com/khinsen>`_
-* `Will Kahn-Greene <https://github.com/willkg>`_
-* `Paul Tagliamonte <https://github.com/paultag>`_
+* `Kodi B. Arfer <https://github.com/Kodiologist>`_
 * `Nicolas Dandrimont <https://github.com/olasd>`_
-* `Berker Peksag <https://github.com/berkerpeksag>`_
-* `Clinton N. Dreisbach <https://github.com/cndreisbach>`_
-* `han semaj <https://github.com/microamp>`_
-* `Kodi Arfer <https://github.com/Kodiologist>`_
+* `Julien Danjou <https://github.com/jd>`_
+* `Rob Day <https://github.com/rkday>`_
+* `Simon Gomizelj <https://github.com/vodik>`_
+* `Ryan Gonzalez <https://github.com/refi64>`_
+* `Abhishek Lekshmanan <https://github.com/theanalyst>`_
+* `Morten Linderud <https://github.com/Foxboron>`_
+* `Matthew Odendahl <https://github.com/gilch>`_
+* `Paul Tagliamonte <https://github.com/paultag>`_
+* `Brandon T. Willard <https://github.com/brandonwillard>`_

docs/extra/anaphoric.rst

@@ -17,15 +17,29 @@ To use these macros you need to require the ``hy.extra.anaphoric`` module like s
 
 ``(require [hy.extra.anaphoric [*]])``
 
+These macros are implemented by replacing any use of the designated
+anaphoric symbols (``it``, in most cases) with a gensym. Consequently,
+it's unwise to nest these macros where symbol replacement is happening.
+Symbol replacement typically takes place in ``body`` or ``form``
+parameters, where the output of the expression may be returned. It is also
+recommended to avoid using an affected symbol as something other than a
+variable name, as in ``(print "My favorite Stephen King book is" 'it)``.
+
 .. _ap-if:
 
 ap-if
 =====
 
-Usage: ``(ap-if (foo) (print it))``
+Usage: ``(ap-if test-form then-form else-form)``
 
-Evaluates the first form for truthiness, and bind it to ``it`` in both the
-true and false branches.
+As :ref:`if <if>`, but the result of the test form is named ``it`` in
+the subsequent forms. As with ``if``, the else-clause is optional.
+
+.. code-block:: hy
+
+   => (import os)
+   => (ap-if (.get os.environ "PYTHONPATH")
+   ...  (print "Your PYTHONPATH is" it))
 
 
 .. _ap-each:
@@ -33,9 +47,17 @@ true and false branches.
 ap-each
 =======
 
-Usage: ``(ap-each [1 2 3 4 5] (print it))``
+Usage: ``(ap-each xs body…)``
 
-Evaluate the form for each element in the list for side-effects.
+Evaluate the body forms for each element ``it`` of ``xs`` and return
+``None``.
+
+.. code-block:: hy
+
+   => (ap-each [1 2 3] (print it))
+   1
+   2
+   3
 
 
 .. _ap-each-while:
@@ -43,10 +65,10 @@ Evaluate the form for each element in the list for side-effects.
 ap-each-while
 =============
 
-Usage: ``(ap-each-while list pred body)``
+Usage: ``(ap-each-while xs pred body…)``
 
-Evaluate the form for each element where the predicate form returns
-``True``.
+As ``ap-each``, but the form ``pred`` is run before the body forms on
+each iteration, and the loop ends if ``pred`` is false.
 
 .. code-block:: hy
 
@@ -60,11 +82,10 @@ Evaluate the form for each element where the predicate form returns
 ap-map
 ======
 
-Usage: ``(ap-map form list)``
+Usage: ``(ap-map form xs)``
 
-The anaphoric form of map works just like regular map except that
-instead of a function object it takes a Hy form. The special name
-``it`` is bound to the current object from the list in the iteration.
+Create a generator like :py:func:`map` that yields each result of ``form``
+evaluated with ``it`` bound to successive elements of ``xs``.
 
 .. code-block:: hy
 
@@ -77,10 +98,12 @@ instead of a function object it takes a Hy form. The special name
 ap-map-when
 ===========
 
-Usage: ``(ap-map-when predfn rep list)``
+Usage: ``(ap-map-when predfn rep xs)``
 
-Evaluate a mapping over the list using a predicate function to
-determin when to apply the form.
+As ``ap-map``, but the predicate function ``predfn`` (yes, that's a
+function, not an anaphoric form) is applied to each ``it``, and the
+anaphoric mapping form ``rep`` is only applied if the predicate is true.
+Otherwise, ``it`` is yielded unchanged.
 
 .. code-block:: hy
 
@@ -96,11 +119,9 @@ determin when to apply the form.
 ap-filter
 =========
 
-Usage: ``(ap-filter form list)``
+Usage: ``(ap-filter form xs)``
 
-As with ``ap-map`` we take a special form instead of a function to
-filter the elements of the list. The special name ``it`` is bound to
-the current element in the iteration.
+The :py:func:`filter` equivalent of ``ap-map``.
 
 .. code-block:: hy
 
@@ -113,11 +134,9 @@ the current element in the iteration.
 ap-reject
 =========
 
-Usage: ``(ap-reject form list)``
+Usage: ``(ap-reject form xs)``
 
-This function does the opposite of ``ap-filter``, it rejects the
-elements passing the predicate . The special name ``it`` is bound to
-the current element in the iteration.
+Equivalent to ``(ap-filter (not form) xs)``.
 
 .. code-block:: hy
 
@@ -130,10 +149,9 @@ the current element in the iteration.
 ap-dotimes
 ==========
 
-Usage ``(ap-dotimes n body)``
+Usage: ``(ap-dotimes n body…)``
 
-This function evaluates the body *n* times, with the special
-variable ``it`` bound from *0* to *1-n*. It is useful for side-effects.
+Equivalent to ``(ap-each (range n) body…)``.
 
 .. code-block:: hy
 
@@ -148,11 +166,11 @@ variable ``it`` bound from *0* to *1-n*. It is useful for side-effects.
 ap-first
 ========
 
-Usage ``(ap-first predfn list)``
+Usage: ``(ap-first form xs)``
 
-This function returns the first element that passes the predicate or
-``None``, with the special variable ``it`` bound to the current element in
-iteration.
+Evaluate the predicate ``form`` for each element ``it`` of ``xs``. When
+the predicate is true, stop and return ``it``. If the predicate is never
+true, return ``None``.
 
 .. code-block:: hy
 
@@ -165,11 +183,11 @@ iteration.
 ap-last
 ========
 
-Usage ``(ap-last predfn list)``
+Usage: ``(ap-last form list)``
 
-This function returns the last element that passes the predicate or
-``None``, with the special variable ``it`` bound to the current element in
-iteration.
+Evaluate the predicate ``form`` for every element ``it`` of ``xs``.
+Return the last element for which the predicate is true, or ``None`` if
+there is no such element.
 
 .. code-block:: hy
 
@@ -182,14 +200,20 @@ iteration.
 ap-reduce
 =========
 
-Usage ``(ap-reduce form list &optional initial-value)``
+Usage: ``(ap-reduce form xs &optional initial-value)``
 
-This function returns the result of applying form to the first 2
-elements in the body and applying the result and the 3rd element
-etc. until the list is exhausted. Optionally an initial value can be
-supplied so the function will be applied to initial value and the
-first element instead. This exposes the element being iterated as
-``it`` and the current accumulated value as ``acc``.
+This macro is an anaphoric version of :py:func:`reduce`. It works as
+follows:
+
+- Bind ``acc`` to the first element of ``xs``, bind ``it`` to the
+  second, and evaluate ``form``.
+- Bind ``acc`` to the result, bind ``it`` to the third value of ``xs``,
+  and evaluate ``form`` again.
+- Bind ``acc`` to the result, and continue until ``xs`` is exhausted.
+
+If ``initial-value`` is supplied, the process instead begins with
+``acc`` set to ``initial-value`` and ``it`` set to the first element of
+``xs``.
 
 .. code-block:: hy
 
@@ -202,7 +226,7 @@ first element instead. This exposes the element being iterated as
 #%
 ==
 
-Usage ``#% expr``
+Usage: ``#% expr``
 
 Makes an expression into a function with an implicit ``%`` parameter list.
 

docs/hacking.rst

@@ -1,3 +1,5 @@
+.. _hacking:
+
 ===============
  Hacking on Hy
 ===============

docs/index.rst

@@ -1,36 +1,28 @@
-Welcome to Hy's documentation!
-==============================
+The Hy Manual
+=============
 
 .. image:: _static/hy-logo-small.png
    :alt: Hy
    :align: left
 
-:Try Hy: https://try-hy.appspot.com
 :PyPI: https://pypi.python.org/pypi/hy
 :Source: https://github.com/hylang/hy
 :List: `hylang-discuss <https://groups.google.com/forum/#!forum/hylang-discuss>`_
-:IRC: ``#hy`` on Freenode
-:Build status:
-    .. image:: https://secure.travis-ci.org/hylang/hy.png
-        :alt: Travis CI
-        :target: http://travis-ci.org/hylang/hy
+:IRC: irc://chat.freenode.net/hy
+:Stack Overflow: `The [hy] tag <https://stackoverflow.com/questions/tagged/hy>`_
 
-Hy is a wonderful dialect of Lisp that's embedded in Python.
+Hy is a Lisp dialect that's embedded in Python. Since Hy transforms its Lisp
+code into Python abstract syntax tree (AST) objects, you have the whole
+beautiful world of Python at your fingertips, in Lisp form.
 
-Since Hy transforms its Lisp code into the Python Abstract Syntax
-Tree, you have the whole beautiful world of Python at your fingertips,
-in Lisp form!
-
-
-Documentation Index
-===================
-
-Contents:
+To install the latest stable release of Hy, just use the command ``pip3 install
+--user hy``. Then you can start an interactive read-eval-print loop (REPL) with
+the command ``hy``, or run a Hy program with ``hy myprogram.hy``.
 
 .. toctree::
    :maxdepth: 3
 
-   quickstart
+   whyhy
    tutorial
    style-guide
    language/index

docs/language/api.rst

@@ -1,3 +1,5 @@
+.. _special-forms:
+
 =================
 Built-Ins
 =================
@@ -6,6 +8,48 @@ Hy features a number of special forms that are used to help generate
 correct Python AST. The following are "special" forms, which may have
 behavior that's slightly unexpected in some situations.
 
+^
+-
+
+The ``^`` symbol is used to denote annotations in three different contexts:
+
+- Standalone variable annotations.
+- Variable annotations in a setv call.
+- Function argument annotations.
+
+They implement `PEP 526 <https://www.python.org/dev/peps/pep-0526/>`_ and
+`PEP 3107 <https://www.python.org/dev/peps/pep-3107/>`_.
+
+Here is some example syntax of all three usages:
+
+.. code-block:: clj
+
+  ; Annotate the variable x as an int (equivalent to `x: int`).
+  (^int x)
+  ; Can annotate with expressions if needed (equivalent to `y: f(x)`).
+  (^(f x) y)
+
+  ; Annotations with an assignment: each annotation (int, str) covers the term that
+  ; immediately follows.
+  ; Equivalent to: x: int = 1; y = 2; z: str = 3
+  (setv ^int x 1 y 2 ^str z 3)
+
+  ; Annotate a as an int, c as an int, and b as a str.
+  ; Equivalent to: def func(a: int, b: str = None, c: int = 1): ...
+  (defn func [^int a &optional ^str b ^int [c 1]] ...)
+
+The rules are:
+
+- The value to annotate with is the value that immediately follows the caret.
+- There must be no space between the caret and the value to annotate, otherwise it will be
+  interpreted as a bitwise XOR like the Python operator.
+- The annotation always comes (and is evaluated) *before* the value being annotated. This is
+  unlike Python, where it comes and is evaluated *after* the value being annotated.
+
+Note that variable annotations are only supported on Python 3.6+.
+
+For annotating items with generic types, the of_ macro will likely be of use.
+
 .
 -
 
@@ -224,6 +268,23 @@ Examples of usage:
 .. note:: ``assoc`` modifies the datastructure in place and returns ``None``.
 
 
+await
+-----
+
+``await`` creates an :ref:`await expression <py:await>`. It takes exactly one
+argument: the object to wait for.
+
+::
+
+    => (import asyncio)
+    => (defn/a main []
+    ...   (print "hello")
+    ...   (await (asyncio.sleep 1))
+    ...   (print "world"))
+    => (asyncio.run (main))
+    hello
+    world
+
 break
 -----
 
@@ -238,6 +299,38 @@ as the user enters *k*.
                   (print "Try again")))
 
 
+cmp
+---
+
+``cmp`` creates a :ref:`comparison expression <py:comparisons>`. It isn't
+required for unchained comparisons, which have only one comparison operator,
+nor for chains of the same operator. For those cases, you can use the
+comparison operators directly with Hy's usual prefix syntax, as in ``(= x 1)``
+or ``(< 1 2 3)``. The use of ``cmp`` is to construct chains of heterogeneous
+operators, such as ``x <= y < z``. It uses an infix syntax with the general
+form
+
+::
+
+    (cmp ARG OP ARG OP ARG…)
+
+Hence, ``(cmp x <= y < z)`` is equivalent to ``(and (<= x y) (< y z))``,
+including short-circuiting, except that ``y`` is only evaluated once.
+
+Each ``ARG`` is an arbitrary form, which does not itself use infix syntax. Use
+:ref:`py-specialform` if you want fully Python-style operator syntax. You can
+also nest ``cmp`` forms, although this is rarely useful. Each ``OP`` is a
+literal comparison operator; other forms that resolve to a comparison operator
+are not allowed.
+
+At least two ``ARG``\ s and one ``OP`` are required, and every ``OP`` must be
+followed by an ``ARG``.
+
+As elsewhere in Hy, the equality operator is spelled ``=``, not ``==`` as in
+Python.
+
+
+
 comment
 -------
 
@@ -262,6 +355,8 @@ This is completely discarded and doesn't expand to anything, not even ``None``.
    Hy
 
 
+.. _cond:
+
 cond
 ----
 
@@ -321,6 +416,8 @@ is only called on every other value in the list.
       (side-effect2 x))
 
 
+.. _do:
+
 do
 ----------
 
@@ -383,6 +480,8 @@ the second of which becomes each value.
     {0: 0, 1: 10, 2: 20, 3: 30, 4: 40}
 
 
+.. _setv:
+
 setv
 ----
 
@@ -399,20 +498,19 @@ For example:
     => (counter [1 2 3 4 5 2 3] 2)
     2
 
-They can be used to assign multiple variables at once:
+You can provide more than one target–value pair, and the assignments will be made in order::
 
-.. code-block:: hy
+    (setv  x 1  y x  x 2)
+    (print x y)  ; => 2 1
 
-    => (setv a 1 b 2)
-    (1L, 2L)
-    => a
-    1L
-    => b
-    2L
-    =>
+You can perform parallel assignments or unpack the source value with square brackets and :ref:`unpack-iterable`::
 
+    (setv duo ["tim" "eric"])
+    (setv [guy1 guy2] duo)
+    (print guy1 guy2)  ; => tim eric
 
-``setv`` always returns ``None``.
+    (setv [letter1 letter2 #* others] "abcdefg")
+    (print letter1 letter2 others)   ; => a b ['c', 'd', 'e', 'f', 'g']
 
 
 setx
@@ -427,19 +525,21 @@ Whereas ``setv`` creates an assignment statement, ``setx`` creates an assignment
     3 is greater than 0
 
 
+.. _defclass:
+
 defclass
 --------
 
-New classes are declared with ``defclass``. It can take three optional parameters in the following order:
-a list defining (a) possible super class(es), a string (:term:`py:docstring`) and another list containing
-attributes of the new class along with their corresponding values.
+New classes are declared with ``defclass``. It can take optional parameters in the following order:
+a list defining (a) possible super class(es) and a string (:term:`py:docstring`).
 
 .. code-block:: clj
 
     (defclass class-name [super-class-1 super-class-2]
       "docstring"
-      [attribute1 value1
-       attribute2 value2]
+
+      (setv attribute1 value1)
+      (setv attribute2 value2)
 
       (defn method [self] (print "hello!")))
 
@@ -449,8 +549,8 @@ below:
 .. code-block:: clj
 
     => (defclass Cat []
-    ...  [age None
-    ...   colour "white"]
+    ...  (setv age None)
+    ...  (setv colour "white")
     ...
     ...  (defn speak [self] (print "Meow")))
 
@@ -564,8 +664,6 @@ requires.
           File "<input>", line 1, in <module>
         TypeError: compare() missing 1 required keyword-only argument: 'keyfn'
 
-    Availability: Python 3.
-
 &kwargs
     Like ``&rest``, but for keyword arugments.
     The following parameter will contain 0 or more keyword arguments.
@@ -899,6 +997,9 @@ raising an exception.
     => (first [])
     None
 
+
+.. _for:
+
 for
 ---
 
@@ -977,6 +1078,8 @@ written without accidental variable name clashes.
 
    Section :ref:`using-gensym`
 
+.. _get:
+
 get
 ---
 
@@ -1007,6 +1110,8 @@ successive elements in a nested structure. Example usage:
           index that is out of bounds.
 
 
+.. _gfor:
+
 gfor
 ----
 
@@ -1048,6 +1153,8 @@ keyword, the second function would have raised a ``NameError``.
     (set-a 5)
     (print-a)
 
+.. _if:
+
 if / if* / if-not
 -----------------
 
@@ -1057,7 +1164,7 @@ if / if* / if-not
 ``if / if* / if-not`` respect Python *truthiness*, that is, a *test* fails if it
 evaluates to a "zero" (including values of ``len`` zero, ``None``, and
 ``False``), and passes otherwise, but values with a ``__bool__`` method
-(``__nonzero__`` in Python 2) can overrides this.
+can override this.
 
 The ``if`` macro is for conditionally selecting an expression for evaluation.
 The result of the selected expression becomes the result of the entire ``if``
@@ -1173,6 +1280,8 @@ that ``import`` can be used.
     (import [sys [*]])
 
 
+.. _fn:
+
 fn
 -----------
 
@@ -1238,6 +1347,8 @@ last
     6
 
 
+.. _lfor:
+
 lfor
 ----
 
@@ -1296,19 +1407,12 @@ fact, these forms are implemented as generator functions whenever they
 contain Python statements, with the attendant consequences for calling
 ``return``. By contrast, ``for`` shares the caller's scope.
 
-.. note:: An exception to the above scoping rules occurs on Python 2 for
-   ``lfor`` specifically (and not ``sfor``, ``gfor``, or ``dfor``) when
-   Hy can implement the ``lfor`` as a Python list comprehension. Then,
-   variables will leak to the surrounding scope.
-
 
 nonlocal
 --------
 
 .. versionadded:: 0.11.1
 
-**PYTHON 3.0 AND UP ONLY!**
-
 ``nonlocal`` can be used to mark a symbol as not local to the current scope.
 The parameters are the names of symbols to mark as nonlocal.  This is necessary
 to modify variables through nested ``fn`` scopes:
@@ -1376,17 +1480,75 @@ parameter will be returned.
     True
 
 
-print
------
+of
+--
 
-``print`` is used to output on screen. Example usage:
+``of`` is an alias for get, but with special semantics designed for handling PEP 484's generic
+types.
+
+``of`` has three forms:
+
+- ``(of T)`` will simply become ``T``.
+- ``(of T x)`` will become ``(get T x)``.
+- ``(of T x y ...)`` (where the ``...`` represents zero or more arguments) will become
+  ``(get T (, x y ...))``.
+
+For instance:
 
 .. code-block:: clj
 
-    (print "Hello world!")
+  (of str)  ; => str
 
-.. note:: ``print`` always returns ``None``.
+  (of List int)  ; => List[int]
+  (of Set int)   ; => Set[int]
 
+  (of Dict str str)   ; => Dict[str, str]
+  (of Tuple str int)  ; => Tuple[str, int]
+
+  (of Callable [int str] str)  ; => Callable[[int, str], str]
+
+
+.. _py-specialform:
+
+py
+--
+
+``py`` parses the given Python code at compile-time and inserts the result into
+the generated abstract syntax tree. Thus, you can mix Python code into a Hy
+program. Only a Python expression is allowed, not statements; use
+:ref:`pys-specialform` if you want to use Python statements. The value of the
+expression is returned from the ``py`` form. ::
+
+    (print "A result from Python:" (py "'hello' + 'world'"))
+
+The code must be given as a single string literal, but you can still use
+macros, :ref:`eval-fn`, and related tools to construct the ``py`` form. If
+having to backslash-escape internal double quotes is getting you down, try a
+:ref:`bracket string <syntax-bracket-strings>`. If you want to evaluate some
+Python code that's only defined at run-time, try the standard Python function
+:func:`eval`.
+
+Python code need not syntactically round-trip if you use ``hy2py`` on a Hy
+program that uses ``py`` or ``pys``. For example, comments will be removed.
+
+
+.. _pys-specialform:
+
+pys
+---
+
+As :ref:`py-specialform`, but the code can consist of zero or more statements,
+including compound statements such as ``for`` and ``def``. ``pys`` always
+returns ``None``. Also, the code string is dedented with
+:func:`textwrap.dedent` before parsing, which allows you to intend the code to
+match the surrounding Hy code, but significant leading whitespace in embedded
+string literals will be removed. ::
+
+    (pys "myvar = 5")
+    (print "myvar is" myvar)
+
+
+.. _quasiquote:
 
 quasiquote
 ----------
@@ -1406,6 +1568,8 @@ using ``unquote`` (``~``). The evaluated form can also be spliced using
     ; equivalent to '(foo bar baz)
 
 
+.. _quote:
+
 quote
 -----
 
@@ -1423,6 +1587,8 @@ alternatively be written using the apostrophe (``'``) symbol.
     Hello World
 
 
+.. _require:
+
 require
 -------
 
@@ -1563,6 +1729,8 @@ sfor
 equivalent to ``(set (lfor CLAUSES VALUE))``. See `lfor`_.
 
 
+.. _cut:
+
 cut
 -----
 
@@ -1669,6 +1837,8 @@ the given conditional is ``False``. The following shows the expansion of this ma
       (do statement))
 
 
+.. _unpack-iterable:
+
 unpack-iterable, unpack-mapping
 -------------------------------
 
@@ -1693,7 +1863,7 @@ object (respectively) to provide positional or keywords arguments
     => (f #* [1 2] #** {"c" 3 "d" 4})
     [1, 2, 3, 4]
 
-With Python 3, unpacking is allowed in more contexts, and you can unpack
+Unpacking is allowed in a variety of contexts, and you can unpack
 more than once in one expression (:pep:`3132`, :pep:`448`).
 
 .. code-block:: clj
@@ -1709,6 +1879,8 @@ more than once in one expression (:pep:`3132`, :pep:`448`).
     [1, 2, 3, 4]
 
 
+.. _unquote:
+
 unquote
 -------
 
@@ -1784,6 +1956,8 @@ following shows the expansion of the macro.
     (if conditional (do statement))
 
 
+.. _while:
+
 while
 -----
 
@@ -1843,6 +2017,9 @@ prints
     In condition
     At end of outer loop
 
+
+.. _with:
+
 with
 ----
 
@@ -2038,8 +2215,6 @@ yield-from
 
 .. versionadded:: 0.9.13
 
-**PYTHON 3.3 AND UP ONLY!**
-
 ``yield-from`` is used to call a subgenerator.  This is useful if you
 want your coroutine to be able to delegate its processes to another
 coroutine, say, if using something fancy like

docs/language/core.rst

@@ -240,19 +240,6 @@ otherwise ``False``. Return ``True`` if *coll* is empty.
    True
 
 
-.. _exec-fn:
-
-exec
-----
-
-Equivalent to Python 3's built-in function :py:func:`exec`.
-
-.. code-block:: clj
-
-    => (exec "print(a + b)" {"a" 1} {"b" 2})
-    3
-
-
 .. _float?-fn:
 
 float?
@@ -385,8 +372,7 @@ integer?
 
 Usage: ``(integer? x)``
 
-Returns `True` if *x* is an integer. For Python 2, this is
-either ``int`` or ``long``. For Python 3, this is ``int``.
+Returns `True` if *x* is an integer (``int``).
 
 .. code-block:: hy
 
@@ -800,6 +786,29 @@ Returns ``True`` if *x* is odd. Raises ``TypeError`` if
    => (odd? 0)
    False
 
+.. _parse-args:
+
+parse-args
+----------
+
+Usage: ``(parse-args spec &optional args &kwargs parser-args)``
+
+Return arguments namespace parsed from *args* or ``sys.argv`` with
+:py:meth:`argparse.ArgumentParser.parse_args` according to *spec*.
+
+*spec* should be a list of arguments which will be passed to repeated
+calls to :py:meth:`argparse.ArgumentParser.add_argument`.  *parser-args*
+may be a list of keyword arguments to pass to the
+:py:class:`argparse.ArgumentParser` constructor.
+
+.. code-block:: hy
+
+    => (parse-args [["strings" :nargs "+" :help "Strings"]
+                    ["-n" "--numbers" :action "append" :type int :help "Numbers"]]
+                   ["a" "b" "-n" "1" "-n" "2"]
+                   :description "Parse strings and numbers from args")
+    Namespace(numbers=[1, 2], strings=['a', 'b'])
+
 .. _partition-fn:
 
 partition
@@ -924,7 +933,7 @@ string?
 
 Usage: ``(string? x)``
 
-Returns ``True`` if *x* is a string.
+Returns ``True`` if *x* is a string (``str``).
 
 .. code-block:: hy
 

docs/language/internals.rst

@@ -44,9 +44,9 @@ If this is causing issues,
 it can be turned off globally by setting ``hy.models.PRETTY`` to ``False``,
 or temporarily by using the ``hy.models.pretty`` context manager.
 
-Hy also attempts to color pretty reprs using ``clint.textui.colored``.
-This module has a flag to disable coloring,
-and a method ``clean`` to strip colored strings of their color tags.
+Hy also attempts to color pretty reprs and errors using ``colorama``. These can
+be turned off globally by setting ``hy.models.COLORED`` and ``hy.errors.COLORED``,
+respectively, to ``False``.
 
 .. _hysequence:
 
@@ -60,6 +60,10 @@ Adding a HySequence to another iterable object reuses the class of the
 left-hand-side object, a useful behavior when you want to concatenate Hy
 objects in a macro, for instance.
 
+HySequences are (mostly) immutable: you can't add, modify, or remove
+elements. You can still append to a variable containing a HySequence with
+``+=`` and otherwise construct new HySequences out of old ones.
+
 
 .. _hylist:
 
@@ -90,11 +94,6 @@ HyDict
 ``hy.models.HyDict`` inherits :ref:`HySequence` for curly-bracketed
 ``{}`` expressions, which compile down to a Python dictionary literal.
 
-The decision of using a list instead of a dict as the base class for
-``HyDict`` allows easier manipulation of dicts in macros, with the added
-benefit of allowing compound expressions as dict keys (as, for instance,
-the :ref:`HyExpression` Python class isn't hashable).
-
 Atomic Models
 -------------
 
@@ -120,9 +119,7 @@ HyString
 ~~~~~~~~
 
 ``hy.models.HyString`` represents string literals (including bracket strings),
-which compile down to unicode string literals in Python. ``HyStrings`` inherit
-unicode objects in Python 2, and string objects in Python 3 (and are therefore
-not encoding-dependent).
+which compile down to unicode string literals (``str``) in Python.
 
 ``HyString``\s are immutable.
 
@@ -140,15 +137,15 @@ HyBytes
 ~~~~~~~
 
 ``hy.models.HyBytes`` is like ``HyString``, but for sequences of bytes.
-It inherits from ``bytes`` on Python 3 and ``str`` on Python 2.
+It inherits from ``bytes``.
 
 .. _hy_numeric_models:
 
 Numeric Models
 ~~~~~~~~~~~~~~
 
-``hy.models.HyInteger`` represents integer literals (using the
-``long`` type on Python 2, and ``int`` on Python 3).
+``hy.models.HyInteger`` represents integer literals, using the ``int``
+type.
 
 ``hy.models.HyFloat`` represents floating-point literals.
 

docs/language/interop.rst

@@ -1,10 +1,9 @@
+.. _interop:
+
 =====================
 Hy <-> Python interop
 =====================
 
-   “Keep in mind we’re not Clojure. We’re not Common Lisp. We’re Homoiconic
-   Python, with extra bits that make sense.” — Hy Style Guide
-
 Despite being a Lisp, Hy aims to be fully compatible with Python. That means
 every Python module or package can be imported in Hy code, and vice versa.
 
@@ -17,9 +16,11 @@ Hy and Python. For example, Python's ``str.format_map`` can be written
 Using Python from Hy
 ====================
 
-Using Python from Hy is nice and easy, you just have to :ref:`import` it.
+You can embed Python code directly into a Hy program with the special operators
+:ref:`py-specialform` and :ref:`pys-specialform`.
 
-If you have the following in ``greetings.py`` in Python::
+Using a Python module from Hy is nice and easy: you just have to :ref:`import`
+it. If you have the following in ``greetings.py`` in Python::
 
     def greet(name):
         print("hello," name)

docs/language/syntax.rst

@@ -1,3 +1,5 @@
+.. _syntax:
+
 ==============
 Syntax
 ==============
@@ -10,7 +12,7 @@ An identifier consists of a nonempty sequence of Unicode characters that are not
 numeric literals
 ----------------
 
-In addition to regular numbers, standard notation from Python 3 for non-base 10
+In addition to regular numbers, standard notation from Python for non-base 10
 integers is used. ``0x`` for Hex, ``0o`` for Octal, ``0b`` for Binary.
 
 .. code-block:: clj
@@ -35,6 +37,8 @@ Hy allows double-quoted strings (e.g., ``"hello"``), but not single-quoted
 strings like Python. The single-quote character ``'`` is reserved for
 preventing the evaluation of a form (e.g., ``'(+ 1 1)``), as in most Lisps.
 
+.. _syntax-bracket-strings:
+
 Python's so-called triple-quoted strings (e.g., ``'''hello'''`` and
 ``"""hello"""``) aren't supported. However, in Hy, unlike Python, any string
 literal can contain newlines. Furthermore, Hy supports an alternative form of
@@ -60,13 +64,9 @@ Plain string literals support :ref:`a variety of backslash escapes
 literally, prefix the string with ``r``, as in ``r"slash\not"``. Bracket
 strings are always raw strings and don't allow the ``r`` prefix.
 
-Whether running under Python 2 or Python 3, Hy treats all string literals as
-sequences of Unicode characters by default, and allows you to prefix a plain
-string literal (but not a bracket string) with ``b`` to treat it as a sequence
-of bytes. So when running under Python 3, Hy translates ``"foo"`` and
-``b"foo"`` to the identical Python code, but when running under Python 2,
-``"foo"`` is translated to ``u"foo"`` and ``b"foo"`` is translated to
-``"foo"``.
+Like Python, Hy treats all string literals as sequences of Unicode characters
+by default. You may prefix a plain string literal (but not a bracket string)
+with ``b`` to treat it as a sequence of bytes.
 
 Unlike Python, Hy only recognizes string prefixes (``r``, etc.) in lowercase.
 

docs/quickstart.rst

@@ -1,54 +0,0 @@
-==========
-Quickstart
-==========
-
-.. image:: _static/cuddles-transparent-small.png
-   :alt: Karen Rustard's Cuddles
-
-(Thanks to Karen Rustad for Cuddles!)
-
-
-**HOW TO GET HY REAL FAST**:
-
-1. Create a `Virtual Python Environment
-   <https://pypi.python.org/pypi/virtualenv>`_.
-2. Activate your Virtual Python Environment.
-3. Install `hy from GitHub <https://github.com/hylang/hy>`_ with ``$ pip install git+https://github.com/hylang/hy.git``.
-4. Start a REPL with ``hy``.
-5. Type stuff in the REPL::
-
-       => (print "Hy!")
-       Hy!
-       => (defn salutationsnm [name] (print (+ "Hy " name "!")))
-       => (salutationsnm "YourName")
-       Hy YourName!
-
-       etc
-
-6. Hit CTRL-D when you're done.
-7. If you're familiar with Python, start the REPL using ``hy --spy`` to check what happens inside::
-
-       => (+ "Hyllo " "World" "!")
-       'Hyllo ' + 'World' + '!'
-
-       'Hyllo World!'
-
-*OMG! That's amazing! I want to write a Hy program.*
-
-8. Open up an elite programming editor and type::
-
-       #! /usr/bin/env hy
-       (print "I was going to code in Python syntax, but then I got Hy.")
-
-9. Save as ``awesome.hy``.
-10. Make it executable::
-
-        chmod +x awesome.hy
-
-11. And run your first Hy program::
-
-        ./awesome.hy
-
-12. Take a deep breath so as to not hyperventilate.
-13. Smile villainously and sneak off to your hydeaway and do
-    unspeakable things.

docs/tutorial.rst

@@ -2,275 +2,174 @@
 Tutorial
 ========
 
-.. TODO
-..
-..  - How do I index into arrays or dictionaries?
-..  - Blow your mind with macros!
-..  - Where's my banana???
+.. image:: _static/cuddles-transparent-small.png
+   :alt: Karen Rustard's Cuddles
 
-Welcome to the Hy tutorial!
+This chapter provides a quick introduction to Hy. It assumes a basic background
+in programming, but no specific prior knowledge of Python or Lisp.
 
-In a nutshell, Hy is a Lisp dialect, but one that converts its
-structure into Python ... literally a conversion into Python's abstract
-syntax tree!  (Or to put it in more crude terms, Hy is lisp-stick on a
-Python!)
+Lisp-stick on a Python
+======================
 
-This is pretty cool because it means Hy is several things:
+Let's start with the classic::
 
- - A Lisp that feels very Pythonic
- - For Lispers, a great way to use Lisp's crazy powers but in the wide
-   world of Python's libraries (why yes, you now can write a Django
-   application in Lisp!)
- - For Pythonistas, a great way to start exploring Lisp, from the
-   comfort of Python!
- - For everyone: a pleasant language that has a lot of neat ideas!
+    (print "Hy, world!")
 
-Now this tutorial assumes you're running Hy on Python 3. So know things
-are a bit different if you're still using Python 2.
+This program calls the :func:`print` function, which, like all of Python's
+:ref:`built-in functions <py:built-in-funcs>`, is available in Hy.
 
-
-Basic intro to Lisp for Pythonistas
-===================================
-
-Okay, maybe you've never used Lisp before, but you've used Python!
-
-A "hello world" program in Hy is actually super simple. Let's try it:
-
-.. code-block:: clj
-
-   (print "hello world")
-
-See?  Easy!  As you may have guessed, this is the same as the Python
-version of::
-
-  print("hello world")
-
-To add up some super simple math, we could do:
-
-.. code-block:: clj
+All of Python's :ref:`binary and unary operators <py:expressions>` are
+available, too, although ``==`` is spelled ``=`` in deference to Lisp
+tradition. Here's how we'd use the addition operator ``+``::
 
     (+ 1 3)
 
-Which would return 4 and would be the equivalent of:
+This code returns ``4``. It's equivalent to ``1 + 3`` in Python and many other
+languages. Languages in the `Lisp
+<https://en.wikipedia.org/wiki/Lisp_(programming_language)>`_ family, including
+Hy, use a prefix syntax: ``+``, just like ``print`` or ``sqrt``, appears before
+all of its arguments. The call is delimited by parentheses, but the opening
+parenthesis appears before the operator being called instead of after it, so
+instead of ``sqrt(2)``, we write ``(sqrt 2)``. Multiple arguments, such as the
+two integers in ``(+ 1 3)``, are separated by whitespace. Many operators,
+including ``+``, allow more than two arguments: ``(+ 1 2 3)`` is equivalent to
+``1 + 2 + 3``.
 
-.. code-block:: clj
+Here's a more complex example::
 
-   1 + 3
+    (- (* (+ 1 3 88) 2) 8)
 
-What you'll notice is that the first item in the list is the function
-being called and the rest of the arguments are the arguments being
-passed in.  In fact, in Hy (as with most Lisps) we can pass in
-multiple arguments to the plus operator:
+This code returns ``176``. Why? We can see the infix equivalent with the
+command ``echo "(- (* (+ 1 3 88) 2) 8)" | hy2py``, which returns the Python
+code corresponding to the given Hy code, or by passing the ``--spy`` option to
+Hy when starting the REPL, which shows the Python equivalent of each input line
+before the result. The infix equivalent in this case is:
 
-.. code-block:: clj
+.. code-block:: python
 
-   (+ 1 3 55)
+    ((1 + 3 + 88) * 2) - 8
 
-Which would return 59.
+To evaluate this infix expression, you'd of course evaluate the innermost
+parenthesized expression first and work your way outwards. The same goes for
+Lisp. Here's what we'd get by evaluating the above Hy code one step at a time::
 
-Maybe you've heard of Lisp before but don't know much about it.  Lisp
-isn't as hard as you might think, and Hy inherits from Python, so Hy
-is a great way to start learning Lisp.  The main thing that's obvious
-about Lisp is that there's a lot of parentheses.  This might seem
-confusing at first, but it isn't so hard.  Let's look at some simple
-math that's wrapped in a bunch of parentheses that we could enter into
-the Hy interpreter:
+    (- (* (+ 1 3 88) 2) 8)
+    (- (* 92 2) 8)
+    (- 184 8)
+    176
 
-.. code-block:: clj
+The basic unit of Lisp syntax, which is similar to a C or Python expression, is
+the **form**. ``92``, ``*``, and ``(* 92 2)`` are all forms. A Lisp program
+consists of a sequence of forms nested within forms. Forms are typically
+separated from each other by whitespace, but some forms, such as string
+literals (``"Hy, world!"``), can contain whitespace themselves. An
+**expression** is a form enclosed in parentheses; its first child form, called
+the **head**, determines what the expression does, and should generally be a
+function, macro, or special operator. Functions are the most ordinary sort of
+head, whereas macros (described in more detail below) are functions executed at
+compile-time instead and return code to be executed at run-time. Special
+operators are one of :ref:`a fixed set of names <special-forms>` that are
+hard-coded into the compiler, and used to implement everything else.
 
-   (setv result (- (/ (+ 1 3 88) 2) 8))
+Comments start with a ``;`` character and continue till the end of the line. A
+comment is functionally equivalent to whitespace. ::
 
-This would return 38.0  But why?  Well, we could look at the equivalent
-expression in python::
+    (print (** 2 64))   ; Max 64-bit unsigned integer value
 
-  result = ((1 + 3 + 88) / 2) - 8
+Although ``#`` isn't a comment character in Hy, a Hy program can begin with a
+`shebang line <https://en.wikipedia.org/wiki/Shebang_(Unix)>`_, which Hy itself
+will ignore::
 
-If you were to try to figure out how the above were to work in python,
-you'd of course figure out the results by solving each inner
-parenthesis.  That's the same basic idea in Hy.  Let's try this
-exercise first in Python::
+   #!/usr/bin/env hy
+   (print "Make me executable, and run me!")
 
-  result = ((1 + 3 + 88) / 2) - 8
-  # simplified to...
-  result = (92 / 2) - 8
-  # simplified to...
-  result = 46.0 - 8
-  # simplified to...
-  result = 38.0
+Literals
+========
 
-Now let's try the same thing in Hy:
+Hy has :ref:`literal syntax <syntax>` for all of the same data types that
+Python does. Here's an example of Hy code for each type and the Python
+equivalent.
 
-.. code-block:: clj
+==============  ================  =================
+Hy              Python            Type
+==============  ================  =================
+``1``           ``1``             :class:`int`
+``1.2``         ``1.2``           :class:`float`
+``4j``          ``4j``            :class:`complex`
+``True``        ``True``          :class:`bool`
+``None``        ``None``          :class:`NoneType`
+``"hy"``        ``'hy'``          :class:`str`
+``b"hy"``       ``b'hy'``         :class:`bytes`
+``(, 1 2 3)``   ``(1, 2, 3)``     :class:`tuple`
+``[1 2 3]``     ``[1, 2, 3]``     :class:`list`
+``#{1 2 3}``    ``{1, 2, 3}``     :class:`set`
+``{1 2  3 4}``  ``{1: 2, 3: 4}``  :class:`dict`
+==============  ================  =================
 
-   (setv result (- (/ (+ 1 3 88) 2) 8))
-   ; simplified to...
-   (setv result (- (/ 92 2) 8))
-   ; simplified to...
-   (setv result (- 46.0 8))
-   ; simplified to...
-   (setv result 38.0)
+In addition, Hy has a Clojure-style literal syntax for
+:class:`fractions.Fraction`: ``1/3`` is equivalent to ``fractions.Fraction(1,
+3)``.
 
-As you probably guessed, this last expression with ``setv`` means to
-assign the variable "result" to 38.0.
-
-See?  Not too hard!
-
-This is the basic premise of Lisp. Lisp stands for "list
-processing"; this means that the structure of the program is
-actually lists of lists.  (If you're familiar with Python lists,
-imagine the entire same structure as above but with square brackets
-instead, and you'll be able to see the structure above as both a
-program and a data structure.)  This is easier to understand with more
-examples, so let's write a simple Python program, test it, and then
-show the equivalent Hy program::
-
-  def simple_conversation():
-      print("Hello!  I'd like to get to know you.  Tell me about yourself!")
-      name = input("What is your name? ")
-      age = input("What is your age? ")
-      print("Hello " + name + "!  I see you are " + age + " years old.")
-
-  simple_conversation()
-
-If we ran this program, it might go like::
-
-  Hello!  I'd like to get to know you.  Tell me about yourself!
-  What is your name? Gary
-  What is your age? 38
-  Hello Gary!  I see you are 38 years old.
-
-Now let's look at the equivalent Hy program:
-
-.. code-block:: clj
-
-   (defn simple-conversation []
-      (print "Hello!  I'd like to get to know you.  Tell me about yourself!")
-      (setv name (input "What is your name? "))
-      (setv age (input "What is your age? "))
-      (print (+ "Hello " name "!  I see you are "
-                 age " years old.")))
-
-   (simple-conversation)
-
-If you look at the above program, as long as you remember that the
-first element in each list of the program is the function (or
-macro... we'll get to those later) being called and that the rest are
-the arguments, it's pretty easy to figure out what this all means.
-(As you probably also guessed, ``defn`` is the Hy method of defining
-methods.)
-
-Still, lots of people find this confusing at first because there's so
-many parentheses, but there are plenty of things that can help make
-this easier: keep indentation nice and use an editor with parenthesis
-matching (this will help you figure out what each parenthesis pairs up
-with) and things will start to feel comfortable.
-
-There are some advantages to having a code structure that's actually a
-very simple data structure as the core of Lisp is based on.  For one
-thing, it means that your programs are easy to parse and that the
-entire actual structure of the program is very clearly exposed to you.
-(There's an extra step in Hy where the structure you see is converted
-to Python's own representations ... in "purer" Lisps such as Common
-Lisp or Emacs Lisp, the data structure you see in the code and the
-data structure that is executed is much more literally close.)
-
-Another implication of this is macros: if a program's structure is a
-simple data structure, that means you can write code that can write
-code very easily, meaning that implementing entirely new language
-features can be very fast.  Previous to Hy, this wasn't very possible
-for Python programmers ... now you too can make use of macros'
-incredible power (just be careful to not aim them footward)!
-
-
-Hy is a Lisp-flavored Python
-============================
-
-Hy converts to Python's own abstract syntax tree, so you'll soon start
-to find that all the familiar power of python is at your fingertips.
-
-You have full access to Python's data types and standard library in
-Hy.  Let's experiment with this in the hy interpreter::
+The Hy REPL prints output in Python syntax by default::
 
   => [1 2 3]
   [1, 2, 3]
-  => {"dog" "bark"
-  ... "cat" "meow"}
-  {'dog': 'bark', 'cat': 'meow'}
-  => (, 1 2 3)
-  (1, 2, 3)
-  => #{3 1 2}
-  {1, 2, 3}
-  => 1/2
-  Fraction(1, 2)
 
-Notice the last two lines: Hy has a fraction literal like Clojure.
-
-If you start Hy like this (a shell alias might be helpful)::
+But if you start Hy like this (a shell alias might be helpful)::
 
   $ hy --repl-output-fn=hy.contrib.hy-repr.hy-repr
 
-the interactive mode will use :ref:`hy-repr-fn` instead of Python's
-native ``repr`` function to print out values, so you'll see values in
-Hy syntax rather than Python syntax::
+the interactive mode will use :ref:`hy-repr-fn` instead of Python's native
+``repr`` function to print out values, so you'll see values in Hy syntax::
 
   => [1 2 3]
   [1 2 3]
-  => {"dog" "bark"
-  ... "cat" "meow"}
-  {"dog" "bark" "cat" "meow"}
 
-If you are familiar with other Lisps, you may be interested that Hy
-supports the Common Lisp method of quoting:
 
-.. code-block:: clj
+Basic operations
+================
 
-   => '(1 2 3)
-   (1 2 3)
+Set variables with :ref:`setv`::
 
-You also have access to all the built-in types' nice methods::
+    (setv zone-plane 8)
 
-  => (.strip " fooooo   ")
-  "fooooo"
+Access the elements of a list, dictionary, or other data structure with
+:ref:`get`::
 
-What's this?  Yes indeed, this is precisely the same as::
+    (setv fruit ["apple" "banana" "cantaloupe"])
+    (print (get fruit 0))  ; => apple
+    (setv (get fruit 1) "durian")
+    (print (get fruit 1))  ; => durian
 
-  " fooooo   ".strip()
+Access a range of elements in an ordered structure with :ref:`cut`::
 
-That's right---Lisp with dot notation!  If we have this string
-assigned as a variable, we can also do the following:
+    (print (cut "abcdef" 1 4))  ; => bcd
 
-.. code-block:: clj
+Conditional logic can be built with :ref:`if`::
 
-   (setv this-string " fooooo   ")
-   (this-string.strip)
+    (if (= 1 1)
+      (print "Math works. The universe is safe.")
+      (print "Math has failed. The universe is doomed."))
 
-What about conditionals?:
+As in this example, ``if`` is called like ``(if CONDITION THEN ELSE)``. It
+executes and returns the form ``THEN`` if ``CONDITION`` is true (according to
+:class:`bool`) and ``ELSE`` otherwise. If ``ELSE`` is omitted, ``None`` is used
+in its place.
 
-.. code-block:: clj
+What if you want to use more than form in place of the ``THEN`` or ``ELSE``
+clauses, or in place of ``CONDITION``, for that matter? Use the special
+operator :ref:`do` (known more traditionally in Lisp as ``progn``), which
+combines several forms into one, returning the last::
 
-   (if (try-some-thing)
-     (print "this is if true")
-     (print "this is if false"))
+   (if (do (print "Let's check.") (= 1 1))
+     (do
+       (print "Math works.")
+       (print "The universe is safe."))
+     (do
+       (print "Math has failed.")
+       (print "The universe is doomed.")))
 
-As you can tell above, the first argument to ``if`` is a truth test, the
-second argument is the body if true, and the third argument (optional!)
-is if false (ie. ``else``).
-
-If you need to do more complex conditionals, you'll find that you
-don't have ``elif`` available in Hy.  Instead, you should use something
-called ``cond``.  In Python, you might do something like::
-
-  somevar = 33
-  if somevar > 50:
-      print("That variable is too big!")
-  elif somevar < 10:
-      print("That variable is too small!")
-  else:
-      print("That variable is jussssst right!")
-
-In Hy, you would do:
-
-.. code-block:: clj
+For branching on more than one case, try :ref:`cond`::
 
    (setv somevar 33)
    (cond
@@ -281,306 +180,140 @@ In Hy, you would do:
     [True
      (print "That variable is jussssst right!")])
 
-What you'll notice is that ``cond`` switches off between a statement
-that is executed and checked conditionally for true or falseness, and
-then a bit of code to execute if it turns out to be true.  You'll also
-notice that the ``else`` is implemented at the end simply by checking
-for ``True`` -- that's because ``True`` will always be true, so if we get
-this far, we'll always run that one!
+The macro ``(when CONDITION THEN-1 THEN-2 …)`` is shorthand for ``(if CONDITION
+(do THEN-1 THEN-2 …))``. ``unless`` works the same as ``when``, but inverts the
+condition with ``not``.
 
-You might notice above that if you have code like:
+Hy's basic loops are :ref:`while` and :ref:`for`::
 
-.. code-block:: clj
+    (setv x 3)
+    (while (> x 0)
+      (print x)
+      (setv x (- x 1)))  ; => 3 2 1
 
-   (if some-condition
-     (body-if-true)
-     (body-if-false))
+    (for [x [1 2 3]]
+      (print x))         ; => 1 2 3
 
-But wait!  What if you want to execute more than one statement in the
-body of one of these?
+A more functional way to iterate is provided by the comprehension forms such as
+:ref:`lfor`. Whereas ``for`` always returns ``None``, ``lfor`` returns a list
+with one element per iteration. ::
 
-You can do the following:
+    (print (lfor  x [1 2 3]  (* x 2)))  ; => [2, 4, 6]
 
-.. code-block:: clj
 
-   (if (try-some-thing)
-     (do
-       (print "this is if true")
-       (print "and why not, let's keep talking about how true it is!"))
-     (print "this one's still simply just false"))
+Functions, classes, and modules
+===============================
 
-You can see that we used ``do`` to wrap multiple statements.  If you're
-familiar with other Lisps, this is the equivalent of ``progn``
-elsewhere.
+Define named functions with :ref:`defn`::
 
-Comments start with semicolons:
+    (defn fib [n]
+      (if (< n 2)
+        n
+        (+ (fib (- n 1)) (fib (- n 2)))))
+    (print (fib 8))  ; => 21
 
-.. code-block:: clj
+Define anonymous functions with :ref:`fn`::
 
-  (print "this will run")
-  ; (print "but this will not")
-  (+ 1 2 3)  ; we'll execute the addition, but not this comment!
+    (print (list (filter (fn [x] (% x 2)) (range 10))))
+      ; => [1, 3, 5, 7, 9]
 
-Hashbang (``#!``) syntax is supported:
+Special symbols in the parameter list of ``defn`` or ``fn`` allow you to
+indicate optional arguments, provide default values, and collect unlisted
+arguments::
 
-.. code-block:: clj
+    (defn test [a b &optional c [d "x"] &rest e]
+      [a b c d e])
+    (print (test 1 2))            ; => [1, 2, None, 'x', ()]
+    (print (test 1 2 3 4 5 6 7))  ; => [1, 2, 3, 4, (5, 6, 7)]
 
-   #! /usr/bin/env hy
-   (print "Make me executable, and run me!")
+Set a function parameter by name with a ``:keyword``::
 
-Looping is not hard but has a kind of special structure.  In Python,
-we might do::
+    (test 1 2 :d "y")             ; => [1, 2, None, 'y', ()]
 
-  for i in range(10):
-      print("'i' is now at " + str(i))
+Define classes with :ref:`defclass`::
 
-The equivalent in Hy would be:
-
-.. code-block:: clj
-
-  (for [i (range 10)]
-    (print (+ "'i' is now at " (str i))))
-
-Python's collections indexes and slices are implemented
-by the ``get`` and ``cut`` built-in:
-
-.. code-block:: clj
-
-  (setv array [0 1 2])
-  (get array 1)
-  (cut array -3 -1)
-
-which is equivalent to::
-
-  array[1]
-  array[-3:-1]
-
-You can also import and make use of various Python libraries.  For
-example:
-
-.. code-block:: clj
-
-   (import os)
-
-   (if (os.path.isdir "/tmp/somedir")
-     (os.mkdir "/tmp/somedir/anotherdir")
-     (print "Hey, that path isn't there!"))
-
-Python's context managers (``with`` statements) are used like this:
-
-.. code-block:: clj
-
-     (with [f (open "/tmp/data.in")]
-       (print (.read f)))
-
-which is equivalent to::
-
-  with open("/tmp/data.in") as f:
-      print(f.read())
-
-And yes, we do have List comprehensions!  In Python you might do::
-
-  odds_squared = [
-    pow(num, 2)
-    for num in range(100)
-    if num % 2 == 1]
-
-In Hy, you could do these like:
-
-.. code-block:: clj
-
-  (setv odds-squared
-    (lfor
-      num (range 100)
-      :if (= (% num 2) 1)
-      (pow num 2)))
-
-.. code-block:: clj
-
-  ; And, an example stolen shamelessly from a Clojure page:
-  ; Let's list all the blocks of a Chessboard:
-
-  (lfor
-    x (range 8)
-    y "ABCDEFGH"
-    (, x y))
-
-  ; [(0, 'A'), (0, 'B'), (0, 'C'), (0, 'D'), (0, 'E'), (0, 'F'), (0, 'G'), (0, 'H'),
-  ;  (1, 'A'), (1, 'B'), (1, 'C'), (1, 'D'), (1, 'E'), (1, 'F'), (1, 'G'), (1, 'H'),
-  ;  (2, 'A'), (2, 'B'), (2, 'C'), (2, 'D'), (2, 'E'), (2, 'F'), (2, 'G'), (2, 'H'),
-  ;  (3, 'A'), (3, 'B'), (3, 'C'), (3, 'D'), (3, 'E'), (3, 'F'), (3, 'G'), (3, 'H'),
-  ;  (4, 'A'), (4, 'B'), (4, 'C'), (4, 'D'), (4, 'E'), (4, 'F'), (4, 'G'), (4, 'H'),
-  ;  (5, 'A'), (5, 'B'), (5, 'C'), (5, 'D'), (5, 'E'), (5, 'F'), (5, 'G'), (5, 'H'),
-  ;  (6, 'A'), (6, 'B'), (6, 'C'), (6, 'D'), (6, 'E'), (6, 'F'), (6, 'G'), (6, 'H'),
-  ;  (7, 'A'), (7, 'B'), (7, 'C'), (7, 'D'), (7, 'E'), (7, 'F'), (7, 'G'), (7, 'H')]
-
-
-Python has support for various fancy argument and keyword arguments.
-In Python we might see::
-
-  >>> def optional_arg(pos1, pos2, keyword1=None, keyword2=42):
-  ...   return [pos1, pos2, keyword1, keyword2]
-  ...
-  >>> optional_arg(1, 2)
-  [1, 2, None, 42]
-  >>> optional_arg(1, 2, 3, 4)
-  [1, 2, 3, 4]
-  >>> optional_arg(keyword1=1, pos2=2, pos1=3, keyword2=4)
-  [3, 2, 1, 4]
-
-The same thing in Hy::
-
-  => (defn optional-arg [pos1 pos2 &optional keyword1 [keyword2 42]]
-  ...  [pos1 pos2 keyword1 keyword2])
-  => (optional-arg 1 2)
-  [1 2 None 42]
-  => (optional-arg 1 2 3 4)
-  [1 2 3 4]
-
-You can call keyword arguments like this::
-
-  => (optional-arg :keyword1 1
-  ...              :pos2 2
-  ...              :pos1 3
-  ...              :keyword2 4)
-  [3, 2, 1, 4]
-
-You can unpack arguments with the syntax ``#* args`` and ``#** kwargs``,
-similar to `*args` and `**kwargs` in Python::
-
-  => (setv args [1 2])
-  => (setv kwargs {"keyword2" 3
-  ...              "keyword1" 4})
-  => (optional-arg #* args #** kwargs)
-  [1, 2, 4, 3]
-
-Hy also supports ``*args`` and ``**kwargs`` in parameter lists.  In Python::
-
-  def some_func(foo, bar, *args, **kwargs):
-    import pprint
-    pprint.pprint((foo, bar, args, kwargs))
-
-The Hy equivalent:
-
-.. code-block:: clj
-
-  (defn some-func [foo bar &rest args &kwargs kwargs]
-    (import pprint)
-    (pprint.pprint (, foo bar args kwargs)))
-
-Finally, of course we need classes!  In Python, we might have a class
-like::
-
-  class FooBar(object):
-      """
-      Yet Another Example Class
-      """
-      def __init__(self, x):
-          self.x = x
-
-      def get_x(self):
-          """
-          Return our copy of x
-          """
-          return self.x
-
-And we might use it like::
-
-  bar = FooBar(1)
-  print(bar.get_x())
-
-
-In Hy:
-
-.. code-block:: clj
-
-  (defclass FooBar [object]
-    "Yet Another Example Class"
-
-    (defn --init-- [self x]
+    (defclass FooBar []
+      (defn __init__ [self x]
         (setv self.x x))
-
       (defn get-x [self]
-      "Return our copy of x"
         self.x))
 
-And we can use it like:
+Here we create a new instance ``fb`` of ``FooBar`` and access its attributes by
+various means::
 
-.. code-block:: clj
+    (setv fb (FooBar 15))
+    (print fb.x)         ; => 15
+    (print (. fb x))     ; => 15
+    (print (.get-x fb))  ; => 15
+    (print (fb.get-x))   ; => 15
 
-  (setv bar (FooBar 1))
-  (print (bar.get-x))
+Note that syntax like ``fb.x`` and ``fb.get-x`` only works when the object
+being invoked (``fb``, in this case) is a simple variable name. To get an
+attribute or call a method of an arbitrary form ``FORM``, you must use the
+syntax ``(. FORM x)`` or ``(.get-x FORM)``.
 
-Or using the leading dot syntax!
+Access an external module, whether written in Python or Hy, with
+:ref:`import`::
 
-.. code-block:: clj
+    (import math)
+    (print (math.sqrt 2))  ; => 1.4142135623730951
 
-  (print (.get-x (FooBar 1)))
-
-
-You can also do class-level attributes.  In Python::
-
-  class Customer(models.Model):
-      name = models.CharField(max_length=255)
-      address = models.TextField()
-      notes = models.TextField()
-
-In Hy:
-
-.. code-block:: clj
-
-  (defclass Customer [models.Model]
-    [name (models.CharField :max-length 255})
-     address (models.TextField)
-     notes (models.TextField)])
+Python can import a Hy module like any other module so long as Hy itself has
+been imported first, which, of course, must have already happened if you're
+running a Hy program.
 
 Macros
 ======
 
-One really powerful feature of Hy are macros. They are small functions that are
-used to generate code (or data). When program written in Hy is started, the
-macros are executed and their output is placed in the program source. After this,
-the program starts executing normally. Very simple example:
+Macros are the basic metaprogramming tool of Lisp. A macro is a function that
+is called at compile time (i.e., when a Hy program is being translated to
+Python :mod:`ast` objects) and returns code, which becomes part of the final
+program. Here's a simple example::
 
-.. code-block:: clj
+    (print "Executing")
+    (defmacro m []
+      (print "Now for a slow computation")
+      (setv x (% (** 10 10 7) 3))
+      (print "Done computing")
+      x)
+    (print "Value:" (m))
+    (print "Done executing")
 
-  => (defmacro hello [person]
-  ...  `(print "Hello there," ~person))
-  => (hello "Tuukka")
-  Hello there, Tuukka
+If you run this program twice in a row, you'll see this::
 
-The thing to notice here is that hello macro doesn't output anything on
-screen. Instead it creates piece of code that is then executed and prints on
-screen. This macro writes a piece of program that looks like this (provided that
-we used "Tuukka" as parameter):
+    $ hy example.hy
+    Now for a slow computation
+    Done computing
+    Executing
+    Value: 1
+    Done executing
+    $ hy example.hy
+    Executing
+    Value: 1
+    Done executing
 
-.. code-block:: clj
+The slow computation is performed while compiling the program on its first
+invocation. Only after the whole program is compiled does normal execution
+begin from the top, printing "Executing". When the program is called a second
+time, it is run from the previously compiled bytecode, which is equivalent to
+simply::
 
-  (print "Hello there," "Tuukka")
+    (print "Executing")
+    (print "Value:" 1)
+    (print "Done executing")
 
-We can also manipulate code with macros:
-
-.. code-block:: clj
-
-  => (defmacro rev [code]
-  ...  (setv op (last code) params (list (butlast code)))
-  ...  `(~op ~@params))
-  => (rev (1 2 3 +))
-  6
-
-The code that was generated with this macro just switched around some of the
-elements, so by the time program started executing, it actually reads:
-
-.. code-block:: clj
-
-  (+ 1 2 3)
+Our macro ``m`` has an especially simple return value, an integer, which at
+compile-time is converted to an integer literal. In general, macros can return
+arbitrary Hy forms to be executed as code. There are several special operators
+and macros that make it easy to construct forms programmatically, such as
+:ref:`quote` (``'``), :ref:`quasiquote` (`````), :ref:`unquote` (``~``), and
+:ref:`defmacro!`. The previous chapter has :ref:`a simple example <do-while>`
+of using ````` and ``~`` to define a new control construct ``do-while``.
 
 Sometimes it's nice to be able to call a one-parameter macro without
-parentheses. Tag macros allow this. The name of a tag macro is typically
-one character long, but since Hy operates well with Unicode, we aren't running
-out of characters that soon:
-
-.. code-block:: clj
+parentheses. Tag macros allow this. The name of a tag macro is often just one
+character long, but since Hy allows most Unicode characters in the name of a
+macro (or ordinary variable), you won't out of characters soon. ::
 
   => (deftag ↻ [code]
   ...  (setv op (last code) params (list (butlast code)))
@@ -588,106 +321,23 @@ out of characters that soon:
   => #↻(1 2 3 +)
   6
 
-Macros are useful when one wishes to extend Hy or write their own
-language on top of that. Many features of Hy are macros, like ``when``,
-``cond`` and ``->``.
-
-What if you want to use a macro that's defined in a different
-module? The special form ``import`` won't help, because it merely
-translates to a Python ``import`` statement that's executed at
-run-time, and macros are expanded at compile-time, that is,
-during the translate from Hy to Python. Instead, use ``require``,
-which imports the module and makes macros available at
-compile-time. ``require`` uses the same syntax as ``import``.
-
-.. code-block:: clj
+What if you want to use a macro that's defined in a different module?
+``import`` won't help, because it merely translates to a Python ``import``
+statement that's executed at run-time, and macros are expanded at compile-time,
+that is, during the translation from Hy to Python. Instead, use :ref:`require`,
+which imports the module and makes macros available at compile-time.
+``require`` uses the same syntax as ``import``. ::
 
    => (require tutorial.macros)
    => (tutorial.macros.rev (1 2 3 +))
    6
 
-Hy <-> Python interop
-=====================
+Next steps
+==========
 
-Using Hy from Python
---------------------
+You now know enough to be dangerous with Hy. You may now smile villainously and
+sneak off to your Hydeaway to do unspeakable things.
 
-You can use Hy modules in Python!
-
-If you save the following in ``greetings.hy``:
-
-.. code-block:: clj
-
-    (defn greet [name] (print "hello from hy," name))
-
-Then you can use it directly from Python, by importing Hy before importing
-the module. In Python::
-
-    import hy
-    import greetings
-
-    greetings.greet("Foo")
-
-Using Python from Hy
---------------------
-
-You can also use any Python module in Hy!
-
-If you save the following in ``greetings.py`` in Python::
-
-    def greet(name):
-        print("hello, %s" % (name))
-
-You can use it in Hy (see :ref:`import`):
-
-.. code-block:: clj
-
-    (import greetings)
-    (.greet greetings "foo")
-
-More information on :doc:`../language/interop`.
-
-
-Protips!
-========
-
-Hy also features something known as the "threading macro", a really neat
-feature of Clojure's. The "threading macro" (written as ``->``) is used
-to avoid deep nesting of expressions.
-
-The threading macro inserts each expression into the next expression's first
-argument place.
-
-Let's take the classic:
-
-.. code-block:: clj
-
-    (require [hy.contrib.loop [loop]])
-
-    (loop (print (eval (read))))
-
-Rather than write it like that, we can write it as follows:
-
-.. code-block:: clj
-
-    (require [hy.contrib.loop [loop]])
-
-    (-> (read) (eval) (print) (loop))
-
-Now, using `python-sh <http://amoffat.github.com/sh/>`_, we can show
-how the threading macro (because of python-sh's setup) can be used like
-a pipe:
-
-.. code-block:: clj
-
-    => (import [sh [cat grep wc]])
-    => (-> (cat "/usr/share/dict/words") (grep "-E" "^hy") (wc "-l"))
-    210
-
-Which, of course, expands out to:
-
-.. code-block:: clj
-
-    (wc (grep (cat "/usr/share/dict/words") "-E" "^hy") "-l")
-
-Much more readable, no? Use the threading macro!
+Refer to Python's documention for the details of Python semantics, and the rest
+of this manual for Hy-specific features. Like Hy itself, the manual is
+incomplete, but :ref:`contributions <hacking>` are always welcome.

docs/whyhy.rst

@@ -0,0 +1,142 @@
+=======
+Why Hy?
+=======
+
+Hy is a multi-paradigm general-purpose programming language in the `Lisp family
+<https://en.wikipedia.org/wiki/Lisp_(programming_language)>`_. It's implemented
+as a kind of alternative syntax for Python. Compared to Python, Hy offers a
+variety of extra features, generalizations, and syntactic simplifications, as
+would be expected of a Lisp. Compared to other Lisps, Hy provides direct access
+to Python's built-ins and third-party Python libraries, while allowing you to
+freely mix imperative, functional, and object-oriented styles of programming.
+
+
+Hy versus Python
+----------------
+
+The first thing a Python programmer will notice about Hy is that it has Lisp's
+traditional parenthesis-heavy prefix syntax in place of Python's C-like infix
+syntax. For example, ``print("The answer is", 2 + object.method(arg))`` could
+be written ``(print "The answer is" (+ 2 (.method object arg)))`` in Hy.
+Consequently, Hy is free-form: structure is indicated by parentheses rather
+than whitespace, making it convenient for command-line use.
+
+As in other Lisps, the value of a simplistic syntax is that it facilitates
+Lisp's signature feature: `metaprogramming
+<https://en.wikipedia.org/wiki/Metaprogramming>`_ through macros, which are
+functions that manipulate code objects at compile time to produce new code
+objects, which are then executed as if they had been part of the original code.
+In fact, Hy allows arbitrary computation at compile-time. For example, here's a
+simple macro that implements a C-style do-while loop, which executes its body
+for as long as the condition is true, but at least once.
+
+.. _do-while:
+
+::
+
+    (defmacro do-while [condition &rest body]
+      `(do
+        ~body
+        (while ~condition
+          ~body)))
+
+    (setv x 0)
+    (do-while x
+      (print "This line is executed once."))
+
+Hy also removes Python's restrictions on mixing expressions and statements,
+allowing for more direct and functional code. For example, Python doesn't allow
+:ref:`with <py:with>` blocks, which close a resource once you're done using it,
+to return values. They can only execute a set of statements:
+
+.. code-block:: python
+
+    with open("foo") as o:
+       f1 = o.read()
+    with open("bar") as o:
+       f2 = o.read()
+    print(len(f1) + len(f2))
+
+In Hy, :ref:`with` returns the value of its last body form, so you can use it
+like an ordinary function call::
+
+   (print (+
+     (len (with [o (open "foo")] (.read o))
+     (len (with [o (open "bar")] (.read o))))))
+
+To be even more concise, you can put a ``with`` form in a :ref:`generator
+expression <gfor>`::
+
+   (print (sum (gfor
+     filename ["foo" "bar"]
+     (len (with [o (open filename)] (.read o))))))
+
+Finally, Hy offers several generalizations to Python's binary operators.
+Operators can be given more than two arguments (e.g., ``(+ 1 2 3)``), including
+augmented assignment operators (e.g., ``(+= x 1 2 3)``). They are also provided
+as ordinary first-class functions of the same name, allowing them to be passed
+to higher-order functions: ``(sum xs)`` could be written ``(reduce + xs)``.
+
+The Hy compiler works by reading Hy source code into Hy model objects and
+compiling the Hy model objects into Python abstract syntax tree (:py:mod:`ast`)
+objects. Python AST objects can then be compiled and run by Python itself,
+byte-compiled for faster execution later, or rendered into Python source code.
+You can even :ref:`mix Python and Hy code in the same project, or even the same
+file,<interop>` which can be a good way to get your feet wet in Hy.
+
+
+Hy versus other Lisps
+---------------------
+
+At run-time, Hy is essentially Python code. Thus, while Hy's design owes a lot
+to `Clojure <https://clojure.org>`_, it is more tightly coupled to Python than
+Clojure is to Java; a better analogy is `CoffeeScript's
+<https://coffeescript.org>`_ relationship to JavaScript. Python's built-in
+:ref:`functions <py:built-in-funcs>` and :ref:`data structures
+<py:bltin-types>` are directly available::
+
+    (print (int "deadbeef" :base 16))  ; 3735928559
+    (print (len [1 10 100]))           ; 3
+
+The same goes for third-party Python libraries from `PyPI <https://pypi.org>`_
+and elsewhere. Here's a tiny `CherryPy <https://cherrypy.org>`_ web application
+in Hy::
+
+    (import cherrypy)
+
+    (defclass HelloWorld []
+      #@(cherrypy.expose (defn index [self]
+        "Hello World!")))
+
+    (cherrypy.quickstart (HelloWorld))
+
+You can even run Hy on `PyPy <https://pypy.org>`_ for a particularly speedy
+Lisp.
+
+Like all Lisps, Hy is `homoiconic
+<https://en.wikipedia.org/wiki/Homoiconicity>`_. Its syntax is represented not
+with cons cells or with Python's basic data structures, but with simple
+subclasses of Python's basic data structures called :ref:`models <models>`.
+Using models in place of plain ``list``\s, ``set``\s, and so on has two
+purposes: models can keep track of their line and column numbers for the
+benefit of error messages, and models can represent syntactic features that the
+corresponding primitive type can't, such as the order in which elements appear
+in a set literal. However, models can be concatenated and indexed just like
+plain lists, and you can return ordinary Python types from a macro or give them
+to ``eval`` and Hy will automatically promote them to models.
+
+Hy takes much of its semantics from Python. For example, Hy is a Lisp-1 because
+Python functions use the same namespace as objects that aren't functions. In
+general, any Python code should be possible to literally translate to Hy. At
+the same time, Hy goes to some lengths to allow you to do typical Lisp things
+that aren't straightforward in Python. For example, Hy provides the
+aforementioned mixing of statements and expressions, :ref:`name mangling
+<mangling>` that transparently converts symbols with names like ``valid?`` to
+Python-legal identifiers, and a :ref:`let` macro to provide block-level scoping
+in place of Python's usual function-level scoping.
+
+Overall, Hy, like Common Lisp, is intended to be an unopinionated big-tent
+language that lets you do what you want. If you're interested in a more
+small-and-beautiful approach to Lisp, in the style of Scheme, check out
+`Hissp <https://github.com/gilch/hissp>`_, another Lisp embedded in Python
+that was created by a Hy developer.

get_version.py

@@ -5,6 +5,9 @@ import os, subprocess, runpy
 os.chdir(os.path.split(os.path.abspath(__file__))[0])
 VERSIONFILE = os.path.join("hy", "version.py")
 
+if "HY_VERSION" in os.environ:
+    __version__ = os.environ["HY_VERSION"]
+else:
     try:
         __version__ = (subprocess.check_output
                        (["git", "describe", "--tags", "--dirty"])

hy/__main__.py

@@ -11,5 +11,5 @@ import sys
 
 if len(sys.argv) > 1:
     sys.argv.pop(0)
-    imp.load_source("__main__", sys.argv[0])
+    hy.importer._import_from_path('__main__', sys.argv[0])
     sys.exit(0)  # right?

hy/_compat.py

@@ -1,109 +1,16 @@
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 
-try:
-    import __builtin__ as builtins
-except ImportError:
-    import builtins  # NOQA
-import sys, keyword, textwrap
+import sys
 
-PY3 = sys.version_info[0] >= 3
 PY36 = sys.version_info >= (3, 6)
 PY37 = sys.version_info >= (3, 7)
 PY38 = sys.version_info >= (3, 8)
 
-# The value of UCS4 indicates whether Unicode strings are stored as UCS-4.
-# It is always true on Pythons >= 3.3, which use USC-4 on all systems.
-UCS4 = sys.maxunicode == 0x10FFFF
-
-str_type     = str   if PY3 else unicode      # NOQA
-bytes_type   = bytes if PY3 else str          # NOQA
-long_type    = int   if PY3 else long         # NOQA
-string_types = str   if PY3 else basestring   # NOQA
-
-#
-# Inspired by the same-named `six` functions.
-#
-if PY3:
-    raise_src = textwrap.dedent('''
-    def raise_from(value, from_value):
-        raise value from from_value
-    ''')
 
 def reraise(exc_type, value, traceback=None):
     try:
         raise value.with_traceback(traceback)
     finally:
         traceback = None
-
-    code_obj_args = ['argcount', 'kwonlyargcount', 'nlocals', 'stacksize',
-                     'flags', 'code', 'consts', 'names', 'varnames',
-                     'filename', 'name', 'firstlineno', 'lnotab', 'freevars',
-                     'cellvars']
-else:
-    def raise_from(value, from_value=None):
-        raise value
-
-    raise_src = textwrap.dedent('''
-    def reraise(exc_type, value, traceback=None):
-        try:
-            raise exc_type, value, traceback
-        finally:
-            traceback = None
-    ''')
-
-    code_obj_args = ['argcount', 'nlocals', 'stacksize', 'flags', 'code',
-                     'consts', 'names', 'varnames', 'filename', 'name',
-                     'firstlineno', 'lnotab', 'freevars', 'cellvars']
-
-raise_code = compile(raise_src, __file__, 'exec')
-exec(raise_code)
-
-
-def rename_function(func, new_name):
-    """Creates a copy of a function and [re]sets the name at the code-object
-    level.
-    """
-    c = func.__code__
-    new_code = type(c)(*[getattr(c, 'co_{}'.format(a))
-                         if a != 'name' else str(new_name)
-                         for a in code_obj_args])
-
-    _fn = type(func)(new_code, func.__globals__, str(new_name),
-                     func.__defaults__, func.__closure__)
-    _fn.__dict__.update(func.__dict__)
-
-    return _fn
-
-
-def isidentifier(x):
-    if x in ('True', 'False', 'None', 'print'):
-        # `print` is special-cased here because Python 2's
-        # keyword.iskeyword will count it as a keyword, but we
-        # use the __future__ feature print_function, which makes
-        # it a non-keyword.
-        return True
-    if keyword.iskeyword(x):
-        return False
-    if PY3:
-        return x.isidentifier()
-    if x.rstrip() != x:
-        return False
-    import tokenize as T
-    from io import StringIO
-    try:
-        tokens = list(T.generate_tokens(StringIO(x).readline))
-    except (T.TokenError, IndentationError):
-        return False
-    # Some versions of Python 2.7 (including one that made it into
-    # Ubuntu 18.10) have a Python 3 backport that adds a NEWLINE
-    # token. Remove it if it's present.
-    # https://bugs.python.org/issue33899
-    tokens = [t for t in tokens if t[0] != T.NEWLINE]
-    return len(tokens) == 2 and tokens[0][0] == T.NAME
-
-try:
-    FileNotFoundError = FileNotFoundError
-except NameError:
-    FileNotFoundError = IOError

hy/cmdline.py

@@ -1,9 +1,12 @@
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 
 from __future__ import print_function
 
+import colorama
+colorama.init()
+
 import argparse
 import code
 import ast
@@ -19,6 +22,7 @@ import time
 import linecache
 import hashlib
 import codeop
+import builtins
 
 import astor.code_gen
 
@@ -35,7 +39,6 @@ from hy.importer import runhy
 from hy.completer import completion, Completer
 from hy.macros import macro, require
 from hy.models import HyExpression, HyString, HySymbol
-from hy._compat import builtins, PY3, FileNotFoundError
 
 
 sys.last_type = None
@@ -59,9 +62,19 @@ class HyQuitter(object):
             pass
         raise SystemExit(code)
 
+class HyHelper(object):
+    def __repr__(self):
+        return ("Use (help) for interactive help, or (help object) for help "
+                "about object.")
+
+    def __call__(self, *args, **kwds):
+        import pydoc
+        return pydoc.help(*args, **kwds)
+
 
 builtins.quit = HyQuitter('quit')
 builtins.exit = HyQuitter('exit')
+builtins.help = HyHelper()
 
 @contextmanager
 def extend_linecache(add_cmdline_cache):
@@ -256,7 +269,7 @@ class HyREPL(code.InteractiveConsole, object):
                 module, f = '.'.join(parts[:-1]), parts[-1]
                 self.output_fn = getattr(importlib.import_module(module), f)
             else:
-                self.output_fn = __builtins__[mangle(output_fn)]
+                self.output_fn = getattr(builtins, mangle(output_fn))
 
         # Pre-mangle symbols for repl recent results: *1, *2, *3
         self._repl_results_symbols = [mangle("*{}".format(i + 1)) for i in range(3)]
@@ -661,7 +674,7 @@ def hy2py_main():
     if options.with_source:
         # need special printing on Windows in case the
         # codepage doesn't support utf-8 characters
-        if PY3 and platform.system() == "Windows":
+        if platform.system() == "Windows":
             for h in hst:
                 try:
                     print(h)
@@ -676,7 +689,7 @@ def hy2py_main():
         _ast = hy_compile(hst, '__main__', filename=filename, source=source)
 
     if options.with_ast:
-        if PY3 and platform.system() == "Windows":
+        if platform.system() == "Windows":
             _print_for_windows(astor.dump_tree(_ast))
         else:
             print(astor.dump_tree(_ast))
@@ -684,7 +697,7 @@ def hy2py_main():
         print()
 
     if not options.without_python:
-        if PY3 and platform.system() == "Windows":
+        if platform.system() == "Windows":
             _print_for_windows(astor.code_gen.to_source(_ast))
         else:
             print(astor.code_gen.to_source(_ast))

hy/completer.py

@@ -1,4 +1,4 @@
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 
@@ -6,10 +6,10 @@ import contextlib
 import os
 import re
 import sys
+import builtins
 
 import hy.macros
 import hy.compiler
-from hy._compat import builtins, string_types
 
 
 docomplete = True
@@ -78,7 +78,7 @@ class Completer(object):
         matches = []
         for p in self.path:
             for k in p.keys():
-                if isinstance(k, string_types):
+                if isinstance(k, str):
                     k = k.replace("_", "-")
                     if k.startswith(text):
                         matches.append(k)
@@ -89,7 +89,7 @@ class Completer(object):
         matches = []
         for p in self.tag_path:
             for k in p.keys():
-                if isinstance(k, string_types):
+                if isinstance(k, str):
                     if k.startswith(text):
                         matches.append("#{}".format(k))
         return matches
@@ -123,7 +123,7 @@ def completion(completer=None):
         try:
             readline.read_history_file(history)
         except IOError:
-            open(history, 'a').close()
+            pass
 
         readline.parse_and_bind(readline_bind)
 
@@ -131,4 +131,7 @@ def completion(completer=None):
         yield
     finally:
         if docomplete:
+            try:
                 readline.write_history_file(history)
+            except IOError:
+                pass

hy/contrib/botsbuildbots.hy

@@ -1,4 +1,4 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 

hy/contrib/hy_repr.hy

@@ -1,4 +1,4 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
@@ -7,7 +7,7 @@
   re
   datetime
   collections
-  [hy._compat [PY3 PY36 str-type bytes-type long-type]]
+  [hy._compat [PY36]]
   [hy.models [HyObject HyExpression HySymbol HyKeyword HyInteger HyFloat HyComplex HyList HyDict HySet HyString HyBytes]])
 
 (try
@@ -84,10 +84,10 @@
     (+ "(" (-cat x) ")"))))
 
 (hy-repr-register [HySymbol HyKeyword] str)
-(hy-repr-register [str-type bytes-type] (fn [x]
+(hy-repr-register [str bytes] (fn [x]
   (setv r (.lstrip (-base-repr x) "ub"))
   (+
-    (if (instance? bytes-type x) "b" "")
+    (if (instance? bytes x) "b" "")
     (if (.startswith "\"" r)
       ; If Python's built-in repr produced a double-quoted string, use
       ; that.
@@ -96,10 +96,6 @@
       ; convert it.
       (+ "\"" (.replace (cut r 1 -1) "\"" "\\\"") "\"")))))
 (hy-repr-register bool str)
-(if (not PY3) (hy-repr-register int (fn [x]
-  (.format "(int {})" (-base-repr x)))))
-(if (not PY3) (hy-repr-register long_type (fn [x]
-  (.rstrip (-base-repr x) "L"))))
 (hy-repr-register float (fn [x]
   (if
     (isnan x)  "NaN"
@@ -131,7 +127,7 @@
     (-repr-time-innards x))))
 (defn -repr-time-innards [x]
   (.rstrip (+ " " (.join " " (filter identity [
-    (if x.microsecond (str-type x.microsecond))
+    (if x.microsecond (str x.microsecond))
     (if (not (none? x.tzinfo)) (+ ":tzinfo " (hy-repr x.tzinfo)))
     (if (and PY36 (!= x.fold 0)) (+ ":fold " (hy-repr x.fold)))])))))
 (defn -strftime-0 [x fmt]
@@ -148,7 +144,7 @@
     (hy-repr (dict x)))))
 
 (for [[types fmt] (partition [
-    list "[...]"
+    [list HyList] "[...]"
     [set HySet] "#{...}"
     frozenset "(frozenset #{...})"
     dict-keys "(dict-keys [...])"

hy/contrib/loop.hy

@@ -1,5 +1,5 @@
 ;;; Hy tail-call optimization
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 

hy/contrib/multi.hy

@@ -1,96 +0,0 @@
-;; Hy Arity-overloading
-;; Copyright 2019 the authors.
-;; This file is part of Hy, which is free software licensed under the Expat
-;; license. See the LICENSE.
-
-(import [collections [defaultdict]]
-        [hy [HyExpression HyList HyString]])
-
-(defclass MultiDispatch [object] [
-
-  _fns (defaultdict dict)
-
-  __init__ (fn [self f]
-    (setv self.f f)
-    (setv self.__doc__ f.__doc__)
-    (unless (in f.__name__ (.keys (get self._fns f.__module__)))
-      (setv (get self._fns f.__module__ f.__name__) {}))
-    (setv values f.__code__.co_varnames)
-    (setv (get self._fns f.__module__ f.__name__ values) f))
-
-  fn? (fn [self v args kwargs]
-    "Compare the given (checked fn) to the called fn"
-    (setv com (+ (list args) (list (.keys kwargs))))
-    (and
-      (= (len com) (len v))
-      (.issubset (frozenset (.keys kwargs)) com)))
-
-  __call__ (fn [self &rest args &kwargs kwargs]
-    (setv func None)
-    (for [[i f] (.items (get self._fns self.f.__module__ self.f.__name__))]
-      (when (.fn? self i args kwargs)
-        (setv func f)
-        (break)))
-    (if func
-      (func #* args #** kwargs)
-      (raise (TypeError "No matching functions with this signature"))))])
-
-(defn multi-decorator [dispatch-fn]
-  (setv inner (fn [&rest args &kwargs kwargs]
-                (setv dispatch-key (dispatch-fn #* args #** kwargs))
-                (if (in dispatch-key inner.--multi--)
-                  ((get inner.--multi-- dispatch-key) #* args #** kwargs)
-                  (inner.--multi-default-- #* args #** kwargs))))
-  (setv inner.--multi-- {})
-  (setv inner.--doc-- dispatch-fn.--doc--)
-  (setv inner.--multi-default-- (fn [&rest args &kwargs kwargs] None))
-  inner)
-
-(defn method-decorator [dispatch-fn &optional [dispatch-key None]]
-  (setv apply-decorator
-        (fn [func]
-          (if (is dispatch-key None)
-            (setv dispatch-fn.--multi-default-- func)
-           (assoc dispatch-fn.--multi-- dispatch-key func))
-          dispatch-fn))
-  apply-decorator)
- 
-(defmacro defmulti [name params &rest body]
-  `(do (import [hy.contrib.multi [multi-decorator]])
-       (with-decorator multi-decorator
-         (defn ~name ~params ~@body))))
-
-(defmacro defmethod [name multi-key params &rest body]
-  `(do (import [hy.contrib.multi [method-decorator]])
-       (with-decorator (method-decorator ~name ~multi-key)
-         (defn ~name ~params ~@body))))
-
-(defmacro default-method [name params &rest body]
-  `(do (import [hy.contrib.multi [method-decorator]])
-       (with-decorator (method-decorator ~name)
-         (defn ~name ~params ~@body))))
-
-(defn head-tail [l]
-  (, (get l 0) (cut l 1)))
-
-(defmacro defn [name &rest bodies]
-  (setv arity-overloaded? (fn [bodies]
-                            (if (isinstance (first bodies) HyString)
-                                (arity-overloaded? (rest bodies))
-                                (isinstance (first bodies) HyExpression))))
-
-  (if (arity-overloaded? bodies)
-    (do
-     (setv comment (HyString))
-     (if (= (type (first bodies)) HyString)
-       (setv [comment bodies] (head-tail bodies)))
-     (setv ret `(do))
-     (.append ret '(import [hy.contrib.multi [MultiDispatch]]))
-     (for [body bodies]
-       (setv [let-binds body] (head-tail body))
-       (.append ret 
-                `(with-decorator MultiDispatch (defn ~name ~let-binds ~comment ~@body))))
-     ret)
-    (do
-     (setv [lambda-list body] (head-tail bodies))
-     `(setv ~name (fn* ~lambda-list ~@body)))))

hy/contrib/profile.hy

@@ -1,5 +1,5 @@
 ;;; Hy profiling macros
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
@@ -19,9 +19,7 @@
   `(do
      (import cProfile pstats)
 
-     (if-python2
-       (import [StringIO [StringIO]])
-       (import [io [StringIO]]))
+     (import [io [StringIO]])
 
      (setv ~g!hy-pr (.Profile cProfile))
      (.enable ~g!hy-pr)

hy/contrib/sequences.hy

@@ -1,14 +1,16 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
 (defclass Sequence []
-  [--init-- (fn [self func]
+
+  (defn --init-- [self func]
     "initialize a new sequence with a function to compute values"
     (setv (. self func) func)
     (setv (. self cache) [])
     (setv (. self high-water) -1))
-   --getitem-- (fn [self n]
+
+  (defn --getitem-- [self n]
     "get nth item of sequence"
     (if (hasattr n "start")
     (gfor x (range n.start n.stop (or n.step 1))
@@ -23,7 +25,8 @@
                (setv (. self high-water) (inc (. self high-water)))
                (.append (. self cache) (.func self (. self high-water))))
              (get self n))))))
-   --iter-- (fn [self]
+
+   (defn --iter-- [self]
      "create iterator for this sequence"
      (setv index 0)
      (try (while True
@@ -31,7 +34,8 @@
             (setv index (inc index)))
           (except [IndexError]
             (return))))
-   --len-- (fn [self]
+
+   (defn --len-- [self]
      "length of the sequence, dangerous for infinite sequences"
      (setv index (. self high-water))
      (try (while True
@@ -39,17 +43,20 @@
             (setv index (inc index)))
           (except [IndexError]
             (len (. self cache)))))
-   max-items-in-repr 10
-   --str-- (fn [self]
+
+   (setv max-items-in-repr 10)
+
+   (defn --str-- [self]
      "string representation of this sequence"
      (setv items (list (take (inc self.max-items-in-repr) self)))
      (.format (if (> (len items) self.max-items-in-repr)
                 "[{0}, ...]"
                 "[{0}]")
               (.join ", " (map str items))))
-   --repr-- (fn [self]
+
+   (defn --repr-- [self]
      "string representation of this sequence"
-              (.--str-- self))])
+     (.--str-- self)))
 
 (defmacro seq [param &rest seq-code]
   `(Sequence (fn ~param (do ~@seq-code))))

hy/contrib/walk.hy

@@ -1,9 +1,10 @@
 ;;; Hy AST walker
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
 (import [hy [HyExpression HyDict]]
+        [hy.models [HySequence]]
         [functools [partial]]
         [importlib [import-module]]
         [collections [OrderedDict]]
@@ -17,9 +18,7 @@
   (cond
    [(instance? HyExpression form)
     (outer (HyExpression (map inner form)))]
-   [(instance? HyDict form)
-    (HyDict (outer (HyExpression (map inner form))))]
-   [(list? form)
+   [(or (instance? HySequence form) (list? form))
     ((type form) (outer (HyExpression (map inner form))))]
    [(coll? form)
     (walk inner outer (list form))]
@@ -46,22 +45,24 @@
   (setv module (or (and module-name
                         (import-module module-name))
                    (calling-module))
-        quote-level [0]
+        quote-level 0
         ast-compiler (HyASTCompiler module))  ; TODO: make nonlocal after dropping Python2
   (defn traverse [form]
     (walk expand identity form))
   (defn expand [form]
+    (nonlocal quote-level)
     ;; manages quote levels
     (defn +quote [&optional [x 1]]
+      (nonlocal quote-level)
       (setv head (first form))
-      (+= (get quote-level 0) x)
-      (when (neg? (get quote-level 0))
+      (+= quote-level x)
+      (when (neg? quote-level)
         (raise (TypeError "unquote outside of quasiquote")))
       (setv res (traverse (cut form 1)))
-      (-= (get quote-level 0) x)
+      (-= quote-level x)
       `(~head ~@res))
     (if (call? form)
-        (cond [(get quote-level 0)
+        (cond [quote-level
                (cond [(in (first form) '[unquote unquote-splice])
                       (+quote -1)]
                      [(= (first form) 'quasiquote) (+quote)]
@@ -89,7 +90,7 @@ splits a fn argument list into sections based on &-headers.
 returns an OrderedDict mapping headers to sublists.
 Arguments without a header are under None.
 "
-  (setv headers '[&optional &rest &kwonly &kwargs]
+  (setv headers ['&optional '&rest '&kwonly '&kwargs]
         sections (OrderedDict [(, None [])])
         header None)
   (for [arg form]
@@ -169,7 +170,7 @@ Arguments without a header are under None.
                                                 #{})))))
   (defn handle-args-list [self]
     (setv protected #{}
-          argslist `[])
+          argslist [])
     (for [[header section] (-> self (.tail) first lambda-list .items)]
       (if header (.append argslist header))
       (cond [(in header [None '&rest '&kwargs])

hy/core/bootstrap.hy

@@ -1,5 +1,5 @@
 ;;; Hy bootstrap macros
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
@@ -60,14 +60,12 @@
 (defmacro macro-error [expression reason &optional [filename '--name--]]
   `(raise (hy.errors.HyMacroExpansionError ~reason ~filename ~expression None)))
 
-(defmacro defn [name lambda-list &rest body]
-  "Define `name` as a function with `lambda-list` signature and body `body`."
+(defmacro defn [name &rest args]
+  "Define `name` as a function with `args` as the signature, annotations, and body."
   (import hy)
   (if (not (= (type name) hy.HySymbol))
     (macro-error name "defn takes a name as first argument"))
-  (if (not (isinstance lambda-list hy.HyList))
-    (macro-error name "defn takes a parameter list as second argument"))
-  `(setv ~name (fn* ~lambda-list ~@body)))
+  `(setv ~name (fn* ~@args)))
 
 (defmacro defn/a [name lambda-list &rest body]
   "Define `name` as a function with `lambda-list` signature and body `body`."
@@ -77,10 +75,3 @@
   (if (not (isinstance lambda-list hy.HyList))
     (macro-error name "defn/a takes a parameter list as second argument"))
   `(setv ~name (fn/a ~lambda-list ~@body)))
-
-(defmacro if-python2 [python2-form python3-form]
-  "If running on python2, execute python2-form, else, execute python3-form"
-  (import sys)
-  (if (< (get sys.version_info 0) 3)
-    python2-form
-    python3-form))

hy/core/language.hy

@@ -1,4 +1,4 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
@@ -11,7 +11,7 @@
 (import [fractions [Fraction :as fraction]])
 (import operator)  ; shadow not available yet
 (import sys)
-(import [hy._compat [long-type]]) ; long for python2, int for python3
+(import [collections.abc :as cabc])
 (import [hy.models [HySymbol HyKeyword]])
 (import [hy.lex [tokenize mangle unmangle read read-str]])
 (import [hy.lex.exceptions [LexException PrematureEndOfInput]])
@@ -21,10 +21,6 @@
 
 (require [hy.core.bootstrap [*]])
 
-(if-python2
-  (import [collections :as cabc])
-  (import [collections.abc :as cabc]))
-
 (defn butlast [coll]
   "Return an iterator of all but the last item in `coll`."
   (drop-last 1 coll))
@@ -86,47 +82,12 @@ If the second argument `codegen` is true, generate python code instead."
        (yield val)
        (.add seen val)))))
 
-(if-python2
- (setv
-   remove itertools.ifilterfalse
-   zip-longest itertools.izip_longest
-   ;; not builtin in Python3
-   reduce reduce
-   ;; hy is more like Python3
-   filter itertools.ifilter
-   input raw_input
-   map itertools.imap
-   range xrange
-   zip itertools.izip)
 (setv
   remove itertools.filterfalse
   zip-longest itertools.zip_longest
   ;; was builtin in Python2
   reduce functools.reduce
-   ;; Someone can import these directly from `hy.core.language`;
-   ;; we'll make some duplicates.
-   filter filter
-   input input
-   map map
-   range range
-   zip zip))
-
-(if-python2
-  (defn exec [$code &optional $globals $locals]
-    "Execute Python code.
-
-The parameter names contain weird characters to discourage calling this
-function with keyword arguments, which isn't supported by Python 3's `exec`."
-    (if
-      (none? $globals) (do
-        (setv frame (._getframe sys (int 1)))
-        (try
-          (setv $globals frame.f_globals  $locals frame.f_locals)
-          (finally (del frame))))
-      (none? $locals)
-        (setv $locals $globals))
-    (exec* $code $globals $locals))
-  (setv exec exec))
+  accumulate itertools.accumulate)
 
 ;; infinite iterators
 (setv
@@ -152,18 +113,6 @@ function with keyword arguments, which isn't supported by Python 3's `exec`."
   permutations itertools.permutations
   product itertools.product)
 
-;; also from itertools, but not in Python2, and without func option until 3.3
-(defn accumulate [iterable &optional [func operator.add]]
-  "Accumulate `func` on `iterable`.
-
-Return series of accumulated sums (or other binary function results)."
-  (setv it (iter iterable)
-        total (next it))
-  (yield total)
-  (for [element it]
-    (setv total (func total element))
-    (yield total)))
-
 (defn drop [count coll]
   "Drop `count` elements from `coll` and yield back the rest."
   (islice coll count None))
@@ -252,13 +201,9 @@ Return series of accumulated sums (or other binary function results)."
   "Perform `isinstance` with reversed arguments."
   (isinstance x klass))
 
-(defn integer [x]
-  "Return Hy kind of integer for `x`."
-  (long-type x))
-
 (defn integer? [x]
   "Check if `x` is an integer."
-  (isinstance x (, int long-type)))
+  (isinstance x int))
 
 (defn integer-char? [x]
   "Check if char `x` parses as an integer."
@@ -388,17 +333,9 @@ with overlap."
   "Return the first logical true value of applying `pred` in `coll`, else None."
   (first (filter None (map pred coll))))
 
-(defn string [x]
-  "Cast `x` as the current python version's string implementation."
-  (if-python2
-   (unicode x)
-   (str x)))
-
 (defn string? [x]
   "Check if `x` is a string."
-  (if-python2
-    (isinstance x (, str unicode))
-    (isinstance x str)))
+  (isinstance x str))
 
 (defn take [count coll]
   "Take `count` elements from `coll`."
@@ -433,7 +370,7 @@ Strings numbers and even objects with the __name__ magic will work."
           (HyKeyword (unmangle value))
           (try
             (unmangle (.__name__ value))
-            (except [] (HyKeyword (string value)))))))
+            (except [] (HyKeyword (str value)))))))
 
 (defn name [value]
   "Convert `value` to a string.
@@ -446,7 +383,7 @@ Even objects with the __name__ magic will work."
           (unmangle value)
           (try
             (unmangle (. value __name__))
-            (except [] (string value))))))
+            (except [] (str value))))))
 
 (defn xor [a b]
   "Perform exclusive or between `a` and `b`."
@@ -454,14 +391,37 @@ Even objects with the __name__ magic will work."
     False
     (or a b)))
 
+(defn parse-args [spec &optional args &kwargs parser-args]
+  "Return arguments namespace parsed from `args` or `sys.argv` with `argparse.ArgumentParser.parse-args` according to `spec`.
+
+`spec` should be a list of arguments to pass to repeated calls to
+`argparse.ArgumentParser.add-argument`.  `parser-args` may be a list
+of keyword arguments to pass to the `argparse.ArgumentParser`
+constructor."
+  (import argparse)
+  (setv parser (argparse.ArgumentParser #** parser-args))
+  (for [arg spec]
+    (setv positional-arguments []
+          keyword-arguments []
+          value-of-keyword? False)
+    (for [item arg]
+      (if value-of-keyword?
+          (.append (get keyword-arguments -1) item)
+          (if (keyword? item)
+              (.append keyword-arguments [(name item)])
+              (.append positional-arguments item)))
+      (setv value-of-keyword? (and (not value-of-keyword?) (keyword? item))))
+    (parser.add-argument #* positional-arguments #** (dict keyword-arguments)))
+  (.parse-args parser args))
+
 (setv EXPORTS
   '[*map accumulate butlast calling-module calling-module-name chain coll?
     combinations comp complement compress constantly count cycle dec distinct
-    disassemble drop drop-last drop-while empty? eval even? every? exec first
-    filter flatten float? fraction gensym group-by identity inc input instance?
-    integer integer? integer-char? interleave interpose islice iterable?
+    disassemble drop drop-last drop-while empty? eval even? every? first
+    flatten float? fraction gensym group-by identity inc instance?
+    integer? integer-char? interleave interpose islice iterable?
     iterate iterator? juxt keyword keyword? last list? macroexpand
-    macroexpand-1 mangle map merge-with multicombinations name neg? none? nth
-    numeric? odd? partition permutations pos? product range read read-str
-    remove repeat repeatedly rest reduce second some string string? symbol?
-    take take-nth take-while tuple? unmangle xor tee zero? zip zip-longest])
+    macroexpand-1 mangle merge-with multicombinations name neg? none? nth
+    numeric? odd? parse-args partition permutations pos? product read read-str
+    remove repeat repeatedly rest reduce second some string? symbol?
+    take take-nth take-while tuple? unmangle xor tee zero? zip-longest])

hy/core/macros.hy

@@ -1,5 +1,5 @@
 ;;; Hy core macros
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
@@ -48,16 +48,14 @@ be associated in pairs."
 
 
 (defn _with [node args body]
-  (if (not (empty? args))
-    (do
-     (if (>= (len args) 2)
-       (do
-        (setv p1 (.pop args 0)
-              p2 (.pop args 0)
-              primary [p1 p2])
-        `(~node [~@primary] ~(_with node args body)))
-       `(~node [~@args] ~@body)))
-    `(do ~@body)))
+  (if
+    (not args)
+      `(do ~@body)
+    (<= (len args) 2)
+      `(~node [~@args] ~@body)
+    True (do
+      (setv [p1 p2 #* args] args)
+      `(~node [~p1 ~p2] ~(_with node args body)))))
 
 
 (defmacro with [args &rest body]
@@ -87,29 +85,19 @@ Shorthand for nested with/a* loops:
 
 The result in the bracket may be omitted, in which case the condition is also
 used as the result."
-  (if (empty? branches)
-    None
-    (do
-     (setv branches (iter branches))
-     (setv branch (next branches))
-     (defn check-branch [branch]
-       "check `cond` branch for validity, return the corresponding `if` expr"
-       (if (not (= (type branch) HyList))
-         (macro-error branch "cond branches need to be a list"))
-       (if (< (len branch) 2)
-         (do
+  (or branches
+    (return))
+
+  `(if ~@(reduce + (gfor
+    branch branches
+    (if
+      (not (and (is (type branch) hy.HyList) branch))
+        (macro-error branch "each cond branch needs to be a nonempty list")
+      (= (len branch) 1) (do
         (setv g (gensym))
-           `(if (do (setv ~g ~(first branch)) ~g) ~g))
-         `(if ~(first branch) (do ~@(cut branch 1)))))
-
-     (setv root (check-branch branch))
-     (setv latest-branch root)
-
-     (for [branch branches]
-       (setv cur-branch (check-branch branch))
-       (.append latest-branch cur-branch)
-       (setv latest-branch cur-branch))
-     root)))
+        [`(do (setv ~g ~(first branch)) ~g) g])
+      True
+        [(first branch) `(do ~@(cut branch 1))])))))
 
 
 (defmacro -> [head &rest args]
@@ -151,6 +139,20 @@ the second form, the second result is inserted into the third form, and so on."
   ret)
 
 
+(defmacro of [base &rest args]
+  "Shorthand for indexing for type annotations.
+
+If only one arguments are given, this expands to just that argument. If two arguments are
+given, it expands to indexing the first argument via the second. Otherwise, the first argument
+is indexed using a tuple of the rest.
+
+E.g. `(of List int)` -> `List[int]`, `(of Dict str str)` -> `Dict[str, str]`."
+  (if
+    (empty? args) base
+    (= (len args) 1) `(get ~base ~@args)
+    `(get ~base (, ~@args))))
+
+
 (defmacro if-not [test not-branch &optional yes-branch]
   "Like `if`, but execute the first branch when the test fails"
   `(if* (not ~test) ~not-branch ~yes-branch))
@@ -213,7 +215,7 @@ Such 'o!' params are available within `body` as the equivalent 'g!' symbol."
   (defn extract-o!-sym [arg]
     (cond [(and (symbol? arg) (.startswith arg "o!"))
            arg]
-          [(and (list? arg) (.startswith (first arg) "o!"))
+          [(and (instance? HyList arg) (.startswith (first arg) "o!"))
            (first arg)]))
   (setv os (list (filter identity (map extract-o!-sym args)))
         gs (lfor s os (HySymbol (+ "g!" (cut s 2)))))

hy/core/shadow.hy

@@ -1,16 +1,14 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
 ;;;; Hy shadow functions
 
 (import operator)
-(import [hy._compat [PY3]])
 
 (require [hy.core.bootstrap [*]])
 
-(if PY3
-  (import [functools [reduce]]))
+(import [functools [reduce]])
 
 (defn + [&rest args]
   "Shadowed `+` operator adds `args`."
@@ -60,10 +58,9 @@
   "Shadowed `%` operator takes `x` modulo `y`."
   (% x y))
 
-(if PY3
 (defn @ [a1 &rest a-rest]
   "Shadowed `@` operator matrix multiples `a1` by each `a-rest`."
-      (reduce operator.matmul a-rest a1)))
+  (reduce operator.matmul a-rest a1))
 
 (defn << [a1 a2 &rest a-rest]
   "Shadowed `<<` operator performs left-shift on `a1` by `a2`, ..., `a-rest`."
@@ -120,6 +117,12 @@
 (defn is-not [a1 a2 &rest a-rest]
   "Shadowed `is-not` keyword perform is-not on `a1` by `a2`, ..., `a-rest`."
   (comp-op operator.is-not a1 (+ (, a2) a-rest)))
+(defn in [a1 a2 &rest a-rest]
+  "Shadowed `in` keyword perform `a1` in `a2` in …."
+  (comp-op (fn [x y] (in x y)) a1 (+ (, a2) a-rest)))
+(defn not-in [a1 a2 &rest a-rest]
+  "Shadowed `not in` keyword perform `a1` not in `a2` not in…."
+  (comp-op (fn [x y] (not-in x y)) a1 (+ (, a2) a-rest)))
 (defn >= [a1 &rest a-rest]
   "Shadowed `>=` operator perform ge comparison on `a1` by each `a-rest`."
   (comp-op operator.ge a1 a-rest))
@@ -151,14 +154,6 @@
   "Shadowed `not` keyword perform not on `x`."
   (not x))
 
-(defn in [x y]
-  "Shadowed `in` keyword perform `x` in `y`."
-  (in x y))
-
-(defn not-in [x y]
-  "Shadowed `not in` keyword perform `x` not in `y`."
-  (not-in x y))
-
 (defn get [coll key1 &rest keys]
   "Access item in `coll` indexed by `key1`, with optional `keys` nested-access."
   (setv coll (get coll key1))
@@ -173,5 +168,3 @@
   'and 'or 'not
   'is 'is-not 'in 'not-in
   'get])
-(if (not PY3)
-  (.remove EXPORTS '@))

hy/errors.py

@@ -1,5 +1,5 @@
 # -*- encoding: utf-8 -*-
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 import os
@@ -9,14 +9,13 @@ import traceback
 import pkgutil
 
 from functools import reduce
+from colorama import Fore
 from contextlib import contextmanager
 from hy import _initialize_env_var
 
-from clint.textui import colored
-
 _hy_filter_internal_errors = _initialize_env_var('HY_FILTER_INTERNAL_ERRORS',
                                                  True)
-_hy_colored_errors = _initialize_env_var('HY_COLORED_ERRORS', False)
+COLORED = _initialize_env_var('HY_COLORED_ERRORS', False)
 
 
 class HyError(Exception):
@@ -108,15 +107,12 @@ class HyLanguageError(HyError):
         """Provide an exception message that includes SyntaxError-like source
         line information when available.
         """
-        global _hy_colored_errors
-
         # Syntax errors are special and annotate the traceback (instead of what
         # we would do in the message that follows the traceback).
         if isinstance(self, SyntaxError):
             return super(HyLanguageError, self).__str__()
-
         # When there isn't extra source information, use the normal message.
-        if not isinstance(self, SyntaxError) and not self.text:
+        elif not self.text:
             return super(HyLanguageError, self).__str__()
 
         # Re-purpose Python's builtin syntax error formatting.
@@ -142,15 +138,14 @@ class HyLanguageError(HyError):
             output[arrow_idx] = '{}{}^\n'.format(output[arrow_idx].rstrip('\n'),
                                                  '-' * (self.arrow_offset - 1))
 
-        if _hy_colored_errors:
-            from clint.textui import colored
-            output[msg_idx:] = [colored.yellow(o) for o in output[msg_idx:]]
+        if COLORED:
+            output[msg_idx:] = [Fore.YELLOW + o + Fore.RESET for o in output[msg_idx:]]
             if arrow_idx:
-                output[arrow_idx] = colored.green(output[arrow_idx])
+                output[arrow_idx] = Fore.GREEN + output[arrow_idx] + Fore.RESET
             for idx, line in enumerate(output[::msg_idx]):
                 if line.strip().startswith(
                         'File "{}", line'.format(self.filename)):
-                    output[idx] = colored.red(line)
+                    output[idx] = Fore.RED + line + Fore.RESET
 
         # This resulting string will come after a "<class-name>:" prompt, so
         # put it down a line.

hy/extra/anaphoric.hy

@@ -1,114 +1,93 @@
 ;;; Hy anaphoric macros
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
+;;; Macro to help write anaphoric macros
+
+(defmacro rit [&rest body]
+  """Supply `it` as a gensym and R as a function to replace `it` with the
+  given gensym throughout expressions."""
+  `(do
+    (setv it (gensym))
+    (defn R [form]
+      "Replace `it` with a gensym throughout `form`."
+      (recur-sym-replace {'it it} form))
+    ~@body))
+
+
 ;;; These macros make writing functional programs more concise
 
 (defmacro ap-if [test-form then-form &optional else-form]
-  `(do
-     (setv it ~test-form)
-     (if it ~then-form ~else-form)))
+  (rit `(do
+     (setv ~it ~test-form)
+     (if ~it ~(R then-form) ~(R else-form)))))
 
 
-(defmacro ap-each [lst &rest body]
-  "Evaluate the body form for each element in the list."
-  `(for [it ~lst] ~@body))
+(defmacro ap-each [xs &rest body]
+  (rit `(for [~it ~xs] ~@(R body))))
 
 
-(defmacro ap-each-while [lst form &rest body]
-  "Evaluate the body form for each element in the list while the
-  predicate form evaluates to True."
-  (setv p (gensym))
-  `(do
-     (defn ~p [it] ~form)
-     (for [it ~lst]
-       (if (~p it)
-           ~@body
-           (break)))))
+(defmacro ap-each-while [xs form &rest body]
+  (rit `(for [~it ~xs]
+    (unless ~(R form)
+      (break))
+    ~@(R body))))
 
 
-(defmacro ap-map [form lst]
-  "Yield elements evaluated in the form for each element in the list."
-  (setv v (gensym 'v)  f (gensym 'f))
-  `((fn []
-      (defn ~f [it] ~form)
-      (for [~v ~lst]
-        (yield (~f ~v))))))
+(defmacro ap-map [form xs]
+  (rit `(gfor  ~it ~xs  ~(R form))))
 
 
-(defmacro ap-map-when [predfn rep lst]
-  "Yield elements evaluated for each element in the list when the
-  predicate function returns True."
-  (setv f (gensym))
-  `((fn []
-      (defn ~f [it] ~rep)
-      (for [it ~lst]
-        (if (~predfn it)
-            (yield (~f it))
-            (yield it))))))
+(defmacro ap-map-when [predfn rep xs]
+  (rit `(gfor  ~it ~xs  (if (~predfn ~it) ~(R rep) ~it))))
 
 
-(defmacro ap-filter [form lst]
-  "Yield elements returned when the predicate form evaluates to True."
-  (setv pred (gensym))
-  `((fn []
-      (defn ~pred [it] ~form)
-      (for [val ~lst]
-        (if (~pred val)
-            (yield val))))))
+(defmacro ap-filter [form xs]
+  (rit `(gfor  ~it ~xs  :if ~(R form)  ~it)))
 
 
-(defmacro ap-reject [form lst]
-  "Yield elements returned when the predicate form evaluates to False"
-  `(ap-filter (not ~form) ~lst))
+(defmacro ap-reject [form xs]
+  (rit `(gfor  ~it ~xs  :if (not ~(R form))  ~it)))
 
 
 (defmacro ap-dotimes [n &rest body]
-  "Execute body for side effects `n' times, with it bound from 0 to n-1"
-  (unless (numeric? n)
-    (raise (TypeError (.format "{!r} is not a number" n))))
-  `(ap-each (range ~n) ~@body))
+  (rit `(for [~it (range ~n)]
+    ~@(R body))))
 
 
-(defmacro ap-first [predfn lst]
-  "Yield the first element that passes `predfn`"
-  (with-gensyms [n]
+(defmacro ap-first [form xs]
+  (rit `(next
+    (gfor  ~it ~xs  :if ~(R form)  ~it)
+    None)))
+
+
+(defmacro ap-last [form xs]
+  (setv x (gensym))
+  (rit `(do
+    (setv ~x None)
+    (for  [~it ~xs  :if ~(R form)]
+      (setv ~x ~it))
+    ~x)))
+
+(defmacro! ap-reduce [form o!xs &optional [initial-value None]]
+  (setv
+    it (gensym)
+    acc (gensym))
+  (defn R [form]
+    (recur-sym-replace {'it it  'acc acc} form))
   `(do
-       (setv ~n None)
-       (ap-each ~lst (when ~predfn (setv ~n it) (break)))
-       ~n)))
-
-
-(defmacro ap-last [predfn lst]
-  "Yield the last element that passes `predfn`"
-  (with-gensyms [n]
+    (setv ~acc ~(if (none? initial-value)
       `(do
-       (setv ~n None)
-       (ap-each ~lst (none? ~n)
-                (when ~predfn
-                  (setv ~n it)))
-       ~n)))
-
-
-(defmacro ap-reduce [form lst &optional [initial-value None]]
-  "Anaphoric form of reduce, `acc' and `it' can be used for a form"
-  `(do
-     (setv acc ~(if (none? initial-value) `(get ~lst 0) initial-value))
-     (ap-each ~(if (none? initial-value) `(cut ~lst 1) lst)
-              (setv acc ~form))
-     acc))
+        (setv ~g!xs (iter ~g!xs))
+        (next ~g!xs))
+      initial-value))
+    (for [~it ~g!xs]
+      (setv ~acc ~(R form)))
+    ~acc))
 
 
 (deftag % [expr]
-  "Makes an expression into a function with an implicit `%` parameter list.
-
-   A `%i` symbol designates the (1-based) ith parameter (such as `%3`).
-   Only the maximum `%i` determines the number of `%i` parameters--the
-   others need not appear in the expression.
-   `%*` and `%**` name the `&rest` and `&kwargs` parameters, respectively.
-
-   Nesting of `#%` forms is not recommended."
   (setv %symbols (sfor a (flatten [expr])
                        :if (and (symbol? a)
                                 (.startswith a '%))
@@ -129,3 +108,18 @@
               '(&kwargs %**))]
      ~expr))
 
+
+;;; --------------------------------------------------
+;;; Subroutines
+;;; --------------------------------------------------
+
+
+(defn recur-sym-replace [d form]
+  "Recursive symbol replacement."
+  (cond
+    [(instance? HySymbol form)
+      (.get d form form)]
+    [(coll? form)
+      ((type form) (gfor  x form  (recur-sym-replace d x)))]
+    [True
+      form]))

hy/extra/reserved.hy

@@ -1,5 +1,5 @@
 ;;; Get a frozenset of Hy reserved words
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 

hy/importer.py

@@ -1,4 +1,4 @@
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 
@@ -19,33 +19,6 @@ from contextlib import contextmanager
 
 from hy.compiler import hy_compile, hy_ast_compile_flags
 from hy.lex import hy_parse
-from hy._compat import PY3
-
-
-def cache_from_source(source_path):
-    """Get the cached bytecode file name for a given source file name.
-
-    This function's name is set to mirror Python 3.x's
-    `importlib.util.cache_from_source`, which is also used when available.
-
-    Parameters
-    ----------
-    source_path : str
-        Path of the source file
-
-    Returns
-    -------
-    out : str
-        Path of the corresponding bytecode file that may--or may
-        not--actually exist.
-    """
-    if PY3:
-        return importlib.util.cache_from_source(source_path)
-    else:
-        # If source_path has a file extension, replace it with ".pyc".
-        # Otherwise, just append ".pyc".
-        d, f = os.path.split(source_path)
-        return os.path.join(d, re.sub(r"(?:\.[^.]+)?\Z", ".pyc", f))
 
 
 @contextmanager
@@ -135,10 +108,9 @@ def _get_code_from_file(run_name, fname=None,
                 source = f.read().decode('utf-8')
             code = compile(source, fname, 'exec')
 
-    return (code, fname) if PY3 else code
+    return (code, fname)
 
 
-if PY3:
 importlib.machinery.SOURCE_SUFFIXES.insert(0, '.hy')
 _py_source_to_code = importlib.machinery.SourceFileLoader.source_to_code
 
@@ -167,339 +139,11 @@ if PY3:
 # Do this one just in case?
 importlib.invalidate_caches()
 
-    # XXX: These and the 2.7 counterparts below aren't truly cross-compliant.
+# XXX: These aren't truly cross-compliant.
 # They're useful for testing, though.
 HyImporter = importlib.machinery.FileFinder
 HyLoader = importlib.machinery.SourceFileLoader
 
-else:
-    import imp
-    import py_compile
-    import marshal
-    import struct
-    import traceback
-
-    from pkgutil import ImpImporter, ImpLoader
-
-    def _could_be_hy_src(filename):
-        return (filename.endswith('.hy') or
-                (os.path.isfile(filename) and
-                 not any(filename.endswith(s[0]) for s in imp.get_suffixes())))
-
-    class HyLoader(ImpLoader, object):
-        def __init__(self, fullname, filename, fileobj=None, etc=None):
-            """This constructor is designed for some compatibility with
-            SourceFileLoader."""
-            if etc is None and filename is not None:
-                if _could_be_hy_src(filename):
-                    etc = ('.hy', 'U', imp.PY_SOURCE)
-                    if fileobj is None:
-                        fileobj = io.open(filename, 'rU', encoding='utf-8')
-
-            super(HyLoader, self).__init__(fullname, fileobj, filename, etc)
-
-        def __getattr__(self, item):
-            # We add these for Python >= 3.4 Loader interface compatibility.
-            if item == 'path':
-                return self.filename
-            elif item == 'name':
-                return self.fullname
-            else:
-                return super(HyLoader, self).__getattr__(item)
-
-        def exec_module(self, module, fullname=None):
-            fullname = self._fix_name(fullname)
-            code = self.get_code(fullname)
-            eval(code, module.__dict__)
-
-        def load_module(self, fullname=None):
-            """Same as `pkgutil.ImpLoader`, with an extra check for Hy
-            source and the option to not run `self.exec_module`."""
-            fullname = self._fix_name(fullname)
-            ext_type = self.etc[0]
-            mod_type = self.etc[2]
-            mod = None
-            pkg_path = os.path.join(self.filename, '__init__.hy')
-            if ext_type == '.hy' or (
-                    mod_type == imp.PKG_DIRECTORY and
-                    os.path.isfile(pkg_path)):
-
-                was_in_sys = fullname in sys.modules
-                if was_in_sys:
-                    mod = sys.modules[fullname]
-                else:
-                    mod = sys.modules.setdefault(
-                        fullname, types.ModuleType(fullname))
-
-                # TODO: Should we set these only when not in `sys.modules`?
-                if mod_type == imp.PKG_DIRECTORY:
-                    mod.__file__ = pkg_path
-                    mod.__path__ = [self.filename]
-                    mod.__package__ = fullname
-                else:
-                    # mod.__path__ = self.filename
-                    mod.__file__ = self.get_filename(fullname)
-                    mod.__package__ = '.'.join(fullname.split('.')[:-1])
-
-                mod.__name__ = fullname
-
-                try:
-                    self.exec_module(mod, fullname=fullname)
-                except Exception:
-                    # Follow Python 2.7 logic and only remove a new, bad
-                    # module; otherwise, leave the old--and presumably
-                    # good--module in there.
-                    if not was_in_sys:
-                        del sys.modules[fullname]
-                    raise
-
-            if mod is None:
-                self._reopen()
-                try:
-                    mod = imp.load_module(fullname, self.file, self.filename,
-                                          self.etc)
-                finally:
-                    if self.file:
-                        self.file.close()
-
-            mod.__loader__ = self
-            return mod
-
-        def _reopen(self):
-            """Same as `pkgutil.ImpLoader`, with an extra check for Hy
-            source"""
-            if self.file and self.file.closed:
-                ext_type = self.etc[0]
-                if ext_type == '.hy':
-                    self.file = io.open(self.filename, 'rU', encoding='utf-8')
-                else:
-                    super(HyLoader, self)._reopen()
-
-        def byte_compile_hy(self, fullname=None):
-            fullname = self._fix_name(fullname)
-            if fullname is None:
-                fullname = self.fullname
-
-            hy_source = self.get_source(fullname)
-            hy_tree = hy_parse(hy_source, filename=self.filename)
-
-            with loader_module_obj(self) as module:
-                hy_ast = hy_compile(hy_tree, module)
-
-            code = compile(hy_ast, self.filename, 'exec',
-                           hy_ast_compile_flags)
-
-            if not sys.dont_write_bytecode:
-                try:
-                    hyc_compile(code, module=fullname)
-                except IOError:
-                    pass
-            return code
-
-        def get_code(self, fullname=None):
-            """Same as `pkgutil.ImpLoader`, with an extra check for Hy
-            source"""
-            fullname = self._fix_name(fullname)
-            ext_type = self.etc[0]
-            if ext_type == '.hy':
-                # Looks like we have to manually check for--and update--
-                # the bytecode.
-                t_py = long(os.stat(self.filename).st_mtime)
-                pyc_file = cache_from_source(self.filename)
-                if os.path.isfile(pyc_file):
-                    t_pyc = long(os.stat(pyc_file).st_mtime)
-
-                    if t_pyc is not None and t_pyc >= t_py:
-                        with open(pyc_file, 'rb') as f:
-                            if f.read(4) == imp.get_magic():
-                                t = struct.unpack('<I', f.read(4))[0]
-                                if t == t_py:
-                                    self.code = marshal.load(f)
-
-                if self.code is None:
-                    # There's no existing bytecode, or bytecode timestamp
-                    # is older than the source file's.
-                    self.code = self.byte_compile_hy(fullname)
-
-            if self.code is None:
-                super(HyLoader, self).get_code(fullname=fullname)
-
-            return self.code
-
-        def _get_delegate(self):
-            return HyImporter(self.filename).find_module('__init__')
-
-    class HyImporter(ImpImporter, object):
-        def __init__(self, path=None):
-            # We need to be strict about the types of files this importer will
-            # handle.  To start, if the path is not the current directory in
-            # (represented by '' in `sys.path`), then it must be a supported
-            # file type or a directory.  If it isn't, this importer is not
-            # suitable: throw an exception.
-
-            if path == '' or os.path.isdir(path) or (
-                    os.path.isfile(path) and path.endswith('.hy')):
-                self.path = path
-            else:
-                raise ImportError('Invalid path: {}'.format(path))
-
-        def find_loader(self, fullname):
-            return self.find_module(fullname, path=None)
-
-        def find_module(self, fullname, path=None):
-
-            subname = fullname.split(".")[-1]
-
-            if subname != fullname and self.path is None:
-                return None
-
-            if self.path is None:
-                path = None
-            else:
-                path = [os.path.realpath(self.path)]
-
-            fileobj, file_path, etc = None, None, None
-
-            # The following are excerpts from the later pure Python
-            # implementations of the `imp` module (e.g. in Python 3.6).
-            if path is None:
-                path = sys.path
-
-            for entry in path:
-                if (os.path.isfile(entry) and subname == '__main__' and
-                    entry.endswith('.hy')):
-                    file_path = entry
-                    fileobj = io.open(file_path, 'rU', encoding='utf-8')
-                    etc = ('.hy', 'U', imp.PY_SOURCE)
-                    break
-                else:
-                    file_path = os.path.join(entry, subname)
-                    path_init = os.path.join(file_path, '__init__.hy')
-                    if os.path.isfile(path_init):
-                        fileobj = None
-                        etc = ('', '', imp.PKG_DIRECTORY)
-                        break
-
-                    file_path = file_path + '.hy'
-                    if os.path.isfile(file_path):
-                        fileobj = io.open(file_path, 'rU', encoding='utf-8')
-                        etc = ('.hy', 'U', imp.PY_SOURCE)
-                        break
-            else:
-                try:
-                    fileobj, file_path, etc = imp.find_module(subname, path)
-                except (ImportError, IOError):
-                    return None
-
-            return HyLoader(fullname, file_path, fileobj, etc)
-
-    sys.path_hooks.append(HyImporter)
-    sys.path_importer_cache.clear()
-
-    _py_compile_compile = py_compile.compile
-
-    def hyc_compile(file_or_code, cfile=None, dfile=None, doraise=False,
-                    module=None):
-        """Write a Hy file, or code object, to pyc.
-
-        This is a patched version of Python 2.7's `py_compile.compile`.
-
-        Also, it tries its best to write the bytecode file atomically.
-
-        Parameters
-        ----------
-        file_or_code : str or instance of `types.CodeType`
-            A filename for a Hy or Python source file or its corresponding code
-            object.
-        cfile : str, optional
-            The filename to use for the bytecode file.  If `None`, use the
-            standard bytecode filename determined by `cache_from_source`.
-        dfile : str, optional
-            The filename to use for compile-time errors.
-        doraise : bool, default False
-            If `True` raise compilation exceptions; otherwise, ignore them.
-        module : str or types.ModuleType, optional
-            The module, or module name, in which the Hy tree is expanded.
-            Default is the caller's module.
-
-        Returns
-        -------
-        out : str
-            The resulting bytecode file name.  Python 3.x returns this, but
-            Python 2.7 doesn't; this function does for convenience.
-        """
-
-        if isinstance(file_or_code, types.CodeType):
-            codeobject = file_or_code
-            filename = codeobject.co_filename
-        else:
-            filename = file_or_code
-
-            with open(filename, 'rb') as f:
-                source_str = f.read().decode('utf-8')
-
-            try:
-                flags = None
-                if _could_be_hy_src(filename):
-                    hy_tree = hy_parse(source_str, filename=filename)
-
-                    if module is None:
-                        module = inspect.getmodule(inspect.stack()[1][0])
-                    elif not inspect.ismodule(module):
-                        module = importlib.import_module(module)
-
-                    source = hy_compile(hy_tree, module)
-                    flags = hy_ast_compile_flags
-
-                codeobject = compile(source, dfile or filename, 'exec', flags)
-            except Exception as err:
-
-                py_exc = py_compile.PyCompileError(err.__class__, err,
-                                                   dfile or filename)
-                if doraise:
-                    raise py_exc
-                else:
-                    traceback.print_exc()
-                    return
-
-        timestamp = long(os.stat(filename).st_mtime)
-
-        if cfile is None:
-            cfile = cache_from_source(filename)
-
-        f = None
-        try:
-            f = tempfile.NamedTemporaryFile('wb', dir=os.path.split(cfile)[0],
-                                            delete=False)
-            f.write('\0\0\0\0')
-            f.write(struct.pack('<I', timestamp))
-            f.write(marshal.dumps(codeobject))
-            f.flush()
-            f.seek(0, 0)
-            f.write(imp.get_magic())
-
-            # Make sure it's written to disk.
-            f.flush()
-            os.fsync(f.fileno())
-            f.close()
-
-            # Rename won't replace an existing dest on Windows.
-            if os.name == 'nt' and os.path.isfile(cfile):
-                os.unlink(cfile)
-
-            os.rename(f.name, cfile)
-        except OSError:
-            try:
-                if f is not None:
-                    os.unlink(f.name)
-            except OSError:
-                pass
-
-        return cfile
-
-    py_compile.compile = hyc_compile
-
-
 # We create a separate version of runpy, "runhy", that prefers Hy source over
 # Python.
 runhy = importlib.import_module('runpy')
@@ -513,3 +157,11 @@ runpy = importlib.import_module('runpy')
 
 _runpy_get_code_from_file = runpy._get_code_from_file
 runpy._get_code_from_file = _get_code_from_file
+
+
+def _import_from_path(name, path):
+    """A helper function that imports a module from the given path."""
+    spec = importlib.util.spec_from_file_location(name, path)
+    mod = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(mod)
+    return mod

hy/lex/__init__.py

@@ -1,14 +1,14 @@
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 
 from __future__ import unicode_literals
 
+import keyword
 import re
 import sys
 import unicodedata
 
-from hy._compat import str_type, isidentifier, UCS4
 from hy.lex.exceptions import PrematureEndOfInput, LexException  # NOQA
 from hy.models import HyExpression, HySymbol
 
@@ -116,7 +116,7 @@ def mangle(s):
 
     assert s
 
-    s = str_type(s)
+    s = str(s)
     s = s.replace("-", "_")
     s2 = s.lstrip('_')
     leading_underscores = '_' * (len(s) - len(s2))
@@ -135,7 +135,7 @@ def mangle(s):
                else '{0}{1}{0}'.format(mangle_delim,
                    unicodedata.name(c, '').lower().replace('-', 'H').replace(' ', '_')
                    or 'U{}'.format(unicode_char_to_hex(c)))
-            for c in unicode_to_ucs4iter(s))
+            for c in s)
 
     s = leading_underscores + s
     assert isidentifier(s)
@@ -147,7 +147,7 @@ def unmangle(s):
     form. This may not round-trip, because different Hy symbol names can
     mangle to the same Python identifier."""
 
-    s = str_type(s)
+    s = str(s)
 
     s2 = s.lstrip('_')
     leading_underscores = len(s) - len(s2)
@@ -168,19 +168,6 @@ def unmangle(s):
     return '-' * leading_underscores + s
 
 
-def unicode_to_ucs4iter(ustr):
-    # Covert a unicode string to an iterable object,
-    # elements in the object are single USC-4 unicode characters
-    if UCS4:
-        return ustr
-    ucs4_list = list(ustr)
-    for i, u in enumerate(ucs4_list):
-        if 0xD7FF < ord(u) < 0xDC00:
-            ucs4_list[i] += ucs4_list[i + 1]
-            del ucs4_list[i + 1]
-    return ucs4_list
-
-
 def read(from_file=sys.stdin, eof=""):
     """Read from input and returns a tokenized string.
 
@@ -203,4 +190,12 @@ def read(from_file=sys.stdin, eof=""):
 
 
 def read_str(input):
-    return read(StringIO(str_type(input)))
+    return read(StringIO(str(input)))
+
+
+def isidentifier(x):
+    if x in ('True', 'False', 'None'):
+        return True
+    if keyword.iskeyword(x):
+        return False
+    return x.isidentifier()

hy/lex/exceptions.py

@@ -1,4 +1,4 @@
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 from hy.errors import HySyntaxError

hy/lex/lexer.py

@@ -1,4 +1,4 @@
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 
@@ -10,7 +10,8 @@ lg = LexerGenerator()
 
 # A regexp for something that should end a quoting/unquoting operator
 # i.e. a space or a closing brace/paren/curly
-end_quote = r'(?![\s\)\]\}])'
+end_quote_set = r'\s\)\]\}'
+end_quote = r'(?![%s])' % end_quote_set
 
 identifier = r'[^()\[\]{}\'"\s;]+'
 
@@ -25,6 +26,7 @@ lg.add('QUOTE', r'\'%s' % end_quote)
 lg.add('QUASIQUOTE', r'`%s' % end_quote)
 lg.add('UNQUOTESPLICE', r'~@%s' % end_quote)
 lg.add('UNQUOTE', r'~%s' % end_quote)
+lg.add('ANNOTATION', r'\^(?![=%s])' % end_quote_set)
 lg.add('DISCARD', r'#_')
 lg.add('HASHSTARS', r'#\*+')
 lg.add('BRACKETSTRING', r'''(?x)

hy/lex/parser.py

@@ -1,5 +1,5 @@
 # -*- encoding: utf-8 -*-
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 
@@ -9,7 +9,6 @@ from functools import wraps
 
 from rply import ParserGenerator
 
-from hy._compat import str_type
 from hy.models import (HyBytes, HyComplex, HyDict, HyExpression, HyFloat,
                        HyInteger, HyKeyword, HyList, HySet, HyString, HySymbol)
 from .lexer import lexer
@@ -135,6 +134,12 @@ def term_unquote_splice(state, p):
     return HyExpression([HySymbol("unquote-splice"), p[1]])
 
 
+@pg.production("term : ANNOTATION term")
+@set_quote_boundaries
+def term_annotation(state, p):
+    return HyExpression([HySymbol("annotate*"), p[1]])
+
+
 @pg.production("term : HASHSTARS term")
 @set_quote_boundaries
 def term_hashstars(state, p):
@@ -214,7 +219,7 @@ def t_string(state, p):
         raise LexException.from_lexer("Can't convert {} to a HyString".format(p[0].value),
                                       state, p[0])
     return (HyString(s, is_format = is_format)
-        if isinstance(s, str_type)
+        if isinstance(s, str)
         else HyBytes(s))
 
 

hy/macros.py

@@ -1,4 +1,4 @@
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 import sys
@@ -9,7 +9,7 @@ import traceback
 
 from contextlib import contextmanager
 
-from hy._compat import PY3, string_types, reraise, rename_function
+from hy._compat import reraise, PY38
 from hy.models import replace_hy_obj, HyExpression, HySymbol, wrap_value
 from hy.lex import mangle
 from hy.errors import (HyLanguageError, HyMacroExpansionError, HyTypeError,
@@ -74,9 +74,6 @@ def tag(name):
     def _(fn):
         _name = mangle('#{}'.format(name))
 
-        if not PY3:
-            _name = _name.encode('UTF-8')
-
         fn = rename_function(fn, _name)
 
         module = inspect.getmodule(fn)
@@ -156,7 +153,7 @@ def require(source_module, target_module, assignments, prefix=""):
         parent_frame = inspect.stack()[1][0]
         target_namespace = parent_frame.f_globals
         target_module = target_namespace.get('__name__', None)
-    elif isinstance(target_module, string_types):
+    elif isinstance(target_module, str):
         target_module = importlib.import_module(target_module)
         target_namespace = target_module.__dict__
     elif inspect.ismodule(target_module):
@@ -311,10 +308,7 @@ def macroexpand(tree, module, compiler=None, once=False):
 
     assert not compiler or compiler.module == module
 
-    while True:
-
-        if not isinstance(tree, HyExpression) or tree == []:
-            break
+    while isinstance(tree, HyExpression) and tree:
 
         fn = tree[0]
         if fn in ("quote", "quasiquote") or not isinstance(fn, HySymbol):
@@ -384,3 +378,25 @@ def tag_macroexpand(tag, tree, module):
         expr.module = inspect.getmodule(tag_macro)
 
     return replace_hy_obj(expr, tree)
+
+
+def rename_function(func, new_name):
+    """Creates a copy of a function and [re]sets the name at the code-object
+    level.
+    """
+    c = func.__code__
+    new_code = type(c)(*[getattr(c, 'co_{}'.format(a))
+                         if a != 'name' else str(new_name)
+                         for a in code_obj_args])
+
+    _fn = type(func)(new_code, func.__globals__, str(new_name),
+                     func.__defaults__, func.__closure__)
+    _fn.__dict__.update(func.__dict__)
+
+    return _fn
+
+code_obj_args = ['argcount', 'posonlyargcount', 'kwonlyargcount', 'nlocals', 'stacksize',
+                 'flags', 'code', 'consts', 'names', 'varnames', 'filename', 'name',
+                 'firstlineno', 'lnotab', 'freevars', 'cellvars']
+if not PY38:
+    code_obj_args.remove("posonlyargcount")

hy/model_patterns.py

@@ -1,4 +1,4 @@
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 

hy/models.py

@@ -1,4 +1,4 @@
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 from __future__ import unicode_literals
@@ -6,13 +6,12 @@ from __future__ import unicode_literals
 from contextlib import contextmanager
 from math import isnan, isinf
 from hy import _initialize_env_var
-from hy._compat import PY3, str_type, bytes_type, long_type, string_types
 from hy.errors import HyWrapperError
 from fractions import Fraction
-from clint.textui import colored
+from colorama import Fore
 
 PRETTY = True
-_hy_colored_ast_objects = _initialize_env_var('HY_COLORED_AST_OBJECTS', False)
+COLORED = _initialize_env_var('HY_COLORED_AST_OBJECTS', False)
 
 
 @contextmanager
@@ -29,6 +28,18 @@ def pretty(pretty=True):
         PRETTY = old
 
 
+class _ColoredModel:
+    """
+    Mixin that provides a helper function for models that have color.
+    """
+
+    def _colored(self, text):
+        if COLORED:
+            return self.color + text + Fore.RESET
+        else:
+            return text
+
+
 class HyObject(object):
     """
     Generic Hy Object model. This is helpful to inject things into all the
@@ -88,7 +99,7 @@ def repr_indent(obj):
     return repr(obj).replace("\n", "\n  ")
 
 
-class HyString(HyObject, str_type):
+class HyString(HyObject, str):
     """
     Generic Hy String object. Helpful to store string literals from Hy
     scripts. It's either a ``str`` or a ``unicode``, depending on the
@@ -100,20 +111,20 @@ class HyString(HyObject, str_type):
         value.brackets = brackets
         return value
 
-_wrappers[str_type] = HyString
+_wrappers[str] = HyString
 
 
-class HyBytes(HyObject, bytes_type):
+class HyBytes(HyObject, bytes):
     """
     Generic Hy Bytes object. It's either a ``bytes`` or a ``str``, depending
     on the Python version.
     """
     pass
 
-_wrappers[bytes_type] = HyBytes
+_wrappers[bytes] = HyBytes
 
 
-class HySymbol(HyObject, str_type):
+class HySymbol(HyObject, str):
     """
     Hy Symbol. Basically a string.
     """
@@ -170,42 +181,39 @@ def strip_digit_separators(number):
     # Don't strip a _ or , if it's the first character, as _42 and
     # ,42 aren't valid numbers
     return (number[0] + number[1:].replace("_", "").replace(",", "")
-            if isinstance(number, string_types) and len(number) > 1
+            if isinstance(number, str) and len(number) > 1
             else number)
 
 
-class HyInteger(HyObject, long_type):
+class HyInteger(HyObject, int):
     """
     Internal representation of a Hy Integer. May raise a ValueError as if
-    int(foo) was called, given HyInteger(foo). On python 2.x long will
-    be used instead
+    int(foo) was called, given HyInteger(foo).
     """
 
     def __new__(cls, number, *args, **kwargs):
-        if isinstance(number, string_types):
+        if isinstance(number, str):
             number = strip_digit_separators(number)
             bases = {"0x": 16, "0o": 8, "0b": 2}
             for leader, base in bases.items():
                 if number.startswith(leader):
                     # We've got a string, known leader, set base.
-                    number = long_type(number, base=base)
+                    number = int(number, base=base)
                     break
             else:
                 # We've got a string, no known leader; base 10.
-                number = long_type(number, base=10)
+                number = int(number, base=10)
         else:
             # We've got a non-string; convert straight.
-            number = long_type(number)
+            number = int(number)
         return super(HyInteger, cls).__new__(cls, number)
 
 
 _wrappers[int] = HyInteger
-if not PY3:  # do not add long on python3
-    _wrappers[long_type] = HyInteger
 
 
 def check_inf_nan_cap(arg, value):
-    if isinstance(arg, string_types):
+    if isinstance(arg, str):
         if isinf(value) and "i" in arg.lower() and "Inf" not in arg:
             raise ValueError('Inf must be capitalized as "Inf"')
         if isnan(value) and "NaN" not in arg:
@@ -233,7 +241,7 @@ class HyComplex(HyObject, complex):
     """
 
     def __new__(cls, real, imag=0, *args, **kwargs):
-        if isinstance(real, string_types):
+        if isinstance(real, str):
             value = super(HyComplex, cls).__new__(
                 cls, strip_digit_separators(real)
             )
@@ -247,7 +255,7 @@ class HyComplex(HyObject, complex):
 _wrappers[complex] = HyComplex
 
 
-class HySequence(HyObject, list):
+class HySequence(HyObject, tuple, _ColoredModel):
     """
     An abstract type for sequence-like models to inherit from.
     """
@@ -260,7 +268,8 @@ class HySequence(HyObject, list):
         return self
 
     def __add__(self, other):
-        return self.__class__(super(HySequence, self).__add__(other))
+        return self.__class__(super(HySequence, self).__add__(
+            tuple(other) if isinstance(other, list) else other))
 
     def __getslice__(self, start, end):
         return self.__class__(super(HySequence, self).__getslice__(start, end))
@@ -279,21 +288,25 @@ class HySequence(HyObject, list):
         return str(self) if PRETTY else super(HySequence, self).__repr__()
 
     def __str__(self):
-        global _hy_colored_ast_objects
         with pretty():
-            c = self.color if _hy_colored_ast_objects else str
             if self:
-                return ("{}{}\n  {}{}").format(
-                    c(self.__class__.__name__),
-                    c("(["),
-                    (c(",") + "\n  ").join([repr_indent(e) for e in self]),
-                    c("])"))
+                return self._colored("{}{}\n  {}{}".format(
+                    self._colored(self.__class__.__name__),
+                    self._colored("(["),
+                    self._colored(",\n  ").join(map(repr_indent, self)),
+                    self._colored("])"),
+                ))
+                return self._colored("{}([\n  {}])".format(
+                    self.__class__.__name__,
+                    ','.join(repr_indent(e) for e in self),
+                ))
             else:
-                return '' + c(self.__class__.__name__ + "()")
+                return self._colored(self.__class__.__name__ + "()")
 
 
 class HyList(HySequence):
-    color = staticmethod(colored.cyan)
+    color = Fore.CYAN
+
 
 def recwrap(f):
     return lambda l: f(wrap_value(x) for x in l)
@@ -303,16 +316,14 @@ _wrappers[list] = recwrap(HyList)
 _wrappers[tuple] = recwrap(HyList)
 
 
-class HyDict(HySequence):
+class HyDict(HySequence, _ColoredModel):
     """
     HyDict (just a representation of a dict)
     """
-    color = staticmethod(colored.green)
+    color = Fore.GREEN
 
     def __str__(self):
-        global _hy_colored_ast_objects
         with pretty():
-            g = self.color if _hy_colored_ast_objects else str
             if self:
                 pairs = []
                 for k, v in zip(self[::2],self[1::2]):
@@ -320,20 +331,22 @@ class HyDict(HySequence):
                     pairs.append(
                         ("{0}{c}\n  {1}\n  "
                          if '\n' in k+v
-                         else "{0}{c} {1}").format(k, v, c=g(',')))
+                         else "{0}{c} {1}").format(k, v, c=self._colored(',')))
                 if len(self) % 2 == 1:
                     pairs.append("{}  {}\n".format(
-                        repr_indent(self[-1]), g("# odd")))
+                        repr_indent(self[-1]), self._colored("# odd")))
                 return "{}\n  {}{}".format(
-                    g("HyDict(["), ("{c}\n  ".format(c=g(',')).join(pairs)), g("])"))
+                    self._colored("HyDict(["),
+                    "{c}\n  ".format(c=self._colored(',')).join(pairs),
+                    self._colored("])"))
             else:
-                return '' + g("HyDict()")
+                return self._colored("HyDict()")
 
     def keys(self):
-        return self[0::2]
+        return list(self[0::2])
 
     def values(self):
-        return self[1::2]
+        return list(self[1::2])
 
     def items(self):
         return list(zip(self.keys(), self.values()))
@@ -346,7 +359,7 @@ class HyExpression(HySequence):
     """
     Hy S-Expression. Basically just a list.
     """
-    color = staticmethod(colored.yellow)
+    color = Fore.YELLOW
 
 _wrappers[HyExpression] = recwrap(HyExpression)
 _wrappers[Fraction] = lambda e: HyExpression(
@@ -357,7 +370,7 @@ class HySet(HySequence):
     """
     Hy set (just a representation of a set)
     """
-    color = staticmethod(colored.red)
+    color = Fore.RED
 
 _wrappers[HySet] = recwrap(HySet)
 _wrappers[set] = recwrap(HySet)

scripts/update-coreteam.hy

@@ -1,41 +0,0 @@
-;; Copyright 2019 the authors.
-;; This file is part of Hy, which is free software licensed under the Expat
-;; license. See the LICENSE.
-
-;; You need to install the requests package first
-
-(import os.path)
-(import requests)
-
-
-(setv *api-url* "https://api.github.com/{}")
-(setv *rst-format* "* `{} <{}>`_")
-(setv *missing-names* {"khinsen" "Konrad Hinsen"})
-;; We have three concealed members on the hylang organization
-;; and GitHub only shows public members if the requester is not
-;; an owner of the organization.
-(setv *concealed-members* [(, "aldeka" "Karen Rustad")
-                           (, "tuturto" "Tuukka Turto")
-                           (, "cndreisbach" "Clinton N. Dreisbach")])
-
-(defn get-dev-name [login]
-  (setv name (get (.json (requests.get (.format *api-url* (+ "users/" login)))) "name"))
-  (if-not name
-    (.get *missing-names* login)
-    name))
-
-(setv coredevs (requests.get (.format *api-url* "orgs/hylang/members")))
-
-(setv result (set))
-(for [dev (.json coredevs)]
-  (result.add (.format *rst-format* (get-dev-name (get dev "login"))
-                       (get dev "html_url"))))
-
-(for [(, login name) *concealed-members*]
-  (result.add (.format *rst-format* name (+ "https://github.com/" login))))
-
-(setv filename (os.path.abspath (os.path.join os.path.pardir
-                                              "docs" "coreteam.rst")))
-
-(with [fobj (open filename "w+")]
-  (fobj.write (+ (.join "\n" result) "\n")))

setup.py

@@ -1,9 +1,13 @@
 #!/usr/bin/env python
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 
-import sys, os
+import glob
+import importlib
+import inspect
+import os
+import sys
 
 from setuptools import find_packages, setup
 from setuptools.command.install import install
@@ -20,27 +24,51 @@ make things work nicer, and lets Python and the Hy lisp variant play
 nice together. """
 
 class Install(install):
+    def __compile_hy_bytecode(self):
+        for path in sorted(glob.iglob('hy/**.hy', recursive=True)):
+            importlib.util.cache_from_source(path, optimize=self.optimize)
+
     def run(self):
-        # Import each Hy module to ensure it's compiled.
-        import os, importlib
-        for dirpath, _, filenames in sorted(os.walk("hy")):
-            for filename in sorted(filenames):
-                if filename.endswith(".hy"):
-                    importlib.import_module(
-                        dirpath.replace("/", ".").replace("\\", ".") +
-                        "." + filename[:-len(".hy")])
-        install.run(self)
+        # Don't bother messing around with deps if they wouldn't be installed anyway.
+        # Code is based on setuptools's install.py.
+        if not (self.old_and_unmanageable or self.single_version_externally_managed
+                or not self._called_from_setup(inspect.currentframe())):
+            easy_install = self.distribution.get_command_class('easy_install')
+
+            cmd = easy_install(
+                self.distribution, args="x", root=self.root, record=self.record,
+            )
+            cmd.ensure_finalized()
+            cmd.always_copy_from = '.'
+            cmd.package_index.scan(glob.glob('*.egg'))
+
+            cmd.args = self.distribution.install_requires
+
+            # Avoid deprecation warnings on new setuptools versions.
+            if 'show_deprecation' in inspect.signature(cmd.run).parameters:
+                cmd.run(show_deprecation=False)
+            else:
+                cmd.run()
+
+            # Make sure any new packages get picked up.
+            import site
+            importlib.reload(site)
+            importlib.invalidate_caches()
+
+        self.__compile_hy_bytecode()
+
+        # The deps won't be reinstalled because of:
+        # https://github.com/pypa/setuptools/issues/456
+        return install.run(self)
 
 install_requires = [
     'rply>=0.7.7',
     'astor>=0.8',
     'funcparserlib>=0.3.6',
-    'clint>=0.4']
+    'colorama']
 if os.name == 'nt':
     install_requires.append('pyreadline>=2.1')
 
-ver = sys.version_info[0]
-
 setup(
     name=PKG,
     version=__version__,
@@ -49,11 +77,11 @@ setup(
     entry_points={
         'console_scripts': [
             'hy = hy.cmdline:hy_main',
-            'hy%d = hy.cmdline:hy_main' % ver,
+            'hy3 = hy.cmdline:hy_main',
             'hyc = hy.cmdline:hyc_main',
-            'hyc%d = hy.cmdline:hyc_main' % ver,
+            'hyc3 = hy.cmdline:hyc_main',
             'hy2py = hy.cmdline:hy2py_main',
-            'hy2py%d = hy.cmdline:hy2py_main' % ver,
+            'hy2py3 = hy.cmdline:hy2py_main',
         ]
     },
     packages=find_packages(exclude=['tests*']),
@@ -80,8 +108,6 @@ setup(
         "Operating System :: OS Independent",
         "Programming Language :: Lisp",
         "Programming Language :: Python",
-        "Programming Language :: Python :: 2",
-        "Programming Language :: Python :: 2.7",
         "Programming Language :: Python :: 3",
         "Programming Language :: Python :: 3.5",
         "Programming Language :: Python :: 3.6",

tests/compilers/test_ast.py

@@ -1,5 +1,5 @@
 # -*- encoding: utf-8 -*-
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 
@@ -7,10 +7,10 @@ from __future__ import unicode_literals
 
 from hy import HyString
 from hy.compiler import hy_compile, hy_eval
-from hy.errors import HyCompileError, HyLanguageError, HyError
+from hy.errors import HyLanguageError, HyError
 from hy.lex import hy_parse
 from hy.lex.exceptions import LexException, PrematureEndOfInput
-from hy._compat import PY3, PY36
+from hy._compat import PY36
 
 import ast
 import pytest
@@ -36,13 +36,10 @@ def can_eval(expr):
 def cant_compile(expr):
     with pytest.raises(HyError) as excinfo:
         hy_compile(hy_parse(expr), __name__)
-
-    if issubclass(excinfo.type, HyLanguageError):
-        assert excinfo.value.msg
-        return excinfo.value
-    elif issubclass(excinfo.type, HyCompileError):
     # Anything that can't be compiled should raise a user friendly
     # error, otherwise it's a compiler bug.
+    assert issubclass(excinfo.type, HyLanguageError)
+    assert excinfo.value.msg
     return excinfo.value
 
 
@@ -121,7 +118,6 @@ def test_ast_good_raise():
     can_compile("(raise e)")
 
 
-if PY3:
 def test_ast_raise_from():
     can_compile("(raise Exception :from NameError)")
 
@@ -205,12 +201,12 @@ def test_ast_bad_global():
     cant_compile("(global (foo))")
 
 
-if PY3:
 def test_ast_good_nonlocal():
     "Make sure AST can compile valid nonlocal"
     can_compile("(nonlocal a)")
     can_compile("(nonlocal foo bar)")
 
+
 def test_ast_bad_nonlocal():
     "Make sure AST can't compile invalid nonlocal"
     cant_compile("(nonlocal)")
@@ -226,7 +222,6 @@ def test_ast_good_defclass():
     can_compile("(defclass a [] None (print \"foo\"))")
 
 
-@pytest.mark.skipif(not PY3, reason="Python 3 supports class keywords")
 def test_ast_good_defclass_with_metaclass():
     "Make sure AST can compile valid defclass with keywords"
     can_compile("(defclass a [:metaclass b])")
@@ -299,21 +294,6 @@ import a dotted name."""
     cant_compile("(require [spam [foo.bar]])")
 
 
-def test_ast_no_pointless_imports():
-    def contains_import_from(code):
-        return any([isinstance(node, ast.ImportFrom)
-                   for node in can_compile(code).body])
-    # `reduce` is a builtin in Python 2, but not Python 3.
-    # The version of `map` that returns an iterator is a builtin in
-    # Python 3, but not Python 2.
-    if PY3:
-        assert contains_import_from("reduce")
-        assert not contains_import_from("map")
-    else:
-        assert not contains_import_from("reduce")
-        assert contains_import_from("map")
-
-
 def test_ast_good_get():
     "Make sure AST can compile valid get"
     can_compile("(get x y)")
@@ -454,27 +434,18 @@ def test_lambda_list_keywords_kwargs():
 
 
 def test_lambda_list_keywords_kwonly():
-    """Ensure we can compile functions with &kwonly if we're on Python
-    3, or fail with an informative message on Python 2."""
     kwonly_demo = "(fn [&kwonly a [b 2]] (print 1) (print a b))"
-    if PY3:
     code = can_compile(kwonly_demo)
     for i, kwonlyarg_name in enumerate(('a', 'b')):
         assert kwonlyarg_name == code.body[0].args.kwonlyargs[i].arg
     assert code.body[0].args.kw_defaults[0] is None
     assert code.body[0].args.kw_defaults[1].n == 2
-    else:
-        exception = cant_compile(kwonly_demo)
-        assert isinstance(exception, HyLanguageError)
-        message = exception.args[0]
-        assert message == "&kwonly parameters require Python 3"
 
 
 def test_lambda_list_keywords_mixed():
     """ Ensure we can mix them up."""
     can_compile("(fn [x &rest xs &kwargs kw] (list x xs kw))")
     cant_compile("(fn [x &rest xs &fasfkey {bar \"baz\"}])")
-    if PY3:
     can_compile("(fn [x &rest xs &kwonly kwoxs &kwargs kwxs]"
                 "  (list x xs kwxs kwoxs))")
 
@@ -504,11 +475,11 @@ def test_ast_unicode_strings():
 
 
 def test_ast_unicode_vs_bytes():
-    assert s('"hello"') == u"hello"
-    assert type(s('"hello"')) is (str if PY3 else unicode)  # noqa
-    assert s('b"hello"') == (eval('b"hello"') if PY3 else "hello")
-    assert type(s('b"hello"')) is (bytes if PY3 else str)
-    assert s('b"\\xa0"') == (bytes([160]) if PY3 else chr(160))
+    assert s('"hello"') == "hello"
+    assert type(s('"hello"')) is str
+    assert s('b"hello"') == b"hello"
+    assert type(s('b"hello"')) is bytes
+    assert s('b"\\xa0"') == bytes([160])
 
 
 @pytest.mark.skipif(not PY36, reason='f-strings require Python 3.6+')
@@ -528,7 +499,7 @@ def test_ast_bracket_string():
     assert s(r'#[my delim[fizzle]my delim]') == 'fizzle'
     assert s(r'#[[]]') == ''
     assert s(r'#[my delim[]my delim]') == ''
-    assert type(s('#[X[hello]X]')) is (str if PY3 else unicode)  # noqa
+    assert type(s('#[X[hello]X]')) is str
     assert s(r'#[X[raw\nstring]X]') == 'raw\\nstring'
     assert s(r'#[foozle[aa foozli bb ]foozle]') == 'aa foozli bb '
     assert s(r'#[([unbalanced](]') == 'unbalanced'
@@ -623,30 +594,6 @@ def test_lots_of_comment_lines():
     can_compile(1000 * ";\n")
 
 
-def test_exec_star():
-
-    code = can_compile('(exec* "print(5)")').body[0]
-    assert type(code) == (ast.Expr if PY3 else ast.Exec)
-    if not PY3:
-        assert code.body.s == "print(5)"
-        assert code.globals is None
-        assert code.locals is None
-
-    code = can_compile('(exec* "print(a)" {"a" 3})').body[0]
-    assert type(code) == (ast.Expr if PY3 else ast.Exec)
-    if not PY3:
-        assert code.body.s == "print(a)"
-        assert code.globals.keys[0].s == "a"
-        assert code.locals is None
-
-    code = can_compile('(exec* "print(a + b)" {"a" "x"} {"b" "y"})').body[0]
-    assert type(code) == (ast.Expr if PY3 else ast.Exec)
-    if not PY3:
-        assert code.body.s == "print(a + b)"
-        assert code.globals.keys[0].s == "a"
-        assert code.locals.keys[0].s == "b"
-
-
 def test_compiler_macro_tag_try():
     """Check that try forms within defmacro/deftag are compiled correctly"""
     # https://github.com/hylang/hy/issues/1350
@@ -654,13 +601,11 @@ def test_compiler_macro_tag_try():
     can_compile("(deftag foo [] (try None (except [] None)) `())")
 
 
-@pytest.mark.skipif(not PY3, reason="Python 3 required")
 def test_ast_good_yield_from():
     "Make sure AST can compile valid yield-from"
     can_compile("(yield-from [1 2])")
 
 
-@pytest.mark.skipif(not PY3, reason="Python 3 required")
 def test_ast_bad_yield_from():
     "Make sure AST can't compile invalid yield-from"
     cant_compile("(yield-from)")
@@ -691,3 +636,10 @@ def test_futures_imports():
 
     assert hy_ast.body[0].module == '__future__'
     assert hy_ast.body[1].module == 'hy.core.language'
+
+
+def test_inline_python():
+    can_compile('(py "1 + 1")')
+    cant_compile('(py "1 +")')
+    can_compile('(pys "if 1:\n  2")')
+    cant_compile('(pys "if 1\n  2")')

tests/compilers/test_compiler.py

@@ -1,4 +1,4 @@
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 
@@ -6,7 +6,7 @@ import ast
 
 from hy import compiler
 from hy.models import HyExpression, HyList, HySymbol, HyInteger
-from hy._compat import PY3
+import types
 
 
 def make_expression(*args):
@@ -26,7 +26,7 @@ def test_compiler_bare_names():
                         HySymbol("a"),
                         HySymbol("b"),
                         HySymbol("c"))
-    ret = compiler.HyASTCompiler('test').compile(e)
+    ret = compiler.HyASTCompiler(types.ModuleType('test')).compile(e)
 
     # We expect two statements and a final expr.
 
@@ -55,7 +55,7 @@ def test_compiler_yield_return():
                         HyExpression([HySymbol("+"),
                                       HyInteger(1),
                                       HyInteger(1)]))
-    ret = compiler.HyASTCompiler('test').compile_atom(e)
+    ret = compiler.HyASTCompiler(types.ModuleType('test')).compile_atom(e)
 
     assert len(ret.stmts) == 1
     stmt, = ret.stmts
@@ -64,12 +64,5 @@ def test_compiler_yield_return():
     assert len(body) == 2
     assert isinstance(body[0], ast.Expr)
     assert isinstance(body[0].value, ast.Yield)
-
-    if PY3:
-        # From 3.3+, the final statement becomes a return value
     assert isinstance(body[1], ast.Return)
     assert isinstance(body[1].value, ast.BinOp)
-    else:
-        # In earlier versions, the expression is not returned
-        assert isinstance(body[1], ast.Expr)
-        assert isinstance(body[1].value, ast.BinOp)

tests/importer/test_importer.py

@@ -1,4 +1,4 @@
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 
@@ -10,6 +10,7 @@ import runpy
 import importlib
 
 from fractions import Fraction
+from importlib import reload
 
 import pytest
 
@@ -18,12 +19,7 @@ from hy.lex import hy_parse
 from hy.errors import HyLanguageError
 from hy.lex.exceptions import PrematureEndOfInput
 from hy.compiler import hy_eval, hy_compile
-from hy.importer import HyLoader, cache_from_source
-
-try:
-    from importlib import reload
-except ImportError:
-    from imp import reload
+from hy.importer import HyLoader
 
 
 def test_basics():
@@ -101,7 +97,7 @@ def test_import_autocompiles():
         f.write(b'(defn pyctest [s] (+ "X" s "Y"))')
         f.flush()
 
-        pyc_path = cache_from_source(f.name)
+        pyc_path = importlib.util.cache_from_source(f.name)
 
         try:
             os.remove(pyc_path)
@@ -144,7 +140,7 @@ def test_reload():
 
     def unlink(filename):
         os.unlink(source)
-        bytecode = cache_from_source(source)
+        bytecode = importlib.util.cache_from_source(source)
         if os.path.isfile(bytecode):
             os.unlink(bytecode)
 
@@ -238,6 +234,18 @@ def test_reload():
         unlink(source)
 
 
+def test_reload_reexecute(capsys):
+    """A module is re-executed when it's reloaded, even if it's
+    unchanged.
+
+    https://github.com/hylang/hy/issues/712"""
+    import tests.resources.hello_world
+    assert capsys.readouterr().out == 'hello world\n'
+    assert capsys.readouterr().out == ''
+    reload(tests.resources.hello_world)
+    assert capsys.readouterr().out == 'hello world\n'
+
+
 def test_circular():
     """Test circular imports by creating a temporary file/module that calls a
     function that imports itself."""

tests/importer/test_pyc.py

@@ -1,12 +1,13 @@
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 
 import os
-import imp
+import importlib.util
+import py_compile
 import tempfile
 
-import py_compile
+import hy.importer
 
 
 def test_pyc():
@@ -16,10 +17,11 @@ def test_pyc():
         f.flush()
 
         cfile = py_compile.compile(f.name)
-
         assert os.path.exists(cfile)
 
-        mod = imp.load_compiled('pyc', cfile)
+        try:
+            mod = hy.importer._import_from_path('pyc', cfile)
+        finally:
             os.remove(cfile)
 
         assert mod.pyctest('Foo') == 'XFooY'

tests/macros/test_macro_processor.py

@@ -1,4 +1,4 @@
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 

tests/macros/test_tag_macros.py

@@ -1,4 +1,4 @@
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 

tests/native_tests/comprehensions.hy

@@ -1,7 +1,6 @@
 (import
   types
-  pytest
-  [hy._compat [PY3]])
+  pytest)
 
 
 (defn test-comprehension-types []
@@ -134,8 +133,7 @@
   ; An `lfor` that gets compiled to a real comprehension
   (setv x 0)
   (assert (= (lfor x [1 2 3] (inc x)) [2 3 4]))
-  (assert (= x (if PY3 0 3)))
-    ; Python 2 list comprehensions leak their variables.
+  (assert (= x 0))
 
   ; An `lfor` that gets compiled to a loop
   (setv x 0  l [])

tests/native_tests/contrib/hy_repr.hy

@@ -1,9 +1,9 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
 (import
-  [hy._compat [PY3 PY36 PY37]]
+  [hy._compat [PY36 PY37]]
   [math [isnan]]
   [hy.contrib.hy-repr [hy-repr hy-repr-register]])
 
@@ -79,10 +79,10 @@
   (assert (is (type (get orig 1)) float))
   (assert (is (type (get result 1)) HyFloat)))
 
-(when PY3 (defn test-dict-views []
+(defn test-dict-views []
   (assert (= (hy-repr (.keys {1 2})) "(dict-keys [1])"))
   (assert (= (hy-repr (.values {1 2})) "(dict-values [2])"))
-  (assert (= (hy-repr (.items {1 2})) "(dict-items [(, 1 2)])"))))
+  (assert (= (hy-repr (.items {1 2})) "(dict-items [(, 1 2)])")))
 
 (defn test-datetime []
   (import [datetime :as D])
@@ -91,9 +91,8 @@
     "(datetime.datetime 2009 1 15 15 27 5)"))
   (assert (= (hy-repr (D.datetime 2009 1 15 15 27 5 123))
     "(datetime.datetime 2009 1 15 15 27 5 123)"))
-  (when PY3
   (assert (= (hy-repr (D.datetime 2009 1 15 15 27 5 123 :tzinfo D.timezone.utc))
-      "(datetime.datetime 2009 1 15 15 27 5 123 :tzinfo datetime.timezone.utc)")))
+    "(datetime.datetime 2009 1 15 15 27 5 123 :tzinfo datetime.timezone.utc)"))
   (when PY36
     (assert (= (hy-repr (D.datetime 2009 1 15 15 27 5 :fold 1))
       "(datetime.datetime 2009 1 15 15 27 5 :fold 1)"))
@@ -114,17 +113,11 @@
 (defn test-collections []
   (import collections)
   (assert (= (hy-repr (collections.defaultdict :a 8))
-    (if PY3
-      "(defaultdict None {\"a\" 8})"
-      "(defaultdict None {b\"a\" 8})")))
+    "(defaultdict None {\"a\" 8})"))
   (assert (= (hy-repr (collections.defaultdict int :a 8))
-    (if PY3
-      "(defaultdict <class 'int'> {\"a\" 8})"
-      "(defaultdict <type 'int'> {b\"a\" 8})")))
+    "(defaultdict <class 'int'> {\"a\" 8})"))
   (assert (= (hy-repr (collections.Counter [15 15 15 15]))
-    (if PY3
-      "(Counter {15 4})"
-      "(Counter {15 (int 4)})")))
+    "(Counter {15 4})"))
   (setv C (collections.namedtuple "Fooey" ["cd" "a_b"]))
   (assert (= (hy-repr (C 11 12))
     "(Fooey :cd 11 :a_b 12)")))
@@ -155,9 +148,8 @@
   (setv mo (re.search "b+" "aaaabbbccc"))
   (assert (= (hy-repr mo)
     (.format
-      #[[<{} object; :span {} :match "bbb">]]
-      (if PY37 "re.Match" (+ (. (type mo) __module__) ".SRE_Match"))
-      (if PY3 "(, 4 7)" "(, (int 4) (int 7))")))))
+      #[[<{} object; :span (, 4 7) :match "bbb">]]
+      (if PY37 "re.Match" (+ (. (type mo) __module__) ".SRE_Match"))))))
 
 (defn test-hy-repr-custom []
 
@@ -166,8 +158,8 @@
   (assert (= (hy-repr (C)) "cuddles"))
 
   (defclass Container [object]
-    [__init__ (fn [self value]
-      (setv self.value value))])
+    (defn __init__ [self value]
+      (setv self.value value)))
   (hy-repr-register Container :placeholder "(Container ...)" (fn [x]
     (+ "(Container " (hy-repr x.value) ")")))
   (setv container (Container 5))
@@ -178,5 +170,5 @@
 
 (defn test-hy-repr-fallback []
   (defclass D [object]
-    [__repr__ (fn [self] "cuddles")])
+    (defn __repr__ [self] "cuddles"))
   (assert (= (hy-repr (D)) "cuddles")))

tests/native_tests/contrib/loop.hy

@@ -1,4 +1,4 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 

tests/native_tests/contrib/multi.hy

@@ -1,123 +0,0 @@
-;; Copyright 2019 the authors.
-;; This file is part of Hy, which is free software licensed under the Expat
-;; license. See the LICENSE.
-
-(require [hy.contrib.multi [defmulti defmethod default-method defn]])
-(import pytest)
-
-(defn test-different-signatures []
-  "NATIVE: Test multimethods with different signatures"
-  (defmulti fun [&rest args]
-    (len args))
-
-  (defmethod fun 0 []
-    "Hello!")
-
-  (defmethod fun 1 [a]
-    a)
-
-  (defmethod fun 2 [a b]
-    "a b")
-
-  (defmethod fun 3 [a b c]
-    "a b c")
-
-  (assert (= (fun) "Hello!"))
-  (assert (= (fun "a") "a"))
-  (assert (= (fun "a" "b") "a b"))
-  (assert (= (fun "a" "b" "c") "a b c")))
-
-(defn test-different-signatures-defn []
-  "NATIVE: Test defn with different signatures"
-  (defn f1
-    ([] "")
-    ([a] "a")
-    ([a b] "a b"))
-
-  (assert (= (f1) ""))
-  (assert (= (f1 "a") "a"))
-  (assert (= (f1 "a" "b") "a b"))
-  (with [(pytest.raises TypeError)]
-    (f1 "a" "b" "c")))
-
-(defn test-basic-dispatch []
-  "NATIVE: Test basic dispatch"
-  (defmulti area [shape]
-    (:type shape))
-  
-  (defmethod area "square" [square]
-    (* (:width square)
-       (:height square)))
-  
-  (defmethod area "circle" [circle]
-    (* (** (:radius circle) 2) 
-       3.14))
-
-  (default-method area [shape]
-    0)
-
-  (assert (< 0.784 (area {:type "circle" :radius 0.5}) 0.786))
-  (assert (= (area {:type "square" :width 2 :height 2})) 4)
-  (assert (= (area {:type "non-euclid rhomboid"}) 0)))
-
-(defn test-docs []
-  "NATIVE: Test if docs are properly handled"
-  (defmulti fun [a b]
-    "docs"
-    a)
-
-  (defmethod fun "foo" [a b]
-    "foo was called")
-
-  (defmethod fun "bar" [a b]
-    "bar was called")
-  
-  (assert (= fun.--doc-- "docs")))
-
-(defn test-kwargs-handling []
-  "NATIVE: Test handling of kwargs with multimethods"
-  (defmulti fun [&kwargs kwargs]
-    (get kwargs "type"))
-
-  (defmethod fun "foo" [&kwargs kwargs]
-    "foo was called")
-
-  (defmethod fun "bar" [&kwargs kwargs]
-    "bar was called")
-
-  (assert (= (fun :type "foo" :extra "extra") "foo was called")))
-
-(defn test-basic-multi []
-  "NATIVE: Test a basic arity overloaded defn"
-  (defn f2
-    ([] "Hello!")
-    ([a] a)
-    ([a b] "a b")
-    ([a b c] "a b c"))
-
-  (assert (= (f2) "Hello!"))
-  (assert (= (f2 "a") "a"))
-  (assert (= (f2 "a" "b") "a b"))
-  (assert (= (f2 "a" "b" "c") "a b c")))
-
-
-(defn test-kw-args []
-  "NATIVE: Test if kwargs are handled correctly for arity overloading"
-  (defn f3
-    ([a] a)
-    ([&optional [a "nop"] [b "p"]] (+ a b)))
-   
-  (assert (= (f3 1) 1))
-  (assert (= (f3 :a "t") "t"))
-  (assert (= (f3 "hello " :b "world") "hello world"))
-  (assert (= (f3 :a "hello " :b "world") "hello world")))
-
-
-(defn test-docs []
-  "NATIVE: Test if docs are properly handled for arity overloading"
-  (defn f4
-    "docs"
-    ([a] (print a))
-    ([a b] (print b)))
-  
-  (assert (= f4.--doc-- "docs")))

tests/native_tests/contrib/sequences.hy

@@ -1,4 +1,4 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 

tests/native_tests/contrib/walk.hy

@@ -1,4 +1,4 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
@@ -21,17 +21,16 @@
              walk-form)))
 
 (defn test-walk []
-  (setv acc '())
-  (assert (= (walk (partial collector acc) identity walk-form)
+  (setv acc [])
+  (assert (= (list (walk (partial collector acc) identity walk-form))
              [None None]))
-  (assert (= acc walk-form))
+  (assert (= acc (list walk-form)))
   (setv acc [])
   (assert (= (walk identity (partial collector acc) walk-form)
              None))
   (assert (= acc [walk-form])))
 
 (defn test-walk-iterators []
-  (setv acc [])
   (assert (= (walk (fn [x] (* 2 x)) (fn [x] x)
                    (drop 1 [1 [2 [3 [4]]]]))
              [[2 [3 [4]] 2 [3 [4]]]])))
@@ -44,19 +43,19 @@
   (assert (= (macroexpand-all '(foo-walk))
              42))
   (assert (= (macroexpand-all '(with [a 1]))
-             '(with* [a 1] (do))))
+             '(with* [a 1])))
   (assert (= (macroexpand-all '(with [a 1 b 2 c 3] (for [d c] foo)))
-             '(with* [a 1] (with* [b 2] (with* [c 3] (do (for [d c] foo)))))))
+             '(with* [a 1] (with* [b 2] (with* [c 3] (for [d c] foo))))))
   (assert (= (macroexpand-all '(with [a 1]
                                  '(with [b 2])
                                  `(with [c 3]
                                     ~(with [d 4])
                                     ~@[(with [e 5])])))
              '(with* [a 1]
-                (do '(with [b 2])
+                '(with [b 2])
                 `(with [c 3]
-                       ~(with* [d 4] (do))
-                       ~@[(with* [e 5] (do))])))))
+                   ~(with* [d 4])
+                   ~@[(with* [e 5])]))))
 
   (defmacro require-macro []
     `(do
@@ -130,7 +129,7 @@
                '(foo `(bar a ~a ~"x"))))
     (assert (= `(foo ~@[a])
                '(foo "x")))
-    (assert (= `(foo `(bar [a] ~@[a] ~@~[a 'a `a] ~~@[a]))
+    (assert (= `(foo `(bar [a] ~@[a] ~@~(HyList [a 'a `a]) ~~@[a]))
                '(foo `(bar [a] ~@[a] ~@["x" a a] ~"x"))))))
 
 (defn test-let-except []

tests/native_tests/core.hy

@@ -1,9 +1,7 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
-(import [hy._compat [PY3]])
-
 ;;;; some simple helpers
 
 (defn assert-true [x]
@@ -287,7 +285,7 @@ result['y in globals'] = 'y' in globals()")
   (setv s3 (gensym "xx"))
   (assert (= 0 (.find s2 "_xx\uffff")))
   (assert (not (= s2 s3)))
-  (assert (not (= (string s2) (string s3)))))
+  (assert (not (= (str s2) (str s3)))))
 
 (defn test-identity []
   "NATIVE: testing the identity function"
@@ -302,7 +300,7 @@ result['y in globals'] = 'y' in globals()")
   (assert-requires-num inc)
 
   (defclass X [object]
-    [__add__ (fn [self other] (.format "__add__ got {}" other))])
+    (defn __add__ [self other] (.format "__add__ got {}" other)))
   (assert-equal (inc (X)) "__add__ got 1"))
 
 (defn test-instance []
@@ -324,8 +322,8 @@ result['y in globals'] = 'y' in globals()")
   (assert-true (integer? 0))
   (assert-true (integer? 3))
   (assert-true (integer? -3))
-  (assert-true (integer? (integer "-3")))
-  (assert-true (integer? (integer 3)))
+  (assert-true (integer? (int "-3")))
+  (assert-true (integer? (int 3)))
   (assert-false (integer? 4.2))
   (assert-false (integer? None))
   (assert-false (integer? "foo")))
@@ -334,7 +332,7 @@ result['y in globals'] = 'y' in globals()")
   "NATIVE: testing the integer-char? function"
   (assert-true (integer-char? "1"))
   (assert-true (integer-char? "-1"))
-  (assert-true (integer-char? (str (integer 300))))
+  (assert-true (integer-char? (str (int 300))))
   (assert-false (integer-char? "foo"))
   (assert-false (integer-char? None)))
 
@@ -429,8 +427,7 @@ result['y in globals'] = 'y' in globals()")
   (assert-true (neg? -2))
   (assert-false (neg? 1))
   (assert-false (neg? 0))
-  (when PY3
-    (assert-requires-num neg?)))
+  (assert-requires-num neg?))
 
 (defn test-zero []
   "NATIVE: testing the zero? function"
@@ -481,6 +478,16 @@ result['y in globals'] = 'y' in globals()")
   (assert-false (odd? 0))
   (assert-requires-num odd?))
 
+(defn test-parse-args []
+  "NATIVE: testing the parse-args function"
+  ; https://github.com/hylang/hy/issues/1875
+  (setv parsed-args (parse-args [["strings" :nargs "+" :help "Strings"]
+                                 ["-n" :action "append" :type int :help "Numbers" "--numbers"]]
+                                ["a" "b" "-n" "1" "--numbers" "2"]
+                                :description "Parse strings and numbers from args"))
+  (assert-equal parsed-args.strings ["a" "b"])
+  (assert-equal parsed-args.numbers [1 2]))
+
 (defn test-partition []
   "NATIVE: testing the partition function"
   (setv ten (range 10))
@@ -519,8 +526,7 @@ result['y in globals'] = 'y' in globals()")
   (assert-true (pos? 2))
   (assert-false (pos? -1))
   (assert-false (pos? 0))
-  (when PY3
-    (assert-requires-num pos?)))
+  (assert-requires-num pos?))
 
 (defn test-remove []
   "NATIVE: testing the remove function"
@@ -638,8 +644,8 @@ result['y in globals'] = 'y' in globals()")
   "NATIVE: testing the keyword? function"
   (assert (keyword? ':bar))
   (assert (keyword? ':baz))
-  (assert (keyword? :bar))
-  (assert (keyword? :baz))
+  (setv x :bar)
+  (assert (keyword? x))
   (assert (not (keyword? "foo")))
   (assert (not (keyword? ":foo")))
   (assert (not (keyword? 1)))

tests/native_tests/defclass.hy

@@ -1,4 +1,4 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
@@ -27,7 +27,7 @@
 (defn test-defclass-attrs []
   "NATIVE: test defclass attributes"
   (defclass A []
-    [x 42])
+    (setv x 42))
   (assert (= A.x 42))
   (assert (= (getattr (A) "x")  42)))
 
@@ -35,9 +35,9 @@
 (defn test-defclass-attrs-fn []
   "NATIVE: test defclass attributes with fn"
   (defclass B []
-    [x 42
-     y (fn [self value]
-         (+ self.x value))])
+    (setv x 42)
+    (setv y (fn [self value]
+      (+ self.x value))))
   (assert (= B.x 42))
   (assert (= (.y (B) 5) 47))
   (setv b (B))
@@ -48,17 +48,17 @@
 (defn test-defclass-dynamic-inheritance []
   "NATIVE: test defclass with dynamic inheritance"
   (defclass A [((fn [] (if True list dict)))]
-    [x 42])
+    (setv x 42))
   (assert (isinstance (A) list))
   (defclass A [((fn [] (if False list dict)))]
-    [x 42])
+    (setv x 42))
   (assert (isinstance (A) dict)))
 
 
 (defn test-defclass-no-fn-leak []
   "NATIVE: test defclass attributes with fn"
   (defclass A []
-    [x (fn [] 1)])
+    (setv x (fn [] 1)))
   (try
    (do
     (x)
@@ -68,13 +68,13 @@
 (defn test-defclass-docstring []
   "NATIVE: test defclass docstring"
   (defclass A []
-    [--doc-- "doc string"
-     x 1])
+    (setv --doc-- "doc string")
+    (setv x 1))
   (setv a (A))
   (assert (= a.__doc__ "doc string"))
   (defclass B []
     "doc string"
-    [x 1])
+    (setv x 1))
   (setv b (B))
   (assert (= b.x 1))
   (assert (= b.__doc__ "doc string"))
@@ -82,7 +82,7 @@
     "begin a very long multi-line string to make
      sure that it comes out the way we hope
      and can span 3 lines end."
-    [x 1])
+    (setv x 1))
   (setv mL (MultiLine))
   (assert (= mL.x 1))
   (assert (in "begin" mL.__doc__))
@@ -100,8 +100,8 @@
   "NATIVE: test defclass syntax with properties and methods and side-effects"
   (setv foo 1)
   (defclass A []
-    [x 1
-     y 2]
+    (setv x 1)
+    (setv y 2)
     (global foo)
     (setv foo 2)
     (defn greet [self]
@@ -117,7 +117,7 @@
 (defn test-defclass-implicit-none-for-init []
   "NATIVE: test that defclass adds an implicit None to --init--"
   (defclass A []
-    [--init-- (fn [self] (setv self.x 1) 42)])
+    (setv --init-- (fn [self] (setv self.x 1) 42)))
   (defclass B []
     (defn --init-- [self]
       (setv self.x 2)

tests/native_tests/extra/anaphoric.hy

@@ -1,134 +1,209 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
 (import [hy.errors [HyMacroExpansionError]])
 (require [hy.extra.anaphoric [*]])
 
-;;;; some simple helpers
-
-(defn assert-true [x]
-  (assert (= True x)))
-
-(defn assert-false [x]
-  (assert (= False x)))
-
-(defn assert-equal [x y]
-  (assert (= x y)))
-
-
 (defn test-ap-if []
-  "NATIVE: testing anaphoric if"
-  (ap-if True (assert-true it))
-  (ap-if False True (assert-false it)))
+  (ap-if True (assert (is it True)))
+  (ap-if False True (assert (is it False)))
+
+  ; https://github.com/hylang/hy/issues/1847
+  (setv it "orig")
+  (setv out (ap-if (+ 1 1) (+ it 1) (+ it 10)))
+  (assert (= out 3))
+  (assert (= it "orig"))
+
+  (ap-if
+    (->> [1 2 3 4 5]
+      (ap-filter (= (% it 2) 0))
+      (list))
+    (assert (= it [2 4]))))
+
 
 (defn test-ap-each []
-  "NATIVE: testing anaphoric each"
   (setv res [])
-  (ap-each [1 2 3 4] (.append res it))
-  (assert-equal res [1 2 3 4]))
+  (assert (is (ap-each [1 2 3 4] (.append res it)) None))
+  (assert (= res [1 2 3 4]))
+
+  (setv res [])
+  (ap-each
+    (->> [1 2 3 4]
+      (ap-map (+ 1 it))
+      (list))
+    (.append res it))
+  (assert (= res [2 3 4 5])))
 
 (defn test-ap-each-while []
-  "NATIVE: testing anaphoric each-while"
   (setv res [])
   (ap-each-while [2 2 4 3 4 5 6] (even? it) (.append res it))
-  (assert-equal res [2 2 4]))
+  (assert (= res [2 2 4]))
+
+  (setv res [])
+  (ap-each-while
+    (->> [2 2 4 3 4 5 6]
+      (ap-map (+ 1 it))
+      (list))
+    (odd? it) (.append res it))
+  (assert (= res [3 3 5])))
 
 (defn test-ap-map []
-  "NATIVE: testing anaphoric map"
-  (assert-equal (list (ap-map (* it 3) [1 2 3]))
-                [3 6 9])
-  (assert-equal (list (ap-map (* it 3) []))
-                [])
-  (assert-equal (do (setv v 1 f 1) (list (ap-map (it v f) [(fn [a b] (+ a b))])))
+  (assert (= (list (ap-map (* it 3) [1 2 3]))
+             [3 6 9]))
+  (assert (= (list (ap-map (* it 3) []))
+             []))
+  (assert (= (do (setv v 1 f 1) (list (ap-map (it v f) [(fn [a b] (+ a b))])))
              [2]))
 
+  (assert (=
+    (->> [1 2 3]
+      (ap-filter (even? it))
+      (ap-map (* 3 it))
+      (list))
+    [6])))
+
 (defn test-ap-map-when []
-  "NATIVE: testing anaphoric map-when"
-  (assert-equal (list (ap-map-when even? (* it 2) [1 2 3 4]))
+  (assert (= (list (ap-map-when even? (* it 2) [1 2 3 4]))
              [1 4 3 8]))
 
+  (assert (=
+    (->> [1 2 3 4]
+      (ap-map (+ 1 it))
+      (ap-map-when even? (* 2 it))
+      (list))
+    [4 3 8 5])))
+
 (defn test-ap-filter []
-  "NATIVE: testing anaphoric filter"
-  (assert-equal (list (ap-filter (> it 2) [1 2 3 4]))
-                [3 4])
-  (assert-equal (list (ap-filter (even? it) [1 2 3 4]))
+  (assert (= (list (ap-filter (> it 2) [1 2 3 4]))
+             [3 4]))
+  (assert (= (list (ap-filter (even? it) [1 2 3 4]))
              [2 4]))
 
+  (assert (=
+    (->> [1 2 3 4]
+      (ap-map (+ 3 it))
+      (ap-filter (even? it))
+      (list))
+    [4 6])))
+
 (defn test-ap-reject []
-  "NATIVE: testing anaphoric filter"
-  (assert-equal (list (ap-reject (> it 2) [1 2 3 4]))
-                [1 2])
-  (assert-equal (list (ap-reject (even? it) [1 2 3 4]))
+  (assert (= (list (ap-reject (> it 2) [1 2 3 4]))
+             [1 2]))
+  (assert (= (list (ap-reject (even? it) [1 2 3 4]))
              [1 3]))
 
+  (assert (=
+    (->> [1 2 3 4]
+      (ap-map (+ 3 it))
+      (ap-reject (even? it))
+      (list))
+    [5 7])))
+
 (defn test-ap-dotimes []
-  "NATIVE: testing anaphoric dotimes"
-  (assert-equal (do (setv n []) (ap-dotimes 3 (.append n 3)) n)
-                [3 3 3])
-  (assert-equal (do (setv n []) (ap-dotimes 3 (.append n it)) n)
+  (assert (= (do (setv n []) (ap-dotimes 3 (.append n 3)) n)
+             [3 3 3]))
+  (assert (= (do (setv n []) (ap-dotimes 3 (.append n it)) n)
              [0 1 2]))
 
+  ; https://github.com/hylang/hy/issues/1853
+  (setv n 5)
+  (setv x "")
+  (ap-dotimes n (+= x "."))
+  (assert (= x "....."))
+
+  (assert (=
+    (do
+      (setv n [])
+      (ap-dotimes
+        (ap-first (odd? it) [2 4 5 6 3 8])
+        (.append n it))
+      n)
+    [0 1 2 3 4])))
+
 (defn test-ap-first []
-  "NATIVE: testing anaphoric first"
-  (assert-equal (ap-first (> it 5) (range 10)) 6)
-  (assert-equal (ap-first (even? it) [1 2 3 4]) 2)
-  (assert-equal (ap-first (> it 10) (range 10)) None))
+  (assert (= (ap-first (> it 5) (range 10)) 6))
+  (assert (= (ap-first (even? it) [1 2 3 4]) 2))
+  (assert (= (ap-first (> it 10) (range 10)) None))
+
+  (assert (=
+    (->> [1 2 3 4]
+      (ap-map (+ 4 it))
+      (ap-first (even? it)))
+    6)))
 
 (defn test-ap-last []
-  "NATIVE: testing anaphoric last"
-  (assert-equal (ap-last (> it 5) (range 10)) 9)
-  (assert-equal (ap-last (even? it) [1 2 3 4]) 4)
-  (assert-equal (ap-last (> it 10) (range 10)) None))
+  (assert (= (ap-last (> it 5) (range 10)) 9))
+  (assert (= (ap-last (even? it) [1 2 3 4]) 4))
+  (assert (= (ap-last (> it 10) (range 10)) None))
+
+  (assert (=
+    (->> [1 2 3 4]
+      (ap-map (+ 4 it))
+      (ap-last (odd? it)))
+    7)))
 
 (defn test-ap-reduce []
-  "NATIVE: testing anaphoric reduce"
-  (assert-equal (ap-reduce (* acc it) [1 2 3]) 6)
-  (assert-equal (ap-reduce (* acc it) [1 2 3] 6) 36)
-  (assert-equal (ap-reduce (+ acc " on " it) ["Hy" "meth"])
-                "Hy on meth")
-  (assert-equal (ap-reduce (+ acc it) [] 1) 1))
+  (assert (= (ap-reduce (* acc it) [1 2 3]) 6))
+  (assert (= (ap-reduce (* acc it) [1 2 3] 6) 36))
+  (assert (= (ap-reduce (+ acc " on " it) ["Hy" "meth"])
+              "Hy on meth"))
+  (assert (= (ap-reduce (+ acc it) [] 1) 1))
+
+  ; https://github.com/hylang/hy/issues/1848
+  (assert (= (ap-reduce (* acc it) (map inc [1 2 3])) 24))
+  (assert (= (ap-reduce (* acc it) (map inc [1 2 3]) 4) 96))
+
+  (setv expr-evaluated 0)
+  (assert (=
+    (ap-reduce (* acc it) (do (+= expr-evaluated 1) [4 5 6])))
+    120)
+  (assert (= expr-evaluated 1))
+
+  (assert (=
+    (->> [1 2 3]
+      (ap-map (+ 2 it))
+      (ap-reduce (* acc it)))
+    60)))
 
 (defn test-tag-fn []
-  "NATIVE: testing #%() forms"
   ;; test ordering
-  (assert-equal (#%(/ %1 %2) 2 4) 0.5)
-  (assert-equal (#%(/ %2 %1) 2 4) 2)
-  (assert-equal (#%(identity (, %5 %4 %3 %2 %1)) 1 2 3 4 5) (, 5 4 3 2 1))
-  (assert-equal (#%(identity (, %1 %2 %3 %4 %5)) 1 2 3 4 5) (, 1 2 3 4 5))
-  (assert-equal (#%(identity (, %1 %5 %2 %3 %4)) 1 2 3 4 5) (, 1 5 2 3 4))
+  (assert (= (#%(/ %1 %2) 2 4) 0.5))
+  (assert (= (#%(/ %2 %1) 2 4) 2))
+  (assert (= (#%(identity (, %5 %4 %3 %2 %1)) 1 2 3 4 5) (, 5 4 3 2 1)))
+  (assert (= (#%(identity (, %1 %2 %3 %4 %5)) 1 2 3 4 5) (, 1 2 3 4 5)))
+  (assert (= (#%(identity (, %1 %5 %2 %3 %4)) 1 2 3 4 5) (, 1 5 2 3 4)))
   ;; test &rest
-  (assert-equal (#%(sum %*) 1 2 3) 6)
-  (assert-equal (#%(identity (, %1 %*)) 10 1 2 3) (, 10 (, 1 2 3)))
+  (assert (= (#%(sum %*) 1 2 3) 6))
+  (assert (= (#%(identity (, %1 %*)) 10 1 2 3) (, 10 (, 1 2 3))))
   ;; no parameters
-  (assert-equal (#%(list)) [])
-  (assert-equal (#%(identity "Hy!")) "Hy!")
-  (assert-equal (#%(identity "%*")) "%*")
-  (assert-equal (#%(+ "Hy " "world!")) "Hy world!")
+  (assert (= (#%(list)) []))
+  (assert (= (#%(identity "Hy!")) "Hy!"))
+  (assert (= (#%(identity "%*")) "%*"))
+  (assert (= (#%(+ "Hy " "world!")) "Hy world!"))
   ;; test skipped parameters
-  (assert-equal (#%(identity [%3 %1]) 1 2 3) [3 1])
+  (assert (= (#%(identity [%3 %1]) 1 2 3) [3 1]))
   ;; test nesting
-  (assert-equal (#%(identity [%1 (, %2 [%3] "Hy" [%*])]) 1 2 3 4 5)
-                [1 (, 2 [3] "Hy" [(, 4 5)])])
+  (assert (= (#%(identity [%1 (, %2 [%3] "Hy" [%*])]) 1 2 3 4 5)
+             [1 (, 2 [3] "Hy" [(, 4 5)])]))
   ;; test arg as function
-  (assert-equal (#%(%1 2 4) +) 6)
-  (assert-equal (#%(%1 2 4) -) -2)
-  (assert-equal (#%(%1 2 4) /) 0.5)
+  (assert (= (#%(%1 2 4) +) 6))
+  (assert (= (#%(%1 2 4) -) -2))
+  (assert (= (#%(%1 2 4) /) 0.5))
   ;; test &rest &kwargs
-  (assert-equal (#%(, %* %**) 1 2 :a 'b)
+  (assert (= (#%(, %* %**) 1 2 :a 'b)
              (, (, 1 2)
-                   (dict :a 'b)))
+                (dict :a 'b))))
   ;; test other expression types
-  (assert-equal (#% %* 1 2 3)
-                (, 1 2 3))
-  (assert-equal (#% %** :foo 2)
-                (dict :foo 2))
-  (assert-equal (#%[%3 %2 %1] 1 2 3)
-                [3 2 1])
-  (assert-equal (#%{%1 %2} 10 100)
-                {10 100})
-  (assert-equal (#% #{%3 %2 %1} 1 3 2)
-                #{3 1 2})  ; sets are not ordered.
-  (assert-equal (#% "%1")
-                "%1"))
-
+  (assert (= (#% %* 1 2 3)
+             (, 1 2 3)))
+  (assert (= (#% %** :foo 2)
+             (dict :foo 2)))
+  (assert (= (#%[%3 %2 %1] 1 2 3)
+             [3 2 1]))
+  (assert (= (#%{%1 %2} 10 100)
+             {10 100}))
+  (assert (= (#% #{%3 %2 %1} 1 3 2)
+             #{3 1 2}))  ; sets are not ordered.
+  (assert (= (#% "%1")
+             "%1")))

tests/native_tests/extra/anaphoric_single.hy

@@ -0,0 +1,11 @@
+;; Copyright 2019 the authors.
+;; This file is part of Hy, which is free software licensed under the Expat
+;; license. See the LICENSE.
+
+(require [hy.extra.anaphoric [ap-last]])
+
+(defn test-anaphoric-single-require []
+  ; https://github.com/hylang/hy/issues/1853#issuecomment-568192529
+  ; `ap-last` should work even if `require`d without anything else
+  ; from the anaphoric module.
+  (assert (= (ap-last (> it 0) [-1 1 0 3 2 0 -1]) 2)))

tests/native_tests/extra/reserved.hy

@@ -1,14 +1,13 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
-(import [hy.extra.reserved [names]] [hy._compat [PY3]])
+(import [hy.extra.reserved [names]])
 
 (defn test-reserved []
   (assert (is (type (names)) frozenset))
   (assert (in "and" (names)))
-  (when PY3
-    (assert (in "False" (names))))
+  (assert (in "False" (names)))
   (assert (in "pass" (names)))
   (assert (in "class" (names)))
   (assert (in "defclass" (names)))

tests/native_tests/language.hy

@@ -1,4 +1,4 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
@@ -11,7 +11,7 @@
         pytest)
 (import sys)
 
-(import [hy._compat [PY3 PY37 PY38]])
+(import [hy._compat [PY38]])
 
 (defn test-sys-argv []
   "NATIVE: test sys.argv"
@@ -68,16 +68,15 @@
   "NATIVE: test that setv doesn't work on names Python can't assign to
   and that we can't mangle"
   (try (eval '(setv None 1))
-       (except [e [SyntaxError]] (assert (in "Can't assign to" (str e)))))
+       (except [e [SyntaxError]] (assert (in "illegal target for assignment" (str e)))))
   (try (eval '(defn None [] (print "hello")))
-       (except [e [SyntaxError]] (assert (in "Can't assign to" (str e)))))
-  (when PY3
+       (except [e [SyntaxError]] (assert (in "illegal target for assignment" (str e)))))
   (try (eval '(setv False 1))
-         (except [e [SyntaxError]] (assert (in "Can't assign to" (str e)))))
+       (except [e [SyntaxError]] (assert (in "illegal target for assignment" (str e)))))
   (try (eval '(setv True 0))
-         (except [e [SyntaxError]] (assert (in "Can't assign to" (str e)))))
+       (except [e [SyntaxError]] (assert (in "illegal target for assignment" (str e)))))
   (try (eval '(defn True [] (print "hello")))
-         (except [e [SyntaxError]] (assert (in "Can't assign to" (str e)))))))
+       (except [e [SyntaxError]] (assert (in "illegal target for assignment" (str e))))))
 
 
 (defn test-setv-pairs []
@@ -130,8 +129,8 @@
   (assert (none? (setv (get {} "x") 42)))
   (setv l [])
   (defclass Foo [object]
-    [__setattr__ (fn [self attr val]
-      (.append l [attr val]))])
+    (defn __setattr__ [self attr val]
+      (.append l [attr val])))
   (setv x (Foo))
   (assert (none? (setv x.eggs "ham")))
   (assert (not (hasattr x "eggs")))
@@ -198,7 +197,23 @@
     (.append l 1)
     (len l))
   (while (!= (f) 4) (do))
-  (assert (= l [1 1 1 1])))
+  (assert (= l [1 1 1 1]))
+
+  ; only compile the condition once
+  ; https://github.com/hylang/hy/issues/1790
+  (global while-cond-var)
+  (setv while-cond-var 10)
+  (eval
+    '(do
+      (defmacro while-cond []
+        (global while-cond-var)
+        (assert (= while-cond-var 10))
+        (+= while-cond-var 1)
+        `(do
+          (setv x 3)
+          False))
+      (while (while-cond))
+      (assert (= x 3)))))
 
 (defn test-while-loop-else []
   (setv count 5)
@@ -444,9 +459,9 @@
 
   (defclass X [object] [])
   (defclass M [object]
-    [meth (fn [self &rest args &kwargs kwargs]
+    (defn meth [self &rest args &kwargs kwargs]
       (.join " " (+ (, "meth") args
-        (tuple (map (fn [k] (get kwargs k)) (sorted (.keys kwargs)))))))])
+        (tuple (map (fn [k] (get kwargs k)) (sorted (.keys kwargs))))))))
 
   (setv x (X))
   (setv m (M))
@@ -513,9 +528,7 @@
   (setv passed False)
   (try
    (raise)
-   ;; Python 2 raises IndexError here (due to the previous test)
-   ;; Python 3 raises RuntimeError
-   (except [[IndexError RuntimeError]]
+   (except [RuntimeError]
      (setv passed True)))
   (assert passed)
 
@@ -747,16 +760,11 @@
 (defn test-yield-with-return []
   "NATIVE: test yield with return"
   (defn gen [] (yield 3) "goodbye")
-  (if PY3
-    (do (setv gg (gen))
+  (setv gg (gen))
   (assert (= 3 (next gg)))
   (try (next gg)
        (except [e StopIteration] (assert (hasattr e "value"))
                                  (assert (= (getattr e "value") "goodbye")))))
-    (do (setv gg (gen))
-        (assert (= 3 (next gg)))
-        (try (next gg)
-             (except [e StopIteration] (assert (not (hasattr e "value"))))))))
 
 
 (defn test-yield-in-try []
@@ -900,6 +908,22 @@
   (assert (= mooey.__name__ "mooey")))
 
 
+(defn test-defn-annotations []
+  "NATIVE: test that annotations in defn work"
+
+  (defn f [^int p1 p2 ^str p3 &optional ^str o1 ^int [o2 0]
+           &rest ^str rest &kwonly ^str k1 ^int [k2 0] &kwargs ^bool kwargs])
+
+  (assert (= (. f __annotations__ ["p1"]) int))
+  (assert (= (. f __annotations__ ["p3"]) str))
+  (assert (= (. f __annotations__ ["o1"]) str))
+  (assert (= (. f __annotations__ ["o2"]) int))
+  (assert (= (. f __annotations__ ["rest"]) str))
+  (assert (= (. f __annotations__ ["k1"]) str))
+  (assert (= (. f __annotations__ ["k2"]) int))
+  (assert (= (. f __annotations__ ["kwargs"]) bool)))
+
+
 (defn test-return []
 
   ; `return` in main line
@@ -1096,7 +1120,8 @@
 
   (assert (= :foo :foo))
   (assert (= :foo ':foo))
-  (assert (is (type :foo) (type ':foo)))
+  (setv x :foo)
+  (assert (is (type x) (type ':foo)))
   (assert (= (get {:foo "bar"} :foo) "bar"))
   (assert (= (get {:bar "quux"} (get {:foo :bar} :foo)) "quux")))
 
@@ -1111,9 +1136,9 @@
 (defn test-empty-keyword []
   "NATIVE: test that the empty keyword is recognized"
   (assert (= : :))
-  (assert (keyword? :))
+  (assert (keyword? ':))
   (assert (!= : ":"))
-  (assert (= (name :) "")))
+  (assert (= (name ':) "")))
 
 
 (defn test-nested-if []
@@ -1190,12 +1215,15 @@
       5j 5.1j 2+1j 1.2+3.4j
       "" b""
       "apple bloom" b"apple bloom" "⚘" b"\x00"
-      :mykeyword
       [] #{} {}
       [1 2 3] #{1 2 3} {"a" 1 "b" 2}]]
     (assert (= (eval `(identity ~x)) x))
     (assert (= (eval x) x)))
 
+  (setv kw :mykeyword)
+  (assert (= (get (eval `[~kw]) 0) kw))
+  (assert (= (eval kw) kw))
+
   ; Tuples wrap to HyLists, not HyExpressions.
   (assert (= (eval (,)) []))
   (assert (= (eval (, 1 2 3)) [1 2 3]))
@@ -1242,19 +1270,14 @@ cee\"} dee" "ey bee\ncee dee"))
   ; Conversion characters and format specifiers
   (setv p:9 "other")
   (setv !r "bar")
-  (defn u [s]
-    ; Add a "u" prefix for Python 2.
-    (if PY3
-      s
-      (.replace (.replace s "'" "u'" 1) "  " " " 1)))
-  (assert (= f"a{p !r}" (u "a'xyzzy'")))
+  (assert (= f"a{p !r}" "a'xyzzy'"))
   (assert (= f"a{p :9}" "axyzzy    "))
   (assert (= f"a{p:9}" "aother"))
-  (assert (= f"a{p !r :9}" (u "a'xyzzy'  ")))
-  (assert (= f"a{p !r:9}" (u "a'xyzzy'  ")))
+  (assert (= f"a{p !r :9}" "a'xyzzy'  "))
+  (assert (= f"a{p !r:9}" "a'xyzzy'  "))
   (assert (= f"a{p:9 :9}" "aother    "))
   (assert (= f"a{!r}" "abar"))
-  (assert (= f"a{!r !r}" (u "a'bar'")))
+  (assert (= f"a{!r !r}" "a'bar'"))
 
   ; Fun with `r`
   (assert (= f"hello {r\"\\n\"}" r"hello \n"))
@@ -1278,9 +1301,18 @@ cee\"} dee" "ey bee\ncee dee"))
   (assert (= f"{(C) :  {(str (+ 1 1)) !r :x<5}}" "C[  '2'xx]"))
 
   ; Format bracket strings
-  (assert (= #[f[a{p !r :9}]f] (u "a'xyzzy'  ")))
+  (assert (= #[f[a{p !r :9}]f] "a'xyzzy'  "))
   (assert (= #[f-string[result: {value :{width}.{precision}}]f-string]
-    "result:      12.34")))
+    "result:      12.34"))
+
+  ; Quoting shouldn't evaluate the f-string immediately
+  ; https://github.com/hylang/hy/issues/1844
+  (setv quoted 'f"hello {world}")
+  (assert quoted.is-format)
+  (with [(pytest.raises NameError)]
+    (eval quoted))
+  (setv world "goodbye")
+  (assert (= (eval quoted) "hello goodbye")))
 
 
 (defn test-import-syntax []
@@ -1482,11 +1514,6 @@ cee\"} dee" "ey bee\ncee dee"))
   (assert (= y [5])))
 
 
-(defn test-string []
-  (assert (string? (string "a")))
-  (assert (string? (string 1)))
-  (assert (= u"unicode" (string "unicode"))))
-
 (defn test-del []
   "NATIVE: Test the behavior of del"
   (setv foo 42)
@@ -1549,17 +1576,12 @@ cee\"} dee" "ey bee\ncee dee"))
 
 (defn test-disassemble []
   "NATIVE: Test the disassemble function"
-  (assert (= (disassemble '(do (leaky) (leaky) (macros))) (cond
-    [PY3 (.format "Module(
+  (assert (= (disassemble '(do (leaky) (leaky) (macros)))
+    (.format "Module(
     body=[Expr(value=Call(func=Name(id='leaky'), args=[], keywords=[])),
         Expr(value=Call(func=Name(id='leaky'), args=[], keywords=[])),
         Expr(value=Call(func=Name(id='macros'), args=[], keywords=[]))]{})"
-      (if PY38 ",\n    type_ignores=[]" ""))]
-    [True "Module(
-    body=[
-        Expr(value=Call(func=Name(id='leaky'), args=[], keywords=[], starargs=None, kwargs=None)),
-        Expr(value=Call(func=Name(id='leaky'), args=[], keywords=[], starargs=None, kwargs=None)),
-        Expr(value=Call(func=Name(id='macros'), args=[], keywords=[], starargs=None, kwargs=None))])"])))
+      (if PY38 ",\n    type_ignores=[]" ""))))
   (assert (= (disassemble '(do (leaky) (leaky) (macros)) True)
              "leaky()
 leaky()
@@ -1597,9 +1619,7 @@ macros()
 
 (defn test-read []
   "NATIVE: test that read takes something for stdin and reads"
-  (if-python2
-    (import [StringIO [StringIO]])
-    (import [io [StringIO]]))
+  (import [io [StringIO]])
   (import [hy.models [HyExpression]])
 
   (setv stdin-buffer (StringIO "(+ 2 2)\n(- 2 2)"))
@@ -1641,7 +1661,8 @@ macros()
   (assert (= (keyword 'foo) :foo))
   (assert (= (keyword 'foo-bar) :foo-bar))
   (assert (= (keyword 1) :1))
-  (assert (= (keyword :foo_bar) :foo-bar)))
+  (setv x :foo_bar)
+  (assert (= (keyword x) :foo-bar)))
 
 (defn test-name-conversion []
   "NATIVE: Test name conversion"
@@ -1653,8 +1674,8 @@ macros()
   (assert (= (name 'foo_bar) "foo-bar"))
   (assert (= (name 1) "1"))
   (assert (= (name 1.0) "1.0"))
-  (assert (= (name :foo) "foo"))
-  (assert (= (name :foo_bar) "foo-bar"))
+  (assert (= (name ':foo) "foo"))
+  (assert (= (name ':foo_bar) "foo-bar"))
   (assert (= (name test-name-conversion) "test-name-conversion")))
 
 (defn test-keywords []
@@ -1678,21 +1699,20 @@ macros()
    (= (identify-keywords 1 "bloo" :foo)
       ["other" "other" "keyword"])))
 
-#@(pytest.mark.xfail
 (defn test-assert-multistatements []
   ; https://github.com/hylang/hy/issues/1390
-  (setv s (set))
+  (setv l [])
   (defn f [x]
-    (.add s x)
+    (.append l x)
     False)
   (with [(pytest.raises AssertionError)]
     (assert (do (f 1) (f 2)) (do (f 3) (f 4))))
-  (assert (= s #{1 2 3 4}))))
+  (assert (= l [1 2 3 4])))
 
 (defn test-underscore_variables []
   ; https://github.com/hylang/hy/issues/1340
   (defclass XYZ []
-    [_42 6])
+    (setv _42 6))
   (setv x (XYZ))
   (assert (= (. x _42) 6)))
 
@@ -1716,3 +1736,206 @@ macros()
   "Make sure relative imports work properly"
   (import [..resources [tlib]])
   (assert (= tlib.SECRET-MESSAGE "Hello World")))
+
+
+(defn test-exception-cause []
+  (try (raise ValueError :from NameError)
+       (except [e [ValueError]]
+         (assert (= (type (. e __cause__)) NameError)))))
+
+
+(defn test-kwonly []
+  "NATIVE: test keyword-only arguments"
+  ;; keyword-only with default works
+  (defn kwonly-foo-default-false [&kwonly [foo False]] foo)
+  (assert (= (kwonly-foo-default-false) False))
+  (assert (= (kwonly-foo-default-false :foo True) True))
+  ;; keyword-only without default ...
+  (defn kwonly-foo-no-default [&kwonly foo] foo)
+  (setv attempt-to-omit-default (try
+                                  (kwonly-foo-no-default)
+                                  (except [e [Exception]] e)))
+  ;; works
+  (assert (= (kwonly-foo-no-default :foo "quux") "quux"))
+  ;; raises TypeError with appropriate message if not supplied
+  (assert (isinstance attempt-to-omit-default TypeError))
+  (assert (in "missing 1 required keyword-only argument: 'foo'"
+              (. attempt-to-omit-default args [0])))
+  ;; keyword-only with other arg types works
+  (defn function-of-various-args [a b &rest args &kwonly foo &kwargs kwargs]
+    (, a b args foo kwargs))
+  (assert (= (function-of-various-args 1 2 3 4 :foo 5 :bar 6 :quux 7)
+             (, 1 2 (, 3 4)  5 {"bar" 6 "quux" 7}))))
+
+
+(defn test-extended-unpacking-1star-lvalues []
+  (setv [x #*y] [1 2 3 4])
+  (assert (= x 1))
+  (assert (= y [2 3 4]))
+  (setv [a #*b c] "ghijklmno")
+  (assert (= a "g"))
+  (assert (= b (list "hijklmn")))
+  (assert (= c "o")))
+
+
+(defn test-yield-from []
+  "NATIVE: testing yield from"
+  (defn yield-from-test []
+    (for [i (range 3)]
+      (yield i))
+    (yield-from [1 2 3]))
+  (assert (= (list (yield-from-test)) [0 1 2 1 2 3])))
+
+
+(defn test-yield-from-exception-handling []
+  "NATIVE: Ensure exception handling in yield from works right"
+  (defn yield-from-subgenerator-test []
+    (yield 1)
+    (yield 2)
+    (yield 3)
+    (assert 0))
+  (defn yield-from-test []
+    (for [i (range 3)]
+      (yield i))
+    (try
+      (yield-from (yield-from-subgenerator-test))
+      (except [e AssertionError]
+        (yield 4))))
+  (assert (= (list (yield-from-test)) [0 1 2 1 2 3 4])))
+
+(require [hy.contrib.walk [let]])
+
+(defn test-let-optional []
+  (let [a 1
+        b 6
+        d 2]
+       (defn foo [&kwonly [a a] b [c d]]
+         (, a b c))
+       (assert (= (foo :b "b")
+                  (, 1 "b" 2)))
+       (assert (= (foo :b 20 :a 10 :c 30)
+                  (, 10 20 30)))))
+
+(defn test-pep-3115 []
+  (defclass member-table [dict]
+    (defn --init-- [self]
+      (setv self.member-names []))
+
+    (defn --setitem-- [self key value]
+      (if (not-in key self)
+          (.append self.member-names key))
+      (dict.--setitem-- self key value)))
+
+  (defclass OrderedClass [type]
+    (setv --prepare-- (classmethod (fn [metacls name bases]
+      (member-table))))
+
+    (defn --new-- [cls name bases classdict]
+      (setv result (type.--new-- cls name bases (dict classdict)))
+      (setv result.member-names classdict.member-names)
+      result))
+
+  (defclass MyClass [:metaclass OrderedClass]
+    (defn method1 [self] (pass))
+    (defn method2 [self] (pass)))
+
+  (assert (= (. (MyClass) member-names)
+             ["__module__" "__qualname__" "method1" "method2"])))
+
+
+(import [asyncio [get-event-loop sleep]])
+
+
+(defn test-unpacking-pep448-1star []
+  (setv l [1 2 3])
+  (setv p [4 5])
+  (assert (= ["a" #*l "b" #*p #*l] ["a" 1 2 3 "b" 4 5 1 2 3]))
+  (assert (= (, "a" #*l "b" #*p #*l) (, "a" 1 2 3 "b" 4 5 1 2 3)))
+  (assert (= #{"a" #*l "b" #*p #*l} #{"a" "b" 1 2 3 4 5}))
+  (defn f [&rest args] args)
+  (assert (= (f "a" #*l "b" #*p #*l) (, "a" 1 2 3 "b" 4 5 1 2 3)))
+  (assert (= (+ #*l #*p) 15))
+  (assert (= (and #*l) 3)))
+
+
+(defn test-unpacking-pep448-2star []
+  (setv d1 {"a" 1 "b" 2})
+  (setv d2 {"c" 3 "d" 4})
+  (assert (= {1 "x" #**d1 #**d2 2 "y"} {"a" 1 "b" 2 "c" 3 "d" 4 1 "x" 2 "y"}))
+  (defn fun [&optional a b c d e f] [a b c d e f])
+  (assert (= (fun #**d1 :e "eee" #**d2) [1 2 3 4 "eee" None])))
+
+
+(defn run-coroutine [coro]
+  "Run a coroutine until its done in the default event loop."""
+  (.run_until_complete (get-event-loop) (coro)))
+
+
+(defn test-fn/a []
+  (assert (= (run-coroutine (fn/a [] (await (sleep 0)) [1 2 3]))
+             [1 2 3])))
+
+
+(defn test-defn/a []
+  (defn/a coro-test []
+    (await (sleep 0))
+    [1 2 3])
+  (assert (= (run-coroutine coro-test) [1 2 3])))
+
+
+(defn test-decorated-defn/a []
+  (defn decorator [func] (fn/a [] (/ (await (func)) 2)))
+
+  #@(decorator
+      (defn/a coro-test []
+        (await (sleep 0))
+        42))
+  (assert (= (run-coroutine coro-test) 21)))
+
+
+(defclass AsyncWithTest []
+  (defn --init-- [self val]
+    (setv self.val val)
+    None)
+
+  (defn/a --aenter-- [self]
+    self.val)
+
+  (defn/a --aexit-- [self tyle value traceback]
+    (setv self.val None)))
+
+
+(defn test-single-with/a []
+  (run-coroutine
+    (fn/a []
+      (with/a [t (AsyncWithTest 1)]
+        (assert (= t 1))))))
+
+(defn test-two-with/a []
+  (run-coroutine
+    (fn/a []
+      (with/a [t1 (AsyncWithTest 1)
+               t2 (AsyncWithTest 2)]
+        (assert (= t1 1))
+        (assert (= t2 2))))))
+
+(defn test-thrice-with/a []
+  (run-coroutine
+    (fn/a []
+      (with/a [t1 (AsyncWithTest 1)
+               t2 (AsyncWithTest 2)
+               t3 (AsyncWithTest 3)]
+        (assert (= t1 1))
+        (assert (= t2 2))
+        (assert (= t3 3))))))
+
+(defn test-quince-with/a []
+  (run-coroutine
+    (fn/a []
+      (with/a [t1 (AsyncWithTest 1)
+               t2 (AsyncWithTest 2)
+               t3 (AsyncWithTest 3)
+               _ (AsyncWithTest 4)]
+        (assert (= t1 1))
+        (assert (= t2 2))
+        (assert (= t3 3))))))

tests/native_tests/mangling.hy

@@ -1,11 +1,8 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
 
-(import [hy._compat [PY3]])
-
-
 (defn test-hyphen []
   (setv a-b 1)
   (assert (= a-b 1))
@@ -63,9 +60,7 @@
 (defn test-higher-unicode []
   (setv 😂 "emoji")
   (assert (= 😂 "emoji"))
-  (if PY3
-    (assert (= hyx_Xface_with_tears_of_joyX "emoji"))
-    (assert (= hyx_XU1f602X "emoji"))))
+  (assert (= hyx_Xface_with_tears_of_joyX "emoji")))
 
 
 (defn test-nameless-unicode []

tests/native_tests/mathematics.hy

@@ -1,210 +0,0 @@
-;; Copyright 2019 the authors.
-;; This file is part of Hy, which is free software licensed under the Expat
-;; license. See the LICENSE.
-
-(import [hy._compat [PY3]])
-
-(setv square (fn [x]
-               (* x x)))
-
-
-(setv test_basic_math (fn []
-                        "NATIVE: Test basic math."
-                        (assert (= (+ 2 2) 4))))
-
-(setv test_mult (fn []
-                  "NATIVE: Test multiplication."
-                  (assert (= 4 (square 2)))
-                  (assert (= 8 (* 8)))
-                  (assert (= 1 (*)))))
-
-
-(setv test_sub (fn []
-                 "NATIVE: Test subtraction"
-                 (assert (= 4 (- 8 4)))
-                 (assert (= -8 (- 8)))))
-
-
-(setv test_add (fn []
-                 "NATIVE: Test addition"
-                 (assert (= 4 (+ 1 1 1 1)))
-                 (assert (= 8 (+ 8)))
-                 (assert (= 0 (+)))))
-
-
-(defn test-add-unary []
-  "NATIVE: test that unary + calls __pos__"
-
-  (defclass X [object]
-    [__pos__ (fn [self] "called __pos__")])
-  (assert (= (+ (X)) "called __pos__"))
-
-  ; Make sure the shadowed version works, too.
-  (setv f +)
-  (assert (= (f (X)) "called __pos__")))
-
-
-(setv test_div (fn []
-                 "NATIVE: Test division"
-                 (assert (= 25 (/ 100 2 2)))
-                                ; Commented out until float constants get implemented
-                                ; (assert (= 0.5 (/ 1 2)))
-                 (assert (= 1 (* 2 (/ 1 2))))))
-
-(setv test_int_div (fn []
-                     "NATIVE: Test integer division"
-                     (assert (= 25 (// 101 2 2)))))
-
-(defn test-modulo []
-  "NATIVE: test mod"
-  (assert (= (% 10 2) 0)))
-
-(defn test-pow []
-  "NATIVE: test pow"
-  (assert (= (** 10 2) 100)))
-
-(defn test-lshift []
-  "NATIVE: test lshift"
-  (assert (= (<< 1 2) 4)))
-
-(defn test-rshift []
-  "NATIVE: test lshift"
-  (assert (= (>> 8 1) 4)))
-
-(defn test-bitor []
-  "NATIVE: test lshift"
-  (assert (= (| 1 2) 3)))
-
-(defn test-bitxor []
-  "NATIVE: test xor"
-  (assert (= (^ 1 2) 3)))
-
-(defn test-bitand []
-  "NATIVE: test lshift"
-  (assert (= (& 1 2) 0)))
-
-(defn test-augassign-add []
-  "NATIVE: test augassign add"
-  (setv x 1)
-  (+= x 41)
-  (assert (= x 42)))
-
-(defn test-augassign-sub []
-  "NATIVE: test augassign sub"
-  (setv x 1)
-  (-= x 41)
-  (assert (= x -40)))
-
-(defn test-augassign-mult []
-  "NATIVE: test augassign mult"
-  (setv x 1)
-  (*= x 41)
-  (assert (= x 41)))
-
-(defn test-augassign-div []
-  "NATIVE: test augassign div"
-  (setv x 42)
-  (/= x 2)
-  (assert (= x 21)))
-
-(defn test-augassign-floordiv []
-  "NATIVE: test augassign floordiv"
-  (setv x 42)
-  (//= x 2)
-  (assert (= x 21)))
-
-(defn test-augassign-mod []
-  "NATIVE: test augassign mod"
-  (setv x 42)
-  (%= x 2)
-  (assert (= x 0)))
-
-(defn test-augassign-pow []
-  "NATIVE: test augassign pow"
-  (setv x 2)
-  (**= x 3)
-  (assert (= x 8)))
-
-(defn test-augassign-lshift []
-  "NATIVE: test augassign lshift"
-  (setv x 2)
-  (<<= x 2)
-  (assert (= x 8)))
-
-(defn test-augassign-rshift []
-  "NATIVE: test augassign rshift"
-  (setv x 8)
-  (>>= x 1)
-  (assert (= x 4)))
-
-(defn test-augassign-bitand []
-  "NATIVE: test augassign bitand"
-  (setv x 8)
-  (&= x 1)
-  (assert (= x 0)))
-
-(defn test-augassign-bitor []
-  "NATIVE: test augassign bitand"
-  (setv x 0)
-  (|= x 2)
-  (assert (= x 2)))
-
-(defn test-augassign-bitxor []
-  "NATIVE: test augassign bitand"
-  (setv x 1)
-  (^= x 1)
-  (assert (= x 0)))
-
-(defn overflow-int-to-long []
-  "NATIVE: test if int does not raise an overflow exception"
-  (assert (integer? (+ 1 1000000000000000000000000))))
-
-
-(defclass HyTestMatrix [list]
-  [--matmul--
-   (fn [self other]
-     (setv n (len self)
-           m (len (. other [0]))
-           result [])
-     (for [i (range m)]
-       (setv result-row [])
-       (for [j (range n)]
-         (setv dot-product 0)
-         (for [k (range (len (. self [0])))]
-           (+= dot-product (* (. self [i] [k])
-                              (. other [k] [j]))))
-         (.append result-row dot-product))
-       (.append result result-row))
-     result)])
-
-(setv first-test-matrix (HyTestMatrix [[1 2 3]
-                                       [4 5 6]
-                                       [7 8 9]]))
-
-(setv second-test-matrix (HyTestMatrix [[2 0 0]
-                                        [0 2 0]
-                                        [0 0 2]]))
-
-(setv product-of-test-matrices (HyTestMatrix [[ 2  4  6]
-                                              [ 8 10 12]
-                                              [14 16 18]]))
-
-(defn test-matmul []
-  "NATIVE: test matrix multiplication"
-  (if PY3
-    (assert (= (@ first-test-matrix second-test-matrix)
-               product-of-test-matrices))
-    ;; Python <= 3.4
-    (do
-      (setv matmul-attempt (try (@ first-test-matrix second-test-matrix)
-                                (except [e [Exception]] e)))
-      (assert (isinstance matmul-attempt NameError)))))
-
-(defn test-augassign-matmul []
-  "NATIVE: test augmented-assignment matrix multiplication"
-  (setv matrix first-test-matrix
-        matmul-attempt (try (@= matrix second-test-matrix)
-                              (except [e [Exception]] e)))
-  (if PY3
-    (assert (= product-of-test-matrices matrix))
-    (assert (isinstance matmul-attempt NameError))))

tests/native_tests/model_patterns.hy

@@ -1,4 +1,4 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 

tests/native_tests/native_macros.hy

@@ -1,4 +1,4 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
@@ -140,11 +140,6 @@
 (assert initialized)
 (assert (test-initialized))
 
-(defn test-if-python2 []
-  (import sys)
-  (assert (= (get sys.version_info 0)
-             (if-python2 2 3))))
-
 (defn test-gensym-in-macros []
   (import ast)
   (import [astor.code-gen [to-source]])
@@ -391,7 +386,7 @@ in expansions."
   ;; Now, let's use a `require`d macro that depends on another macro defined only
   ;; in this scope.
   (defmacro local-test-macro [x]
-    (.format "This is the local version of `nonlocal-test-macro` returning {}!" x))
+    (.format "This is the local version of `nonlocal-test-macro` returning {}!" (int x)))
 
   (assert (= "This is the local version of `nonlocal-test-macro` returning 3!"
              (test-module-macro-2 3)))
@@ -409,9 +404,9 @@ in expansions."
  Additionally, we confirm that `require` statements are executed via loaded bytecode."
 
   (import os sys marshal types)
-  (import [hy.importer [cache-from-source]])
+  (import importlib)
 
-  (setv pyc-file (cache-from-source
+  (setv pyc-file (importlib.util.cache-from-source
                    (os.path.realpath
                      (os.path.join
                        "tests" "resources" "macro_with_require.hy"))))

tests/native_tests/operators.hy

@@ -1,9 +1,7 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
-(import pytest [hy._compat [PY3]])
-
 (defmacro op-and-shadow-test [op &rest body]
   ; Creates two tests with the given `body`, one where all occurrences
   ; of the symbol `f` are syntactically replaced with `op` (a test of
@@ -36,7 +34,7 @@
 
   (assert (= (f) 0))
 
-  (defclass C [object] [__pos__ (fn [self] "called __pos__")])
+  (defclass C [object] (defn __pos__ [self] "called __pos__"))
   (assert (= (f (C)) "called __pos__"))
 
   (assert (= (f 1 2) 3))
@@ -102,14 +100,14 @@
   (forbid (f 1 2 3)))
 
 
-(when PY3 (op-and-shadow-test @
-  (defclass C [object] [
-    __init__ (fn [self content] (setv self.content content))
-    __matmul__ (fn [self other] (C (+ self.content other.content)))])
+(op-and-shadow-test @
+  (defclass C [object]
+    (defn __init__ [self content] (setv self.content content))
+    (defn __matmul__ [self other] (C (+ self.content other.content))))
   (forbid (f))
   (assert (do (setv c (C "a")) (is (f c) c)))
   (assert (= (. (f (C "b") (C "c")) content) "bc"))
-  (assert (= (. (f (C "d") (C "e") (C "f")) content) "def"))))
+  (assert (= (. (f (C "d") (C "e") (C "f")) content) "def")))
 
 
 (op-and-shadow-test <<
@@ -173,11 +171,11 @@
 
   ; Make sure chained comparisons use `and`, not `&`.
   ; https://github.com/hylang/hy/issues/1191
-  (defclass C [object] [
-    __init__ (fn [self x]
+  (defclass C [object]
+    (defn __init__ [self x]
       (setv self.x x))
-    __lt__ (fn [self other]
-      self.x)])
+    (defn __lt__ [self other]
+      self.x))
   (assert (= (f (C "a") (C "b") (C "c")) "b")))
 
 
@@ -296,7 +294,8 @@
   (forbid (f 3))
   (assert (is (f 3 [1 2]) (!= f-name "in")))
   (assert (is (f 2 [1 2]) (= f-name "in")))
-  (forbid (f 2 [1 2] [3 4])))
+  (assert (is (f 2 [1 2] [[1 2] 3]) (= f-name "in")))
+  (assert (is (f 3 [1 2] [[2 2] 3]) (!= f-name "in"))))
 
 
 (op-and-shadow-test [get]
@@ -305,3 +304,58 @@
   (assert (= (f "hello" 1) "e"))
   (assert (= (f [[1 2 3] [4 5 6] [7 8 9]] 1 2) 6))
   (assert (= (f {"x" {"y" {"z" 12}}} "x" "y" "z") 12)))
+
+
+(defn test-chained-comparison []
+  (assert (cmp 2 = (+ 1 1) = (- 3 1)))
+  (assert (not (cmp 2 = (+ 1 1) = (+ 3 1))))
+
+  (assert (cmp 2 = 2 > 1))
+  (assert (cmp 2 = (+ 1 1) > 1))
+  (setv x 2)
+  (assert (cmp 2 = x > 1))
+  (assert (cmp 2 = x > (> 4 3)))
+  (assert (not (cmp (> 4 3) = x > 1)))
+
+  (assert (cmp 1 in [1] in [[1] [2 3]] not-in [5]))
+  (assert (not (cmp 1 in [1] not-in [[1] [2 3]] not-in [5]))))
+
+
+(defn test-augassign []
+  (setv b 2  c 3  d 4)
+  (defmacro same-as [expr1 expr2 expected-value]
+    `(do
+      (setv a 4)
+      ~expr1
+      (setv expr1-value a)
+      (setv a 4)
+      ~expr2
+      (assert (= expr1-value a ~expected-value))))
+  (same-as (+= a b c d) (+= a (+ b c d)) 13)
+  (same-as (-= a b c d) (-= a (+ b c d)) -5)
+  (same-as (*= a b c d) (*= a (* b c d)) 96)
+  (same-as (**= a b c) (**= a (** b c)) 65,536)
+  (same-as (/= a b c d) (/= a (* b c d)) (/ 1 6))
+  (same-as (//= a b c d) (//= a (* b c d)) 0)
+  (same-as (<<= a b c d) (<<= a (+ b c d)) 0b10_00000_00000)
+  (same-as (>>= a b c d) (>>= a (+ b c d)) 0)
+  (same-as (&= a b c d) (&= a (& b c d)) 0)
+  (same-as (|= a b c d) (|= a (| b c d)) 0b111)
+
+  (defclass C [object]
+    (defn __init__ [self content] (setv self.content content))
+    (defn __matmul__ [self other] (C (+ self.content other.content))))
+  (setv  a (C "a")  b (C "b")  c (C "c")  d (C "d"))
+  (@= a b c d)
+  (assert (= a.content "abcd"))
+  (setv a (C "a"))
+  (@= a (@ b c d))
+  (assert (= a.content "abcd"))
+
+  (setv a 15)
+  (%= a 9)
+  (assert (= a 6))
+
+  (setv a 0b1100)
+  (^= a 0b1010)
+  (assert (= a 0b0110)))

tests/native_tests/py36_only_tests.hy

@@ -1,4 +1,4 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
@@ -6,6 +6,7 @@
 ;; conftest.py skips this file when running on Python <3.6.
 
 (import [asyncio [get-event-loop sleep]])
+(import [typing [get-type-hints List Dict]])
 
 
 (defn run-coroutine [coro]
@@ -38,10 +39,24 @@
         (else (setv x (+ x 50))))
       (assert (= x 53)))))
 
+(defn test-variable-annotations []
+  (defclass AnnotationContainer []
+    (setv ^int x 1 y 2)
+    (^bool z))
+
+  (setv annotations (get-type-hints AnnotationContainer))
+  (assert (= (get annotations "x") int))
+  (assert (= (get annotations "z") bool)))
+
+(defn test-of []
+  (assert (= (of str) str))
+  (assert (= (of List int) (get List int)))
+  (assert (= (of Dict str str) (get Dict (, str str)))))
+
 (defn test-pep-487 []
   (defclass QuestBase []
-    [--init-subclass-- (fn [cls swallow &kwargs kwargs]
-                         (setv cls.swallow swallow))])
+    (defn --init-subclass-- [cls swallow &kwargs kwargs]
+      (setv cls.swallow swallow)))
 
   (defclass Quest [QuestBase :swallow "african"])
   (assert (= (. (Quest) swallow) "african")))

tests/native_tests/py38_only_tests.hy

@@ -1,4 +1,4 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 

tests/native_tests/py3_only_tests.hy

@@ -1,207 +0,0 @@
-;; Copyright 2019 the authors.
-;; This file is part of Hy, which is free software licensed under the Expat
-;; license. See the LICENSE.
-
-;; Tests where the emitted code relies on Python 3.
-;; conftest.py skips this file when running on Python 2.
-
-
-(defn test-exception-cause []
-  (try (raise ValueError :from NameError)
-       (except [e [ValueError]]
-         (assert (= (type (. e __cause__)) NameError)))))
-
-
-(defn test-kwonly []
-  "NATIVE: test keyword-only arguments"
-  ;; keyword-only with default works
-  (defn kwonly-foo-default-false [&kwonly [foo False]] foo)
-  (assert (= (kwonly-foo-default-false) False))
-  (assert (= (kwonly-foo-default-false :foo True) True))
-  ;; keyword-only without default ...
-  (defn kwonly-foo-no-default [&kwonly foo] foo)
-  (setv attempt-to-omit-default (try
-                                  (kwonly-foo-no-default)
-                                  (except [e [Exception]] e)))
-  ;; works
-  (assert (= (kwonly-foo-no-default :foo "quux") "quux"))
-  ;; raises TypeError with appropriate message if not supplied
-  (assert (isinstance attempt-to-omit-default TypeError))
-  (assert (in "missing 1 required keyword-only argument: 'foo'"
-              (. attempt-to-omit-default args [0])))
-  ;; keyword-only with other arg types works
-  (defn function-of-various-args [a b &rest args &kwonly foo &kwargs kwargs]
-    (, a b args foo kwargs))
-  (assert (= (function-of-various-args 1 2 3 4 :foo 5 :bar 6 :quux 7)
-             (, 1 2 (, 3 4)  5 {"bar" 6 "quux" 7}))))
-
-
-(defn test-extended-unpacking-1star-lvalues []
-  (setv [x #*y] [1 2 3 4])
-  (assert (= x 1))
-  (assert (= y [2 3 4]))
-  (setv [a #*b c] "ghijklmno")
-  (assert (= a "g"))
-  (assert (= b (list "hijklmn")))
-  (assert (= c "o")))
-
-
-(defn test-yield-from []
-  "NATIVE: testing yield from"
-  (defn yield-from-test []
-    (for [i (range 3)]
-      (yield i))
-    (yield-from [1 2 3]))
-  (assert (= (list (yield-from-test)) [0 1 2 1 2 3])))
-
-
-(defn test-yield-from-exception-handling []
-  "NATIVE: Ensure exception handling in yield from works right"
-  (defn yield-from-subgenerator-test []
-    (yield 1)
-    (yield 2)
-    (yield 3)
-    (assert 0))
-  (defn yield-from-test []
-    (for [i (range 3)]
-      (yield i))
-    (try
-      (yield-from (yield-from-subgenerator-test))
-      (except [e AssertionError]
-        (yield 4))))
-  (assert (= (list (yield-from-test)) [0 1 2 1 2 3 4])))
-
-(require [hy.contrib.walk [let]])
-
-(defn test-let-optional []
-  (let [a 1
-        b 6
-        d 2]
-       (defn foo [&kwonly [a a] b [c d]]
-         (, a b c))
-       (assert (= (foo :b "b")
-                  (, 1 "b" 2)))
-       (assert (= (foo :b 20 :a 10 :c 30)
-                  (, 10 20 30)))))
-
-(defn test-pep-3115 []
-  (defclass member-table [dict]
-    [--init-- (fn [self] (setv self.member-names []))
-
-     --setitem-- (fn [self key value]
-                   (if (not-in key self)
-                       (.append self.member-names key))
-                   (dict.--setitem-- self key value))])
-
-  (defclass OrderedClass [type]
-    [--prepare-- (classmethod (fn [metacls name bases] (member-table)))
-
-     --new-- (fn [cls name bases classdict]
-               (setv result (type.--new-- cls name bases (dict classdict)))
-               (setv result.member-names classdict.member-names)
-               result)])
-
-  (defclass MyClass [:metaclass OrderedClass]
-    [method1 (fn [self] (pass))
-     method2 (fn [self] (pass))])
-
-  (assert (= (. (MyClass) member-names)
-             ["__module__" "__qualname__" "method1" "method2"])))
-
-
-(import [asyncio [get-event-loop sleep]])
-
-
-(defn test-unpacking-pep448-1star []
-  (setv l [1 2 3])
-  (setv p [4 5])
-  (assert (= ["a" #*l "b" #*p #*l] ["a" 1 2 3 "b" 4 5 1 2 3]))
-  (assert (= (, "a" #*l "b" #*p #*l) (, "a" 1 2 3 "b" 4 5 1 2 3)))
-  (assert (= #{"a" #*l "b" #*p #*l} #{"a" "b" 1 2 3 4 5}))
-  (defn f [&rest args] args)
-  (assert (= (f "a" #*l "b" #*p #*l) (, "a" 1 2 3 "b" 4 5 1 2 3)))
-  (assert (= (+ #*l #*p) 15))
-  (assert (= (and #*l) 3)))
-
-
-(defn test-unpacking-pep448-2star []
-  (setv d1 {"a" 1 "b" 2})
-  (setv d2 {"c" 3 "d" 4})
-  (assert (= {1 "x" #**d1 #**d2 2 "y"} {"a" 1 "b" 2 "c" 3 "d" 4 1 "x" 2 "y"}))
-  (defn fun [&optional a b c d e f] [a b c d e f])
-  (assert (= (fun #**d1 :e "eee" #**d2) [1 2 3 4 "eee" None])))
-
-
-(defn run-coroutine [coro]
-  "Run a coroutine until its done in the default event loop."""
-  (.run_until_complete (get-event-loop) (coro)))
-
-
-(defn test-fn/a []
-  (assert (= (run-coroutine (fn/a [] (await (sleep 0)) [1 2 3]))
-             [1 2 3])))
-
-
-(defn test-defn/a []
-  (defn/a coro-test []
-    (await (sleep 0))
-    [1 2 3])
-  (assert (= (run-coroutine coro-test) [1 2 3])))
-
-
-(defn test-decorated-defn/a []
-  (defn decorator [func] (fn/a [] (/ (await (func)) 2)))
-
-  #@(decorator
-      (defn/a coro-test []
-        (await (sleep 0))
-        42))
-  (assert (= (run-coroutine coro-test) 21)))
-
-
-(defclass AsyncWithTest []
-  (defn --init-- [self val]
-    (setv self.val val)
-    None)
-
-  (defn/a --aenter-- [self]
-    self.val)
-
-  (defn/a --aexit-- [self tyle value traceback]
-    (setv self.val None)))
-
-
-(defn test-single-with/a []
-  (run-coroutine
-    (fn/a []
-      (with/a [t (AsyncWithTest 1)]
-        (assert (= t 1))))))
-
-(defn test-two-with/a []
-  (run-coroutine
-    (fn/a []
-      (with/a [t1 (AsyncWithTest 1)
-               t2 (AsyncWithTest 2)]
-        (assert (= t1 1))
-        (assert (= t2 2))))))
-
-(defn test-thrice-with/a []
-  (run-coroutine
-    (fn/a []
-      (with/a [t1 (AsyncWithTest 1)
-               t2 (AsyncWithTest 2)
-               t3 (AsyncWithTest 3)]
-        (assert (= t1 1))
-        (assert (= t2 2))
-        (assert (= t3 3))))))
-
-(defn test-quince-with/a []
-  (run-coroutine
-    (fn/a []
-      (with/a [t1 (AsyncWithTest 1)
-               t2 (AsyncWithTest 2)
-               t3 (AsyncWithTest 3)
-               _ (AsyncWithTest 4)]
-        (assert (= t1 1))
-        (assert (= t2 2))
-        (assert (= t3 3))))))

tests/native_tests/quote.hy

@@ -1,4 +1,4 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
@@ -9,7 +9,7 @@
   "NATIVE: test for quoting functionality"
   (setv q (quote (a b c)))
   (assert (= (len q) 3))
-  (assert (= q [(quote a) (quote b) (quote c)])))
+  (assert (= q (HyExpression [(quote a) (quote b) (quote c)]))))
 
 
 (defn test-basic-quoting []

tests/native_tests/tag_macros.hy

@@ -1,4 +1,4 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 

tests/native_tests/with_decorator.hy

@@ -1,4 +1,4 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
@@ -27,7 +27,7 @@
     cls)
   (with-decorator bardec
     (defclass cls []
-      [attr1 123]))
+      (setv attr1 123)))
   (assert (= cls.attr1 123))
   (assert (= cls.attr2 456)))
 

tests/native_tests/with_test.hy

@@ -1,4 +1,4 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 

tests/resources/argparse_ex.hy

@@ -1,5 +1,5 @@
 #!/usr/bin/env hy
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 

tests/resources/hello_world.hy

@@ -0,0 +1 @@
+(print "hello world")

tests/resources/pydemo.hy

@@ -1,4 +1,4 @@
-;; Copyright 2019 the authors.
+;; Copyright 2020 the authors.
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
@@ -49,6 +49,10 @@ Call me Ishmael. Some years ago—never mind how long precisely—having little
 (setv condexpr (if "" "x" "y"))
 (setv mylambda (fn [x] (+ x "z")))
 
+(setv fstring1 f"hello {(+ 1 1)} world")
+(setv p "xyzzy")
+(setv fstring2 f"a{p !r :9}")
+
 (setv augassign 103)
 (//= augassign 4)
 
@@ -142,13 +146,24 @@ Call me Ishmael. Some years ago—never mind how long precisely—having little
 
 (defclass C2 [C1]
   "class docstring"
-  [attr1 5  attr2 6]
-  (setv attr3 7))
+  (setv attr1 5)
+  (setv attr2 6))
 
 (import [contextlib [closing]])
 (setv closed [])
 (defclass Closeable []
-  [close (fn [self] (.append closed self.x))])
+  (defn close [self] (.append closed self.x)))
 (with [c1 (closing (Closeable)) c2 (closing (Closeable))]
   (setv c1.x "v1")
   (setv c2.x "v2"))
+(setv closed1 (.copy closed))
+
+(pys "
+  closed = []
+  pys_accum = []
+  for i in range(5):
+      with closing(Closeable()) as o:
+          class C: pass
+          o.x = C()
+      pys_accum.append(i)")
+(setv py-accum (py "''.join(map(str, pys_accum))"))

tests/test_bin.py

@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 # -*- encoding: utf-8 -*-
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 
@@ -8,14 +8,12 @@ import os
 import re
 import shlex
 import subprocess
+import builtins
 
-from hy.importer import cache_from_source
-from hy._compat import PY3
+from importlib.util import cache_from_source
 
 import pytest
 
-from hy._compat import builtins
-
 
 hy_dir = os.environ.get('HY_DIR', '')
 
@@ -497,9 +495,8 @@ def test_bin_hy_tracebacks():
     os.environ['HY_DEBUG'] = ''
 
     def req_err(x):
-        assert x == '{}HyRequireError: No module named {}'.format(
-            'hy.errors.' if PY3 else '',
-            (repr if PY3 else str)('not_a_real_module'))
+        assert (x == 'hy.errors.HyRequireError: No module named '
+            "'not_a_real_module'")
 
     # Modeled after
     #   > python -c 'import not_a_real_module'
@@ -512,7 +509,7 @@ def test_bin_hy_tracebacks():
         del error_lines[-1]
     assert len(error_lines) <= 10
       # Rough check for the internal traceback filtering
-    req_err(error_lines[4 if PY3 else -1])
+    req_err(error_lines[4])
 
     _, error = run_cmd('hy -c "(require not-a-real-module)"', expect=1)
     error_lines = error.splitlines()
@@ -522,7 +519,7 @@ def test_bin_hy_tracebacks():
     output, error = run_cmd('hy -i "(require not-a-real-module)"')
     assert output.startswith('=> ')
     print(error.splitlines())
-    req_err(error.splitlines()[2 if PY3 else -3])
+    req_err(error.splitlines()[2])
 
     # Modeled after
     #   > python -c 'print("hi'
@@ -535,9 +532,8 @@ def test_bin_hy_tracebacks():
         r'Traceback \(most recent call last\):\n'
         r'  File "(?:<string>|string-[0-9a-f]+)", line 1\n'
         r'    \(print "\n'
-        r'           \^\n' +
-        r'{}PrematureEndOfInput: Partial string literal\n'.format(
-            r'hy\.lex\.exceptions\.' if PY3 else ''))
+        r'           \^\n'
+        r'hy.lex.exceptions.PrematureEndOfInput: Partial string literal\n')
     assert re.search(peoi_re, error)
 
     # Modeled after

tests/test_hy2py.py

@@ -1,11 +1,12 @@
 # -*- encoding: utf-8 -*-
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 
 import math, itertools
 from hy import mangle
 from hy._compat import PY36
+import hy.importer
 
 
 def test_direct_import():
@@ -19,7 +20,8 @@ def test_hy2py_import(tmpdir):
         ["hy2py", "tests/resources/pydemo.hy"]).decode("UTF-8")
     path = tmpdir.join("pydemo.py")
     path.write(python_code)
-    assert_stuff(import_from_path("pydemo", path))
+    # Note: explicit "str" is needed for 3.5.
+    assert_stuff(hy.importer._import_from_path("pydemo", str(path)))
 
 
 def assert_stuff(m):
@@ -77,6 +79,9 @@ def assert_stuff(m):
     assert type(m.mylambda) is type(lambda x: x + "z")
     assert m.mylambda("a") == "az"
 
+    assert m.fstring1 == "hello 2 world"
+    assert m.fstring2 == "a'xyzzy'  "
+
     assert m.augassign == 25
 
     assert m.delstatement == ["a", "c", "d", "e"]
@@ -113,18 +118,12 @@ def assert_stuff(m):
 
     assert m.C2.__doc__ == "class docstring"
     assert issubclass(m.C2, m.C1)
-    assert (m.C2.attr1, m.C2.attr2, m.C2.attr3) == (5, 6, 7)
+    assert (m.C2.attr1, m.C2.attr2) == (5, 6)
 
-    assert m.closed == ["v2", "v1"]
+    assert m.closed1 == ["v2", "v1"]
 
-
-def import_from_path(name, path):
-    if PY36:
-        import importlib.util
-        spec = importlib.util.spec_from_file_location(name, path)
-        m = importlib.util.module_from_spec(spec)
-        spec.loader.exec_module(m)
-    else:
-        import imp
-        m = imp.load_source(name, str(path))
-    return m
+    assert len(m.closed) == 5
+    for a, b in itertools.combinations(m.closed, 2):
+        assert type(a) is not type(b)
+    assert m.pys_accum == [0, 1, 2, 3, 4]
+    assert m.py_accum == "01234"

tests/test_lex.py

@@ -1,4 +1,4 @@
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 import sys

tests/test_models.py

@@ -1,18 +1,18 @@
-# Copyright 2019 the authors.
+# Copyright 2020 the authors.
 # This file is part of Hy, which is free software licensed under the Expat
 # license. See the LICENSE.
 
 import copy
 import hy
-from clint.textui.colored import clean
-from hy._compat import long_type, str_type
 from hy.models import (wrap_value, replace_hy_obj, HyString, HyInteger, HyList,
                        HyDict, HySet, HyExpression, HyComplex, HyFloat, pretty)
 
+hy.models.COLORED = False
 
-def test_wrap_long_type():
+
+def test_wrap_int():
     """ Test conversion of integers."""
-    wrapped = wrap_value(long_type(0))
+    wrapped = wrap_value(0)
     assert type(wrapped) == HyInteger
 
 
@@ -26,27 +26,27 @@ def test_wrap_tuple():
 
 def test_wrap_nested_expr():
     """ Test conversion of HyExpressions with embedded non-HyObjects."""
-    wrapped = wrap_value(HyExpression([long_type(0)]))
+    wrapped = wrap_value(HyExpression([0]))
     assert type(wrapped) == HyExpression
     assert type(wrapped[0]) == HyInteger
     assert wrapped == HyExpression([HyInteger(0)])
 
 
-def test_replace_long_type():
+def test_replace_int():
     """ Test replacing integers."""
-    replaced = replace_hy_obj(long_type(0), HyInteger(13))
+    replaced = replace_hy_obj(0, HyInteger(13))
     assert replaced == HyInteger(0)
 
 
 def test_replace_string_type():
     """Test replacing python string"""
-    replaced = replace_hy_obj(str_type("foo"), HyString("bar"))
+    replaced = replace_hy_obj("foo", HyString("bar"))
     assert replaced == HyString("foo")
 
 
 def test_replace_tuple():
     """ Test replacing tuples."""
-    replaced = replace_hy_obj((long_type(0), ), HyInteger(13))
+    replaced = replace_hy_obj((0, ), HyInteger(13))
     assert type(replaced) == HyList
     assert type(replaced[0]) == HyInteger
     assert replaced == HyList([HyInteger(0)])
@@ -57,8 +57,8 @@ def test_list_add():
     a = HyList([1, 2, 3])
     b = HyList([3, 4, 5])
     c = a + b
-    assert c == [1, 2, 3, 3, 4, 5]
-    assert c.__class__ == HyList
+    assert c == HyList([1, 2, 3, 3, 4, 5])
+    assert type(c) is HyList
 
 
 def test_list_slice():
@@ -92,7 +92,7 @@ hyset = HySet([3, 1, 2, 2])
 
 
 def test_set():
-    assert hyset == [3, 1, 2, 2]
+    assert list(hyset) == [3, 1, 2, 2]
 
 
 def test_number_model_copy():
@@ -183,13 +183,13 @@ def test_compound_model_repr():
             assert eval(repr(model([1, 2, 3]))) == model([1, 2, 3])
         for k, v in PRETTY_STRINGS.items():
             # `str` should be pretty, even under `pretty(False)`.
-            assert clean(str(hy.read_str(k))) == v
+            assert str(hy.read_str(k)) == v
         for k in PRETTY_STRINGS.keys():
             assert eval(repr(hy.read_str(k))) == hy.read_str(k)
     with pretty(True):
         for model in HY_LIST_MODELS:
-            assert eval(clean(repr(model()))).__class__ is model
-            assert eval(clean(repr(model([1, 2])))) == model([1, 2])
-            assert eval(clean(repr(model([1, 2, 3])))) == model([1, 2, 3])
+            assert eval(repr(model())).__class__ is model
+            assert eval(repr(model([1, 2]))) == model([1, 2])
+            assert eval(repr(model([1, 2, 3]))) == model([1, 2, 3])
         for k, v in PRETTY_STRINGS.items():
-            assert clean(repr(hy.read_str(k))) == v
+            assert repr(hy.read_str(k)) == v