Merge branch 'master' into pr/584

.mailmap

@@ -0,0 +1,12 @@
+Paul R. Tagliamonte <paultag@debian.org> Paul Tagliamonte <paultag@debian.org>
+Paul R. Tagliamonte <paultag@debian.org> Paul Tagliamonte <tag@pault.ag>
+Paul R. Tagliamonte <paultag@debian.org> Paul Tagliamonte <paultag@sunlightfoundation.com>
+Paul R. Tagliamonte <paultag@debian.org> Paul Tagliamonte <paultag@ubuntu.com>
+Morten Linderud <mcfoxax@gmail.com> Foxboron <mcfoxax@gmail.com>
+Morten Linderud <mcfoxax@gmail.com> <fox@velox.pw>
+James King <james@agentultra.com> agentultra <james@agentultra.com>
+James King <james@agentultra.com> J Kenneth King <james@agentultra.com>
+Abhishek L <abhishek.lekshmanan@gmail.com> <abhishekl.2006@gmail.com>
+Bob Tolbert <bob@tolbert.org> Bob Tolbert <bob@eyesopen.com>
+Guillermo Vaya <guivaya@gmail.com> Guillermo Vaya <guillermo.vaya@gigas.com>
+Gergely Nagy <algernon@balabit.hu> Gergely Nagy <algernon@madhouse-project.org>

.travis.yml

@@ -6,10 +6,12 @@ python:
   - "3.2"
   - "3.3"
   - "3.4"
+cache:
+  - $HOME/.pip-cache
 # command to install dependencies
 install:
-  - pip install -r requirements-dev.txt
+  - pip install -r requirements-travis.txt --download-cache $HOME/.pip-cache
-  - pip install coveralls
+  - pip install coveralls --download-cache $HOME/.pip-cache
   - if [[ $TRAVIS_PYTHON_VERSION == '2.6' ]]; then pip install unittest2; fi  # needs for running tests
   - pip install --allow-all-external -e .
 # command to run tests

AUTHORS

@@ -46,3 +46,4 @@
 * Brendan Curran-Johnson <brendan@bcjbcj.ca>
 * Ivan Kozik <ivan@ludios.org>
 * Allison Kaptur <allison.kaptur@gmail.com>
+* Matthew Wampler-Doty <matthew.wampler.doty@gmail.com>

CONTRIBUTING.rst

@@ -0,0 +1,44 @@
+Contributions are welcome & greatly appreciated, every little bit
+helps in making Hy more awesome.
+
+Pull requests are great! We love them, here is a quick guide:
+
+- Fork the repo, create a topic branch for a feature/fix. Avoid
+  making changes directly on the master branch
+
+- All incoming features should be accompanied with tests
+
+- Before you submit a PR, please run the tests and check your code
+  against the style guide.  You can do both these things at once::
+
+    $ make d
+
+- Make commits into logical units, so that it is easier to track &
+  navigate later. Before submitting a PR, try squashing the commits
+  into changesets that are easy to come back to later. Also make sure
+  you don't leave spurious whitespace in the changesets, this avoids
+  creation of whitespace fix commits later.
+
+- As far as commit messages go, try to adhere to
+  the following:
+
+  + Try sticking to the 50 character limit for the first line of git
+    commit messages
+
+  + For more explanations etc. follow this up with a blank line and
+    continue describing the commit in detail
+
+
+- Finally add yourself to the AUTHORS file (as a separate commit), you
+  deserve it :)
+
+- All incoming changes need to be acked by 2 different members of
+  Hylang's core team. Additional review is clearly welcome, but we need
+  a minimum of 2 signoffs for any change.
+
+- If a core member is sending in a PR, please find 2 core members that doesn't
+  include the PR submitter. The idea here is that one can work with the PR
+  author, and a second acks the entire change set.
+
+- For documentation & other trivial changes, we're good to merge after one
+  ACK. We've got low coverage, so it'd be great to keep that barrier low.

Dockerfile

@@ -0,0 +1,10 @@
+# Base image
+#
+# VERSION   0.2
+FROM        python:3
+MAINTAINER  Paul R. Tagliamonte <paultag@debian.org>
+
+ADD . /opt/hylang/hy
+RUN pip3 install -e /opt/hylang/hy
+
+CMD ["hy"]

NEWS

@@ -60,7 +60,7 @@ Changes from 0.9.12
   * "clean" target added to Makefile 
   * hy2py supports a bunch of commandline options to show AST, source etc.
   * Sub-object mangling: every identifier is split along the dots & mangled
-    seperately
+    separately
   
   [ Bug Fixes ]
   * Empty MacroExpansions work as expected
@@ -88,7 +88,7 @@ Changes from Hy 0.9.11
     what exactly was added. 
     
     The biggest feature, Reader Macros, landed later
-    in the cycle, but were big enough to warrent a release on it's
+    in the cycle, but were big enough to warrant a release on its
     own. A huge thanks goes to Foxboron for implementing them
     and a massive hug goes out to olasd for providing ongoing
     reviews during the development.
@@ -267,7 +267,7 @@ Changes from Hy 0.9.6
 
     * UTF-8 encoded hy symbols are now hy_... rather than __hy_..., it's
       silly to prefex them as such. (PT)
-    * `j' is no longer always intepreted as a complex number; we use it much
+    * `j' is no longer always interpreted as a complex number; we use it much
       more as a symbol. (ND)
     * (decorate-with) has been moved to (with-decorate) (JD)
     * New (unless) macro (JD)
@@ -286,7 +286,7 @@ Changes from Hy 0.9.6
       and jd for making this happen. This solves just an insane number
       of bugs. (ND, PT, JD)
     * Eval no longer sucks with statements (ND)
-    * New magic binary flags / mis fixes with the hy intepreter
+    * New magic binary flags / mis fixes with the hy interpreter
       (WKG + @eigenhombre)
 
 
@@ -311,7 +311,7 @@ Changes from Hy 0.9.5
     =============== WARNING: WARNING: READ ME: READ ME: ===================
     From here on out, we will only support "future division" as part of hy.
     This is actually quite a pain for us, but it's going to be quite an
-    amazing feautre.
+    amazing feature.
 
     This also normalizes behavior from Py 2 --> Py 3.
 
@@ -346,7 +346,7 @@ Changes from Hy 0.9.4
 
     * Statements in the `fn' path early will not return anymore. (PT)
     * Added "not" as the inline "not" operator. It's advised to still
-      use "not-in" or "is-not" rather then nesting. (JD)
+      use "not-in" or "is-not" rather than nesting. (JD)
     * `let' macro added (PT)
     * Added "~" as the "invert" operator. (JD)
     * `catch' now accepts a new format: (JD)

README.md

@@ -25,7 +25,7 @@ OK, so, why?
 ------------
 
 Well. Python is awesome. So awesome, that we have so many tools to alter the
-languge in a *core* way, but we never use them.
+language in a *core* way, but we never use them.
 
 Why?
 

docs/hacking.rst

@@ -89,26 +89,10 @@ To build the docs in HTML::
 Write docs---docs are good! Even this doc!
 
 
-Core Development Rules
+Contributing
-======================
+============
-
-Pull requests are good!
-
-Before you submit a PR, please run the tests and check your code against the style guide.  You can do both these things at once::
-
-    $ make d
-
-All incoming changes need to be acked by 2 different members of Hylang's
-core team. Additional review is clearly welcome, but we need a minimum of
-2 signoffs for any change.
-
-If a core member is sending in a PR, please find 2 core members that don't
-include the PR submitter. The idea here is that one can work with the PR
-author, and a second acks the entire change set.
-
-If the change is adding documentation, feel free to just merge after one
-ACK. We've got low coverage, so it'd be great to keep that barrier low.
 
+.. include:: ../CONTRIBUTING.rst
 
 Core Team
 =========

docs/language/api.rst

@@ -271,6 +271,21 @@ however is called only for every other value in the list.
         (side-effect2 x)))
 
 
+dict-comp
+---------
+
+`dict-comp` is used to create dictionaries. It takes three or four parameters.
+The first two parameters are for controlling the return value
+(key-value pair), while the third is used to select items from a sequence. The
+fourth and optional parameter can be used to filter out some of the items in
+the sequence based on a conditional expression.
+
+.. code-block:: hy
+
+    => (dict-comp x (* x 2) [x (range 10)] (odd? x))
+    {1: 2, 3: 6, 9: 18, 5: 10, 7: 14}
+
+
 do / progn
 ----------
 
@@ -290,7 +305,7 @@ Some example usage:
     Yeah, really!
 
     ;; assuming that (side-effect) is a function that we want to call for each
-    ;; and every value in the list, but which return values we do not care
+    ;; and every value in the list, but whose return value we do not care about
     => (list-comp (do (side-effect x) 
     ...               (if (< x 5) (* 2 x) 
     ...                   (* 4 x))) 
@@ -401,7 +416,7 @@ Parameters may have following keywords in front of them:
     arguments may be specified after this one.
 
     The following code example defines a function that can be given 0 to n
-    numerical parameters. It then sums every odd number and substracts
+    numerical parameters. It then sums every odd number and subtracts
     every even number.
 
     .. code-block:: clj
@@ -441,6 +456,44 @@ symbols for function names as the first parameter, `defn-alias` and
   => (alias)
   "Hello!"
 
+
+defmain
+-------
+
+.. versionadded:: 0.10.1
+
+The `defmain` macro defines a main function that is immediately called
+with sys.argv as arguments if and only if this file is being executed
+as a script.  In other words this:
+
+.. code-block:: clj
+
+   (defmain [&rest args]
+     (do-something-with args))
+
+is the equivalent of::
+
+   def main(*args):
+       do_something_with(args)
+       return 0
+
+   if __name__ == "__main__":
+       import sys
+       retval = main(*sys.arg)
+
+       if isinstance(retval, int):
+           sys.exit(retval)
+
+Note, as you can see above, if you return an integer from this
+function, this will be used as the exit status for your script.
+(Python defaults to exit status 0 otherwise, which means everything's
+okay!)
+
+(Since (sys.exit 0) is not run explicitly in case of a non-integer
+return from defmain, it's good to put (defmain) as the last bit of
+code in your file.)
+
+
 .. _defmacro:
 
 defmacro
@@ -560,6 +613,27 @@ del
   => dic
   {}
 
+doto
+----
+
+.. versionadded:: 0.10.1
+
+`doto` macro is used to make a sequence of method calls for an object easy.
+
+.. code-block:: clj
+
+  => (doto [] (.append 1) (.append 2) .reverse)
+  [2 1]
+
+.. code-block:: clj
+
+  => (setv collection [])
+  => (.append collection 1)
+  => (.append collection 2)
+  => (.reverse collection)
+  => collection
+  [2 1]
+
 eval
 ----
 
@@ -629,6 +703,24 @@ normally. If the execution is halted with `break`, the `else` does not execute.
     loop finished
 
 
+genexpr
+-------
+
+`genexpr` is used to create generator expressions. It takes two or three parameters.
+The first parameter is the expression controlling the return value, while
+the second is used to select items from a list. The third and optional
+parameter can be used to filter out some of the items in the list based on a 
+conditional expression. `genexpr` is similar to `list-comp`, except that it returns
+an iterable that evaluates values one by one instead of evaluating them immediately.
+
+.. code-block:: hy
+
+    => (def collection (range 10))
+    => (def filtered (genexpr x [x collection] (even? x)))
+    => (list filtered)
+    [0, 2, 4, 6, 8]
+
+
 .. _gensym:
 
 gensym
@@ -1003,6 +1095,22 @@ element:
     [1, 2, 3, 4, 5, 6, 7, 8, 9]
 
 
+set-comp
+--------
+
+`set-comp` is used to create sets. It takes two or three parameters.
+The first parameter is for controlling the return value, while the second is
+used to select items from a sequence. The third and optional parameter can be
+used to filter out some of the items in the sequence based on a conditional
+expression.
+
+.. code-block:: hy
+
+    => (setv data [1 2 3 4 5 2 3 4 5 3 4 5])
+    => (set-comp x [x data] (odd? x))
+    {1, 3, 5}
+
+
 slice
 -----
 
@@ -1057,7 +1165,7 @@ Example usage
     ; Throw an IOError("foobar")
 
 
-`throw` can acccept a single argument (an `Exception` class or instance), or
+`throw` can accept a single argument (an `Exception` class or instance), or
 no arguments to re-raise the last Exception.
 
 
@@ -1151,7 +1259,7 @@ while
 `while` form is used to execute a single or more blocks as long as a condition
 is being met.
 
-The following example will output "hello world!" on screen indefinetely:
+The following example will output "hello world!" on screen indefinitely:
 
 .. code-block:: clj
 
@@ -1185,23 +1293,48 @@ file is automatically closed after it has been processed.
 with-decorator
 --------------
 
-`with-decorator` is used to wrap a function with another. The function performing
+`with-decorator` is used to wrap a function with another. The function
-decoration should accept a single value, the function being decorated and return
+performing decoration should accept a single value, the function being
-a new function. `with-decorator` takes two parameters, the function performing
+decorated and return a new function. `with-decorator` takes a minimum
-decoration and the function being decorated.
+of two parameters, the function performing decoration and the function
+being decorated. More than one decorator function can be applied, they
+will be applied in order from outermost to innermost, ie. the first
+decorator will be the outermost one & so on. Decorators with arguments
+are called just like a function call.
 
-In the following example, `inc-decorator` is used to decorate function `addition`
+.. code-block:: clj
-with a function that takes two parameters and calls the decorated function with
+
-values that are incremented by 1. When decorated `addition` is called with values
+   (with-decorator decorator-fun
-1 and 1, the end result will be 4 (1+1 + 1+1).
+      (defn some-function [] ...)
+
+   (with-decorator decorator1 decorator2 ...
+      (defn some-function [] ...)
+
+   (with-decorator (decorator arg) ..
+      (defn some-function [] ...)
+
+
+
+In the following example, `inc-decorator` is used to decorate function
+`addition` with a function that takes two parameters and calls the
+decorated function with values that are incremented by 1. When
+decorated `addition` is called with values 1 and 1, the end result
+will be 4 (1+1 + 1+1).
 
 .. code-block:: clj
 
     => (defn inc-decorator [func]
     ...  (fn [value-1 value-2] (func (+ value-1 1) (+ value-2 1))))
+    => (defn inc2-decorator [func]
+    ...  (fn [value-1 value-2] (func (+ value-1 2) (+ value-2 2))))
+
     => (with-decorator inc-decorator (defn addition [a b] (+ a b)))
     => (addition 1 1)
     4
+    => (with-decorator inc2-decorator inc-decorator
+    ...	 (defn addition [a b] (+ a b)))
+    => (addition 1 1)
+    8
 
 
 .. _with-gensyms:

docs/language/core.rst

@@ -290,6 +290,46 @@ either ``int`` or ``long``. For Python 3, this is ``int``.
    False
 
 
+.. _interleave-fn:
+
+interleave
+----------
+
+.. versionadded:: 0.10.1
+
+Usage: ``(interleave seq1 seq2 ...)``
+
+Return an iterable of the first item in each of the sequences, then the second etc.
+
+.. code-block:: hy
+
+   => (list (interleave (range 5) (range 100 105)))
+   [0, 100, 1, 101, 2, 102, 3, 103, 4, 104]
+
+   => (list (interleave (range 1000000) "abc"))
+   [0, 'a', 1, 'b', 2, 'c']
+
+
+.. _interpose-fn:
+
+interpose
+---------
+
+.. versionadded:: 0.10.1
+
+Usage: ``(interpose item seq)``
+
+Return an iterable of the elements of the sequence separated by the item.
+
+.. code-block:: hy
+
+   => (list (interpose "!" "abcd"))
+   ['a', '!', 'b', '!', 'c', '!', 'd']
+
+   => (list (interpose -1 (range 5)))
+   [0, -1, 1, -1, 2, -1, 3, -1, 4]
+
+
 .. _iterable?-fn:
 
 iterable?
@@ -862,6 +902,33 @@ Return an iterator of `x`, `fn(x)`, `fn(fn(x))`.
    [5, 25, 625, 390625, 152587890625]
 
 
+.. _read-fn:
+
+read
+----
+
+Usage: ``(read [from-file eof])``
+
+Reads the next hy expression from `from-file` (defaults to `sys.stdin`), and
+can take a single byte as EOF (defaults to an empty string).
+Raises an `EOFError` if `from-file` ends before a complete expression can be
+parsed.
+
+.. code-block:: hy
+   => (read)
+   (+ 2 2)
+   ('+' 2 2)
+   => (eval (read))
+   (+ 2 2) 
+   4
+   => (import io)
+   => (def buffer (io.StringIO "(+ 2 2)\n(- 2 1)"))
+   => (eval (apply read [] {"from_file" buffer}))
+   4
+   => (eval (apply read [] {"from_file" buffer}))
+   1
+
+
 .. _remove-fn:
 
 remove

docs/language/internals.rst

@@ -387,7 +387,7 @@ for a more complete description.) ``nif`` is an example, something like a numeri
 where based on the expression, one of the 3 forms is called depending on if the
 expression is positive, zero or negative.
 
-A first pass might be someting like:
+A first pass might be something like:
 
 .. code-block:: hy
 

docs/language/readermacros.rst

@@ -70,4 +70,4 @@ and expression is passed to the correct function.
 .. warning::
    Because of a limitation in Hy's lexer and parser, reader macros can't
    redefine defined syntax such as ``()[]{}``. This will most likely be
-   adressed in the future.
+   addressed in the future.

docs/tutorial.rst

@@ -500,7 +500,7 @@ Let's take the classic:
 
     (loop (print (eval (read))))
 
-Rather then write it like that, we can write it as follows:
+Rather than write it like that, we can write it as follows:
 
 .. code-block:: clj
 

eg/sunlight/party-count.hy

@@ -1,6 +1,6 @@
 #!/usr/bin/env hy
 ;; Copyright (c) Paul R. Tagliamonte <paultag@debian.org>, 2013 under the terms
-;; of the Expat license, a copy of which you have should have recieved with
+;; of the Expat license, a copy of which you should have received with
 ;; the source.
 
 (import sys)

hy/__init__.py

@@ -23,7 +23,6 @@ from hy.version import __version__, __appname__  # NOQA
 
 
 from hy.models.expression import HyExpression  # NOQA
-from hy.models.lambdalist import HyLambdaListKeyword  # NOQA
 from hy.models.integer import HyInteger  # NOQA
 from hy.models.keyword import HyKeyword  # NOQA
 from hy.models.complex import HyComplex  # NOQA

hy/__main__.py

@@ -23,7 +23,7 @@ import imp
 import sys
 
 # This just mocks the normalish behavior of the Python interp. Helpful to aid
-# with shiming existing apps that don't really "work" with Hy.
+# with shimming existing apps that don't really "work" with Hy.
 #
 # You could say this script helps Hyjack a file.
 #

hy/cmdline.py

@@ -47,7 +47,7 @@ from hy.models.expression import HyExpression
 from hy.models.string import HyString
 from hy.models.symbol import HySymbol
 
-from hy._compat import builtins
+from hy._compat import builtins, PY3
 
 
 class HyQuitter(object):
@@ -203,6 +203,7 @@ def run_file(filename):
 
 
 def run_repl(hr=None, spy=False):
+    import platform
     sys.ps1 = "=> "
     sys.ps2 = "... "
 
@@ -210,10 +211,15 @@ def run_repl(hr=None, spy=False):
         if not hr:
             hr = HyREPL(spy)
 
-        hr.interact("{appname} {version}".format(
+        hr.interact("{appname} {version} using "
-            appname=hy.__appname__,
+                    "{py}({build}) {pyversion} on {os}".format(
-            version=hy.__version__
+                        appname=hy.__appname__,
-        ))
+                        version=hy.__version__,
+                        py=platform.python_implementation(),
+                        build=platform.python_build()[0],
+                        pyversion=platform.python_version(),
+                        os=platform.system()
+                    ))
 
     return 0
 
@@ -321,6 +327,7 @@ def hyc_main():
 
 # entry point for cmd line script "hy2py"
 def hy2py_main():
+    import platform
     module_name = "<STDIN>"
 
     options = dict(prog="hy2py", usage="%(prog)s [options] FILE",
@@ -343,17 +350,42 @@ def hy2py_main():
 
     if options.with_source:
         hst = import_file_to_hst(options.args[0])
-        print(hst)
+        # need special printing on Windows in case the
+        # codepage doesn't support utf-8 characters
+        if PY3 and platform.system() == "Windows":
+            for h in hst:
+                try:
+                    print(h)
+                except:
+                    print(str(h).encode('utf-8'))
+        else:
+            print(hst)
         print()
         print()
 
     _ast = import_file_to_ast(options.args[0], module_name)
     if options.with_ast:
-        print(astor.dump(_ast))
+        if PY3 and platform.system() == "Windows":
+            _print_for_windows(astor.dump(_ast))
+        else:
+            print(astor.dump(_ast))
         print()
         print()
 
     if not options.without_python:
-        print(astor.codegen.to_source(_ast))
+        if PY3 and platform.system() == "Windows":
+            _print_for_windows(astor.codegen.to_source(_ast))
+        else:
+            print(astor.codegen.to_source(_ast))
 
     parser.exit(0)
+
+
+# need special printing on Windows in case the
+# codepage doesn't support utf-8 characters
+def _print_for_windows(src):
+    for line in src.split("\n"):
+        try:
+            print(line)
+        except:
+            print(line.encode('utf-8'))

hy/compiler.py

@@ -1,10 +1,10 @@
 # -*- encoding: utf-8 -*-
 #
-# Copyright (c) 2013 Paul Tagliamonte <paultag@debian.org>
+# Copyright (c) 2013, 2014 Paul Tagliamonte <paultag@debian.org>
 # Copyright (c) 2013 Julien Danjou <julien@danjou.info>
 # Copyright (c) 2013 Nicolas Dandrimont <nicolas.dandrimont@crans.org>
 # Copyright (c) 2013 James King <james@agentultra.com>
-# Copyright (c) 2013 Bob Tolbert <bob@tolbert.org>
+# Copyright (c) 2013, 2014 Bob Tolbert <bob@tolbert.org>
 #
 # Permission is hereby granted, free of charge, to any person obtaining a
 # copy of this software and associated documentation files (the "Software"),
@@ -24,7 +24,6 @@
 # FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
 # DEALINGS IN THE SOFTWARE.
 
-from hy.models.lambdalist import HyLambdaListKeyword
 from hy.models.expression import HyExpression
 from hy.models.keyword import HyKeyword
 from hy.models.integer import HyInteger
@@ -48,6 +47,7 @@ import importlib
 import codecs
 import ast
 import sys
+import keyword
 
 from collections import defaultdict
 
@@ -74,6 +74,21 @@ def load_stdlib():
             _stdlib[e] = module
 
 
+# True, False and None included here since they
+# are assignable in Python 2.* but become
+# keywords in Python 3.*
+def _is_hy_builtin(name, module_name):
+    extras = ['True', 'False', 'None',
+              'true', 'false', 'nil', 'null']
+    if name in extras or keyword.iskeyword(name):
+        return True
+    # for non-Hy modules, check for pre-existing name in
+    # _compile_table
+    if not module_name.startswith("hy."):
+        return name in _compile_table
+    return False
+
+
 _compile_table = {}
 
 
@@ -437,6 +452,7 @@ class HyASTCompiler(object):
 
     def _parse_lambda_list(self, exprs):
         """ Return FunctionDef parameter values from lambda list."""
+        ll_keywords = ("&rest", "&optional", "&key", "&kwargs")
         ret = Result()
         args = []
         defaults = []
@@ -446,10 +462,7 @@ class HyASTCompiler(object):
 
         for expr in exprs:
 
-            if isinstance(expr, HyLambdaListKeyword):
+            if expr in ll_keywords:
-                if expr not in expr._valid_types:
-                    raise HyTypeError(expr, "{0} is not a valid "
-                                      "lambda-keyword.".format(repr(expr)))
                 if expr == "&rest" and lambda_keyword is None:
                     lambda_keyword = expr
                 elif expr == "&optional":
@@ -623,7 +636,7 @@ class HyASTCompiler(object):
 
             return imports, ret.replace(form), False
 
-        elif isinstance(form, (HySymbol, HyLambdaListKeyword)):
+        elif isinstance(form, HySymbol):
             return imports, HyExpression([HySymbol(name),
                                           HyString(form)]).replace(form), False
 
@@ -723,7 +736,8 @@ class HyASTCompiler(object):
                              lineno=expr.start_line,
                              col_offset=expr.start_column)
 
-        returnable = Result(expr=expr_name, temp_variables=[expr_name, name])
+        returnable = Result(expr=expr_name, temp_variables=[expr_name, name],
+                            contains_yield=body.contains_yield)
 
         body += ast.Assign(targets=[name],
                            value=body.force_expr,
@@ -1015,7 +1029,10 @@ class HyASTCompiler(object):
     @checkargs(max=1)
     def compile_yield_expression(self, expr):
         expr.pop(0)
-        ret = Result(contains_yield=True)
+        if PY33:
+            ret = Result(contains_yield=False)
+        else:
+            ret = Result(contains_yield=True)
 
         value = None
         if expr != []:
@@ -1528,7 +1545,7 @@ class HyASTCompiler(object):
     def compile_require(self, expression):
         """
         TODO: keep track of what we've imported in this run and then
-        "unimport" it after we've completed `thing' so that we don't polute
+        "unimport" it after we've completed `thing' so that we don't pollute
         other envs.
         """
         expression.pop(0)
@@ -1766,19 +1783,26 @@ class HyASTCompiler(object):
 
     def _compile_assign(self, name, result,
                         start_line, start_column):
+
+        str_name = "%s" % name
+        if _is_hy_builtin(str_name, self.module_name):
+            raise HyTypeError(name,
+                              "Can't assign to a builtin: `%s'" % str_name)
+
         result = self.compile(result)
-
-        if result.temp_variables and isinstance(name, HyString):
-            result.rename(name)
-            return result
-
         ld_name = self.compile(name)
-        st_name = self._storeize(ld_name)
 
-        result += ast.Assign(
+        if result.temp_variables \
-            lineno=start_line,
+           and isinstance(name, HyString) \
-            col_offset=start_column,
+           and '.' not in name:
-            targets=[st_name], value=result.force_expr)
+            result.rename(name)
+        else:
+            st_name = self._storeize(ld_name)
+            result += ast.Assign(
+                lineno=start_line,
+                col_offset=start_column,
+                targets=[st_name],
+                value=result.force_expr)
 
         result += ld_name
         return result
@@ -1871,7 +1895,7 @@ class HyASTCompiler(object):
         ret, args, defaults, stararg, kwargs = self._parse_lambda_list(arglist)
 
         if PY34:
-            # Python 3.4+ requres that args are an ast.arg object, rather
+            # Python 3.4+ requires that args are an ast.arg object, rather
             # than an ast.Name or bare string.
             args = [ast.arg(arg=ast_str(x),
                             annotation=None,  # Fix me!

hy/contrib/anaphoric.hy

@@ -46,9 +46,10 @@
 
 (defmacro ap-map [form lst]
   "Yield elements evaluated in the form for each element in the list."
-  `(let [[f (lambda [it] ~form)]]
+  (let [[v (gensym 'v)] [f (gensym 'f)]]
-     (for [v ~lst]
+    `(let [[~f (lambda [it] ~form)]]
-       (yield (f v)))))
+       (for [~v ~lst]
+         (yield (~f ~v))))))
 
 
 (defmacro ap-map-when [predfn rep lst]
@@ -83,18 +84,20 @@
 
 (defmacro ap-first [predfn lst]
   "Yield the first element that passes `predfn`"
-  `(let [[n (gensym)]]
+  (with-gensyms [n]
-     (ap-each ~lst (when ~predfn (setv n it) (break)))
+    `(let [[~n None]]
-     n))
+       (ap-each ~lst (when ~predfn (setv ~n it) (break)))
+       ~n)))
 
 
 (defmacro ap-last [predfn lst]
   "Yield the last element that passes `predfn`"
-  `(let [[n (gensym)]]
+  (with-gensyms [n]
-     (ap-each ~lst (none? n)
+    `(let [[~n None]]
-	      (when ~predfn
+       (ap-each ~lst (none? ~n)
-		(setv n it)))
+                (when ~predfn
-	n))
+                  (setv ~n it)))
+       ~n)))
 
 
 (defmacro ap-reduce [form lst &optional [initial-value None]]

hy/core/__init__.py

@@ -1,3 +1,4 @@
 STDLIB = [
-    "hy.core.language"
+    "hy.core.language",
+    "hy.core.shadow"
 ]

hy/core/language.hy

@@ -26,14 +26,20 @@
 (import itertools)
 (import functools)
 (import collections)
+(import sys) 
 (import [hy._compat [long-type]]) ; long for python2, int for python3
 (import [hy.models.cons [HyCons]])
+(import [hy.lex [LexException PrematureEndOfInput tokenize]])
 
 
 (defn _numeric-check [x]
   (if (not (numeric? x))
     (raise (TypeError (.format "{0!r} is not a number" x)))))
 
+(defn butlast [coll]
+  "Returns coll except of last element."
+  (itertools.islice coll 0 (dec (len coll))))
+
 (defn coll? [coll]
   "Checks whether item is a collection"
   (and (iterable? coll) (not (string? coll))))
@@ -58,17 +64,17 @@
   (- n 1))
 
 (defn disassemble [tree &optional [codegen false]]
-  "Dump the python AST for a given Hy tree to standard output
+  "Return the python AST for a quoted Hy tree as a string.
    If the second argument is true, generate python code instead."
   (import astor)
   (import hy.compiler)
 
   (fake-source-positions tree)
   (setv compiled (hy.compiler.hy_compile tree (calling-module-name)))
-  (print ((if codegen
+  ((if codegen
             astor.codegen.to_source
             astor.dump)
-          compiled)))
+          compiled))
 
 (defn distinct [coll]
   "Return a generator from the original collection with duplicates
@@ -88,7 +94,8 @@
     (setv map itertools.imap)
     (setv zip itertools.izip)
     (setv range xrange)
-    (setv input raw_input))
+    (setv input raw_input)
+    (setv reduce reduce))
   (do
     (setv reduce functools.reduce)
     (setv filterfalse itertools.filterfalse)
@@ -106,6 +113,7 @@
 (setv drop-while itertools.dropwhile)
 (setv take-while itertools.takewhile)
 (setv zipwith map)
+(setv remove filterfalse)
 
 (defn drop [count coll]
   "Drop `count` elements from `coll` and yield back the rest"
@@ -203,6 +211,14 @@
     (catch [e ValueError] False)
     (catch [e TypeError] False)))
 
+(defn interleave [&rest seqs]
+  "Return an iterable of the first item in each of seqs, then the second etc."
+  (itertools.chain.from_iterable (apply zip seqs)))
+
+(defn interpose [item seq]
+  "Return an iterable of the elements of seq separated by item"
+  (drop 1 (interleave (itertools.repeat item) seq)))
+
 (defn iterable? [x]
   "Return true if x is iterable"
   (isinstance x collections.Iterable))
@@ -270,13 +286,6 @@
   (_numeric_check n)
   (> n 0))
 
-(defn remove [pred coll]
-  "Return coll with elements removed that pass `pred`"
-  (let [[citer (iter coll)]]
-    (for* [val citer]
-      (if (not (pred val))
-        (yield val)))))
-
 (defn rest [coll]
   "Get all the elements of a coll, except the first."
   (drop 1 coll))
@@ -327,10 +336,29 @@
   (_numeric_check n)
   (= n 0))
 
-(def *exports* '[calling-module-name coll? cons cons? cycle dec distinct
+(defn read [&optional [from-file sys.stdin]
-                 disassemble drop drop-while empty? even? every? first filter
+                      [eof ""]]
-                 flatten float? gensym identity inc instance? integer
+  "Read from input and returns a tokenized string.
-                 integer? integer-char? iterable? iterate iterator? keyword?
+   Can take a given input buffer to read from"
-                 list* macroexpand macroexpand-1 map neg? nil? none? nth
+  (def buff "")
-                 numeric? odd? pos? range remove repeat repeatedly rest second
+  (while true
-                 some string string? take take-nth take-while zero? zip zipwith])
+    (def inn (str (.read from-file 1)))
+    (if (= inn eof)
+      (throw (EOFError "Reached end of file" )))
+    (setv buff (+ buff inn))
+    (try
+      (def parsed (first (tokenize buff)))
+      (except [e [LexException PrematureEndOfInput IndexError]]) 
+      (else (if parsed (break)))))
+    parsed)
+
+
+(def *exports* '[butlast calling-module-name coll? cons cons? cycle
+                 dec distinct disassemble drop drop-while empty? even?
+                 every? first filter filterfalse flatten float? gensym identity
+                 inc input instance? integer integer? integer-char? interleave
+                 interpose iterable? iterate iterator? keyword? list*
+                 macroexpand macroexpand-1 map neg? nil? none? nth
+                 numeric? odd? pos? range read remove repeat repeatedly
+                 rest reduce second some string string? take take-nth
+                 take-while zero? zip zip_longest zipwith])

hy/core/macros.hy

@@ -31,34 +31,6 @@
         [hy._compat [PY33 PY34]])
 
 
-(defmacro for [args &rest body]
-  "shorthand for nested for loops:
-  (for [x foo
-        y bar]
-    baz) ->
-  (for* [x foo]
-    (for* [y bar]
-      baz))"
-
-  (if (odd? (len args))
-    (macro-error args "`for' requires an even number of args."))
-
-  (if (empty? body)
-    (macro-error None "`for' requires a body to evaluate"))
-
-  (if (empty? args)
-    `(do ~@body)
-    (if (= (len args) 2)
-      ; basecase, let's just slip right in.
-      `(for* [~@args] ~@body)
-      ; otherwise, let's do some legit handling.
-      (let [[alist (slice args 0 nil 2)]
-            [ilist (slice args 1 nil 2)]]
-        `(do
-           (import itertools)
-           (for* [(, ~@alist) (itertools.product ~@ilist)] ~@body))))))
-
-
 (defmacro with [args &rest body]
   "shorthand for nested for* loops:
   (with [[x foo] [y bar]] baz) ->
@@ -116,6 +88,26 @@
   root)
 
 
+(defmacro for [args &rest body]
+  "shorthand for nested for loops:
+  (for [x foo
+        y bar]
+    baz) ->
+  (for* [x foo]
+    (for* [y bar]
+      baz))"
+  (cond 
+   [(odd? (len args))
+    (macro-error args "`for' requires an even number of args.")]
+   [(empty? body)
+    (macro-error None "`for' requires a body to evaluate")]
+   [(empty? args) `(do ~@body)]
+   [(= (len args) 2)  `(for* [~@args] ~@body)]
+   [true 
+    (let [[alist (slice args 0 nil 2)]]
+      `(for* [(, ~@alist) (genexpr (, ~@alist) [~@args])] ~@body))]))
+
+
 (defmacro -> [head &rest rest]
   ;; TODO: fix the docstring by someone who understands this
   (setv ret head)
@@ -127,6 +119,18 @@
   ret)
 
 
+(defmacro doto [form &rest expressions]
+  "Performs a sequence of potentially mutating actions
+   on an initial object, returning the resulting object"
+  (setv f (gensym))
+  (defn build-form [expression]
+    (if (isinstance expression HyExpression)
+      `(~(first expression) ~f ~@(rest expression))
+      `(~expression ~f)))
+  `(let [[~f ~form]]
+     ~@(map build-form expressions)
+     ~f))
+
 (defmacro ->> [head &rest rest]
   ;; TODO: fix the docstring by someone who understands this
   (setv ret head)
@@ -165,11 +169,26 @@
     ~@body))
 
 (defmacro defmacro/g! [name args &rest body]
-  (let [[syms (list (distinct (filter (fn [x] (.startswith x "g!")) (flatten body))))]]
+  (let [[syms (list (distinct (filter (fn [x] (and (hasattr x "startswith") (.startswith x "g!"))) (flatten body))))]]
     `(defmacro ~name [~@args]
        (let ~(HyList (map (fn [x] `[~x (gensym (slice '~x 2))]) syms))
             ~@body))))
 
+
+(defmacro defmain [args &rest body]
+  "Write a function named \"main\" and do the if __main__ dance"
+  (let [[retval (gensym)]]
+    `(do
+      (defn main [~@args]
+        ~@body)
+
+      (when (= --name-- "__main__")
+        (import sys)
+        (setv ~retval (apply main sys.argv))
+        (if (integer? ~retval)
+          (sys.exit ~retval))))))
+
+
 (defmacro-alias [defn-alias defun-alias] [names lambda-list &rest body]
   "define one function with several names"
   (let [[main (first names)]

hy/core/shadow.hy

@@ -0,0 +1,61 @@
+;; Copyright (c) 2014 Paul Tagliamonte <paultag@debian.org>
+;; Copyright (c) 2014 James King <james@agentultra.com>
+
+;; Permission is hereby granted, free of charge, to any person obtaining a
+;; copy of this software and associated documentation files (the "Software"),
+;; to deal in the Software without restriction, including without limitation
+;; the rights to use, copy, modify, merge, publish, distribute, sublicense,
+;; and/or sell copies of the Software, and to permit persons to whom the
+;; Software is furnished to do so, subject to the following conditions:
+
+;; The above copyright notice and this permission notice shall be included in
+;; all copies or substantial portions of the Software.
+
+;; THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+;; IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+;; FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+;; THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+;; LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+;; FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+;; DEALINGS IN THE SOFTWARE.
+
+;;;; Hy shadow functions
+
+(import operator)
+
+
+(defn + [&rest args]
+  "Shadow + operator for when we need to import / map it against something"
+  (if (= (len args) 0)
+    0
+    (sum args)))  ; shortcut here.
+
+
+(defn - [&rest args]
+  "Shadow - operator for when we need to import / map it against something"
+  (let [[count (len args)]]
+    (if (= count 0)
+      (raise (TypeError "Need at least 1 argument to subtract"))
+      (if (= count 1)
+        (- (get args 0))
+        (reduce operator.sub args)))))
+
+
+(defn * [&rest args]
+  "Shadow * operator for when we need to import / map it against something"
+  (if (= (len args) 0)
+    1  ; identity
+    (reduce operator.mul args)))
+
+
+(defn / [&rest args]
+  "Shadow / operator for when we need to import / map it against something"
+  (let [[count (len args)]]
+    (if (= count 0)
+      (raise (TypeError "Need at least 1 argument to divide"))
+      (if (= count 1)
+        (operator.truediv 1 (get args 0))
+        (reduce operator.truediv args)))))
+
+
+(setv *exports* ['+ '- '* '/])

hy/errors.py

@@ -28,7 +28,7 @@ import traceback
 
 class HyError(Exception):
     """
-    Generic Hy error. All interal Exceptions will be subclassed from this
+    Generic Hy error. All internal Exceptions will be subclassed from this
     Exception.
     """
     pass

hy/lex/parser.py

@@ -30,7 +30,6 @@ from hy.models.expression import HyExpression
 from hy.models.float import HyFloat
 from hy.models.integer import HyInteger
 from hy.models.keyword import HyKeyword
-from hy.models.lambdalist import HyLambdaListKeyword
 from hy.models.list import HyList
 from hy.models.string import HyString
 from hy.models.symbol import HySymbol
@@ -271,9 +270,6 @@ def t_identifier(p):
     if obj.startswith(":"):
         return HyKeyword(obj)
 
-    if obj.startswith("&"):
-        return HyLambdaListKeyword(obj)
-
     def mangle(p):
         if p.startswith("*") and p.endswith("*") and p not in ("*", "**"):
             p = p[1:-1].upper()

hy/models/complex.py

@@ -23,7 +23,7 @@ from hy.models import HyObject
 
 class HyComplex(HyObject, complex):
     """
-    Internal represntation of a Hy Complex. May raise a ValueError as if
+    Internal representation of a Hy Complex. May raise a ValueError as if
     complex(foo) was called, given HyComplex(foo).
     """
 

hy/models/float.py

@@ -23,7 +23,7 @@ from hy.models import HyObject
 
 class HyFloat(HyObject, float):
     """
-    Internal represntation of a Hy Float. May raise a ValueError as if
+    Internal representation of a Hy Float. May raise a ValueError as if
     float(foo) was called, given HyFloat(foo).
     """
 

hy/models/integer.py

@@ -24,7 +24,7 @@ from hy._compat import long_type
 
 class HyInteger(HyObject, long_type):
     """
-    Internal represntation of a Hy Integer. May raise a ValueError as if
+    Internal representation of a Hy Integer. May raise a ValueError as if
     int(foo) was called, given HyInteger(foo). On python 2.x long will
     be used instead
     """

hy/models/lambdalist.py

@@ -1,39 +0,0 @@
-# Copyright (c) 2013 James King <james@agentultra.com>
-#
-# Permission is hereby granted, free of charge, to any person obtaining a
-# copy of this software and associated documentation files (the "Software"),
-# to deal in the Software without restriction, including without limitation
-# the rights to use, copy, modify, merge, publish, distribute, sublicense,
-# and/or sell copies of the Software, and to permit persons to whom the
-# Software is furnished to do so, subject to the following conditions:
-#
-# The above copyright notice and this permission notice shall be included in
-# all copies or substantial portions of the Software.
-#
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
-# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
-# DEALINGS IN THE SOFTWARE.
-
-from hy.models.string import HyString
-
-
-class HyLambdaListKeyword(HyString):
-    """
-    Hy LambdaListKeyword. Demarcates arguments in an argument list.
-
-    (defun my-fun (x &rest xs &optional (foo "default string")))
-
-    becomes:
-
-    def my_fun(x, *xs, foo="default string"):
-        pass
-    """
-
-    _valid_types = ["&rest", "&optional", "&key", "&kwargs"]
-
-    def __init__(self, string):
-        self += string

requirements-dev.txt

@@ -1,11 +1,8 @@
+-r requirements-travis.txt
 # test tools
 nose
 tox
 
-# code quality
-flake8
-coverage
-
 # documentation
 Pygments>=1.6
 Sphinx

requirements-travis.txt

@@ -0,0 +1,3 @@
+# code quality
+flake8
+coverage

tests/lex/test_lex.py

@@ -21,7 +21,6 @@
 
 from hy.models.expression import HyExpression
 from hy.models.integer import HyInteger
-from hy.models.lambdalist import HyLambdaListKeyword
 from hy.models.float import HyFloat
 from hy.models.complex import HyComplex
 from hy.models.symbol import HySymbol
@@ -85,14 +84,6 @@ def test_lex_expression_integer():
     assert objs == [HyExpression([HySymbol("foo"), HyInteger(2)])]
 
 
-def test_lex_lambda_list_keyword():
-    """ Make sure expressions can produce lambda list keywords """
-    objs = tokenize("(x &rest xs)")
-    assert objs == [HyExpression([HySymbol("x"),
-                                  HyLambdaListKeyword("&rest"),
-                                  HySymbol("xs")])]
-
-
 def test_lex_symbols():
     """ Make sure that symbols are valid expressions"""
     objs = tokenize("foo ")

tests/native_tests/contrib/anaphoric.hy

@@ -54,7 +54,9 @@
   (assert-equal (list (ap-map (* it 3) [1 2 3]))
                 [3 6 9])
   (assert-equal (list (ap-map (* it 3) []))
-                []))
+                [])
+  (assert-equal (let [[v 1] [f 1]] (list (ap-map (it v f) [(fn [a b] (+ a b))])))
+                [2]))
 
 (defn test-ap-map-when []
   "NATIVE: testing anaphoric map-when"
@@ -85,12 +87,14 @@
 (defn test-ap-first []
   "NATIVE: testing anaphoric first"
   (assert-equal (ap-first (> it 5) (range 10)) 6)
-  (assert-equal (ap-first (even? it) [1 2 3 4]) 2))
+  (assert-equal (ap-first (even? it) [1 2 3 4]) 2)
+  (assert-equal (ap-first (> it 10) (range 10)) None))
 
 (defn test-ap-last []
   "NATIVE: testing anaphoric last"
   (assert-equal (ap-last (> it 5) (range 10)) 9)
-  (assert-equal (ap-last (even? it) [1 2 3 4]) 4))
+  (assert-equal (ap-last (even? it) [1 2 3 4]) 4)
+  (assert-equal (ap-last (> it 10) (range 10)) None))
 
 (defn test-ap-reduce []
   "NATIVE: testing anaphoric reduce"

tests/native_tests/core.hy

@@ -57,6 +57,34 @@
   (try (do (dec None) (assert False))
        (catch [e [TypeError]] (assert (in "not a number" (str e))))))
 
+(defn test-setv []
+  "NATIVE: testing setv mutation"
+  (setv x 1)
+  (setv y 1)
+  (assert-equal x y)
+  (setv x (setv y  12))
+  (assert-equal x 12)
+  (assert-equal y 12)
+  (setv x (setv y (fn [x] 9)))
+  (assert-equal (x y) 9)
+  (assert-equal (y x) 9)
+  (try (do (setv a.b 1) (assert False))
+       (catch [e [NameError]] (assert (in "name 'a' is not defined" (str e)))))
+  (try (do (setv b.a (fn [x] x)) (assert False))
+       (catch [e [NameError]] (assert (in "name 'b' is not defined" (str e)))))
+  (import itertools)
+  (setv foopermutations (fn [x] (itertools.permutations x)))
+  (setv p (set [(, 1 3 2) (, 3 2 1) (, 2 1 3) (, 3 1 2) (, 1 2 3) (, 2 3 1)]))
+  (assert-equal (set (itertools.permutations [1 2 3])) p)
+  (assert-equal (set (foopermutations [3 1 2])) p)
+  (setv permutations- itertools.permutations)
+  (setv itertools.permutations (fn [x] 9))
+  (assert-equal (itertools.permutations p) 9)
+  (assert-equal (foopermutations foopermutations) 9)
+  (setv itertools.permutations permutations-)
+  (assert-equal (set (itertools.permutations [2 1 3])) p)
+  (assert-equal (set (foopermutations [2 3 1])) p))
+
 (defn test-distinct []
   "NATIVE: testing the distinct function"
   (setv res (list (distinct [ 1 2 3 4 3 5 2 ])))
@@ -238,6 +266,33 @@
   (assert-false (integer-char? "foo"))
   (assert-false (integer-char? None)))
 
+(defn test-interleave []
+  "NATIVE: testing the interleave function"
+  ;; with more than 2 sequences
+  (assert-equal (list (take 9 (interleave (range 10)
+                                          (range 10 20)
+                                          (range 20 30))))
+                [0 10 20 1 11 21 2 12 22])
+  ;; with sequences of different length
+  (assert-equal (list (interleave (range 1000000)
+                                  (range 0 -3 -1)))
+                [0 0 1 -1 2 -2])
+  ;; with infinite sequences
+  (import itertools)
+  (assert-equal (list (take 10 (interleave (itertools.count)
+                                           (itertools.count 100))))
+                [0 100 1 101 2 102 3 103 4 104]))
+
+(defn test-interpose []
+  "NATIVE: testing the interpose function"
+  ;; with a list
+  (assert-equal (list (interpose "!" ["a" "b" "c"]))
+                ["a" "!" "b" "!" "c"])
+  ;; with an infinite sequence
+  (import itertools)
+  (assert-equal (list (take 7 (interpose -1 (itertools.count))))
+                [0 -1 1 -1 2 -1 3]))
+
 (defn test-iterable []
   "NATIVE: testing iterable? function"
   ;; should work for a string
@@ -482,6 +537,16 @@
   (setv res (zipwith operator.sub [3 7 9] [1 2 4]))
   (assert-equal (list res) [2 5 5]))
 
+(defn test-doto []
+  "NATIVE: testing doto macro"
+  (setv collection [])
+  (doto collection (.append 1) (.append 2) (.append 3))
+  (assert-equal collection [1 2 3])
+  (setv res (doto (set) (.add 2) (.add 1)))
+  (assert-equal res (set [1 2]))
+  (setv res (doto [] (.append 1) (.append 2) .reverse))
+  (assert-equal res [2 1]))
+
 (defn test-is-keyword []
   "NATIVE: testing the keyword? function"
   (assert (keyword? ':bar))

tests/native_tests/language.hy

@@ -3,6 +3,7 @@
         [sys :as systest])
 (import sys)
 
+(import [hy._compat [PY33 PY34]])
 
 (defn test-sys-argv []
   "NATIVE: test sys.argv"
@@ -27,9 +28,31 @@
   (setv (get foo 0) 12)
   (assert (= (get foo 0) 12)))
 
+(defn test-setv-builtin []
+  "NATIVE: test that setv doesn't work on builtins"
+  (try (eval '(setv False 1))
+       (catch [e [TypeError]] (assert (in "Can't assign to a builtin" (str e)))))
+  (try (eval '(setv True 0))
+       (catch [e [TypeError]] (assert (in "Can't assign to a builtin" (str e)))))
+  (try (eval '(setv None 1))
+       (catch [e [TypeError]] (assert (in "Can't assign to a builtin" (str e)))))
+  (try (eval '(setv false 1))
+       (catch [e [TypeError]] (assert (in "Can't assign to a builtin" (str e)))))
+  (try (eval '(setv true 0))
+       (catch [e [TypeError]] (assert (in "Can't assign to a builtin" (str e)))))
+  (try (eval '(setv nil 1))
+       (catch [e [TypeError]] (assert (in "Can't assign to a builtin" (str e)))))
+  (try (eval '(setv null 1))
+       (catch [e [TypeError]] (assert (in "Can't assign to a builtin" (str e)))))
+  (try (eval '(defn defclass [] (print "hello")))
+       (catch [e [TypeError]] (assert (in "Can't assign to a builtin" (str e)))))
+  (try (eval '(defn get [] (print "hello")))
+       (catch [e [TypeError]] (assert (in "Can't assign to a builtin" (str e)))))
+  (try (eval '(defn lambda [] (print "hello")))
+       (catch [e [TypeError]] (assert (in "Can't assign to a builtin" (str e))))))
 
 (defn test-for-loop []
-  "NATIVE: test for loops?"
+  "NATIVE: test for loops"
   (setv count 0)
   (for [x [1 2 3 4 5]]
     (setv count (+ count x)))
@@ -38,7 +61,11 @@
   (for [x [1 2 3 4 5]
         y [1 2 3 4 5]]
     (setv count (+ count x y)))
-  (assert (= count 150)))
+  (assert (= count 150))
+  (assert (= (list ((fn [] (for [x [[1] [2 3]] y x] (yield y)))))
+             (list-comp y [x [[1] [2 3]] y x])))
+  (assert (= (list ((fn [] (for [x [[1] [2 3]] y x z (range 5)] (yield z)))))
+             (list-comp z [x [[1] [2 3]] y x z (range 5)]))))
 
 
 (defn test-nasty-for-nesting []
@@ -444,6 +471,29 @@
   (for [y (gen)] (setv ret (+ ret y)))
   (assert (= ret 10)))
 
+(defn test-yield-with-return []
+  "NATIVE: test yield with return"
+  (defn gen [] (yield 3) "goodbye")
+  (if PY33
+    (do (setv gg (gen))
+        (assert (= 3 (next gg)))
+        (try (next gg)
+             (except [e StopIteration] (assert (hasattr e "value"))
+                                       (assert (= (getattr e "value") "goodbye")))))
+    (do (setv gg (gen))
+        (assert (= 3 (next gg)))
+        (try (next gg)
+             (except [e StopIteration] (assert (not (hasattr e "value"))))))))
+
+
+(defn test-yield-in-try []
+  "NATIVE: test yield in try"
+  (defn gen []
+    (let [[x 1]]
+    (try (yield x)
+         (finally (print x)))))
+  (setv output (list (gen)))
+  (assert (= [1] output)))
 
 (defn test-first []
   "NATIVE: test firsty things"
@@ -942,22 +992,10 @@
 
 (defn test-disassemble []
   "NATIVE: Test the disassemble function"
-  (import sys)
+  (assert (= (disassemble '(do (leaky) (leaky) (macros)))
-  (if-python2
+             "Module(\n    body=[\n        Expr(value=Call(func=Name(id='leaky'), args=[], keywords=[], starargs=None, kwargs=None)),\n        Expr(value=Call(func=Name(id='leaky'), args=[], keywords=[], starargs=None, kwargs=None)),\n        Expr(value=Call(func=Name(id='macros'), args=[], keywords=[], starargs=None, kwargs=None))])"))
-   (import [io [BytesIO :as StringIO]])
+  (assert (= (disassemble '(do (leaky) (leaky) (macros)) true)
-   (import [io [StringIO]]))
+             "leaky()\nleaky()\nmacros()")))
-  (setv prev-stdout sys.stdout)
-  (setv sys.stdout (StringIO))
-  (disassemble '(do (leaky) (leaky) (macros)))
-  (setv stdout (.getvalue sys.stdout))
-  (setv sys.stdout prev-stdout)
-  (assert (in "leaky" stdout))
-  (assert (in "macros" stdout))
-  (setv sys.stdout (StringIO))
-  (disassemble '(do (leaky) (leaky) (macros)) true)
-  (setv stdout (.getvalue sys.stdout))
-  (setv sys.stdout prev-stdout)
-  (assert (= stdout "leaky()\nleaky()\nmacros()\n")))
 
 
 (defn test-attribute-access []
@@ -983,3 +1021,36 @@
   "NATIVE: test keyword quoting magic"
   (assert (= :foo "\ufdd0:foo"))
   (assert (= `:foo "\ufdd0:foo")))
+
+(defn test-only-parse-lambda-list-in-defn []
+  "NATIVE: test lambda lists are only parsed in defn"
+  (try
+   (foo [&rest spam] 1)
+   (catch [NameError] True)
+   (else (raise AssertionError))))
+
+(defn test-read []
+  "NATIVE: test that read takes something for stdin and reads"
+  (if-python2
+    (import [StringIO [StringIO]])
+    (import [io [StringIO]]))
+  (import [hy.models.expression [HyExpression]])
+ 
+  (def stdin-buffer (StringIO "(+ 2 2)\n(- 2 2)"))
+  (assert (= (eval (read stdin-buffer)) 4))
+  (assert (isinstance (read stdin-buffer) HyExpression))
+  
+  "Multiline test"
+  (def stdin-buffer (StringIO "(\n+\n41\n1\n)\n(-\n2\n1\n)"))
+  (assert (= (eval (read stdin-buffer)) 42))
+  (assert (= (eval (read stdin-buffer)) 1))
+
+  "EOF test"
+  (def stdin-buffer (StringIO "(+ 2 2)"))
+  (read stdin-buffer)
+  (try 
+    (read stdin-buffer)
+    (catch [e Exception]
+      (assert (isinstance e EOFError)))))
+
+

tests/native_tests/native_macros.hy

@@ -170,7 +170,12 @@
   (setv s2 (to_source _ast2))
   (assert (in ":res_" s1))
   (assert (in ":res_" s2))
-  (assert (not (= s1 s2))))
+  (assert (not (= s1 s2)))
+
+  ;; defmacro/g! didn't like numbers initially because they
+  ;; don't have a startswith method and blew up during expansion
+  (setv macro2 "(defmacro/g! two-point-zero [] `(+ (float 1) 1.0))")
+  (assert (import_buffer_to_ast macro2 "foo")))
 
 
 (defn test-if-not []

tests/native_tests/quote.hy

@@ -77,12 +77,6 @@
   (assert (= q qq)))
 
 
-(defn test-quote-lambdalistkeyword []
-  "NATIVE: test quoting lambda list keywords"
-  (setv opt (quote &optional))
-  (assert (isinstance opt hy.HyLambdaListKeyword))
-  (assert (= (str opt) "&optional")))
-
 (defmacro doodle [&rest body]
   `(do ~@body))
 

tests/native_tests/shadow.hy

@@ -0,0 +1,41 @@
+
+
+(defn test-shadow-addition []
+  "NATIVE: test shadow addition"
+  (let [[x +]]
+    (assert (= (x) 0))
+    (assert (= (x 1 2 3 4) 10))
+    (assert (= (x 1 2 3 4 5) 15))))
+
+
+(defn test-shadow-subtraction []
+  "NATIVE: test shadow subtraction"
+  (let [[x -]]
+    (assert (try
+             (x)
+             (catch [TypeError] True)
+             (else (throw AssertionError))))
+    (assert (= (x 1) -1))
+    (assert (= (x 2 1) 1))
+    (assert (= (x 2 1 1) 0))))
+
+
+(defn test-shadow-multiplication []
+  "NATIVE: test shadow multiplication"
+  (let [[x *]]
+    (assert (= (x) 1))
+    (assert (= (x 3) 3))
+    (assert (= (x 3 3) 9))))
+
+
+(defn test-shadow-division []
+  "NATIVE: test shadow division"
+  (let [[x /]]
+    (assert (try
+             (x)
+             (catch [TypeError] True)
+             (else (throw AssertionError))))
+    (assert (= (x 1) 1))
+    (assert (= (x 8 2) 4))
+    (assert (= (x 8 2 2) 2))
+    (assert (= (x 8 2 2 2) 1))))

tests/resources/bin/main.hy

@@ -0,0 +1,5 @@
+(defmain [&rest args]
+  (print args)
+  (print "Hello World")
+  (if (in "exit1" args)
+    1))

tests/resources/bin/nomain.hy

@@ -0,0 +1,4 @@
+(print "This Should Still Works")
+
+(defn main []
+    (print "This Should Not Work"))

tests/test_bin.py

@@ -146,3 +146,27 @@ def test_bin_hy_builtins():
 
     assert str(exit) == "Use (exit) or Ctrl-D (i.e. EOF) to exit"
     assert str(quit) == "Use (quit) or Ctrl-D (i.e. EOF) to exit"
+
+
+def test_bin_hy_main():
+    ret = run_cmd("hy tests/resources/bin/main.hy")
+    assert ret[0] == 0
+    assert "Hello World" in ret[1]
+
+
+def test_bin_hy_main_args():
+    ret = run_cmd("hy tests/resources/bin/main.hy test 123")
+    assert ret[0] == 0
+    assert "test" in ret[1]
+    assert "123" in ret[1]
+
+
+def test_bin_hy_main_exitvalue():
+    ret = run_cmd("hy tests/resources/bin/main.hy exit1")
+    assert ret[0] == 1
+
+
+def test_bin_hy_no_main():
+    ret = run_cmd("hy tests/resources/bin/nomain.hy")
+    assert ret[0] == 0
+    assert "This Should Still Work" in ret[1]