Much like how `can_compile` returns the compilation result, which some
tests make use of, it may be useful for for `cant_compile` to return the
exception object that it caught, for more specific assertions.
Python 3 supports keyword-only arguments as described in the immortal
PEP 3102. This commit implements keyword-only argument support for Hy
using a `&kwonly` lambda-list-keyword with semantics analogous how
`&optional` arguments are handled: `&kwonly` arguments are either a
symbol, in which case the keyword argument so named is mandatory, or a
two-element list, the first of which is the symbolic name of the keyword
argument and the second of which is its default value if not
supplied. If Hy is running under Python 2, attempting to use `&kwonly`
args will raise a HyTypeError.
This effort is with the aim of resolving #453.
Expressions can sometimes contain itertools.islice objects, which we can
only walk if we force them into a list. To do this, the walk function
has to be taught that collections that are not instances of list should
be forced into a list.
Signed-off-by: Gergely Nagy <algernon@madhouse-project.org>
Python 3.5's PEP 448 ("Additional Unpacking Generalizations") allows the
iterable- and dictionary- unpacking operators to be used more than once;
the implementation (see https://hg.python.org/cpython/rev/a65f685ba8c0)
gets rid of the optional `starargs` and `kwargs` arguments to `ast.Call`
and `ast.ClassDef`, instead using `ast.Starred` and `ast.keyword`
objects inside of the normal `args` and `keywords` lists,
respectively. This commit allows Hy's `apply` to work correctly with
this revised AST when running under Python 3.5.
As reported in issue #748, there was a bug in which passing a lambda
as the value of a :keyword argument would fail—
$ hy --spy
hy 0.10.1 using CPython(default) 3.4.0 on Linux
=> (sorted (range 10) :key (fn [x] (- x)))
from hy.core.language import range
sorted(range(10), key=_hy_anon_fn_1)
Traceback (most recent call last):
File "<input>", line 1, in <module>
NameError: name '_hy_anon_fn_1' is not defined
The function call would appear in the generated AST without being
preceded by the appropriate function definition corresponding to the
anonymous function argument value in the Hy source, causing either a
NameError (as in the example above), or erroneous reuse of whatever
function was already pointed to by the `_hy_anon_fn_` name referenced
in the list of keywords passed to `ast.Call`.
This commit aims to fix the problem by handling it in same way that
the expression/statement gap is bridged many other places in the
compiler, by adding the compiled value of the keyword argument to the
Result object being built during `_compile_collect`, with the
understanding that any Python statements implied by the argument value
will be appropriately preserved therein.
Python 3.5 will have a new commercial-at infix operator with the magic
methods __matmul__, __rmatmul__, and __imatmul__, unused as yet in the
standard library, but intended to represent matrix multiplication in
numerical code; see PEP 465 (https://www.python.org/dev/peps/pep-0465/)
for details. This commit (developed against Python 3.5 alpha 3) brings
support for this operator to Hy when running under Python 3.5 (or,
hypothetically as yet, greater). For Hy under Python <= 3.4, attempting
to use `@` in function-call position currently results in a NameError;
this commit does not change that behavior.
This is intended to resolve#668.
jcrocholl/pep8 (used by flake8, used in Hy's continuous integration
builds) introduced an imports-at-top-of-file check in 1.6.0 and a
line-breaks-around-binary-operators check in 1.6.2. This commit makes
nonfunctional changes to bring the Hy codebase in compliance with this
tool, fixing #764.
This code is heavily, *heavily* based off of Guillermo Vaya
(willyfrog)'s work... instead of defining its own keyword arg though, it
uses the "standard" :kwarg type, which is the main difference from
willyfrog's original branch.
Included tests and some documentation in the tutorial.
Also documented "apply" separately as an example of reproducing
*args and **kwargs.
When (fn) or (defn) does not get an arglist as first/second parameter,
emit a more descriptive error message, rather than an ugly traceback.
Fixes#716.
Reported-by: Joakim Tall
Signed-off-by: Gergely Nagy <algernon@madhouse-project.org>
Our MetaImporter was being inserted at the end of sys.meta_path.
For Python prior to 3.3, this was fine since sys.meta_path
was empty by default. As of the completion of PEP 302 in Py3.3 and
later, there are several importers registered by default. One of
these was trying (and failing) to import simple Hy modules,
resulting in a failure to import anything inside __init__.hy.
This change simply inserts the Hy-specific importer at the front
of the list.
This was noted in issue #620 (great catch @algernon)
A tribute to Portal 2, this function will return an infinite list of the
contents of the AUTHORS file on GitHub master (assuming requests is
installed). Except, the macro does this, the function never gets called,
it is purely there for tribute reasons.
Signed-off-by: Gergely Nagy <algernon@madhouse-project.org>
Python has the keyword.iskeyword method we can leverage for Python
keywords, but we also need to address Hy builtins like 'get' or
'slice'.
And to make behavior compatible with Python 2 or 3, we also make
a special case to prevent assignment to False, True or None as
well as the Hy versions: false, true, null, and nil.
For non-Hy modules, we also check to make sure the symbol is not
part of the compiler. This allows shadow.hy to override "+" but
prevents general use from re-defn-ing 'get' or 'do'.
As noted in #600, Python 3 allows a return inside a generator
method, that raises a StopIteration and passes the return value
inside the 'value' attr of the exception.
To allow this behaviour we simple set 'contains_yield' while compiling
'yield', thus allowing a return statement, but only for Python 3. Then
when compiling the try-except, we check for contains_yield to decide
whether there will be a return.
This allows code like:
(defn gen []
(yield 3)
"goodbye")
to compile in both Py2 and Py3. The return value is simply ignored in
Python 2.
hy2py in Python 2 gives:
def g():
yield 3L
u'goodbye'
while hy2py in Python 3 gives:
def g():
yield 3
return 'goodbye'
Turns out return in yield started in Python 3.3
This new core module allows us to shadow the builtin Python operators so
they may be passed to sequence functions that expect functions:
=> (map / [1 2 3 4 5])
[1.0, 0.5, 0.3333333333333333, 0.25]
Currently, defmacro/g! doesn't respond well when it comes across a
HyObject that doesn't respond to the instance method startswith (e.g.
HyInteger, HyFloat, etc.). This updates defmacro/g! to be a little
safer when searching for the gensyms it needs to create.
This also breaks out the PY3 only tests into their own file. We need to do this because raise from is a syntax error in PY2, so we can't rely on the previous hack of catching a HyCompileError - it would compile fine through Hy and then be a syntax error in Python.
As a result:
* functions such as `nth` should work correctly on iterators;
* `nth` will raise `IndexError` (in a fashion consistent with `get`)
when the index is out of bounds;
* `take`, etc. will raise `ValueError` instead of returning
an ambiguous value if the index is negative;
* `map`, `zip`, `range`, `input`, `filter` work the same way (Py3k one)
on both Python 2 and 3 (see #523 and #331).
Also small DRYing in try handling.
Previously, writing a bare (try (foo)) would invoke Pokemon
exception catching (gotta catch 'em all) instead of the correct
behavior, which is to raise the exception if no handler is provided.
Note that this is a cute feature of Hy, as a `try` with no `except`
is a syntax error. We avoid the syntax error here because we don't
use Python's compiler, which is the only thing that can throw
Syntax Errors. :D
Fixes#555.
The yield-from that existed previously wasn't actually implementing the
full complexity of "yield from":
http://legacy.python.org/dev/peps/pep-0380/#formal-semantics
... this includes passing along errors, and many other things.
Also removes the yield-from backport macro, since it does not seem
possible at present to conditionally build macros.
Thus, there is no longer yield-from on pre-python-3.3 systems.
Includes updated docs and tests to reflect all this.
Example:
(defmain [&rest args]
(print "now we're having a fun time!")
(print args))
Which outputs:
$ hy test.hy
now we're having a fun time!
(['test.hy'],)
Includes documentation and tests.
One would expect the form:
> (defmacro a (&rest b) b)
> (a 1 2)
To return a tuple object but we have no Hy model so it returns a HyList.
Not sure if this is the right thing to do.
When (import) encounters anything but a HySymbol or HyList, raise an
exception, as that is not valid in Hy. Previously, anything other than a
HySymbol or HyList was simply ignored, turning that particular import
into a no-op, which was both wrong and confusing.
Reported-by: Richard Parsons <richard.lee.parsons@gmail.com>
Signed-off-by: Gergely Nagy <algernon@madhouse-project.org>
This function will recursively perform all possible macroexpansions in
the supplied form. Unfortunately, it also traverses into quasiquoted
parts, where it shouldn't, but it is a useful estimation of macro
expansion anyway.
Signed-off-by: Gergely Nagy <algernon@madhouse-project.org>
The hy.contrib.walk module provides a few functions to walk the Hy AST,
and potentially transform it along the way. The main entry point
is (walk), which takes two functions and a form as arguments, and
applies the first (inner) function to each element of the form, building
up a data structure of the same type as the original. Then applies outer
(the second function) to the result.
Two convenience functions are provided: (postwalk) and (prewalk), which
do a depth-first, post/pre-order traversal of the form.
Signed-off-by: Gergely Nagy <algernon@madhouse-project.org>
* hy/core/language.hy: Adding a simple `identity` function that returns
the argument supplied to it
* docs/language/core.rst: Updated docs with identity function
Sometimes it is better to start with the false condition, sometimes that
makes the code clearer. For that, the (if-not) macro, which simply
reverses the order of the condition blocks, can be of great use.
Signed-off-by: Gergely Nagy <algernon@madhouse-project.org>
In the same vein as defmacro-alias, this implements defn-alias /
defun-alias, which does essentially the same thing as defmacro-alias,
but for functions.
Signed-off-by: Gergely Nagy <algernon@balabit.hu>
With this patch, every identifier is split up along dots, each part gets
separately mangled, and then it is all joined back together. This allows
for fun stuff like (.foo? (Foo)), and even more contrived examples.
Signed-off-by: Gergely Nagy <algernon@madhouse-project.org>
The hy2py tool has been very useful for me, but most of the time, it's
only a part of its output that one is interested in. The whole output,
with source code, AST and python code together is one big monstrosity.
So instead of printing all that, lets have a few handy command-line
options to control which part gets printed.
By default, only the generated python source is, as that's what the name
of the tool implies.
Also, don't run it. That's what hy is for.
Signed-off-by: Gergely Nagy <algernon@madhouse-project.org>
There's no reason why one would need to choose between --spy and -i, so
pass down options.spy to run_icommand, and then to HyREPL, so we can
have both.
Signed-off-by: Gergely Nagy <algernon@balabit.hu>
* hy/core/language.hy:
-Added a simple coll? function that checks whether the given argument
is an iterable and not a string,
- Also replaced the check in `flatten` by coll?
* tests/native_tests/core.hy: Tests updated for checking coll?
This cleans up a number of doc warnings, including a bad
underline for zero?
While there, added a nil? function to match up with the
new nil is None.
Also un-hid myself from coreteam.
This version is much simpler.
At the point that the exception is raised, we don't have access to
the actual source, just the current expression. but as the
exception percolates up, we can intercept it, add the source and
the re-raise it.
Then at the final point, in the cmdline handler, we can choose to
let the entire traceback print, or just the simpler, direct error
message.
And even with the full traceback, the last bit is nicely formatted
just like the shorter, simpler message.
The error message is colored if clint is installed, but to avoid
yet another dependency, you get monochrome without clint.
I'm sure there is a better way to do the markup, the current method
is kludgy but works.
I wish there was more shared code between HyTypeError and LexException
but they are kind of different in some fundamental ways.
This doesn't work (yet) with runtime errors generated from Python,
like NameError, but I have a method that can catch NameError and turn it
into a more pleasing output.
Finally, there is no obvious way to raise HyTypeError from pure Hy code,
so methods in core/language.hy throw ugly TypeError/ValueError.
When calling get with more than two arguments, treat the rest as indexes
into the expression from the former. That is, (get foo "bar" "baz")
would translate to foo["bar"]["baz"], and so on.
This fixes#362.
Requested-by: Sean B. Palmer <sean@miscoranda.com>
Signed-off-by: Gergely Nagy <algernon@madhouse-project.org>
* hy/contrib/anaphoric.hy: The following anaphoric macros have been
added
`ap-reject` : Opposite of ap-filter, yields the elements when a `pred`
evaluates to false
`ap-dotimes` : Execute body forms (possibly for side-effects) n times
with `it` bound from 0 to n-1
`ap-first` : return the first element that passes predicate
`ap-last` : return the last element that passes predicate
`ap-reduce`: anaphoric form of reduce that allows `acc` and `it` to
create a function that is applied over the list
* docs/contrib/anaphoric.rst: updated docs to reflect these changes
* tests/__init__.py: updated to explicitly include tests for anaphoric
macros
Apply didn't work on method calls (i.e. `(apply .foo [bar]) broke).
This slipped through because there were no tests of this behavior. I noticed
it while trying to merge the `meth` fixes.
Added first iteration of reader macros
Refactored defmacro and defreader
Added test inn hy/tests/lex/test_lex.py
Added new test in hy/tests/native/tests
Added new test in hy/tests/macros.
changed the error given in the dispatch macro and added some handling for missing symbol and invalid characters
Adding to the manual gensym for macros are 2 new
macros, but very literal from the CL in
letoverlambda.
The first is the (with-gensyms ...) macro that
can generate a small set of syms for a macro. Works
something like:
(defmacro adder2 [A B]
(with-gensyms [a b]
`(let [[~a ~A] [~b ~B]]
(+ ~a ~b))))
and ~a and ~b will be replaced with (gensym "a") and
(gensym "b") respectively.
Then the final macro is a new defmacro that will automatically
replace symbols prefaced with "g!" with a new gensym based on the
rest of the symbol. So in this final version of 'nif':
(defmacro/g! nif4 (expr pos zero neg)
`(let [[~g!result ~expr]]
(cond [(pos? ~g!result) ~pos]
[(zero? ~g!result) ~zero]
[(neg? ~g!result) ~neg])))
all uses of ~g!result will be replaced with (gensym "result").
Simple implementation of gensym in Hy.
Returns a new HySymbol.
Usable in macros like:
(defmacro nif [expr pos zero neg]
(let [[g (gensym)]]
`(let [[~g ~expr]]
(cond [(pos? ~g) ~pos]
[(zero? ~g) ~zero]
[(neg? ~g) ~neg]))))
This addresses all the general comments about (gensym), and doesn't
try to implement "auto-gensym" yet. But clearly the macro approach
instead of the pre-processor approach (as described in the
letoverlambda (http://letoverlambda.com/index.cl/guest/chap3.html#sec_5)
is the way to go
Like other lisps, operators `+` and `*` return their identity values
when called with no arguments. Also with a single operand they return
the operand.
This fixes#372
`hy --spy` fails on hy 0.9.11.
$ hy --spy
hy 0.9.11
=> (type "hy")
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/hy/cmdline.py", line 68, in print_python_code
import astor.codegen
ImportError: No module named astor.codegen
The fancypants Hy award goes to Nick for coming up with the quoted
symbol hack for exports. This broke with foo?, since the export string
needs to be is_foo, but using a quoted string will pick up the change
due to it being a Symbol.
Mad clown love for that, @olasd.
A couple of more macros:
hy> (--each-while [1 2 3 4 5] (< it 3) (print it))
1
2
3
hy>
```--each-while``` continues to evaluate the body form while the
predicate form is true for each element in the list.
```--map-when``` uses a predicate form to determine when to apply the
map form upon the element in the list:
hy> (list (--map-when (even? it) (* it 3) [1 2 3 4]))
[1, 6, 3, 12]
Anaphoric macros reduce the need to specify a lambda by binding a
special name in a form passed as a parameter to the macro. This allows
you to write more concise code:
(= (list (--filter (even? it) [1 2 3 4])) [2 4])
This patch just adds a few basic ones. Other forms that can be
converted to anaphoric versions include reduce, remove, enumerate,
etc.
This gets rid of the dichotomy between bootstrap.py and macros.hy,
by making both files hy modules.
I added some error checking to make the macros more resilient. The
biggest (user-visible) change is the change in cond, which now only
accepts lists as arguments. Tests updated accordingly.
Closes: #176 (whoops, no more bootstrap)
This rounds out the first pass at a set of core functions, adding
some that were not in the first PR.
From here I'm working on a contrib.seq and contrib.io module to
hold less obvious but maybe interesting native functions that can
move to core if desired.
This should also close out issure #150 asking for some core
functions like these.
This will let us use (basic) yield from behavior from Python 2. This
isn't complete, and is low-hanging fruit for others willing to hack
on hy.
I've also changed the macrosystem to allow for proper bootstrapping.
This is similar to how it's done elsewhere in the codebase (stdlib
stuff).
Updated most methods to replace While with For, and added tons of new tests
for things like (cycle []) and lists with None's in them.
thanks @olasd
Add set of new core functions
Add set of new core functions to the stdlib.
Moved the auto-import code from compile_expression to
HySymbol so that "even?' in this style expression will
be found and imported.
(list (filter even? [1 2 3 4 5]))
The core functions are documented in 2 sections, one
for basic functions like (even?..) and (nth ...) and
one for all the sequence functions.
Update: This removes all the caching decorators, misnamed as
'lazy-seq' from the core. All sequence methods now just use
yield to return a generator, so they are Python-lazy
Further refinements of core functions
Cleaned up the docs to use 'iterator' instead of 'generator'
Fixed drop to just return the iterator instead of an extra
yield loop. But also added a test to catch dropping too
many.
This adds real command line handling to 'hyc' for issue #256
This fix catches missing/unreadable files and prints a nice
error message instead of a nasty stack trace when trying to
compile a non-existent file.
Also add this non-existent file check to hy to prevent the
current stack trace from something like "hy foobarbaz" when
"foobarbaz" doesn't exist.
also changes the failure return value to 2 to match Python.
This adds a class to avoid returning when we have a Yieldable
expression contained in the body of the function. This breaks Python
2.x, and ought to break Python 3.x, but doesn't.
We need this fo' context managers, etc.
This commit also has work from @rwtolbert adding new testcases and
fixes for yielded entries behind a while / for.
Summary: This update does away with the scripts in bin and changes
setup.py to use entry_points in cmdline.py for the scripts 'hy' and
'hyc'.
This fixes installing and running on Windows.
The tests are updated to run the 'hy' script produced by setup.py
and not from bin/hy. This is more correct and makes the tox tests
run on both Window and *nix.
For running hy or nosetests directly in the source tree, you do have
to run 'python setup.py develop' first. But since tox runs and builds
dists, all tox tests pass on all platforms.
Also, since there is no built-in readline on Windows, the setup.py
only on Windows requires 'pyreadline' as a replacement.
Switched from optparse to argparse in cmdline.py
Instead of trying to manually separate args meant for
hy from args meant for a hy script, this switches from
optparse to argparse for the CLI.
argparse automatically peels out args meant for hy and leaves
the rest, including the user hy script in options.args.
This fixes the issue @paultag found running "hy foo" where
foo is not a real file. Also added a test that makes sure
trying to run a non-existent script exits instead of dropping
the user into the REPL.
Added argparse as setup.py resource (and removed from tox.ini) as well as removed uses of deprecated setf
Add set of new core functions to the stdlib.
Moved the auto-import code from compile_expression to
HySymbol so that "even?' in this style expression will
be found and imported.
(list (filter even? [1 2 3 4 5]))
The core functions are documented in 2 sections, one
for basic functions like (even?..) and (nth ...) and
one for all the sequence functions.
Update: This removes all the caching decorators, misnamed as
'lazy-seq' from the core. All sequence methods now just use
yield to return a generator, so they are Python-lazy
Further refinements of core functions
Cleaned up the docs to use 'iterator' instead of 'generator'
Fixed drop to just return the iterator instead of an extra
yield loop. But also added a test to catch dropping too
many.
This will let us implement common functions seen in other lisps,
and allow them to be importable, without explicit imports. The goal
is to keep this as small as we can; we don't want too much magic.
I've added `take' and `drop' as examples of what we can do.
A macro is available in the module where it was defined and
in any module that does a require of the defining module.
Only macros defined in hy.core are globally available.
Fixes#181
We actually only generate an ast.Lambda if (lambda) was called, as a lot of code
expect ast.FunctionDefs (most notably with_decorator).
This allows empty lambdas too.
This fixes#165.
This object allows to coerce statements to an expression, if we need to use
them that way, which, with a lisp, is often.
This was collaborative work that has been rebased to make it bisectable.
Helped-by: Paul Tagliamonte <paultag@debian.org>
Helped-by: Julien Danjou <julien@danjou.info>
The new and improved (import) can handle all cases import-as and
import-from did, so drop the latter two from the language. To do this,
the import builtin had to be changed a little: if there's a single
import statement to return, return it as-is, otherwise return a list of
imports.
Signed-off-by: Gergely Nagy <algernon@madhouse-project.org>
This also fixes a bug in the pass optimize missing branch where the code is
something like: [stmt, [], stmt]; in such case we want to filter out [], so
if we end up with [] we can optimize it. This fix is needed otherwise (do)
inside (do) are not properly optimized.
Signed-off-by: Julien Danjou <julien@danjou.info>
With these changes, the import function will become a lot smarter, and
will combine all of import, import-from and import-as in a hyly lispy
syntax:
(import sys os whatever_else)
(import [sys [exit argv]] [os :as real_os]
[whatever_else [some_function :as sf]])
That is, each argument of import can be:
- A plain symbol, which will be imported
- A list, which will be handled specially
If the argument is a list, the first element will always be the module
name to import, the second member can be either of these:
- A list of symbols to import
- The ':as' keyword
- Nothing
If it is the ':as' keyword, the third argument must be an alias. If it
is a list of symbols to import, we'll iterate through that list too. If
any symbol is followed by an ':as' keyword, we'll pick all three, and
treat the third member as an alias. If there is nothing else in the
list, we'll import the module as-is.
All this combined fixes#113.
Signed-off-by: Gergely Nagy <algernon@madhouse-project.org>
Fixes#106
Note: This is implemented by replacing all calls to Python's
builtin "compile" function by calls to hy.importer.compile_,
which adds the "future division" flag. Anyone using "compile"
in future work will have to remember this.
This implements keywords, ":" prefixed symbols that are able to look
themselves up in a collection. They're internally stored as strings that
start with "\ufdd0".
This fixes#22.
Signed-off-by: Gergely Nagy <algernon@madhouse-project.org>
This is a bit tricky, since we'll also have to support `finally' in the end,
I've introduced an Else statement on my own to be able to recognize it.
This fixes#74
Signed-off-by: Julien Danjou <julien@danjou.info>
The (or) function is to be constructed similarly to (and), so refactor
the compile_and_operator function to handle or aswell.
Signed-off-by: Gergely Nagy <algernon@madhouse-project.org>
The function takes at least two arguments, and turns it into a pythonic
and statement, which returns the last True-ish value, or False.
Signed-off-by: Gergely Nagy <algernon@madhouse-project.org>
We can know use any amount and type of bytes to build a HyString, meaning we
can use Unicode and UTF-8 for our function and variables.
Eat that, snake!
Signed-off-by: Julien Danjou <julien@danjou.info>
This allows us to translate lisp argument lists to Python ones.
(defun foo (x y &rest z &optional {foo 3} &aux kwargs))
translates roughly to:
def foo(x, y, *z, foo=3, **kwargs):
pass
The name "lambda list keyword" is a historical artifact from the CLHS
and not optimal (http://clhs.lisp.se/Body/26_glo_l.htm).
The name may change at some point.
