This commit refactors the exception/error classes and their handling.
It also retains Hy source strings and their originating file information, when
available, all throughout the core parser and compiler functions.
As well, with these changes, calling code is no longer responsible for providing
source and file details to exceptions,
Closeshylang/hy#657.
Functions and variables relating to compilation and parsing have been moved to
`compiler.py` and `lex/__init__.py`, respectively. Those functions are
- `hy_parse` from `hy.importer` to `hy.lex`
- `hy_eval`, `ast_compile`, and `calling_module` from `hy.importer` to
`hy.compiler`
Closeshylang/hy#1695.
Newly imported modules with compile and/or run-time errors were not being
removed from `sys.modules`. This commit modifies the Python 2.7 loader so that
it follows Python's failed-initial-import logic and removes the module from
`sys.modules`.
This change a Hy-preferring `runhy` that is used by cmdline Hy. Standard
`runpy` is still patched so that it can run `.hy` files, but the default
behaviour for unknown filetypes is preserved (i.e. assume they are Python
source).
Closeshylang/hy#1677.
Python 3.x is patched in a way that integrates `.hy` source files into
Pythons default `importlib` machinery. In Python 2.7, a PEP-302 "importer"
and "loader" is implemented according to the standard `import` logic (via
`pkgutil` and later pure-Python `imp` package code).
In both cases, the entry-point for the loaders is through `sys.path_hooks` only.
As well, the import semantics have been updated all throughout to utilize
`importlib` and follow aspects of PEP-420. This, along with some light
patches, should allow for basic use of `runpy`, `py_compile` and `reload`.
In all cases, if a `.hy` file is shadowed by a `.py`, Hy will silently use
`.hy`.
Auto-promotion now occurs in only two cases: when we start the compiler and when we expand a macro. It's fully recursive so even a non-model nested in a model will be promoted.
This change fixes some regressions induced by the stricter type checks of the pattern-matching compiler.
Importing or executing a Hy file now loads the byte-compiled version if it exists and is up to date, and if not, the source is byte-compiled after it's parsed.
This change can speed up Hy a lot. Here are some examples comparing run times of the current master (491b474e) to this commit, on my laptop with Python 3.6:
- `nosetests --exclude='test_bin'` goes from 3.8 s to 0.7 s (a 5-fold speedup)
- `hy -c '(print "hello world")` goes from 0.47 s to 0.20 s (a 2-fold speedup)
- Rogue TV's startup goes from 3.6 s to 0.4 s (a 9-fold speedup)
Accompanying changes include:
- `setup.py` now creates and installs bytecode for `hy.core`, `hy.contrib`, and `hy.extra`.
- The `hyc` command under Python 3 now creates bytecode in `__pycache__`, as usual for Python 3, instead of putting the `.pyc` right next to the source file like Python 2 does.
I've removed a test of `hy.extra.anaphoric.a-if` that triggers #1268 when the test file is byte-compiled and then hits some weird `macroexpand` bug or something when I try to work around that—Nose crashes when trying to produce an error message, and I can't seem to replicate the bug without Nose.
Previously, Hy scripts that raised a subclass of IOError would be caught
by code intended to deal with IOErrors raised when Hy couldn't import
the script itself, resulting in either a misleading "Can't open file"
error message, or a misleading TypeError traceback from not being able
to format the "Can't open file" message (for IOErrors for which the
errno attribute was None). This commit (a straightforward implementation
of the idea proposed by @slimetree in #714) introduces a new HyIOError
class, raises that when `import_file_to_hst` can't open the file path,
and catches it in the `cmdline_handler`.
This is believed to fix#513, #714, and #727.
Currently '.replace' method is used to replace hy objects. This is not
safe when we are not sure if the 'obj' in 'obj.replace(other)' is an
instance of HyObject.
In these cases, we can use function 'replace_hy_obj(obj, other)'
instead. This function will try to wrap 'obj' if it's not an instance of
HyObject.
This also means that we need a wrapping function in hy.models'. Hence I
moved the '_wrap_value' function from hy.macros into hy.models. To avoid
circular importing, the wrapper functions are provided individually by
each model type's own file.
Our MetaImporter was being inserted at the end of sys.meta_path.
For Python prior to 3.3, this was fine since sys.meta_path
was empty by default. As of the completion of PEP 302 in Py3.3 and
later, there are several importers registered by default. One of
these was trying (and failing) to import simple Hy modules,
resulting in a failure to import anything inside __init__.hy.
This change simply inserts the Hy-specific importer at the front
of the list.
This was noted in issue #620 (great catch @algernon)
This version is much simpler.
At the point that the exception is raised, we don't have access to
the actual source, just the current expression. but as the
exception percolates up, we can intercept it, add the source and
the re-raise it.
Then at the final point, in the cmdline handler, we can choose to
let the entire traceback print, or just the simpler, direct error
message.
And even with the full traceback, the last bit is nicely formatted
just like the shorter, simpler message.
The error message is colored if clint is installed, but to avoid
yet another dependency, you get monochrome without clint.
I'm sure there is a better way to do the markup, the current method
is kludgy but works.
I wish there was more shared code between HyTypeError and LexException
but they are kind of different in some fundamental ways.
This doesn't work (yet) with runtime errors generated from Python,
like NameError, but I have a method that can catch NameError and turn it
into a more pleasing output.
Finally, there is no obvious way to raise HyTypeError from pure Hy code,
so methods in core/language.hy throw ugly TypeError/ValueError.
There was a couple of duplicate imports and type checkings in the
codebase. So I added a new module to unify all Python 2 and 3
compatibility codes.
Also, this is a somewhat common pattern in Python. See Jinja2 for
example:
https://github.com/mitsuhiko/jinja2/blob/master/jinja2/_compat.py
Kodi Arfer