Functions and variables relating to compilation and parsing have been moved to
`compiler.py` and `lex/__init__.py`, respectively. Those functions are
- `hy_parse` from `hy.importer` to `hy.lex`
- `hy_eval`, `ast_compile`, and `calling_module` from `hy.importer` to
`hy.compiler`
Closeshylang/hy#1695.
Fixes:
>>> from hy._compat import isidentifier
>>> isidentifier(u" 0\n 0")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "hy/_compat.py", line 47, in isidentifier
tokens = list(T.generate_tokens(StringIO(x).readline))
File "/usr/lib/python2.7/tokenize.py", line 374, in generate_tokens
("<tokenize>", lnum, pos, line))
File "<tokenize>", line 2
0
^
IndentationError: unindent does not match any outer indentation level
This commit adds just enough namespacing to resolve a macro first in the macro's
defining module's namespace (i.e. the module assigned to the `HyASTCompiler`),
then in the namespace/module it's evaluated in. Namespacing is accomplished by
adding a `module` attribute to `HySymbol`, so that `HyExpression`s can be
checked for this definition namespace attribute and their car symbol resolved
per the above.
As well, a couple tests have been added that cover
- the loading of module-level macros
- e.g. that only macros defined in the `require`d module are added
- the AST generated for `require`
- using macros loaded from modules imported via bytecode
- the non-local macro namespace resolution described above
- a `require`d macro that uses a macro `require` exclusively in its
module-level namespace
- and that (second-degree `require`d) macros can reference variables within
their module-level namespaces.
Closeshylang/hy#1268, closeshylang/hy#1650, closeshylang/hy#1416.
Newly imported modules with compile and/or run-time errors were not being
removed from `sys.modules`. This commit modifies the Python 2.7 loader so that
it follows Python's failed-initial-import logic and removes the module from
`sys.modules`.
This change a Hy-preferring `runhy` that is used by cmdline Hy. Standard
`runpy` is still patched so that it can run `.hy` files, but the default
behaviour for unknown filetypes is preserved (i.e. assume they are Python
source).
Closeshylang/hy#1677.
Python 3.x is patched in a way that integrates `.hy` source files into
Pythons default `importlib` machinery. In Python 2.7, a PEP-302 "importer"
and "loader" is implemented according to the standard `import` logic (via
`pkgutil` and later pure-Python `imp` package code).
In both cases, the entry-point for the loaders is through `sys.path_hooks` only.
As well, the import semantics have been updated all throughout to utilize
`importlib` and follow aspects of PEP-420. This, along with some light
patches, should allow for basic use of `runpy`, `py_compile` and `reload`.
In all cases, if a `.hy` file is shadowed by a `.py`, Hy will silently use
`.hy`.
Auto-promotion now occurs in only two cases: when we start the compiler and when we expand a macro. It's fully recursive so even a non-model nested in a model will be promoted.
This change fixes some regressions induced by the stricter type checks of the pattern-matching compiler.
