Functions and variables relating to compilation and parsing have been moved to
`compiler.py` and `lex/__init__.py`, respectively. Those functions are
- `hy_parse` from `hy.importer` to `hy.lex`
- `hy_eval`, `ast_compile`, and `calling_module` from `hy.importer` to
`hy.compiler`
Closeshylang/hy#1695.
Fixes:
>>> from hy._compat import isidentifier
>>> isidentifier(u" 0\n 0")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "hy/_compat.py", line 47, in isidentifier
tokens = list(T.generate_tokens(StringIO(x).readline))
File "/usr/lib/python2.7/tokenize.py", line 374, in generate_tokens
("<tokenize>", lnum, pos, line))
File "<tokenize>", line 2
0
^
IndentationError: unindent does not match any outer indentation level
This commit adds just enough namespacing to resolve a macro first in the macro's
defining module's namespace (i.e. the module assigned to the `HyASTCompiler`),
then in the namespace/module it's evaluated in. Namespacing is accomplished by
adding a `module` attribute to `HySymbol`, so that `HyExpression`s can be
checked for this definition namespace attribute and their car symbol resolved
per the above.
As well, a couple tests have been added that cover
- the loading of module-level macros
- e.g. that only macros defined in the `require`d module are added
- the AST generated for `require`
- using macros loaded from modules imported via bytecode
- the non-local macro namespace resolution described above
- a `require`d macro that uses a macro `require` exclusively in its
module-level namespace
- and that (second-degree `require`d) macros can reference variables within
their module-level namespaces.
Closeshylang/hy#1268, closeshylang/hy#1650, closeshylang/hy#1416.
Python 3.x is patched in a way that integrates `.hy` source files into
Pythons default `importlib` machinery. In Python 2.7, a PEP-302 "importer"
and "loader" is implemented according to the standard `import` logic (via
`pkgutil` and later pure-Python `imp` package code).
In both cases, the entry-point for the loaders is through `sys.path_hooks` only.
As well, the import semantics have been updated all throughout to utilize
`importlib` and follow aspects of PEP-420. This, along with some light
patches, should allow for basic use of `runpy`, `py_compile` and `reload`.
In all cases, if a `.hy` file is shadowed by a `.py`, Hy will silently use
`.hy`.
Auto-promotion now occurs in only two cases: when we start the compiler and when we expand a macro. It's fully recursive so even a non-model nested in a model will be promoted.
This change fixes some regressions induced by the stricter type checks of the pattern-matching compiler.
It's rarely useful, because it catches all exceptions, but it doesn't let you do anything other than return `None`. You can still get the same effect with `(except [])`.
HyKeywords are no longer an instances of string with a particular
prefix, but a completely separate object.
This means keywords no longer trip isinstance str checks, adding a
little bit of type safety to the compiler.
It also means that HyKeywords evaluate to themselves.
Closes#1352
Python Class AST expects a body which is a list of ast.Expr. Force
every entry to be stored as a statement. This means we'll preserve
print statements.
Python also doesn't construct docstrings in classes by setting a
__doc__ attribute, it does it by inspecting the first ast.Expr node of
the class. But this means we can remove the special handler for it.
