Add set of new core functions

Add set of new core functions to the stdlib.

Moved the auto-import code from compile_expression to
HySymbol so that "even?' in this style expression will
be found and imported.

(list (filter even? [1 2 3 4 5]))

The core functions are documented in 2 sections, one
for basic functions like (even?..) and (nth ...) and
one for all the sequence functions.

Update: This removes all the caching decorators, misnamed as
'lazy-seq' from the core. All sequence methods now just use
yield to return a generator, so they are Python-lazy

Further refinements of core functions

Cleaned up the docs to use 'iterator' instead of 'generator'

Fixed drop to just return the iterator instead of an extra
yield loop. But also added a test to catch dropping too
many.
This commit is contained in:
Bob Tolbert 2013-07-07 20:36:26 -06:00
parent d4aa9b8632
commit 62f1f40830
6 changed files with 929 additions and 12 deletions

552
docs/language/core.rst Normal file
View File

@ -0,0 +1,552 @@
=================
Hy Core
=================
Core Functions
===============
.. _dec-fn:
dec
---
Usage: ``(dec x)``
Return one less than x. Equivalent to ``(- x 1)``.
.. code-block:: clojure
=> (dec 3)
2
=> (dec 0)
-1
=> (dec 12.3)
11.3
.. _even?-fn:
even?
-----
Usage: ``(even? x)``
Return True if x is even.
.. code-block:: clojure
=> (even? 2)
True
=> (even? 13)
False
=> (even? 0)
True
.. _inc-fn:
inc
---
Usage: ``(inc x)``
Return one more than x. Equivalent to ``(+ x 1)``.
.. code-block:: clojure
=> (inc 3)
4
=> (inc 0)
1
=> (inc 12.3)
13.3
.. _instance?-fn:
instance?
---------
Usage: ``(instance? CLASS x)``
Return True if x is an instance of CLASS.
.. code-block:: clojure
=> (instance? float 1.0)
True
=> (instance? int 7)
True
=> (instance? str (str "foo"))
True
=> (defclass TestClass [object])
=> (setv inst (TestClass))
=> (instance? TestClass inst)
True
.. _iterable?-fn:
iterable?
---------
Usage: ``(iterable? x)``
Return True if x is iterable. Iterable objects return a new iterator
when ``(iter x)`` is called. Contrast with :ref:`iterator?-fn`.
.. code-block:: clojure
=> ;; works for strings
=> (iterable? (str "abcde"))
True
=> ;; works for lists
=> (iterable? [1 2 3 4 5])
True
=> ;; works for tuples
=> (iterable? (, 1 2 3))
True
=> ;; works for dicts
=> (iterable? {:a 1 :b 2 :c 3})
True
=> ;; works for iterators/generators
=> (iterable? (repeat 3))
True
.. _iterator?-fn:
iterator?
---------
Usage: ``(iterator? x)``
Return True if x is an iterator. Iterators are objects that return
themselves as an iterator when ``(iter x)`` is called.
Contrast with :ref:`iterable?-fn`.
.. code-block:: clojure
=> ;; doesn't work for a list
=> (iterator? [1 2 3 4 5])
False
=> ;; but we can get an iter from the list
=> (iterator? (iter [1 2 3 4 5]))
True
=> ;; doesn't work for dict
=> (iterator? {:a 1 :b 2 :c 3})
False
=> ;; create an iterator from the dict
=> (iterator? (iter {:a 1 :b 2 :c 3}))
True
.. _neg?-fn:
neg?
----
Usage: ``(neg? x)``
Return True if x is less than zero (0).
.. code-block:: clojure
=> (neg? -2)
True
=> (neg? 3)
False
=> (neg? 0)
False
.. _none?-fn:
none?
-----
Usage: ``(none? x)``
Return True if x is None.
.. code-block:: clojure
=> (none? None)
True
=> (none? 0)
False
=> (setf x None)
=> (none? x)
True
=> ;; list.append always returns None
=> (none? (.append [1 2 3] 4))
True
.. _nth-fn:
nth
---
Usage: ``(nth coll n)``
Return the `nth` item in a collection, counting from 0. Unlike
``get``, ``nth`` works on both iterators and iterables. Returns ``None``
if the `n` is outside the range of `coll`.
.. code-block:: clojure
=> (nth [1 2 4 7] 1)
2
=> (nth [1 2 4 7] 3)
7
=> (none? (nth [1 2 4 7] 5))
True
=> (nth (take 3 (drop 2 [1 2 3 4 5 6])) 2))
5
.. _odd?-fn:
odd?
----
Usage: ``(odd? x)``
Return True if x is odd.
.. code-block:: clojure
=> (odd? 13)
True
=> (odd? 2)
False
=> (odd? 0)
False
.. _pos?-fn:
pos?
----
Usage: ``(pos? x)``
Return True if x is greater than zero (0).
.. code-block:: clojure
=> (pos? 3)
True
=> (pos? -2)
False
=> (pos? 0)
False
Sequence Functions
=======================
Sequence functions can either create or operate on a potentially
infinite sequence without requiring the sequence be fully realized in
a list or similar container. They do this by returning a Python
iterator.
We can use the canonical infinite Fibonacci number generator
as an example of how to use some of these functions.
.. code-block:: clojure
(defn fib []
(setf a 0)
(setf b 1)
(while true
(yield a)
(setf (, a b) (, b (+ a b)))))
Note the ``(while true ...)`` loop. If we run this in the REPL,
.. code-block:: clojure
=> (fib)
<generator object fib at 0x101e642d0>
Calling the function only returns an iterator, but does no
work until we consume it. Trying something like this is not recommend as
the infinite loop will run until it consumes all available RAM, or
in this case until I killed it.
.. code-block:: clojure
=> (list (fib))
[1] 91474 killed hy
To get the first 10 Fibonacci numbers, use :ref:`take-fn`. Note that
:ref:`take-fn` also returns a generator, so I create a list from it.
.. code-block:: clojure
=> (list (take 10 (fib)))
[0, 1, 1, 2, 3, 5, 8, 13, 21, 34]
To get the Fibonacci number at index 9, (starting from 0):
.. code-block:: clojure
=> (nth (fib) 9)
34
.. _cycle-fn:
cycle
------
Usage: ``(cycle coll)``
Return an infinite iterator of the members of coll.
.. code-block:: clj
=> (list (take 7 (cycle [1 2 3])))
[1, 2, 3, 1, 2, 3, 1]
=> (list (take 2 (cycle [1 2 3])))
[1, 2]
.. _distinct-fn:
distinct
--------
Usage: ``(distinct coll)``
Returns an iterator containing only the unique members in ``coll``.
.. code-block:: clojure
=> (list (distinct [ 1 2 3 4 3 5 2 ]))
[1, 2, 3, 4, 5]
=> (list (distinct []))
[]
=> (list (distinct (iter [ 1 2 3 4 3 5 2 ])))
[1, 2, 3, 4, 5]
.. _drop-fn:
drop
----
Usage: ``(drop n coll)``
Return an iterator, skipping the first ``n`` members of ``coll``
.. code-block:: clojure
=> (list (drop 2 [1 2 3 4 5]))
[3, 4, 5]
=> (list (drop 4 [1 2 3 4 5]))
[5]
=> (list (drop 0 [1 2 3 4 5]))
[1, 2, 3, 4, 5]
=> (list (drop 6 [1 2 3 4 5]))
[]
.. _filter-fn:
filter
------
Usage: ``(filter pred coll)``
Return an iterator for all items in ``coll`` that pass the predicate ``pred``.
See also :ref:`remove-fn`.
.. code-block:: clojure
=> (list (filter pos? [1 2 3 -4 5 -7]))
[1, 2, 3, 5]
=> (list (filter even? [1 2 3 -4 5 -7]))
[2, -4]
.. _iterate-fn:
iterate
-------
Usage: ``(iterate fn x)``
Return an iterator of `x`, `fn(x)`, `fn(fn(x))`.
.. code-block:: clojure
=> (list (take 5 (iterate inc 5)))
[5, 6, 7, 8, 9]
=> (list (take 5 (iterate (fn [x] (* x x)) 5)))
[5, 25, 625, 390625, 152587890625]
.. _remove-fn:
remove
------
Usage: ``(remove pred coll)``
Return an iterator from ``coll`` with elements that pass the
predicate, ``pred``, removed.
See also :ref:`filter-fn`.
.. code-block:: clojure
=> (list (remove odd? [1 2 3 4 5 6 7]))
[2, 4, 6]
=> (list (remove pos? [1 2 3 4 5 6 7]))
[]
=> (list (remove neg? [1 2 3 4 5 6 7]))
[1, 2, 3, 4, 5, 6, 7]
.. _repeat-fn:
repeat
------
Usage: ``(repeat x)``
Return an iterator (infinite) of ``x``.
.. code-block:: clojure
=> (list (take 6 (repeat "s")))
[u's', u's', u's', u's', u's', u's']
.. _repeatedly-fn:
repeatedly
----------
Usage: ``(repeatedly fn)``
Return an iterator by calling ``fn`` repeatedly.
.. code-block:: clojure
=> (import [random [randint]])
=> (list (take 5 (repeatedly (fn [] (randint 0 10)))))
[6, 2, 0, 6, 7]
.. _take-fn:
take
----
Usage: ``(take n coll)``
Return an iterator containing the first ``n`` members of ``coll``.
.. code-block:: clojure
=> (list (take 3 [1 2 3 4 5]))
[1, 2, 3]
=> (list (take 4 (repeat "s")))
[u's', u's', u's', u's']
=> (list (take 0 (repeat "s")))
[]
.. _take-nth-fn:
take-nth
--------
Usage: ``(take-nth n coll)``
Return an iterator containing every ``nth`` member of ``coll``.
.. code-block:: clojure
=> (list (take-nth 2 [1 2 3 4 5 6 7]))
[1, 3, 5, 7]
=> (list (take-nth 3 [1 2 3 4 5 6 7]))
[1, 4, 7]
=> (list (take-nth 4 [1 2 3 4 5 6 7]))
[1, 5]
=> (list (take-nth 10 [1 2 3 4 5 6 7]))
[1]
.. _take-while-fn:
take-while
----------
Usage: ``(take-while pred coll)``
Return an iterator from ``coll`` as long as predicate, ``pred`` returns True.
.. code-block:: clojure
=> (list (take-while pos? [ 1 2 3 -4 5]))
[1, 2, 3]
=> (list (take-while neg? [ -4 -3 1 2 5]))
[-4, -3]
=> (list (take-while neg? [ 1 2 3 -4 5]))
[]

View File

@ -8,4 +8,5 @@ Contents:
:maxdepth: 3 :maxdepth: 3
api api
core
internals internals

View File

@ -1445,9 +1445,6 @@ class HyASTCompiler(object):
if ret: if ret:
return ret return ret
if fn in _stdlib:
self.imports[_stdlib[fn]].add(fn)
if fn.startswith("."): if fn.startswith("."):
# (.split "test test") -> "test test".split() # (.split "test test") -> "test test".split()
@ -1757,6 +1754,9 @@ class HyASTCompiler(object):
) )
return ret return ret
if symbol in _stdlib:
self.imports[_stdlib[symbol]].add(symbol)
return ast.Name(id=ast_str(symbol), return ast.Name(id=ast_str(symbol),
arg=ast_str(symbol), arg=ast_str(symbol),
ctx=ast.Load(), ctx=ast.Load(),

View File

@ -3,20 +3,146 @@
;;;; ;;;;
(defn cycle [coll]
"Yield an infinite repetition of the items in coll"
(while true
(for [x coll]
(yield x))))
(defn dec [n]
"Decrement n by 1"
(- n 1))
(defn distinct [coll]
"Return a generator from the original collection with duplicates
removed"
(let [ [seen [] ]
[citer (iter coll)]
[val (next citer)] ]
(while (not (none? val))
(do
(if (not_in val seen)
(do
(yield val)
(.append seen val)))
(setv val (next citer))))))
(defn drop [count coll]
"Drop `count` elements from `coll` and yield back the rest"
(let [ [citer (iter coll)] ]
(try (for [i (range count)]
(next citer))
(catch [StopIteration]))
citer))
(defn even? [n]
"Return true if n is an even number"
(= (% n 2) 0))
(defn filter [pred coll]
"Return all elements from coll that pass filter"
(let [ [citer (iter coll)] [val (next citer)] ]
(while (not (none? val))
(if (pred val)
(yield val))
(setv val (next citer)))))
(defn inc [n]
"Increment n by 1"
(+ n 1))
(defn instance? [klass x]
(isinstance x klass))
(defn iterable? [x]
"Return true if x is iterable"
(try (do (iter x) true)
(catch [Exception] false)))
(defn iterate [f x]
(setv val x)
(while true
(yield val)
(setv val (f val))))
(defn iterator? [x]
"Return true if x is an iterator"
(try (= x (iter x))
(catch [TypeError] false)))
(defn neg? [n]
"Return true if n is < 0"
(< n 0))
(defn none? [x]
"Return true if x is None"
(is x None))
(defn nth [coll index]
"Return nth item in collection or sequence, counting from 0"
(if (not (neg? index))
(if (iterable? coll)
(try (first (list (take 1 (drop index coll))))
(catch [IndexError] None))
(try (get coll index)
(catch [IndexError] None)))
None))
(defn odd? [n]
"Return true if n is an odd number"
(= (% n 2) 1))
(defn pos? [n]
"Return true if n is > 0"
(> n 0))
(defn remove [pred coll]
"Return coll with elements removed that pass `pred`"
(let [ [citer (iter coll)] [val (next citer)] ]
(while (not (none? val))
(if (not (pred val))
(yield val))
(setv val (next citer)))))
(defn repeat [x &optional n]
"Yield x forever or optionally n times"
(if (none? n)
(setv dispatch (fn [] (while true (yield x))))
(setv dispatch (fn [] (for [_ (range n)] (yield x)))))
(dispatch))
(defn repeatedly [func]
"Yield result of running func repeatedly"
(while true
(yield (func))))
(defn take [count what] (defn take [count what]
"Take `count` elements from `what`, or the whole set if the total "Take `count` elements from `what`, or the whole set if the total
number of entries in `what` is less than `count`." number of entries in `what` is less than `count`."
(setv what (iter what)) (setv what (iter what))
(for [i (range count)] (for [i (range count)]
(yield (next what)))) (yield (next what))))
(defn take-nth [n coll]
"Return every nth member of coll
raises ValueError for (not (pos? n))"
(if (pos? n)
(let [ [citer (iter coll)] [val (next citer)] ]
(while (not (none? val))
(yield val)
(for [_ (range (dec n))]
(next citer))
(setv val (next citer))))
(raise (ValueError "n must be positive"))))
(defn drop [count coll] (defn take-while [pred coll]
"Drop `count` elements from `coll` and return the iter" "Take all elements while `pred` is true"
(let [ [citer (iter coll)] ] (let [ [citer (iter coll)] [val (next citer)] ]
(for [i (range count)] (while (pred val)
(next citer)) (yield val)
citer)) (setv val (next citer)))))
(def *exports* ["cycle" "dec" "distinct" "drop" "even?" "filter" "inc"
(def *exports* ["take" "drop"]) "instance?" "iterable?" "iterate" "iterator?" "neg?"
"none?" "nth" "odd?" "pos?" "remove" "repeat" "repeatedly"
"take" "take_nth" "take_while"])

View File

@ -10,3 +10,4 @@ from .native_tests.language import * # noqa
from .native_tests.unless import * # noqa from .native_tests.unless import * # noqa
from .native_tests.when import * # noqa from .native_tests.when import * # noqa
from .native_tests.with_decorator import * # noqa from .native_tests.with_decorator import * # noqa
from .native_tests.core import * # noqa

237
tests/native_tests/core.hy Normal file
View File

@ -0,0 +1,237 @@
(import time)
;;;; some simple helpers
(defn assert-true [x]
(assert (= True x)))
(defn assert-false [x]
(assert (= False x)))
(defn assert-equal [x y]
(assert (= x y)))
(defn test-cycle []
"NATIVE: testing cycle"
(assert-equal (list (take 7 (cycle [1 2 3]))) [1 2 3 1 2 3 1])
(assert-equal (list (take 2 (cycle [1 2 3]))) [1 2]))
(defn test-dec []
"NATIVE: testing the dec function"
(assert-equal 0 (dec 1))
(assert-equal -1 (dec 0))
(assert-equal 0 (dec (dec 2))))
(defn test-distinct []
"NATIVE: testing the distinct function"
(setv res (list (distinct [ 1 2 3 4 3 5 2 ])))
(assert-equal res [1 2 3 4 5])
;; distinct of an empty list should be []
(setv res (list (distinct [])))
(assert-equal res [])
;; now with an iter
(setv test_iter (iter [1 2 3 4 3 5 2]))
(setv res (list (distinct test_iter)))
(assert-equal res [1 2 3 4 5]))
(defn test-drop []
"NATIVE: testing drop function"
(setv res (list (drop 2 [1 2 3 4 5])))
(assert-equal res [ 3 4 5])
(setv res (list (drop 3 (iter [1 2 3 4 5]))))
(assert-equal res [ 4 5])
(setv res (list (drop 0 [1 2 3 4 5])))
(assert-equal res [1 2 3 4 5])
(setv res (list (drop -1 [1 2 3 4 5])))
(assert-equal res [1 2 3 4 5])
(setv res (list (drop 6 (iter [1 2 3 4 5]))))
(assert-equal res [])
(setv res (list (take 5 (drop 2 (iterate inc 0)))))
(assert-equal res [2 3 4 5 6]))
(defn test-even []
"NATIVE: testing the even? function"
(assert-true (even? -2))
(assert-false (even? 1))
(assert-true (even? 0)))
(defn test-filter []
"NATIVE: testing the filter function"
(setv res (list (filter pos? [ 1 2 3 -4 5])))
(assert-equal res [ 1 2 3 5 ])
;; test with iter
(setv res (list (filter pos? (iter [ 1 2 3 -4 5 -6]))))
(assert-equal res [ 1 2 3 5])
(setv res (list (filter neg? [ -1 -4 5 3 4])))
(assert-false (= res [1 2])))
(defn test-inc []
"NATIVE: testing the inc function"
(assert-equal 3 (inc 2))
(assert-equal 0 (inc -1)))
(defn test-instance []
"NATIVE: testing instance? function"
(defclass Foo [object])
(defclass Foo2 [object])
(defclass Foo3 [Foo])
(setv foo (Foo))
(setv foo3 (Foo3))
(assert-true (instance? Foo foo))
(assert-false (instance? Foo2 foo))
(assert-true (instance? Foo foo3))
(assert-true (instance? float 1.0))
(assert-true (instance? int 3))
(assert-true (instance? str (str "hello"))))
(defn test-iterable []
"NATIVE: testing iterable? function"
;; should work for a string
(setv s (str "abcde"))
(assert-true (iterable? s))
;; should work for unicode
(setv u "hello")
(assert-true (iterable? u))
(assert-true (iterable? (iter u)))
;; should work for a list
(setv l [1 2 3 4])
(assert-true (iterable? l))
(assert-true (iterable? (iter l)))
;; should work for a dict
(setv d {:a 1 :b 2 :c 3})
(assert-true (iterable? d))
;; should work for a tuple?
(setv t (, 1 2 3 4))
(assert-true (iterable? t))
;; should work for a generator
(assert-true (iterable? (repeat 3)))
;; shouldn't work for an int
(assert-false (iterable? 5)))
(defn test-iterate []
"NATIVE: testing the iterate function"
(setv res (list (take 5 (iterate inc 5))))
(assert-equal res [5 6 7 8 9])
(setv res (list (take 3 (iterate (fn [x] (* x x)) 5))))
(assert-equal res [5 25 625])
(setv f (take 4 (iterate inc 5)))
(assert-equal (list f) [5 6 7 8]))
(defn test-iterator []
"NATIVE: testing iterator? function"
;; should not work for a list
(setv l [1 2 3 4])
(assert-false (iterator? l))
;; should work for an iter over a list
(setv i (iter [1 2 3 4]))
(assert-true (iterator? i))
;; should not work for a dict
(setv d {:a 1 :b 2 :c 3})
(assert-false (iterator? d))
;; should not work for a tuple?
(setv t (, 1 2 3 4))
(assert-false (iterator? t))
;; should work for a generator
(assert-true (iterator? (repeat 3)))
;; should not work for an int
(assert-false (iterator? 5)))
(defn test-neg []
"NATIVE: testing the neg? function"
(assert-true (neg? -2))
(assert-false (neg? 1))
(assert-false (neg? 0)))
(defn test-none []
"NATIVE: testing for `is None`"
(assert-true (none? None))
(setv f None)
(assert-true (none? f))
(assert-false (none? 0))
(assert-false (none? "")))
(defn test-nth []
"NATIVE: testing the nth function"
(assert-equal 2 (nth [1 2 4 7] 1))
(assert-equal 7 (nth [1 2 4 7] 3))
(assert-true (none? (nth [1 2 4 7] 5)))
(assert-true (none? (nth [1 2 4 7] -1)))
;; now for iterators
(assert-equal 2 (nth (iter [1 2 4 7]) 1))
(assert-equal 7 (nth (iter [1 2 4 7]) 3))
(assert-true (none? (nth (iter [1 2 4 7]) -1)))
(assert-equal 5 (nth (take 3 (drop 2 [1 2 3 4 5 6])) 2)))
(defn test-odd []
"NATIVE: testing the odd? function"
(assert-true (odd? -3))
(assert-true (odd? 1))
(assert-false (odd? 0)))
(defn test-pos []
"NATIVE: testing the pos? function"
(assert-true (pos? 2))
(assert-false (pos? -1))
(assert-false (pos? 0)))
(defn test-remove []
"NATIVE: testing the remove function"
(setv r (list (remove odd? [1 2 3 4 5 6 7])))
(assert-equal r [2 4 6])
(assert-equal (list (remove even? [1 2 3 4 5])) [1 3 5])
(assert-equal (list (remove neg? [1 2 3 4 5])) [1 2 3 4 5])
(assert-equal (list (remove pos? [1 2 3 4 5])) []))
(defn test-repeat []
"NATIVE: testing repeat"
(setv r (repeat 10))
(assert-equal (list (take 5 r)) [10 10 10 10 10])
(assert-equal (list (take 4 r)) [10 10 10 10])
(setv r (repeat 10 3))
(assert-equal (list r) [10 10 10]))
(defn test-repeatedly []
"NATIVE: testing repeatedly"
(setv r (repeatedly (fn [] (inc 4))))
(assert-equal (list (take 5 r)) [5 5 5 5 5])
(assert-equal (list (take 4 r)) [5 5 5 5])
(assert-equal (list (take 6 r)) [5 5 5 5 5 5]))
(defn test-take []
"NATIVE: testing the take function"
(setv res (list (take 3 [1 2 3 4 5])))
(assert-equal res [1 2 3])
(setv res (list (take 4 (repeat "s"))))
(assert-equal res ["s" "s" "s" "s"])
(setv res (list (take 0 (repeat "s"))))
(assert-equal res [])
(setv res (list (take -1 (repeat "s"))))
(assert-equal res []))
(defn test-take-nth []
"NATIVE: testing the take-nth function"
(setv res (list (take-nth 2 [1 2 3 4 5 6 7])))
(assert-equal res [1 3 5 7])
(setv res (list (take-nth 3 [1 2 3 4 5 6 7])))
(assert-equal res [1 4 7])
(setv res (list (take-nth 4 [1 2 3 4 5 6 7])))
(assert-equal res [1 5])
(setv res (list (take-nth 5 [1 2 3 4 5 6 7])))
(assert-equal res [1 6])
(setv res (list (take-nth 6 [1 2 3 4 5 6 7])))
(assert-equal res [1 7])
(setv res (list (take-nth 7 [1 2 3 4 5 6 7])))
(assert-equal res [1])
;; using 0 should raise ValueError
(let [[passed false]]
(try
(setv res (list (take-nth 0 [1 2 3 4 5 6 7])))
(catch [ValueError] (setv passed true)))
(assert passed)))
(defn test-take-while []
"NATIVE: testing the take-while function"
(setv res (list (take-while pos? [ 1 2 3 -4 5])))
(assert-equal res [ 1 2 3 ])
(setv res (list (take-while neg? [ -1 -4 5 3 4])))
(assert-false (= res [1 2])))