Allow numpy arrays to be passed by value into tasks (and inlined in the task spec). (#1816)

* Allow numpy arrays and larger objects to be passed by value in task specifications.

* Fix bug.

* Fix bug. Inline all bug numpy object arrays.

* Increase size limit for inlining args in task spec.

* Give numpy init different signatures in Python 2 and Python 3.

* Simplify code.

* Fix test.

* Use import_array1 instead of import_array.
This commit is contained in:
Robert Nishihara
2018-04-15 20:36:01 -07:00
committed by Philipp Moritz
parent 6bd944ae0d
commit 6ca2c2a609
7 changed files with 41 additions and 13 deletions
+6 -3
View File
@@ -30,7 +30,9 @@ def random_task_id():
BASE_SIMPLE_OBJECTS = [
0, 1, 100000, 0.0, 0.5, 0.9, 100000.1, (), [], {}, "", 990 * "h", u"",
990 * u"h"
990 * u"h",
np.ones(3),
np.array([True, False]), None, True, False
]
if sys.version_info < (3, 0):
@@ -60,8 +62,9 @@ class Foo(object):
BASE_COMPLEX_OBJECTS = [
999 * "h", 999 * u"h", lst,
Foo(), 10 * [10 * [10 * [1]]]
15000 * "h", 15000 * u"h", lst,
Foo(), 100 * [100 * [10 * [1]]],
np.array([Foo()])
]
LIST_COMPLEX_OBJECTS = [[obj] for obj in BASE_COMPLEX_OBJECTS]