* Cross language serialization for Java and Python
* Use strict types when Python serializing
* Handle recursive objects in Python; Pin msgpack >= 0.6.0, < 1.0.0
* Disable gc for optimizing msgpack loads
* Fix merge bug
* Java call Python use returnType; Fix ClassLoaderTest
* Fix RayMethodsTest
* Fix checkstyle
* Fix lint
* prepare_args raises exception if try to transfer a non-deserializable object to another language
* Fix CrossLanguageInvocationTest.java, Python msgpack treat float as double
* Minor fixes
* Fix compile error on linux
* Fix lint in java/BUILD.bazel
* Fix test_failure
* Fix lint
* Class<?> to Class<T>; Refine metadata bytes.
* Rename FST to Fst; sort java dependencies
* Change Class<?>[] to Optional<Class<?>>; sort requirements in setup.py
* Improve CrossLanguageInvocationTest
* Refactor MessagePackSerializer.java
* Refactor MessagePackSerializer.java; Refine CrossLanguageInvocationTest.java
* Remove unnecessary dependencies for Java; Add getReturnType() for RayFunction in Java
* Fix bug
* Remove custom cross language type support
* Replace Serializer.Meta with MutableBoolean
* Remove @SuppressWarnings support from checkstyle.xml; Add null test in CrossLanguageInvocationTest.java
* Refine MessagePackSerializer.pack
* Ray.get support RayObject as input
* Improve comments and error info
* Remove classLoader argument from serializer
* Separate msgpack from pickle5 in Python
* Pair<byte[], MutableBoolean> to Pair<byte[], Boolean>
* Remove public static <T> T get(RayObject<T> object), use RayObject.get() instead
* Refine test
* small fixes
Co-authored-by: 刘宝 <po.lb@antfin.com>
Co-authored-by: Hao Chen <chenh1024@gmail.com>
* Skeleton plus a unit test for simple borrower case
* First unit test passes - forward an ID and task returns with 1 submitted task pending on the inner ID
* Invariant for contained_in
* Unit test passes for testing task return without creating a borrower
* Wrap ref count functionality in test case
* Fix bad delete
* Unit test and fix for borrowers creating more borrowers
* Unit test and fix for simple borrowing, but owner sends call after borrower's ref count goes to 0
* Refactor:
- keep a sentinel ref count for task argument IDs
- keep contained_in_borrowed in addition to contained_in_owned
* Unit test for nested IDs passes
* Refactor so that an object ID can only be contained in 1 borrowed ID at a time
* Add check
* Fix
* Unit test (passes) to test nesting object IDs but no borrowers created
* Unit test for nested objects from different owners passes, refactor to unset contained_in when popping refs
* Unit tests for borrowers receiving an ObjectID from multiple sources,
skip adding ownership info if we already have it to handle duplicate
refs
* Unit test for returning object ID passes
* More unit tests for returning object IDs pass
* Add serialized ID tests
* fix serialization issue
* remove swap
* It builds!
* debugging and some fixes:
- register handler for WaitForRefRemoved
- don't create a python reference for arg IDs
- pass in client factory into ReferenceCounter
- fix bad decrement in PopBorrowerRefs
* Fix accounting for serialized IDs:
- don't decrement for IDs on dependency resolution, wait until task finished
- add object IDs that were inlined when building the arguments to the task spec, pin these on the task executor until task finishes
* mu_ -> mutex_
* lint
* fix build
* clear outer_object_id
* add direct call type check
* Fix test for direct call IDs and return IDs for actor calls
* Fix CoreWorkerClient.Addr()
* Remove unneeded lock
* Remove unnecessary ObjectID refs
* Fix worker holding serialized refs test
* Fix hex IDs
* fix
* fix tests
* fix tests
* refactor and cleanups
* lint
* Put inlined Ids in task args and some cleanup
* Add back gc.collect() line for test case
* Refactor and fixes:
- store inlined IDs in RayObject
- allow storing objects with inlined IDs in memory store
- pin objects that were promoted to plasma
* oops
* make sure worker ID is set in address, pass in rpc::Address to CoreWorkerClient
* todos
* cleanups and test builds
* Fix tests
* Add feature flag
* cleanups
* address comments and some cleanups
* cleanup
* fix recursive test
* Comments for tests
* Turn off ref counting by default
* Skip tests
* Fix some bugs for test_array.py, java build
* Don't include nested objects in the ref count when the feature flag is off
* C++ feature flag does not work...
* Remove
* Turn on python tests and add a warning when plasma objects are evicted before being pinned
* Fix build and remove irrelevant test
* Fix for java
* Revert "Fix build and remove irrelevant test"
This reverts commit 056cca9b263ed05b0f9ab2250907338edcbca2d5.
* Fix ray.internal.free
* Fixes and skip some flaky tests
* fix java build
* fix windows build
* Add IDs contained in owned objects
* Update src/ray/protobuf/core_worker.proto
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Update src/ray/core_worker/reference_count.cc
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Update src/ray/protobuf/core_worker.proto
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Update src/ray/protobuf/core_worker.proto
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Update src/ray/core_worker/reference_count.h
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Update src/ray/core_worker/reference_count.h
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Update src/ray/core_worker/reference_count.cc
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Apply suggestions from code review
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* update
* Try to fix ::test_direct_call_serialized_id_eviction
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
* ensure data_ won't be null_ptr when size == 0
* when data_sizes[i] == 0, we should Allocate an empty buffer
* work around for pyarrow.py_buffer
* fix comments
* add null ptr check
* add test for bytes
* lint
* Do not start threadpool when using async
* Turn function_executor into a generator
* Add new test for high concurrency and bump the default
* Set direct call
Clark Zinzow