* Added support for submission-time task names.
* Suggestions from code review: add missing consts
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
* Add num_returns arg to actor method options docstring example.
* Add process name line and proctitle assertion to submission-time task name section of advanced docs.
* Add submission-time task name --> proctitle test for Python worker.
* Added Python actor options tests for num_returns and name.
* Added Java test for submission-time task names.
* Add dashboard image to task name docs section.
* Move to fstrings.
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
* Fix SC2006: Use $(...) notation instead of legacy backticked `...`.
* Fix SC2016: Expressions don't expand in single quotes, use double quotes for that.
* Fix SC2046: Quote this to prevent word splitting.
* Fix SC2053: Quote the right-hand side of == in [[ ]] to prevent glob matching.
* Fix SC2068: Double quote array expansions to avoid re-splitting elements.
* Fix SC2086: Double quote to prevent globbing and word splitting.
* Fix SC2102: Ranges can only match single chars (mentioned due to duplicates).
* Fix SC2140: Word is of the form "A"B"C" (B indicated). Did you mean "ABC" or "A\"B\"C"?
* Fix SC2145: Argument mixes string and array. Use * or separate argument.
* Fix SC2209: warning: Use var=$(command) to assign output (or quote to assign string).
Co-authored-by: Mehrdad <noreply@github.com>
* Get rid of system() calls
* Work around '/usr/share/mini' showing up on GitHub Actions (probably due to psutil truncation)
https://github.com/ray-project/ray/runs/722480047?check_suite_focus=true
* Don't check for socket max path length on Windows
* Don't check for socket existence on Windows
* Fix race condition in Windows fate-sharing
* Work around missing .exe extension for Redis tests
* Add more tests to GitHub Actions
Co-authored-by: Mehrdad <noreply@github.com>
* Python
* Consolidate state in the direct actor transport, set the caller starts at
* todo
* Remove unused
* Update and unit tests
* Doc
* Remove unused
* doc
* Remove debug
* Update src/ray/core_worker/transport/direct_actor_transport.h
Co-authored-by: Eric Liang <ekhliang@gmail.com>
* Update src/ray/core_worker/transport/direct_actor_transport.cc
Co-authored-by: Eric Liang <ekhliang@gmail.com>
* lint and fix build
* Update
* Fix build
* Fix tests
* Unit test for max_task_retries=0
* Fix java?
* Fix bad test
* Cross language fix
* fix java
Co-authored-by: Eric Liang <ekhliang@gmail.com>
* Move some Java tests into ci.sh
* Move C++ worker tests into ci.sh
* Define run()
* Prepare to move Python tests into ci.sh
* Fix issues in install-dependencies.sh
* Reload environment for GitHub Actions
* Move wheels to ci.sh and fix related issues
* Don't bypass failures in install-ray.sh anymore
* Make CI a little quieter
* Move linting into ci.sh
* Add vitals test right after build
* Fix os.uname() unavailability on Windows
Co-authored-by: Mehrdad <noreply@github.com>
* Smol comment
* WIP, not passing ray.init
* Fixed small problem
* wip
* Pseudo interrupt things
* Basic prototype operational
* correct proc title
* Mostly done
* Cleanup
* cleaner raylet error
* Cleaning up a few loose ends
* Fixing Race Conds
* Prelim testing
* Fixing comments and adding second_check for kill
* Working_new_impl
* demo_ready
* Fixing my english
* Fixing a few problems
* Small problems
* Cleaning up
* Response to changes
* Fixing error passing
* Merged to master
* fixing lock
* Cleaning up print statements
* Format
* Fixing Unit test build failure
* mock_worker fix
* java_fix
* Canel
* Switching to Cancel
* Responding to Review
* FixFormatting
* Lease cancellation
* FInal comments?
* Moving exist check to CoreWorker
* Fix Actor Transport Test
* Fixing task manager test
* chaning clock repr
* Fix build
* fix white space
* lint fix
* Updating to medium size
* Fixing Java test compilation issue
* lengthen bad timeouts
* try to trigger mock transfer tests ci
* execute transfer tests
* show all logs when bazel test streaming
* temporary repeated ci runs
* Revert "temporary repeated ci runs"
This reverts commit dc77d2f9f79b5fa7b490221a8e9089e6349e067d.
* Fix cyclic dependency
Headers in ray/util should not depend on those in ray/common
* Move random generations to ray/common/test_util.h
* Add license header
Co-authored-by: Mehrdad <noreply@github.com>
Co-authored-by: Philipp Moritz <pcmoritz@gmail.com>
* streaming writer use event driven model.
* add RefreshChannelInfo
* fix name
* minor changes according reviewer comments
* Fix according to reviewer's comments
* fix bazel lint
* code polished
* Add more comments
* rename Stop & Start of EventQueue to Freeze and Unfreeze.
* add override
* fix
* fix return value
* support flow control
* add flow control ut in mock transfer
* minor changes according to comments
* add java and python worker adaption
Co-authored-by: wanxing <wanxing.wwx@alibaba-inc.com>
* Skeleton plus a unit test for simple borrower case
* First unit test passes - forward an ID and task returns with 1 submitted task pending on the inner ID
* Invariant for contained_in
* Unit test passes for testing task return without creating a borrower
* Wrap ref count functionality in test case
* Fix bad delete
* Unit test and fix for borrowers creating more borrowers
* Unit test and fix for simple borrowing, but owner sends call after borrower's ref count goes to 0
* Refactor:
- keep a sentinel ref count for task argument IDs
- keep contained_in_borrowed in addition to contained_in_owned
* Unit test for nested IDs passes
* Refactor so that an object ID can only be contained in 1 borrowed ID at a time
* Add check
* Fix
* Unit test (passes) to test nesting object IDs but no borrowers created
* Unit test for nested objects from different owners passes, refactor to unset contained_in when popping refs
* Unit tests for borrowers receiving an ObjectID from multiple sources,
skip adding ownership info if we already have it to handle duplicate
refs
* Unit test for returning object ID passes
* More unit tests for returning object IDs pass
* Add serialized ID tests
* fix serialization issue
* remove swap
* It builds!
* debugging and some fixes:
- register handler for WaitForRefRemoved
- don't create a python reference for arg IDs
- pass in client factory into ReferenceCounter
- fix bad decrement in PopBorrowerRefs
* Fix accounting for serialized IDs:
- don't decrement for IDs on dependency resolution, wait until task finished
- add object IDs that were inlined when building the arguments to the task spec, pin these on the task executor until task finishes
* mu_ -> mutex_
* lint
* fix build
* clear outer_object_id
* add direct call type check
* Fix test for direct call IDs and return IDs for actor calls
* Fix CoreWorkerClient.Addr()
* Remove unneeded lock
* Remove unnecessary ObjectID refs
* Fix worker holding serialized refs test
* Fix hex IDs
* fix
* fix tests
* fix tests
* refactor and cleanups
* lint
* Put inlined Ids in task args and some cleanup
* Add back gc.collect() line for test case
* Refactor and fixes:
- store inlined IDs in RayObject
- allow storing objects with inlined IDs in memory store
- pin objects that were promoted to plasma
* oops
* make sure worker ID is set in address, pass in rpc::Address to CoreWorkerClient
* todos
* cleanups and test builds
* Fix tests
* Add feature flag
* cleanups
* address comments and some cleanups
* cleanup
* fix recursive test
* Comments for tests
* Turn off ref counting by default
* Skip tests
* Fix some bugs for test_array.py, java build
* Don't include nested objects in the ref count when the feature flag is off
* C++ feature flag does not work...
* Remove
* Turn on python tests and add a warning when plasma objects are evicted before being pinned
* Fix build and remove irrelevant test
* Fix for java
* Revert "Fix build and remove irrelevant test"
This reverts commit 056cca9b263ed05b0f9ab2250907338edcbca2d5.
* Fix ray.internal.free
* Fixes and skip some flaky tests
* fix java build
* fix windows build
* Add IDs contained in owned objects
* Update src/ray/protobuf/core_worker.proto
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Update src/ray/core_worker/reference_count.cc
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Update src/ray/protobuf/core_worker.proto
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Update src/ray/protobuf/core_worker.proto
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Update src/ray/core_worker/reference_count.h
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Update src/ray/core_worker/reference_count.h
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Update src/ray/core_worker/reference_count.cc
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Apply suggestions from code review
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* update
* Try to fix ::test_direct_call_serialized_id_eviction
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
* Fix common.fbs rename (due to apache/arrow/commit/bef9a1c251397311a6415d3dc362ef419d154caa)
* Add missing COPTS
* Use socketpair(AF_INET) if boost::asio::local is unavailable (e.g. on Windows)
* Fix compile bug in service_based_gcs_client_test.cc (fix build breakage in #6686)
* Work around googletest/gmock inability to specify override to avoid -Werror,-Winconsistent-missing-override
* Fix missing override on IsPlasmaBuffer()
* Fix missing libraries for streaming
* Factor out install-toolchains.sh
* Put some Bazel flags into .bazelrc
* Fix jni_md.h missing inclusion
* Add ~/bin to PATH for Bazel
* Change echo $$(date) > $@ to date > $@
* Fix lots of unquoted paths
* Add system() call checks for Windows
Co-authored-by: GitHub Web Flow <noreply@github.com>
* streaming writer use event driven model.
* minor changes according reviewer comments
* Fix according to reviewer's comments
* fix bazel lint
* code polished
* Add more comments
* rename Stop & Start of EventQueue to Freeze and Unfreeze.
* streaming message bundle use inplacement instance
* fix typo & enable common test
* fix compiler warning
* block copy for serilization
* add reference
* remove streaming common test to travis script
Lixin Wei