* Use set/dict literal syntax
Ran code through [pyupgrade](https://github.com/asottile/pyupgrade). This is
supported in every Python version 2.7+.
* Drop unnecessary string format specification
No need to specify 0,1.. if paramters are passed in order.
* Revert "Drop unnecessary string format specification"
This reverts commit efa5ec85d30ff69f34e5ed93e31343fea7647bcb.
* Undo changes to cloudpickle
Drop use of set literal until cloudpickle uses it.
* Reformat code with YAPF
We need to set up a git pre-push hook to automatically run this stuff.
* Treat actor creation like a regular task.
* Small cleanups.
* Change semantics of actor resource handling.
* Bug fix.
* Minor linting
* Bug fix
* Fix jenkins test.
* Fix actor tests
* Some cleanups
* Bug fix
* Fix bug.
* Remove cached actor tasks when a driver is removed.
* Add more info to taskspec in global state API.
* Fix cyclic import bug in tune.
* Fix
* Fix linting.
* Fix linting.
* Don't schedule any tasks (especially actor creaiton tasks) on local schedulers with 0 CPUs.
* Bug fix.
* Add test for 0 CPU case
* Fix linting
* Address comments.
* Fix typos and add comment.
* Add assertion and fix test.
* Define execution dependencies flatbuffer and add to Redis commands
* Convert TaskSpec to TaskExecutionSpec
* Add execution dependencies to Python bindings
* Submitting actor tasks uses execution dependency API instead of dummy argument
* Fix dependency getters and some cleanup for fetching missing dependencies
* C++ convention
* Make TaskExecutionSpec a C++ class
* Convert local scheduler to use TaskExecutionSpec class
* Convert some pointers to references
* Finish conversion to TaskExecutionSpec class
* fix
* Fix
* Fix memory errors?
* Cast flatbuffers GetSize to size_t
* Fixes
* add more retries in global scheduler unit test
* fix linting and cast fbb.GetSize to size_t
* Style and doc
* Fix linting and simplify from_flatbuf.
* Enable scheduling with custom resource labels.
* Fix.
* Minor fixes and ref counting fix.
* Linting
* Use .data() instead of .c_str().
* Fix linting.
* Fix ResourcesTest.testGPUIDs test by waiting for workers to start up.
* Sleep in test so that all tasks are submitted before any completes.
* Object table lookup returns vector of DBClientID instead of address strings
* Add node IP address to DBClient notification
* DB client cache stores entire DB client, convert addresses to std::string
* get cached db client returns the client
* Expose a call to initialize the redis cache
* Local scheduler filters out dead clients during reconstruction
* Remove node ip address from dbclient, use aux_address for plasma managers
* Get entire db client entry when not found in cache
* Fix common tests
* Fix address in tests
* Push error to driver if driver task did the put
* Address Robert's comments and cleanup
* Remove unused Redis command
* Fix db test
* WIP: removing OL, OI, TT on client exit; no saving yet.
* ray_redis_module.cc: update header comment.
* Cleanup: just the removal.
* Reformat via yapf: use pep8 style instead of google.
* Checkpoint addressing comments (partially)
* Add 'b' marker before strings (py3 compat)
* Add MonitorTest.
* Use `isort` to sort imports.
* Remove some loggings
* Fix flake8 noqa marker runtest.py
* Try to separate tests out to monitor_test.py
* Rework cleanup algorithm: correct logic
* Extend tests to cover multi-shard cases
* Add some small comments and formatting changes.
* Clone catapult and generate static html during setup.
* Include UI files in installation.
* Fix directory to clone catapult to and fix linting.
* Use absolute path.
* Make sure we find a sufficiently new version of python2 when building wheels.
* Copy the trace_viewer_full.html file to the local directory if it is not present.
* Make sure wheels fail to build if UI is not included.
* Embedded timeline
* Yeah
* Fixed arrows not showing up.
* Fixed arrows not showing up, and added check boxes for the kinds of dependencies that should be included in the trace.
* first
* Fixes
* Fixed typo in comments, added more comments. fixed linting.
* Added more comments.
* Formatting.
* fixes
* Fixed state.py linting.
* Fixed ui.py linting errors.
* Fixed linting errors.
* Renamed task dependencies and included instructions for viewing arrows.
* Fixed according to PR comments.
* Fixed bug.
* Undid changes to metadata blocks.
* Fixes according to comments.
* Fixed linting.
* Fixed linting.
* NOQA keyword added to link line.
* adding support for the user-interpretable label(UIR)
* more plumbing for num_uirs further upstream; set to infty when specified on cmd line
* pass default num_uirs for actors; update GlobalStateAPI
* support num_uirs in ray.init()
* local scheduler resource accounting: support num_uirs; prep for vectorized resource accounting
* global scheduler test updated
* Fix bug introduced by rebase.
* Rename UIR -> CustomResource and add test.
* Small changes and use constexpr instead of macros.
* Linting and some renaming.
* Reorder some code.
* Remove cpus_in_use and fix bug.
* Add another test and make a small change.
* Rephrase documentation about feature stability.
* Rebase Ray on top of Plasma in Apache Arrow
* add thirdparty building scripts
* use rebased arrow
* fix
* fix build
* fix python visibility
* comment out C tests for now
* fix multithreading
* fix
* reduce logging
* fix plasma manager multithreading
* make sure old and new object IDs can coexist peacefully
* more rebasing
* update
* fixes
* fix
* install pyarrow
* install cython
* fix
* install newer cmake
* fix
* rebase on top of latest arrow
* getting runtest.py run locally (needed to comment out a test for that to work)
* work on plasma tests
* more fixes
* fix local scheduler tests
* fix global scheduler test
* more fixes
* fix python 3 bytes vs string
* fix manager tests valgrind
* fix documentation building
* fix linting
* fix c++ linting
* fix linting
* add tests back in
* Install without sudo.
* Set PKG_CONFIG_PATH in build.sh so that Ray can find plasma.
* Install pkg-config
* Link -lpthread, note that find_package(Threads) doesn't seem to work reliably.
* Comment in testGPUIDs in runtest.py.
* Set PKG_CONFIG_PATH when building pyarrow.
* Pull apache/arrow and not pcmoritz/arrow.
* Fix installation in docker image.
* adapt to changes of the plasma api
* Fix installation of pyarrow module.
* Fix linting.
* Use correct python executable to build pyarrow.
* 4 space indentation for actor.py.
* 4 space indentation for worker.py.
* 4 space indentation for more files.
* 4 space indentation for some test files.
* Check indentation in Travis.
* 4 space indentation for some rl files.
* Fix failure test.
* Fix multi_node_test.
* 4 space indentation for more files.
* 4 space indentation for remaining files.
* Fixes.
* Updated task_profiles function to avoid future repetitive parsing.
* Fix indentation.
* Fixed according to comments.
* Included updated test for task_profiles function.
* Simplify test.
* Fix indentation.
* Fix.
* added log_table function and a test
* fixed log_files and added task_profiles
* fixed formatting
* fixed linting errors
* fixes
* removed file
* more fixes
* hopefully fixed
* Small changes.
* Fix linting.
* Fix bug in log monitor.
* Small changes.
* Fix bug in travis.
* Including data_size and hash in the ResultTableReply.
* Included data_size and hash info in object_table.
* Fixed bugs in ray_redis_module.cc.
* Removing commented out code.
* Fixes
* Freed hash and data_size strings after using, and checked if they're null along with task_id and is_put.
* Changed it so that data_size is set correctly.
* Removed iostream import.
* Included a check to ensure that the Redis string to long long conversion was successful.
* Included separate data_size and hash null checks.
* Fixed bug.
* Made linting changes.
* Another linting error.
* Slight simplication.
* Implement sharding in the Ray core
* Single node Python modifications to do sharding
* Do the sharding in redis.cc
* Pipe num_redis_shards through start_ray.py and worker.py.
* Use multiple redis shards in multinode tests.
* first steps for sharding ray.global_state
* Fix problem in multinode docker test.
* fix runtest.py
* fix some tests
* fix redis shard startup
* fix redis sharding
* fix
* fix bug introduced by the map-iterator being consumed
* fix sharding bug
* shard event table
* update number of Redis clients to be 64K
* Fix object table tests by flushing shards in between unit tests
* Fix local scheduler tests
* Documentation
* Register shard locations in the primary shard
* Add plasma unit tests back to build
* lint
* lint and fix build
* Fix
* Address Robert's comments
* Refactor start_ray_processes to start Redis shard
* lint
* Fix global scheduler python tests
* Fix redis module test
* Fix plasma test
* Fix component failure test
* Fix local scheduler test
* Fix runtest.py
* Fix global scheduler test for python3
* Fix task_table_test_and_update bug, from actor task table submission race
* Fix jenkins tests.
* Retry Redis shard connections
* Fix test cases
* Convert database clients to DBClient struct
* Fix race condition when subscribing to db client table
* Remove unused lines, add APITest for sharded Ray
* Fix
* Fix memory leak
* Suppress ReconstructionTests output
* Suppress output for APITestSharded
* Reissue task table add/update commands if initial command does not publish to any subscribers.
* fix
* Fix linting.
* fix tests
* fix linting
* fix python test
* fix linting
* Perform ray.register_class under the hood.
* Fix bug.
* Release worker lock when waiting for imports to arrive in get.
* Remove calls to register_class from examples and tests.
* Clear serialization state between tests.
* Fix bug and add test for multiple custom classes with same name.
* Fix failure test.
* Fix linting and cleanups to python code.
* Fixes to documentation.
* Implement recursion depth for recursively registering classes.
* Fix linting.
* Push warning to user if waiting for class for too long.
* Fix typos.
* Don't export FunctionToRun if pickling the function fails.
* Don't broadcast class definition when pickling class.
* Clean up state when drivers exit.
* Remove unnecessary field in ActorMapEntry struct.
* Have monitor release GPU resources in Redis when driver exits.
* Enable multiple drivers in multi-node tests and test driver cleanup.
* Make redis GPU allocation a redis transaction and small cleanups.
* Fix multi-node test.
* Small cleanups.
* Make global scheduler take node_ip_address so it appears in the right place in the client table.
* Cleanups.
* Fix linting and cleanups in local scheduler.
* Fix removed_driver_test.
* Fix bug related to vector -> list.
* Fix linting.
* Cleanup.
* Fix multi node tests.
* Fix jenkins tests.
* Add another multi node test with many drivers.
* Fix linting.
* Make the actor creation notification a flatbuffer message.
* Revert "Make the actor creation notification a flatbuffer message."
This reverts commit af99099c8084dbf9177fb4e34c0c9b1a12c78f39.
* Add comment explaining flatbuffer problems.
Robert Nishihara