diff --git a/site/_posts/2017-11-30-ray-0.3-release.markdown b/site/_posts/2017-11-30-ray-0.3-release.markdown new file mode 100644 index 000000000..f433ccb4f --- /dev/null +++ b/site/_posts/2017-11-30-ray-0.3-release.markdown @@ -0,0 +1,147 @@ +--- +layout: post +title: "Ray: 0.3 Release" +excerpt: "This post announces the release of Ray 0.3." +date: 2017-11-30 14:00:00 +--- + +We are pleased to announce the Ray 0.3 release. This release introduces +[**distributed actor handles**][1] and [**Ray.tune**][2] --- a new +hyperparameter search library. It also includes a number of bug fixes and +stability enhancements. + +To upgrade to the latest version, run + +``` +pip install -U ray +``` + +## Hyperparameter Search Tool + +This release adds [**Ray.tune**][2], a distributed hyperparameter evaluation +tool for long-running tasks such as reinforcement learning and deep learning +training. It currently includes the following features: + +- Pluggable **early stopping algorithms** including the Median Stopping Rule and + [Hyperband][5]. +- Integration with **visualization tools** such as [TensorBoard][6], + [rllab's VisKit][7], and a [parallel coordinates visualization][8]. +- Flexible **trial variant generation**, including grid search, random search, + and conditional parameter distributions. +- **Resource-aware scheduling**, including support for concurrent runs of + algorithms that require GPUs or are themselves parallel and distributed. + +Ray.tune provides a Python API for use with deep learning and other compute +intensive training tasks. Here is a toy example illustrating usage. + +```python +from ray.tune import register_trainable, grid_search, run_experiments + +def my_func(config, reporter): + import time, numpy as np + i = 0 + while True: + reporter(timesteps_total=i, mean_accuracy=i ** config['alpha']) + + i += config['beta'] + time.sleep(0.01) + +register_trainable('my_func', my_func) + +run_experiments({ + 'my_experiment': { + 'run': 'my_func', + 'resources': {'cpu': 1, 'gpu': 0}, + 'stop': {'mean_accuracy': 100}, + 'config': { + 'alpha': grid_search([0.2, 0.4, 0.6]), + 'beta': grid_search([1, 2]), + }, + } +}) +``` + +In-progress results can be visualized live using tools such as Tensorboard and +rllab's VisKit (or you can read the JSON format logs directly from the driver +node): + +View the [documentation][2] and the [code][9]. + +### Ray.tune integration with RLlib + +You can try out RLlib with Ray.tune with the following example. + +```bash +cd ray/python/ray/rllib +python train.py -f tuned_examples/cartpole-grid-search-example.yaml +``` + +The `tuned_examples` directory also contains pre-tuned hyperparameter +configurations for common benchmark tasks such as Pong and Humanoid. View the +[RLlib documentation][3]. + +### Initial Support for PyTorch in RLlib + +A modern reinforcement learning library should work with multiple deep learning +frameworks. As a step toward this goal, 0.3 adds support for PyTorch models for +A3C in RLlib. You can try this out with the following A3C config. + +```bash +cd ray/python/ray/rllib +./train.py --run=A3C \ + --env=PongDeterministic-v4 \ + --config='{"use_pytorch": true, "num_workers": 8, "use_lstm": false, "model": {"grayscale": true, "zero_mean": false, "dim": 80, "channel_major": true}}' +``` + +## Distributed Actor Handles + +Ray 0.3 adds support for [distributed actor handles][1], that is, the ability to +have multiple callers invoke methods on the same actor. The actor’s creator may +pass the actor handle as an argument to other tasks or to other actor methods. +Here’s an example in which the driver creates an actor to log messages and +passes an actor handle to other tasks: + +```python +import ray + +ray.init() + +@ray.remote +class Logger(object): + def __init__(self): + self.logs = [] + def log(self, log_msg): + self.logs.append(log_msg) + def read_logs(self): + return self.logs + +@ray.remote +def task(logger, task_index): + # Do some work. + logged = logger.log.remote('Task {} is done'.format(task_index)) + +# Create an actor and get a handle to it. +logger = Logger.remote() +# Pass the actor handle to some tasks. +tasks = [task.remote(logger, i) for i in range(10)] +# Wait for the tasks to finish. +ray.get(tasks) +# Get the logs from the tasks. +print(ray.get(logger.read_logs.remote())) +``` + +This feature is still considered experimental, but we've already found +distributed actor handles useful for implementing [**parameter server**][10] and +[**streaming MapReduce**][11] applications. + +[1]: http://ray.readthedocs.io/en/latest/actors.html#passing-around-actor-handles-experimental +[2]: http://ray.readthedocs.io/en/latest/tune.html +[3]: http://ray.readthedocs.io/en/latest/rllib.html +[4]: https://research.google.com/pubs/pub46180.html +[5]: https://arxiv.org/abs/1603.06560 +[6]: https://www.tensorflow.org/get_started/summaries_and_tensorboard +[7]: https://media.readthedocs.org/pdf/rllab/latest/rllab.pdf +[8]: https://en.wikipedia.org/wiki/Parallel_coordinates +[9]: https://github.com/ray-project/ray/tree/master/python/ray/tune +[10]: http://ray.readthedocs.io/en/latest/example-parameter-server.html +[11]: http://ray.readthedocs.io/en/latest/example-streaming.html