We are pleased to announce the Ray 0.3 release. This release introduces distributed actor handles and Ray.tune — a new hyperparameter search library. It also includes a number of bug fixes and stability enhancements.

To upgrade to the latest version, run

pip install -U ray

Hyperparameter Search Tool

This release adds Ray.tune, a distributed hyperparameter evaluation tool for long-running tasks such as reinforcement learning and deep learning training. It currently includes the following features:

  • Pluggable early stopping algorithms including the Median Stopping Rule and Hyperband.
  • Integration with visualization tools such as TensorBoard, rllab’s VisKit, and a parallel coordinates visualization.
  • Flexible trial variant generation, including grid search, random search, and conditional parameter distributions.
  • Resource-aware scheduling, including support for concurrent runs of algorithms that require GPUs or are themselves parallel and distributed.

Ray.tune provides a Python API for use with deep learning and other compute intensive training tasks. Here is a toy example illustrating usage.

from ray.tune import register_trainable, grid_search, run_experiments

def my_func(config, reporter):
    import time, numpy as np
    i = 0
    while True:
        reporter(timesteps_total=i, mean_accuracy=i ** config['alpha'])

        i += config['beta']
        time.sleep(0.01)

register_trainable('my_func', my_func)

run_experiments({
    'my_experiment': {
        'run': 'my_func',
        'resources': {'cpu': 1, 'gpu': 0},
        'stop': {'mean_accuracy': 100},
        'config': {
            'alpha': grid_search([0.2, 0.4, 0.6]),
            'beta': grid_search([1, 2]),
        },
    }
})

In-progress results can be visualized live using tools such as Tensorboard and rllab’s VisKit (or you can read the JSON format logs directly from the driver node):

View the documentation and the code.

Ray.tune integration with RLlib

You can try out RLlib with Ray.tune with the following example.

cd ray/python/ray/rllib
python train.py -f tuned_examples/cartpole-grid-search-example.yaml

The tuned_examples directory also contains pre-tuned hyperparameter configurations for common benchmark tasks such as Pong and Humanoid. View the RLlib documentation.

Initial Support for PyTorch in RLlib

A modern reinforcement learning library should work with multiple deep learning frameworks. As a step toward this goal, 0.3 adds support for PyTorch models for A3C in RLlib. You can try this out with the following A3C config.

cd ray/python/ray/rllib
./train.py --run=A3C \
           --env=PongDeterministic-v4 \
           --config='{"use_pytorch": true, "num_workers": 8, "use_lstm": false, "model": {"grayscale": true, "zero_mean": false, "dim": 80, "channel_major": true}}'

Distributed Actor Handles

Ray 0.3 adds support for distributed actor handles, that is, the ability to have multiple callers invoke methods on the same actor. The actor’s creator may pass the actor handle as an argument to other tasks or to other actor methods. Here’s an example in which the driver creates an actor to log messages and passes an actor handle to other tasks:

import ray

ray.init()

@ray.remote
class Logger(object):
    def __init__(self):
        self.logs = []
    def log(self, log_msg):
        self.logs.append(log_msg)
    def read_logs(self):
        return self.logs

@ray.remote
def task(logger, task_index):
    # Do some work.
    logged = logger.log.remote('Task {} is done'.format(task_index))

# Create an actor and get a handle to it.
logger = Logger.remote()
# Pass the actor handle to some tasks.
tasks = [task.remote(logger, i) for i in range(10)]
# Wait for the tasks to finish.
ray.get(tasks)
# Get the logs from the tasks.
print(ray.get(logger.read_logs.remote()))

This feature is still considered experimental, but we’ve already found distributed actor handles useful for implementing parameter server and streaming MapReduce applications.