mirror of
https://github.com/wassname/ray.git
synced 2026-08-12 12:20:11 +08:00
[Tune] Pbt Function API (#9958)
* adding function convnet example * add unit test * update test * update example * wip * move error from experiment to tune * wip * Fix checkpoint deletion * updating code * adding smoke test * updating pbt guide * formatting * fix build * add best checkpoint analysis util * update test * add comments * remove class api * fix example * add setup and teardown to tests * formatting * Update python/ray/tune/tests/test_trial_scheduler_pbt.py Co-authored-by: Kai Fricke <kai@anyscale.com> Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
This commit is contained in:
Amog Kamsetty
co-authored by
Kai Fricke
Richard Liaw
Kai Fricke
Richard Liaw
parent
fba5906ce3
commit
f87a4aa45d
@@ -14,9 +14,8 @@ hyperparameters and allocate resources to promising models. Let's walk through h
|
||||
:local:
|
||||
:backlinks: none
|
||||
|
||||
|
||||
Trainable API with Population Based Training
|
||||
--------------------------------------------
|
||||
Function API with Population Based Training
|
||||
-------------------------------------------
|
||||
|
||||
PBT takes its inspiration from genetic algorithms where each member of the population
|
||||
can exploit information from the remainder of the population. For example, a worker might
|
||||
@@ -31,23 +30,24 @@ This means that PBT can quickly exploit good hyperparameters, can dedicate more
|
||||
promising models and, crucially, can adapt the hyperparameter values throughout training,
|
||||
leading to automatic learning of the best configurations.
|
||||
|
||||
First, we define a Trainable that wraps a ConvNet model.
|
||||
First we define a training function that trains a ConvNet model using SGD.
|
||||
|
||||
.. literalinclude:: /../../python/ray/tune/examples/pbt_convnet_example.py
|
||||
:language: python
|
||||
:start-after: __trainable_begin__
|
||||
:end-before: __trainable_end__
|
||||
.. literalinclude:: /../../python/ray/tune/examples/pbt_convnet_function_example.py
|
||||
:language: python
|
||||
:start-after: __train_begin__
|
||||
:end-before: __train_end__
|
||||
|
||||
The example reuses some of the functions in ray/tune/examples/mnist_pytorch.py, and is also a good
|
||||
demo for how to decouple the tuning logic and original training code.
|
||||
|
||||
Here, we also override ``reset_config``. This method is optional but can be implemented to speed
|
||||
up algorithms such as PBT, and to allow performance optimizations such as running experiments
|
||||
with ``reuse_actors=True``.
|
||||
Here, we also need to take in a ``checkpoint_dir`` arg since checkpointing is required for the exploitation process in PBT.
|
||||
We have to both load in the checkpoint if one is provided, and periodically save our
|
||||
model state in a checkpoint- in this case every 5 iterations. With SGD, there's no need to checkpoint the optimizer
|
||||
since it does not depend on previous states, but this is necessary with other optimizers like Adam.
|
||||
|
||||
Then, we define a PBT scheduler:
|
||||
|
||||
.. literalinclude:: /../../python/ray/tune/examples/pbt_convnet_example.py
|
||||
.. literalinclude:: /../../python/ray/tune/examples/pbt_convnet_function_example.py
|
||||
:language: python
|
||||
:start-after: __pbt_begin__
|
||||
:end-before: __pbt_end__
|
||||
@@ -67,7 +67,7 @@ Some of the most important parameters are:
|
||||
|
||||
Now we can kick off the tuning process by invoking tune.run:
|
||||
|
||||
.. literalinclude:: /../../python/ray/tune/examples/pbt_convnet_example.py
|
||||
.. literalinclude:: /../../python/ray/tune/examples/pbt_convnet_function_example.py
|
||||
:language: python
|
||||
:start-after: __tune_begin__
|
||||
:end-before: __tune_end__
|
||||
@@ -77,19 +77,19 @@ During the training, we can constantly check the status of the models from conso
|
||||
.. code-block:: bash
|
||||
|
||||
== Status ==
|
||||
Memory usage on this node: 11.6/16.0 GiB
|
||||
PopulationBasedTraining: 5 checkpoints, 4 perturbs
|
||||
Resources requested: 0/16 CPUs, 0/0 GPUs, 0.0/3.96 GiB heap, 0.0/1.37 GiB objects
|
||||
Memory usage on this node: 11.2/16.0 GiB
|
||||
PopulationBasedTraining: 12 checkpoints, 5 perturbs
|
||||
Resources requested: 0/16 CPUs, 0/0 GPUs, 0.0/4.83 GiB heap, 0.0/1.66 GiB objects
|
||||
Result logdir: /Users/foo/ray_results/pbt_test
|
||||
Number of trials: 4 (4 TERMINATED)
|
||||
+------------------------------+------------+-------+-----------+------------+----------+--------+------------------+
|
||||
| Trial name | status | loc | lr | momentum | acc | iter | total time (s) |
|
||||
|------------------------------+------------+-------+-----------+------------+----------+--------+------------------|
|
||||
| PytorchTrainable_ba982_00000 | TERMINATED | | 0.0457501 | 0.99 | 0.6375 | 25 | 5.35712 |
|
||||
| PytorchTrainable_ba982_00001 | TERMINATED | | 0.175808 | 0.0667043 | 0.909375 | 29 | 6.18802 |
|
||||
| PytorchTrainable_ba982_00002 | TERMINATED | | 0.21097 | 0.99 | 0.040625 | 29 | 6.19634 |
|
||||
| PytorchTrainable_ba982_00003 | TERMINATED | | 0.0571876 | 0.852088 | 0.96875 | 30 | 6.37298 |
|
||||
+------------------------------+------------+-------+-----------+------------+----------+--------+------------------+
|
||||
+---------------------------+------------+-------+-----------+------------+----------+--------+------------------+
|
||||
| Trial name | status | loc | lr | momentum | acc | iter | total time (s) |
|
||||
|---------------------------+------------+-------+-----------+------------+----------+--------+------------------|
|
||||
| train_convnet_b2732_00000 | TERMINATED | | 0.221776 | 0.608416 | 0.95625 | 59 | 13.0862 |
|
||||
| train_convnet_b2732_00001 | TERMINATED | | 0.0734679 | 0.1484 | 0.934375 | 59 | 13.1084 |
|
||||
| train_convnet_b2732_00002 | TERMINATED | | 0.0376862 | 0.8 | 0.971875 | 46 | 10.2909 |
|
||||
| train_convnet_b2732_00003 | TERMINATED | | 0.0471078 | 0.8 | 0.95 | 51 | 11.3355 |
|
||||
+---------------------------+------------+-------+-----------+------------+----------+--------+------------------+
|
||||
|
||||
In {LOG_DIR}/{MY_EXPERIMENT_NAME}/, all mutations are logged in pbt_global.txt
|
||||
and individual policy perturbations are recorded in pbt_policy_{i}.txt. Tune logs:
|
||||
@@ -145,7 +145,6 @@ thus just use the same ``Trainable`` for the replay run.
|
||||
scheduler=replay,
|
||||
stop={"training_iteration": 100})
|
||||
|
||||
|
||||
DCGAN with Trainable and PBT
|
||||
----------------------------
|
||||
|
||||
|
||||
Reference in New Issue
Block a user