[docs] rewrite (#5175)

This commit is contained in:
Richard Liaw
2019-08-05 23:33:14 -07:00
committed by Eric Liang
parent 5d7afe8092
commit a08ea09760
32 changed files with 1456 additions and 1627 deletions
+4 -7
View File
@@ -4,26 +4,23 @@ Using Ray with TensorFlow
This document describes best practices for using Ray with TensorFlow.
To see more involved examples using TensorFlow, take a look at
`A3C`_, `ResNet`_, `Policy Gradients`_, and `LBFGS`_.
`A3C`_, `ResNet`_, and `LBFGS`_.
.. _`A3C`: http://ray.readthedocs.io/en/latest/example-a3c.html
.. _`ResNet`: http://ray.readthedocs.io/en/latest/example-resnet.html
.. _`Policy Gradients`: http://ray.readthedocs.io/en/latest/example-policy-gradient.html
.. _`LBFGS`: http://ray.readthedocs.io/en/latest/example-lbfgs.html
If you are training a deep network in the distributed setting, you may need to
ship your deep network between processes (or machines). For example, you may
update your model on one machine and then use that model to compute a gradient
on another machine. However, shipping the model is not always straightforward.
ship your deep network between processes (or machines). However, shipping the model is not always straightforward.
For example, a straightforward attempt to pickle a TensorFlow graph gives mixed
A straightforward attempt to pickle a TensorFlow graph gives mixed
results. Some examples fail, and some succeed (but produce very large strings).
The results are similar with other pickling libraries as well.
Furthermore, creating a TensorFlow graph can take tens of seconds, and so
serializing a graph and recreating it in another process will be inefficient.
The better solution is to create the same TensorFlow graph on each worker once
The better solution is to replicate the same TensorFlow graph on each worker once
at the beginning and then to ship only the weights between the workers.
Suppose we have a simple network definition (this one is modified from the