diff --git a/docker/autoscaler/README.md b/docker/autoscaler/README.md
new file mode 100644
index 000000000..807c37fff
--- /dev/null
+++ b/docker/autoscaler/README.md
@@ -0,0 +1,11 @@
+# DEPRECATED -- Please use [`rayproject/ray-ml`](https://hub.docker.com/repository/docker/rayproject/ray-ml)
+## About
+This image used to be the base image for the Ray autoscaler, but it has been replaced by [`rayproject/ray-ml`](https://hub.docker.com/repository/docker/rayproject/ray-ml).
+Please use that instead, *this image will be removed in the near future*.
+
+
+## Tags
+* [`:latest`](https://hub.docker.com/repository/docker/rayproject/autoscaler/tags?page=1&name=latest) - The most recent Ray release.
+* `:1.x.x` - A specific release build.
+* [`:nightly`](https://hub.docker.com/repository/docker/rayproject/autoscaler/tags?page=1&name=nightly) - The most recent nightly build.
+* `:SHA` - A specific nightly build.
\ No newline at end of file
diff --git a/docker/base-deps/README.md b/docker/base-deps/README.md
new file mode 100644
index 000000000..9a75db782
--- /dev/null
+++ b/docker/base-deps/README.md
@@ -0,0 +1,26 @@
+## About
+This is an internal image, the [`rayproject/ray`](https://hub.docker.com/repository/docker/rayproject/ray) or [`rayproject/ray-ml`](https://hub.docker.com/repository/docker/rayproject/ray-ml) should be used!
+
+
+This image has the system-level dependencies for `Ray` and the `Ray Autoscaler`. The `ray-deps` image is built on top of this. This image is built periodically or when dependencies are added. [Find the Dockerfile here.](https://github.com/ray-project/ray/blob/master/docker/base-deps/Dockerfile)
+
+
+
+
+## Tags
+* [`:latest`](https://hub.docker.com/repository/docker/rayproject/base-deps/tags?page=1&name=latest) - The most recent Ray release.
+* `:1.x.x` - A specific release build.
+* [`:nightly`](https://hub.docker.com/repository/docker/rayproject/base-deps/tags?page=1&name=nightly) - The most recent nightly build.
+* `:DATE` - A specific build.
+
+### Suffixes
+* `-gpu` - These are based off of an `NVIDIA CUDA` image. They require the [Nvidia Docker Runtime](https://github.com/NVIDIA/nvidia-docker) to be installed on the host for the container to access GPUs.
+* `-cpu`- These are based off of an `Ubuntu` image.
+* Tags without a suffix refer to `-cpu` images
+
+## Other Images
+* [`rayproject/ray`](https://hub.docker.com/repository/docker/rayproject/ray) - Ray and all of its dependencies.
+* [`rayproject/ray-ml`](https://hub.docker.com/repository/docker/rayproject/ray-ml) - This image with common ML libraries to make development & deployment more smooth!
+
+
+* [`rayproject/ray-deps`](https://hub.docker.com/repository/docker/rayproject/ray-deps) - Internal image with python-level dependencies.
diff --git a/docker/ray-deps/README.md b/docker/ray-deps/README.md
new file mode 100644
index 000000000..a393bd752
--- /dev/null
+++ b/docker/ray-deps/README.md
@@ -0,0 +1,24 @@
+## About
+This is an internal image, the [`rayproject/ray`](https://hub.docker.com/repository/docker/rayproject/ray) or [`rayproject/ray-ml`](https://hub.docker.com/repository/docker/rayproject/ray-ml) should be used!
+
+This has the python-level dependencies for `Ray` and the `Ray Autoscaler`. The `ray` image is built on top of this. This image is built periodically or when dependencies are added. [Find the Dockerfile here.](https://github.com/ray-project/ray/blob/master/docker/ray-deps/Dockerfile)
+
+
+
+
+## Tags
+* [`:latest`](https://hub.docker.com/repository/docker/rayproject/ray-deps/tags?page=1&name=latest) - The most recent Ray release.
+* `:1.x.x` - A specific release build.
+* [`:nightly`](https://hub.docker.com/repository/docker/rayproject/ray-deps/tags?page=1&name=nightly) - The most recent nightly build.
+* `:DATE` - A specific build.
+
+### Suffixes
+* `-gpu` - These are based off of an `NVIDIA CUDA` image. They require the [Nvidia Docker Runtime](https://github.com/NVIDIA/nvidia-docker) to be installed on the host for the container to access GPUs.
+* `-cpu`- These are based off of an `Ubuntu` image.
+* Tags without a suffix refer to `-cpu` images
+
+## Other Images
+* [`rayproject/ray`](https://hub.docker.com/repository/docker/rayproject/ray) - Ray and all of its dependencies.
+* [`rayproject/ray-ml`](https://hub.docker.com/repository/docker/rayproject/ray-ml) - This image with common ML libraries to make development & deployment more smooth!
+
+* [`rayproject/base-deps`](https://hub.docker.com/repository/docker/rayproject/base-deps) - Internal image with system-level dependencies.
diff --git a/docker/ray-ml/README.md b/docker/ray-ml/README.md
new file mode 100644
index 000000000..f96147754
--- /dev/null
+++ b/docker/ray-ml/README.md
@@ -0,0 +1,19 @@
+## About
+This image is an extension of the [`rayproject/ray`](https://hub.docker.com/repository/docker/rayproject/ray) image. It includes all extended requirements of `RLlib`, `Serve` and `RLLIB`. It is a well-provisioned starting point for trying out the Ray ecosystem. [Find the Dockerfile here.](https://github.com/ray-project/ray/blob/master/docker/ray-ml/Dockerfile)
+
+
+
+
+## Tags
+* [`:latest`](https://hub.docker.com/repository/docker/rayproject/ray-ml/tags?page=1&name=latest) - The most recent Ray release.
+* `:1.x.x` - A specific release build.
+* [`:nightly`](https://hub.docker.com/repository/docker/rayproject/ray-ml/tags?page=1&name=nightly) - The most recent nightly build.
+* `:SHA` - A specific nightly build.
+
+### Suffixes
+* `-gpu` - These are based off of an `NVIDIA CUDA` image. They require the [Nvidia Docker Runtime](https://github.com/NVIDIA/nvidia-docker) to be installed on the host for the container to access GPUs.
+* `-cpu`- These are based off of an `Ubuntu` image.
+* Tags without a suffix refer to `-cpu` images
+
+## Other Images
+* [`rayproject/ray`](https://hub.docker.com/repository/docker/rayproject/ray) - Ray and all of its dependencies.
diff --git a/docker/ray/README.md b/docker/ray/README.md
new file mode 100644
index 000000000..deca5499e
--- /dev/null
+++ b/docker/ray/README.md
@@ -0,0 +1,20 @@
+## About
+Default docker images for [Ray](https://github.com/ray-project/ray)! This includes
+everything needed to get started with running Ray! They work for both local development and *are ideal* for use with the [Ray Cluster Launcher](https://docs.ray.io/en/latest/cluster/launcher.html). [Find the Dockerfile here.](https://github.com/ray-project/ray/blob/master/docker/ray/Dockerfile)
+
+
+
+
+## Tags
+* [`:latest`](https://hub.docker.com/repository/docker/rayproject/ray/tags?page=1&name=latest) - The most recent Ray release.
+* `:1.x.x` - A specific release build.
+* [`:nightly`](https://hub.docker.com/repository/docker/rayproject/ray/tags?page=1&name=nightly) - The most recent nightly build.
+* `:SHA` - A specific nightly build.
+
+### Suffixes
+* `-gpu` - These are based off of an `NVIDIA CUDA` image. They require the [Nvidia Docker Runtime](https://github.com/NVIDIA/nvidia-docker) to be installed on the host for the container to access GPUs.
+* `-cpu`- These are based off of an `Ubuntu` image.
+* Tags without a suffix refer to `-cpu` images
+
+## Other Images
+* [`rayproject/ray-ml`](https://hub.docker.com/repository/docker/rayproject/ray-ml) - This image with common ML libraries to make development & deployment more smooth!