[rllib] Document auto-concat in a3c (#2533)

* docs

* update hyperparm docs

doc/source/rllib-training.rst

@@ -41,7 +41,7 @@ can be used) and for choosing the algorithm with ``--run``
 Specifying Parameters
 ~~~~~~~~~~~~~~~~~~~~~
 
-Each algorithm has specific hyperparameters that can be set with ``--config``. See the
+Each algorithm has specific hyperparameters that can be set with ``--config``, in addition to a number of `common hyperparameters <https://github.com/ray-project/ray/blob/master/python/ray/rllib/agents/agent.py>`__. See the
 `algorithms documentation <rllib-algorithms.html>`__ for more information.
 
 In an example below, we train A3C by specifying 8 workers through the config flag.

python/ray/rllib/agents/a3c/a3c.py

@@ -30,7 +30,8 @@ DEFAULT_CONFIG = with_common_config({
     "use_gpu_for_workers": False,
     # Whether to emit extra summary stats
     "summarize": False,
-    # Workers sample async
+    # Workers sample async. Note that this increases the effective
+    # sample_batch_size by up to 5x due to async buffering of batches.
     "sample_async": True,
     # Model and preprocessor options
     "model": {