[rllib] Learner should not see clipped actions (#3496)

doc/source/rllib-env.rst

@@ -24,18 +24,21 @@ ARS             **Yes**                 **Yes**             No           No
 
 .. _`+parametric`: rllib-models.html#variable-length-parametric-action-spaces
 
-You can pass either a string name or a Python class to specify an environment. By default, strings will be interpreted as a gym `environment name <https://gym.openai.com/envs>`__. Custom env classes must take a single ``env_config`` parameter in their constructor:
+You can pass either a string name or a Python class to specify an environment. By default, strings will be interpreted as a gym `environment name <https://gym.openai.com/envs>`__. Custom env classes passed directly to the agent must take a single ``env_config`` parameter in their constructor:
 
 .. code-block:: python
 
-    import ray
+    import gym, ray
     from ray.rllib.agents import ppo
 
     class MyEnv(gym.Env):
         def __init__(self, env_config):
-            self.action_space = ...
-            self.observation_space = ...
-        ...
+            self.action_space = <gym.Space>
+            self.observation_space = <gym.Space>
+        def reset(self):
+            return <obs>
+        def step(self, action):
+            return <obs>, <reward: float>, <done: bool>, <info: dict>
 
     ray.init()
     trainer = ppo.PPOAgent(env=MyEnv, config={