[rllib] Add a simple REST policy server and client example (#2232)

* wip * cls * re * wip * wip * a3c working * torch support * pg works * lint * rm v2 * consumer id * clean up pg * clean up more * fix python 2.7 * tf session management * docs * dqn wip * fix compile * dqn * apex runs * up * impotrs * ddpg * quotes * fix tests * fix last r * fix tests * lint * pass checkpoint restore * kwar * nits * policy graph * fix yapf * com * class * pyt * vectorization * update * test cpe * unit test * fix ddpg2 * changes * wip * args * faster test * common * fix * add alg option * batch mode and policy serving * multi serving test * todo * wip * serving test * doc async env * num envs * comments * thread * remove init hook * update * policy serve * spaces * checkpoint * no train * fix ppo * comments1 * fix * updates * add jenkins tests * fix * fix pytorch * fix * fixes * fix a3c policy * fix squeeze * fix trunc on apex * fix squeezing for real * update * remove horizon test for now * fix race condition * update * com * updat * add test * Update run_multi_node_tests.sh * use curl * curl * kill * Update run_multi_node_tests.sh * Update run_multi_node_tests.sh * fix import * update
2026-07-09 21:18:08 +08:00 · 2018-06-20 13:22:39 -07:00
parent 418cd6804a
commit e5724a9cfe
15 changed files with 384 additions and 78 deletions
@@ -188,7 +188,7 @@ def _env_runner(

    while True:
        # Get observations from ready envs
-        unfiltered_obs, rewards, dones, _, off_policy_actions = \
+        unfiltered_obs, rewards, dones, infos, off_policy_actions = \
            async_vector_env.poll()
        ready_eids = []
        ready_obs = []
@@ -216,24 +216,25 @@ def _env_runner(
            else:
                done = False

-            episode.batch_builder.add_values(
-                obs=episode.last_observation,
-                actions=episode.last_action_flat(),
-                rewards=rewards[eid],
-                dones=done,
-                new_obs=filtered_obs,
-                **episode.last_pi_info)
+            if infos[eid].get("training_enabled", True):
+                episode.batch_builder.add_values(
+                    obs=episode.last_observation,
+                    actions=episode.last_action_flat(),
+                    rewards=rewards[eid],
+                    dones=done,
+                    new_obs=filtered_obs,
+                    **episode.last_pi_info)

-            # Cut the batch if we're not packing multiple episodes into one,
-            # or if we've exceeded the requested batch size.
-            if (done and not pack) or \
-                    episode.batch_builder.count >= num_local_steps:
-                yield episode.batch_builder.build_and_reset(
-                    policy.postprocess_trajectory)
-            elif done:
-                # Make sure postprocessor never goes across episode boundaries
-                episode.batch_builder.postprocess_batch_so_far(
-                    policy.postprocess_trajectory)
+                # Cut the batch if we're not packing multiple episodes into
+                # one, or if we've exceeded the requested batch size.
+                if (done and not pack) or \
+                        episode.batch_builder.count >= num_local_steps:
+                    yield episode.batch_builder.build_and_reset(
+                        policy.postprocess_trajectory)
+                elif done:
+                    # Make sure postprocessor never crosses episode boundaries
+                    episode.batch_builder.postprocess_batch_so_far(
+                        policy.postprocess_trajectory)

            if done:
                # Handle episode termination