Multiagent model using concatenated observations (#1416)

* working multi action distribution and multiagent model

* currently working but the splits arent done in the right place

* added shared models

* added categorical support and mountain car example

* now compatible with generalized advantage estimation

* working multiagent code with discrete and continuous example

* moved reshaper to utils

* code review changes made, ppo action placeholder moved to model catalog, all multiagent code moved out of fcnet

* added examples in

* added PEP8 compliance

* examples are mostly pep8 compliant

* removed all flake errors

* added examples to jenkins tests

* fixed custom options bug

* added lines to let docker file find multiagent tests

* shortened example run length

* corrected nits

* fixed flake errors

python/ray/rllib/utils/sampler.py

@@ -5,6 +5,7 @@ from __future__ import print_function
 import six.moves.queue as queue
 import threading
 from collections import namedtuple
+import numpy as np
 
 
 class PartialRollout(object):
@@ -226,6 +227,10 @@ def _env_runner(env, policy, num_local_steps, horizon, obs_filter):
             if length >= horizon:
                 terminal = True
 
+            # Concatenate multiagent actions
+            if isinstance(action, list):
+                action = np.concatenate(action, axis=0).flatten()
+
             # Collect the experience.
             rollout.add(observations=last_observation,
                         actions=action,