[rllib] Additional support for Shared Models in A3C (#866)

* Code for Supporting Shared Models

Running (with vnet modification) - needs to be tested for performance

Summaries

Small refactoring + generalized to more domains

Small fix for jenkins

Linting

linting

Addressing changes

Addressing changes

Update envs.py

Addressing changes

convnet

Merge - new model

final touches

final linting

Changing iterations back

removed extra change

changes for fast experimentation

changes to enable a2c

TEMP FOR DEBUGGING

ContinuousActions - Still doesn't work

InvertedPendulum trains with 8 workers - k=200

huber loss

Maxes for InvertedPendulum-v1 - 16w,200steps

temp: working with a2c

Back to shared model

more fixes

small

nit

LSTM to shared models

need to fix last_features

tuning pong

Best record for hitting 0 - with k=16,n=20

nit

a2cremoval

remove A2c reference and nits

nit

removed a2c vestiges

removing a2c

removing example.py

Linting

nit

* Linting + Removing vestigal code

* Final Touches

* nits

* rerun travis

python/ray/rllib/models/catalog.py

@@ -10,7 +10,6 @@ from ray.rllib.models.preprocessors import (
     NoPreprocessor, AtariRamPreprocessor, AtariPixelPreprocessor)
 from ray.rllib.models.fcnet import FullyConnectedNetwork
 from ray.rllib.models.visionnet import VisionNetwork
-from ray.rllib.models.convnet import ConvolutionalNetwork
 
 
 class ModelCatalog(object):
@@ -67,10 +66,6 @@ class ModelCatalog(object):
 
         return FullyConnectedNetwork(inputs, num_outputs, options)
 
-    @staticmethod
-    def ConvolutionalNetwork(inputs, num_outputs, options=dict()):
-        return ConvolutionalNetwork(inputs, num_outputs, options)
-
     @staticmethod
     def get_preprocessor(env_name, obs_shape):
         """Returns a suitable processor for the given environment.