Support for multiple val_dataloaders (#97)

* Added support for multiple validation dataloaders

* Fix typo in README.md

* Update trainer.py

* Add support for multiple dataloaders

* Rename dataloader_index to dataloader_i

* Added warning to check val_dataloaders

Added a warning to ensure that all val_dataloaders were DistributedSamplers if ddp is enabled

* Updated DistributedSampler warning

* Fixed typo

* Added multiple val_dataloaders

* Multiple val_dataloader test

* Update lightning_module_template.py

Added dataloader_i to validation_step parameters

* Update trainer.py

* Reverted template changes

* Create multi_val_module.py

* Update no_val_end_module.py

* New MultiValModel

* Rename MultiValModel to MultiValTestModel

* Revert to LightningTestModel

* Update test_models.py

* Update trainer.py

* Update test_models.py

* multiple val_dataloaders in test template

* Fixed flake8 warnings

* Update trainer.py

* Fix flake errors

* Fixed Flake8 errors

* Update lm_test_module.py

keep this test model with a single dataset for val

* Update trainer.py

* Update trainer.py

* Update trainer.py

* Update trainer.py

* Update trainer.py

* Update test_models.py

* Update trainer.py

* Update trainer.py

* Update trainer.py

* Update trainer.py

* Update trainer.py

* Update trainer.py

* Update RequiredTrainerInterface.md

* Update RequiredTrainerInterface.md

* Update test_models.py

* Update trainer.py

dont need the else clause, val_dataloader is either a list or none because of get_dataloaders()

* Update trainer.py

fixed flake errors

* Update trainer.py

tests/test_models.py

@@ -252,7 +252,7 @@ def test_cpu_restore_training():
         # if model and state loaded correctly, predictions will be good even though we
         # haven't trained with the new loaded model
         trainer.model.eval()
-        run_prediction(trainer.val_dataloader, trainer.model)
+        _ = [run_prediction(dataloader, trainer.model) for dataloader in trainer.val_dataloader]
 
     model.on_sanity_check_start = assert_good_acc
 
@@ -761,6 +761,37 @@ def test_ddp_sampler_error():
     clear_save_dir()
 
 
+def test_multiple_val_dataloader():
+    """
+    Verify multiple val_dataloader
+    :return:
+    """
+    hparams = get_hparams()
+    model = LightningTemplateModel(hparams)
+
+    save_dir = init_save_dir()
+
+    # exp file to get meta
+    trainer_options = dict(
+        max_nb_epochs=1,
+        val_percent_check=0.1,
+        train_percent_check=0.1,
+    )
+
+    # fit model
+    trainer = Trainer(**trainer_options)
+    result = trainer.fit(model)
+
+    # verify tng completed
+    assert result == 1
+
+    # verify there are 2 val loaders
+    assert len(trainer.val_dataloader) == 2, 'Multiple val_dataloaders not initiated properly'
+
+    # make sure predictions are good for each val set
+    [run_prediction(dataloader, trainer.model) for dataloader in trainer.val_dataloader]
+
+
 # ------------------------------------------------------------------------
 # UTILS
 # ------------------------------------------------------------------------