* Add tests for distributed backend config
* Refactor set_distributed_mode
* Use gloo backend on cpu
* Use 127.0.0.1 instead of 127.0.0.2
Not totally clear on why this is necessary, but it seemt to work
* Update LightningDDP so that it works with CPU
* Add ddp_cpu backend and num_processes Trainer arg
* PEP8
* Fix test skipping. Inequalities are hard :/
* Skip ddp_cpu test on Windows
* Make a few more cases fall back to ddp_cpu
* New function name
* Flake8
* Don't test distributed on MacOS with torch < 1.3
Support for distributed in MacOS was added in Torch 1.3.0
* Add ddp_cpu and num_processes to docs
* Parametrize trainer config tests
* Tweak warning
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Remove redundant test
* Replace pass branches with comments
* Add missing warnings import
* save_path -> root_dir
* Use new rank_zero_warn
* Whitespace
* Apply suggestions from code review
* formatting
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
* remove error when test dataloader used in test
* remove error when test dataloader used in test
* remove error when test dataloader used in test
* remove error when test dataloader used in test
* remove error when test dataloader used in test
* remove error when test dataloader used in test
* fix lost model reference
* remove error when test dataloader used in test
* fix lost model reference
* moved optimizer types
* moved optimizer types
* moved optimizer types
* moved optimizer types
* moved optimizer types
* moved optimizer types
* moved optimizer types
* moved optimizer types
* added tests for warning
* fix lost model reference
* fix lost model reference
* added tests for warning
* added tests for warning
* refactoring
* refactoring
* fix imports
* refactoring
* fix imports
* refactoring
* fix tests
* fix mnist
* flake8
* review
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
* Add automatic GPU choice to trainer
This commit adds the `gpu_choice` parameter to Trainer. By default,
this parameter is set to 'manual' which causes no observable
difference in behavior.
When `gpu_choice` is set to "auto" and `gpus` is an int, then the
trainer will automatically allocate the first available GPU.
This is especially useful when GPUs are configured to be in "exclusive
mode", which means that only one process at a time can use them.
* Rename gpu_choice -> auto_select_gpus
* Allow reinits in sub procs
* Dont create an experiment on pickle, name, or project
* Comments consistency
* Fix test
* Apply suggestions from code review
Co-authored-by: Chris Van Pelt <vanpelt@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Make training_epoch_end behave like validation_epoch_end + minor fixes in docstrings.
* Minor fixes (Borda's comments).
* Detach tensors in batch_output (to avoid possible memory leak) + doc fix.
Co-authored-by: Jean-Baptiste SCHIRATTI <jean-baptisteschiratti@MacBook-Pro-de-Jean-Baptiste.local>
* show progress bar dependent on refresh_rate
* test progress_bar_refresh control show bar
* remove show_progress_bar from other tests
* borda fixes
* flake8 fix
* changelog update prog bar refresh rate
* move show_progress_bar to deprecated 0.9 api
* rm show_progress_bar references, test deprecated
* Update pytorch_lightning/trainer/__init__.py
* fix test
* changelog
* minor CHANGELOG.md format
* Update pytorch_lightning/trainer/__init__.py
* Update pytorch_lightning/trainer/trainer.py
Co-authored-by: Gerard Bentley <gbkh2015@mymail.pomona.edu>
Co-authored-by: William Falcon <waf2107@columbia.edu>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
* fix RunningMean
* changelog
* fix none
* Update supporters.py
just needed to multiply by zero for init
* Revert "Update supporters.py"
This reverts commit 7e0da6c6
* fix NaN
* formatting
Co-authored-by: William Falcon <waf2107@columbia.edu>
* added custom mnist without torchvision dep
* move files so it does not conflict with mnist gitignore
* mock torchvision for tests
* fix line too long
* fix line too long
* fix "module level import not at top of file" warning
* move mock imports to __init__.py
* simplify MNIST a lot and download directly the .pt files
* further simplify and clean up mnist
* revert import overrides
* make as before
* drop PIL requirement
* move mnist.py to datasets subfolder
* use logging instead of print
* choose same name as in torchvision
* remove torchvision and pillow also from yml file
* refactor if train
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* capitalized class attr
* moved mnist to models
* re-added datsets ignore
* better name for file variable
* Update mnist.py
* move dataset classes to datasets.py
* new line
* update
* update
* fix automerge
* move to base folder
* adapt testingmnist to new mnist base class
* remove temporal fix
* fix datatype
* remove old testingmnist
* readable
* fix import
* fix whitespace
* docstring
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/base/datasets.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* changelog
* added types
* Update CHANGELOG.md
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* exist->isfile
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* index -> idx
* temporary fix for trains error
* better changelog message
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* `add_argparse_args` method fixed (argument types added)
* CHANGELOG.md upd
* autopep8 fixes
* --gpus=0 removed from test (for ci tests)
* typo fixed
* reduce on plateau scheduler fixed
* Trainer cli related tests moved to test_trainer_cli.py
* refactored: get_init_arguments_and_types is a public classmethod of the Trainer now
* test_get_init_arguments_and_types added
* autopep8 fixes
* Trainer cli related tests moved to test_trainer_cli.py
* refactored: get_init_arguments_and_types is a public classmethod of the Trainer now
* test_get_init_arguments_and_types added
* autopep8 fixes
* Trainer cli related tests moved to test_trainer_cli.py
* refactored: get_init_arguments_and_types is a public classmethod of the Trainer now
* test_get_init_arguments_and_types added
* autopep8 fixes
* Trainer cli related tests moved to test_trainer_cli.py
* test_get_init_arguments_and_types added
* autopep8 fixes
* Apply suggestions from code review
* cosmetics
* cosmetics
* Update pytorch_lightning/trainer/trainer.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* `Trainer.get_init_arguments_and_types` now returns arg types wrapped in tuples (not in sets)
* deprecated args are now ignored in argparser
* get_deprecated_arg_names small refactor
* get_deprecated_arg_names bug fixed
* Trainer cli related tests moved to test_trainer_cli.py
* refactored: get_init_arguments_and_types is a public classmethod of the Trainer now
* test_get_init_arguments_and_types added
* autopep8 fixes
* Trainer cli related tests moved to test_trainer_cli.py
* autopep8 fixes
* Trainer cli related tests moved to test_trainer_cli.py
* Trainer cli related tests moved to test_trainer_cli.py
* test_get_init_arguments_and_types added
* autopep8 fixes
* autopep8 fixes
* Apply suggestions from code review
* cosmetics
* cosmetics
* Update pytorch_lightning/trainer/trainer.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* `Trainer.get_init_arguments_and_types` now returns arg types wrapped in tuples (not in sets)
* deprecated args are now ignored in argparser
* get_deprecated_arg_names small refactor
* get_deprecated_arg_names bug fixed
* Update pytorch_lightning/trainer/trainer.py
Co-Authored-By: Joe Davison <joe@huggingface.co>
* Update pytorch_lightning/trainer/trainer.py
Co-Authored-By: Joe Davison <joe@huggingface.co>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Joe Davison <joe@huggingface.co>
Co-authored-by: William Falcon <waf2107@columbia.edu>
* pylint
* model API
* update test
* formatting
* disable logger
* fix checking overwrite
* fix test
* typo
* deprecated model
* fix for DDP
* drop Flake8 in GH actions
* Update pytorch_lightning/trainer/evaluation_loop.py
* fix imports
Co-authored-by: Nic Eggert <nic@eggert.io>
* check for nan values
* test nan detection on loss
* sys.exit
* whitespace
* detect nan and inf values in loss and params
* update
* added documentation
* moved detect nan to training loop, remove flag for print
* blank line
* test
* rename
* deprecate print_nan_grads
* deprecated print_nan_grads
* remove unused imports
* update changelog
* fix line too long
* correct deprecated version
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* raise exception instead of sysexit
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* raise exception instead of sysexit
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/trainer/training_tricks.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/trainer/training_tricks.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
* fix test
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
William Falcon