74 Commits
Author SHA1 Message Date
Jirka Borovec bee0392c37 extend arg parser (#1842)
* extend arg parser

* flake8

* tests

* example

* fix test
2020-05-14 17:56:11 -04:00
William Falcon 53d9316a56 fixes ddp bugs (#1819)
* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug
2020-05-13 19:17:04 -04:00
William FalconandNate Raw 648d516668 Use store_true for bool args (#1822)
*  Use store_true for bool args

* debug

Co-authored-by: Nate Raw <nxr9266@g.rit.edu>
2020-05-13 19:12:06 -04:00
663b90035c Bugfix: accumulation and suggestion for learning rate finder (#1801)
* fix suggestion being too naive

* fix accumulation error and added new tests

* fix styling

* update CHANGELOG.md

* update based on review

* fix tests

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-05-13 14:40:44 -04:00
22d7d03118 Replace meta_tags.csv with hparams.yaml (#1271)
* Add support for hierarchical dict

* Support nested Namespace

* Add docstring

* Migrate hparam flattening to each logger

* Modify URLs in CHANGELOG

* typo

* Simplify the conditional branch about Namespace

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Update CHANGELOG.md

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* added examples section to docstring

* renamed _dict -> input_dict

* mata_tags.csv -> hparams.yaml

* code style fixes

* add pyyaml

* remove unused import

* create the member NAME_HPARAMS_FILE

* improve tests

* Update tensorboard.py

* pass the local test w/o relavents of Horovod

* formatting

* update dependencies

* fix dependencies

* Apply suggestions from code review

* add savings

* warn

* docstrings

* tests

* Apply suggestions from code review

* saving

* Apply suggestions from code review

* use default

* remove logging

* typo fixes

* update docs

* update CHANGELOG

* clean imports

* add blank lines

* Update pytorch_lightning/core/lightning.py

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Update pytorch_lightning/core/lightning.py

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* back to namespace

* add docs

* test fix

* update dependencies

* add space

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2020-05-13 15:05:15 +02:00
William FalconandJirka 10b16dbfab made ddp the default if no backend specified with multiple GPUs (#1789)
* made ddp the default if no backend specified with multiple GPUs

* fix

* spawn

Co-authored-by: Jirka <jirka.borovec@seznam.cz>
2020-05-12 06:54:23 -04:00
Kevin Chen de1fdd8d3b Removed test_dataloader call in check_testing_model_configuration (#1670)
* Removed test_dataloader call

* Check if test_dataloader is actually overriden

* Fixed method spelling

* Replaced lambdas

* Replaced None with super method

* Fixed testpass
2020-05-12 00:08:07 -04:00
William Falcon 5bb6b41b78 dataloaders with fast_dev_run (#1787)
* dataloaders with fast_dev_run

* dataloaders with fast_dev_run

* dataloaders with fast_dev_run

* fix

* pep 8
2020-05-11 23:32:44 -04:00
Jirka Borovec 134eb61e1a Tests: refactor cleanup (#1744)
* wip

* cleaning

* optim imports

* -

* default hparams

* fix restore

* fix imports
2020-05-10 13:15:28 -04:00
4970927ec8 Feature: auto scale batch size (#1638)
* auto batch finder

* fix styling

* add description

* add different modes

* fix copy paste error

* better organised code

* fix styling

* add tests

* fix

* fix

* add some documentation

* added CHANGELOG.md

* some documentation

* update based on review

* Update trainer.py

* Update docs/source/training_tricks.rst

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Update tests/trainer/test_trainer_tricks.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update tests/trainer/test_trainer_tricks.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* use EvalModelTemplate

* param tests

* rename

* wrap params

* rename function

* rename

* rename param

* fix

* abs

* rename

* refactor code

* add docs

* try

* arg

* loop

* exept

* loop

* drop bool

* docs

* docs

* added check and test for passing dataloader to fit

* styling fix

* update based on review

Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: William Falcon <waf2107@columbia.edu>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
2020-05-09 08:28:36 -04:00
Shunta Komatsu f656882942 Fix typo (#1750) 2020-05-07 09:25:54 -04:00
Adrian Wälchli 2b03d34931 complete test (#1705) 2020-05-05 14:08:15 -04:00
Jirka BorovecandWilliam Falcon 2a2f303ae9 Tests: refactor trainer dataloaders (#1690)
* refactor default model

* drop redundant seeds

* refactor dataloaders tests

* fix multiple

* fix conf

* flake8

* Apply suggestions from code review

Co-authored-by: William Falcon <waf2107@columbia.edu>

Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-05-05 12:31:15 -04:00
Jirka Borovec 043ae697c2 Tests: refactor callbacks (#1688)
* refactor default model

* drop redundant seeds

* path

* refactor callback tests

* update

* fix sch

* wip

* fix return

* review
2020-05-04 16:52:22 -04:00
Jirka Borovec 6d58fb1353 Tests: refactor trainer (#1728)
* lr

* optim

* wip

* wip

* fix mean

* flake8
2020-05-04 16:51:39 -04:00
Nicki SkafteandNicki Skafte e865b046b1 Bugfix/lr finder (#1676)
* fix early stopping bug

* allow val dataloader

* update CHANGELOG.md

* fix early stopping bug

* allow val dataloader

* update CHANGELOG.md

Co-authored-by: Nicki Skafte <nugginea@gmail.com>
2020-05-04 11:38:51 -04:00
Adrian Wälchli d28b145393 Update type hints for multiple dataloaders in .fit() and .test() (#1723)
* update typehints

* change log
2020-05-04 08:24:34 -04:00
Jirka Borovec 595ec65796 refactor trainer checks (#1651)
* refactor trainer checks

* opt

* none

* Apply suggestions from code review

* imports

* fix tensors
2020-05-04 01:25:48 -04:00
Adrian WälchliandJ. Borovec e6b34ef90d [WIP] Reduction when batch size < num gpus (#1609)
* reduce if <= num_gpus

* add test with explanation

* chlog

* fix changelog

Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-05-02 11:01:44 -04:00
Jirka Borovec f380027951 refactor default model (#1652)
* refactor default model

* drop redundant seeds

* formatting

* path

* formatting

* rename
2020-05-02 08:38:22 -04:00
Jirka Borovec 34bc149359 move unnecessary dict trainer_options (#1469)
* move unnecessary dict trainer_options

* fix tests

* fix tests

* formatting

* missing
2020-05-01 10:43:58 -04:00
Peter Yu 8d564b5e38 call on_load_checkpoint() when resuming from checkpoint (#1666) 2020-04-30 07:57:24 -04:00
William Falcon e7ea564df2 pep8 2020-04-27 08:47:19 -04:00
William Falcon 710dbf5a12 ddp pickle 2020-04-27 08:04:02 -04:00
William Falcon a3cebb4469 ddp pickle 2020-04-27 08:04:02 -04:00
William Falcon 5c0118fe9d ddp pickle 2020-04-27 08:03:32 -04:00
William Falcon b993a3ed39 ddp pickle 2020-04-27 07:21:58 -04:00
William Falcon e7a0d98277 fix hparams issue 2020-04-26 17:34:33 -04:00
William Falcon d40b4c8f80 fix hparams issue 2020-04-26 17:30:50 -04:00
Jirka Borovec 570b2c7aeb fix depreated call (#1596)
* fix parity

* update deprecated call
2020-04-24 14:45:43 -04:00
3e8f2d99a9 Progress bar callback (#1450)
* squash and rebase

sanity check hooks


sanity check callback hook finish


moved core progress bar functionality into callback


wip


remove duplicate merge


clean up


imports


docs


sanity check progress bar main


sanity


move callback calls


init progrss bar callback


configuration and docs


changelog


rate decorator


pass process_position


disable on rank > 0


position index


is_enabled


remove decorator


refactor init tqdm bars


callback method ordering 


cannot reset when disabled


sequence -> list


default values


fix has no attr _time() 


move on_val_end to proper place


fix the pickle issue


update warning


properties


check for None


remove old comment


switch order


pull out non-tqdm functionality into base class


documentation for the base class


docs


fix refresh rate issue in validation


restrict type hint of trainer arg


more docs


update trainer docs


rst docs


fix lines too long


fix test


add missing type hints


fix typo


move docstring to __init__ solves doctest failures


remove doctest :(( can't fix the pickle error


fix example


simplify by saving trainer reference


fix docs errors


move docstring


initial value


multiple val checks per epoch


simpler handling of inf dataset sizes


update inf docs


renamed training_tqdm_dict


rename get_tqdm_dict


rename occurences of tqdm 


update changelog


fix doctest


fix formatting errors


added callback tests


progress bar on off test


more tests for progress bar


weird test fix?


add ignored property


disable default progress bar in LR finder


change enable/disable behavior


trying doctest in CI again


undo doctest pickle error


undo doctest pickle error :((


remove progress_bar_callback Trainer arg and fix tests


restore progress bar after auto lr find


update docs


fix rebase


fix wrong negation

* fix fast dev run total

* more thorough testing

* remove old args

* fix merge

* fix merge

* separate tests

* type hint total batches

* reduce if

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* is_disabled

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* is_enabled

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* rename enabled/disabled

* move deprecated api

* remove duplicated test from merge

* fix rename is_disabled

* newline

* test also testprogress for fast dev run

Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-04-23 20:46:18 -04:00
Nicki SkafteandNicki Skafte e977d1cde5 Default value for ModelCheckpoint filepath (#1548)
* allow determine of filepath at runtime

* typing

Co-authored-by: Nicki Skafte <nugginea@gmail.com>
2020-04-23 11:50:58 -04:00
Jirka Borovec c1c6e3b6c9 default test logger (#1478)
* default test logger

* fix tests

* spawn

* try

* simplify tests

* simplify tests

* formatting

* loggers

* loggers

* revert to TestTube

* default

* default

* wraps

* world size

* optim imports
2020-04-21 20:33:10 -04:00
Adrian Wälchli 452fa858f4 skip warning test (#1533) 2020-04-20 08:04:37 +00:00
e3001a0929 Add ddp_cpu backend for testing ddp without GPUs (#1158)
* Add tests for distributed backend config

* Refactor set_distributed_mode

* Use gloo backend on cpu

* Use 127.0.0.1 instead of 127.0.0.2

Not totally clear on why this is necessary, but it seemt to work

* Update LightningDDP so that it works with CPU

* Add ddp_cpu backend and num_processes Trainer arg

* PEP8

* Fix test skipping. Inequalities are hard :/

* Skip ddp_cpu test on Windows

* Make a few more cases fall back to ddp_cpu

* New function name

* Flake8

* Don't test distributed on MacOS with torch < 1.3

Support for distributed in MacOS was added in Torch 1.3.0

* Add ddp_cpu and num_processes to docs

* Parametrize trainer config tests

* Tweak warning

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Remove redundant test

* Replace pass branches with comments

* Add missing warnings import

* save_path -> root_dir

* Use new rank_zero_warn

* Whitespace

* Apply suggestions from code review

* formatting

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-15 23:17:31 -04:00
William FalconandJ. Borovec 3431c62d41 Remove error when test dataloader used in test (#1495)
* remove error when test dataloader used in test

* remove error when test dataloader used in test

* remove error when test dataloader used in test

* remove error when test dataloader used in test

* remove error when test dataloader used in test

* remove error when test dataloader used in test

* fix lost model reference

* remove error when test dataloader used in test

* fix lost model reference

* moved optimizer types

* moved optimizer types

* moved optimizer types

* moved optimizer types

* moved optimizer types

* moved optimizer types

* moved optimizer types

* moved optimizer types

* added tests for warning

* fix lost model reference

* fix lost model reference

* added tests for warning

* added tests for warning

* refactoring

* refactoring

* fix imports

* refactoring

* fix imports

* refactoring

* fix tests

* fix mnist

* flake8

* review

Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-15 22:16:40 -04:00
Ethan Harris 8544b334e4 Replace automatic nan check with optional flag (#1475)
* Replace automatic nan check with optional flag

* Update CHANGELOG.md
2020-04-13 14:06:25 -04:00
3f09b32df3 Learning Rate finder (#1347)
* initial structure

* rebase

* incorporate suggestions

* update CHANGELOG.md

* initial docs

* fixes based on reviews

* added trainer arg

* update docs

* added saving/restore of model state

* initial tests

* fix styling

* added more tests

* fix docs, backward compatility and progressbar

* fix styling

* docs update

* updates based on review

* changed saving to standard functions

* consistent naming

* fix formatting

* improve docs, added support for nested fields, improve codecov

* update CHANGELOG.md

* Update lr_finder.rst

* Update pytorch_lightning/trainer/trainer.py

* Update trainer.py

* Update CHANGELOG.md

* Update path

* restoring

* test

* attribs

* docs

* doc typo

Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: William Falcon <waf2107@columbia.edu>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-10 14:34:23 -04:00
William Falcon b78c3d4da8 Fix weights path (#1445)
* renamed default path to actual root_dir

* added default weights path

* added default weights path

* added default weights path
2020-04-10 12:02:59 -04:00
Allard Hendriksen 7ac1580a31 Add automatic GPU choice to trainer (#1426)
* Add automatic GPU choice to trainer

This commit adds the `gpu_choice` parameter to Trainer. By default,
this parameter is set to 'manual' which causes no observable
difference in behavior.

When `gpu_choice` is set to "auto" and `gpus` is an int, then the
trainer will automatically allocate the first available GPU.
This is especially useful when GPUs are configured to be in "exclusive
mode", which means that only one process at a time can use them.

* Rename gpu_choice -> auto_select_gpus
2020-04-10 11:45:29 -04:00
Rohit Gupta e79ae18cae Add test_dataloaders to test method (#1434)
* Add test_dataloaders to test method

* Remove test_dataloaders from .fit()

* Fix code comment

* Fix tests

* Add test_dataloaders to test method (#1393)

* Fix failing tests

* Update docs (#1393)
2020-04-10 11:44:03 -04:00
Alexey Karnachev 4c34d16a34 Fixed configure optimizer from dict without "scheduler" key (#1443)
* `configure_optimizer` from dict with only "optimizer" key. bug fixed

* autopep8

* pep8speaks suggested fixes

* CHANGELOG.md upd
2020-04-10 11:43:06 -04:00
Alex Sergeev 8dd9b80d7a Fix gradient clipping (#1438)
* Fix gradient clipping

* Relax accuracy constraint
2020-04-09 21:08:28 -04:00
ddbf7de6dc Added accumulation of loggers' metrics for the same steps (#1278)
* `add_argparse_args` method fixed (argument types added)

* autopep8 fixes

* --gpus=0 removed from test (for ci tests)

* Update pytorch_lightning/trainer/trainer.py

Co-Authored-By: Joe Davison <joe@huggingface.co>

* test_with_accumulate_grad_batches added

* agg_and_log_metrics logic added to the base logger class

* small format fix

* agg metrics strategies removed (not to complicate stuff)

* agg metrics: handle zero step

* autopep8

* changelog upd

* flake fix

* metrics aggregators factored out, metrics_agg.py added + tests

* metrics agg default value added

* Update pytorch_lightning/loggers/metrics_agg.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* metrics aggregators factored out, metrics_agg.py added + tests

* metrics agg default value added

* Update pytorch_lightning/loggers/metrics_agg.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* remove .item which causes sync issues (#1254)

* remove .item which causes sync issues

* fixed gradient acc sched

* fixed gradient acc sched

* test_metrics_agg.py removed (all tested in doctrings), agg metrics refactored

* test_metrics_agg.py removed (all tested in doctrings), agg metrics refactored

* autopep8

* loggers base.py types fixed

* test

* test

* metrics aggregation for loggers: each key now has a specific function (or default one)

* metrics aggregation for loggers: each key now has a specific function (or default one)

* docstrings upd

* manual typehints removed from docstrings

* batch_size decreased for test `test_with_accumulate_grad_batches`

* extend running accum

* refactor

* fix tests

* fix tests

* allowed_types generator scoped

* trainer.py distutils was imported twice, fixed

* TensorRunningAccum refactored

* TensorRunningAccum added to change log (Changed)

* change log pull link added

Co-authored-by: Joe Davison <joe@huggingface.co>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: William Falcon <waf2107@columbia.edu>
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-08 08:35:47 -04:00
Jeremy Jordan 91c9b29d47 add trainer attribute to denote if interrupted (#1368)
* add trainer attribute to denote if interrupted

* bugfix and formatting
2020-04-05 11:12:41 -04:00
Ethan HarrisandJirka Borovec b18accc64c Add warning for few workers (#1378)
* Add warning for few workers

* Fix style issue

* Update CHANGELOG.md

* Update test

* formatting

* formatting

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-04-05 11:07:16 -04:00
Justus SchockandJ. Borovec f6a86e8551 generalize reinstantiation of dataloader (#1346)
* generalize reinstantiation of dataloader

* fix condition

* add test

* update changelog

* fix changelog

Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-03 17:55:08 -04:00
Adrian Wälchli ebd9fc9530 Fix for incorrect run on the validation set with overwritten validation_epoch_end and test_end (#1353)
* reorder if clauses

* fix wrong method overload in test

* fix formatting

* update change_log

* fix line too long
2020-04-03 09:25:32 -04:00
f33b5a8d99 Simplify progress bar args (#1108)
* show progress bar dependent on refresh_rate

* test progress_bar_refresh control show bar

* remove show_progress_bar from other tests

* borda fixes

* flake8 fix

* changelog update prog bar refresh rate

* move show_progress_bar to deprecated 0.9 api

* rm show_progress_bar references, test deprecated

* Update pytorch_lightning/trainer/__init__.py

* fix test

* changelog

* minor CHANGELOG.md format

* Update pytorch_lightning/trainer/__init__.py

* Update pytorch_lightning/trainer/trainer.py

Co-authored-by: Gerard Bentley <gbkh2015@mymail.pomona.edu>
Co-authored-by: William Falcon <waf2107@columbia.edu>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-03 00:53:00 +02:00
Jirka Borovec 724b787cd1 faster CI testing (#1323)
* MNIST digits

* increase test acc

* smaller parity

* drone builds

* increase GH action timeout

* drone format

* fix paths

* drone cache

* circle cache

* fix test

* lower nb epochs

* circleCI

* user orb

* fix test

* fix test

* circle cache

* circle cache

* circle cache

* comment caches

* benchmark batch size

* cache dataset

* smaller dataset

* smaller dataset

* fix nb samples

* batch size

* fix test
2020-04-02 12:28:44 -04:00