Commit Graph
255 Commits
Author SHA1 Message Date
2ae2bd2b46 Print test results only if prog_bar_metrics is not empty (#1411)
* Print test results only if prog_bar_metrics is not empty

* Update evaluation_loop.py

Co-authored-by: vitor-guizilini <vitor.guizilini@tri.global>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-04-08 11:51:52 -04:00
7d0c2c7db8 Loader docs (#1416)
* added multiple loader docs

* added multiple loader docs

* added multiple loader docs

* added multiple loader docs

* added multiple loader docs

* Apply suggestions from code review

* added multiple loader docs

* added build docs script

* typo

* added build docs script

* added build docs script

* added build docs script

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-08 11:38:12 -04:00
ddbf7de6dc Added accumulation of loggers' metrics for the same steps (#1278)
* `add_argparse_args` method fixed (argument types added)

* autopep8 fixes

* --gpus=0 removed from test (for ci tests)

* Update pytorch_lightning/trainer/trainer.py

Co-Authored-By: Joe Davison <joe@huggingface.co>

* test_with_accumulate_grad_batches added

* agg_and_log_metrics logic added to the base logger class

* small format fix

* agg metrics strategies removed (not to complicate stuff)

* agg metrics: handle zero step

* autopep8

* changelog upd

* flake fix

* metrics aggregators factored out, metrics_agg.py added + tests

* metrics agg default value added

* Update pytorch_lightning/loggers/metrics_agg.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* metrics aggregators factored out, metrics_agg.py added + tests

* metrics agg default value added

* Update pytorch_lightning/loggers/metrics_agg.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* remove .item which causes sync issues (#1254)

* remove .item which causes sync issues

* fixed gradient acc sched

* fixed gradient acc sched

* test_metrics_agg.py removed (all tested in doctrings), agg metrics refactored

* test_metrics_agg.py removed (all tested in doctrings), agg metrics refactored

* autopep8

* loggers base.py types fixed

* test

* test

* metrics aggregation for loggers: each key now has a specific function (or default one)

* metrics aggregation for loggers: each key now has a specific function (or default one)

* docstrings upd

* manual typehints removed from docstrings

* batch_size decreased for test `test_with_accumulate_grad_batches`

* extend running accum

* refactor

* fix tests

* fix tests

* allowed_types generator scoped

* trainer.py distutils was imported twice, fixed

* TensorRunningAccum refactored

* TensorRunningAccum added to change log (Changed)

* change log pull link added

Co-authored-by: Joe Davison <joe@huggingface.co>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: William Falcon <waf2107@columbia.edu>
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-08 08:35:47 -04:00
Asaf Manor 09668df726 Update optimizers.py (#1383) 2020-04-07 09:09:23 -04:00
Paweł Rzepiński b8ff9bc1d2 Fix unimplemented type() on TPU (#1396)
* Fix unimplemented type() on TPU

* Add changelog entry

* Add quotation marks
2020-04-06 20:29:55 -04:00
areshytkoandAlexander Reshytko 9754c5da55 load_spawn_weights only in proc rank 0 (#1385)
Co-authored-by: Alexander Reshytko <areshytko@Alexanders-MacBook-Pro.local>
2020-04-06 10:17:16 -04:00
4ed3027309 Set precision=16 when use_amp is passed as True (#1145)
* Set precision=16 when use_amp is passed as True

* Update CHANGELOG.md

* add use_amp to deprecated API

* Update trainer.py

* Update trainer.py

* move the use_amp attribute to deprecated API

* move use_amp deprecation back to Trainer's __init__

* drop unsed

* drop deprecated

* reorder imports

* typing

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: William Falcon <waf2107@columbia.edu>
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-06 08:13:24 -04:00
Jeremy Jordan 91c9b29d47 add trainer attribute to denote if interrupted (#1368)
* add trainer attribute to denote if interrupted

* bugfix and formatting
2020-04-05 11:12:41 -04:00
Ethan HarrisandJirka Borovec b18accc64c Add warning for few workers (#1378)
* Add warning for few workers

* Fix style issue

* Update CHANGELOG.md

* Update test

* formatting

* formatting

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-04-05 11:07:16 -04:00
William FalconandShubham Agarwal 16f4cc9ff0 Shubhamagarwal92 master (#1349)
* SA: for #958: set torch cuda device when finding root

* SA: for #958: removing root gpu hack in trainer/evaluation_loop

* SA: setting torch cuda device

* comment line too long

* check if root gpu exists or available

* Incorporating suggestions on #1094

* since root gpu returns none instead of -1 for cpu

* undo changes

* fixed dp memory thing

Co-authored-by: Shubham Agarwal <shubhamagarwal92@gmail.com>
2020-04-03 17:56:19 -04:00
Justus SchockandJ. Borovec f6a86e8551 generalize reinstantiation of dataloader (#1346)
* generalize reinstantiation of dataloader

* fix condition

* add test

* update changelog

* fix changelog

Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-03 17:55:08 -04:00
Jean-Baptiste SCHIRATTI e570d2e1ca Doc fixes (#1362)
* Doc fixes from #1357 (awaelchli's comments) + changelog.

* Fix indentation.

* Add blank line to fix doc build?
2020-04-03 15:02:20 -04:00
Tullie Murrell 38e89dd890 Fix fast_dev_run running validation twice (#1365) 2020-04-03 15:00:26 -04:00
Adrian Wälchli ebd9fc9530 Fix for incorrect run on the validation set with overwritten validation_epoch_end and test_end (#1353)
* reorder if clauses

* fix wrong method overload in test

* fix formatting

* update change_log

* fix line too long
2020-04-03 09:25:32 -04:00
Jean-Baptiste SCHIRATTIandJean-Baptiste SCHIRATTI 868b172f05 Make training_epoch_end behave like validation_epoch_end (#1357)
* Make training_epoch_end behave like validation_epoch_end + minor fixes in docstrings.

* Minor fixes (Borda's comments).

* Detach tensors in batch_output (to avoid possible memory leak) + doc fix.

Co-authored-by: Jean-Baptiste SCHIRATTI <jean-baptisteschiratti@MacBook-Pro-de-Jean-Baptiste.local>
2020-04-03 14:43:26 +02:00
William Falcon 2eca8a9ef2 quick patch __code__ (#1352)
* quick patch

* testing fix

* testing fix

* testing fix

* testing fix

* testing fix

* testing fix

* testing fix

* testing fix

* testing fix

* testing fix

* testing fix

* testing fix

* testing fix
2020-04-03 08:40:02 -04:00
Santiago Castro 1576ad9963 Fix docs typo (#1355)
* Fix typo

* Fix typo
2020-04-03 07:35:09 +02:00
f33b5a8d99 Simplify progress bar args (#1108)
* show progress bar dependent on refresh_rate

* test progress_bar_refresh control show bar

* remove show_progress_bar from other tests

* borda fixes

* flake8 fix

* changelog update prog bar refresh rate

* move show_progress_bar to deprecated 0.9 api

* rm show_progress_bar references, test deprecated

* Update pytorch_lightning/trainer/__init__.py

* fix test

* changelog

* minor CHANGELOG.md format

* Update pytorch_lightning/trainer/__init__.py

* Update pytorch_lightning/trainer/trainer.py

Co-authored-by: Gerard Bentley <gbkh2015@mymail.pomona.edu>
Co-authored-by: William Falcon <waf2107@columbia.edu>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-03 00:53:00 +02:00
2912239fe6 Add useful errors when model is not configured correctly (#1199)
* add check_model_configuration method

* trying to fix errors

* trying to fix tests

* added test_epoch_end to lightning template

* fix tests

* fix new test after rebase

* fix spelling

* added more checks

* updated formating

* added tests

* fixed CHANGELOG

* Apply suggestions from code review

* move test to new module

* change check on configure_optimizers

Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-04-02 11:53:37 -04:00
Ethan HarrisandJirka Borovec 28242f02d1 Remove default optimizer, add None optimizer option (#1279)
* Add warning when using default optimizer

* Refactor optimizer tests to test_optimizers

* Remove default optimizer, add option to use no optimizer

* Update CHANGELOG.md

* Update pytorch_lightning/trainer/optimizers.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Fix style

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-04-02 11:48:53 -04:00
04935ea718 fixed extra dataloader bug (#1196)
* fixed extra dataloader bug

* Update pytorch_lightning/trainer/training_loop.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* updated CHANGELOG

* Small non-repetition change

self.get_model() => model as it was already defined

* Update CHANGELOG.md

* changed argument name to reload_train_dataloader_every_epoch

* fixed doc underline too short

* reverted to `reload_dataloaders_every_epoch`

* fixed val and test reloading

* fixed val and test reloading

Co-authored-by: TevenLeScao <teven.lescao@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-04-02 11:41:56 +02:00
William Falcon e48422df38 Sampler (#1328)
* sampler

* check for dataloader type

* check for dataloader type

* fixed sampler cases
2020-04-01 12:57:36 -04:00
William Falcon 7de51f78ac Sampler (#1318)
* sampler

* sampler

* sampler

* check for dataloader type

* check for dataloader type
2020-03-31 18:22:45 -04:00
Asaf ManorandJirka Borovec aca8c7e6f3 Optimizer Frequencies logic, and new configure_optimizers (#1269)
* init_optimizers accepts Dict, Sequence[Dict]
and returns optimizer_frequencies.
optimizer_frequencies was added as a member of Trainer.

* Optimizer frequencies logic implemented in training_loop.
Description added to configure_optimizers in LightningModule

* optimizer frequencies tests added to test_gpu

* Fixed formatting for merging PR #1269

* Apply suggestions from code review

* Apply suggestions from code review

Co-Authored-By: Asaf Manor <32155911+asafmanor@users.noreply.github.com>

* Update trainer.py

* Moving get_optimizers_iterable() outside.

* Update note

* Apply suggestions from code review

* formatting

* formatting

* Update CHANGELOG.md

* formatting

* Update CHANGELOG.md

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-03-31 16:41:24 +00:00
Jirka Borovec ee68d5ba8e reorder deprecated args (#1230) 2020-03-31 09:01:47 -04:00
Jirka Borovec 6ddb03922a Profiler summary (#1259)
* refactor and add types

* add Prorfiler summary

* fix imports

* Revert "refactor and add types"

This reverts commit b4c552fa

* changelog

* revert rename

* fix test

* mute verbose
2020-03-31 08:57:48 -04:00
Adrian Wälchliandmergify[bot] 4dcb9d3e30 fixed type hint for weights_summary arg (#1313)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2020-03-31 13:42:29 +02:00
Adrian Wälchli 1aba411da9 Early stopping when validation is disabled (#1235)
* early stop fallback to train epoch

* added test

* fix imports

* update docs

* update changelog

* fix typo
2020-03-31 06:24:26 +00:00
2ccc7456ca Error on zero length dataloaders (#1280)
* error_on_zero_length

* update CHANGELOG.md

* added test

* Update pytorch_lightning/trainer/data_loading.py

Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-03-30 18:43:53 -04:00
09167efdb5 Checkpointing interval (#1272)
* formatting

* formatting

* fix interval

* fix train loop

* fix test

* parametrize test

* Apply suggestions from code review

Co-Authored-By: Adrian Wälchli <adrian.waelchli@students.unibe.ch>

* fix calling

* flake8

* add types

Co-authored-by: Adrian Wälchli <adrian.waelchli@students.unibe.ch>
Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-03-30 18:37:02 -04:00
Jirka BorovecandWilliam Falcon 31017120fd fix incomplete RunningMean (#1309)
* fix RunningMean

* changelog

* fix none

* Update supporters.py

just needed to multiply by zero for init

* Revert "Update supporters.py"

This reverts commit 7e0da6c6

* fix NaN

* formatting

Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-03-30 18:28:31 -04:00
Jirka Borovec c869dd8b8f make evaluate private (#1260)
* make evaluate private

* changelog
2020-03-30 12:14:27 -04:00
sneimanandseth 6dfe9951e1 validation and training loops run the partial dataset (#1192)
* auto_add_sampler() fix

* auto_add_sampler() fix

Co-authored-by: seth <seth@duckpapa.com>
2020-03-30 12:13:34 -04:00
William Falcon 31b71483c4 remove .item which causes sync issues (#1254)
* remove .item which causes sync issues

* fixed gradient acc sched

* fixed gradient acc sched
2020-03-29 20:20:34 -04:00
Paweł Rzepiński b74a3c5106 Fix training resuming docs (#1265) 2020-03-29 15:29:48 -04:00
Ethan Harris ab09faa15e Add support for iterable datasets when val_check_interval=1.0 (#1283)
* Add support for iterable datasets when val_check_interval=1.0

* Update CHANGELOG.md
2020-03-29 15:27:44 -04:00
Adrian Wälchli 0b0180406b Fix outdated docs follow up (#1266)
* save_top_k

* num_gpus

* print_nan_grads fix leftovers

* undo doctest removal
2020-03-28 11:30:57 -04:00
Jack Pertschuk 12b39a74b4 Changed smoothing in tqdm to decrease variability of time remaining between training / eval (#1194) 2020-03-27 09:22:53 -04:00
Shunta Komatsu 3a93aaf9e2 Fix outdated docs (#1227) 2020-03-27 08:42:52 -04:00
Adrian Wälchli 2a4cd479e2 Disable validation when val_percent_check=0 (#1251)
* fix disable validation

* add test

* update changelog

* update docs for val_percent_check

* make "fast training" docs consistent
2020-03-27 02:07:22 +00:00
ced662fc27 Custom argparser extension with Trainer arguments (argument types added) (#1147)
* `add_argparse_args` method fixed (argument types added)

* CHANGELOG.md upd

* autopep8 fixes

* --gpus=0 removed from test (for ci tests)

* typo fixed

* reduce on plateau scheduler fixed

* Trainer cli related tests moved to test_trainer_cli.py

* refactored: get_init_arguments_and_types is a public classmethod of the Trainer now

* test_get_init_arguments_and_types added

* autopep8 fixes

* Trainer cli related tests moved to test_trainer_cli.py

* refactored: get_init_arguments_and_types is a public classmethod of the Trainer now

* test_get_init_arguments_and_types added

* autopep8 fixes

* Trainer cli related tests moved to test_trainer_cli.py

* refactored: get_init_arguments_and_types is a public classmethod of the Trainer now

* test_get_init_arguments_and_types added

* autopep8 fixes

* Trainer cli related tests moved to test_trainer_cli.py

* test_get_init_arguments_and_types added

* autopep8 fixes

* Apply suggestions from code review

* cosmetics

* cosmetics

* Update pytorch_lightning/trainer/trainer.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* `Trainer.get_init_arguments_and_types` now returns arg types wrapped in tuples (not in sets)

* deprecated args are now ignored in argparser

* get_deprecated_arg_names small refactor

* get_deprecated_arg_names bug fixed

* Trainer cli related tests moved to test_trainer_cli.py

* refactored: get_init_arguments_and_types is a public classmethod of the Trainer now

* test_get_init_arguments_and_types added

* autopep8 fixes

* Trainer cli related tests moved to test_trainer_cli.py

* autopep8 fixes

* Trainer cli related tests moved to test_trainer_cli.py

* Trainer cli related tests moved to test_trainer_cli.py

* test_get_init_arguments_and_types added

* autopep8 fixes

* autopep8 fixes

* Apply suggestions from code review

* cosmetics

* cosmetics

* Update pytorch_lightning/trainer/trainer.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* `Trainer.get_init_arguments_and_types` now returns arg types wrapped in tuples (not in sets)

* deprecated args are now ignored in argparser

* get_deprecated_arg_names small refactor

* get_deprecated_arg_names bug fixed

* Update pytorch_lightning/trainer/trainer.py

Co-Authored-By: Joe Davison <joe@huggingface.co>

* Update pytorch_lightning/trainer/trainer.py

Co-Authored-By: Joe Davison <joe@huggingface.co>

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Joe Davison <joe@huggingface.co>
Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-03-24 14:55:27 -04:00
So Uchida 60b8246bc3 Pretty test results with pprint (#1176) 2020-03-24 14:52:57 -04:00
Adrian Wälchli d735055e6f add missing overfit_pct docs (#1204)
* add missing overfit_pct docs

* move arg to old position

* move arg docs
2020-03-24 14:49:11 -04:00
Jirka BorovecandNic Eggert 3be81cb54e test deprecated - model (#1074)
* pylint

* model API

* update test

* formatting

* disable logger

* fix checking overwrite

* fix test

* typo

* deprecated model

* fix for DDP

* drop Flake8 in GH actions

* Update pytorch_lightning/trainer/evaluation_loop.py

* fix imports

Co-authored-by: Nic Eggert <nic@eggert.io>
2020-03-20 20:51:14 +01:00
792962ecc9 CI: Force docs warnings to be raised as errors (+ fix all) (#1191)
* add argument to force warn

* fix automodule error

* fix permalink error

* fix indentation warning

* fix warning

* fix import warnings

* fix duplicate label warning

* fix bullet point indentation warning

* fix duplicate label warning

* fix "import not top level" warning

* line too long

* fix indentation

* fix bullet points indentation warning

* fix hooks warnings

* fix reference problem with excluded test_tube

* fix indentation in print

* change imports for trains logger

* remove pandas type annotation

* Update pytorch_lightning/core/lightning.py

* include bullet points inside note

* remove old quick start guide (unused)

* fix unused warning

* fix formatting

* fix duplicate label issue

* fix duplicate label warning (replaced by class ref)

* fix tick

* fix indentation warnings

* docstring ticks

* remove obsolete docstring typing

* Revert "remove old quick start guide (unused)"

This reverts commit d51bb40695442c8fa11bc9df74f6db56264f7509.

* added old quick start guide to navigation

* remove unused  tutorials file

* ignore some modules that got deprecated and are not used anymore

* fix duplicate label warning

* move examples doc and exclude pl_examples from autodoc

* fix formatting for configure_optimizer

* fix no blank line warnings

* fix "see also" labels and add paramref extension

* fix more reference problems

* fix multi-gpu reference

* fix weird warning

* fix indentation and unrecognized characters in code block

* fix warning "... not included in toctree"

* fix PIL import error

* fix duplicate target "here" warning

* fix broken link

* revert accidentally moved pl_examples

* changelog

* stdout

* note some things to know

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-03-20 20:49:01 +01:00
Adrian WälchliandJirka Borovec 732eaee4d7 nan detection and intervention (#1097)
* check for nan values

* test nan detection on loss

* sys.exit

* whitespace

* detect nan and inf values in loss and params

* update

* added documentation

* moved detect nan to training loop, remove flag for print

* blank line

* test

* rename

* deprecate print_nan_grads

* deprecated print_nan_grads

* remove unused imports

* update changelog

* fix line too long

* correct deprecated version

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* raise exception instead of sysexit

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* raise exception instead of sysexit

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Update pytorch_lightning/trainer/training_tricks.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Update pytorch_lightning/trainer/training_tricks.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* fix test

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-03-19 09:24:45 -04:00
22a7264e9a improve partial Codecov (#1172)
* ignore in setup

* show report

* abs imports

* abstract pass

* cover loggers

* doctest trains

* locals

* pass

* revert tensorboard

* use tensorboardX

* revert tensorboardX

* fix trains

* Add TrainsLogger.set_credentials (#1179)

* Add TrainsLogger.set_credentials to control trains server configuration and authentication from code. Sync trains package version.
Fix CI Trains tests

* Add global TrainsLogger set_bypass_mode (#1187)

* Add global TrainsLogger set_bypass_mode skips all external communication

Co-authored-by: bmartinn <>

* rm some no-cov

Co-authored-by: Martin.B <51887611+bmartinn@users.noreply.github.com>
2020-03-19 09:14:29 -04:00
Jacob Zhong 1a73fa0b03 change default logger to dedicated one (#1064)
Fix test


Fix format

Update pytorch_lightning/__init__.py
Separate imports
2020-03-17 18:44:00 -04:00
Jirka Borovec e461ec0037 fixing Win failed import (#1163)
* version

* try fix distrib

* update try import
2020-03-16 20:50:36 -04:00
William Falcon 49d000c0c9 Docs (#1164)
* fixed docs

* fixed docs

* fixed docs
2020-03-16 20:50:14 -04:00