Commit Graph
2181 Commits
Author SHA1 Message Date
William Falcon 890458fdbd Fixes automatic parser bug (#1585)
* fixes gpu parsing

* fixes gpu parsing
2020-04-23 21:00:41 -04:00
3e8f2d99a9 Progress bar callback (#1450)
* squash and rebase

sanity check hooks


sanity check callback hook finish


moved core progress bar functionality into callback


wip


remove duplicate merge


clean up


imports


docs


sanity check progress bar main


sanity


move callback calls


init progrss bar callback


configuration and docs


changelog


rate decorator


pass process_position


disable on rank > 0


position index


is_enabled


remove decorator


refactor init tqdm bars


callback method ordering 


cannot reset when disabled


sequence -> list


default values


fix has no attr _time() 


move on_val_end to proper place


fix the pickle issue


update warning


properties


check for None


remove old comment


switch order


pull out non-tqdm functionality into base class


documentation for the base class


docs


fix refresh rate issue in validation


restrict type hint of trainer arg


more docs


update trainer docs


rst docs


fix lines too long


fix test


add missing type hints


fix typo


move docstring to __init__ solves doctest failures


remove doctest :(( can't fix the pickle error


fix example


simplify by saving trainer reference


fix docs errors


move docstring


initial value


multiple val checks per epoch


simpler handling of inf dataset sizes


update inf docs


renamed training_tqdm_dict


rename get_tqdm_dict


rename occurences of tqdm 


update changelog


fix doctest


fix formatting errors


added callback tests


progress bar on off test


more tests for progress bar


weird test fix?


add ignored property


disable default progress bar in LR finder


change enable/disable behavior


trying doctest in CI again


undo doctest pickle error


undo doctest pickle error :((


remove progress_bar_callback Trainer arg and fix tests


restore progress bar after auto lr find


update docs


fix rebase


fix wrong negation

* fix fast dev run total

* more thorough testing

* remove old args

* fix merge

* fix merge

* separate tests

* type hint total batches

* reduce if

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* is_disabled

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* is_enabled

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* rename enabled/disabled

* move deprecated api

* remove duplicated test from merge

* fix rename is_disabled

* newline

* test also testprogress for fast dev run

Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-04-23 20:46:18 -04:00
Guy Davidson fe2b6666e0 Fixing a small issue in trainer logging (#1563)
* The epoch was being logged to metrics, which isn't read, rather than to current_metrics.

* Updated the tests to account for the epoch arriving at the logger.
2020-04-23 17:52:41 -04:00
Jirka Borovec 7989ca844c test deprecation warnings (#1470)
* check deprecation warnings

* extend warning test

* try

* unimport modules

* update
2020-04-23 17:34:47 -04:00
Alexey Karnachev edb8d7a23c Nested metrics dictionaries now can be passed to the loggers (#1582)
* now func merge_dicts works with nested dictionaries

* CHANGELOG.md upd
2020-04-23 17:32:36 -04:00
Jirka Borovec 94e53444c6 fix changelog (#1583) 2020-04-23 16:57:37 -04:00
William Falcon 5ab5084f7b Update __init__.py 2020-04-23 15:32:40 -04:00
William Falcon 47629536e2 Amp2 (#1580)
* fixed new amp bugs

* fixed new amp bugs
2020-04-23 15:24:02 -04:00
William Falcon 68ca577919 why copy? (#1579) 2020-04-23 15:03:39 -04:00
29ebe92208 support for native amp (#1561)
* adding native amp suppport

* adding native amp suppport

* adding native amp suppport

* adding native amp suppport

* autocast

* autocast

* autocast

* autocast

* autocast

* autocast

* removed comments

* removed comments

* added state saving

* added state saving

* try install amp again

* added state saving

* drop Apex reinstall

Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-04-23 14:47:08 -04:00
karlinjfandJosh Karlin 41b6cbb3ca Don't copy the batch when training on a single gpu (#1576)
* fix

* whitespace

Co-authored-by: Josh Karlin <karlinjf@gmail.com>
2020-04-23 14:28:20 -04:00
Jirka Borovec 0b22b64a10 Tests/docker (#1573)
* devel image

* try parallel

* new image
2020-04-23 12:52:59 -04:00
Nicki SkafteandNicki Skafte e977d1cde5 Default value for ModelCheckpoint filepath (#1548)
* allow determine of filepath at runtime

* typing

Co-authored-by: Nicki Skafte <nugginea@gmail.com>
2020-04-23 11:50:58 -04:00
Ferdinand Schlatt 545b38ec5f fix boolean argparse (#1571)
* fix boolean argparse #1570

* update change log
2020-04-23 11:44:18 -04:00
William Falcon 759557050a Update __init__.py 2020-04-23 11:04:57 -04:00
Lezwon Castelino 831842972f check for kaggle env variable (#1568)
* check for kaggle env variable

* added changelog
2020-04-23 07:12:54 -04:00
William Falcon 990fd22488 Update __init__.py 2020-04-22 20:16:04 -04:00
7024177f7d Added Horovod distributed backend (#1529)
* Initial commit of Horovod distributed backend implementation

* Update distrib_data_parallel.py

* Update distrib_data_parallel.py

* Update tests/models/test_horovod.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Update tests/models/test_horovod.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Fixed tests

* Added six

* tests

* Install tox for GitHub CI

* Retry tests

* Catch all exceptions

* Skip cache

* Remove tox

* Restore pip cache

* Remove the cache

* Restore pip cache

* Remove AMP

Co-authored-by: William Falcon <waf2107@columbia.edu>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-22 17:39:08 -04:00
Jirka Borovec 4d24032ea5 tests for pytorch 1.5 (#1552)
* tests for pytorch 1.5

* up Win

* win

* win

* win

* win

* win

* win
2020-04-22 10:10:23 -04:00
Jirka Borovec c1c6e3b6c9 default test logger (#1478)
* default test logger

* fix tests

* spawn

* try

* simplify tests

* simplify tests

* formatting

* loggers

* loggers

* revert to TestTube

* default

* default

* wraps

* world size

* optim imports
2020-04-21 20:33:10 -04:00
Kevin Chen bafdeca42f Replace GPU device idx with current process index (#1541) 2020-04-21 14:29:15 -04:00
Justus SchockandJ. Borovec 29c7d2f195 Revert namespace package search to normal package search (#1545)
* Revert this

* typos

* version++

Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-21 08:26:47 -04:00
Justus Schock 8035c10f37 Prepare Namespace package (#1543)
* Update __init__.py

* Update setup.py
2020-04-21 07:12:02 -04:00
Jirka Borovec bd168819f2 fix changelog (#1452)
* fix changelog

* formatting

* add ddp_cpu

* docs

* add another
2020-04-20 17:36:26 -04:00
Roshan RaoandWilliam Falcon 0203938af8 Update learning rate on each backward pass instead of each forward pass. (#1477)
* change lr scheduler step interval to update every backwards pass instead of every forwards pass

* update CHANGELOG

* fix spacing

* Add TODO to lr schedule update

* remove trailing whitespace

Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-04-20 08:03:52 -04:00
Adrian Wälchli 4fca994d0e Fix callback default (horror bug!) (#1534)
* fix horror bug

* update changelog

* fix doctest

* liine too long
2020-04-20 07:02:53 -04:00
Adrian Wälchli 452fa858f4 skip warning test (#1533) 2020-04-20 08:04:37 +00:00
William Falcon b0bf51f99f Update __init__.py 2020-04-19 17:58:45 -04:00
d0c9472cb3 Add SLURM check in ddp_train() and init_ddp_connection() (#1387)
* slurm check in ddp_train and init_ddp_connection

* Remove code example in init_ddp_connection

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* remove blank line

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* improve for test coverage

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* update changelog

* Default values and warnings for DDP env variables

* fix merge artifacts

* update localhost value

* change to NODE_RANK

Co-authored-by: Alexander Reshytko <areshytko@Alexanders-MacBook-Pro.local>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-04-19 17:08:19 -04:00
Adrian Wälchli 7131685ae3 CI: Docs build preview in each PR (#1494)
* store html artifacts in circle ci

* add note to contributing.md
2020-04-19 17:07:16 -04:00
Justus Schock c71bd73acb DDP sampler (#1513)
* Add explicit flag for ddp sampler replacement

* Add flag for sampler replacement in ddp

* Update data_loading.py

* Update CHANGELOG.md

* pep8 fixes

* pep8
2020-04-19 16:58:57 -04:00
William Falcon ae2e14e3ed fixed memory leak from opt return (#1528)
* fixed memory leak from opt return

* fixed memory leak from opt return

* fixed memory leak from opt return

* fixed memory leak from opt return

* fixed memory leak from opt return

* fixed memory leak from opt return

* fixed memory leak from opt return

* fixed memory leak from opt return

* fixed memory leak from opt return

* fixed memory leak from opt return

* fixed memory leak from opt return

* fixed memory leak from opt return

* fixed memory leak from opt return

* fixed memory leak from opt return

* fixed memory leak from opt return

* fixed memory leak from opt return

* fixed memory leak from opt return

* fixed memory leak from opt return

* fixed memory leak from opt return
2020-04-19 16:41:54 -04:00
Hengjian (Henry) JiaandJirka Borovec 3c6f856f23 Fix Mixing hparams and arguments in LightningModule (#1505)
* Attempt to fix #1468

* Remove the if statement, it doesn't actually make any difference

* Update docs

* Correct warnings I caused in the last commit

* Add to changelog

* Actually add to changelog

* Clarify documentation and examples

* Update CHANGELOG.md

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-04-19 07:03:40 -04:00
Jonas-Jaeger e02146943d Removed redundant computations in clip_gradients that slowed down the gradient clipping. (#1523)
Fixes #1522
2020-04-18 23:07:15 -04:00
Krishna PenukondaandAdrian Wälchli a22a8142ac Allow Trainer's gpus arg type to be subclass of currently accepted types (#1423)
* Fixed Trainer `gpus` arg type issue

Fixes #1388

* Disallow boolean gpus parameter

Co-Authored-By: Adrian Wälchli <aedu.waelchli@gmail.com>

* Fixed missing paranthesis

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2020-04-17 18:18:29 -04:00
Jirka BorovecandAdrian Wälchli 1ee2837d62 add QA to docs (#1374)
* add QA to docs

* not about doc updates by @awaelchli

* Apply suggestions from code review

Co-Authored-By: Adrian Wälchli <adrian.waelchli@students.unibe.ch>

* help

Co-authored-by: Adrian Wälchli <adrian.waelchli@students.unibe.ch>
2020-04-17 18:17:33 -04:00
William Falcon 3ddf3f1fb4 change name (#1519) 2020-04-17 14:45:23 -04:00
9b31272cf0 feat: save checkpoint before deleting old ones (#1453)
* feat: save checkpoint before deleting old ones

* fix: make sure that the new model is not deleted

* changelog

Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-04-16 16:40:51 +00:00
Adrian Wälchli 2ab2f7d08d Improved docs for pytorch_lightning.core (continued) (#1483)
* improved docs for core

update links


add references to hooks lifecycle


wip


continue with __init__.py


improve docs for memory.py


improve docs for saving.py


simpler links


fix formatting

* move hooks lifecycle to top of file

* fix doctest import problem

* add missing hook in lifecycle
2020-04-16 12:04:55 -04:00
Adrian WälchliandJirka Borovec 6e1d72d98a Improved docs for Loggers (#1484)
* improve __init__

* improve logger base

* improve comet logger docs

* improved docs for mlflow

* improved nepune logger docs

* fix matplotlib import issue

* improve tensorboard docs

* improve docs for test tube

* improved trains logger docs

* improve wandb logger docs

* improved docs in experiment_logging.rst

* added MLflow to the list of loggers

* fix too long lines

* fix trains doctest

* fix neptune doctest

* fix mlflow doctest

* Apply suggestions from code review

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Apply suggestions from code review

* fix whitespace

* try bypass mode for neptune (fix doctest api key error)

* try "test" as api key

* Revert "try "test" as api key"

This reverts commit fd77db26d551f08b4b4a12bb93cbd8f7a0814f29.

* try test as api key

* update neptune docs

* bump neptune minimal version

* revert unnecessary bypass code

* test if CI runs doctests in .rst files

* Revert "test if CI runs doctests in .rst files"

This reverts commit a45aeb460a8c4b7445a35dd7b49265f48d11c485.

* add doctest directive

* neptune demo links

* added tutorial link for W&B

* fix line too long

* fix merge error

* fix merge error

* add instructions how to install loggers

* add instructions how to install the loggers

* hide _abc_impl property from docs

* review Borda, 4 spaces

* indentation in example sections

* blank

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-04-16 12:04:12 -04:00
Adrian Wälchli 3c549e8ae3 Call on_before_zero_grad model hook (#1493)
* call on_before_zero_grad

* update changelog

* add note about overriding both hooks

* added test

* move test_hooks.py to models folder
2020-04-16 12:01:41 -04:00
Boris DaymaandJ. Borovec 06e6eadfaf feat(semseg): allow model customization (#1371)
* feat(semantic_segmentation): allow customization of unet

* feat(semseg): allow model customization

* style(semseg): format to PEP8

* fix(semseg): rename logger

* docs(changelog): updated semantic segmentation example

* suggestions

* suggestions

* flake8

Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-16 12:00:24 -04:00
e3001a0929 Add ddp_cpu backend for testing ddp without GPUs (#1158)
* Add tests for distributed backend config

* Refactor set_distributed_mode

* Use gloo backend on cpu

* Use 127.0.0.1 instead of 127.0.0.2

Not totally clear on why this is necessary, but it seemt to work

* Update LightningDDP so that it works with CPU

* Add ddp_cpu backend and num_processes Trainer arg

* PEP8

* Fix test skipping. Inequalities are hard :/

* Skip ddp_cpu test on Windows

* Make a few more cases fall back to ddp_cpu

* New function name

* Flake8

* Don't test distributed on MacOS with torch < 1.3

Support for distributed in MacOS was added in Torch 1.3.0

* Add ddp_cpu and num_processes to docs

* Parametrize trainer config tests

* Tweak warning

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Remove redundant test

* Replace pass branches with comments

* Add missing warnings import

* save_path -> root_dir

* Use new rank_zero_warn

* Whitespace

* Apply suggestions from code review

* formatting

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-15 23:17:31 -04:00
William FalconandJ. Borovec 3431c62d41 Remove error when test dataloader used in test (#1495)
* remove error when test dataloader used in test

* remove error when test dataloader used in test

* remove error when test dataloader used in test

* remove error when test dataloader used in test

* remove error when test dataloader used in test

* remove error when test dataloader used in test

* fix lost model reference

* remove error when test dataloader used in test

* fix lost model reference

* moved optimizer types

* moved optimizer types

* moved optimizer types

* moved optimizer types

* moved optimizer types

* moved optimizer types

* moved optimizer types

* moved optimizer types

* added tests for warning

* fix lost model reference

* fix lost model reference

* added tests for warning

* added tests for warning

* refactoring

* refactoring

* fix imports

* refactoring

* fix imports

* refactoring

* fix tests

* fix mnist

* flake8

* review

Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-15 22:16:40 -04:00
Jirka Borovec 8322f1b039 neptune online (#1499) 2020-04-15 11:14:29 -04:00
Jirka Borovec b3fe17ddeb fix flushing loggers (#1459)
* flushing loggers

* flushing loggers

* flushing loggers

* flushing loggers

* changelog

* typo

* fix trains

* optimize imports

* add logger test all

* add logger test pickle

* flake8

* fix benchmark

* hanging loggers

* try

* del

* all

* cleaning
2020-04-14 20:32:33 -04:00
William FalconandJirka Borovec c96c6a6b33 attempting to remove some speed issues (#1482)
* removed some .items

* added speed tests

* added speed tests

* Update benchmarks/test_rnn_parity.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Update benchmarks/test_trainer_parity.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* fix lost model reference

* added speed tests

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-04-14 20:23:36 -04:00
Justus Schock f293c9b5f4 Merge pull request #1473 from aiyolo/patch-3
docs typo
2020-04-14 10:04:28 +02:00
Ethan Harris 8544b334e4 Replace automatic nan check with optional flag (#1475)
* Replace automatic nan check with optional flag

* Update CHANGELOG.md
2020-04-13 14:06:25 -04:00
Caldera 1b45ddcd17 typing error
it should be val_batch here.
2020-04-14 00:16:54 +08:00