karlinjf and Josh Karlin
41b6cbb3ca
Don't copy the batch when training on a single gpu ( #1576 )
...
* fix
* whitespace
Co-authored-by: Josh Karlin <karlinjf@gmail.com >
2020-04-23 14:28:20 -04:00
Jirka Borovec
0b22b64a10
Tests/docker ( #1573 )
...
* devel image
* try parallel
* new image
2020-04-23 12:52:59 -04:00
Nicki Skafte and Nicki Skafte
e977d1cde5
Default value for ModelCheckpoint filepath ( #1548 )
...
* allow determine of filepath at runtime
* typing
Co-authored-by: Nicki Skafte <nugginea@gmail.com >
2020-04-23 11:50:58 -04:00
Ferdinand Schlatt
545b38ec5f
fix boolean argparse ( #1571 )
...
* fix boolean argparse #1570
* update change log
2020-04-23 11:44:18 -04:00
William Falcon
759557050a
Update __init__.py
2020-04-23 11:04:57 -04:00
Lezwon Castelino
831842972f
check for kaggle env variable ( #1568 )
...
* check for kaggle env variable
* added changelog
2020-04-23 07:12:54 -04:00
William Falcon
990fd22488
Update __init__.py
2020-04-22 20:16:04 -04:00
7024177f7d
Added Horovod distributed backend ( #1529 )
...
* Initial commit of Horovod distributed backend implementation
* Update distrib_data_parallel.py
* Update distrib_data_parallel.py
* Update tests/models/test_horovod.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com >
* Update tests/models/test_horovod.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com >
* Fixed tests
* Added six
* tests
* Install tox for GitHub CI
* Retry tests
* Catch all exceptions
* Skip cache
* Remove tox
* Restore pip cache
* Remove the cache
* Restore pip cache
* Remove AMP
Co-authored-by: William Falcon <waf2107@columbia.edu >
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com >
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz >
2020-04-22 17:39:08 -04:00
Jirka Borovec
4d24032ea5
tests for pytorch 1.5 ( #1552 )
...
* tests for pytorch 1.5
* up Win
* win
* win
* win
* win
* win
* win
2020-04-22 10:10:23 -04:00
Jirka Borovec
c1c6e3b6c9
default test logger ( #1478 )
...
* default test logger
* fix tests
* spawn
* try
* simplify tests
* simplify tests
* formatting
* loggers
* loggers
* revert to TestTube
* default
* default
* wraps
* world size
* optim imports
2020-04-21 20:33:10 -04:00
Kevin Chen
bafdeca42f
Replace GPU device idx with current process index ( #1541 )
2020-04-21 14:29:15 -04:00
Justus Schock and J. Borovec
29c7d2f195
Revert namespace package search to normal package search ( #1545 )
...
* Revert this
* typos
* version++
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz >
2020-04-21 08:26:47 -04:00
Justus Schock
8035c10f37
Prepare Namespace package ( #1543 )
...
* Update __init__.py
* Update setup.py
2020-04-21 07:12:02 -04:00
Jirka Borovec
bd168819f2
fix changelog ( #1452 )
...
* fix changelog
* formatting
* add ddp_cpu
* docs
* add another
2020-04-20 17:36:26 -04:00
Roshan Rao and William Falcon
0203938af8
Update learning rate on each backward pass instead of each forward pass. ( #1477 )
...
* change lr scheduler step interval to update every backwards pass instead of every forwards pass
* update CHANGELOG
* fix spacing
* Add TODO to lr schedule update
* remove trailing whitespace
Co-authored-by: William Falcon <waf2107@columbia.edu >
2020-04-20 08:03:52 -04:00
Adrian Wälchli
4fca994d0e
Fix callback default (horror bug!) ( #1534 )
...
* fix horror bug
* update changelog
* fix doctest
* liine too long
2020-04-20 07:02:53 -04:00
Adrian Wälchli
452fa858f4
skip warning test ( #1533 )
2020-04-20 08:04:37 +00:00
William Falcon
b0bf51f99f
Update __init__.py
2020-04-19 17:58:45 -04:00
d0c9472cb3
Add SLURM check in ddp_train() and init_ddp_connection() ( #1387 )
...
* slurm check in ddp_train and init_ddp_connection
* Remove code example in init_ddp_connection
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com >
* remove blank line
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com >
* improve for test coverage
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com >
* update changelog
* Default values and warnings for DDP env variables
* fix merge artifacts
* update localhost value
* change to NODE_RANK
Co-authored-by: Alexander Reshytko <areshytko@Alexanders-MacBook-Pro.local >
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com >
Co-authored-by: William Falcon <waf2107@columbia.edu >
2020-04-19 17:08:19 -04:00
Adrian Wälchli
7131685ae3
CI: Docs build preview in each PR ( #1494 )
...
* store html artifacts in circle ci
* add note to contributing.md
2020-04-19 17:07:16 -04:00
Justus Schock
c71bd73acb
DDP sampler ( #1513 )
...
* Add explicit flag for ddp sampler replacement
* Add flag for sampler replacement in ddp
* Update data_loading.py
* Update CHANGELOG.md
* pep8 fixes
* pep8
2020-04-19 16:58:57 -04:00
William Falcon
ae2e14e3ed
fixed memory leak from opt return ( #1528 )
...
* fixed memory leak from opt return
* fixed memory leak from opt return
* fixed memory leak from opt return
* fixed memory leak from opt return
* fixed memory leak from opt return
* fixed memory leak from opt return
* fixed memory leak from opt return
* fixed memory leak from opt return
* fixed memory leak from opt return
* fixed memory leak from opt return
* fixed memory leak from opt return
* fixed memory leak from opt return
* fixed memory leak from opt return
* fixed memory leak from opt return
* fixed memory leak from opt return
* fixed memory leak from opt return
* fixed memory leak from opt return
* fixed memory leak from opt return
* fixed memory leak from opt return
2020-04-19 16:41:54 -04:00
Hengjian (Henry) Jia and Jirka Borovec
3c6f856f23
Fix Mixing hparams and arguments in LightningModule ( #1505 )
...
* Attempt to fix #1468
* Remove the if statement, it doesn't actually make any difference
* Update docs
* Correct warnings I caused in the last commit
* Add to changelog
* Actually add to changelog
* Clarify documentation and examples
* Update CHANGELOG.md
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com >
2020-04-19 07:03:40 -04:00
Jonas-Jaeger
e02146943d
Removed redundant computations in clip_gradients that slowed down the gradient clipping. ( #1523 )
...
Fixes #1522
2020-04-18 23:07:15 -04:00
Krishna Penukonda and Adrian Wälchli
a22a8142ac
Allow Trainer's gpus arg type to be subclass of currently accepted types ( #1423 )
...
* Fixed Trainer `gpus` arg type issue
Fixes #1388
* Disallow boolean gpus parameter
Co-Authored-By: Adrian Wälchli <aedu.waelchli@gmail.com >
* Fixed missing paranthesis
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com >
2020-04-17 18:18:29 -04:00
Jirka Borovec and Adrian Wälchli
1ee2837d62
add QA to docs ( #1374 )
...
* add QA to docs
* not about doc updates by @awaelchli
* Apply suggestions from code review
Co-Authored-By: Adrian Wälchli <adrian.waelchli@students.unibe.ch >
* help
Co-authored-by: Adrian Wälchli <adrian.waelchli@students.unibe.ch >
2020-04-17 18:17:33 -04:00
William Falcon
3ddf3f1fb4
change name ( #1519 )
2020-04-17 14:45:23 -04:00
9b31272cf0
feat: save checkpoint before deleting old ones ( #1453 )
...
* feat: save checkpoint before deleting old ones
* fix: make sure that the new model is not deleted
* changelog
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz >
Co-authored-by: William Falcon <waf2107@columbia.edu >
2020-04-16 16:40:51 +00:00
Adrian Wälchli
2ab2f7d08d
Improved docs for pytorch_lightning.core (continued) ( #1483 )
...
* improved docs for core
update links
add references to hooks lifecycle
wip
continue with __init__.py
improve docs for memory.py
improve docs for saving.py
simpler links
fix formatting
* move hooks lifecycle to top of file
* fix doctest import problem
* add missing hook in lifecycle
2020-04-16 12:04:55 -04:00
Adrian Wälchli and Jirka Borovec
6e1d72d98a
Improved docs for Loggers ( #1484 )
...
* improve __init__
* improve logger base
* improve comet logger docs
* improved docs for mlflow
* improved nepune logger docs
* fix matplotlib import issue
* improve tensorboard docs
* improve docs for test tube
* improved trains logger docs
* improve wandb logger docs
* improved docs in experiment_logging.rst
* added MLflow to the list of loggers
* fix too long lines
* fix trains doctest
* fix neptune doctest
* fix mlflow doctest
* Apply suggestions from code review
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com >
* Apply suggestions from code review
* fix whitespace
* try bypass mode for neptune (fix doctest api key error)
* try "test" as api key
* Revert "try "test" as api key"
This reverts commit fd77db26d551f08b4b4a12bb93cbd8f7a0814f29.
* try test as api key
* update neptune docs
* bump neptune minimal version
* revert unnecessary bypass code
* test if CI runs doctests in .rst files
* Revert "test if CI runs doctests in .rst files"
This reverts commit a45aeb460a8c4b7445a35dd7b49265f48d11c485.
* add doctest directive
* neptune demo links
* added tutorial link for W&B
* fix line too long
* fix merge error
* fix merge error
* add instructions how to install loggers
* add instructions how to install the loggers
* hide _abc_impl property from docs
* review Borda, 4 spaces
* indentation in example sections
* blank
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com >
2020-04-16 12:04:12 -04:00
Adrian Wälchli
3c549e8ae3
Call on_before_zero_grad model hook ( #1493 )
...
* call on_before_zero_grad
* update changelog
* add note about overriding both hooks
* added test
* move test_hooks.py to models folder
2020-04-16 12:01:41 -04:00
Boris Dayma and J. Borovec
06e6eadfaf
feat(semseg): allow model customization ( #1371 )
...
* feat(semantic_segmentation): allow customization of unet
* feat(semseg): allow model customization
* style(semseg): format to PEP8
* fix(semseg): rename logger
* docs(changelog): updated semantic segmentation example
* suggestions
* suggestions
* flake8
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz >
2020-04-16 12:00:24 -04:00
e3001a0929
Add ddp_cpu backend for testing ddp without GPUs ( #1158 )
...
* Add tests for distributed backend config
* Refactor set_distributed_mode
* Use gloo backend on cpu
* Use 127.0.0.1 instead of 127.0.0.2
Not totally clear on why this is necessary, but it seemt to work
* Update LightningDDP so that it works with CPU
* Add ddp_cpu backend and num_processes Trainer arg
* PEP8
* Fix test skipping. Inequalities are hard :/
* Skip ddp_cpu test on Windows
* Make a few more cases fall back to ddp_cpu
* New function name
* Flake8
* Don't test distributed on MacOS with torch < 1.3
Support for distributed in MacOS was added in Torch 1.3.0
* Add ddp_cpu and num_processes to docs
* Parametrize trainer config tests
* Tweak warning
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com >
* Remove redundant test
* Replace pass branches with comments
* Add missing warnings import
* save_path -> root_dir
* Use new rank_zero_warn
* Whitespace
* Apply suggestions from code review
* formatting
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com >
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz >
2020-04-15 23:17:31 -04:00
William Falcon and J. Borovec
3431c62d41
Remove error when test dataloader used in test ( #1495 )
...
* remove error when test dataloader used in test
* remove error when test dataloader used in test
* remove error when test dataloader used in test
* remove error when test dataloader used in test
* remove error when test dataloader used in test
* remove error when test dataloader used in test
* fix lost model reference
* remove error when test dataloader used in test
* fix lost model reference
* moved optimizer types
* moved optimizer types
* moved optimizer types
* moved optimizer types
* moved optimizer types
* moved optimizer types
* moved optimizer types
* moved optimizer types
* added tests for warning
* fix lost model reference
* fix lost model reference
* added tests for warning
* added tests for warning
* refactoring
* refactoring
* fix imports
* refactoring
* fix imports
* refactoring
* fix tests
* fix mnist
* flake8
* review
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz >
2020-04-15 22:16:40 -04:00
Jirka Borovec
8322f1b039
neptune online ( #1499 )
2020-04-15 11:14:29 -04:00
Jirka Borovec
b3fe17ddeb
fix flushing loggers ( #1459 )
...
* flushing loggers
* flushing loggers
* flushing loggers
* flushing loggers
* changelog
* typo
* fix trains
* optimize imports
* add logger test all
* add logger test pickle
* flake8
* fix benchmark
* hanging loggers
* try
* del
* all
* cleaning
2020-04-14 20:32:33 -04:00
William Falcon and Jirka Borovec
c96c6a6b33
attempting to remove some speed issues ( #1482 )
...
* removed some .items
* added speed tests
* added speed tests
* Update benchmarks/test_rnn_parity.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com >
* Update benchmarks/test_trainer_parity.py
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com >
* fix lost model reference
* added speed tests
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com >
2020-04-14 20:23:36 -04:00
Justus Schock
f293c9b5f4
Merge pull request #1473 from aiyolo/patch-3
...
docs typo
2020-04-14 10:04:28 +02:00
Ethan Harris
8544b334e4
Replace automatic nan check with optional flag ( #1475 )
...
* Replace automatic nan check with optional flag
* Update CHANGELOG.md
2020-04-13 14:06:25 -04:00
Caldera
1b45ddcd17
typing error
...
it should be val_batch here.
2020-04-14 00:16:54 +08:00
Nicki Skafte and Nicki Skafte
3f1e4b953f
fix lr scheduler docs ( #1446 )
...
Co-authored-by: Nicki Skafte <nugginea@gmail.com >
2020-04-10 16:14:51 -04:00
3f09b32df3
Learning Rate finder ( #1347 )
...
* initial structure
* rebase
* incorporate suggestions
* update CHANGELOG.md
* initial docs
* fixes based on reviews
* added trainer arg
* update docs
* added saving/restore of model state
* initial tests
* fix styling
* added more tests
* fix docs, backward compatility and progressbar
* fix styling
* docs update
* updates based on review
* changed saving to standard functions
* consistent naming
* fix formatting
* improve docs, added support for nested fields, improve codecov
* update CHANGELOG.md
* Update lr_finder.rst
* Update pytorch_lightning/trainer/trainer.py
* Update trainer.py
* Update CHANGELOG.md
* Update path
* restoring
* test
* attribs
* docs
* doc typo
Co-authored-by: Nicki Skafte <nugginea@gmail.com >
Co-authored-by: William Falcon <waf2107@columbia.edu >
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com >
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz >
2020-04-10 14:34:23 -04:00
Jirka Borovec
d05ac813dc
fix deprecated default_save_path ( #1449 )
2020-04-10 14:32:56 -04:00
Jirka Borovec
dcda5194df
continues develop ( #1419 )
...
* continues develop
* changelog
* typo
2020-04-10 13:10:46 -04:00
William Falcon
b78c3d4da8
Fix weights path ( #1445 )
...
* renamed default path to actual root_dir
* added default weights path
* added default weights path
* added default weights path
2020-04-10 12:02:59 -04:00
Allard Hendriksen
7ac1580a31
Add automatic GPU choice to trainer ( #1426 )
...
* Add automatic GPU choice to trainer
This commit adds the `gpu_choice` parameter to Trainer. By default,
this parameter is set to 'manual' which causes no observable
difference in behavior.
When `gpu_choice` is set to "auto" and `gpus` is an int, then the
trainer will automatically allocate the first available GPU.
This is especially useful when GPUs are configured to be in "exclusive
mode", which means that only one process at a time can use them.
* Rename gpu_choice -> auto_select_gpus
2020-04-10 11:45:29 -04:00
Rohit Gupta
e79ae18cae
Add test_dataloaders to test method ( #1434 )
...
* Add test_dataloaders to test method
* Remove test_dataloaders from .fit()
* Fix code comment
* Fix tests
* Add test_dataloaders to test method (#1393 )
* Fix failing tests
* Update docs (#1393 )
2020-04-10 11:44:03 -04:00
Alexey Karnachev
4c34d16a34
Fixed configure optimizer from dict without "scheduler" key ( #1443 )
...
* `configure_optimizer` from dict with only "optimizer" key. bug fixed
* autopep8
* pep8speaks suggested fixes
* CHANGELOG.md upd
2020-04-10 11:43:06 -04:00
William Falcon
7857a73710
Update README.md
2020-04-10 09:55:42 -04:00
William Falcon
e052883de7
Update README.md
2020-04-10 09:54:09 -04:00
