38 Commits
Author SHA1 Message Date
Devashish ShankarandJirka Borovec ade3f36b7a Raise an error when lightning replaces an existing sampler (#2020)
* Raise an error when lightning replaces an existing sampler

Currently, Trainer replaces the existing sampler with DistributedSampler
if running distributing training and `replace_sampler_ddp=True` (default
behaviour). If a user has configured an existing sampler, this would
lead to widely different results if running a distributed vs
non-distributed training.

This PR fixes this by raising an Error if user has configured a sampler
and uses `replace_sampler_ddp=True`. The recommended behavior from now
on is to either remove the sampler or set `replace_sampler_ddp=False`

* Fix tests

* Simpler fix

* Fix tests

* Make inner method protected

* Apply suggestions from code review

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-06-02 18:52:04 -04:00
Ashraful Islam 981169cacc add warning for shuffling in test/val (#1865) 2020-05-18 09:53:02 -04:00
William Falcon 53d9316a56 fixes ddp bugs (#1819)
* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug
2020-05-13 19:17:04 -04:00
Yuri Brovmanandybrovman 3a642601e8 added warning for None dataloader (#1745)
* added warning for None dataloader

* fixed variable style

* updated warning message

* remove unused import

Co-authored-by: ybrovman <ybrovman@ebay.com>
2020-05-07 09:26:41 -04:00
Shunta Komatsu f656882942 Fix typo (#1750) 2020-05-07 09:25:54 -04:00
Yuri Brovmanandybrovman 35bbe178bd fix _reset_eval_dataloader() for IterableDataset (#1560)
* removed if dl from _reset_eval_dataloader()

* changed to if dl != None to be more safe

* hints from pep8speaks

Co-authored-by: ybrovman <ybrovman@ebay.com>
2020-05-05 14:09:48 -04:00
Jirka BorovecandWilliam Falcon 2a2f303ae9 Tests: refactor trainer dataloaders (#1690)
* refactor default model

* drop redundant seeds

* refactor dataloaders tests

* fix multiple

* fix conf

* flake8

* Apply suggestions from code review

Co-authored-by: William Falcon <waf2107@columbia.edu>

Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-05-05 12:31:15 -04:00
William Falcon cebc74f7bc ddp pickle 2020-04-27 07:19:12 -04:00
William Falcon 2181ad1bc7 ddp pickle 2020-04-27 07:12:04 -04:00
William FalconandJ. Borovec b620d86c54 diable val and test shuffling (#1600)
* diable val and test shuffling

* diable val and test shuffling

* diable val and test shuffling

* diable val and test shuffling

* log

* condition

* shuffle

* refactor

Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-25 16:45:20 -04:00
William Falcon d0faf97893 fixed dataset stuff + docs (#1599)
* Fixed dataset docs and disabled auto-sampler for iterable dataset
2020-04-24 16:51:26 -04:00
7024177f7d Added Horovod distributed backend (#1529)
* Initial commit of Horovod distributed backend implementation

* Update distrib_data_parallel.py

* Update distrib_data_parallel.py

* Update tests/models/test_horovod.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Update tests/models/test_horovod.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Fixed tests

* Added six

* tests

* Install tox for GitHub CI

* Retry tests

* Catch all exceptions

* Skip cache

* Remove tox

* Restore pip cache

* Remove the cache

* Restore pip cache

* Remove AMP

Co-authored-by: William Falcon <waf2107@columbia.edu>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-22 17:39:08 -04:00
Jirka Borovec c1c6e3b6c9 default test logger (#1478)
* default test logger

* fix tests

* spawn

* try

* simplify tests

* simplify tests

* formatting

* loggers

* loggers

* revert to TestTube

* default

* default

* wraps

* world size

* optim imports
2020-04-21 20:33:10 -04:00
Justus Schock c71bd73acb DDP sampler (#1513)
* Add explicit flag for ddp sampler replacement

* Add flag for sampler replacement in ddp

* Update data_loading.py

* Update CHANGELOG.md

* pep8 fixes

* pep8
2020-04-19 16:58:57 -04:00
Jirka Borovec 17f58d2e11 add rank warning (#1428)
* add rank warning

* changelog

* use rank_zero_warn

* user trainer_init

* replace warnings

* fix test

* flake8

* docs

* changelog

* bug lol
2020-04-09 14:05:46 -04:00
William Falcon b4eb3884cf workers warning not on windows (#1430) 2020-04-09 12:42:30 -04:00
William Falcon 21a1972921 fixed default sampler (#1425) 2020-04-09 08:52:15 -04:00
Ethan HarrisandJirka Borovec b18accc64c Add warning for few workers (#1378)
* Add warning for few workers

* Fix style issue

* Update CHANGELOG.md

* Update test

* formatting

* formatting

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-04-05 11:07:16 -04:00
Justus SchockandJ. Borovec f6a86e8551 generalize reinstantiation of dataloader (#1346)
* generalize reinstantiation of dataloader

* fix condition

* add test

* update changelog

* fix changelog

Co-authored-by: J. Borovec <jirka.borovec@seznam.cz>
2020-04-03 17:55:08 -04:00
William Falcon e48422df38 Sampler (#1328)
* sampler

* check for dataloader type

* check for dataloader type

* fixed sampler cases
2020-04-01 12:57:36 -04:00
William Falcon 7de51f78ac Sampler (#1318)
* sampler

* sampler

* sampler

* check for dataloader type

* check for dataloader type
2020-03-31 18:22:45 -04:00
Jirka Borovec 6ddb03922a Profiler summary (#1259)
* refactor and add types

* add Prorfiler summary

* fix imports

* Revert "refactor and add types"

This reverts commit b4c552fa

* changelog

* revert rename

* fix test

* mute verbose
2020-03-31 08:57:48 -04:00
2ccc7456ca Error on zero length dataloaders (#1280)
* error_on_zero_length

* update CHANGELOG.md

* added test

* Update pytorch_lightning/trainer/data_loading.py

Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-03-30 18:43:53 -04:00
sneimanandseth 6dfe9951e1 validation and training loops run the partial dataset (#1192)
* auto_add_sampler() fix

* auto_add_sampler() fix

Co-authored-by: seth <seth@duckpapa.com>
2020-03-30 12:13:34 -04:00
Ethan Harris ab09faa15e Add support for iterable datasets when val_check_interval=1.0 (#1283)
* Add support for iterable datasets when val_check_interval=1.0

* Update CHANGELOG.md
2020-03-29 15:27:44 -04:00
Jirka Borovec e461ec0037 fixing Win failed import (#1163)
* version

* try fix distrib

* update try import
2020-03-16 20:50:36 -04:00
Ethan Harris 2b3f443f6b Add support for IterableDatasets everywhere (#1104)
* Add support for IterableDatasets everywhere

* Added type hints, simplified code and improved coverage in data_loading.py

* Update CHANGELOG.md
2020-03-12 12:46:02 -04:00
William Falcon 6f2b296b64 prepare_data before spawns (#1067)
* prepare_data assignment

* prepare_data assignment

* prepare_data assignment

* prepare_data assignment

* prepare_data assignment

* prepare_data assignment
2020-03-06 05:49:18 -05:00
Jirka Borovec 7beed7cae6 Trainer cleanup (#934)
* Trainer cleanup

* update abstract

* remove ...

* remove __init__

* update mixin types

* update callbacks

* fix

* lower test acc
2020-02-27 16:21:14 -05:00
Ethan Harris b2e9607362 Refactor dataloading (#955)
* Refactor dataloading

* Refactor dataloading

* Refactor dataloading

* Add shuffle to test
2020-02-26 16:55:18 -05:00
Jirka Borovec 5dd2afeab1 Fixing tests (#936)
* abs import

* rename test model

* update trainer

* revert test_step check

* move tags

* fix test_step

* clean tests

* fix template

* update dataset path

* fix parent order
2020-02-25 13:06:24 -05:00
William Falcon 1015a00506 Clean up dataloader logic (#926)
* added get dataloaders directly using a getter

* deleted decorator

* added prepare_data hook

* refactored dataloader init

* refactored dataloader init

* added dataloader reset flag and main loop

* added dataloader reset flag and main loop

* added dataloader reset flag and main loop

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* made changes

* fixed bad loaders

* fixed bad loaders

* fixed bad loaders

* fixed bad loaders

* fixed bad loaders

* fixed bad loaders

* fixed bad loaders

* fixed bad loaders

* fixed bad loaders

* fixed error in .fit with loaders

* fixed error in .fit with loaders

* fixed error in .fit with loaders

* fixed error in .fit with loaders

* fixed error in .fit with loaders

* fixed error in .fit with loaders

* fixed error in .fit with loaders

* fixed error in .fit with loaders

* fixed error in .fit with loaders

* fixed error in .fit with loaders

* fixed error in .fit with loaders

* fixed error in .fit with loaders

* fixed error in .fit with loaders

* fixes #909

* fixes #909

* bug fix

* Fixes #902
2020-02-24 22:23:25 -05:00
srush 5778a4131c Add tags to the rendezvous calls for TPU. (#921)
* Update data_loading.py

* Update training_io.py

* Update trainer.py
2020-02-23 15:00:32 -05:00
William Falcon c00a8a10dd finished dist (#911) 2020-02-21 20:39:12 -05:00
d4a31f02e0 Enable TPU support (#868)
* added tpu docs

* added tpu flags

* add tpu docs + init training call

* amp

* amp

* amp

* amp

* optimizer step

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* fix test pkg create (#873)

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added test return and print

* added test return and print

* added test return and print

* added test return and print

* added test return and print

* Update pytorch_lightning/trainer/trainer.py

Co-Authored-By: Luis Capelo <luiscape@gmail.com>

* Fix segmentation example (#876)

* removed torchvision model and added custom model

* minor fix

* Fixed relative imports issue

* Fix/typo (#880)

* Update greetings.yml

* Update greetings.yml

* Changelog (#869)

* Create CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update PULL_REQUEST_TEMPLATE.md

* Update PULL_REQUEST_TEMPLATE.md

* Add PR links to Version 0.6.0 in CHANGELOG.md

* Add PR links for Unreleased in CHANGELOG.md

* Update PULL_REQUEST_TEMPLATE.md

* Fixing Function Signatures (#871)

* added tpu docs

* added tpu flags

* add tpu docs + init training call

* amp

* amp

* amp

* amp

* optimizer step

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added auto data transfer to TPU

* added test return and print

* added test return and print

* added test return and print

* added test return and print

* added test return and print

* added test return and print

* added test return and print

* added test return and print

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Luis Capelo <luiscape@gmail.com>
Co-authored-by: Akshay Kulkarni <akshayk.vnit@gmail.com>
Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
Co-authored-by: Shikhar Chauhan <xssChauhan@users.noreply.github.com>
2020-02-17 16:01:20 -05:00
Jirka Borovec ea59a99426 update org paths & convert logos (#685)
* fix typos

* update org paths

* update links from READMe to docs

* add svg logo

* add svg logo-text

* update logos

* testing temp paths

* prune links from readme

* optimize imports

* update logo

* update paths in README

* missing imports
2020-01-20 14:50:31 -05:00
Vadim Bereznyuk 7824b5c5f5 Fix percent_checks (#649)
* fix percent_checks

* Added _percent_range_check

* remove max
2020-01-05 14:36:06 -05:00
Jirka Borovec 1d4b6be17b rename trainer modules, drop _mixin (#571)
* rename trainer modules, drop _mixin

* fix imports
2019-12-04 11:39:14 -05:00