1633 Commits
Author SHA1 Message Date
William Falcon 8f966797b7 release v0.5.3.2 0.5.3.2 2019-11-09 10:51:33 -05:00
William Falcon c10ca47ab8 fix imagenet example 2019-11-09 07:15:07 -05:00
Jason Phang d56750899f ImageNet Example (#476)
* ImageNet example

* cleanup

* cleanup

* Minor changes from feedback

* More cleanup
2019-11-09 01:02:21 -05:00
Jirka Borovec 1fd1e42aa6 Fix setup-doc for pypi (#472)
* add Twine to CI

* freeze Twine

* freeze Twine

* minor refactoring

* try another

* fix req.

* update README

* fix __doc__

* fix multiple req. test-tube
2019-11-09 00:59:14 -05:00
Tullie Murrell a3f785dfca Fix tbptt docs (#484) 2019-11-08 21:21:36 -05:00
Anton Bakhtin e22dea228f Remove errnonoeous comma in logging call (#474) 2019-11-08 08:50:39 -05:00
Jirka Borovec 2acdfe57a7 make checkboxes (#473) 2019-11-07 15:32:00 -05:00
William Falcon 1fd2cfcffd release v0.5.3.1 2019-11-07 08:35:32 -05:00
William Falcon e41bf0a047 fix install 0.5.3.1 2019-11-07 08:34:32 -05:00
William Falcon cd594a1d1a Update setup.py 2019-11-06 16:09:45 -05:00
William Falcon d923acd606 Update README.md 2019-11-06 15:12:46 -05:00
William Falcon b35229d9ab need to fix readme for pypi 2019-11-06 14:58:53 -05:00
William Falcon 978519fc33 release v0.5.3 2019-11-06 14:56:51 -05:00
William Falcon aab4fe860d release v0.5.3 0.5.3 2019-11-06 14:51:49 -05:00
William Falcon a7f3974f95 Release (#467)
* smurf ethics

* smurf ethics

* removed auto ddp fix

* removed auto ddp fix

* removed auto ddp fix

* removed auto ddp fix

* removed auto ddp fix

* removed auto ddp fix
2019-11-06 14:34:50 -05:00
William Falcon 7ce5b3f62f Update README.md 2019-11-05 11:53:12 -05:00
William Falcon f95eca3ea5 Update README.md 2019-11-05 11:52:50 -05:00
William Falcon f9a3e03e1c Update CONTRIBUTING.md 2019-11-05 10:49:22 -05:00
Nic Eggert 9fa2806605 Fix ModelCheckpoint default paths (#413)
* Make name and version properties required

* Warn before deleting files in checkpoint directory

* Get default checkpoint path from any logger

* Fix typos

* Uncomment logger tests

* Whitespace

* Update callback_config_mixin.py

checkpoints and version file names would just have a number. it's easy to tell what you're looking at with version_ prepended

* Address comments

* Fix broken tests
2019-11-05 10:41:59 -05:00
William Falcon 3e38005a61 Ddp2 fix (#448)
* added training_end

* added training_end

* added training_end

* added training_end

* added training_end

* added training_end

* added training_end

* added training_end

* added training_end

* added training_end

* added training_end

* added training_end

* allow ddp and apex to be configured

* allow ddp and apex to be configured

* bananas

* bananas

* bananas

* bananas

* bananas

* bananas

* bananas

* bananas

* bananas

* bananas

* bananas

* bananas

* bananas

* bananas

* bananas

* added eval and train for redundancy

* added eval and train for redundancy

* added training_end

* added training_end

* added training_end

* added training_end

* added training_end

* added training_end

* added training_end

* added training_end

* added training_end

* added training_end

* added training_end

* added training_end

* allow ddp and apex to be configured

* allow ddp and apex to be configured

* bananas

* bananas

* bananas

* bananas

* bananas

* bananas

* bananas

* bananas

* bananas

* bananas

* bananas

* bananas

* bananas

* bananas

* bananas

* added eval and train for redundancy

* added eval and train for redundancy
2019-11-05 10:01:52 -05:00
William Falcon 8fbaccddae added eval and train for redundancy (#464) 2019-11-05 09:14:33 -05:00
Yongrae Jo 32dd803b1e Fix min_max gpu memory logging bug (#453)
* #452 Fix ValueError

* #452 Use subprocess.run

* #452 Simplify code for gpu_memory_map

* #452 Simplify code for min max memory

* #452 Add test for get_memory_profile

* #452 Use os.sep

* #452 Use os.linesep
2019-11-05 08:55:44 -05:00
Ir1dXD 5a9afb11cc change print to logging (#457)
* change print to logging

* always use logging.info

* use f-strings

* update code style

* set logging configs

* remove unused code
2019-11-05 08:43:21 -05:00
Ir1dXD 9a5307dc30 add coding styleguide (#460)
* Update CONTRIBUTING.md

add coding styleguide

* Update CONTRIBUTING.md
2019-11-05 08:29:24 -05:00
William Falcon 37729f0a17 fixing test (#451) 2019-11-03 08:52:22 -05:00
Vadim Bereznyuk 446a1b5d45 Split progress bar (#449)
* Splitted progress bars

* Iterable dataset total batches fix

* Use dynamic ncols and use batch as units

* Count epochs from 1 in progress bar

* Fix for disabled progress bar

* Code simplifications
2019-11-03 05:42:53 -05:00
s-rog 4e9fd95f79 packed sequence clarification in train_dataloader (#443)
* packed sequence clarification in train_dataloader

* moved changes to training loop

* removed changes from required interface

* added index entry
2019-11-03 05:26:27 -05:00
Pattarawat Chormai 1865de1ff8 [WIP] Fix wrong example paths in README.md (#444)
* Fix wrong example paths

* correct dataloading wrong condition in Readme
2019-11-01 07:55:37 -04:00
William Falcon 661a1c6fe6 Update README.md 2019-10-31 10:49:07 -04:00
Tullie Murrell 248495b1d1 Add tbptt (#429)
* Add truncated bptt

* Fix rebase error

* AutoPep8

* Address comments, incl default bptt_split impl

* Add tbptt test

* Add default split for lists/tuples

* Add tbptt docs

* Fix trainer spacing

* Update RequiredTrainerInterface.md
2019-10-31 06:45:28 -04:00
Joel Wong f6b8b175bb Update Docs for current checkpointing behaviour (#445)
Related issue #432

The old documentation suggested that the way to restore a training session is to use a test_tube Experiment. Trainer no longer takes an experiment as a parameter, so it seems the current way to restore a training session is to pass an experiment via a TestTubeLogger. Even if this is not the most elegant solution, updating the docs will at least point new users in the right direction.
2019-10-31 06:40:32 -04:00
Vadim Bereznyuk f79bdf2327 Set total number of batches in progress bar while testing (#425) 2019-10-30 12:14:28 -04:00
Vadim Bereznyuk 9f8ab7c29e Fixed total number of batches (#439)
* Fixed total number of batches

* Fixed flake8 warning

* Update train_loop_mixin.py

* Update train_loop_mixin.py
2019-10-30 12:13:40 -04:00
William Falcon 8347a6c87e mem clear (#440)
* mem clear

* mem clear
2019-10-30 12:11:21 -04:00
Nic Eggert 4df4d4cc03 Catch exceptions when optional dependencies are missing (#442) 2019-10-30 12:03:52 -04:00
Jirka Borovec 37647d835a add package info (#395)
* add package info #358

* Update __init__.py

* wrap package info

* update CI

* fix package info

* fix for #388

* prune duplicated configs

* fix install

* use req from file

* move info to sep. module
drop comments from req

* add setup req.

* add setup req.

* update get info

* refactor init

* update pip

* fix failing on buildins

* fix failing open

* fix test imports

* fix tests

* fix pep8
2019-10-28 18:41:13 -04:00
William Falcon b86d223889 makes checkpoint process safe (#431) 2019-10-25 08:57:05 -04:00
William Falcon d5ca464cc6 Back hook (#424)
* Fixes #356

* Fixes #356

* Fixes #356

* Fixes #356

* Fixes #356

* Fixes #356
2019-10-24 07:56:56 -04:00
William Falcon a4b43ce095 Loaders (#422)
* refactor dataloading

* refactor dataloading

* refactor dataloading

* refactor dataloading

* refactor dataloading

* refactor dataloading

* refactor dataloading

* refactor dataloading
2019-10-24 06:43:35 -04:00
William Falcon 28c3bcb0c0 Update CONTRIBUTING.md 2019-10-24 06:26:39 -04:00
William Falcon 48eabf0751 Update CONTRIBUTING.md 2019-10-24 06:25:56 -04:00
William Falcon 7fb868bfd8 Update README.md 2019-10-24 06:23:00 -04:00
William Falcon 5db90e32eb hpc restore takes priority over non hpc weights (#419)
* hpc restore takes priority over non hpc weights

* hpc restore takes priority over non hpc weights

* hpc restore takes priority over non hpc weights

* hpc restore takes priority over non hpc weights

* hpc restore takes priority over non hpc weights

* hpc restore takes priority over non hpc weights

* hpc restore takes priority over non hpc weights
2019-10-23 20:18:26 -04:00
William Falcon c6244594a6 clear memory cache before train starts (#418)
* clear memory cache before train starts

* clear memory cache before train starts
2019-10-23 11:41:00 -04:00
William Falcon d955baa235 Update README.md 2019-10-23 06:13:31 -04:00
William Falcon b47b881f78 Update README.md 2019-10-23 06:13:00 -04:00
David Kossnick 56fa2075a5 Move global_step incrementing (#412)
* Move global_step incrementing to the end of a batch loop, per https://github.com/williamFalcon/pytorch-lightning/issues/411

* Move met_batch_limit condition to the end

* cleanup whitespace

* Update train_loop_mixin.py
2019-10-23 06:11:18 -04:00
William Falcon 5afae59715 refactored tests (#417)
* refactored tests

* refactored tests

* refactored tests

* refactored tests

* refactored tests

* refactored tests

* refactored tests

* refactored tests

* refactored tests
2019-10-23 06:10:13 -04:00
Vismantas 2aba70e228 parse_gpu_ids fix (#382)
* Unit tests for num_gpu property as proxy for __parse_gpu_ids.

* Refactoring __parse_gpu_ids

* Moved the function outside the class as it is
an utility function and did not depend on class in any way.
* Added unit tests for it.

* Mocked torch.cuda.device_count function in tests.

This allows the tests to be run on machines that do not have gpus.

* Fixed the parse_gpu_ids function to handle -1 case.

Function now handles -1 the same way as it does for '-1'.

* Unit tests for root_gpu added.

Added backend as a parameter as currently depending on backend set
or not, code fails with exception in certain circumstances, before
giving a wrong answer.

* Moved __set_root_gpu function out of the class.

This function does not depend on the class and can be tested
more easily this way.
Also added unit tests for this function. They simply reuse
data for the root_gpu property.

* determine_root_gpu_device passes unit tests.

* num_gpus passes unit tests.

Also added a None test for this function.

* parse_gpu_ids tests changed to reflect desired state after refactoring.

Planning to refactor parse_gpu_ids to always return list of ints.
This will simplify code that use output of this function.

* * parse_gpu_ids always returns lists
* parse_gpu_ids checks given ids against available ids
* parse_gpu_ids raises exception for non existant ids
* parse_gpu_ids returns None when no gpus are available
* cleaned up determine_root_gpu_device
* cleaned up num_gpus property
* Updated unit tests to reflect changes in the functions

* Flake8 fixes

* Moved fixture code up before where it is used.

* Updated documentation.

* Changed tests to match the API:
* gpus=-1 or gpus='-1' should use all available gpu devices
* gpus=N
    * N=0: no gpus should be used.
    * N>0: N gpus should be used
* gpus=list of ints or a comma separated string of numbers:
    Use the gpus indicated by the list or the string.

* Fixed code to pass all the changed tests for parsing gpus param.

* Refactoring parse_gpu_ids function.

* flake8 fixes.

* Updating documentation.

* flake8 fixes.

* flake8 fixes.

* flake8 fixes

* Update trainer.py

* Update dp_mixin.py

* Make reduce_distributed_output a stand alone function.
Fix imports.
Fix flake8.

* Add comet_ml dependency to tests requirements.txt

* Revert "Make reduce_distributed_output a stand alone function. Fix imports. Fix flake8."

This reverts commit eac0338

* Merge with master.
2019-10-23 05:05:09 -04:00
Nic Eggert 05cea3ff8b Save / Load Hyperparameters with checkpoint (#415)
* Save and load hparams from checkpoints

* Update docs

* Add warning when not saving hparams

* Missing import

* Update .run_local_tests.sh

* Update lm_test_module_mixins.py

* Update lightning_module_template.py
2019-10-23 04:48:24 -04:00