wassname/pytorch-lightning
1
0
Fork 0
mirror of https://github.com/wassname/pytorch-lightning.git synced 2026-09-11 12:31:23 +08:00
Code Issues Packages Projects Releases Wiki Activity

Docs format - Trainer & LModule (#1055)

Browse Source 
* format Trainer

* format LModule

* format LModule

* linted

* Update lightning.py

Co-authored-by: William Falcon <waf2107@columbia.edu>
This commit is contained in:
William Falcon
2020-03-05 18:52:17 -05:00
committed by GitHub
parent 8ff19dda22
commit 8827bd3a3b
4 changed files with 704 additions and 719 deletions
Download Patch File Download Diff File Expand all files Collapse all files
pytorch_lightning/core/__init__.py
+47 -56
View File
@@ -7,57 +7,48 @@ A LightningModule organizes your PyTorch code into the following sections:
Notice a few things.
1. It's the SAME code.
1. It's the SAME code.
2. The PyTorch code IS NOT abstracted - just organized.
3. All the other code that didn't go in the LightningModule has been automated for you by the trainer
.. code-block:: python
2. The PyTorch code IS NOT abstracted - just organized.
net = Net()
trainer = Trainer()
trainer.fit(net)
3. All the other code that didn't go in the LightningModule has been automated
for you by the trainer
4. There are no .cuda() or .to() calls... Lightning does these for you.
.. code-block:: python
.. code-block:: python
# don't do in lightning
x = torch.Tensor(2, 3)
x = x.cuda()
x = x.to(device)
net = Net()
trainer = Trainer()
trainer.fit(net)
# do this instead
x = x # leave it alone!
4. There are no .cuda() or .to() calls... Lightning does these for you.
# or to init a new tensor
new_x = torch.Tensor(2, 3)
new_x = new_x.type_as(x.type())
.. code-block:: python
5. There are no samplers for distributed, Lightning also does this for you.
.. code-block:: python
# don't do in lightning
x = torch.Tensor(2, 3)
x = x.cuda()
x = x.to(device)
# Don't do in Lightning...
data = MNIST(...)
sampler = DistributedSampler(data)
DataLoader(data, sampler=sampler)
# do this instead
x = x # leave it alone!
# do this instead
data = MNIST(...)
DataLoader(data)
# or to init a new tensor
new_x = torch.Tensor(2, 3)
new_x = new_x.type_as(x.type())
6. A LightingModule is a torch.nn.Module but with added functionality. Use it as such!
.. code-block:: python
5. There are no samplers for distributed, Lightning also does this for you.
.. code-block:: python
# Don't do in Lightning...
data = MNIST(...)
sampler = DistributedSampler(data)
DataLoader(data, sampler=sampler)
# do this instead
data = MNIST(...)
DataLoader(data)
6. A LightingModule is a torch.nn.Module but with added functionality. Use it as such!
.. code-block:: python
net = Net.load_from_checkpoint(PATH)
net.freeze()
out = net(x)
net = Net.load_from_checkpoint(PATH)
net.freeze()
out = net(x)
Thus, to use Lightning, you just need to organize your code which takes about 30 minutes,
(and let's be real, you probably should do anyhow).
@@ -110,7 +101,7 @@ Which you can train by doing:
trainer.fit(model)
----------
---
Training loop structure
-----------------------
@@ -190,7 +181,7 @@ don't run your test data by accident. Instead you have to explicitly call:
trainer = Trainer()
trainer.test(model)
-------------
---
Training_step_end method
------------------------
@@ -220,7 +211,7 @@ which allows you to operate on the pieces of the batch
# like calculate validation set accuracy or loss
training_epoch_end(val_outs)
-------------
---
Remove cuda calls
-----------------
@@ -239,7 +230,7 @@ When you init a new tensor in your code, just use type_as
z = sample_noise()
z = z.type_as(x.type())
-------------
---
Data preparation
----------------
@@ -262,27 +253,27 @@ allow for this
# do stuff that writes to disk or should be done once
# this will only happen from the master GPU or TPU core
.. note:: ```prepare_data``` is called once.
.. note:: ``prepare_data`` is called once.
Lifecycle
---------
The methods in the LightningModule are called in this order:
1. ```__init__```
2. ```prepare_data```
3. ```configure_optimizers```
4. ```prepare_data```
5. ```train_dataloader```
1. ```__init__```
2. ```prepare_data```
3. ```configure_optimizers```
4. ```prepare_data```
5. ```train_dataloader```
If you define a validation loop then
If you define a validation loop then
6. ```val_dataloader```
6. ```val_dataloader```
And if you define a test loop:
And if you define a test loop:
7. ```test_dataloader```
7. ```test_dataloader```
.. note:: test_dataloader is only called with .test()
.. note:: ``test_dataloader`` is only called with ``.test()``
In every epoch, the loop methods are called in this frequency:
pytorch_lightning/core/lightning.py
+642 -648
View File
File diff suppressed because it is too large Load Diff
pytorch_lightning/trainer/__init__.py
+1 -1
View File
@@ -54,7 +54,7 @@ main.py file this way
main(args)
So you can run it like so:
So you can run it like so:distributed_backend
.. code-block:: bash
pytorch_lightning/trainer/trainer.py
+14 -14
View File
@@ -138,15 +138,15 @@ class Trainer(TrainerIOMixin,
gradient_clip_val: 0 means don't clip.
gradient_clip:
.. warning:: deprecated 0.6.1 Use `gradient_clip_val` instead. Will remove 0.8.0.
.. warning:: deprecated 0.6.1 Use `gradient_clip_val` instead. Will remove 0.8.0.
process_position: orders the tqdm bar when running multiple models on same machine.
num_nodes: number of GPU nodes for distributed training.
nb_gpu_nodes:
.. warning:: .. deprecated:: 0.6.1
Use `num_nodes` instead. Will remove 0.8.0.
.. warning:: .. deprecated:: 0.6.1
Use `num_nodes` instead. Will remove 0.8.0.
gpus: Which GPUs to train on.
@@ -169,14 +169,14 @@ class Trainer(TrainerIOMixin,
max_epochs: Stop training once this number of epochs is reached.
max_nb_epochs:
.. warning:: .. deprecated:: 0.6.1
Use `max_epochs` instead. Will remove 0.8.0.
.. warning:: .. deprecated:: 0.6.1
Use `max_epochs` instead. Will remove 0.8.0.
min_epochs: Force training for at least these many epochs
min_nb_epochs:
.. warning:: .. deprecated:: 0.6.1
Use `min_epochs` instead. Will remove 0.8.0.
.. warning:: .. deprecated:: 0.6.1
Use `min_epochs` instead. Will remove 0.8.0.
max_steps: Stop training after this number of steps. Disabled by default (None).
@@ -195,14 +195,14 @@ class Trainer(TrainerIOMixin,
row_log_interval: How often to add logging rows (does not write to disk)
add_row_log_interval:
.. warning:: .. deprecated:: 0.6.1
Use `row_log_interval` instead. Will remove 0.8.0.
.. warning:: .. deprecated:: 0.6.1
Use `row_log_interval` instead. Will remove 0.8.0.
distributed_backend: The distributed backend to use.
use_amp:
.. warning:: .. deprecated:: 0.7.0
Use `precision` instead. Will remove 0.8.0.
.. warning:: .. deprecated:: 0.7.0
Use `precision` instead. Will remove 0.8.0.
precision: Full precision (32), half precision (16).
@@ -217,8 +217,8 @@ class Trainer(TrainerIOMixin,
num_sanity_val_steps: Sanity check runs n batches of val before starting the training routine.
nb_sanity_val_steps:
.. warning:: .. deprecated:: 0.7.0
Use `num_sanity_val_steps` instead. Will remove 0.8.0.
.. warning:: .. deprecated:: 0.7.0
Use `num_sanity_val_steps` instead. Will remove 0.8.0.
truncated_bptt_steps: Truncated back prop breaks performs backprop every k steps of
@@ -228,7 +228,7 @@ class Trainer(TrainerIOMixin,
reload_dataloaders_every_epoch: Set to True to reload dataloaders every epoch
benchmark (bool): If true enables cudnn.benchmark.
benchmark: If true enables cudnn.benchmark.
"""
# Init callbacks