No auto load weights (#985)

* remove autoload

* remove autoload

* added weights loading docs

* checkpoint loading saving docs

* checkpoint loading saving docs

* checkpoint loading saving docs

* docs (#1010)

* remove autoload

* remove autoload

* added weights loading docs

* checkpoint loading saving docs

* checkpoint loading saving docs

* checkpoint loading saving docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs

* docs
This commit is contained in:
William Falcon
2020-03-02 17:12:22 -05:00
committed by GitHub
parent e15c0419c0
commit 2a04be0386
10 changed files with 112 additions and 325 deletions
-80
View File
@@ -1,80 +0,0 @@
Checkpointing
==============
.. _model-saving:
Model saving
-------------------
To save a LightningModule, provide a :meth:`pytorch_lightning.callbacks.ModelCheckpoint` callback.
The Lightning checkpoint also saves the hparams (hyperparams) passed into the LightningModule init.
.. note:: hparams is a `Namespace <https://docs.python.org/2/library/argparse.html#argparse.Namespace>`_ or dictionary.
.. code-block:: python
:emphasize-lines: 8
from argparse import Namespace
# usually these come from command line args
args = Namespace(**{'learning_rate':0.001})
# define you module to have hparams as the first arg
# this means your checkpoint will have everything that went into making
# this model (in this case, learning rate)
class MyLightningModule(pl.LightningModule):
def __init__(self, hparams, ...):
self.hparams = hparams
my_model = MyLightningModule(args)
# auto-saves checkpoint
checkpoint_callback = ModelCheckpoint(filepath='my_path')
Trainer(checkpoint_callback=checkpoint_callback)
Model loading
-----------------------------------
To load a model, use :meth:`pytorch_lightning.core.LightningModule.load_from_checkpoint`
.. note:: If lightning created your checkpoint, your model will receive all the hyperparameters used
to create the checkpoint. (See: :ref:`model-saving`).
.. code-block:: python
# load weights without mapping
MyLightningModule.load_from_checkpoint('path/to/checkpoint.ckpt')
# load weights mapping all weights from GPU 1 to GPU 0
map_location = {'cuda:1':'cuda:0'}
MyLightningModule.load_from_checkpoint('path/to/checkpoint.ckpt', map_location=map_location)
Restoring training session
-----------------------------------
If you want to pick up training from where you left off, you have a few options.
1. Pass in a logger with the same experiment version to continue training.
.. code-block:: python
# train the first time and set the version number
logger = TensorboardLogger(version=10)
trainer = Trainer(logger=logger)
trainer.fit(model)
# when you init another logger with that same version, the model
# will continue where it left off
logger = TensorboardLogger(version=10)
trainer = Trainer(logger=logger)
trainer.fit(model)
2. A second option is to pass in a path to a checkpoint (see: :ref:`pytorch_lightning.trainer.trainer.Trainer`).
.. code-block:: python
# train the first time and set the version number
trainer = Trainer(resume_from_checkpoint='some/path/to/my_checkpoint.ckpt')
trainer.fit(model)
+1 -1
View File
@@ -45,7 +45,6 @@ PyTorch-Lightning Documentation
:caption: Common Use Cases
apex
checkpointing
slurm
debugging
experiment_logging
@@ -54,6 +53,7 @@ PyTorch-Lightning Documentation
fast_training
hooks
multi_gpu
weights_loading
single_gpu
sequences
training_tricks
+76
View File
@@ -0,0 +1,76 @@
Saving and loading weights
==========================
Lightning can automate saving and loading checkpoints.
Checkpoint saving
-----------------
Checkpointing is enabled by default to the current working directory.
To change the checkpoint path pass in:
.. code-block:: python
Trainer(default_save_path='/your/path/to/save/checkpoints')
To modify the behavior of checkpointing pass in your own callback.
.. code-block:: python
from pytorch_lightning.callbacks import ModelCheckpoint
# DEFAULTS used by the Trainer
checkpoint_callback = ModelCheckpoint(
filepath=os.getcwd(),
save_best_only=True,
verbose=True,
monitor='val_loss',
mode='min',
prefix=''
)
trainer = Trainer(checkpoint_callback=checkpoint_callback)
Or disable it by passing
.. code-block:: python
trainer = Trainer(checkpoint_callback=False)
The Lightning checkpoint also saves the hparams (hyperparams) passed into the LightningModule init.
.. note:: hparams is a `Namespace <https://docs.python.org/2/library/argparse.html#argparse.Namespace>`_.
.. code-block:: python
:emphasize-lines: 8
from argparse import Namespace
# usually these come from command line args
args = Namespace(**{'learning_rate':0.001})
# define you module to have hparams as the first arg
# this means your checkpoint will have everything that went into making
# this model (in this case, learning rate)
class MyLightningModule(pl.LightningModule):
def __init__(self, hparams, ...):
self.hparams = hparams
Checkpoint Loading
------------------
You might want to not only load a model but also continue training it. Use this method to
restore the trainer state as well. This will continue from the epoch and global step you last left off.
However, the dataloaders will start from the first batch again (if you shuffled it shouldn't matter).
.. code-block:: python
model = MyLightingModule.load_from_checkpoint(PATH)
model.eval()
y_hat = model(x)
A LightningModule is no different than a nn.Module. This means you can load it and use it for
predictions as you would a nn.Module.