mirror of
https://github.com/wassname/pytorch-lightning.git
synced 2026-09-12 12:40:20 +08:00
No auto load weights (#985)
* remove autoload * remove autoload * added weights loading docs * checkpoint loading saving docs * checkpoint loading saving docs * checkpoint loading saving docs * docs (#1010) * remove autoload * remove autoload * added weights loading docs * checkpoint loading saving docs * checkpoint loading saving docs * checkpoint loading saving docs * docs * docs * docs * docs * docs * docs * docs * docs * docs * docs * docs * docs * docs * docs * docs * docs * docs * docs * docs * docs * docs * docs * docs * docs * docs * docs * docs * docs
This commit is contained in:
@@ -1,80 +0,0 @@
|
||||
Checkpointing
|
||||
==============
|
||||
|
||||
.. _model-saving:
|
||||
|
||||
Model saving
|
||||
-------------------
|
||||
To save a LightningModule, provide a :meth:`pytorch_lightning.callbacks.ModelCheckpoint` callback.
|
||||
|
||||
The Lightning checkpoint also saves the hparams (hyperparams) passed into the LightningModule init.
|
||||
|
||||
.. note:: hparams is a `Namespace <https://docs.python.org/2/library/argparse.html#argparse.Namespace>`_ or dictionary.
|
||||
|
||||
.. code-block:: python
|
||||
:emphasize-lines: 8
|
||||
|
||||
from argparse import Namespace
|
||||
|
||||
# usually these come from command line args
|
||||
args = Namespace(**{'learning_rate':0.001})
|
||||
|
||||
# define you module to have hparams as the first arg
|
||||
# this means your checkpoint will have everything that went into making
|
||||
# this model (in this case, learning rate)
|
||||
class MyLightningModule(pl.LightningModule):
|
||||
|
||||
def __init__(self, hparams, ...):
|
||||
self.hparams = hparams
|
||||
|
||||
my_model = MyLightningModule(args)
|
||||
|
||||
# auto-saves checkpoint
|
||||
checkpoint_callback = ModelCheckpoint(filepath='my_path')
|
||||
Trainer(checkpoint_callback=checkpoint_callback)
|
||||
|
||||
|
||||
Model loading
|
||||
-----------------------------------
|
||||
|
||||
To load a model, use :meth:`pytorch_lightning.core.LightningModule.load_from_checkpoint`
|
||||
|
||||
.. note:: If lightning created your checkpoint, your model will receive all the hyperparameters used
|
||||
to create the checkpoint. (See: :ref:`model-saving`).
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
# load weights without mapping
|
||||
MyLightningModule.load_from_checkpoint('path/to/checkpoint.ckpt')
|
||||
|
||||
# load weights mapping all weights from GPU 1 to GPU 0
|
||||
map_location = {'cuda:1':'cuda:0'}
|
||||
MyLightningModule.load_from_checkpoint('path/to/checkpoint.ckpt', map_location=map_location)
|
||||
|
||||
Restoring training session
|
||||
-----------------------------------
|
||||
|
||||
If you want to pick up training from where you left off, you have a few options.
|
||||
|
||||
1. Pass in a logger with the same experiment version to continue training.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
# train the first time and set the version number
|
||||
logger = TensorboardLogger(version=10)
|
||||
trainer = Trainer(logger=logger)
|
||||
trainer.fit(model)
|
||||
|
||||
# when you init another logger with that same version, the model
|
||||
# will continue where it left off
|
||||
logger = TensorboardLogger(version=10)
|
||||
trainer = Trainer(logger=logger)
|
||||
trainer.fit(model)
|
||||
|
||||
2. A second option is to pass in a path to a checkpoint (see: :ref:`pytorch_lightning.trainer.trainer.Trainer`).
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
# train the first time and set the version number
|
||||
trainer = Trainer(resume_from_checkpoint='some/path/to/my_checkpoint.ckpt')
|
||||
trainer.fit(model)
|
||||
@@ -45,7 +45,6 @@ PyTorch-Lightning Documentation
|
||||
:caption: Common Use Cases
|
||||
|
||||
apex
|
||||
checkpointing
|
||||
slurm
|
||||
debugging
|
||||
experiment_logging
|
||||
@@ -54,6 +53,7 @@ PyTorch-Lightning Documentation
|
||||
fast_training
|
||||
hooks
|
||||
multi_gpu
|
||||
weights_loading
|
||||
single_gpu
|
||||
sequences
|
||||
training_tricks
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
Saving and loading weights
|
||||
==========================
|
||||
|
||||
Lightning can automate saving and loading checkpoints.
|
||||
|
||||
Checkpoint saving
|
||||
-----------------
|
||||
|
||||
Checkpointing is enabled by default to the current working directory.
|
||||
To change the checkpoint path pass in:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
Trainer(default_save_path='/your/path/to/save/checkpoints')
|
||||
|
||||
To modify the behavior of checkpointing pass in your own callback.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from pytorch_lightning.callbacks import ModelCheckpoint
|
||||
|
||||
# DEFAULTS used by the Trainer
|
||||
checkpoint_callback = ModelCheckpoint(
|
||||
filepath=os.getcwd(),
|
||||
save_best_only=True,
|
||||
verbose=True,
|
||||
monitor='val_loss',
|
||||
mode='min',
|
||||
prefix=''
|
||||
)
|
||||
|
||||
trainer = Trainer(checkpoint_callback=checkpoint_callback)
|
||||
|
||||
|
||||
Or disable it by passing
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
trainer = Trainer(checkpoint_callback=False)
|
||||
|
||||
|
||||
The Lightning checkpoint also saves the hparams (hyperparams) passed into the LightningModule init.
|
||||
|
||||
.. note:: hparams is a `Namespace <https://docs.python.org/2/library/argparse.html#argparse.Namespace>`_.
|
||||
|
||||
.. code-block:: python
|
||||
:emphasize-lines: 8
|
||||
|
||||
from argparse import Namespace
|
||||
|
||||
# usually these come from command line args
|
||||
args = Namespace(**{'learning_rate':0.001})
|
||||
|
||||
# define you module to have hparams as the first arg
|
||||
# this means your checkpoint will have everything that went into making
|
||||
# this model (in this case, learning rate)
|
||||
class MyLightningModule(pl.LightningModule):
|
||||
|
||||
def __init__(self, hparams, ...):
|
||||
self.hparams = hparams
|
||||
|
||||
Checkpoint Loading
|
||||
------------------
|
||||
|
||||
You might want to not only load a model but also continue training it. Use this method to
|
||||
restore the trainer state as well. This will continue from the epoch and global step you last left off.
|
||||
However, the dataloaders will start from the first batch again (if you shuffled it shouldn't matter).
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
model = MyLightingModule.load_from_checkpoint(PATH)
|
||||
model.eval()
|
||||
y_hat = model(x)
|
||||
|
||||
A LightningModule is no different than a nn.Module. This means you can load it and use it for
|
||||
predictions as you would a nn.Module.
|
||||
Reference in New Issue
Block a user