mirror of
https://github.com/wassname/pytorch-lightning.git
synced 2026-08-21 11:20:03 +08:00
Adrian WälchliandJirka Borovec
a6de1b8d75
* add doctest to circleci * Revert "add doctest to circleci" This reverts commit c45b34ea911a81f87989f6c3a832b1e8d8c471c6. * Revert "Revert "add doctest to circleci"" This reverts commit 41fca97fdcfe1cf4f6bdb3bbba75d25fa3b11f70. * doctest docs rst files * Revert "doctest docs rst files" This reverts commit b4a2e83e3da5ed1909de500ec14b6b614527c07f. * doctest only rst * doctest debugging.rst * doctest apex * doctest callbacks * doctest early stopping * doctest for child modules * doctest experiment reporting * indentation * doctest fast training * doctest for hyperparams * doctests for lr_finder * doctests multi-gpu * more doctest * make doctest drone * fix label build error * update fast training * update invalid imports * fix problem with int device count * rebase stuff * wip * wip * wip * intro guide * add missing code block * circleci * logger import for doctest * test if doctest runs on drone * fix mnist download * also run install deps for building docs * install cmake * try sudo * hide output * try pip stuff * try to mock horovod * Tranfer -> Transfer * add torchvision to extras * revert pip stuff * mlflow file location * do not mock torch * torchvision * drone extra req. * try higher sphinx version * Revert "try higher sphinx version" This reverts commit 490ac28e46d6fd52352640dfdf0d765befa56988. * try coverage command * try coverage command * try undoc flag * newline * undo drone * report coverage * review Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com> * remove torchvision from extras * skip tests only if torchvision not available * fix testoutput torchvision Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
128 lines
3.6 KiB
ReStructuredText
128 lines
3.6 KiB
ReStructuredText
.. testsetup:: *
|
|
|
|
from pytorch_lightning.trainer.trainer import Trainer
|
|
|
|
|
|
Experiment Reporting
|
|
=====================
|
|
|
|
Lightning supports many different experiment loggers. These loggers allow you to monitor losses, images, text, etc...
|
|
as training progresses. They usually provide a GUI to visualize and can sometimes even snapshot hyperparameters
|
|
used in each experiment.
|
|
|
|
|
|
Control logging frequency
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
It may slow training down to log every single batch. Trainer has an option to log every k batches instead.
|
|
|
|
.. testcode::
|
|
|
|
k = 10
|
|
trainer = Trainer(row_log_interval=k)
|
|
|
|
Control log writing frequency
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Writing to a logger can be expensive. In Lightning you can set the interval at which you
|
|
want to log using this trainer flag.
|
|
|
|
.. seealso::
|
|
:class:`~pytorch_lightning.trainer.trainer.Trainer`
|
|
|
|
.. testcode::
|
|
|
|
k = 100
|
|
trainer = Trainer(log_save_interval=k)
|
|
|
|
Log metrics
|
|
^^^^^^^^^^^
|
|
|
|
To plot metrics into whatever logger you passed in (tensorboard, comet, neptune, TRAINS, etc...)
|
|
|
|
1. training_epoch_end, validation_epoch_end, test_epoch_end will all log anything in the "log" key of the return dict.
|
|
|
|
.. testcode::
|
|
|
|
def training_epoch_end(self, outputs):
|
|
loss = some_loss()
|
|
...
|
|
|
|
logs = {'train_loss': loss}
|
|
results = {'log': logs}
|
|
return results
|
|
|
|
def validation_epoch_end(self, outputs):
|
|
loss = some_loss()
|
|
...
|
|
|
|
logs = {'val_loss': loss}
|
|
results = {'log': logs}
|
|
return results
|
|
|
|
def test_epoch_end(self, outputs):
|
|
loss = some_loss()
|
|
...
|
|
|
|
logs = {'test_loss': loss}
|
|
results = {'log': logs}
|
|
return results
|
|
|
|
2. In addition, you can also use any arbitrary functionality from a particular logger from within your LightningModule.
|
|
For instance, here we log images using tensorboard.
|
|
|
|
.. testcode::
|
|
:skipif: not TORCHVISION_AVAILABLE
|
|
|
|
def training_step(self, batch, batch_idx):
|
|
self.generated_imgs = self.decoder.generate()
|
|
|
|
sample_imgs = self.generated_imgs[:6]
|
|
grid = torchvision.utils.make_grid(sample_imgs)
|
|
self.logger.experiment.add_image('generated_images', grid, 0)
|
|
|
|
...
|
|
return results
|
|
|
|
Modify progress bar
|
|
^^^^^^^^^^^^^^^^^^^
|
|
|
|
Each return dict from the training_end, validation_end, testing_end and training_step also has
|
|
a key called "progress_bar".
|
|
|
|
Here we show the validation loss in the progress bar
|
|
|
|
.. testcode::
|
|
|
|
def validation_epoch_end(self, outputs):
|
|
loss = some_loss()
|
|
...
|
|
|
|
logs = {'val_loss': loss}
|
|
results = {'progress_bar': logs}
|
|
return results
|
|
|
|
Snapshot hyperparameters
|
|
^^^^^^^^^^^^^^^^^^^^^^^^
|
|
When training a model, it's useful to know what hyperparams went into that model.
|
|
When Lightning creates a checkpoint, it stores a key "hparams" with the hyperparams.
|
|
|
|
.. code-block:: python
|
|
|
|
lightning_checkpoint = torch.load(filepath, map_location=lambda storage, loc: storage)
|
|
hyperparams = lightning_checkpoint['hparams']
|
|
|
|
Some loggers also allow logging the hyperparams used in the experiment. For instance,
|
|
when using the TestTubeLogger or the TensorBoardLogger, all hyperparams will show
|
|
in the `hparams tab <https://pytorch.org/docs/stable/tensorboard.html#torch.utils.tensorboard.writer.SummaryWriter.add_hparams>`_.
|
|
|
|
Snapshot code
|
|
^^^^^^^^^^^^^
|
|
Loggers also allow you to snapshot a copy of the code used in this experiment.
|
|
For example, TestTubeLogger does this with a flag:
|
|
|
|
.. testcode::
|
|
|
|
from pytorch_lightning.loggers import TestTubeLogger
|
|
logger = TestTubeLogger('.', create_git_tag=True)
|