add more detail to tbptt example (#755)

* add more detail to tbptt example

* warn user about new arg in training_step
This commit is contained in:
Jeremy Jordan
2020-02-01 15:51:42 -05:00
committed by GitHub
parent 76a1c67d87
commit 589815f6ab
2 changed files with 12 additions and 0 deletions
+8
View File
@@ -168,6 +168,14 @@ class LightningModule(ABC, GradInformation, ModelIO, ModelHooks):
# Truncated back-propagation through time
def training_step(self, batch, batch_idx, hiddens):
# hiddens are the hiddens from the previous truncated backprop step
...
out, hiddens = self.lstm(data, hiddens)
...
return {
"loss": ...,
"hiddens": hiddens # remember to detach() this
}
You can also return a -1 instead of a dict to stop the current loop. This is useful
if you want to break out of the current training epoch early.
+4
View File
@@ -448,6 +448,10 @@ class Trainer(TrainerIOMixin,
# backprop every 5 steps in a batch
trainer = Trainer(truncated_bptt_steps=5)
Using this feature requires updating your LightningModule's `training_step()` to include
a `hiddens` arg.
resume_from_checkpoint (str): To resume training from a specific checkpoint pass in the path here.k
Example::