mirror of
https://github.com/wassname/pytorch-lightning.git
synced 2026-09-10 12:21:57 +08:00
packed sequence clarification in train_dataloader (#443)
* packed sequence clarification in train_dataloader * moved changes to training loop * removed changes from required interface * added index entry
This commit is contained in:
@@ -186,7 +186,7 @@ break out of the current training epoch early.
|
||||
def train_dataloader(self)
|
||||
```
|
||||
Called by lightning during training loop. Make sure to use the @pl.data_loader decorator, this ensures not calling this function until the data are needed.
|
||||
If you want to change the data during every epoch DON'T use the data_loader decorator.
|
||||
If you want to change the data during every epoch DON'T use the data_loader decorator.
|
||||
|
||||
##### Return
|
||||
PyTorch DataLoader
|
||||
|
||||
@@ -92,6 +92,25 @@ trainer = Trainer(train_percent_check=1.0)
|
||||
trainer = Trainer(train_percent_check=0.1)
|
||||
```
|
||||
|
||||
---
|
||||
#### Packed sequences as inputs
|
||||
When using PackedSequence, do 2 things:
|
||||
1. return either a padded tensor in dataset or a list of variable length tensors in the dataloader collate_fn (example above shows the list implementation).
|
||||
2. Pack the sequence in forward or training and validation steps depending on use case.
|
||||
|
||||
``` {.python}
|
||||
# For use in dataloader
|
||||
def collate_fn(batch):
|
||||
x = [item[0] for item in batch]
|
||||
y = [item[1] for item in batch]
|
||||
return x, y
|
||||
|
||||
# In module
|
||||
def training_step(self, batch, batch_nb):
|
||||
x = rnn.pack_sequence(batch[0], enforce_sorted=False)
|
||||
y = rnn.pack_sequence(batch[1], enforce_sorted=False)
|
||||
```
|
||||
|
||||
---
|
||||
#### Truncated Back Propagation Through Time
|
||||
There are times when multiple backwards passes are needed for each batch. For example, it may save memory to use Truncated Back Propagation Through Time when training RNNs.
|
||||
|
||||
@@ -71,6 +71,7 @@ But of course the fun is in all the advanced things it can do:
|
||||
- [Use multiple optimizers (like GANs)](https://williamfalcon.github.io/pytorch-lightning/LightningModule/RequiredTrainerInterface/#configure_optimizers)
|
||||
- [Set how much of the training set to check (1-100%)](https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#set-how-much-of-the-training-set-to-check)
|
||||
- [Step optimizers at arbitrary intervals](https://williamfalcon.github.io/pytorch-lightning/Trainer/hooks/#optimizer_step)
|
||||
- [Packed sequences](https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#packed-sequences-as-inputs)
|
||||
- [Truncated Back Propagation Through Time](https://williamfalcon.github.io/pytorch-lightning//Training%20Loop/#truncated-back-propation-through-time)
|
||||
|
||||
**Validation loop**
|
||||
|
||||
Reference in New Issue
Block a user