From 4e9fd95f79f4d60868ad258d7a2ab38c55e0c4f3 Mon Sep 17 00:00:00 2001
From: s-rog <55400948+s-rog@users.noreply.github.com>
Date: Sun, 3 Nov 2019 18:26:27 +0800
Subject: [PATCH] packed sequence clarification in train_dataloader (#443)

* packed sequence clarification in train_dataloader

* moved changes to training loop

* removed changes from required interface

* added index entry
---
 .../RequiredTrainerInterface.md               |  2 +-
 docs/Trainer/Training Loop.md                 | 19 +++++++++++++++++++
 docs/Trainer/index.md                         |  1 +
 3 files changed, 21 insertions(+), 1 deletion(-)

diff --git a/docs/LightningModule/RequiredTrainerInterface.md b/docs/LightningModule/RequiredTrainerInterface.md
index e0299615..53bfc821 100644
--- a/docs/LightningModule/RequiredTrainerInterface.md
+++ b/docs/LightningModule/RequiredTrainerInterface.md
@@ -186,7 +186,7 @@ break out of the current training epoch early.
 def train_dataloader(self)
 ```
 Called by lightning during training loop. Make sure to use the @pl.data_loader decorator, this ensures not calling this function until the data are needed.   
-If you want to change the data during every epoch DON'T use the data_loader decorator.   
+If you want to change the data during every epoch DON'T use the data_loader decorator.
 
 ##### Return
 PyTorch DataLoader
diff --git a/docs/Trainer/Training Loop.md b/docs/Trainer/Training Loop.md
index f471c2a2..bc0c83c8 100644
--- a/docs/Trainer/Training Loop.md	
+++ b/docs/Trainer/Training Loop.md	
@@ -92,6 +92,25 @@ trainer = Trainer(train_percent_check=1.0)
 trainer = Trainer(train_percent_check=0.1)
 ```
 
+---
+#### Packed sequences as inputs
+When using PackedSequence, do 2 things:
+1. return either a padded tensor in dataset or a list of variable length tensors in the dataloader collate_fn (example above shows the list implementation).    
+2. Pack the sequence in forward or training and validation steps depending on use case.
+
+``` {.python}
+# For use in dataloader
+def collate_fn(batch):
+    x = [item[0] for item in batch]
+    y = [item[1] for item in batch]
+    return x, y
+
+# In module
+def training_step(self, batch, batch_nb):
+    x = rnn.pack_sequence(batch[0], enforce_sorted=False)
+    y = rnn.pack_sequence(batch[1], enforce_sorted=False)
+```
+
 ---
 #### Truncated Back Propagation Through Time
 There are times when multiple backwards passes are needed for each batch. For example, it may save memory to use Truncated Back Propagation Through Time when training RNNs.
diff --git a/docs/Trainer/index.md b/docs/Trainer/index.md
index d71c07ed..13df0539 100644
--- a/docs/Trainer/index.md
+++ b/docs/Trainer/index.md
@@ -71,6 +71,7 @@ But of course the fun is in all the advanced things it can do:
 - [Use multiple optimizers (like GANs)](https://williamfalcon.github.io/pytorch-lightning/LightningModule/RequiredTrainerInterface/#configure_optimizers)
 - [Set how much of the training set to check (1-100%)](https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#set-how-much-of-the-training-set-to-check)
 - [Step optimizers at arbitrary intervals](https://williamfalcon.github.io/pytorch-lightning/Trainer/hooks/#optimizer_step)
+- [Packed sequences](https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#packed-sequences-as-inputs)
 - [Truncated Back Propagation Through Time](https://williamfalcon.github.io/pytorch-lightning//Training%20Loop/#truncated-back-propation-through-time)
 
 **Validation loop**