Deployed 5fd0bf8 with MkDocs version: 1.0.4

This commit is contained in:
William Falcon
2019-08-05 00:21:12 -05:00
parent e9e43b28f1
commit 69a655280a
7 changed files with 68 additions and 45 deletions
@@ -280,6 +280,13 @@
Minimal example
</a>
</li>
<li class="md-nav__item">
<a href="#how-do-these-methods-fit-into-the-broader-training" title="How do these methods fit into the broader training?" class="md-nav__link">
How do these methods fit into the broader training?
</a>
</li>
<li class="md-nav__item">
@@ -696,6 +703,13 @@
Minimal example
</a>
</li>
<li class="md-nav__item">
<a href="#how-do-these-methods-fit-into-the-broader-training" title="How do these methods fit into the broader training?" class="md-nav__link">
How do these methods fit into the broader training?
</a>
</li>
<li class="md-nav__item">
@@ -980,6 +994,15 @@ class CoolModel(ptl.LightningModule):
return DataLoader(MNIST(os.getcwd(), train=True, download=True, transform=transforms.ToTensor()), batch_size=32)
</code></pre>
<hr />
<h3 id="how-do-these-methods-fit-into-the-broader-training">How do these methods fit into the broader training?</h3>
<p>The LightningModule interface is on the right. Each method corresponds to a part of a research project. Lightning automates everything not in blue. </p>
<p align="center">
<a href="https://github.com/williamFalcon/pytorch-lightning/blob/master/docs/source/_static/overview_flat.jpg">
<img alt="" src="https://github.com/williamFalcon/pytorch-lightning/blob/master/docs/source/_static/overview_flat.jpg" height="900px">
</a>
</p>
<hr />
<h3 id="training_step">training_step</h3>
<pre><code class="python">def training_step(self, data_batch, batch_nb)
+2 -2
View File
@@ -657,7 +657,7 @@ trainer = Trainer(process_position=1)
<h4 id="save-a-snapshot-of-all-hyperparameters">Save a snapshot of all hyperparameters</h4>
<p>Whenever you call .save() on the test-tube experiment it logs all the hyperparameters in current use.
Give lightning a test-tube Experiment object to automate this for you.</p>
<pre><code class="python">from test-tube import Experiment
<pre><code class="python">from test_tube import Experiment
exp = Experiment(...)
Trainer(experiment=exp)
@@ -667,7 +667,7 @@ Trainer(experiment=exp)
<h4 id="snapshot-code-for-a-training-run">Snapshot code for a training run</h4>
<p>Whenever you call .save() on the test-tube experiment it snapshows all code and pushes to a git tag.
Give lightning a test-tube Experiment object to automate this for you.</p>
<pre><code class="python">from test-tube import Experiment
<pre><code class="python">from test_tube import Experiment
exp = Experiment(create_git_tag=True)
Trainer(experiment=exp)
+28 -28
View File
@@ -506,40 +506,40 @@ trainer.fit(model)
<p>But of course the fun is in all the advanced things it can do:</p>
<p><strong>Checkpointing</strong> </p>
<ul>
<li>Model saving</li>
<li>Model loading </li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Checkpointing/#model-saving">Model saving</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/LightningModule/methods/#load-from-metrics">Model loading</a> </li>
</ul>
<p><strong>Computing cluster (SLURM)</strong> </p>
<ul>
<li><a href="SLURM%20Managed%20Cluster/#running-grid-search-on-a-cluster">Running grid search on a cluster</a></li>
<li><a href="SLURM%20Managed%20Cluster/#walltime-auto-resubmit">Walltime auto-resubmit</a> </li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/SLURM%20Managed%20Cluster#running-grid-search-on-a-cluster">Running grid search on a cluster</a> </li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/SLURM%20Managed%20Cluster#walltime-auto-resubmit">Walltime auto-resubmit</a> </li>
</ul>
<p><strong>Debugging</strong> </p>
<ul>
<li><a href="Debugging/#fast-dev-run">Fast dev run</a></li>
<li><a href="Debugging/#inspect-gradient-norms">Inspect gradient norms</a></li>
<li><a href="Debugging/#Log-gpu-usage">Log GPU usage</a></li>
<li><a href="Debugging/#make-model-overfit-on-subset-of-data">Make model overfit on subset of data</a></li>
<li><a href="Debugging/#print-the-parameter-count-by-layer">Print the parameter count by layer</a></li>
<li><a href="Debugging/#print-which-gradients-are-nan">Pring which gradients are nan</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/debugging/#fast-dev-run">Fast dev run</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/debugging/#inspect-gradient-norms">Inspect gradient norms</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/debugging/#Log-gpu-usage">Log GPU usage</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/debugging/#make-model-overfit-on-subset-of-data">Make model overfit on subset of data</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/debugging/#print-the-parameter-count-by-layer">Print the parameter count by layer</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/debugging/#print-which-gradients-are-nan">Pring which gradients are nan</a></li>
</ul>
<p><strong>Distributed training</strong> </p>
<ul>
<li><a href="Distributed%20training/#16-bit-mixed-precision">16-bit mixed precision</a></li>
<li><a href="Distributed%20training/#Multi-GPU">Multi-GPU</a></li>
<li><a href="Distributed%20training/#Multi-node">Multi-node</a></li>
<li><a href="Distributed%20training/#single-gpu">Single GPU</a></li>
<li><a href="Distributed%20training/#self-balancing-architecture">Self-balancing architecture</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Distributed%20training/#16-bit-mixed-precision">16-bit mixed precision</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Distributed%20training/#Multi-GPU">Multi-GPU</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Distributed%20training/#Multi-node">Multi-node</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Distributed%20training/#single-gpu">Single GPU</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Distributed%20training/#self-balancing-architecture">Self-balancing architecture</a></li>
</ul>
<p><strong>Experiment Logging</strong> </p>
<ul>
<li><a href="Logging/#display-metrics-in-progress-bar">Display metrics in progress bar</a></li>
<li><a href="Logging/#log-metric-row-every-k-batches">Log metric row every k batches</a></li>
<li><a href="Logging/#process-position">Process position</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Logging/#display-metrics-in-progress-bar">Display metrics in progress bar</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Logging/#log-metric-row-every-k-batches">Log metric row every k batches</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Logging/#process-position">Process position</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Logging/#tensorboard-support">Tensorboard support</a></li>
<li><a href="Logging/#save-a-snapshot-of-all-hyperparameters">Save a snapshot of all hyperparameters</a> </li>
<li><a href="Logging/#snapshot-code-for-a-training-run">Snapshot code for a training run</a> </li>
<li><a href="Logging/#write-logs-file-to-csv-every-k-batches">Write logs file to csv every k batches</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Logging/#save-a-snapshot-of-all-hyperparameters">Save a snapshot of all hyperparameters</a> </li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Logging/#snapshot-code-for-a-training-run">Snapshot code for a training run</a> </li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Logging/#write-logs-file-to-csv-every-k-batches">Write logs file to csv every k batches</a></li>
</ul>
<p><strong>Training loop</strong> </p>
<ul>
@@ -547,19 +547,19 @@ trainer.fit(model)
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#force-training-for-min-or-max-epochs">Force training for min or max epochs</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#force-disable-early-stop">Force disable early stop</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#gradient-clipping">Gradient Clipping</a></li>
<li><a href="hooks">Hooks</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/hooks/">Hooks</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/LightningModule/RequiredTrainerInterface/#configure_optimizers">Learning rate scheduling</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/LightningModule/RequiredTrainerInterface/#configure_optimizers">Use multiple optimizers (like GANs)</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#set-how-much-of-the-training-set-to-check">Set how much of the training set to check (1-100%)</a></li>
</ul>
<p><strong>Validation loop</strong> </p>
<ul>
<li><a href="Validation%20Loop/#check-validation-every-n-epochs">Check validation every n epochs</a></li>
<li><a href="hooks">Hooks</a></li>
<li><a href="Validation%20Loop/#set-how-much-of-the-validation-set-to-check">Set how much of the validation set to check</a></li>
<li><a href="Validation%20Loop/#set-how-much-of-the-test-set-to-check">Set how much of the test set to check</a></li>
<li><a href="Validation%20Loop/#set-validation-check-frequency-within-1-training-epoch">Set validation check frequency within 1 training epoch</a></li>
<li><a href="Validation%20Loop/#set-the-number-of-validation-sanity-steps">Set the number of validation sanity steps</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Validation%20loop/#check-validation-every-n-epochs">Check validation every n epochs</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/hooks/">Hooks</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Validation%20loop/#set-how-much-of-the-validation-set-to-check">Set how much of the validation set to check</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Validation%20loop/#set-how-much-of-the-test-set-to-check">Set how much of the test set to check</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Validation%20loop/#set-validation-check-frequency-within-1-training-epoch">Set validation check frequency within 1 training epoch</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Validation%20loop/#set-the-number-of-validation-sanity-steps">Set the number of validation sanity steps</a></li>
</ul>
File diff suppressed because one or more lines are too long
+14 -14
View File
@@ -2,72 +2,72 @@
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>None</loc>
<lastmod>2019-08-01</lastmod>
<lastmod>2019-08-05</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-08-01</lastmod>
<lastmod>2019-08-05</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-08-01</lastmod>
<lastmod>2019-08-05</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-08-01</lastmod>
<lastmod>2019-08-05</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-08-01</lastmod>
<lastmod>2019-08-05</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-08-01</lastmod>
<lastmod>2019-08-05</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-08-01</lastmod>
<lastmod>2019-08-05</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-08-01</lastmod>
<lastmod>2019-08-05</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-08-01</lastmod>
<lastmod>2019-08-05</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-08-01</lastmod>
<lastmod>2019-08-05</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-08-01</lastmod>
<lastmod>2019-08-05</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-08-01</lastmod>
<lastmod>2019-08-05</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-08-01</lastmod>
<lastmod>2019-08-05</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-08-01</lastmod>
<lastmod>2019-08-05</lastmod>
<changefreq>daily</changefreq>
</url>
</urlset>
BIN
View File
Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 410 KiB