mirror of
https://github.com/wassname/pytorch-lightning.git
synced 2026-09-09 11:32:07 +08:00
Deployed 5fd0bf8 with MkDocs version: 1.0.4
This commit is contained in:
@@ -280,6 +280,13 @@
|
||||
Minimal example
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#how-do-these-methods-fit-into-the-broader-training" title="How do these methods fit into the broader training?" class="md-nav__link">
|
||||
How do these methods fit into the broader training?
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
@@ -696,6 +703,13 @@
|
||||
Minimal example
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#how-do-these-methods-fit-into-the-broader-training" title="How do these methods fit into the broader training?" class="md-nav__link">
|
||||
How do these methods fit into the broader training?
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
@@ -980,6 +994,15 @@ class CoolModel(ptl.LightningModule):
|
||||
return DataLoader(MNIST(os.getcwd(), train=True, download=True, transform=transforms.ToTensor()), batch_size=32)
|
||||
</code></pre>
|
||||
|
||||
<hr />
|
||||
<h3 id="how-do-these-methods-fit-into-the-broader-training">How do these methods fit into the broader training?</h3>
|
||||
<p>The LightningModule interface is on the right. Each method corresponds to a part of a research project. Lightning automates everything not in blue. </p>
|
||||
<p align="center">
|
||||
<a href="https://github.com/williamFalcon/pytorch-lightning/blob/master/docs/source/_static/overview_flat.jpg">
|
||||
<img alt="" src="https://github.com/williamFalcon/pytorch-lightning/blob/master/docs/source/_static/overview_flat.jpg" height="900px">
|
||||
</a>
|
||||
</p>
|
||||
|
||||
<hr />
|
||||
<h3 id="training_step">training_step</h3>
|
||||
<pre><code class="python">def training_step(self, data_batch, batch_nb)
|
||||
|
||||
@@ -657,7 +657,7 @@ trainer = Trainer(process_position=1)
|
||||
<h4 id="save-a-snapshot-of-all-hyperparameters">Save a snapshot of all hyperparameters</h4>
|
||||
<p>Whenever you call .save() on the test-tube experiment it logs all the hyperparameters in current use.
|
||||
Give lightning a test-tube Experiment object to automate this for you.</p>
|
||||
<pre><code class="python">from test-tube import Experiment
|
||||
<pre><code class="python">from test_tube import Experiment
|
||||
|
||||
exp = Experiment(...)
|
||||
Trainer(experiment=exp)
|
||||
@@ -667,7 +667,7 @@ Trainer(experiment=exp)
|
||||
<h4 id="snapshot-code-for-a-training-run">Snapshot code for a training run</h4>
|
||||
<p>Whenever you call .save() on the test-tube experiment it snapshows all code and pushes to a git tag.
|
||||
Give lightning a test-tube Experiment object to automate this for you.</p>
|
||||
<pre><code class="python">from test-tube import Experiment
|
||||
<pre><code class="python">from test_tube import Experiment
|
||||
|
||||
exp = Experiment(create_git_tag=True)
|
||||
Trainer(experiment=exp)
|
||||
|
||||
+28
-28
@@ -506,40 +506,40 @@ trainer.fit(model)
|
||||
<p>But of course the fun is in all the advanced things it can do:</p>
|
||||
<p><strong>Checkpointing</strong> </p>
|
||||
<ul>
|
||||
<li>Model saving</li>
|
||||
<li>Model loading </li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Checkpointing/#model-saving">Model saving</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/LightningModule/methods/#load-from-metrics">Model loading</a> </li>
|
||||
</ul>
|
||||
<p><strong>Computing cluster (SLURM)</strong> </p>
|
||||
<ul>
|
||||
<li><a href="SLURM%20Managed%20Cluster/#running-grid-search-on-a-cluster">Running grid search on a cluster</a></li>
|
||||
<li><a href="SLURM%20Managed%20Cluster/#walltime-auto-resubmit">Walltime auto-resubmit</a> </li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/SLURM%20Managed%20Cluster#running-grid-search-on-a-cluster">Running grid search on a cluster</a> </li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/SLURM%20Managed%20Cluster#walltime-auto-resubmit">Walltime auto-resubmit</a> </li>
|
||||
</ul>
|
||||
<p><strong>Debugging</strong> </p>
|
||||
<ul>
|
||||
<li><a href="Debugging/#fast-dev-run">Fast dev run</a></li>
|
||||
<li><a href="Debugging/#inspect-gradient-norms">Inspect gradient norms</a></li>
|
||||
<li><a href="Debugging/#Log-gpu-usage">Log GPU usage</a></li>
|
||||
<li><a href="Debugging/#make-model-overfit-on-subset-of-data">Make model overfit on subset of data</a></li>
|
||||
<li><a href="Debugging/#print-the-parameter-count-by-layer">Print the parameter count by layer</a></li>
|
||||
<li><a href="Debugging/#print-which-gradients-are-nan">Pring which gradients are nan</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/debugging/#fast-dev-run">Fast dev run</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/debugging/#inspect-gradient-norms">Inspect gradient norms</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/debugging/#Log-gpu-usage">Log GPU usage</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/debugging/#make-model-overfit-on-subset-of-data">Make model overfit on subset of data</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/debugging/#print-the-parameter-count-by-layer">Print the parameter count by layer</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/debugging/#print-which-gradients-are-nan">Pring which gradients are nan</a></li>
|
||||
</ul>
|
||||
<p><strong>Distributed training</strong> </p>
|
||||
<ul>
|
||||
<li><a href="Distributed%20training/#16-bit-mixed-precision">16-bit mixed precision</a></li>
|
||||
<li><a href="Distributed%20training/#Multi-GPU">Multi-GPU</a></li>
|
||||
<li><a href="Distributed%20training/#Multi-node">Multi-node</a></li>
|
||||
<li><a href="Distributed%20training/#single-gpu">Single GPU</a></li>
|
||||
<li><a href="Distributed%20training/#self-balancing-architecture">Self-balancing architecture</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Distributed%20training/#16-bit-mixed-precision">16-bit mixed precision</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Distributed%20training/#Multi-GPU">Multi-GPU</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Distributed%20training/#Multi-node">Multi-node</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Distributed%20training/#single-gpu">Single GPU</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Distributed%20training/#self-balancing-architecture">Self-balancing architecture</a></li>
|
||||
</ul>
|
||||
<p><strong>Experiment Logging</strong> </p>
|
||||
<ul>
|
||||
<li><a href="Logging/#display-metrics-in-progress-bar">Display metrics in progress bar</a></li>
|
||||
<li><a href="Logging/#log-metric-row-every-k-batches">Log metric row every k batches</a></li>
|
||||
<li><a href="Logging/#process-position">Process position</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Logging/#display-metrics-in-progress-bar">Display metrics in progress bar</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Logging/#log-metric-row-every-k-batches">Log metric row every k batches</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Logging/#process-position">Process position</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Logging/#tensorboard-support">Tensorboard support</a></li>
|
||||
<li><a href="Logging/#save-a-snapshot-of-all-hyperparameters">Save a snapshot of all hyperparameters</a> </li>
|
||||
<li><a href="Logging/#snapshot-code-for-a-training-run">Snapshot code for a training run</a> </li>
|
||||
<li><a href="Logging/#write-logs-file-to-csv-every-k-batches">Write logs file to csv every k batches</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Logging/#save-a-snapshot-of-all-hyperparameters">Save a snapshot of all hyperparameters</a> </li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Logging/#snapshot-code-for-a-training-run">Snapshot code for a training run</a> </li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Logging/#write-logs-file-to-csv-every-k-batches">Write logs file to csv every k batches</a></li>
|
||||
</ul>
|
||||
<p><strong>Training loop</strong> </p>
|
||||
<ul>
|
||||
@@ -547,19 +547,19 @@ trainer.fit(model)
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#force-training-for-min-or-max-epochs">Force training for min or max epochs</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#force-disable-early-stop">Force disable early stop</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#gradient-clipping">Gradient Clipping</a></li>
|
||||
<li><a href="hooks">Hooks</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/hooks/">Hooks</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/LightningModule/RequiredTrainerInterface/#configure_optimizers">Learning rate scheduling</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/LightningModule/RequiredTrainerInterface/#configure_optimizers">Use multiple optimizers (like GANs)</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#set-how-much-of-the-training-set-to-check">Set how much of the training set to check (1-100%)</a></li>
|
||||
</ul>
|
||||
<p><strong>Validation loop</strong> </p>
|
||||
<ul>
|
||||
<li><a href="Validation%20Loop/#check-validation-every-n-epochs">Check validation every n epochs</a></li>
|
||||
<li><a href="hooks">Hooks</a></li>
|
||||
<li><a href="Validation%20Loop/#set-how-much-of-the-validation-set-to-check">Set how much of the validation set to check</a></li>
|
||||
<li><a href="Validation%20Loop/#set-how-much-of-the-test-set-to-check">Set how much of the test set to check</a></li>
|
||||
<li><a href="Validation%20Loop/#set-validation-check-frequency-within-1-training-epoch">Set validation check frequency within 1 training epoch</a></li>
|
||||
<li><a href="Validation%20Loop/#set-the-number-of-validation-sanity-steps">Set the number of validation sanity steps</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Validation%20loop/#check-validation-every-n-epochs">Check validation every n epochs</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/hooks/">Hooks</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Validation%20loop/#set-how-much-of-the-validation-set-to-check">Set how much of the validation set to check</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Validation%20loop/#set-how-much-of-the-test-set-to-check">Set how much of the test set to check</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Validation%20loop/#set-validation-check-frequency-within-1-training-epoch">Set validation check frequency within 1 training epoch</a></li>
|
||||
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Validation%20loop/#set-the-number-of-validation-sanity-steps">Set the number of validation sanity steps</a></li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
File diff suppressed because one or more lines are too long
+14
-14
@@ -2,72 +2,72 @@
|
||||
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-08-01</lastmod>
|
||||
<lastmod>2019-08-05</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-08-01</lastmod>
|
||||
<lastmod>2019-08-05</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-08-01</lastmod>
|
||||
<lastmod>2019-08-05</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-08-01</lastmod>
|
||||
<lastmod>2019-08-05</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-08-01</lastmod>
|
||||
<lastmod>2019-08-05</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-08-01</lastmod>
|
||||
<lastmod>2019-08-05</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-08-01</lastmod>
|
||||
<lastmod>2019-08-05</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-08-01</lastmod>
|
||||
<lastmod>2019-08-05</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-08-01</lastmod>
|
||||
<lastmod>2019-08-05</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-08-01</lastmod>
|
||||
<lastmod>2019-08-05</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-08-01</lastmod>
|
||||
<lastmod>2019-08-05</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-08-01</lastmod>
|
||||
<lastmod>2019-08-05</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-08-01</lastmod>
|
||||
<lastmod>2019-08-05</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-08-01</lastmod>
|
||||
<lastmod>2019-08-05</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
</urlset>
|
||||
Binary file not shown.
Binary file not shown.
|
After Width: | Height: | Size: 410 KiB |
Reference in New Issue
Block a user