mirror of
https://github.com/wassname/pytorch-lightning.git
synced 2026-09-09 11:32:07 +08:00
Deployed a093d11 with MkDocs version: 1.0.4
This commit is contained in:
William Falcon
@@ -360,6 +360,13 @@
|
||||
Choosing a backend
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#cuda-flags" title="CUDA flags" class="md-nav__link">
|
||||
CUDA flags
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
@@ -544,6 +551,13 @@
|
||||
Choosing a backend
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#cuda-flags" title="CUDA flags" class="md-nav__link">
|
||||
CUDA flags
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
@@ -621,6 +635,15 @@ trainer = Trainer(distributed_backend='ddp')
|
||||
We recommend you use DistributedDataparallel even for single-node multi-GPU training. It is MUCH faster than DP but <em>may</em>
|
||||
have configuration issues depending on your cluster.</p>
|
||||
<p>For a deeper understanding of what lightning is doing, feel free to read <a href="https://medium.com/@_willfalcon/9-tips-for-training-lightning-fast-neural-networks-in-pytorch-8e63a502f565">this guide</a>. </p>
|
||||
<hr />
|
||||
<h4 id="cuda-flags">CUDA flags</h4>
|
||||
<p>CUDA flags make certain GPUs visible to your script.
|
||||
Lightning sets these for you automatically, there's NO NEED to do this yourself.</p>
|
||||
<pre><code class="python"># lightning will set according to what you give the trainer
|
||||
# os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
|
||||
# os.environ["CUDA_VISIBLE_DEVICES"] = "0"
|
||||
</code></pre>
|
||||
|
||||
<hr />
|
||||
<h4 id="16-bit-mixed-precision">16-bit mixed precision</h4>
|
||||
<p>16 bit precision can cut your memory footprint by half. If using volta architecture GPUs it can give a dramatic training speed-up as well. <br />
|
||||
@@ -638,11 +661,7 @@ trainer = Trainer(amp_level='O2', use_amp=False)
|
||||
<hr />
|
||||
<h4 id="single-gpu">Single-gpu</h4>
|
||||
<p>Make sure you're on a GPU machine. </p>
|
||||
<pre><code class="python"># set these flags
|
||||
os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
|
||||
os.environ["CUDA_VISIBLE_DEVICES"] = "0"
|
||||
|
||||
# DEFAULT
|
||||
<pre><code class="python"># DEFAULT
|
||||
trainer = Trainer(gpus=[0])
|
||||
</code></pre>
|
||||
|
||||
@@ -650,14 +669,7 @@ trainer = Trainer(gpus=[0])
|
||||
<h4 id="multi-gpu">multi-gpu</h4>
|
||||
<p>Make sure you're on a GPU machine. You can set as many GPUs as you want.
|
||||
In this setting, the model will run on all 8 GPUs at once using DataParallel under the hood.</p>
|
||||
<pre><code class="python"># set these flags
|
||||
# lightning sets these flags for you automatically
|
||||
# no need to set yourself
|
||||
# os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
|
||||
# os.environ["CUDA_VISIBLE_DEVICES"] = "0,1,2,3,4,5,6,7"
|
||||
|
||||
|
||||
# to use DataParallel (default)
|
||||
<pre><code class="python"># to use DataParallel (default)
|
||||
trainer = Trainer(gpus=[0,1,2,3,4,5,6,7], distributed_backend='dp')
|
||||
|
||||
# RECOMMENDED use DistributedDataParallel
|
||||
|
||||
File diff suppressed because one or more lines are too long
+14
-14
@@ -2,72 +2,72 @@
|
||||
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-07-26</lastmod>
|
||||
<lastmod>2019-07-27</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-07-26</lastmod>
|
||||
<lastmod>2019-07-27</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-07-26</lastmod>
|
||||
<lastmod>2019-07-27</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-07-26</lastmod>
|
||||
<lastmod>2019-07-27</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-07-26</lastmod>
|
||||
<lastmod>2019-07-27</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-07-26</lastmod>
|
||||
<lastmod>2019-07-27</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-07-26</lastmod>
|
||||
<lastmod>2019-07-27</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-07-26</lastmod>
|
||||
<lastmod>2019-07-27</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-07-26</lastmod>
|
||||
<lastmod>2019-07-27</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-07-26</lastmod>
|
||||
<lastmod>2019-07-27</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-07-26</lastmod>
|
||||
<lastmod>2019-07-27</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-07-26</lastmod>
|
||||
<lastmod>2019-07-27</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-07-26</lastmod>
|
||||
<lastmod>2019-07-27</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
<url>
|
||||
<loc>None</loc>
|
||||
<lastmod>2019-07-26</lastmod>
|
||||
<lastmod>2019-07-27</lastmod>
|
||||
<changefreq>daily</changefreq>
|
||||
</url>
|
||||
</urlset>
|
||||
Binary file not shown.
Reference in New Issue
Block a user