wassname/pytorch-lightning
1
0
Fork 0
mirror of https://github.com/wassname/pytorch-lightning.git synced 2026-09-12 12:40:20 +08:00
Code Issues Packages Projects Releases Wiki Activity

Deployed 5a9320d with MkDocs version: 1.0.4

Browse Source
This commit is contained in:
William Falcon
2019-09-26 10:28:41 -05:00
parent 8f53e506cf
commit a5cc9ec2dc
23 changed files with 417 additions and 349 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Trainer/Distributed training/index.html
+47 -23
View File
@@ -34,7 +34,7 @@
<meta name="lang:search.tokenizer" content="[\s\-]+">
<link rel="shortcut icon" href="../../assets/images/favicon.png">
<meta name="generator" content="mkdocs-1.0.4, mkdocs-material-4.4.0">
<meta name="generator" content="mkdocs-1.0.4, mkdocs-material-4.4.2">
@@ -42,7 +42,7 @@
<link rel="stylesheet" href="../../assets/stylesheets/application.0284f74d.css">
<link rel="stylesheet" href="../../assets/stylesheets/application.30686662.css">
@@ -171,7 +171,7 @@
<main class="md-main">
<main class="md-main" role="main">
<div class="md-main__inner md-grid" data-md-component="container">
@@ -356,56 +356,56 @@
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="#choosing-a-backend" title="Choosing a backend" class="md-nav__link">
<a href="#choosing-a-backend" class="md-nav__link">
Choosing a backend
</a>
</li>
<li class="md-nav__item">
<a href="#distributed-and-16-bit-precision" title="Distributed and 16-bit precision." class="md-nav__link">
<a href="#distributed-and-16-bit-precision" class="md-nav__link">
Distributed and 16-bit precision.
</a>
</li>
<li class="md-nav__item">
<a href="#cuda-flags" title="CUDA flags" class="md-nav__link">
<a href="#cuda-flags" class="md-nav__link">
CUDA flags
</a>
</li>
<li class="md-nav__item">
<a href="#16-bit-mixed-precision" title="16-bit mixed precision" class="md-nav__link">
<a href="#16-bit-mixed-precision" class="md-nav__link">
16-bit mixed precision
</a>
</li>
<li class="md-nav__item">
<a href="#single-gpu" title="Single-gpu" class="md-nav__link">
<a href="#single-gpu" class="md-nav__link">
Single-gpu
</a>
</li>
<li class="md-nav__item">
<a href="#multi-gpu" title="multi-gpu" class="md-nav__link">
<a href="#multi-gpu" class="md-nav__link">
multi-gpu
</a>
</li>
<li class="md-nav__item">
<a href="#multi-node" title="Multi-node" class="md-nav__link">
<a href="#multi-node" class="md-nav__link">
Multi-node
</a>
</li>
<li class="md-nav__item">
<a href="#self-balancing-architecture" title="Self-balancing architecture" class="md-nav__link">
<a href="#self-balancing-architecture" class="md-nav__link">
Self-balancing architecture
</a>
@@ -566,56 +566,56 @@
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="#choosing-a-backend" title="Choosing a backend" class="md-nav__link">
<a href="#choosing-a-backend" class="md-nav__link">
Choosing a backend
</a>
</li>
<li class="md-nav__item">
<a href="#distributed-and-16-bit-precision" title="Distributed and 16-bit precision." class="md-nav__link">
<a href="#distributed-and-16-bit-precision" class="md-nav__link">
Distributed and 16-bit precision.
</a>
</li>
<li class="md-nav__item">
<a href="#cuda-flags" title="CUDA flags" class="md-nav__link">
<a href="#cuda-flags" class="md-nav__link">
CUDA flags
</a>
</li>
<li class="md-nav__item">
<a href="#16-bit-mixed-precision" title="16-bit mixed precision" class="md-nav__link">
<a href="#16-bit-mixed-precision" class="md-nav__link">
16-bit mixed precision
</a>
</li>
<li class="md-nav__item">
<a href="#single-gpu" title="Single-gpu" class="md-nav__link">
<a href="#single-gpu" class="md-nav__link">
Single-gpu
</a>
</li>
<li class="md-nav__item">
<a href="#multi-gpu" title="multi-gpu" class="md-nav__link">
<a href="#multi-gpu" class="md-nav__link">
multi-gpu
</a>
</li>
<li class="md-nav__item">
<a href="#multi-node" title="Multi-node" class="md-nav__link">
<a href="#multi-node" class="md-nav__link">
Multi-node
</a>
</li>
<li class="md-nav__item">
<a href="#self-balancing-architecture" title="Self-balancing architecture" class="md-nav__link">
<a href="#self-balancing-architecture" class="md-nav__link">
Self-balancing architecture
</a>
@@ -768,10 +768,34 @@ SLURM will set these for you. </p>
<h4 id="16-bit-mixed-precision">16-bit mixed precision</h4>
<p>16 bit precision can cut your memory footprint by half. If using volta architecture GPUs it can give a dramatic training speed-up as well. <br />
First, install apex (if install fails, look <a href="https://github.com/NVIDIA/apex">here</a>):</p>
<table class="codehilitetable"><tr><td class="linenos"><div class="linenodiv"><pre>1
2
3</pre></div></td><td class="code"><div class="codehilite"><pre><span></span>$ git clone https://github.com/NVIDIA/apex
<table class="codehilitetable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15</pre></div></td><td class="code"><div class="codehilite"><pre><span></span>$ git clone https://github.com/NVIDIA/apex
$ <span class="nb">cd</span> apex
<span class="c1"># ------------------------</span>
<span class="c1"># OPTIONAL: on your cluster you might need to load cuda 10 or 9</span>
<span class="c1"># depending on how you installed PyTorch</span>
<span class="c1"># see available modules</span>
module avail
<span class="c1"># load correct cuda before install</span>
module load cuda-10.0
<span class="c1"># ------------------------</span>
$ pip install -v --no-cache-dir --global-option<span class="o">=</span><span class="s2">&quot;--cpp_ext&quot;</span> --global-option<span class="o">=</span><span class="s2">&quot;--cuda_ext&quot;</span> ./
</pre></div>
</td></tr></table>
@@ -963,7 +987,7 @@ portion of your dataset onto each GPU. (World_size = gpus_per_node * nb_nodes).
</div>
<script src="../../assets/javascripts/application.245445c6.js"></script>
<script src="../../assets/javascripts/application.c648116f.js"></script>
<script>app.initialize({version:"1.0.4",url:{base:"../.."}})</script>