Deployed babaa08 with MkDocs version: 1.0.4

Trainer/Distributed training/index.html

@@ -624,13 +624,30 @@ In this setting, the model will run on all 8 GPUs at once using DataParallel und
 os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
 os.environ["CUDA_VISIBLE_DEVICES"] = "0,1,2,3,4,5,6,7"
 
-# DEFAULT
+
 trainer = Trainer(gpus=[0,1,2,3,4,5,6,7])
 </code></pre>
 
 <hr />
 <h4 id="multi-node">Multi-node</h4>
-<p>COMING SOON.</p>
+<p>Multi-node training is easily done by specifying these flags.</p>
+<pre><code class="python"># train on 12*8 GPUs
+trainer = Trainer(gpus=[0,1,2,3,4,5,6,7], nb_gpu_nodes=12)
+</code></pre>
+
+<p>In addition, make sure to set up your SLURM job correctly via the <a href="https://williamfalcon.github.io/test-tube/hpc/SlurmCluster/">SlurmClusterObject</a>. In particular, specify the number of tasks per node correctly.</p>
+<pre><code class="python">cluster = SlurmCluster(
+    hyperparam_optimizer=test_tube.HyperOptArgumentParser(),
+    log_path='/some/path/to/save',
+)
+
+# configure cluster
+cluster.per_experiment_nb_nodes = 12 
+cluster.per_experiment_nb_gpus = 8
+
+cluster.add_slurm_cmd(cmd='ntasks-per-node', value=8, comment='1 task per gpu')
+</code></pre>
+
 <hr />
 <h4 id="self-balancing-architecture">Self-balancing architecture</h4>
 <p>Here lightning distributes parts of your module across available GPUs to optimize for speed and memory.   </p>

sitemap.xml

@@ -2,72 +2,72 @@
 <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
     <url>
      <loc>None</loc>
-     <lastmod>2019-07-17</lastmod>
+     <lastmod>2019-07-21</lastmod>
      <changefreq>daily</changefreq>
     </url>
     <url>
      <loc>None</loc>
-     <lastmod>2019-07-17</lastmod>
+     <lastmod>2019-07-21</lastmod>
      <changefreq>daily</changefreq>
     </url>
     <url>
      <loc>None</loc>
-     <lastmod>2019-07-17</lastmod>
+     <lastmod>2019-07-21</lastmod>
      <changefreq>daily</changefreq>
     </url>
     <url>
      <loc>None</loc>
-     <lastmod>2019-07-17</lastmod>
+     <lastmod>2019-07-21</lastmod>
      <changefreq>daily</changefreq>
     </url>
     <url>
      <loc>None</loc>
-     <lastmod>2019-07-17</lastmod>
+     <lastmod>2019-07-21</lastmod>
      <changefreq>daily</changefreq>
     </url>
     <url>
      <loc>None</loc>
-     <lastmod>2019-07-17</lastmod>
+     <lastmod>2019-07-21</lastmod>
      <changefreq>daily</changefreq>
     </url>
     <url>
      <loc>None</loc>
-     <lastmod>2019-07-17</lastmod>
+     <lastmod>2019-07-21</lastmod>
      <changefreq>daily</changefreq>
     </url>
     <url>
      <loc>None</loc>
-     <lastmod>2019-07-17</lastmod>
+     <lastmod>2019-07-21</lastmod>
      <changefreq>daily</changefreq>
     </url>
     <url>
      <loc>None</loc>
-     <lastmod>2019-07-17</lastmod>
+     <lastmod>2019-07-21</lastmod>
      <changefreq>daily</changefreq>
     </url>
     <url>
      <loc>None</loc>
-     <lastmod>2019-07-17</lastmod>
+     <lastmod>2019-07-21</lastmod>
      <changefreq>daily</changefreq>
     </url>
     <url>
      <loc>None</loc>
-     <lastmod>2019-07-17</lastmod>
+     <lastmod>2019-07-21</lastmod>
      <changefreq>daily</changefreq>
     </url>
     <url>
      <loc>None</loc>
-     <lastmod>2019-07-17</lastmod>
+     <lastmod>2019-07-21</lastmod>
      <changefreq>daily</changefreq>
     </url>
     <url>
      <loc>None</loc>
-     <lastmod>2019-07-17</lastmod>
+     <lastmod>2019-07-21</lastmod>
      <changefreq>daily</changefreq>
     </url>
     <url>
      <loc>None</loc>
-     <lastmod>2019-07-17</lastmod>
+     <lastmod>2019-07-21</lastmod>
      <changefreq>daily</changefreq>
     </url>
 </urlset>