wassname/pytorch-lightning
1
0
Fork 0
mirror of https://github.com/wassname/pytorch-lightning.git synced 2026-09-09 11:32:07 +08:00
Code Issues Packages Projects Releases Wiki Activity

Deployed 28618a3 with MkDocs version: 1.0.4

Browse Source
This commit is contained in:
William Falcon
2019-06-28 16:46:09 -05:00
parent f1d00ca90d
commit ee71cdbcfa
17 changed files with 560 additions and 101 deletions
Download Patch File Download Diff File Expand all files Collapse all files
404.html
+6 -2
View File
@@ -48,11 +48,15 @@
<li class="toctree-l1">
<span class="caption-text">Pytorch Lightning</span>
<span class="caption-text">LightningModule</span>
<ul class="subnav">
<li class="">
<a class="" href="/Pytorch-Lightning/LightningModule/">Lightning module</a>
<a class="" href="/LightningModule/RequiredTrainerInterface/">Lightning Module interface</a>
</li>
<li class="">
<a class="" href="/LightningModule/methods/">Methods</a>
</li>
</ul>
</li>
Pytorch-Lightning/LightningModule/index.html → LightningModule/RequiredTrainerInterface/index.html
+16 -12
View File
@@ -8,7 +8,7 @@
<link rel="shortcut icon" href="../../img/favicon.ico">
<title>Lightning module - Pytorch lightning Documentation</title>
<title>Lightning Module interface - Pytorch lightning Documentation</title>
<link href='https://fonts.googleapis.com/css?family=Lato:400,700|Roboto+Slab:400,700|Inconsolata:400,700' rel='stylesheet' type='text/css'>
<link rel="stylesheet" href="../../css/theme.css" type="text/css" />
@@ -17,8 +17,8 @@
<script>
// Current page data
var mkdocs_page_name = "Lightning module";
var mkdocs_page_input_path = "Pytorch-Lightning/LightningModule.md";
var mkdocs_page_name = "Lightning Module interface";
var mkdocs_page_input_path = "LightningModule/RequiredTrainerInterface.md";
var mkdocs_page_url = null;
</script>
@@ -55,14 +55,14 @@
<li class="toctree-l1">
<span class="caption-text">Pytorch Lightning</span>
<span class="caption-text">LightningModule</span>
<ul class="subnav">
<li class=" current">
<a class="current" href="./">Lightning module</a>
<a class="current" href="./">Lightning Module interface</a>
<ul class="subnav">
<li class="toctree-l3"><a href="#lightning-module">Lightning module</a></li>
<li class="toctree-l3"><a href="#lightning-module-interface">Lightning Module interface</a></li>
<ul>
@@ -93,6 +93,10 @@
</ul>
</li>
<li class="">
<a class="" href="../methods/">Methods</a>
</li>
</ul>
</li>
@@ -161,14 +165,14 @@
<li>Pytorch Lightning &raquo;</li>
<li>LightningModule &raquo;</li>
<li>Lightning module</li>
<li>Lightning Module interface</li>
<li class="wy-breadcrumbs-aside">
<a href="https://github.com/williamFalcon/pytorch-lightning/edit/master/docs/Pytorch-Lightning/LightningModule.md"
<a href="https://github.com/williamFalcon/pytorch-lightning/edit/master/docs/LightningModule/RequiredTrainerInterface.md"
class="icon icon-github"> Edit on GitHub</a>
</li>
@@ -178,7 +182,7 @@
<div role="main">
<div class="section">
<h1 id="lightning-module">Lightning module</h1>
<h1 id="lightning-module-interface">Lightning Module interface</h1>
<p>[<a href="https://github.com/williamFalcon/pytorch-lightning/blob/master/pytorch_lightning/root_module/root_module.py">Github Code</a>]</p>
<p>A lightning module is a strict superclass of nn.Module, it provides a standard interface for the trainer to interact with the model.</p>
<p>The easiest thing to do is copy <a href="../../examples/new_project_templates/lightning_module_template.py">this template</a> and modify accordingly. </p>
@@ -590,7 +594,7 @@ def add_model_specific_args(parent_parser, root_dir):
<div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
<a href="../../Trainer/" class="btn btn-neutral float-right" title="Trainer">Next <span class="icon icon-circle-arrow-right"></span></a>
<a href="../methods/" class="btn btn-neutral float-right" title="Methods">Next <span class="icon icon-circle-arrow-right"></span></a>
<a href="../.." class="btn btn-neutral" title="PYTORCH-LIGHTNING DOCUMENTATION"><span class="icon icon-circle-arrow-left"></span> Previous</a>
@@ -624,7 +628,7 @@ def add_model_specific_args(parent_parser, root_dir):
<span><a href="../.." style="color: #fcfcfc;">&laquo; Previous</a></span>
<span style="margin-left: 15px"><a href="../../Trainer/" style="color: #fcfcfc">Next &raquo;</a></span>
<span style="margin-left: 15px"><a href="../methods/" style="color: #fcfcfc">Next &raquo;</a></span>
</span>
</div>
LightningModule/methods/index.html
+274
View File
@@ -0,0 +1,274 @@
<!DOCTYPE html>
<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="shortcut icon" href="../../img/favicon.ico">
<title>Methods - Pytorch lightning Documentation</title>
<link href='https://fonts.googleapis.com/css?family=Lato:400,700|Roboto+Slab:400,700|Inconsolata:400,700' rel='stylesheet' type='text/css'>
<link rel="stylesheet" href="../../css/theme.css" type="text/css" />
<link rel="stylesheet" href="../../css/theme_extra.css" type="text/css" />
<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/styles/github.min.css">
<script>
// Current page data
var mkdocs_page_name = "Methods";
var mkdocs_page_input_path = "LightningModule/methods.md";
var mkdocs_page_url = null;
</script>
<script src="../../js/jquery-2.1.1.min.js" defer></script>
<script src="../../js/modernizr-2.8.3.min.js" defer></script>
<script src="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/highlight.min.js"></script>
<script>hljs.initHighlightingOnLoad();</script>
</head>
<body class="wy-body-for-nav" role="document">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side stickynav">
<div class="wy-side-nav-search">
<a href="../.." class="icon icon-home"> Pytorch lightning Documentation</a>
<div role="search">
<form id ="rtd-search-form" class="wy-form" action="../../search.html" method="get">
<input type="text" name="q" placeholder="Search docs" title="Type search term here" />
</form>
</div>
</div>
<div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
<ul class="current">
<li class="toctree-l1">
<a class="" href="../..">PYTORCH-LIGHTNING DOCUMENTATION</a>
</li>
<li class="toctree-l1">
<span class="caption-text">LightningModule</span>
<ul class="subnav">
<li class="">
<a class="" href="../RequiredTrainerInterface/">Lightning Module interface</a>
</li>
<li class=" current">
<a class="current" href="./">Methods</a>
<ul class="subnav">
<li class="toctree-l3"><a href="#freeze">freeze</a></li>
<li class="toctree-l3"><a href="#load_from_metrics">load_from_metrics</a></li>
<li class="toctree-l3"><a href="#unfreeze">unfreeze</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1">
<span class="caption-text">Trainer</span>
<ul class="subnav">
<li class="">
<a class="" href="../../Trainer/">Trainer</a>
</li>
<li class="">
<a class="" href="../../Trainer/Checkpointing/">Checkpointing</a>
</li>
<li class="">
<a class="" href="../../Trainer/Distributed training/">Distributed training</a>
</li>
<li class="">
<a class="" href="../../Trainer/Logging/">Logging</a>
</li>
<li class="">
<a class="" href="../../Trainer/SLURM Managed Cluster/">SLURM Managed Cluster</a>
</li>
<li class="">
<a class="" href="../../Trainer/Training Loop/">Training Loop</a>
</li>
<li class="">
<a class="" href="../../Trainer/Validation loop/">Validation loop</a>
</li>
<li class="">
<a class="" href="../../Trainer/debugging/">Debugging</a>
</li>
<li class="">
<a class="" href="../../Trainer/hooks/">Hooks</a>
</li>
</ul>
</li>
</ul>
</div>
&nbsp;
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
<nav class="wy-nav-top" role="navigation" aria-label="top navigation">
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="../..">Pytorch lightning Documentation</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="breadcrumbs navigation">
<ul class="wy-breadcrumbs">
<li><a href="../..">Docs</a> &raquo;</li>
<li>LightningModule &raquo;</li>
<li>Methods</li>
<li class="wy-breadcrumbs-aside">
<a href="https://github.com/williamFalcon/pytorch-lightning/edit/master/docs/LightningModule/methods.md"
class="icon icon-github"> Edit on GitHub</a>
</li>
</ul>
<hr/>
</div>
<div role="main">
<div class="section">
<p>Lightning modules are strict superclasses of torch.nn.Module. A LightningModule offers the following in addition to that API.</p>
<hr />
<h3 id="freeze">freeze</h3>
<p>Freeze all params for inference</p>
<pre><code class="python">model = MyLightningModule(...)
model.freeze()
</code></pre>
<hr />
<h3 id="load_from_metrics">load_from_metrics</h3>
<p>This is the easiest/fastest way which uses the meta_tags.csv file from test-tube to rebuild the model.
The meta_tags.csv file can be found in the test-tube experiment save_dir. </p>
<pre><code class="python">pretrained_model = MyLightningModule.load_from_metrics(
weights_path='/path/to/pytorch_checkpoint.ckpt',
tags_csv='/path/to/test_tube/experiment/version/meta_tags.csv',
on_gpu=True,
map_location=None
)
# predict
pretrained_model.freeze()
y_hat = pretrained_model(x)
</code></pre>
<p><strong>Params</strong> </p>
<table>
<thead>
<tr>
<th>Param</th>
<th>description</th>
</tr>
</thead>
<tbody>
<tr>
<td>weights_path</td>
<td>Path to a pytorch checkpoint</td>
</tr>
<tr>
<td>tags_csv</td>
<td>Path to meta_tags.csv file generated by the test-tube Experiment</td>
</tr>
<tr>
<td>on_gpu</td>
<td>if True, puts model on GPU. Make sure to use transforms option if model devices have changed</td>
</tr>
<tr>
<td>map_location</td>
<td>A dictionary mapping saved weight GPU devices to new GPU devices</td>
</tr>
</tbody>
</table>
<p><strong>Returns</strong> </p>
<p>LightningModule - The pretrained LightningModule</p>
<hr />
<h3 id="unfreeze">unfreeze</h3>
<p>Unfreeze all params for inference</p>
<pre><code class="python">model = MyLightningModule(...)
model.unfreeze()
</code></pre>
</div>
</div>
<footer>
<div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
<a href="../../Trainer/" class="btn btn-neutral float-right" title="Trainer">Next <span class="icon icon-circle-arrow-right"></span></a>
<a href="../RequiredTrainerInterface/" class="btn btn-neutral" title="Lightning Module interface"><span class="icon icon-circle-arrow-left"></span> Previous</a>
</div>
<hr/>
<div role="contentinfo">
<!-- Copyright etc -->
</div>
Built with <a href="http://www.mkdocs.org">MkDocs</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<div class="rst-versions" role="note" style="cursor: pointer">
<span class="rst-current-version" data-toggle="rst-current-version">
<a href="https://github.com/williamFalcon/pytorch-lightning/" class="fa fa-github" style="float: left; color: #fcfcfc"> GitHub</a>
<span><a href="../RequiredTrainerInterface/" style="color: #fcfcfc;">&laquo; Previous</a></span>
<span style="margin-left: 15px"><a href="../../Trainer/" style="color: #fcfcfc">Next &raquo;</a></span>
</span>
</div>
<script>var base_url = '../..';</script>
<script src="../../js/theme.js" defer></script>
<script src="../../search/main.js" defer></script>
</body>
</html>
Trainer/Checkpointing/index.html
+25 -3
View File
@@ -55,11 +55,15 @@
<li class="toctree-l1">
<span class="caption-text">Pytorch Lightning</span>
<span class="caption-text">LightningModule</span>
<ul class="subnav">
<li class="">
<a class="" href="../../Pytorch-Lightning/LightningModule/">Lightning module</a>
<a class="" href="../../LightningModule/RequiredTrainerInterface/">Lightning Module interface</a>
</li>
<li class="">
<a class="" href="../../LightningModule/methods/">Methods</a>
</li>
</ul>
</li>
@@ -77,6 +81,9 @@
<a class="current" href="./">Checkpointing</a>
<ul class="subnav">
<li class="toctree-l3"><a href="#model-saving">Model saving</a></li>
</ul>
</li>
<li class="">
@@ -149,7 +156,22 @@
<div role="main">
<div class="section">
<p>Lightning can automate saving and loading checkpoints.</p>
<hr />
<h3 id="model-saving">Model saving</h3>
<p>To enable checkpointing, define the checkpoint callback and give it to the trainer.</p>
<pre><code class="python">from pytorch_lightning.utils.pt_callbacks import ModelCheckpoint
checkpoint_callback = ModelCheckpoint(
filepath='/path/to/store/weights.ckpt',
save_best_only=True,
verbose=True,
monitor='val_loss',
mode='min'
)
trainer = Trainer(checkpoint_callback=checkpoint_callback)
</code></pre>
</div>
</div>
Trainer/Distributed training/index.html
+6 -2
View File
@@ -55,11 +55,15 @@
<li class="toctree-l1">
<span class="caption-text">Pytorch Lightning</span>
<span class="caption-text">LightningModule</span>
<ul class="subnav">
<li class="">
<a class="" href="../../Pytorch-Lightning/LightningModule/">Lightning module</a>
<a class="" href="../../LightningModule/RequiredTrainerInterface/">Lightning Module interface</a>
</li>
<li class="">
<a class="" href="../../LightningModule/methods/">Methods</a>
</li>
</ul>
</li>
Trainer/Logging/index.html
+6 -2
View File
@@ -55,11 +55,15 @@
<li class="toctree-l1">
<span class="caption-text">Pytorch Lightning</span>
<span class="caption-text">LightningModule</span>
<ul class="subnav">
<li class="">
<a class="" href="../../Pytorch-Lightning/LightningModule/">Lightning module</a>
<a class="" href="../../LightningModule/RequiredTrainerInterface/">Lightning Module interface</a>
</li>
<li class="">
<a class="" href="../../LightningModule/methods/">Methods</a>
</li>
</ul>
</li>
Trainer/SLURM Managed Cluster/index.html
+103 -3
View File
@@ -55,11 +55,15 @@
<li class="toctree-l1">
<span class="caption-text">Pytorch Lightning</span>
<span class="caption-text">LightningModule</span>
<ul class="subnav">
<li class="">
<a class="" href="../../Pytorch-Lightning/LightningModule/">Lightning module</a>
<a class="" href="../../LightningModule/RequiredTrainerInterface/">Lightning Module interface</a>
</li>
<li class="">
<a class="" href="../../LightningModule/methods/">Methods</a>
</li>
</ul>
</li>
@@ -89,6 +93,12 @@
<a class="current" href="./">SLURM Managed Cluster</a>
<ul class="subnav">
<li class="toctree-l3"><a href="#running-grid-search-on-a-cluster">Running grid search on a cluster</a></li>
<li class="toctree-l3"><a href="#walltime-auto-resubmit">Walltime auto-resubmit</a></li>
</ul>
</li>
<li class="">
@@ -149,7 +159,97 @@
<div role="main">
<div class="section">
<p>Lightning supports model training on a cluster managed by SLURM in the following cases: </p>
<ol>
<li>Training on single or multi-cpus only.</li>
<li>Training on single or multi-gpus on the same node.</li>
<li>Coming SOON: Training across multiple nodes.</li>
</ol>
<hr />
<h4 id="running-grid-search-on-a-cluster">Running grid search on a cluster</h4>
<p>To use lightning to run a hyperparameter search (grid-search or random-search) on a cluster do 4 things: </p>
<p>(1). Define the parameters for the grid search </p>
<pre><code class="python">from test_tube import HyperOptArgumentParser
# subclass of argparse
parser = HyperOptArgumentParser(strategy='random_search')
parser.add_argument('--learning_rate', default=0.002, type=float, help='the learning rate')
# let's enable optimizing over the number of layers in the network
parser.opt_list('--nb_layers', default=2, type=int, tunable=True, options=[2, 4, 8])
hparams = parser.parse_args()
</code></pre>
<p>(2). Define the cluster options in the <a href="https://williamfalcon.github.io/test-tube/hpc/SlurmCluster/">SlurmCluster object</a> (over 5 nodes and 8 gpus) </p>
<pre><code class="python">from test_tube.hpc import SlurmCluster
# hyperparameters is a test-tube hyper params object
# see https://williamfalcon.github.io/test-tube/hyperparameter_optimization/HyperOptArgumentParser/
hyperparams = args.parse()
# init cluster
cluster = SlurmCluster(
hyperparam_optimizer=hyperparams,
log_path='/path/to/log/results/to',
python_cmd='python3'
)
# let the cluster know where to email for a change in job status (ie: complete, fail, etc...)
cluster.notify_job_status(email='some@email.com', on_done=True, on_fail=True)
# set the job options. In this instance, we'll run 20 different models
# each with its own set of hyperparameters giving each one 1 GPU (ie: taking up 20 GPUs)
cluster.per_experiment_nb_gpus = 8
cluster.per_experiment_nb_nodes = 5
# we'll request 10GB of memory per node
cluster.memory_mb_per_node = 10000
# set a walltime of 10 minues
cluster.job_time = '10:00'
</code></pre>
<p>(3). Give trainer the cluster_manager in your main function: </p>
<pre><code class="python">from pytorch_lightning import Trainer
def train_fx(trial_hparams, cluster_manager, _):
# hparams has a specific set of hyperparams
my_model = MyLightningModel()
# give the trainer the cluster object
trainer = Trainer(cluster=cluster_manager)
trainer.fit(my_model)
</code></pre>
<p>(4). Start the grid search </p>
<pre><code class="python"># run the models on the cluster
cluster.optimize_parallel_cluster_gpu(
train_fx,
nb_trials=20,
job_name='my_grid_search_exp_name',
job_display_name='my_exp')
</code></pre>
<p>That's it! The SlurmCluster object will automatically checkpoint the lightning model and resubmit if it runs into the walltime!</p>
<hr />
<h4 id="walltime-auto-resubmit">Walltime auto-resubmit</h4>
<p>Lightning automatically resubmits jobs when they reach the walltime. You get this behavior for free if you give lightning
a slurm cluster object.</p>
<pre><code class="python">def my_main_fx(hparams, slurm_manager, _):
trainer = Trainer(cluster=slurm_manager)
</code></pre>
<p>(See the grid search example above for cluster configuration).
With this feature lightning will: </p>
<ol>
<li>automatically checkpoint the model</li>
<li>checkpoint the trainer session</li>
<li>resubmit a continuation job.</li>
<li>load the checkpoint and trainer session in the new model</li>
</ol>
</div>
</div>
Trainer/Training Loop/index.html
+6 -2
View File
@@ -55,11 +55,15 @@
<li class="toctree-l1">
<span class="caption-text">Pytorch Lightning</span>
<span class="caption-text">LightningModule</span>
<ul class="subnav">
<li class="">
<a class="" href="../../Pytorch-Lightning/LightningModule/">Lightning module</a>
<a class="" href="../../LightningModule/RequiredTrainerInterface/">Lightning Module interface</a>
</li>
<li class="">
<a class="" href="../../LightningModule/methods/">Methods</a>
</li>
</ul>
</li>
Trainer/Validation loop/index.html
+6 -2
View File
@@ -55,11 +55,15 @@
<li class="toctree-l1">
<span class="caption-text">Pytorch Lightning</span>
<span class="caption-text">LightningModule</span>
<ul class="subnav">
<li class="">
<a class="" href="../../Pytorch-Lightning/LightningModule/">Lightning module</a>
<a class="" href="../../LightningModule/RequiredTrainerInterface/">Lightning Module interface</a>
</li>
<li class="">
<a class="" href="../../LightningModule/methods/">Methods</a>
</li>
</ul>
</li>
Trainer/debugging/index.html
+6 -2
View File
@@ -55,11 +55,15 @@
<li class="toctree-l1">
<span class="caption-text">Pytorch Lightning</span>
<span class="caption-text">LightningModule</span>
<ul class="subnav">
<li class="">
<a class="" href="../../Pytorch-Lightning/LightningModule/">Lightning module</a>
<a class="" href="../../LightningModule/RequiredTrainerInterface/">Lightning Module interface</a>
</li>
<li class="">
<a class="" href="../../LightningModule/methods/">Methods</a>
</li>
</ul>
</li>
Trainer/hooks/index.html
+6 -2
View File
@@ -55,11 +55,15 @@
<li class="toctree-l1">
<span class="caption-text">Pytorch Lightning</span>
<span class="caption-text">LightningModule</span>
<ul class="subnav">
<li class="">
<a class="" href="../../Pytorch-Lightning/LightningModule/">Lightning module</a>
<a class="" href="../../LightningModule/RequiredTrainerInterface/">Lightning Module interface</a>
</li>
<li class="">
<a class="" href="../../LightningModule/methods/">Methods</a>
</li>
</ul>
</li>
Trainer/index.html
+10 -8
View File
@@ -55,11 +55,15 @@
<li class="toctree-l1">
<span class="caption-text">Pytorch Lightning</span>
<span class="caption-text">LightningModule</span>
<ul class="subnav">
<li class="">
<a class="" href="../Pytorch-Lightning/LightningModule/">Lightning module</a>
<a class="" href="../LightningModule/RequiredTrainerInterface/">Lightning Module interface</a>
</li>
<li class="">
<a class="" href="../LightningModule/methods/">Methods</a>
</li>
</ul>
</li>
@@ -172,10 +176,8 @@ trainer.fit(model)
</ul>
<p><strong>Computing cluster (SLURM)</strong> </p>
<ul>
<li>Automatic checkpointing </li>
<li>Automatic saving, loading </li>
<li>Running grid search on a cluster </li>
<li>Walltime auto-resubmit </li>
<li><a href="SLURM%20Managed%20Cluster/#running-grid-search-on-a-cluster">Running grid search on a cluster</a></li>
<li><a href="SLURM%20Managed%20Cluster/#walltime-auto-resubmit">Walltime auto-resubmit</a> </li>
</ul>
<p><strong>Debugging</strong> </p>
<ul>
@@ -231,7 +233,7 @@ trainer.fit(model)
<a href="Checkpointing/" class="btn btn-neutral float-right" title="Checkpointing">Next <span class="icon icon-circle-arrow-right"></span></a>
<a href="../Pytorch-Lightning/LightningModule/" class="btn btn-neutral" title="Lightning module"><span class="icon icon-circle-arrow-left"></span> Previous</a>
<a href="../LightningModule/methods/" class="btn btn-neutral" title="Methods"><span class="icon icon-circle-arrow-left"></span> Previous</a>
</div>
@@ -259,7 +261,7 @@ trainer.fit(model)
<a href="https://github.com/williamFalcon/pytorch-lightning/" class="fa fa-github" style="float: left; color: #fcfcfc"> GitHub</a>
<span><a href="../Pytorch-Lightning/LightningModule/" style="color: #fcfcfc;">&laquo; Previous</a></span>
<span><a href="../LightningModule/methods/" style="color: #fcfcfc;">&laquo; Previous</a></span>
<span style="margin-left: 15px"><a href="Checkpointing/" style="color: #fcfcfc">Next &raquo;</a></span>
index.html
+67 -47
View File
@@ -5,7 +5,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="Documentation for Pytorch Pytorch-Lightning, the researcher version of keras.">
<meta name="description" content="Documentation for Pytorch LightningModule, the researcher version of keras.">
<link rel="shortcut icon" href="img/favicon.ico">
<title>PYTORCH-LIGHTNING DOCUMENTATION - Pytorch lightning Documentation</title>
@@ -63,16 +63,20 @@
<li><a class="toctree-l3" href="#quick-start-examples">Quick start examples</a></li>
<li><a class="toctree-l3" href="#training-loop">Training loop</a></li>
<li><a class="toctree-l3" href="#validation-loop">Validation loop</a></li>
<li><a class="toctree-l3" href="#distributed-training">Distributed training</a></li>
<li><a class="toctree-l3" href="#checkpointing">Checkpointing</a></li>
<li><a class="toctree-l3" href="#computing-cluster-slurm">Computing cluster (SLURM)</a></li>
<li><a class="toctree-l3" href="#debugging">Debugging</a></li>
<li><a class="toctree-l3" href="#distributed-training">Distributed training</a></li>
<li><a class="toctree-l3" href="#experiment-logging">Experiment Logging</a></li>
<li><a class="toctree-l3" href="#training-loop">Training loop</a></li>
<li><a class="toctree-l3" href="#validation-loop">Validation loop</a></li>
</ul>
@@ -81,11 +85,15 @@
<li class="toctree-l1">
<span class="caption-text">Pytorch Lightning</span>
<span class="caption-text">LightningModule</span>
<ul class="subnav">
<li class="">
<a class="" href="Pytorch-Lightning/LightningModule/">Lightning module</a>
<a class="" href="LightningModule/RequiredTrainerInterface/">Lightning Module interface</a>
</li>
<li class="">
<a class="" href="LightningModule/methods/">Methods</a>
</li>
</ul>
</li>
@@ -171,7 +179,7 @@
<h1 id="pytorch-lightning-documentation">PYTORCH-LIGHTNING DOCUMENTATION</h1>
<h6 id="main-docs">Main Docs</h6>
<ul>
<li><a href="Pytorch-Lightning/LightningModule">LightningModule</a> </li>
<li><a href="LightningModule/LightningModule">LightningModule</a> </li>
<li><a href="Trainer/">Trainer</a> </li>
</ul>
<h6 id="new-project-quick-start">New project Quick Start</h6>
@@ -190,42 +198,10 @@
<li>Multi-gpu example </li>
<li>SLURM cluster grid search example </li>
</ul>
<h6 id="training-loop">Training loop</h6>
<ul>
<li>Accumulate gradients</li>
<li>Check GPU usage</li>
<li>Check which gradients are nan</li>
<li>Check validation every n epochs</li>
<li>Display metrics in progress bar</li>
<li>Force training for min or max epochs</li>
<li>Inspect gradient norms</li>
<li>Hooks</li>
<li>Learning rate annealing</li>
<li>Make model overfit on subset of data</li>
<li>Multiple optimizers (like GANs)</li>
<li>Set how much of the training set to check (1-100%)</li>
<li>training_step function</li>
</ul>
<h6 id="validation-loop">Validation loop</h6>
<ul>
<li>Display metrics in progress bar</li>
<li>hooks</li>
<li>Set how much of the validation set to check (1-100%)</li>
<li>Set validation check frequency within 1 training epoch (1-100%)</li>
<li>validation_step function</li>
<li>Why does validation run first for 5 steps?</li>
</ul>
<h6 id="distributed-training">Distributed training</h6>
<ul>
<li>Single-gpu </li>
<li>Multi-gpu </li>
<li>Multi-node </li>
<li>16-bit mixed precision</li>
</ul>
<h6 id="checkpointing">Checkpointing</h6>
<ul>
<li>Model saving</li>
<li>Model loading </li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Checkpointing/#model-saving">Model saving</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/LightningModule/methods/#load-from-metrics">Model loading</a> </li>
</ul>
<h6 id="computing-cluster-slurm">Computing cluster (SLURM)</h6>
<ul>
@@ -233,6 +209,50 @@
<li>Automatic saving, loading </li>
<li>Running grid search on a cluster </li>
<li>Walltime auto-resubmit </li>
</ul>
<h6 id="debugging">Debugging</h6>
<ul>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/debugging/#fast-dev-run">Fast dev run</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/debugging/#inspect-gradient-norms">Inspect gradient norms</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/debugging/#Log-gpu-usage">Log GPU usage</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/debugging/#make-model-overfit-on-subset-of-data">Make model overfit on subset of data</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/debugging/#print-the-parameter-count-by-layer">Print the parameter count by layer</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/debugging/#print-which-gradients-are-nan">Pring which gradients are nan</a></li>
</ul>
<h6 id="distributed-training">Distributed training</h6>
<ul>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Distributed%20training/#16-bit-mixed-precision">16-bit mixed precision</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Distributed%20training/#Multi-GPU">Multi-GPU</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Distributed%20training/#Multi-node">Multi-node</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Distributed%20training/#single-gpu">Single GPU</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Distributed%20training/#self-balancing-architecture">Self-balancing architecture</a></li>
</ul>
<h6 id="experiment-logging">Experiment Logging</h6>
<ul>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Logging/#display-metrics-in-progress-bar">Display metrics in progress bar</a></li>
<li>Log arbitrary metrics</li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Logging/#log-metric-row-every-k-batches">Log metric row every k batches</a></li>
<li><a href="Logging/#process-position">Process position</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Logging/#save-a-snapshot-of-all-hyperparameters">Save a snapshot of all hyperparameters</a> </li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Logging/#snapshot-code-for-a-training-run">Snapshot code for a training run</a> </li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Logging/#write-logs-file-to-csv-every-k-batches">Write logs file to csv every k batches</a></li>
</ul>
<h6 id="training-loop">Training loop</h6>
<ul>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#accumulated-gradients">Accumulate gradients</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#anneal-learning-rate">Anneal Learning rate</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#force-training-for-min-or-max-epochs">Force training for min or max epochs</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#force-disable-early-stop">Force disable early stop</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Pytorch-Lightning/LightningModule/#configure_optimizers">Use multiple optimizers (like GANs)</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#set-how-much-of-the-training-set-to-check">Set how much of the training set to check (1-100%)</a></li>
</ul>
<h6 id="validation-loop">Validation loop</h6>
<ul>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Validation%20loop/#check-validation-every-n-epochs">Check validation every n epochs</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Validation%20loop/#set-how-much-of-the-validation-set-to-check">Set how much of the validation set to check</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Validation%20loop/#set-how-much-of-the-test-set-to-check">Set how much of the test set to check</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Validation%20loop/#set-validation-check-frequency-within-1-training-epoch">Set validation check frequency within 1 training epoch</a></li>
<li><a href="https://williamfalcon.github.io/pytorch-lightning/Trainer/Validation%20loop/#set-the-number-of-validation-sanity-steps">Set the number of validation sanity steps</a></li>
</ul>
</div>
@@ -241,7 +261,7 @@
<div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
<a href="Pytorch-Lightning/LightningModule/" class="btn btn-neutral float-right" title="Lightning module">Next <span class="icon icon-circle-arrow-right"></span></a>
<a href="LightningModule/RequiredTrainerInterface/" class="btn btn-neutral float-right" title="Lightning Module interface">Next <span class="icon icon-circle-arrow-right"></span></a>
</div>
@@ -271,7 +291,7 @@
<span style="margin-left: 15px"><a href="Pytorch-Lightning/LightningModule/" style="color: #fcfcfc">Next &raquo;</a></span>
<span style="margin-left: 15px"><a href="LightningModule/RequiredTrainerInterface/" style="color: #fcfcfc">Next &raquo;</a></span>
</span>
</div>
@@ -284,5 +304,5 @@
<!--
MkDocs version : 1.0.4
Build Date UTC : 2019-06-27 18:33:25
Build Date UTC : 2019-06-28 21:46:09
-->
search.html
+6 -2
View File
@@ -48,11 +48,15 @@
<li class="toctree-l1">
<span class="caption-text">Pytorch Lightning</span>
<span class="caption-text">LightningModule</span>
<ul class="subnav">
<li class="">
<a class="" href="./Pytorch-Lightning/LightningModule/">Lightning module</a>
<a class="" href="./LightningModule/RequiredTrainerInterface/">Lightning Module interface</a>
</li>
<li class="">
<a class="" href="./LightningModule/methods/">Methods</a>
</li>
</ul>
</li>
search/search_index.json
+1 -1
View File
File diff suppressed because one or more lines are too long
sitemap.xml
+16 -11
View File
@@ -2,57 +2,62 @@
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>None</loc>
<lastmod>2019-06-27</lastmod>
<lastmod>2019-06-28</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-06-27</lastmod>
<lastmod>2019-06-28</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-06-27</lastmod>
<lastmod>2019-06-28</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-06-27</lastmod>
<lastmod>2019-06-28</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-06-27</lastmod>
<lastmod>2019-06-28</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-06-27</lastmod>
<lastmod>2019-06-28</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-06-27</lastmod>
<lastmod>2019-06-28</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-06-27</lastmod>
<lastmod>2019-06-28</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-06-27</lastmod>
<lastmod>2019-06-28</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-06-27</lastmod>
<lastmod>2019-06-28</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-06-27</lastmod>
<lastmod>2019-06-28</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2019-06-28</lastmod>
<changefreq>daily</changefreq>
</url>
</urlset>
sitemap.xml.gz
BIN
View File
Binary file not shown.