release v0.5.3.2

* ImageNet example

* cleanup

* cleanup

* Minor changes from feedback

* More cleanup

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,16 +1,16 @@
 # Before submitting
 
-- Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
-- Did you read the [contributor guideline](https://github.com/williamFalcon/pytorch-lightning/blob/master/.github/CONTRIBUTING.md)?
-- Did you make sure to update the docs?   
-- Did you write any new necessary tests?  
+- [ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
+- [ ] Did you read the [contributor guideline](https://github.com/williamFalcon/pytorch-lightning/blob/master/.github/CONTRIBUTING.md)?
+- [ ] Did you make sure to update the docs?   
+- [ ] Did you write any new necessary tests?  
 
 ## What does this PR do?
 Fixes # (issue).
 
 ## PR review    
 Anyone in the community is free to review the PR once the tests have passed.     
-If we didn't discuss your PR in Github issues there's a high chance it will not be merged.   
+If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
 
 ## Did you have fun?
 Make sure you had fun coding 🙃

.travis.yml

@@ -16,16 +16,13 @@ language: python
 
 matrix:
   include:
-    - os: linux
-      dist: xenial  # Ubuntu 16.04
+#    - dist: xenial  # Ubuntu 16.04
+#      python: 3.5
+#      env: TOXENV=py35
+    - dist: bionic  # Ubuntu 18.04
       python: 3.6
       env: TOXENV=py36
-    - os: linux
-      dist: bionic  # Ubuntu 18.04
-      python: 3.6
-      env: TOXENV=py36
-    - os: linux
-      dist: bionic  # Ubuntu 18.04
+    - dist: bionic  # Ubuntu 18.04
       python: 3.7
       env: TOXENV=py37
     - os: osx
@@ -58,6 +55,7 @@ script:
   # integration
   - tox --sitepackages
   - pip install --editable .
+  #- python setup.py install --dry-run --user
 
 after_success:
   - coverage report

README.md

@@ -17,7 +17,7 @@
 [![ReadTheDocs](https://readthedocs.org/projects/pytorch-lightning/badge/?version=latest)](https://pytorch-lightning.readthedocs.io/en/latest)
 [![Gitter](https://badges.gitter.im/PyTorch-Lightning/community.svg)](https://gitter.im/PyTorch-Lightning/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
 [![license](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/williamFalcon/pytorch-lightning/blob/master/LICENSE)
-[![Next Release](https://img.shields.io/badge/Next%20Release-Nov%206-<COLOR>.svg)](https://shields.io/)
+[![Next Release](https://img.shields.io/badge/Next%20Release-Dec%206-<COLOR>.svg)](https://shields.io/)
 
 <!-- 
 removed until codecov badge isn't empy. likely a config error showing nothing on master.
@@ -63,85 +63,85 @@ Lightning sets up all the boilerplate state-of-the-art training for you so you c
 ---
 
 ## How do I do use it?   
-Think about Lightning as refactoring your research code instead of using a new framework. The research code goes into a [LightningModule]((https://williamfalcon.github.io/pytorch-lightning/LightningModule/RequiredTrainerInterface/)) which you fit using a Trainer.    
+Think about Lightning as refactoring your research code instead of using a new framework. The research code goes into a [LightningModule](https://williamfalcon.github.io/pytorch-lightning/LightningModule/RequiredTrainerInterface/) which you fit using a Trainer.    
 
 The LightningModule defines a *system* such as seq-2-seq, GAN, etc... It can ALSO define a simple classifier such as the example below.     
 
 To use lightning do 2 things:  
-1. [Define a LightningModule](https://williamfalcon.github.io/pytorch-lightning/LightningModule/RequiredTrainerInterface/)     
-
-**WARNING:** This syntax is for version 0.5.0+ where abbreviations were removed.    
-```python
-import os
-import torch
-from torch.nn import functional as F
-from torch.utils.data import DataLoader
-from torchvision.datasets import MNIST
-import torchvision.transforms as transforms
-
-import pytorch_lightning as pl
-
-class CoolSystem(pl.LightningModule):
-
-    def __init__(self):
-        super(CoolSystem, self).__init__()
-        # not the best model...
-        self.l1 = torch.nn.Linear(28 * 28, 10)
-
-    def forward(self, x):
-        return torch.relu(self.l1(x.view(x.size(0), -1)))
-
-    def training_step(self, batch, batch_nb):
-        # REQUIRED
-        x, y = batch
-        y_hat = self.forward(x)
-        loss = F.cross_entropy(y_hat, y)
-        tensorboard_logs = {'train_loss': loss}
-        return {'loss': loss, 'log': tensorboard_logs}
-
-    def validation_step(self, batch, batch_nb):
-        # OPTIONAL
-        x, y = batch
-        y_hat = self.forward(x)
-        return {'val_loss': F.cross_entropy(y_hat, y)}
-
-    def validation_end(self, outputs):
-        # OPTIONAL
-        avg_loss = torch.stack([x['val_loss'] for x in outputs]).mean()
-        tensorboard_logs = {'val_loss': avg_loss}
-        return {'avg_val_loss': avg_loss, 'log': tensorboard_logs}
-
-    def configure_optimizers(self):
-        # REQUIRED
-        # can return multiple optimizers and learning_rate schedulers
-        # (LBFGS it is automatically supported, no need for closure function)
-        return torch.optim.Adam(self.parameters(), lr=0.02)
-
-    @pl.data_loader
-    def train_dataloader(self):
-        # REQUIRED
-        return DataLoader(MNIST(os.getcwd(), train=True, download=True, transform=transforms.ToTensor()), batch_size=32)
-
-    @pl.data_loader
-    def val_dataloader(self):
-        # OPTIONAL
-        return DataLoader(MNIST(os.getcwd(), train=True, download=True, transform=transforms.ToTensor()), batch_size=32)
-
-    @pl.data_loader
-    def test_dataloader(self):
-        # OPTIONAL
-        return DataLoader(MNIST(os.getcwd(), train=False, download=True, transform=transforms.ToTensor()), batch_size=32)
-```
+1. [Define a LightningModule](https://williamfalcon.github.io/pytorch-lightning/LightningModule/RequiredTrainerInterface/)
+**WARNING:** This syntax is for version 0.5.0+ where abbreviations were removed.
+    ```python
+    import os
+    
+    import torch
+    from torch.nn import functional as F
+    from torch.utils.data import DataLoader
+    from torchvision.datasets import MNIST
+    from torchvision import transforms
+    
+    import pytorch_lightning as pl
+    
+    class CoolSystem(pl.LightningModule):
+    
+        def __init__(self):
+            super(CoolSystem, self).__init__()
+            # not the best model...
+            self.l1 = torch.nn.Linear(28 * 28, 10)
+    
+        def forward(self, x):
+            return torch.relu(self.l1(x.view(x.size(0), -1)))
+    
+        def training_step(self, batch, batch_nb):
+            # REQUIRED
+            x, y = batch
+            y_hat = self.forward(x)
+            loss = F.cross_entropy(y_hat, y)
+            tensorboard_logs = {'train_loss': loss}
+            return {'loss': loss, 'log': tensorboard_logs}
+    
+        def validation_step(self, batch, batch_nb):
+            # OPTIONAL
+            x, y = batch
+            y_hat = self.forward(x)
+            return {'val_loss': F.cross_entropy(y_hat, y)}
+    
+        def validation_end(self, outputs):
+            # OPTIONAL
+            avg_loss = torch.stack([x['val_loss'] for x in outputs]).mean()
+            tensorboard_logs = {'val_loss': avg_loss}
+            return {'avg_val_loss': avg_loss, 'log': tensorboard_logs}
+    
+        def configure_optimizers(self):
+            # REQUIRED
+            # can return multiple optimizers and learning_rate schedulers
+            # (LBFGS it is automatically supported, no need for closure function)
+            return torch.optim.Adam(self.parameters(), lr=0.02)
+    
+        @pl.data_loader
+        def train_dataloader(self):
+            # REQUIRED
+            return DataLoader(MNIST(os.getcwd(), train=True, download=True, transform=transforms.ToTensor()), batch_size=32)
+    
+        @pl.data_loader
+        def val_dataloader(self):
+            # OPTIONAL
+            return DataLoader(MNIST(os.getcwd(), train=True, download=True, transform=transforms.ToTensor()), batch_size=32)
+    
+        @pl.data_loader
+        def test_dataloader(self):
+            # OPTIONAL
+            return DataLoader(MNIST(os.getcwd(), train=False, download=True, transform=transforms.ToTensor()), batch_size=32)
+    ```
 2. Fit with a [trainer](https://williamfalcon.github.io/pytorch-lightning/Trainer/)    
-```python
-from pytorch_lightning import Trainer
-
-model = CoolSystem()
-
-# most basic trainer, uses good defaults
-trainer = Trainer()    
-trainer.fit(model)   
-```     
+    ```python
+    from pytorch_lightning import Trainer
+    
+    model = CoolSystem()
+    
+    # most basic trainer, uses good defaults
+    trainer = Trainer()    
+    trainer.fit(model)   
+    ```
 
 Trainer sets up a tensorboard logger, early stopping and checkpointing by default (you can modify all of them or
 use something other than tensorboard).   
@@ -166,7 +166,7 @@ trainer.fit(model)
 # view tensorboard logs 
 logging.info(f'View tensorboard logs by running\ntensorboard --logdir {os.getcwd()}')
 logging.info('and going to http://localhost:6006 on your browser')
-```    
+```
 
 When you're all done you can even run the test set separately.   
 ```python
@@ -348,7 +348,8 @@ Lightning also adds a text column with all the hyperparameters for this experime
 - [9 key speed features in Pytorch-Lightning](https://towardsdatascience.com/9-tips-for-training-lightning-fast-neural-networks-in-pytorch-8e63a502f565)    
 - [SLURM, multi-node training with Lightning](https://towardsdatascience.com/trivial-multi-node-training-with-pytorch-lightning-ff75dfb809bd)     
 
----    
+---
+
 ## Asking for help    
 Welcome to the Lightning community!   
 

docs/Trainer/Training Loop.md

@@ -112,8 +112,8 @@ def training_step(self, batch, batch_nb):
 ```
 
 ---
-#### Truncated Back Propagation Through Time
-There are times when multiple backwards passes are needed for each batch. For example, it may save memory to use Truncated Back Propagation Through Time when training RNNs.
+#### Truncated Backpropagation Through Time
+There are times when multiple backwards passes are needed for each batch. For example, it may save memory to use Truncated Backpropagation Through Time when training RNNs.
 
 When this flag is enabled each batch is split into sequences of size truncated_bptt_steps and passed to training_step(...) separately. A default splitting function is provided, however, you can override it for more flexibility. See [tbptt_split_batch](https://williamfalcon.github.io/pytorch-lightning/Trainer/hooks#tbptt_split_batch).
 

docs/Trainer/index.md

@@ -73,7 +73,7 @@ But of course the fun is in all the advanced things it can do:
 - [Set how much of the training set to check (1-100%)](https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#set-how-much-of-the-training-set-to-check)
 - [Step optimizers at arbitrary intervals](https://williamfalcon.github.io/pytorch-lightning/Trainer/hooks/#optimizer_step)
 - [Packed sequences](https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#packed-sequences-as-inputs)
-- [Truncated Back Propagation Through Time](https://williamfalcon.github.io/pytorch-lightning//Training%20Loop/#truncated-back-propation-through-time)
+- [Truncated Backpropagation Through Time](https://williamfalcon.github.io/pytorch-lightning/Trainer/Training%20Loop/#truncated-backpropagtion-through-time)
 
 **Validation loop**    
 

pl_examples/full_examples/imagenet/imagenet_example.py

@@ -0,0 +1,248 @@
+"""
+This example is largely adapted from https://github.com/pytorch/examples/blob/master/imagenet/main.py
+"""
+import argparse
+import os
+import random
+from collections import OrderedDict
+
+import torch
+import torch.backends.cudnn as cudnn
+import torch.nn.parallel
+import torch.nn.functional as F
+import torch.optim as optim
+import torch.optim.lr_scheduler as lr_scheduler
+import torch.utils.data
+import torch.utils.data.distributed
+
+import torchvision.transforms as transforms
+import torchvision.models as models
+import torchvision.datasets as datasets
+
+import pytorch_lightning as pl
+
+
+# pull out resnet names from torchvision models
+MODEL_NAMES = sorted(
+    name for name in models.__dict__
+    if name.islower() and not name.startswith("__") and callable(models.__dict__[name])
+)
+
+
+class ImageNetLightningModel(pl.LightningModule):
+
+    def __init__(self, hparams):
+        super(ImageNetLightningModel, self).__init__()
+        self.hparams = hparams
+        self.model = models.__dict__[self.hparams.arch](pretrained=self.hparams.pretrained)
+
+    def training_step(self, batch, batch_idx):
+        images, target = batch
+        output = self.model(images)
+        loss_val = F.cross_entropy(output, target)
+        acc1, acc5 = self.__accuracy(output, target, topk=(1, 5))
+
+        # in DP mode (default) make sure if result is scalar, there's another dim in the beginning
+        if self.trainer.use_dp or self.trainer.use_ddp2:
+            loss_val = loss_val.unsqueeze(0)
+            acc1 = acc1.unsqueeze(0)
+            acc5 = acc5.unsqueeze(0)
+
+        tqdm_dict = {'train_loss': loss_val}
+        output = OrderedDict({
+            'loss': loss_val,
+            'acc1': acc1,
+            'acc5': acc5,
+            'progress_bar': tqdm_dict,
+            'log': tqdm_dict
+        })
+
+        return output
+
+    def validation_step(self, batch, batch_idx):
+        images, target = batch
+        output = self.model(images)
+        loss_val = F.cross_entropy(output, target)
+        acc1, acc5 = self.__accuracy(output, target, topk=(1, 5))
+
+        # in DP mode (default) make sure if result is scalar, there's another dim in the beginning
+        if self.trainer.use_dp or self.trainer.use_ddp2:
+            loss_val = loss_val.unsqueeze(0)
+            acc1 = acc1.unsqueeze(0)
+            acc5 = acc5.unsqueeze(0)
+
+        output = OrderedDict({
+            'val_loss': loss_val,
+            'val_acc1': acc1,
+            'val_acc5': acc5,
+        })
+
+        return output
+
+    def validation_end(self, outputs):
+
+        tqdm_dict = {}
+
+        for metric_name in ["val_loss", "val_acc1", "val_acc5"]:
+            metric_total = 0
+
+            for output in outputs:
+                metric_value = output[metric_name]
+
+                # reduce manually when using dp
+                if self.trainer.use_dp or self.trainer.use_ddp2:
+                    metric_value = torch.mean(metric_value)
+
+                metric_total += metric_value
+
+            tqdm_dict[metric_name] = metric_total / len(outputs)
+
+        result = {'progress_bar': tqdm_dict, 'log': tqdm_dict, 'val_loss': tqdm_dict["val_loss"]}
+        return result
+
+    @classmethod
+    def __accuracy(cls, output, target, topk=(1,)):
+        """Computes the accuracy over the k top predictions for the specified values of k"""
+        with torch.no_grad():
+            maxk = max(topk)
+            batch_size = target.size(0)
+
+            _, pred = output.topk(maxk, 1, True, True)
+            pred = pred.t()
+            correct = pred.eq(target.view(1, -1).expand_as(pred))
+
+            res = []
+            for k in topk:
+                correct_k = correct[:k].view(-1).float().sum(0, keepdim=True)
+                res.append(correct_k.mul_(100.0 / batch_size))
+            return res
+
+    def configure_optimizers(self):
+        optimizer = optim.SGD(
+            self.parameters(),
+            lr=self.hparams.lr,
+            momentum=self.hparams.momentum,
+            weight_decay=self.hparams.weight_decay
+        )
+        scheduler = lr_scheduler.ExponentialLR(optimizer, gamma=0.1)
+        return [optimizer], [scheduler]
+
+    @pl.data_loader
+    def train_dataloader(self):
+        normalize = transforms.Normalize(
+            mean=[0.485, 0.456, 0.406],
+            std=[0.229, 0.224, 0.225],
+        )
+
+        train_dir = os.path.join(self.hparams.data, 'train')
+        train_dataset = datasets.ImageFolder(
+            train_dir,
+            transforms.Compose([
+                transforms.RandomResizedCrop(224),
+                transforms.RandomHorizontalFlip(),
+                transforms.ToTensor(),
+                normalize,
+            ]))
+
+        if self.use_ddp:
+            train_sampler = torch.utils.data.distributed.DistributedSampler(train_dataset)
+        else:
+            train_sampler = None
+
+        train_loader = torch.utils.data.DataLoader(
+            dataset=train_dataset,
+            batch_size=self.hparams.batch_size,
+            shuffle=(train_sampler is None),
+            num_workers=0,
+            sampler=train_sampler
+        )
+        return train_loader
+
+    @pl.data_loader
+    def val_dataloader(self):
+        normalize = transforms.Normalize(
+            mean=[0.485, 0.456, 0.406],
+            std=[0.229, 0.224, 0.225],
+        )
+        val_dir = os.path.join(self.hparams.data, 'val')
+        val_loader = torch.utils.data.DataLoader(
+            datasets.ImageFolder(val_dir, transforms.Compose([
+                transforms.Resize(256),
+                transforms.CenterCrop(224),
+                transforms.ToTensor(),
+                normalize,
+            ])),
+            batch_size=self.hparams.batch_size,
+            shuffle=False,
+            num_workers=0,
+        )
+        return val_loader
+
+    @staticmethod
+    def add_model_specific_args(parent_parser):  # pragma: no cover
+        parser = argparse.ArgumentParser(parents=[parent_parser])
+        parser.add_argument('-a', '--arch', metavar='ARCH', default='resnet18', choices=MODEL_NAMES,
+                            help='model architecture: ' +
+                                 ' | '.join(MODEL_NAMES) +
+                                 ' (default: resnet18)')
+        parser.add_argument('--epochs', default=90, type=int, metavar='N',
+                            help='number of total epochs to run')
+        parser.add_argument('--seed', type=int, default=None,
+                            help='seed for initializing training. ')
+        parser.add_argument('-b', '--batch-size', default=256, type=int,
+                            metavar='N',
+                            help='mini-batch size (default: 256), this is the total '
+                                 'batch size of all GPUs on the current node when '
+                                 'using Data Parallel or Distributed Data Parallel')
+        parser.add_argument('--lr', '--learning-rate', default=0.1, type=float,
+                            metavar='LR', help='initial learning rate', dest='lr')
+        parser.add_argument('--momentum', default=0.9, type=float, metavar='M',
+                            help='momentum')
+        parser.add_argument('--wd', '--weight-decay', default=1e-4, type=float,
+                            metavar='W', help='weight decay (default: 1e-4)',
+                            dest='weight_decay')
+        parser.add_argument('--pretrained', dest='pretrained', action='store_true',
+                            help='use pre-trained model')
+        return parser
+
+
+def get_args():
+    parent_parser = argparse.ArgumentParser(add_help=False)
+    parent_parser.add_argument('--data-path', metavar='DIR', type=str,
+                               help='path to dataset')
+    parent_parser.add_argument('--save-path', metavar='DIR', default=".", type=str,
+                               help='path to save output')
+    parent_parser.add_argument('--gpus', type=int, default=1,
+                               help='how many gpus')
+    parent_parser.add_argument('--distributed-backend', type=str, default='dp', choices=('dp', 'ddp', 'ddp2'),
+                               help='supports three options dp, ddp, ddp2')
+    parent_parser.add_argument('--use-16bit', dest='use-16bit', action='store_true',
+                               help='if true uses 16 bit precision')
+    parent_parser.add_argument('-e', '--evaluate', dest='evaluate', action='store_true',
+                               help='evaluate model on validation set')
+
+    parser = ImageNetLightningModel.add_model_specific_args(parent_parser)
+    return parser.parse_args()
+
+
+def main(hparams):
+    model = ImageNetLightningModel(hparams)
+    if hparams.seed is not None:
+        random.seed(hparams.seed)
+        torch.manual_seed(hparams.seed)
+        cudnn.deterministic = True
+    trainer = pl.Trainer(
+        default_save_path=hparams.save_path,
+        gpus=hparams.gpus,
+        max_nb_epochs=hparams.epochs,
+        distributed_backend=hparams.distributed_backend,
+        use_amp=hparams.use_16bit
+    )
+    if hparams.evaluate:
+        trainer.run_evaluation()
+    else:
+        trainer.fit(model)
+
+
+if __name__ == '__main__':
+    main(get_args())

pytorch_lightning/__init__.py

@@ -1,14 +1,13 @@
 """Package info"""
 
-__version__ = '0.5.3'
+__version__ = '0.5.3.2'
 __author__ = ' William Falcon et al.'
 __author_email__ = 'waf2107@columbia.edu'
 __license__ = 'Apache-2.0'
-__homepage__ = 'https://github.com/williamFalcon/pytorch-lightning',
-__docs__ = """# PyTorch Lightning
-
-The lightweight PyTorch wrapper for ML researchers. Scale your models. Write less boilerplate.
-"""
+__homepage__ = 'https://github.com/williamFalcon/pytorch-lightning'
+# this has to be simple string, see: https://github.com/pypa/twine/issues/522
+__docs__ = "PyTorch Lightning is the lightweight PyTorch wrapper for ML researchers." \
+           " Scale your models. Write less boilerplate."
 
 
 try:

pytorch_lightning/callbacks/pt_callbacks.py

@@ -253,7 +253,7 @@ class ModelCheckpoint(Callback):
                         if self.verbose > 0:
                             logging.info(
                                 f'\nEpoch {epoch + 1:05d}: {self.monitor} improved'
-                                f' from {self.best:0.5f} to {current:0.5f},',
+                                f' from {self.best:0.5f} to {current:0.5f},'
                                 f' saving model to {filepath}')
                         self.best = current
                         self.save_model(filepath, overwrite=True)

requirements.txt

@@ -1,8 +1,8 @@
 scikit-learn==0.20.2
 tqdm==4.35.0
-twine==1.13.0
 numpy==1.16.4
 torch>=1.2.0
 torchvision>=0.3.0
 pandas>=0.20.3
-# future>=0.17.1  # required for buildins in setup.py
+test-tube>=0.6.9
+# future>=0.17.1  # required for buildins in setup.py

setup.cfg

@@ -41,12 +41,13 @@ exclude_lines =
     break
     pass
     os.makedirs
+# TODO: to be reviewed, this should not be skipped
 omit =
     pytorch_lightning/callbacks/pt_callbacks.py
     tests/test_models.py
     pytorch_lightning/testing_models/lm_test_module.py
     pytorch_lightning/utilities/arg_parse.py
-    examples/templates
+    pl_examples/templates
 
 [flake8]
 # TODO: this should be 88 or 100 according PEP8
@@ -58,3 +59,15 @@ verbose = 2
 # https://pep8.readthedocs.io/en/latest/intro.html#error-codes
 format = pylint
 ignore = E731,W504,F401,F841
+
+[check-manifest]
+ignore =
+    .travis.yml
+    tox.ini
+    .github
+    .github/*
+
+[metadata]
+license_file = LICENSE
+# long_description = file:README.md
+# long_description_content_type = text/markdown

setup.py

@@ -46,15 +46,18 @@ setup(
     url=pytorch_lightning.__homepage__,
     download_url='https://github.com/williamFalcon/pytorch-lightning',
     license=pytorch_lightning.__license__,
-    packages=find_packages(exclude=['examples']),
+    packages=find_packages(exclude=['tests']),
+
     long_description=open('README.md', encoding='utf-8').read(),
     long_description_content_type='text/markdown',
     include_package_data=True,
     zip_safe=False,
+
     keywords=['deep learning', 'pytorch', 'AI'],
     python_requires='>=3.6',
     setup_requires=[],
     install_requires=load_requirements(PATH_ROOT),
+
     classifiers=[
         'Environment :: Console',
         'Natural Language :: English',

tests/requirements.txt

@@ -5,6 +5,7 @@ pytest>=3.0.5
 pytest-cov
 flake8
 check-manifest
-test_tube
+# test_tube  # already installed in main req.
 mlflow
 comet_ml
+twine==1.13.0

tox.ini

@@ -35,10 +35,12 @@ deps =
     -r ./tests/requirements.txt
 commands =
     pip list
-    check-manifest --ignore tox.ini
+    check-manifest
     python setup.py check --metadata --strict
     coverage run --source pytorch_lightning -m py.test pytorch_lightning tests pl_examples -v --doctest-modules
     flake8 .
+    python setup.py sdist
+    twine check dist/*
 
 # DROP, it is duplication of setup.cfg
 # [flake8]