wassname/pytorch-lightning
1
0
Fork 0
mirror of https://github.com/wassname/pytorch-lightning.git synced 2026-09-09 11:32:07 +08:00
Code Issues Packages Projects Releases Wiki Activity

made ddp the default if no backend specified with multiple GPUs (#1789)

Browse Source 
* made ddp the default if no backend specified with multiple GPUs

* fix

* spawn

Co-authored-by: Jirka <jirka.borovec@seznam.cz>
This commit is contained in:
William Falcon
2020-05-12 06:54:23 -04:00
committed by GitHub
co-authored by Jirka
parent acab068c74
commit 10b16dbfab
4 changed files with 6 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/source/multi_gpu.rst
+2
View File
@@ -132,6 +132,8 @@ Lightning allows multiple ways of training
- Horovod (`distributed_backend='horovod'`) (multi-machine, multi-gpu, configured at runtime)
- TPUs (`num_tpu_cores=8|x`) (tpu or TPU pod)
.. note:: If you request multiple GPUs without setting a mode, ddp will be automatically used.
Data Parallel (dp)
^^^^^^^^^^^^^^^^^^
`DataParallel <https://pytorch.org/docs/stable/nn.html#torch.nn.DataParallel>`_ splits a batch across k GPUs. That is, if you have a batch of 32 and use dp with 2 gpus,