mirror of
https://github.com/wassname/pytorch-lightning.git
synced 2026-09-20 13:10:42 +08:00
09167efdb5
* formatting * formatting * fix interval * fix train loop * fix test * parametrize test * Apply suggestions from code review Co-Authored-By: Adrian Wälchli <adrian.waelchli@students.unibe.ch> * fix calling * flake8 * add types Co-authored-by: Adrian Wälchli <adrian.waelchli@students.unibe.ch> Co-authored-by: William Falcon <waf2107@columbia.edu>
Multi-node example
This demo launches a job using 2 GPUs on 2 different nodes (4 GPUs total). To run this demo do the following:
- Log into the jumphost node of your SLURM-managed cluster.
- Create a conda environment with Lightning and a GPU PyTorch version.
- Choose a script to submit
DDP
Submit this job to run with distributedDataParallel (2 nodes, 2 gpus each)
sbatch ddp_job_submit.sh YourEnv
DDP2
Submit this job to run with a different implementation of distributedDataParallel. In this version, each node acts like DataParallel but syncs across nodes like DDP.
sbatch ddp2_job_submit.sh YourEnv