diff --git a/docs/Using SHARCNET.ipynb b/docs/Using SHARCNET.ipynb new file mode 100644 index 0000000..be4a144 --- /dev/null +++ b/docs/Using SHARCNET.ipynb @@ -0,0 +1,163 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Step 1: Register\n", + "\n", + "1. Create an account on the Compute Canada [website](https://ccdb.computecanada.ca/account_application)\n", + "2. You will be asked for Compute Canada Role Identifier (CCRI) of your sponsor or supervisor. A CCRI is an identifier of the form abc-123-01.\n", + "3. You and your supervisor will receive an email after creating the account; please confirm your application." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Step 2: SSH Setup\n", + "1. Update your SSH configuration\n", + "```bash\n", + "cat >> ~/.ssh/config\n", + "```\n", + "\n", + "2. Copy the following content after adding your username \n", + "```bash\n", + "Host sharc\n", + "User \n", + "Hostname graham.computecanada.ca\n", + "ServerAliveInterval 30\n", + "TCPKeepAlive yes\n", + "Protocol 2,1\n", + "Identityfile ~/.ssh/id_rsa\n", + "```\n", + "\n", + "3. Copy your SSH key to the server as an authorized key:\n", + "```bash\n", + "ssh-copy-id sharc\n", + "```\n", + "\n", + "4. Login. You may have to wait for sometime for your account to be updated on the\n", + "server.\n", + "```bash\n", + "ssh sharc\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Step 3: Environment Setup\n", + "1. Download and setup Anaconda\n", + "```bash\n", + "cd projects\n", + "wget https://repo.continuum.io/archive/Anaconda3-5.0.1-Linux-x86_64.sh\n", + "chmod +x Anaconda3-5.0.1-Linux-x86_64.sh\n", + "./Anaconda3-5.0.1-Linux-x86_64.sh\n", + "```\n", + "\n", + "2. Update `~/.bashrc` by adding the following:\n", + "\n", + "```bash\n", + "export PATH=/home//anaconda3/bin:$PATH\n", + "```\n", + "\n", + "3. Source\n", + "```bash\n", + "source ~/.bashrc\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Step 4: Setup SM model\n", + "1. Follow the detailed instructions from [here](https://github.com/castorini/Castor/tree/master/sm_cnn)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Step 5: SHARCNET basics\n", + "\n", + "Following are the list of common commands on SLURM. For more details, refer [this](https://www.sharcnet.ca/help/index.php/SLURM)\n", + "### Basic commands:\n", + "1.To submit a job:\n", + "```bash \n", + "sbatch \n", + "```\n", + "\n", + " Your job should have atleast the following as a part of the preamble:\n", + "\n", + "- CPU job:\n", + "```bash\n", + "#!/bin/bash\n", + "#SBATCH --time=00:01:00\n", + "#SBATCH --account=def-\n", + "```\n", + "\n", + "\n", + "- GPU job:\n", + "```bash\n", + "#!/bin/bash\n", + "#SBATCH --nodes=1\n", + "#SBATCH --gres=gpu:2\n", + "#SBATCH --ntasks-per-node=32\n", + "#SBATCH --mem=12288\n", + "#SBATCH --time=10:00\n", + "#SBATCH --account=def-\n", + "```\n", + "\n", + "The output of the job will be stored in the slurm-<jobid>.out file\n", + "\n", + "2.To monitor jobs in the queue:\n", + "```bash\n", + "squeue -u \n", + "```\n", + "\n", + "3.To see the completed jobs:\n", + "```bash\n", + "sacct\n", + "```\n", + "\n", + "4.To cancel a job:\n", + "```bash\n", + "scancel \n", + "```\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.3" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +}