Castor/docs/Using SHARCNET.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 1: Register\n",
    "\n",
    "1. Create an account on the Compute Canada [website](https://ccdb.computecanada.ca/account_application)\n",
    "2. You will be asked for Compute Canada Role Identifier (CCRI) of your sponsor or supervisor. A CCRI is an identifier of the form abc-123-01.\n",
    "3. You and your supervisor will receive an email after creating the account; please confirm your application."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 2: SSH Setup\n",
    "1. Update your SSH configuration\n",
    "```bash\n",
    "cat >> ~/.ssh/config\n",
    "```\n",
    "\n",
    "2. Copy the following content after adding your username \n",
    "```bash\n",
    "Host sharc\n",
    "User <username>\n",
    "Hostname graham.computecanada.ca\n",
    "ServerAliveInterval 30\n",
    "TCPKeepAlive yes\n",
    "Protocol 2,1\n",
    "Identityfile ~/.ssh/id_rsa\n",
    "```\n",
    "\n",
    "3. Copy your SSH key to the server as an authorized key:\n",
    "```bash\n",
    "ssh-copy-id sharc\n",
    "```\n",
    "\n",
    "4. Login. You may have to wait for sometime for your account to be updated on the\n",
    "server.\n",
    "```bash\n",
    "ssh sharc\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 3: Environment Setup\n",
    "1. Download and setup Anaconda\n",
    "```bash\n",
    "cd projects\n",
    "wget https://repo.continuum.io/archive/Anaconda3-5.0.1-Linux-x86_64.sh\n",
    "chmod +x Anaconda3-5.0.1-Linux-x86_64.sh\n",
    "./Anaconda3-5.0.1-Linux-x86_64.sh\n",
    "```\n",
    "\n",
    "2. Update `~/.bashrc` by adding the following:\n",
    "\n",
    "```bash\n",
    "export PATH=/home/<username>/anaconda3/bin:$PATH\n",
    "```\n",
    "\n",
    "3. Source\n",
    "```bash\n",
    "source ~/.bashrc\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 4: Setup SM model\n",
    "1. Follow the detailed instructions from [here](https://github.com/castorini/Castor/tree/master/sm_cnn)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 5: SHARCNET basics\n",
    "\n",
    "Following are the list of common commands on SLURM. For more details, refer [this](https://www.sharcnet.ca/help/index.php/SLURM)\n",
    "### Basic commands:\n",
    "1.To submit a job:\n",
    "```bash \n",
    "sbatch <job_name.sh>\n",
    "```\n",
    "\n",
    " Your job should have atleast the following as a part of the preamble:\n",
    "\n",
    "- CPU job:\n",
    "```bash\n",
    "#!/bin/bash\n",
    "#SBATCH --time=00:01:00\n",
    "#SBATCH --account=def-<name-of-the-sponsor>\n",
    "```\n",
    "\n",
    "\n",
    "- GPU job:\n",
    "```bash\n",
    "#!/bin/bash\n",
    "#SBATCH --nodes=1\n",
    "#SBATCH --gres=gpu:2\n",
    "#SBATCH --ntasks-per-node=32\n",
    "#SBATCH --mem=12288\n",
    "#SBATCH --time=10:00\n",
    "#SBATCH --account=def-<name-of-the-sponsor>\n",
    "```\n",
    "\n",
    "The output of the job will be stored in the slurm-&lt;jobid&gt;.out file\n",
    "\n",
    "2.To monitor jobs in the queue:\n",
    "```bash\n",
    "squeue -u <username>\n",
    "```\n",
    "\n",
    "3.To see the completed jobs:\n",
    "```bash\n",
    "sacct\n",
    "```\n",
    "\n",
    "4.To cancel a job:\n",
    "```bash\n",
    "scancel <jobid>\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}