Skip to content

DS150 Cluster Walkthrough

This document will help you understand how to use the BOSE supercomputing cluster, and various tools on it, for DS150. If you have any questions, feel free to email us at bgsc.admins@uwec.edu

Open OnDemand | Linux Quick Guide PDF | GlobalProtect VPN

Part 1: Logging in to Open OnDemand

For your convenience, the cluster can be accessed through a graphical user interface (GUI) called Open OnDemand. This interface will allow you to access all of the features you will need for DS150.

  1. Navigate to https://ondemand.hpc.uwec.edu (Requires on-campus or VPN connection).
  2. Enter your UWEC login and authenticate with Okta.
  3. Complete the first time sign-in process if you have never used the cluster before. * Follow the instructions on screen exactly. Make sure you accept all the terms of service.

Trouble with setup?

If you're struggling with first time sign-in or accepting the terms of service, try following our detailed guide at https://docs.hpc.uwec.edu/ood/first-time/

Part 2: The OnDemand File Manager

Open OnDemand has a file browser built-in that can be used for basic operations such a creating, viewing, editing, and deleting files. To access it, click Files > Home Directory in the hotbar or click the pinned app on the OnDemand dashboard.

  1. Open your my_ds150 folder:

Home Directory

  1. Create a new folder named lesson1:

my_ds150 Directory

  1. Navigate back to your home directory by clicking on your username above the list of files, or by clicking "Home Directory" on the left.

  2. Click on ds150_course_files, which is where you'll be able to find all of the course files you need to copy each day.

  3. Copy the practice_notebook_1.ipynb file from /data/users/<your-username>/ds150_course_files/notebooks/ by selecting the file and clicking Copy/Move:

Notebook Copy

  1. A new message box will appear on the left side of your screen. Navigate back to /data/users/<your-username>/my_ds150/lesson1 and click the Copy button from the new box. This will copy practice_notebook_1.ipynb to your own local folder:

Notebook Paste

Part 3: Jupyter

Jupyter Notebook is an interactive computing platform allowing for the dynamic compiling of certain programming languages. To access it, click Interactive Apps > Jupyter Notebook in the hotbar or click the pinned app on the OnDemand dashboard.

The following settings are recommended:

  • Accounting Group: 2251.ds.150.001 (You might have multiple!)
  • Slurm Partition: Week (7 Days) - Default
  • CPU Cores: 1
  • Memory: 5G
  • # GPU Cards: No GPUs
  • Number of hours: How long you intend to actively use Jupyter at the time.
  • Note that it’ll stop once you hit the time limit, in which case you can just submit the form again to pick up where you last saved.
  • Working Directory: Leave blank

Part 4: Bash and Basic Shell Commands

For now, we are going to go into the shell environment, which is less graphical and where you'll be running some commands.

1) Click on the "BOSE Cluster Shell Access" tile on the home page, or go to "Clusters --> BOSE Cluster Shell Access" on the top of the page.

The Prompt

The Bourne Again Shell (BASH) will be your primary method of interacting with your files on the cluster. When you are in the shell, you will see a Command Prompt that looks something like this

[username1234@bose ~]$

The prompt has several components:

  1. Your username
  2. The hostname of the machine. In this example the hostname is bose
  3. Your current working directory. The tilde (~) represents a user's Home Directory

Commands

Refer to the Linux Quick Guide at the front of this packet for helpful shell commands and more information on relative and absolute paths

The dollar sign ($) after the prompt is where you will enter commands. Commands can be used to perform many operations on the cluster, but for now we will be sticking to basic file management. Most commands take Arguments, which are additional information you provide to tell a command what to do. For example, the touch command creates a new file, but you must provide a name for the newly created file: touch myfile.txt A Flag is something which can be added after a command to alter its behavior. Single-letter flags can be combined into one large flag. For example, instead of ls -l -a, you can type ls -la.

Tabbing

When you enter commands, you can press tab to auto-complete both commands and file/directory names. For example, if you want to remove the file myfile.txt, you can type rm myf and then press tab. This will autocomplete the command for you so long as the file exists and there are no other files or directories matching that pattern in your working directory. In the case of ambiguity (such as myfile.txt and myfile2.txt existing in the working directory), you can double-press tab to see all matching entries in the directory. Use this as much as possible to save lots of time and your fingers!

DS150 Instructions:

Below is a list, in order, of the commands you will run for your first assignment:

hello_world.py

  1. Change the working directory to your home directory: 

    cd

  2. Copy provided python (.py) files to your personal lesson1 directory. Note the use of the wildcard character * to copy all files ending in .py: 

    cp ds150_course_files/code/*.py my_ds150/lesson1/

  3. Copy provided database (.csv) files to your personal lesson1 directory. Note the use of the wildcard character * to copy all files ending in .csv: 
    cp ds150_course_files/data/lesson1/*.csv my_ds150/lesson1/

  4. Change your working directory to your lesson1 folder. Run pwd at any time to confirm if your path is correct: 

    cd my_ds150/lesson1/

  5. Load DS150's custom python module. If the module is activated, (ds150) will be added to your prompt: 

    module load ds150

  6. Run the hello_world.py python file: 
    python hello_world.py

Part 5: Visual Studio Code

Visual Studio Code is a versatile text editor developed my Microsoft. It allows for simple text editing paired with extensive features for programmers and developers. To access it, click Interactive Apps > Visual Studio Code in the hotbar or click the pinned app on the OnDemand dashboard.

Launch a session of the app and begin editing your hello_world.py file located at /data/users/<your-username>/my_ds150/lesson1.

Part 6: More Bash and Commands

DS150 Instructions:

hello_world.py and plot_penguins.py

  1. Change your working directory to your lesson1 directory: 
    cd my_ds150/lesson1/
  2. Re-run hello_world.py and see its results with your new edits: 
    python hello_world.py
  3. Run plot_penguins.py and examine its results: 
    python plot_penguins.py