Globus CLI
Overview
Globus CLI is a graphic-less interface for the Globus file manager application. It houses the full feature-set of Globus through the Command Line Interface (CLI) or terminal, which is a great way to automate transfers, such as on a regularily scheduled basis or as part of a job script.
Availability
| Cluster | Module/Version |
|---|---|
| BOSE | 3.41.0 |
| BGSC | Not Available |
Note: It's highly recommended to do module load globus-cli to always activate the most recently installed version of this software. Unlike other software, we'll routinely remove old module versions.
Initial Set Up
Before you can use the Globus CLI tool, you'll need to first log in and link your account.
-
Enable the Globus CLI module:
-
Log into Globus
-
Copy the provided link into your web browser and log into Globus using your campus credentials, or with another Globus account.
- The first time you log in you'll be asked to authorize "Globus CLI", which gives the tool access to your account.
- You'll then be provided a "Native App Authorization Code", which you'll need to copy and paste back into the terminal to finish logging in before the code expires within 10 minutes.
Searching Collections
Before you can transfer any files, you'll need to identifify the UUID of both the source and destination collection. This could be the BOSE cluster, the client on your own computer, or another computing center.
Finding Our Collections: globus endpoint search "UWEC - BOSE"
[username@bose ~]$ globus endpoint search "UWEC - BOSE"
ID | Owner | Display Name
------------------------------------ | -------------------- | -----------------------------
4296d970-aa79-484e-946f-3779ac3f7512 | uwechpc@globusid.org | UWEC - BOSE Group Directories
26970a2d-6e76-4853-adab-c4596a910bb9 | uwechpc@globusid.org | UWEC - BOSE Home Directories
23217f7a-d8c9-46bf-a536-6a89b0971af0 | uwechpc@globusid.org | UWEC - BOSE Globus Endpoint
Finding Your Collections: globus endpoint search --filter-scope my-endpoints
Authorizing a Collection
When connecting to a collection for the first time, you may run into a "data_access" error. This means that your Globus account is currently not set up to allow access to any files in that collection.
Follow the instructions asking you to run globus session consent [...], which involves copy/pasting a URL into your web browser, choosing an identity (see below), and copy/pasting a code back into terminal.
If UW-Eau Claire: Use my <username>@uwec.edu identity
If Other Campuses: Use my <username>@idp.hpc.uwec.edu identity
Listing Files
Before you attempt transferring any files, it would be a good id to verify exactly what files/folders you see in the collection.
For example, this is the command for viewing your home directory on the BOSE cluster.
Transferring Files
In order to transfer files, a user needs to write a transfer command that matches the following syntax:
globus transfer <SRC_UUID>:path/to/file.jpg <DEST_UUID>:./file.jpg # Final destination file name is required.
This is very closely related to the syntax of an scp command. The UUID acts as a host machine followed by a path, and the first path acts as the source while the second path the destination.
More Options
The globus transfer command has a variety of other arguments that may be valuable, such as syncing, dry runs, and inclusion/exclusion filters.
Viewing Activity Logs
Globus keeps an activity log of all transfer attempts, which is very valuable when knowing progress or to identify issues.
Common Issues
| Type | Errror | Possible Resolution |
|---|---|---|
| Permission Denied | Path not allowed |
Verify that your folder/file path doesn't start with a / and is valid relative to the collection starting point. Run globus ls <UUID> to verify. |
| File Access Error | Unable to open file |
If transferring a file, make sure a file name is specified on both the source and destination. |
Real Example
Has your research group used the Globus CLI in a project to automate transfers? Contact the HPC Team as we'd love to learn more about about Globus is used.