Jobbergate CLI Overview

The Jobbergate CLI offers an interactive gateway to the functionalities of the Jobbergate API's. Users can utilize the CLI to manage resources and execute various tasks.

The CLI operates under two primary modes:

Resource Creation: The CLI introduces create subcommands for every resource, allowing users to establish new instances.
Resource Viewing: With list and get-one subcommands available for each resource, users can inspect different detail levels about the resource entities stored in the database.

To ensure secure access, the Jobbergate CLI offers a sign-in mechanism to the Jobbergate API. Once authenticated, users may use all the resources in Jobbergate that their account has been granted access to.

Installation

Jobbergate-cli is available on PyPI, and can be installed by a couple of ways compatible with the Python ecosystem:

pip:
```
pip install jobbergate-cli
```
Note

Installing on the system Python is not considered best practice as it may interfere with the system's package manager. Consider using a virtual environment or other installation methods.
pipx allows for the global installation of Python applications in isolated environments:
```
pipx install jobbergate-cli
```
uv executes and installs command-line tools provided by Python packages, similar to pipx:
```
uv tool install jobbergate-cli
```
venv is a built-in Python module that allows you to create isolated environments for Python projects. You can install in the directory that best suits your needs. For reference, /srv/new-jobbergate-cli-venv is the used on a few provisioning scripts. The commands to create the installation directory, set the virtual environment, install the cli, and create a symlink to enable it to be run from anywhere are as follows:
```
mkdir /srv
/usr/bin/python3.12 -m venv /srv/new-jobbergate-cli-venv
/srv/new-jobbergate-cli-venv/bin/pip install jobbergate-cli
ln -s /srv/new-jobbergate-cli-venv/bin/jobbergate /usr/local/bin/jobbergate
```

Configuration

The base configuration file can be set on /etc/default/jobbergate3-cli or on a .env file in the current working directory. Consult your system administrator for configuration values.

Discovering Command details

You can start learning about the commands and usage of the Jobbergate CLI by starting with this command:

$ jobbergate --help
Usage: jobbergate [OPTIONS] COMMAND [ARGS]...

  Welcome to the Jobbergate CLI!

  More information can be shown for each command listed below by running it
  with the --help option.

Options:
  --verbose / --no-verbose        Enable verbose logging to the terminal
                                  [default: no-verbose]
  --full / --no-full              Print all fields from CRUD commands
                                  [default: no-full]
  --raw / --no-raw                Print output from CRUD commands as raw json
                                  [default: no-raw]
  --version / --no-version        Print the version of jobbergate-cli and exit
                                  [default: no-version]
  --install-completion [bash|zsh|fish|powershell|pwsh]
                                  Install completion for the specified shell.
  --show-completion [bash|zsh|fish|powershell|pwsh]
                                  Show completion for the specified shell, to
                                  copy it or customize the installation.
  --help                          Show this message and exit.

Commands:
  applications     Commands to interact with applications
  job-scripts      Commands to interact with job scripts
  job-submissions  Commands to interact with job submissions
  login            Log in to the jobbergate-cli by storing the supplied...
  logout           Logs out of the jobbergate-cli.
  show-token       Show the token for the logged in user.

If you want to delve deeper and understand the usage of a specific subcommand, you can use the --help flag with that particular subcommand. For example, to better understand the usage of the job-scripts create subcommand, you would run:

jobbergate job-scripts create --help

Logging In

The first thing you need to do with the Jobbergate CLI is to log in:

jobbergate login

Upon executing the command, a message will appear like:

╭───────────────────────────────────────────── Waiting for login ──────────────────────────────────────────────────────╮
│                                                                                                                      │
│   To complete login, please open the following link in a browser:                                                    │
│                                                                                                                      │
│     http://keycloak.local:8080/realms/jobbergate-local/device?user_code=BMVJ-NLZS                                    │
│                                                                                                                      │
│   Waiting up to 5.0 minutes for you to complete the process...                                                       │
│                                                                                                                      │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Waiting for web login... ━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   3% 0:04:50

Next, you will need to:

Open the provided link by either clicking on it (if your terminal supports it) or copy/paste it into a browser.
Enter your login credentials
Complete the sign in process
Return to your terminal

You should see a message like:

╭──────────────────────────────────────────────── Logged in! ──────────────────────────────────────────────────────────╮
│                                                                                                                      │
│   User was logged in with email 'local-user@jobbergate.local'                                                        │
│                                                                                                                      │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Checking the Auth Token

To get access to the auth token you acquired by logging in, run this command:

jobbergate show-token --plain

Executing this command will display the authentication token in a plain text format, without any additional characters or formatting. This makes it easier for you to manually select and copy the token, especially in environments where clipboard access might be restricted, such as when using docker-compose or an SSH connection.

Once the token is displayed, you can copy the token to your clipboard to use with API requests.

It's essential to treat this token with care, as it provides access to the Jobbergate system under your user account. Ensure you don't share it with unauthorized individuals and avoid unintentionally exposing it in logs or scripts.

Resource Commands

Now that you are logged in, you can interact with any of the three main Jobbergate resources. Most of the resources provide the following sub-commands:

create: Create a new instance of the resource
delete: Delete an instance of the resource
get-one: Fetch details about a single instance of the resource
list: Fetch a listing of all the resources limited by filters
update: Update an instance of the resource.

Details for each subcommand can be viewed by passing the --help flag to any of them.

Use the --help option to explore the CLI and disccover the usage and options for all the subcommands.

Usability

When rendering a job-script from a template, the user will be asked a series of questions to fill in the template variables.

The library used for the questionnaire has a limitation that messages can only be displayed in a single line. This means that some of the questions can be truncated and will not be fully visible if the message is too long.

Note

To ensure that you can see the full output of the CLI, we recommend that you use a terminal in a maximized window.

Submission Modes

The Jobbergate CLI supports two modes for submitting jobs:

Remote Job Submission:
- SBATCH_PATH is configured on the CLI, so the payload to create a new submission does not include slurm_job_id.
- The API processes the creation as usual, giving it the status CREATED and adding the new entry on the database.
- The agent pulls pending jobs (status=CREATED) and submit them to slurm on every cycle.
- The agent pulls active jobs (STATUS=SUBMITTED) and update their status on every cycle.
Aiming to provide backward compatibility with Jobbergate-cli v1, on-site submissions are also available (new in 4.2.0):
- SBATCH_PATH is configured on the CLI, so it downloads job-script files, submits them to slurm using sbatch, and includes slurm_job_id on the payload to create a new submissions.
- The API processes the creation giving it the status SUBMITTED and adding the new entry on the database. No need to parse any job-property, since all #SBATCH directives were processed at submission time by sbatch.
- These jobs won't be processed as pending by the agent since their status is not CREATED, remember it was already submitted.
- Just like on remote submissions, the agent will still keep track of active jobs (STATUS=SUBMITTED) and update them as needed.