Contributing to OSD#

OSD is an open-source project, and as such we welcome contributions from developers, engineers, system administrators, and users in general.

There are many ways to contribute:

Feedback#

Feedback is very welcome! We are interested in hearing from you how! Email us @ info@omnivector.solutions.

In the case things aren't working as expected, please file a bug.

Reach out if you have any questions and/or suggestions.

Documentation#

This documentation is built using Sphinx and reStructuredText, and is hosted in GitHub.

The master branch is rebuilt at every commit and published automatically online. If you want to test/show any specific branch, you can do so by "resetting" the testing branch. This branch is automatically built and published to omnivector-solutions.github.io/osd-documentation/testing/. One way to achieve that is:

$ git checkout testing
$ git reset --hard <my-branch>
$ git push --force

General Recommendations#

Use US English.

Wrap the lines at 80 columns when writing documentation.

Sections#

This projects uses the following convention for section headers, from the rsTructuredText Primer document:

=================
This is a heading
=================

This is a section
=================

This is a subsection
--------------------

This is a subsubsection
^^^^^^^^^^^^^^^^^^^^^^^

This is a subsubsubsection
~~~~~~~~~~~~~~~~~~~~~~~~~~

Naming conventions#

Throughout the documentation and code (specially in logs), we stick to these naming conventions with regards to Slurm:

Slurm

SchedMD Slurm project.

slurmXXXd

The daemons for the Slurm component.

For example, slurmd refers to the daemon for the compute nodes, while slurmctld refers to the daemon for the controller node.

Slurm-charms

Omnivector charms.

Code#

OSD source-code is hosted in GitHub repositories. All the charms are in omnivector-solutions/slurm-charms, and a common code shared among them, called slurm-ops-mananger, is hosted at omnivector-solutions/slurm-ops-manager.

The Slurm-charms source code is developed under the MIT license. By contributing to the Slurm-charms project or affiliated code you are implicitly agreeing to your code contributions being governed by this license too.

Git usage#

  • all commits should be atomic

  • all commits should have a proper commit message:

    • the first line should be a brief summary of the changes, no longer than 52 characters

    • second line should be empty

    • subsequent lines should describe the changes and explain why they are needed

  • linear history

    • always rebase with master

    • fast forward merge or squash merge

  • keep user related changes in the CHANGELOG

  • annotated tags for releases, see Release process below for details

  • do not include merge commits in new branches. New branches should contain only new code.

Charm development#

All Python code should be compliant with Python3.6. This is a strong requirement for the charms to run on CentOS7.

The coding style used follows closely PEP-008.

Local setup#

This sections describes how to setup your system for local charm development. This section assumes the system is Linux-based and has support for snaps.

The tool used to build a charm is charmcraft, and it should be installed as a classical snap:

$ sudo snap install --edge charmcraft --classic

The OSD charm code is available in the Git repository slurm-charms, it contains all the nuts and bolts of all Slurm Charms as well as a helper Makefile to build the charms. To clone and build:

$ git clone https://github.com/omnivector-solutions/slurm-charms
$ cd slurm-charms
$ make charms

After the .charm artifacts have been produced, juju can be used to deploy the built charms to a cloud environment of your choosing. There are two primary ways to deploy the charms after building them:

  • use Juju to deploy the built charms directly e.g. juju deploy ./slurmd.charm.

    Note

    slurmctld requires an etcd resource to be deployed. Download it from etcd releases and specify it in the deploy command, e.g. juju deploy ./slurmctld --resource etcd=etcd-v3.5.0-linux-amd64.tar.gz. This is not necessary if using the slurm-bundles Makefile.

  • deploy the local development bundle contained in slurm-bundles. The slurm-bundles contain a helper Makefile that provide a way to easily deploy the built charms to a local LXD cloud by using the command make lxd-focal, or for CentOS7, make lxd-centos. See Installation overview for more details

Slurm-ops-manager#

The slurm-ops-manager is a code base that contains common methods and attributes used across multiple Slurm charms. Code that is used in more than one Slurm charm should not be repeated, it should live in this project instead.

After making a new slurm-ops-manager release, the Slurm-charms project should have one new commit in which the slurm-ops-manager version is incremented:

  • update the version of slurm-ops-manager in the requirements.txt file for each charm

  • update the CHANGELOG.

  • this should be a single commit. If there are any necessary changes in the charms, these should live in separate commits.

Slurm-charms#

Find below the standards and conventions used when contributing code to the slurm-charms.

Actions and Configurations#

In an effort to keep the OSD documentation current and accurate, we generate the documentation for charm actions and charm configurations dynamically. OSD documentation is built in such a way that the actions and configurations that exist in each Slurm charm are what end up in the documentation.

All actions, configurations, and their parameters _should_ have a description. A few things to note around the description field:

  • inline markdown code is translated to inline reStructuredText code

  • if there is a Note: text, this paragraph becomes a Sphinx note:: directive (this should be used to inform the end users of important information related to that action/configuration)

  • if there is an Example usage: text, that paragraph becomes a Sphinx code-block:: with the example given.

There are a few standards around YAML formatting that we closely follow in the Slurm charms:

  • wrap the lines at 80 columns

  • use multi-line strings

CHANGELOG#

The CHANGELOG file contains notable changes for end users, not charm developers. This means that internal changes (e.g., code related to CI, refactoring, unit tests, etc.) should not be documented.

Changes that should be documented include:

  • bug fixes

  • new features

  • new components

  • changes in usage

  • breaking changes

This file should be written in reStructuredText format and it is copied to the documentation Changelog page automatically.

There should always be one, and only one, section Unreleased in the CHANGELOG. New entries should go at the top of this section.

Bats tests#

The Slurm Charms have an extensive suite of tests using Bats.

To run the tests, you need two additional dependencies: - npm and npx (sudo apt install npm) - juju-wait (sudo snap install juju-wait --classic)

And then proceed to install Bats with npm:

$ npm install -g bats

To run all the tests, deploy the base bundle for OSD, then enter in the directory of the slurm-charms and specify the name of the Juju model used:

$ JUJU_MODEL=default npx bats tests/

Note

This will take a some minutes to run.

You can also run specific test-files manually:

$ JUJU_MODEL=default npx bats tests/15-acct-gather.bats

Alternatively, the Makefile contains a tests target that packs the charms, deploys them, and run the tests. The test script creates two Juju models on the active controller (be sure to be using a localhost LXD controller), one for CentOS7 and one for Ubuntu Focal, deploys the charms, and runs all the tests. This target does not remove these new models after the tests finish and takes a considerable amount of time to run.

Note

To run the make tests target, you need a custom CentOS7 image on your LXD installation. Please refer to CentOS7 Deploys on LXD clouds for details on building one.

Versioning#

The Slurm-charms and slurm-ops-manager follow a semantic versioning scheme.

Release process#

Slurm-charms#

To make a new release of the Slurm-charms, use the script/release.sh script. This script will update the CHANGELOG with the version of the new release, create a new commit with the changes, and create a new Git tag with a description of the new release. This script intentionally does not push to GitHub, this way one can review the changes before git push.

After the new release is made in GitHub, pack and release the charms in Charmhub.

Slurm-ops-manager#

The release process for slurm-ops-manager is straightforward:

  • bump the version string in the setup.py file

  • create a new annotated Git tag: git tag --annotate --sign x.y.z. The tag name should be the new version of the library, as explained in Versioning.

  • the tag message should contain a summary of the changes for that release. For example:

    Release 0.7.1

    • use Omnivector's repository for rpm installation

  • push to GitHub: git push --tags.