nomad:csi
Differences
This shows you the differences between two versions of the page.
| — |
nomad:csi [2021/01/01 23:26] (current) ben created |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| + | ====== Overview ====== | ||
| + | |||
| + | [[https:// | ||
| + | |||
| + | ===== Nomad implementation ===== | ||
| + | |||
| + | * [[https:// | ||
| + | * [[https:// | ||
| + | |||
| + | ====== Installation ====== | ||
| + | |||
| + | The CSI plugin is run as nomad jobs with: | ||
| + | |||
| + | * two instances in controller mode as a service job | ||
| + | * one instance per node, running in node mode, as a system job | ||
| + | |||
| + | The controllers are responsible for managing the volumes, and the nodes are responsible for mounting the volumes onto the nomad clients prior to starting a job which wishes to use them. | ||
| + | |||
| + | The job definitions, | ||
| + | |||
| + | ====== Day to day tasks ====== | ||
| + | |||
| + | ===== Creating a volume ===== | ||
| + | |||
| + | Nomad can't provision new volumes itself yet, they must be created manually. This requires '' | ||
| + | GO111MODULE=off go get -u github.com/ | ||
| + | </ | ||
| + | |||
| + | To create a new 100MB volume named '' | ||
| + | ~/ | ||
| + | # " | ||
| + | </ | ||
| + | |||
| + | ===== Registering the volume with Nomad ===== | ||
| + | |||
| + | Create a hcl volume definition file with contents similar to: <code bash vol-acme.json> | ||
| + | id = " | ||
| + | name = " | ||
| + | type = " | ||
| + | external_id = " | ||
| + | plugin_id = " | ||
| + | access_mode = " | ||
| + | attachment_mode = " | ||
| + | mount_options { | ||
| + | fs_type = " | ||
| + | mount_flags = [" | ||
| + | } | ||
| + | context { | ||
| + | node_attach_driver = " | ||
| + | provisioner_driver = " | ||
| + | server = " | ||
| + | share = "/ | ||
| + | } | ||
| + | </ | ||
| + | |||
| + | Register this volume with nomad using: <code bash> | ||
| + | nomad volume register vol-acme.json | ||
| + | </ | ||
| + | |||
| + | ===== Making changes to a volume definition ===== | ||
| + | |||
| + | If it's necessary to make changes to the volume definition, it must be unregistered and reregistered with the new options: <code bash> | ||
| + | nomad volume deregister traefik-acme | ||
| + | nomad volume register vol-acme.json | ||
| + | </ | ||
| + | |||
| + | The volume must not be in use for it to be deregisterable. If a job failed, nomad might not properly record that the allocation is no longer using the volume, in which case it can be forceably deregistered with: <code bash> | ||
| + | nomad volume deregister -force traefik-acme | ||
| + | </ | ||
| + | |||
| + | ===== Resizing a volume ===== | ||
| + | |||
| + | '' | ||
| + | ~/ | ||
| + | </ | ||
| + | |||
| + | This segfaulted for me. However, all it's doing is calling '' | ||
| + | |||
| + | ====== Notes ====== | ||
| + | |||
| + | * The controller creates volumes by running '' | ||
| + | * It uses a root ssh key for this, which is stored in vault, and made available to the controller by nomad | ||
| + | * It might be possible to reduce the permissions required, by creating a dedicated user account for this, and delegating zfs permissions to that user (future investigation). | ||
| + | * Mount option '' | ||
| + | * The controller is configured to enable zfs snapshots by setting the '' | ||
nomad/csi.txt · Last modified: 2021/01/01 23:26 by ben
Page Tools
