Skip to main content

resourceLabels

The resourceLabels directive attaches custom name-value pairs to task executions, for executors that support it.

Usage​

For example:

process hello {
    resourceLabels region: 'some-region', user: 'some-username'

    script:
    """
    your_command --here
    """
}

Resource labels are attached to underlying resources such as cloud VMs, and are intended for operational purposes such as cost tracking. They are not recorded in lineage metadata.

When resourceLabels is specified multiple times in the config, only the last setting is used. Additionally, when resourceLabels is specified both in the config and the process definition, only the process definition is used.

As a best practice, define all resource labels in a single config setting:

process {
    resourceLabels = [ region: 'some-region', user: 'some-username' ]
}

Use process selectors (withName: or withLabel:) to override resource labels for a specific process.

Executor support​

Resource labels are supported by the following executors:

Consider the limits and syntax of the corresponding executor when using resource labels.

Added in version 23.10

Resource labels in Azure are added to auto-pools, rather than jobs, to support cost analysis. A new pool is created for each new set of resource labels. Setting azure.batch.deletePoolsOnCompletion = true is recommended when using process-specific resource labels.

Automatic resource labels​

Added in version 26.10

Nextflow can derive resource labels from the workflow metadata and attach them to the compute resources for each task, in addition to the labels declared with the resourceLabels directive. Automatic labels are disabled by default. Enable them with the tower.autoLabels config option:

// attach all of the available metadata labels
tower.autoLabels = true
// or select the ones to be attached
tower.autoLabels = ['projectName', 'runName', 'workspaceId']

Available names and their label keys:

NameLabel key
projectNamenextflow.io/projectName
userNamenextflow.io/userName
runNamenextflow.io/runName
sessionIdnextflow.io/sessionId
resumenextflow.io/resume
revisionnextflow.io/revision
commitIdnextflow.io/commitId
repositorynextflow.io/repository
manifestNamenextflow.io/manifestName
runtimeVersionnextflow.io/runtimeVersion
workflowIdseqera.io/platform/workflowId
workspaceIdseqera.io/platform/workspaceId
computeEnvIdseqera.io/platform/computeEnvId

Nextflow omits any label whose metadata value is not available. The seqera.io/platform/* labels are attached only when workflow monitoring with Seqera Platform is enabled.

Labels declared with the resourceLabels directive always take precedence. When a declared label has the same key as an automatic label, Nextflow uses the declared value. The normalization described later never modifies declared labels, even when they are not valid for the target executor.

Label normalization​

The executor normalizes automatic labels to meet the requirements of its API. Declared labels are not normalized.

ExecutorNormalizationnextflow.io/runName becomes
AWS BatchLetters, digits, spaces, and + - = . _ : / @. Key up to 128 characters, value up to 256.unchanged
Azure BatchNone, apart from the reserved microsoft name prefix.unchanged
Google Cloud BatchLowercase letters, digits, _, and -. The key must start with a letter. Key and value up to 63 characters.nextflow_io_runname
KubernetesThe key prefix up to the first / is preserved, and any further / becomes _. Values are stripped of their URL scheme and slashes, up to 63 characters.unchanged
Seqera executorNone.unchanged

The same rules apply to label values. For example, the repository value https://github.com/foo/bar becomes github.com_foo_bar on Kubernetes and https___github_com_foo_bar on Google Cloud Batch.

The two-segment seqera.io/platform/* keys are not valid Kubernetes label keys. The Kubernetes executor applies them as seqera.io/platform_workflowId, seqera.io/platform_workspaceId, and seqera.io/platform_computeEnvId.

warning

On Azure Batch, the executor applies resource labels as metadata of the auto-pool and derives the pool identifier from that metadata. A label that changes on every run, such as runName, sessionId, or workflowId, creates a new pool for each run. The pools accumulate unless you enable azure.batch.deletePoolsOnCompletion. Select a stable subset instead, for example ['projectName', 'workspaceId', 'computeEnvId'].

See also​

  • label (for shared process configuration)
  • tag (for per-task identification)
On this Page