Drone

[Feature request] Support for interpolation in nodeSelectors

We’d like to use dynamic node pool provisioning, but to do so, we need to be able to interpolate $DRONE_COMMIT into nodeSelector. This currently isn’t possible.

Not sure I entirely follow, but you can interpolate drone variables in any section of the yaml. It is a pure string find / replace.

node:
  commit: ${DRONE_COMMIT}

When doing this, the server rejects the interpolation since it doesn’t match a regex, a regex which isn’t matched because it appears the commit isn’t expanded. I’ll dig further for logs to confirm but using node.group: “drone-${DRONE_COMMIT}” not only failed but didn’t propagate the error up to the UI, causing a silent fail.

You can find the relevant source code here which might help you debug further. I do not personally use kubernetes so I’m not going to be much help.

[drone-server-7cd4994d7f-j66tl] {"commit":"*****","error":"Job.batch \"drone-job-68-whrzoufutqjpxqrw2\" is invalid: spec.template.spec.nodeSelector: Invalid value: \"drone-${DRONE_BUILD_EVENT}-${DRONE_BUILD_NUMBER}\": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue', or 'my_value', or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')","event":"pull_request","level":"error","msg":"trigger: cannot enqueue build","ref":"refs/pull/*****/head","repo":"*****","time":"2019-06-12T00:55:21Z"}

Looks like it isn’t being interpolated, I’ve dug around drone-yaml/runtime quite a bit with no explanation. The pertinent yaml block goes
node.group: drone-${DRONE_BUILD_EVENT}-${DRONE_BUILD_NUMBER}

node.group: <value> is not a valid yaml syntax, although perhaps I am misunderstanding. I therefore recommend, if reporting issues with yaml or interpolation, that you post your full yaml configuration file.

edit I should also point out that the native kubernetes runtime is experimental and is not recommended for production (per the docs). So you may find yourself in a tight spot if you run into kubernetes-related issues right now since it is not our immediate focus. Just a polite heads up.

According to pipeline.go, the Node field of a Pipeline is a map of strings which is usable in this fashion. Like Kubernetes nodeSelectors (and potentially the Agent’s implementation), the left and right and side of the assignment is effectively a freeform pair of strings. It looks like interpolations aren’t happening to map types, it applies to the key and not the value, or a flag on that field is missing. To be clear, the group part of node.group is the key of the node pool keypair, where the interpolated string is the value.

you are right, there are a few values that are parsed and stored in the database (by the server) before the yaml is interpolated (by the agent or controller) which happens way later in the process. These are the following fields that are read prior to interpolation:

kind: <value>
name: <value>

platform:
  os: <value>
  arch: <value>

node:
  <key>: <value>

Would it be possible to have interpolation of these values supported? While I know native Kubernetes support is experimental, that interpolation provides node pool auto-provisioning for free, an important feature for our use case on GKE.

there is a bit of a chicken and egg issue that we face. The yaml is parsed before the build object has been created (which is not something that can be changed) which means certain variables will be empty, like build number.

Also, if we interpolate the yaml without expected values it can create an invalid yaml file. If we try to interpolate this yaml before we have a build number:

tag: [ ${DRONE_BUILD_NUMBER}, 'latest'

the result would be an invalid yaml that cannot be parsed:

tag: [ , 'latest' ]

one option you might consider is a custom yaml plugin, which can be used to generate a custom yaml on the fly. Not sure if this could work for you, but it is a new feature that is described here https://github.com/drone/drone/issues/1021#issuecomment-417819643

other options may be using something like Starlark instead of yaml, which is planned for a future release and allows more dynamic configuration. Build number would not be available, but some other fields like commit sha would. https://github.com/drone/drone/issues/2585