I see, so the only way to write to the cache currently is to specify the image in a step. Using the Docker plugin to build an image does not add those layers to the cache though and so every build needs to run through the full Dockerfile. That’s what I was thinking about.
So I played with buildx for a little and the
--push option seems to solve the problem in the original ticket. Buildx can push an image to a repository without first tagging the image locally. This means namespaced tags can be used locally and non-namespace tags can then be pushed to a remote repository without ever showing up locally, solving the cache poisoning issue.
Here’s an example
RUN echo "hello world"
# docker -v
Docker version 19.03.12, build 48a66213fe
# export DOCKER_CLI_EXPERIMENTAL=enabled
# docker buildx create
# docker buildx use wizardly_hertz
# docker buildx build -t blopker/dronetest --push .
[+] Building 4.6s (6/6) FINISHED
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 87B 0.0s
=> [internal] load metadata for docker.io/library/busybox:latest 1.3s
=> [1/2] FROM docker.io/library/busybox@sha256:9ddee63a712cea977267342e8750ecbc60d3aab25f04ceacfa795e6fce341793 0.0s
=> => resolve docker.io/library/busybox@sha256:9ddee63a712cea977267342e8750ecbc60d3aab25f04ceacfa795e6fce341793 0.0s
=> CACHED [2/2] RUN echo "hello world" 0.0s
=> exporting to image 3.4s
=> => exporting layers 0.0s
=> => exporting manifest sha256:9ce7dfad6a28040a36e1b0d2b03b6e03979642eebfc3db6fbd661e8647b15014 0.0s
=> => exporting config sha256:b166d61b401655375e40343e5d636d0ada1ee5e74079eb56d66b01b56399d295 0.0s
=> => pushing layers 2.5s
=> => pushing manifest for docker.io/blopker/dronetest:latest
# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
moby/buildkit buildx-stable-1 f2a88cb62c92 3 months ago 82.8MB
Notice that a) the build used cached layers since I had already built the image before and b) the tag
blopker/dronetest does not show up when
docker images is run. It seems like using it this way we don’t even need separate contexts for each pipeline, but just one global one.