Allow to push finalized layers to the registry during build

[StarpTech]

Feature request description

Hi, building an image is a synchronous task and network is slow. I'm curious if it would be possible to build and push layers to the registry simultaneously. If possible, I think this can save a lot of time because while layer 2 is built, layer 1 can be pushed already to the registry.

Suggest potential solution

podman build --layers --push --format v2s2 -t gcr.io/myrepo/myimage

[mtrmac]

This is interesting but also not something we are currently set up to do. c/image only provides an API to copy (“push”) completed images, not layers of unfinished images. And pushing a complete image to a registry requires choosing a tag.

It would probably be possible to hack something together (choose a random long tag, hope that there is no collision; delete the tag afterwards), but that seems fairly messy. Pushing just the layers as they are built would need to be a new API in c/image/copy, only for transports that opt in (like docker:// could).

[StarpTech]

Naively, it can be implemented very similarly to how --cache-from and --cache-to is working with the important difference that constructing and pushing the intermediate layers isn't a blocking operation.

I experienced very slow builds when using podman distributed cache feature. Although layers are cached Using cache .. the layer was pushed again to the registry. In the end, it took more time or similar time to build the image without the cache but this is a story for another issue.

[mtrmac]

Yeah, --cache-to is adding a tag to the repo, for each image pushed. So that would be user-visible.

[StarpTech]

How hard would it be to implement this, as described in #18642 (comment) ? Is there public interest? If someone assists me, I can try to prepare a PR. I think the work has to be done in buildah.

Ref:

• build,cache: support remote cache sources ( via --cache-to, --cache-from ) and introduce cacheKey/layerKey buildah#4118

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@flouthoc could you help @StarpTech