race condition in quadlet pod

[Xyz00777]

Hi, im not sure how to solve my problem.
I am using quadlet to manage a pod with 5 containers inside of it.
and one container have a dependency to start after a database.
I realized the dependency with systemd requires and systemd after, but now i have the problem, that systemd wants to start the webserver container inside the pod before the db container and it says there is no service with the name of the db container. if i restart the start service for the webserver a few moments later manual it knows the service and can start without any problem.

How can i solve this?

[ygalblum]

Not sure I understood all the details. Can you share your Quadlet files, or something to reproduce your issue.

Also, if the DB is a dependency, why not run it outside of the pod?

[ygalblum]

I think that the way to achieve that is by adding dependencies between the container unit the the service units of the other containers.
You can see this as an example: https://github.com/containers/appstore/blob/main/quadlet/minio-prometheus/minio.container
The minio container unit sets Requires=prometheus.service and prometheus.service is generated by the prometheus.container unit.

Come to think about it, maybe there's an RFE here - supporting a dependency key that will translate the service name for the user. @Luap99 WDYT?

[Luap99]

Automatically translating service names .container, .kube, .network, etc... in dependencies sounds reasonable.
I don't think we would need a new key, IMO our suffixes should be unique enough so we can auto translate After=,Before=,Requires= and whatever other keys systemd uses. That to me at least feel the most natural.

[Xyz00777]

I don't have it running and for some reason the generate_systemd field does not populate the values for me. I tried updating the collection, but still nothing.

hmm interesting i can confirm that there are no entrys for requred and after in my .container files too, but as you can see in the systemlog it was included...

the quadlet who is getting created from ansible looks like that:

[Container]
ContainerName=paperless-webserver
Environment=PAPERLESS_REDIS=redis://paperless-redis:6379
Environment=PAPERLESS_DBHOST=paperless-postgresql
Environment=PAPERLESS_TIKA_ENABLED=1
Environment=PAPERLESS_TIKA_GOTENBERG_ENDPOINT=http://paperless-gotenberg:3000
Environment=PAPERLESS_TIKA_ENDPOINT=http://paperless-tika:9998
Environment=PAPERLESS_OCR_LANGUAGE=deu+eng
Environment=PAPERLESS_URL=https://paperless.XXXX.XXX
Environment=PAPERLESS_SECRET_KEY=XXXXXX
Environment=PAPERLESS_CONSUMER_RECURSIVE=true
Environment=PAPERLESS_OCR_DESKEW=True
Environment=PAPERLESS_OCR_CLEAN=clean-final
Environment=PAPERLESS_TIME_ZONE=Europe/Berlin
Environment=USERMAP_UID=1000
Environment=USERMAP_GID=1000
Group=1000
Image=ghcr.io/paperless-ngx/paperless-ngx:latest
Pod=paperless.pod
User=1000
UserNS=keep-id
Volume=/opt/paperlessngx/data:/usr/src/paperless/data
Volume=/opt/paperlessngx/media:/usr/src/paperless/media
Volume=/opt/paperlessngx/export:/usr/src/paperless/export
Volume=/opt/paperlessngx/consume:/usr/src/paperless/consume
PodmanArgs=--requires paperless-tika,paperless-gotenberg,paperless-redis,paperless-postgresql
AutoUpdate=registry
[Install]
WantedBy=multi-user.target

[ygalblum]

It seems that there's an issue with the Ansible collection (or we're missing something in understanding the doc). Let's put that aside for now and focus on the Quadlet part.
I think what you should do is remove the line that adds the --requires flag:

PodmanArgs=--requires paperless-tika,paperless-gotenberg,paperless-redis,paperless-postgresql

And instead add the service dependencies:

[Unit]
Description=A minio server container for local S3 compatible storage
After=paperless-tika.service paperless-gotenberg.service paperless-redis.service paperless-postgresql.service
Requires=paperless-tika.service paperless-gotenberg.service paperless-redis.service paperless-postgresql.service

Let's see if this solves the issue

[Xyz00777]

i wasnt able to test that beside of not being sure how to implement it :)
i think most likely with "quadlet_options:" like the install part

[Xyz00777]

So, sorry for that delay. Now i had time to test it.
Yes when I delete the one line and replace it with the correct [UNIT] definition it works. i tested it now 2 times with restarts of the vm and it works. Thank you very mutch for your help!
I think to open a issue in the ansible community repo would be now the correct way so they change the generate_systemd after and want key to move them into the [UNIT] part and not as part of container args. Did i understood the correct?

[ygalblum]

Happy to hear to works for you.
From what I can tell, the ansible collection is disregarding generate_systemd and does not translate the lines into the [Unit] section. Once this works, you won't need to set the requires field at all.

[Xyz00777]

okay than i will look into it if im able to make a pr or at least a issue, thank you very mutch :)

[ygalblum]

I've opened a PR for dependency translation. It won't solve this issue for now. But, it will make it simpler to add such dependencies.

#25899

[Xyz00777]

understandable, no problem, at least we found out (hopefully) what it was and it can be fixed for the future :)
Thank you very mutch for all the support and trying to understand the problem and helping me <3

[Xyz00777]

so do i understand you correct, that when this pr is merged we still need another pr to fixing this issue? im confused atm

[ygalblum]

From what I can tell, there are two issues here.

1. The Ansible module seems to disregard the generate_systemd key.
2. We need to understand how to achieve what you need assuming it did work. To workaround this issue, I suggested that you manually edit the generated Quadlet files and add the dependencies yourself and then test it.

The PR that I opened removes the need to predict the name of the generated service and let Quadlet translate it for you. But, using it (once merged and released) still requires us to understand how to define these dependencies.