Skip to content

Commit 61d4818

Browse files
scme0scme0
committed
Merge branch 'feat/workflows-codefresh-saas-s3' into chore/merge-logs-and-gitops-operator-bump
# Conflicts: # charts/gitops-runtime/README.md
2 parents eb019a8 + 9f3b43f commit 61d4818

File tree

6 files changed

+71
-51
lines changed

6 files changed

+71
-51
lines changed

‎charts/gitops-runtime/Chart.yaml

-5
Original file line numberDiff line numberDiff line change
@@ -42,11 +42,6 @@ dependencies:
4242
version: 0.7.1
4343
alias: gitops-operator
4444
condition: gitops-operator.enabled
45-
- name: garage
46-
repository: https://codefresh-io.github.io/garage
47-
alias: garage-workflows-artifact-storage
48-
version: 0.5.0-cf.3
49-
condition: garage-workflows-artifact-storage.enabled
5045
- name: cf-argocd-extras
5146
repository: oci://quay.io/codefresh/charts
5247
version: 0.3.21

‎charts/gitops-runtime/README.md

+24-13
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,18 @@ See [Use OCI-based registries](https://helm.sh/docs/topics/registries/)
1616
## Codefresh official documentation:
1717
Prior to running the installation please see the official documentation at: https://codefresh.io/docs/docs/installation/gitops/hybrid-gitops-helm-installation/
1818

19+
## Argo-workflows artifact and log storage
20+
> [!NOTE]
21+
> This version of the chart includes default configuration for storing workflow artifacts and logs in Codefresh provided s3 compatible storage.
22+
23+
If you have your own storage configuration using the default configmap `artifact-repositories` upgrading the chart will override your artifact storage configuration.
24+
To prevent this please set `argo-workflows.controller.workflowDefaults.spec.artifactRepository.configMap` to `artifact-repositories` and `argo-workflows.controller.workflowDefaults.spec.artifactRepository.key`
25+
to the respective key in your configmap identifying the repository.
26+
> [!WARNING]
27+
> It's highly recommended to use your own artifact storage for data privacy reasons.
28+
> Codefresh provided storage has a retention policy of 14 days and limitations on uploaded file sizes.
29+
> Please refer to the official documentation for more details.
30+
1931
## Installation with External ArgoCD
2032

2133
If you want to use an existing ArgoCD installation, you can disable the built-in ArgoCD and configure the GitOps Runtime to use the external ArgoCD.
@@ -259,15 +271,19 @@ sealed-secrets:
259271
| argo-rollouts.enabled | bool | `true` | |
260272
| argo-rollouts.fullnameOverride | string | `"argo-rollouts"` | |
261273
| argo-rollouts.installCRDs | bool | `true` | |
274+
| argo-workflows.codefreshWorkflowLogs | object | `{"endpoint":"gitops-workflow-logs.codefresh.io","insecure":false}` | Argo workflows logs storage on Codefresh platform settings. Don't change unless instructed by Codefresh support. |
275+
| argo-workflows.controller.workflowDefaults.spec.archiveLogs | bool | `true` | |
276+
| argo-workflows.controller.workflowDefaults.spec.artifactRepositoryRef | object | `{"configMap":"codefresh-workflows-log-store","key":"codefresh-workflows-log-store"}` | By default artifact repository is set to a Codefresh provided repository. For data privacy it is reccommended to set your own artifact repository. For instructions see: https://argo-workflows.readthedocs.io/en/latest/configure-artifact-repository/#configuring-your-artifact-repository |
262277
| argo-workflows.crds.install | bool | `true` | Install and upgrade CRDs |
263278
| argo-workflows.enabled | bool | `true` | |
264279
| argo-workflows.executor.resources.requests.ephemeral-storage | string | `"10Mi"` | |
265280
| argo-workflows.fullnameOverride | string | `"argo"` | |
266281
| argo-workflows.mainContainer.resources.requests.ephemeral-storage | string | `"10Mi"` | |
267282
| argo-workflows.server.authModes | list | `["client"]` | auth-mode needs to be set to client to be able to see workflow logs from Codefresh UI |
268283
| argo-workflows.server.baseHref | string | `"/workflows/"` | Do not change. Workflows UI is only accessed through internal router, changing this values will break routing to workflows native UI from Codefresh. |
269-
| cf-argocd-extras | object | `{"libraryMode":true}` | Codefresh extra services for ArgoCD |
284+
| cf-argocd-extras | object | `{"eventReporter":{"affinity":{},"enabled":true,"nodeSelector":{},"tolerations":[]},"libraryMode":true,"sourcesServer":{"affinity":{},"enabled":true,"nodeSelector":{},"tolerations":[]}}` | Codefresh extra services for ArgoCD |
270285
| cf-argocd-extras.libraryMode | bool | `true` | Library mode for the chart. Allows to inject values from gitops runtime chart |
286+
| cf-argocd-extras.sourcesServer | object | `{"affinity":{},"enabled":true,"nodeSelector":{},"tolerations":[]}` | Sources server configuration |
271287
| event-reporters.rollout.eventSource.affinity | object | `{}` | |
272288
| event-reporters.rollout.eventSource.nodeSelector | object | `{}` | |
273289
| event-reporters.rollout.eventSource.replicas | int | `1` | |
@@ -304,14 +320,6 @@ sealed-secrets:
304320
| event-reporters.workflow.sensor.retryStrategy.steps | int | `3` | Number of retries |
305321
| event-reporters.workflow.sensor.tolerations | list | `[]` | |
306322
| event-reporters.workflow.serviceAccount.create | bool | `true` | |
307-
| garage-workflows-artifact-storage | object | `{"deployment":{"kind":"StatefulSet","replicaCount":3},"enabled":false,"fullnameOverride":"garage","garage":{"replicationMode":3},"persistence":{"data":{"size":"100Mi","storageClass":""},"enabled":true,"meta":{"size":"100Mi","storageClass":""}},"resources":{},"tests":{"enabled":false}}` | Builtin Workflows artifacts storage solution. Local S3 backed by local persistence with (PV and PVC) |
308-
| garage-workflows-artifact-storage.deployment.kind | string | `"StatefulSet"` | Only statefulset is supported for Codefresh gitops runtime. Do not change this |
309-
| garage-workflows-artifact-storage.persistence.data | object | `{"size":"100Mi","storageClass":""}` | Volume that stores artifacts and logs for workflows |
310-
| garage-workflows-artifact-storage.persistence.data.storageClass | string | `""` | When empty value empty the default storage class for the cluster will be used |
311-
| garage-workflows-artifact-storage.persistence.meta | object | `{"size":"100Mi","storageClass":""}` | Volume that stores cluster metadata |
312-
| garage-workflows-artifact-storage.persistence.meta.storageClass | string | `""` | When empty value empty the default storage class for the cluster will be used |
313-
| garage-workflows-artifact-storage.resources | object | `{}` | Resources for garage pods. For smaller deployments at least 100m CPU and 1024Mi memory is reccommended. For larger deployments double this size. |
314-
| garage-workflows-artifact-storage.tests | object | `{"enabled":false}` | Helm tests |
315323
| gitops-operator.affinity | object | `{}` | |
316324
| gitops-operator.crds | object | `{"additionalLabels":{},"annotations":{},"install":true,"keep":false}` | Codefresh gitops operator crds |
317325
| gitops-operator.crds.additionalLabels | object | `{}` | Additional labels for gitops operator CRDs |
@@ -369,7 +377,8 @@ sealed-secrets:
369377
| global.external-argo-rollouts | object | `{"rollout-reporter":{"enabled":false}}` | Configuration for external Argo Rollouts |
370378
| global.external-argo-rollouts.rollout-reporter | object | `{"enabled":false}` | Rollout reporter settings |
371379
| global.external-argo-rollouts.rollout-reporter.enabled | bool | `false` | Enable or disable rollout reporter Configuration is defined at .Values.event-reporters.rollout |
372-
| global.runtime | object | `{"cluster":"https://kubernetes.default.svc","codefreshHosted":false,"eventBus":{"annotations":{},"name":"codefresh-eventbus","nats":{"native":{"auth":"token","containerTemplate":{"resources":{"limits":{"cpu":"500m","ephemeral-storage":"2Gi","memory":"4Gi"},"requests":{"cpu":"200m","ephemeral-storage":"2Gi","memory":"1Gi"}}},"maxPayload":"4MB","replicas":3}},"pdb":{"enabled":true,"minAvailable":2}},"gitCredentials":{"password":{"secretKeyRef":{},"value":null},"username":"username"},"ingress":{"annotations":{},"className":"nginx","enabled":false,"hosts":[],"protocol":"https","skipValidation":false,"tls":[]},"ingressUrl":"","isConfigurationRuntime":false,"name":null}` | Runtime level settings |
380+
| global.nodeSelector | object | `{}` | Global nodeSelector for all components |
381+
| global.runtime | object | `{"cluster":"https://kubernetes.default.svc","codefreshHosted":false,"eventBus":{"annotations":{},"name":"codefresh-eventbus","nats":{"native":{"affinity":{},"auth":"token","containerTemplate":{"resources":{"limits":{"cpu":"500m","ephemeral-storage":"2Gi","memory":"4Gi"},"requests":{"cpu":"200m","ephemeral-storage":"2Gi","memory":"1Gi"}}},"maxPayload":"4MB","nodeSelector":{},"replicas":3,"tolerations":[]}},"pdb":{"enabled":true,"minAvailable":2}},"gitCredentials":{"password":{"secretKeyRef":{},"value":null},"username":"username"},"ingress":{"annotations":{},"className":"nginx","enabled":false,"hosts":[],"labels":{},"protocol":"https","skipValidation":false,"tls":[]},"ingressUrl":"","isConfigurationRuntime":false,"name":null}` | Runtime level settings |
373382
| global.runtime.cluster | string | `"https://kubernetes.default.svc"` | Runtime cluster. Should not be changed. |
374383
| global.runtime.codefreshHosted | bool | `false` | Defines whether this is a Codefresh hosted runtime. Should not be changed. |
375384
| global.runtime.eventBus.annotations | object | `{}` | Annotations on EventBus resource |
@@ -381,15 +390,17 @@ sealed-secrets:
381390
| global.runtime.gitCredentials.password.secretKeyRef | object | `{}` | secretKeyReference for Git credentials password. Provide name and key fields. |
382391
| global.runtime.gitCredentials.password.value | string | `nil` | Plain text password |
383392
| global.runtime.gitCredentials.username | string | `"username"` | Username. Optional when using token in password. |
384-
| global.runtime.ingress | object | `{"annotations":{},"className":"nginx","enabled":false,"hosts":[],"protocol":"https","skipValidation":false,"tls":[]}` | Ingress settings |
393+
| global.runtime.ingress | object | `{"annotations":{},"className":"nginx","enabled":false,"hosts":[],"labels":{},"protocol":"https","skipValidation":false,"tls":[]}` | Ingress settings |
385394
| global.runtime.ingress.enabled | bool | `false` | Defines if ingress-based access mode is enabled for runtime. To use tunnel-based (ingressless) access mode, set to false. |
386395
| global.runtime.ingress.hosts | list | `[]` | Hosts for runtime ingress. Note that Codefresh platform will always use the first host in the list to access the runtime. |
387396
| global.runtime.ingress.protocol | string | `"https"` | The protocol that Codefresh platform will use to access the runtime ingress. Can be http or https. |
388397
| global.runtime.ingress.skipValidation | bool | `false` | if set to true, the pre-install hook will validate the existance of appropriate values, but *will not* attempt to make a web request to the ingress host |
389398
| global.runtime.ingressUrl | string | `""` | Explicit url for runtime ingress. Provide this value only if you don't want the chart to create and ingress (global.runtime.ingress.enabled=false) and tunnel-client is not used (tunnel-client.enabled=false) |
390399
| global.runtime.isConfigurationRuntime | bool | `false` | is the runtime set as a "configuration runtime". |
391400
| global.runtime.name | string | `nil` | Runtime name. Must be unique per platform account. |
392-
| installer | object | `{"argoCdVersionCheck":{"argoServerLabels":{"app.kubernetes.io/component":"server","app.kubernetes.io/part-of":"argocd"}},"image":{"pullPolicy":"IfNotPresent","repository":"quay.io/codefresh/gitops-runtime-installer","tag":""},"skipValidation":false}` | Runtime installer used for running hooks and checks on the release |
401+
| global.tolerations | list | `[]` | Global tolerations for all components |
402+
| installer | object | `{"affinity":{},"argoCdVersionCheck":{"argoServerLabels":{"app.kubernetes.io/component":"server","app.kubernetes.io/part-of":"argocd"}},"image":{"pullPolicy":"IfNotPresent","repository":"quay.io/codefresh/gitops-runtime-installer","tag":""},"nodeSelector":{},"skipUsageValidation":false,"skipValidation":false,"tolerations":[]}` | Runtime installer used for running hooks and checks on the release |
403+
| installer.skipUsageValidation | bool | `false` | if set to true, pre-install hook will *not* run |
393404
| installer.skipValidation | bool | `false` | if set to true, pre-install hook will *not* run |
394405
| internal-router.affinity | object | `{}` | |
395406
| internal-router.clusterDomain | string | `"cluster.local"` | |
@@ -424,6 +435,6 @@ sealed-secrets:
424435
| internal-router.serviceAccount.name | string | `""` | |
425436
| internal-router.tolerations | list | `[]` | |
426437
| sealed-secrets | object | `{"fullnameOverride":"sealed-secrets-controller","image":{"registry":"quay.io","repository":"codefresh/sealed-secrets-controller","tag":"0.29.0"},"keyrenewperiod":"720h","resources":{"limits":{"cpu":"500m","memory":"1Gi"},"requests":{"cpu":"200m","memory":"512Mi"}}}` | --------------------------------------------------------------------------------------------------------------------- |
427-
| tunnel-client | object | `{"enabled":true,"libraryMode":true,"tunnelServer":{"host":"register-tunnels.cf-cd.com","subdomainHost":"tunnels.cf-cd.com"}}` | Tunnel based runtime. Not supported for on-prem platform. In on-prem use ingress based runtimes. |
438+
| tunnel-client | object | `{"affinity":{},"enabled":true,"libraryMode":true,"nodeSelector":{},"tolerations":[],"tunnelServer":{"host":"register-tunnels.cf-cd.com","subdomainHost":"tunnels.cf-cd.com"}}` | Tunnel based runtime. Not supported for on-prem platform. In on-prem use ingress based runtimes. |
428439
| tunnel-client.enabled | bool | `true` | Will only be used if global.runtime.ingress.enabled = false |
429440
| tunnel-client.libraryMode | bool | `true` | Do not change this value! Breaks chart logic |

‎charts/gitops-runtime/README.md.gotmpl

+13
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,19 @@ See [Use OCI-based registries](https://helm.sh/docs/topics/registries/)
1616
## Codefresh official documentation:
1717
Prior to running the installation please see the official documentation at: https://codefresh.io/docs/docs/installation/gitops/hybrid-gitops-helm-installation/
1818

19+
## Argo-workflows artifact and log storage
20+
> [!NOTE]
21+
> This version of the chart includes default configuration for storing workflow artifacts and logs in Codefresh provided s3 compatible storage.
22+
23+
If you have your own storage configuration using the default configmap `artifact-repositories` upgrading the chart will override your artifact storage configuration.
24+
To prevent this please set `argo-workflows.controller.workflowDefaults.spec.artifactRepository.configMap` to `artifact-repositories` and `argo-workflows.controller.workflowDefaults.spec.artifactRepository.key`
25+
to the respective key in your configmap identifying the repository.
26+
> [!WARNING]
27+
> It's highly recommended to use your own artifact storage for data privacy reasons.
28+
> Codefresh provided storage has a retention policy of 14 days and limitations on uploaded file sizes.
29+
> Please refer to the official documentation for more details.
30+
31+
1932
## Installation with External ArgoCD
2033

2134
If you want to use an existing ArgoCD installation, you can disable the built-in ArgoCD and configure the GitOps Runtime to use the external ArgoCD.

‎charts/gitops-runtime/templates/codefresh-workflow-log-store.yaml

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
{{- $_ := required "global.codefresh.accountId is required" .Values.global.codefresh.accountId }}
2+
apiVersion: v1
3+
data:
4+
codefresh-workflows-log-store: |
5+
archiveLogs: true
6+
s3:
7+
bucket: {{ .Values.global.codefresh.accountId }}
8+
endpoint: {{ index .Values "argo-workflows" "codefreshWorkflowLogs" "endpoint" }}
9+
insecure: {{ index .Values "argo-workflows" "codefreshWorkflowLogs" "insecure" }}
10+
keyFormat: {{ .Values.global.runtime.name }}/{{ "{{" }}workflow.name{{ "}}" }}/{{ "{{" }}pod.name{{ "}}" }}
11+
accessKeySecret:
12+
name: codefresh-token
13+
key: token
14+
secretKeySecret:
15+
name: codefresh-token
16+
key: token
17+
kind: ConfigMap
18+
metadata:
19+
annotations:
20+
workflows.argoproj.io/default-artifact-repository: codefresh-workflows-log-store
21+
name: codefresh-workflows-log-store

‎charts/gitops-runtime/templates/tunnel-client.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -10,4 +10,4 @@ reduce complexity of installation and number or mandatory values to provide for
1010
{{ $runtimeName := required "runtime.name is required" .Values.global.runtime.name }}
1111
{{ $_ := set $tunnelClientContext.Values.tunnel "subdomainPrefix" (printf "%s-%s" $accoundId $runtimeName)}}
1212
{{- include "codefresh-tunnel-client.resources" $tunnelClientContext }}
13-
{{- end }}
13+
{{- end }}

‎charts/gitops-runtime/values.yaml

+12-32
Original file line numberDiff line numberDiff line change
@@ -315,6 +315,18 @@ argo-workflows:
315315
resources:
316316
requests:
317317
ephemeral-storage: 10Mi
318+
controller:
319+
workflowDefaults:
320+
spec:
321+
archiveLogs: true
322+
# -- By default artifact repository is set to a Codefresh provided repository. For data privacy it is reccommended to set your own artifact repository. For instructions see: https://argo-workflows.readthedocs.io/en/latest/configure-artifact-repository/#configuring-your-artifact-repository
323+
artifactRepositoryRef:
324+
configMap: codefresh-workflows-log-store
325+
key: codefresh-workflows-log-store
326+
# -- Argo workflows logs storage on Codefresh platform settings. Don't change unless instructed by Codefresh support.
327+
codefreshWorkflowLogs:
328+
endpoint: gitops-workflow-logs.codefresh.io
329+
insecure: false
318330

319331
#-----------------------------------------------------------------------------------------------------------------------
320332
# Argo rollouts
@@ -706,38 +718,6 @@ gitops-operator:
706718
cpu: 100m
707719
memory: 128Mi
708720
#-----------------------------------------------------------------------------------------------------------------------
709-
# Garage
710-
#-----------------------------------------------------------------------------------------------------------------------
711-
# -- Builtin Workflows artifacts storage solution. Local S3 backed by local persistence with (PV and PVC)
712-
garage-workflows-artifact-storage:
713-
fullnameOverride: garage
714-
enabled: false
715-
deployment:
716-
# -- Only statefulset is supported for Codefresh gitops runtime. Do not change this
717-
kind: StatefulSet
718-
replicaCount: 3
719-
garage:
720-
#-- Default to 3 replicas, see the replication_mode section at https://garagehq.deuxfleurs.fr/documentation/reference-manual/configuration/#replication-mode
721-
replicationMode: 3
722-
persistence:
723-
enabled: true
724-
# -- Volume that stores cluster metadata
725-
meta:
726-
# -- When empty value empty the default storage class for the cluster will be used
727-
storageClass: ""
728-
size: 100Mi
729-
# -- Volume that stores artifacts and logs for workflows
730-
data:
731-
# -- When empty value empty the default storage class for the cluster will be used
732-
storageClass: ""
733-
size: 100Mi
734-
# -- Resources for garage pods. For smaller deployments at least 100m CPU and 1024Mi memory is reccommended. For larger deployments double this size.
735-
resources: {}
736-
# -- Helm tests
737-
tests:
738-
enabled: false
739-
740-
#-----------------------------------------------------------------------------------------------------------------------
741721
# cf-argocd-extras
742722
#-----------------------------------------------------------------------------------------------------------------------
743723
# -- Codefresh extra services for ArgoCD

0 commit comments

Comments
 (0)