ArgoCD Gotchas: Cache Staleness and the SharedResourceWarning Nobody Explains

kubectl apply reports success. You check the resource and the field you just changed is back to its old value. No error, no event. kubectl get shows the change applied, then a few seconds later it’s gone. I hit this on a k3s cluster on a BMAX Mini PC* running ArgoCD with selfHeal: true. It’s small enough to miss in logs and annoying enough to waste an afternoon.

The cause is ArgoCD’s selfHeal doing what it’s designed to do: re-applying its own rendered idea of what the resource should be. That rendered state comes from a cache, and the cache can lag behind a fresh git push.

This hit the same homelab three times in one day, across three unrelated resources. Below are the pattern, the fix, and a second gotcha that looks different but has a related cause.

View the complete homelab infrastructure source on GitHub 🐙

The Symptom

Three separate incidents, same shape:

• A Tempo PersistentVolumeClaim’s storageClassName kept reverting after being changed.
• Traefik’s tlsStore and dashboard configuration reverted after a Helm values update.
• A paperless-gpt deployment’s volumeMounts reverted after a direct edit.

Each time, the sequence was: edit the live resource or push a change to Git → confirm the change is live → come back later → the old value is back, with no error logged anywhere obvious.

Why This Happens: selfHeal Plus a Stale Cache

ArgoCD’s selfHeal: true continuously reconciles the live cluster state against ArgoCD’s rendered understanding of what the Application’s manifests or Helm chart should produce. That’s the point of GitOps: drift gets corrected automatically, so a manual kubectl edit never silently becomes the new permanent state. A hand edit that isn’t in Git gets reverted by design, cache or no cache.

The surprising case is when the change is in Git. The rendered understanding ArgoCD reconciles against comes from the argocd-repo-server’s manifest and Helm chart cache, and that cache doesn’t always get invalidated promptly after a fresh git push. For a window of time, usually short but long enough to confuse you, ArgoCD’s idea of “what this should look like” is stale, and selfHeal faithfully reverts your change to match it.

From the outside this looks exactly like “ArgoCD is ignoring my change”. What actually happens is that ArgoCD enforces an outdated cached version of what it thinks you want.

The Fix: Force a Hard Refresh

kubectl patch application <name> -n argocd --type merge \
  -p '{"metadata":{"annotations":{"argocd.argoproj.io/refresh":"hard"}}}'

The hard refresh value (as opposed to normal) tells ArgoCD to bypass the repo-server’s manifest cache and re-render from source. Wait roughly 15 seconds, then re-check.

If that alone doesn’t resolve it, the cache itself may need restarting, not just invalidating for one Application:

kubectl rollout restart deployment argocd-repo-server -n argocd

This is a bigger hammer. It affects every Application’s next reconciliation, not only the one you’re debugging, so try the targeted hard refresh annotation first.

The StatefulSet Exception

For the Tempo PVC, neither of the above resolved it on the first try, because volumeClaimTemplates on a StatefulSet are immutable: Kubernetes rejects any attempt to change them on an existing object. Clearing ArgoCD’s stale cache fixes ArgoCD’s intent going forward, but it can’t retroactively fix a field that was never mutable on the live object in the first place.

The fix there is to delete and recreate the StatefulSet itself (the underlying PVC and its data survive deleting the StatefulSet, as long as you don’t also delete the PVC):

kubectl delete statefulset <name> -n <namespace> --cascade=orphan
# re-sync from ArgoCD to recreate the StatefulSet with the new template

--cascade=orphan deletes the StatefulSet object without deleting the Pods or PVCs it owns. ArgoCD’s next sync recreates the StatefulSet with the corrected template, and it re-adopts the existing PVC.

That fixes the template, not the volume. The re-adopted PVC keeps its old storageClassName, because that field can’t be changed on an existing PVC either. To actually move the data to the new class, copy it to a new PVC and delete the old one; only PVCs created after the change pick up the new template.

A Second, Different-Looking Bug With a Related Cause: SharedResourceWarning

A related but distinct symptom: a resource flickers between two different specs, or gets pruned entirely, and .status.conditions on one of the Applications shows a SharedResourceWarning.

This is an ownership conflict, not a cache problem. Two ArgoCD Applications both manage a resource with the same name and namespace. In my case, the Traefik Helm chart’s ingressRoute.dashboard.enabled flag created a dashboard IngressRoute, while a hand-written IngressRoute with the same name lived in a different Application’s manifests. Both claimed the same object.

ArgoCD records ownership with a tracking label or annotation on the live object. When two Applications render the same object, each sync rewrites that marker, and the Application that doesn’t currently own it reports SharedResourceWarning. ArgoCD has no way to know which definition is “correct”, so it flags the conflict instead of guessing.

The fix is to pick exactly one owner and have the other stop claiming the resource:

# kubernetes/system/traefik/application.yml - Helm chart's own dashboard route, disabled
helm:
  values: |
    ingressRoute:
      dashboard:
        enabled: false  # the manual, Authelia-protected route below is canonical

# kubernetes/system/other-ingressroute.yml - the manually-defined route, kept
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: traefik-dashboard
  namespace: traefik
spec:
  # ... Authelia-protected route - this is the one that stays

Once only one Application defines the object, delete the live resource and let the remaining owner’s next sync recreate it with its own tracking marker. The warning clears.

Telling the Two Apart

The diagnostic tell: cache staleness is temporal. The same Application reverts a change made moments ago, and a refresh fixes it. An ownership conflict is structural. Check .status.conditions for SharedResourceWarning first. If it’s there, refreshing the cache won’t help: nothing is stale, both Applications render their own manifests correctly, and the manifests themselves conflict.

The cache-staleness pattern is specific to ArgoCD’s repo-server architecture. The ownership conflict applies to any GitOps tool: Flux has the same failure mode when two Kustomizations or HelmReleases define a resource with the same identity. Checking .status.conditions before assuming a sync or cache problem saves a lot of time chasing the wrong fix.