Operator: manager role lacks watch on clusterroles/clusterrolebindings, informers fail with "cannot watch resource" #7061

Expected Behavior

The operator's RBAC covers every API call the operator makes. The controller-manager log has no permission errors.

Current Behavior

When a FeatureStore uses Kubernetes authorization (the default), the operator logs these errors every few seconds, with client-go backoff:

ERROR controller-runtime.cache.UnhandledError Failed to watch {"reflector": "go/pkg/mod/k8s.io/client-go@v0.35.2/tools/cache/reflector.go:289", "type": "*v1.ClusterRole", "error": "clusterroles.rbac.authorization.k8s.io is forbidden: User \"system:serviceaccount:feast-operator-system:feast-operator-controller-manager\" cannot watch resource \"clusterroles\" in API group \"rbac.authorization.k8s.io\" at the cluster scope"}
ERROR controller-runtime.cache.UnhandledError Failed to watch {"reflector": "go/pkg/mod/k8s.io/client-go@v0.35.2/tools/cache/reflector.go:289", "type": "*v1.ClusterRoleBinding", "error": "clusterrolebindings.rbac.authorization.k8s.io is forbidden: User \"system:serviceaccount:feast-operator-system:feast-operator-controller-manager\" cannot watch resource \"clusterrolebindings\" in API group \"rbac.authorization.k8s.io\" at the cluster scope"}

Reconciliation still succeeds, and the FeatureStore becomes Ready. The informers fill from the initial list, but they never receive watch events. As a result, the operator reads ClusterRoles and ClusterRoleBindings from a cache that goes stale, and the log fills with errors (55 in about 17 minutes with one FeatureStore).

Cause

• The authz code reads and writes ClusterRoles and ClusterRoleBindings through the manager client with controllerutil.CreateOrUpdate and Get. See createFeastClusterRole, createFeastClusterRoleBinding, createOidcClusterRole, createOidcClusterRoleBinding and cleanupKubernetesClusterRbac in internal/controller/authz/authz.go.
• cmd/main.go lists many types in Client.Cache.DisableFor, but not rbacv1.ClusterRole or rbacv1.ClusterRoleBinding (L192). Reads of these two types therefore go through the cache, and controller-runtime starts an informer for each type on first use. An informer needs list and watch.
• The RBAC markers grant only get;list;create;update;delete on clusterrolebindings and create;get;list on clusterroles (featurestore_controller.go L74-L76). Neither marker includes watch.

master has the same markers and the same DisableFor list.

Steps to reproduce

1. Install the operator from infra/feast-operator/dist/install.yaml (v0.67.0).
2. Create a FeatureStore without authz (so it defaults to Kubernetes authorization).
3. kubectl -n feast-operator-system logs deploy/feast-operator-controller-manager | grep 'Failed to watch'

Specifications

• Version: Feast operator v0.67.0 (quay.io/feastdev/feast-operator:0.67.0), controller-runtime v0.23.3
• Platform: vanilla Kubernetes v1.35.6 (not OpenShift)
• Subsystem: feast-operator, RBAC / authz

Possible Solution

Either change fixes the errors:

1. Add watch to the two markers:

   // +kubebuilder:rbac:groups=rbac.authorization.k8s.io,resources=clusterrolebindings,verbs=get;list;watch;create;update;delete
   // +kubebuilder:rbac:groups=rbac.authorization.k8s.io,resources=clusterroles,verbs=create;get;list;watch

   The role already grants list, so watch exposes no additional data.
2. Add &rbacv1.ClusterRole{} and &rbacv1.ClusterRoleBinding{} to Client.Cache.DisableFor. Reads of these two types then go directly to the API server, as reads of Role and RoleBinding already do. With this change, no cluster-wide RBAC informers start at all.

As a workaround, we append a watch-only rule to feast-operator-manager-role with a kustomize patch:

- target:
    kind: ClusterRole
    name: feast-operator-manager-role
  patch: |-
    - op: add
      path: /rules/-
      value:
        apiGroups: [rbac.authorization.k8s.io]
        resources: [clusterrolebindings, clusterroles]
        verbs: [watch]

After a restart, the operator logs no Failed to watch errors.