-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Description
What would you like to be added:
A new CLI flag to support per-resource label selectors, for example:
--label-selectors=nodes=<labelSelector>,pods=<labelSelector>
This would allow specifying Kubernetes label selectors per resource type.
Only the specified resource types would be filtered; others remain cluster-wide.
Example:
--label-selectors=nodes=tenant=team-a,pods=app=frontend
Why is this needed:
KSM currently performs cluster-wide List/Watch operations for most resources.
In some environments, restricting the observed resource set is desirable:
- Running multiple KSM instances for sharding
- Node-scoped deployments (e.g. DaemonSet-based KSM)
- Reducing API server load and KSM memory usage
- Multi-tenant environments
A concrete example:
In multi-tenant clusters, nodes are often labeled per tenant (e.g. tenant=team-a).
It is desirable to expose only the Node metrics matching a specific label selector to each tenant, so that:
- Each tenant receives metrics only for its own nodes
- Node-level isolation is preserved
- KSM does not need to cache the full cluster resource set
Important technical note:
KSM exposes object labels as separate *_labels metrics (e.g. kube_node_labels{node="..."} ...).
However, those label values are not necessarily attached as metric labels on all related resource metrics. In practice, this means Prometheus-side metric_relabel_configs cannot reliably filter Node metrics by node labels, because the label information may only be present in separate *_labels metrics rather than as labels on the Node metrics themselves.
As a result, downstream filtering in Prometheus is not sufficient to implement node-level multi-tenant isolation. Filtering must occur at the informer/KSM level.
Existing mechanisms such as --namespaces are insufficient because:
- Nodes are not namespaced
- Namespace-based filtering does not solve node-level isolation
--nodeonly filters Pods via a field selector and does not apply to Nodes
Describe the solution you'd like:
Introduce:
--label-selectors=<resourceType>=<labelSelector>,...
Behavior:
- Comma-separated
resourceType=labelSelector - Applied to
ListOptions.LabelSelectorof the corresponding informer - Disabled by default (backward compatible)
If both:
--node(field selector for Pods)--label-selectors(label selector)
are specified, they are applied independently:
FieldSelectorfor--nodeLabelSelectorfor--label-selectors
Additional context:
When selectors are applied, kube-state-metrics would expose metrics only for the selected subset of resources. It would not attempt to preserve cluster-wide semantics. This behavior would be explicitly documented.
This feature would enable cleaner sharding models and improve scalability for large or multi-tenant clusters.
Metadata
Metadata
Assignees
Labels
Type
Projects
Status