Skip to content

[kafka/exporter] Provide a way to use custom partitioner implementation #46931

@khushijain21

Description

@khushijain21

Component(s)

kafka/exporter

Is your feature request related to a problem? Please describe.

User may want to partition batches by various methods such as round-robin, hashing etc.
Some of the out of the box implementation in franz-go are listed here https://pkg.go.dev/github.com/twmb/franz-go/pkg/kgo#BasicConsistentPartitioner.

Kafka exporter should also allow using these + any custom partitioning strategies.

Describe the solution you'd like

Expose a new field called record_partitioner or just partitioner field that also supports a custom partition along with the available ones on franz-go

record_partitioner:
  sticky:
  roundRobin: 
  leastBackUp:
  custom:
    // name of the extension implementing the custom partitioner

Describe alternatives you've considered

No response

Additional context

No response

Tip

React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions