Skip to content

New component: kafkatopicsobserver #37665

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wojtekzyla opened this issue Feb 4, 2025 · 1 comment
Closed

New component: kafkatopicsobserver #37665

wojtekzyla opened this issue Feb 4, 2025 · 1 comment
Labels
Accepted Component New component has been sponsored

Comments

@wojtekzyla
Copy link
Contributor

wojtekzyla commented Feb 4, 2025 •
edited by atoulme
Loading

The purpose and use-cases of the new component

The Kafka topics observer extension is a Receiver Creator-compatible "watch observer" that will detect and report
kafka topics in kafka cluster based on regex template. This observer watches available topics and matches them with the
provided regex. If any change in available topics matching the regex is detected, the observer updates the endpoints list.

Example configuration for the component

kafkatopics_observer:
  protocol_version: 3.7.0
  brokers: ["1.2.3.4:9092", "2.3.4.5:9092"]
  topic_regex: "^test[0-9]$"
  topics_sync_interval: 5s
  resolve_canonical_bootstrap_servers_only: false
  session_timeout: 30s
  heartbeat_interval: 20s
  auth:
    plain_text:
      username: fooUser
      password: fooPassword

Telemetry data types supported

logs, metrics, traces

Code Owner(s)

@MovieStoreGuy

Sponsor (optional)

@MovieStoreGuy

Additional context

No response
@wojtekzyla wojtekzyla added needs triage New item requiring triage Sponsor Needed New component seeking sponsor labels Feb 4, 2025
@wojtekzyla wojtekzyla mentioned this issue Feb 4, 2025
Merged
@MovieStoreGuy MovieStoreGuy added Accepted Component New component has been sponsored and removed Sponsor Needed New component seeking sponsor labels Feb 5, 2025
@atoulme atoulme removed the needs triage New item requiring triage label Feb 10, 2025
@github-actions github-actions bot mentioned this issue Feb 11, 2025
Closed
MovieStoreGuy added a commit that referenced this issue Feb 14, 2025
@wojtekzyla @MovieStoreGuy
[extension/kafkatopicsobserver] add kafka topics observer extension (#…
93a3ded 
…37669)

#### Description
Initial PR for adding kafka topics observer extension. Continuation of
#37372 after renaming a branch.

#### Link to tracking issue
New component
#37665

Co-authored-by: Sean Marciniak <[email protected]>
@wojtekzyla wojtekzyla mentioned this issue Feb 19, 2025
Merged
MovieStoreGuy pushed a commit that referenced this issue Feb 24, 2025
@wojtekzyla
Add kafka topics observer implementation (#38060)
ad684d6 
#### Description
Add implementation and tests for kafka topics observer

#### Link to tracking issue
New component
#37665 

#### Testing
Unit tests
@wojtekzyla wojtekzyla mentioned this issue Mar 6, 2025
Merged
MovieStoreGuy pushed a commit that referenced this issue Mar 11, 2025
@wojtekzyla
[kafkatopicsobserver] change kafkatopicsobserver stability to alpha (#…
780bb23 
…38424)

#### Description
Change kafkatopicsobserver stability to alpha

#### Link to tracking issue
New component
#37665
This was referenced Mar 27, 2025
Closed
Merged
@MovieStoreGuy
Copy link
Contributor

Considering this component is now ready to be deployed as of the contrib, it is reasonable to say that this component is accepted and this issues can be closed.

@wojtekzyla wojtekzyla mentioned this issue Apr 1, 2025
Merged
MovieStoreGuy pushed a commit that referenced this issue Apr 7, 2025
@wojtekzyla @atoulme
[receiver/receivercreator] Ffeat/add kafkatopicsobserver to receiverc…
ce4085e 
…reator (#39086)

#### Description
Add kafkatopicsobserver to the receivercreator configuration

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
New component 
#37665 

#### Testing
Manual and unit test

#### Documentation
Added new sections in README file

---------

Co-authored-by: Antoine Toulme <[email protected]>
dmathieu pushed a commit to dmathieu/opentelemetry-collector-contrib that referenced this issue Apr 8, 2025
@wojtekzyla @atoulme @dmathieu
[receiver/receivercreator] Ffeat/add kafkatopicsobserver to receiverc…
5bcec75 
…reator (open-telemetry#39086)

#### Description
Add kafkatopicsobserver to the receivercreator configuration

<!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
New component 
open-telemetry#37665 

#### Testing
Manual and unit test

#### Documentation
Added new sections in README file

---------

Co-authored-by: Antoine Toulme <[email protected]>
LucianoGiannotti pushed a commit to LucianoGiannotti/opentelemetry-collector-contrib that referenced this issue Apr 9, 2025
@wojtekzyla @atoulme @LucianoGiannotti
[receiver/receivercreator] Ffeat/add kafkatopicsobserver to receiverc…
e29a2ec 
…reator (open-telemetry#39086)

#### Description
Add kafkatopicsobserver to the receivercreator configuration

<!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
New component 
open-telemetry#37665 

#### Testing
Manual and unit test

#### Documentation
Added new sections in README file

---------

Co-authored-by: Antoine Toulme <[email protected]>
Fiery-Fenix pushed a commit to Fiery-Fenix/opentelemetry-collector-contrib that referenced this issue Apr 24, 2025
@wojtekzyla @atoulme @Fiery-Fenix
[receiver/receivercreator] Ffeat/add kafkatopicsobserver to receiverc…
840ed30 
…reator (open-telemetry#39086)

#### Description
Add kafkatopicsobserver to the receivercreator configuration

<!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
New component 
open-telemetry#37665 

#### Testing
Manual and unit test

#### Documentation
Added new sections in README file

---------

Co-authored-by: Antoine Toulme <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Accepted Component New component has been sponsored
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@atoulme @MovieStoreGuy @wojtekzyla