Skip to content

Syslog CEF receiver: doc fixes and follow-up TODO from #2452 #2461

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
drewrelmas merged 3 commits into from
Mar 31, 2026
+26 −12
Merged

Syslog CEF receiver: doc fixes and follow-up TODO from #2452 #2461

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
304a917
Minor changes
utpilla Mar 30, 2026
7f3c6fc
Fix markdown
utpilla Mar 30, 2026
f5e4456
Merge branch 'main' into utpilla/Minor-changes-syslog-cef-receiver
lquerel Mar 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 11 additions & 12 deletions rust/otap-dataflow/crates/core-nodes/src/receivers/syslog_cef_receiver/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -356,18 +356,17 @@ To optimize throughput and reduce per-message overhead, the receiver batches
messages into Apache Arrow record batches before sending downstream:

```text
+-------------------------------------------------------------+
| Batching Logic |
+-------------------------------------------------------------+
| |
| Messages arrive --> ArrowRecordsBuilder --> Batch |
| | sent |
| | |
| Flush conditions: | |
| +- Size: max_size messages +--------------------------> |
| +- Time: max_batch_duration_ms | |
| |
+-------------------------------------------------------------+
+-------------------------------------------------------------------+
| Batching Logic |
+-------------------------------------------------------------------+
| |
| Messages arrive --> ArrowRecordsBuilder --> Batch sent |
| | |
| Flush conditions: | |
| +- Size: max_size messages +-------------------------------> |
| +- Time: max_batch_duration_ms +--------------------------> |
| |
+-------------------------------------------------------------------+
```

A batch is flushed when either:
Expand Down
8 changes: 8 additions & 0 deletions rust/otap-dataflow/crates/core-nodes/src/receivers/syslog_cef_receiver/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -568,6 +568,14 @@ impl local::Receiver<OtapPdata> for SyslogCefReceiver {
metrics.borrow_mut().received_logs_truncated.inc();
}

// TODO: When a message exceeds MAX_MESSAGE_SIZE, the truncated
// head is emitted as one record and the remaining tail bytes become
// a separate record with no syslog header context (severity, timestamp,
// etc.). Consider adding fragment-correlation metadata (e.g. a shared
// attribute linking head and tail) or synthesizing a syslog header on
// the continuation fragment so downstream consumers can associate the
// pieces. See https://github.com/open-telemetry/otel-arrow/pull/2452#discussion_r3004024837

// Strip trailing newline if present
// (Complete has it, Truncated does not)
let message_to_parse = if line_bytes.last() == Some(&b'\n') {
Expand Down
7 changes: 7 additions & 0 deletions .../crates/core-nodes/src/receivers/syslog_cef_receiver/syslog-parsing-behavior.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# Syslog Parsing Behavior

> **Developer reference** -- This document describes the internal parsing
> behavior of the syslog CEF receiver. It is useful for contributors
> working on the parser as well as anyone wanting to understand which
> message formats are accepted, how edge cases are handled, and what
> output to expect from valid and invalid input. For user-facing
> configuration and usage, see [README.md](README.md).

## Format Detection Order

The top-level `parse()` function in
Expand Down
Loading