Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KAFKA-18572: Update Kafka Streams metric documenation #18673

Open
wants to merge 4 commits into
base: trunk
Choose a base branch
from
Open

KAFKA-18572: Update Kafka Streams metric documenation #18673

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
8c153ef
KAFKA-18572: Added idempotent-update-skip-rate and idempotent-update-…
ukpagrace Jan 22, 2025
7a38710
KAFKA-18572: Updated documentation with descriptions for thread metri…
ukpagrace Jan 22, 2025
776f17e
KAFKA-18572: Updated documentation with descriptions for task metrics…
ukpagrace Jan 22, 2025
ef1c755
KAFKA-18572: Updated documentation with descriptions for task metrics…
ukpagrace Jan 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 95 additions & 0 deletions docs/ops.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2868,6 +2868,11 @@ <h5 class="anchor-heading"><a id="kafka_streams_thread_monitoring" class="anchor
<td>The maximum execution time in ms, for punctuating.</td>
<td>kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)</td>
</tr>
<tr>
<td>commit-ratio</td>
<td>The fraction of time the thread spent on committing all tasks</td>
<td>kafka.consumer:type=consumer-coordinator-metrics,client-id=([-.\w]+)</td>
</tr>
<tr>
<td>commit-rate</td>
<td>The average number of commits per sec.</td>
Expand All @@ -2878,6 +2883,11 @@ <h5 class="anchor-heading"><a id="kafka_streams_thread_monitoring" class="anchor
<td>The total number of commit calls.</td>
<td>kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)</td>
</tr>
<tr>
<td>poll-ratio</td>
<td>The fraction of time the thread spent on polling records from consumer</td>
<td>kafka.consumer:type=consumer-coordinator-metrics,client-id=([-.\w]+)</td>
</tr>
<tr>
<td>poll-rate</td>
<td>The average number of consumer poll calls per sec.</td>
Expand All @@ -2888,6 +2898,21 @@ <h5 class="anchor-heading"><a id="kafka_streams_thread_monitoring" class="anchor
<td>The total number of consumer poll calls.</td>
<td>kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)</td>
</tr>
<tr>
<td>poll-records-avg</td>
<td>The average number of records polled from consumer within an iteration.</td>
<td>kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)</td>
</tr>
<tr>
<td>poll-records-max</td>
<td>The maximum number of records polled from consumer within an iteration.</td>
<td>kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)</td>
</tr>
<tr>
<td>process-ratio</td>
<td>The fraction of time the thread spent on processing active tasks</td>
<td>kafka.consumer:type=consumer-coordinator-metrics,client-id=([-.\w]+)</td>
</tr>
<tr>
<td>process-rate</td>
<td>The average number of processed records per sec.</td>
Expand All @@ -2898,6 +2923,21 @@ <h5 class="anchor-heading"><a id="kafka_streams_thread_monitoring" class="anchor
<td>The total number of processed records.</td>
<td>kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)</td>
</tr>
<tr>
<td>process-records-avg</td>
<td>The average number of records processed within an iteration (total count of processed records over total number of iterations).</td>
<td>kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)</td>
</tr>
<tr>
<td>process-records-max</td>
<td>The maximum number of records processed within an iteration.</td>
<td>kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)</td>
</tr>
<tr>
<td>punctuate-ratio</td>
<td>"The fraction of time the thread spends performing punctuating actions on active tasks"</td>
<td>kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)</td>
</tr>
<tr>
<td>punctuate-rate</td>
<td>The average number of punctuate calls per sec.</td>
Expand Down Expand Up @@ -2971,6 +3011,26 @@ <h5 class="anchor-heading"><a id="kafka_streams_task_monitoring" class="anchor-l
<td>The total number of processed records across all source processor nodes of this task.</td>
<td>kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)</td>
</tr>
<tr>
<td>punctuate-latency-avg</td>
<td>The average amount of time taken to execute periodic tasks per call to punctuate</td>
<td>kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)</td>
</tr>
<tr>
<td>punctuate-latency-max</td>
<td>The maximum amount of time taken for any single call to punctuate to complete.</td>
<td>kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)</td>
</tr>
<tr>
<td>punctuate-total</td>
<td>The total number of times the punctuate method was called to trigger periodic actions during task processing.</td>
<td>kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)</td>
</tr>
<tr>
<td>punctuate-rate</td>
<td>The average number of calls to punctuate per second.</td>
<td>kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)</td>
</tr>
<tr>
<td>record-lateness-avg</td>
<td>The average observed lateness in ms of records (stream time - record timestamp).</td>
Expand Down Expand Up @@ -3006,6 +3066,11 @@ <h5 class="anchor-heading"><a id="kafka_streams_task_monitoring" class="anchor-l
<td>The fraction of time the stream thread spent on processing this task among all assigned active tasks.</td>
<td>kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)</td>
</tr>
<tr>
<td>active-buffer-count</td>
<td>The count of buffered records that are polled from consumer and not yet processed for this active task.</td>
<td>kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)</td>
</tr>
<tr>
<td>input-buffer-bytes-total</td>
<td>The total number of bytes accumulated by this task,</td>
Expand All @@ -3016,6 +3081,26 @@ <h5 class="anchor-heading"><a id="kafka_streams_task_monitoring" class="anchor-l
<td>The cache size in bytes accumulated by this task.</td>
<td>kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)</td>
</tr>
<tr>
<td>record-rate</td>
<td>The average number of records restored per second.</td>
<td>kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)</td>
</tr>
<tr>
<td>record-total</td>
<td>The total number of records restored</td>
<td>kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)</td>
</tr>
<tr>
<td>update-rate</td>
<td>The average number of records updated per second.</td>
<td>kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)</td>
</tr>
<tr>
<td>update-total</td>
<td>The total number of records updated</td>
<td>kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)</td>
</tr>
</tbody>
</table>

Expand Down Expand Up @@ -3109,6 +3194,16 @@ <h5 class="anchor-heading"><a id="kafka_streams_node_monitoring" class="anchor-l
<td>The total number of records produced by a sink processor node.</td>
<td>kafka.streams:type=stream-topic-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+),topic=([-.\w]+)</td>
</tr>
<tr>
<td>Idempotent-update-skip-rate</td>
<td>The average number of skipped idempotent updates per second.</td>
<td>kafka.streams:type=stream-topic-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+),topic=([-.\w]+)</td>
</tr>
<tr>
<td>Idempotent-update-skip-total</td>
<td>The total number of skipped updates.</td>
<td>kafka.streams:type=stream-topic-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+),topic=([-.\w]+)</td>
</tr>
</tbody>
</table>

Expand Down
Loading