Skip to content

Fix - Expose sink component_sent_bytes_total from startup - #188

Open
nafiul-s1 wants to merge 1 commit into
masterfrom
fix_http_batch_service_sent_bytes_counter
Open

nafiul-s1 wants to merge 1 commit into
masterfrom
fix_http_batch_service_sent_bytes_counter

Conversation

@nafiul-s1

Copy link
Copy Markdown

HttpBatchService created component_sent_bytes_total with emit!(EndpointBytesSent) on the first successful request and dropped the handle, so the series did not exist before the first event and was removed after expire_metrics_secs while the sink was idle. The "sink no data" alert reads this metric.

HttpBatchService now creates the counter once when it is built, holds the handle and increments it per successful request. The endpoint and protocol labels are dropped; only the component_id/kind/type span tags remain. This covers every sink built on HttpBatchService (http, splunk_hec, elasticsearch, honeycomb, clickhouse, stackdriver, appsignal, datadog events, ...).

Test changes:

  • SINK_TESTS: component_sent_bytes_total is checked as an untagged counter and BytesSent is no longer an expected event.
  • metrics::init_test() returns early if the recorder exists, because VectorRecorder::new_test() clears the registry on every call.
  • Tests that build the sink before the compliance wrapper call init_test() first so the counter is created against a live recorder.

HttpBatchService created component_sent_bytes_total with emit!(EndpointBytesSent)
on the first successful request and dropped the handle, so the series did not
exist before the first event and was removed after expire_metrics_secs while
the sink was idle. The "sink no data" alert reads this metric.

HttpBatchService now creates the counter once when it is built, holds the
handle and increments it per successful request. The endpoint and protocol
labels are dropped; only the component_id/kind/type span tags remain. This
covers every sink built on HttpBatchService (http, splunk_hec, elasticsearch,
honeycomb, clickhouse, stackdriver, appsignal, datadog events, ...).

Test changes:
- SINK_TESTS: component_sent_bytes_total is checked as an untagged counter and
  BytesSent is no longer an expected event.
- metrics::init_test() returns early if the recorder exists, because
  VectorRecorder::new_test() clears the registry on every call.
- Tests that build the sink before the compliance wrapper call init_test()
  first so the counter is created against a live recorder.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants