Skip to content

[WIP] perf: serialize telemetry batches on shared thread pool#1883

Open
jpnurmi wants to merge 12 commits into
masterfrom
jpnurmi/perf/thread-pool
Open

[WIP] perf: serialize telemetry batches on shared thread pool#1883
jpnurmi wants to merge 12 commits into
masterfrom
jpnurmi/perf/thread-pool

Conversation

@jpnurmi

@jpnurmi jpnurmi commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Warning

WIP    🚧🔨⏳⛔

  • Introduce a thread pool that runs tasks in parallel and invokes completion callbacks in submission order.
  • Add an internal telemetry module for coordinating logs and metrics startup, shutdown, and flush.
  • Let the telemetry module own a shared serialization pool for enabled telemetry batchers.
  • Use the pool for log and metric batch serialization while keeping completion ordered through the batcher.
image

before-vs-after.ftrace.zip

Close: #1862

@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown
Fails
🚫 Please consider adding a changelog entry for the next release.

Instructions and example for changelog

Please add an entry to CHANGELOG.md to the "Unreleased" section. Make sure the entry includes this PR's number.

Example:

## Unreleased

### Features

- serialize telemetry batches on shared thread pool ([#1883](https://github.com/getsentry/sentry-native/pull/1883))

If none of the above apply, you can opt out of this check by adding #skip-changelog to the PR description or adding a skip-changelog label.

Generated by 🚫 dangerJS against 07e343d

Comment thread src/sentry_telemetry.c
Comment thread src/sentry_batcher.c
@codecov

codecov Bot commented Jul 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.44693% with 105 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.04%. Comparing base (b742b8d) to head (07e343d).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1883      +/-   ##
==========================================
+ Coverage   75.76%   76.04%   +0.27%     
==========================================
  Files          92       93       +1     
  Lines       21989    22371     +382     
  Branches     3918     3991      +73     
==========================================
+ Hits        16661    17011     +350     
- Misses       4447     4467      +20     
- Partials      881      893      +12     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jpnurmi
jpnurmi force-pushed the jpnurmi/perf/thread-pool branch from 2996a56 to ec39dc8 Compare July 22, 2026 14:55
Comment thread src/backends/sentry_backend_inproc.c
Comment thread src/sentry_batcher.c
@jpnurmi
jpnurmi force-pushed the jpnurmi/perf/thread-pool branch from ec39dc8 to 829189b Compare July 22, 2026 15:14
Comment thread src/backends/sentry_backend_breakpad.cpp
@jpnurmi
jpnurmi force-pushed the jpnurmi/perf/thread-pool branch from 829189b to 406b054 Compare July 22, 2026 15:47
Comment thread src/sentry_batcher.c
Comment thread src/sentry_telemetry.c
@jpnurmi
jpnurmi force-pushed the jpnurmi/perf/thread-pool branch from 406b054 to 03159d6 Compare July 22, 2026 16:02
Comment thread src/sentry_batcher.c
Comment thread src/sentry_batcher.c
Comment thread src/sentry_transport.c
Comment thread src/sentry_telemetry.c
jpnurmi added 10 commits July 24, 2026 12:18
Let crash backends suspend the transport's network submission and
automatically route crash-time envelopes through its disk-dump path.
This removes the need for allocating fallback disk transports in the
backends, and prepares for the upcoming thread-pool telemetry module,
which cannot safely drain the transport queue from a crash handler.
Add a bounded thread pool that runs tasks in parallel and invokes completion
callbacks in submission order.

Add unit coverage for ordered parallel execution.
Add an internal telemetry lifecycle layer for logs and metrics startup,
shutdown, force flush, and crash-safe flush.

Keep the existing logs and metrics behavior behind the shared coordination API.
Let the telemetry lifecycle own a shared serialization pool for enabled
telemetry batchers.

Use the pool for log and metric batch serialization while keeping completion
ordered through the batcher flush lifecycle.
@jpnurmi
jpnurmi force-pushed the jpnurmi/perf/thread-pool branch from e417bed to 12f4872 Compare July 24, 2026 10:19
Comment thread src/sentry_telemetry.c

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 12f4872. Configure here.

Comment thread src/sentry_batcher.c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Logs: optimize performance

1 participant