TGViewer
C++ - Reddit C++ - Reddit @r_cpp · 230 subscribers
Post #25437 9
Designing a plugin "collector" interface for a telemetry module — sanity check on C-ABI vs C++, self-describing schemas, and not over-engineering

I'm designing the interface for a telemetry/observability module in a C++ machine-vision inspection platform, and I'd love a sanity check from anyone who's built plugin/collector systems. I've made a bunch of decisions and mostly want to hear *where I'm wrong* or *what I'm not considering*.

**Context.** The platform is plugin-based — a host app dynamically loads modules and device adapters as DLLs/.so. The telemetry module collects HW/OS metrics and correlates them with the app's own activity (which stage is running, how many run concurrently, etc.) into a forensic log, under a hard constraint: **\~zero impact on the hot inspection path**. I want collection to be extensible — add a new metric source *without touching the core interface*.

**What I have so far — a "collector" abstraction with two tiers:**

1. **Internal collectors (in-process, C++ abstract interface).** These subscribe to the host's typed event bus. The subscribe API is a compile-time template keyed on the C++ event type, so these *can't* sit behind a C ABI — they're a C++ `ICollector` compiled into the module.
2. **External collectors (pure C-ABI plugin DLLs).** These only poll OS/HW/kernel (no host types). They implement a small C ABI (`init/finalize/open/close/describe/collect`), are `dlopen`'d at runtime, and run on a dedicated thread.

Boundary rule: *needs the event bus / a host type → internal C++; OS/HW polling only → external C-ABI.*

**Specific decisions I'm unsure about:**

* **Pull-based C-ABI:** host calls `collect(handle, row_sink_cb, userdata)` on a dedicated thread; the collector pushes result rows into the sink callback (scalar = 1 row, top-N = N rows).
* **Self-describing schema:** `describe(handle)` returns a JSON string declaring each field's `{key, type, unit}` \+ gating metadata (`minLevel`, `osMask`). The host's writer/viewer need no code changes for a new metric.
* **"Day-1" safety contracts** baked into the header: (1) every C function wraps its body in try/catch so no C++ exception crosses the `extern "C"` boundary; (2) the `describe()` string is owned by the collector, valid only until the next `describe`/`close`; (3) `close()` must quiesce the collector's threads before returning.

**Questions I'd love opinions on:**

1. **Two parallel interfaces (C++ internal + C-ABI external) — smell or justified?** Maintaining two shapes of the same idea bugs me, but forcing everything through a C ABI loses typed event subscription. Reasonable seam, or is there a cleaner approach (thin C++ wrapper over one C ABI, message-passing instead of typed events, …)?
2. **Self-describing JSON from** `describe()` **vs a versioned C struct.** JSON is flexible and avoids ABI breaks when fields change, but it's stringly-typed. How do you evolve a plugin schema/ABI safely — version constants, capability negotiation at init, a real IDL (flatbuffers/protobuf) at the boundary?
3. **C-ABI safety idioms.** Are try/catch-everywhere + string-ownership + teardown-quiesce the standard playbook for C++ behind a C ABI, or am I missing cleaner patterns (error codes + out-params, never returning owned pointers)? Is documenting these in header comments enough, or should they be enforced structurally?
4. **Pull vs push, mixed cadence.** Some collectors are cheap/fast (NVML poll), others slow/blocking (kernel queues). Everything is "host pulls on one thread" now. Would you isolate blocking collectors (thread per impact class? async with a deadline?), or is one slow-probe thread + an "impact" self-declaration in `describe()` enough?
5. **Versioning a plugin C-ABI for real.** What's worked for forward/backward compat as the ABI grows — a version int from `init`, additive-only rules, a vtable-of-function-pointers struct instead of named exports?
6. **Avoiding over-engineering.** I have exactly *one* real external collector today (kernel nonpaged pool) and a couple internal ones; the team is wary of speculative plugin machinery.
More from @r_cpp
  1. Sep 29, 2026Boost.Graph 1.95 will be C++17 Dear Boost.Graph community, In two release cycles (Boost re…
  2. Sep 26, 2026Token Sequence Injection & Modern Macros: The Most Game-Changing Compile-Time Feature in C…
  3. Sep 25, 2026myStringStream.str("") Considered Harmful Under C++20 I was recently looking at some (rath…
  4. Sep 20, 2026A clever branch free optimization I'm the developer of memlz which is an extremely fast co…
  5. Sep 15, 2026Inside Boost.PolyCollection https://bannalia.blogspot.com/2026/09/inside-boostpolycollecti…
  6. Sep 11, 2026C++26: Standard Library Hardening Experiments https://www.cppstories.com/2026/hardening-ex…
Threads Profile ViewerView any public Threads profile without an account.Open ThreadLook →Writing with AI? Make it sound human.Metric37 rewrites AI drafts so they read naturally. Free AI detector, 1,500 words free.Try Metric37 →