host-plugin

The Kafka capability implemented as a wasmCloud host component plugin.

One long-lived, host-scoped instance serves every workload that binds to it. That split is the point: a Kafka client is stateful — connections to every broker, cached metadata, partition leadership, group membership with heartbeats — and none of it survives a per-request component instance.

Per-workload clients

The plugin does not hold one shared Kafka client. Kafka authorises per principal (ACLs on topics and groups), so a shared client would give every workload identical rights and defeat the isolation the sandbox exists to provide. Instead on-workload-bind delivers each workload's own bootstrap.servers, credentials and topic set, and the plugin opens a client per workload, keyed by workload id.

Later capability calls carry no binding information, so the plugin correlates them back with wasmcloud:host/identity.get-workload-id(). That identity is the only thing distinguishing one tenant's calls from another's, which makes the identity import mandatory rather than a convenience.

Credentials

A workload declares them on its interface binding, and the host merges config -> configFrom -> secretFrom (last wins) before delivering the result as this binding's config:

hostInterfaces:
  - namespace: cosmonic
    package: kafka
    version: "0.1.0"
    interfaces: [producer, consumer]
    config:
      bootstrap.servers: kafka.svc:9092
      group.id: orders-processor
      topics: orders,payments        # what this workload may consume
    secretFrom:
      - kafka-credentials            # sasl.username, sasl.password, ...

Every key is passed to librdkafka verbatim, so any auth mechanism it supports works unchanged: SASL/PLAIN, SCRAM-SHA-256/512, OAUTHBEARER, GSSAPI, or mTLS via ssl.key.pem / ssl.certificate.pem.

The plugin should treat this config as the whole permission grant — a workload that did not declare a topic should not be able to reach it, even though the broker-side ACL is the real enforcement point.

Two shapes of delivery

Pull. A workload holds a consumer resource and reads records(). The call is ingress, so the stream carries records outward without the provider calling in — and the consuming workload must be long-lived to hold it.

Push. The provider calls handler.handle on a workload that exports it, which frees the workload from being long-lived: it is started for a batch and retires when the burst drains. This was not possible when this package was first written — a component plugin's imports resolve against the host's own plugins, never against workloads — and wasmcloud:host/workload-call is what changed it. A plugin opens a target for a workload id and an interface and calls it. Both shapes are optional: a provider may serve either, and a consuming workload picks by importing consumer or exporting handler.