Streaming service

A Streaming service is implemented using a Publish/Subscribe (Pub/Sub) pattern.

Available channels:

• vulnerability

• comment

• bundle

• sighting

Database-level listeners automatically publish new sightings, comments, and bundles to their respective Pub/Sub channels. The various feeders push incoming vulnerabilities directly to the designated vulnerability channel.

Configuration

The streaming service can be configured using the config/stream.json file. An example configuration is provided below.

{
    "register_listeners": true,
    "pubsub_bp": true,
    "global_subscription": false,
    "channels": [
        "vulnerability",
        "comment",
        "bundle",
        "sighting"
    ],
    "_notes": {
        "register_listeners": "Register the listeners for comments, bundles, and sightings.",
        "pubsub_bp": "Activate or deactivate the subscription Blueprint for specific clients.",
        "channels": "The list of channels exposed by the Pub/Sub Blueprint.",
        "global_subscription": "Activate or deactivate the global subscription mechanism for server-side processing of messages."
    }
}

Listeners

When database-level listeners are enabled, new comments, bundles, and sightings are automatically streamed to the Pub/Sub service, with three rules:

• Only published content. A comment held for moderation is not streamed when it is inserted; it is streamed when it gets published. Bundles and sightings have no moderation step and are streamed on insert.

• Only after the commit. A message leaves once the transaction that wrote the row has committed, never for a write that was rolled back.

• An explicit projection. The payload of a message is the object as the REST API returns it (GET /api/comment/<uuid>, /api/bundle/<uuid>, /api/sighting/<uuid>): the author is its public UUID, timestamps are RFC 3339 in UTC, and internal columns such as the author’s database id or the moderation flag never appear.

Every message is a JSON object with the same envelope:

{
  "payload": {
    "uuid": "9c2b2b1e-0b8d-4a44-9a0a-6d1c6b1f2a10",
    "vulnerability_lookup_origin": "1a89b78e-f703-45f3-bb86-59eb712668be",
    "author": "5e0c6a9f-3f22-4a63-8f6e-2d3a8b7c9d01",
    "title": "Patch available",
    "description": "Fixed in 2.4.1.",
    "description_format": "markdown",
    "vulnerability": "CVE-2024-0001",
    "creation_timestamp": "2026-09-11T09:52:22.013928Z",
    "timestamp": "2026-09-11T09:52:22.013928Z",
    "related_vulnerabilities": [],
    "meta": {}
  },
  "instance_uuid": "1a89b78e-f703-45f3-bb86-59eb712668be",
  "timestamp": "2026-09-11T09:52:22.033420Z",
  "data_type": "comment",
  "event": "created",
  "uri": "https://vulnerability.circl.lu/comment/9c2b2b1e-0b8d-4a44-9a0a-6d1c6b1f2a10"
}

data_type is the channel the message was published on. event is created for a row streamed on insert and published for a comment that was moderated after its insert. uri is the web page of the object; it is empty when the row was written outside a web request (by the synchronization with a remote instance, for instance).

Internal streaming

When global_subscription is set to True, an internal subscription mechanism is provided for the components of the application. The internal streaming service is executed in a dedicated thread.

HTTP streaming

The subscription endpoint is gated on the global stream:subscribe permission, which the built-in admin role holds. The stream is a firehose of everything written on the instance, meant for the operator’s automation, so the endpoint is deliberately not open to every confirmed account. To let an automation account read the stream without making it an administrator, create a global role carrying only stream:subscribe and assign it to that account.

Operators should also be aware that a long-lived SSE connection needs the reverse proxy configured for it: streaming enabled (beresp.do_stream in Varnish), buffering off, and a first-byte timeout longer than the interval between events. The endpoint emits an SSE comment line at least every 15 seconds so an idle channel still carries traffic.

Connection to the authenticated HTTP subscribing interface when (pubsub_bp is set to True):

$ curl -H "X-API-KEY: <YOUR-TOKEN>" http://127.0.0.1:10001/pubsub/subscribe/comment

FediVuln can also be used to subscribe to a channel:

$ FediVuln-Publish -t comment

A status will be pushed with the configured Mastodon account (see the documentation of FediVuln).