Synchronization service

Synchronization service#

Vulnerability-Lookup can synchronize objects (comments, bundles, sightings, KEV entries) from remote Vulnerability-Lookup instances. This allows a local instance to pull published data from one or more remote instances via their public APIs.

Configuration#

Remote instances are managed from the admin interface at /admin/remote_instances/. For each remote instance you can configure:

  • Name: a label for the remote instance.

  • Address: the base URL of the remote instance (e.g. https://vulnerability.circl.lu).

  • GNA Identifier: optional GCVE GNA identifier.

  • Sync toggles: choose which object types to synchronize (comments, bundles, sightings, KEV entries).

  • Sync local objects only: when enabled (default), only objects originally created on the remote instance are synced — objects the remote itself synced from elsewhere are skipped.

  • Active: enable or disable the instance.

The last sync date is displayed when editing a remote instance and can be reset to force a full re-pull on the next sync cycle.

Launching the service with poetry#

From the directory used to install Vulnerability-Lookup:

Daemon mode (runs every hour by default):

poetry run flask --app website.app sync

One-shot mode (runs once and exits, ideal for cron):

poetry run flask --app website.app sync --interval 0

Custom interval (e.g. every 30 minutes):

poetry run flask --app website.app sync --interval 1800

Specifying the sync user (synced objects will be attributed to this user):

poetry run flask --app website.app sync --login myuser

With file logging:

poetry run flask --app website.app sync --log-file /var/log/vulnerability-lookup_sync.log

Launching the service with systemd#

Create the file /etc/systemd/system/vulnerability-lookup-sync.service:

[Unit]
Description=Vulnerability-Lookup sync service
After=vulnerability-lookup-web.service
Requires=vulnerability-lookup-web.service

[Service]
Type=simple
User=<system user used to install Vulnerability-Lookup>
Group=<group of the user used to install Vulnerability-Lookup>
WorkingDirectory=<path to the cloned repository>
Environment="VULNERABILITYLOOKUP_HOME=<path to the cloned repository>"
ExecStart=<path to poetry> run python -m flask \
    --app website.app sync \
    --login "sync_user" \
    --interval 3600 \
    --log-file /var/log/vulnerability-lookup/sync.log
StandardOutput=append:/var/log/vulnerability-lookup_sync_message.log
StandardError=append:/var/log/vulnerability-lookup_sync_error.log
Restart=on-failure
RestartSec=15s

[Install]
WantedBy=multi-user.target

Note

Use the same user and group that were used to install Vulnerability-Lookup, and ensure that this user has write permissions to the log files and their parent directory.

Use Requires only if the web service is managed by systemd on the same host, otherwise just use After to ensure the web service starts before the sync service when both are enabled on the same host.

Then reload and enable the service:

sudo systemctl daemon-reload
sudo systemctl enable vulnerability-lookup-sync.service
sudo systemctl start vulnerability-lookup-sync.service
systemctl status vulnerability-lookup-sync.service

To follow the service logs live:

sudo journalctl -u vulnerability-lookup-sync.service -f

Error logs:

tail -f /var/log/vulnerability-lookup_sync_error.log

Sync logs:

tail -f /var/log/vulnerability-lookup_sync.log

Batch deletion of synced objects#

Synced objects can be deleted in bulk via the API using the vulnerability_lookup_origin parameter, which corresponds to the UUID of the remote instance they were pulled from.

# Delete all comments from a specific remote instance
curl -X DELETE -H "X-API-Key: <api_key>" \
    "https://localhost:10001/api/comment/?vulnerability_lookup_origin=<remote-uuid>"

# Delete all bundles from a specific remote instance
curl -X DELETE -H "X-API-Key: <api_key>" \
    "https://localhost:10001/api/bundle/?vulnerability_lookup_origin=<remote-uuid>"

# Delete all sightings from a specific remote instance
curl -X DELETE -H "X-API-Key: <api_key>" \
    "https://localhost:10001/api/sighting/?vulnerability_lookup_origin=<remote-uuid>"
Contents