# High level architecture

```{figure} _static/img/vulnerability-lookup.png
:alt: High level architecture
:target: _static/img/vulnerability-lookup.png

High level architecture
```

The platform is organized in three tiers: a **storage layer**, a **cache layer**, and a **web layer**.

## Storage layer (Kvrocks)

[Kvrocks](https://github.com/apache/kvrocks) is the primary persistent storage for all vulnerability data.
It runs on TCP (default port 10002) and uses RocksDB as its storage engine, providing a Redis-compatible protocol
with significantly lower memory requirements.

Key storage patterns:

- Vulnerabilities are stored by lowercase ID: `cve-2024-1234` → JSON blob
- Metadata: `cve-2024-1234:meta` → hash mapping meta types to UUIDs
- Cross-references: `cve-2024-1234:link` → set of related vulnerability IDs
- Source indexes: `index:<source_name>` → sorted set of vulnerability IDs, by last update
  - `index:<source>:published`, `:reserved`, `:rejected` → the same, by date of that event
  - `index:<source>:state:<state>` → records currently in that state (read by `/api/stats/vulnerability/status`)
- Last update tracking: `last_updates` hash with per-source timestamps
- Prepared statistics, per source (written by `index_vulnerabilities` and `euvd --stats-scan`, read by `/api/stats/*`;
  a missing key means the figure was never prepared, served as `available: false`):
  - `<state>:count:<source>[:<period>]` → record counters per year or month (`published`, `reserved`, `rejected`); the
    total `published:count:<source>` being absent means the indexing pass never ran for that source
  - `state_published:count:<source>[:<period>]` (+ `:prepared_at`) → records published now, withdrawn ones left out
  - `vendors:ranking:<source>[:<period>]`, `assigners:ranking:<source>[:<period>]` → zsets behind the two rankings
  - `stats:<source>:cvss_band:<band>[:<year>]`, `stats:<source>:cvss_version:<version>[:<year>]` → CVSS distribution counters
  - `stats:<source>:time_to_exploit[:<year>]` → hash, the last time to exploit snapshot, whole and per publication year
  - `stats:<source>:kev_listed:total`, `stats:<source>:kev_listed:catalog:<catalog>`,
    `stats:<source>:kev_listed:unattributed` → hashes of month → records a catalogue listed that month
    (bucketed on the *listing* date, not the publication date), with `:catalogs` (set) and `:coverage` (hash) beside them
  - `stats:<source>:scorecard:meta`, `stats:<source>:scorecard:<window|all>:<assigner>` (hashes),
    `stats:<source>:scorecard:ranking:<bucket>`, `stats:<source>:scorecard:records:<bucket>` (zsets) → the
    assigner scorecard, prepared by `scorecard` and every reindex (see `vulnerabilitylookup/scorecard.py`)
  - `stats:kev:scorecard:meta`, `stats:kev:scorecard:<window|all>:<catalogue uuid>` (hashes),
    `stats:kev:scorecard:ranking:<bucket>`, `stats:kev:scorecard:entries:<bucket>` (zsets) → the
    catalogue scorecard, prepared by `kev_scorecard` from the KEV tables (see `vulnerabilitylookup/kev_scorecard.py`)
  - `index:<source>:kev[:<catalog>]` → set of known-exploited records; EUVD ingestion maintains its own, `euvd --stats-scan` rebuilds the others
  - `stats:sources` → set of sources that have a prepared figure

## Cache layer (Valkey)

[Valkey](https://valkey.io) serves as a fast in-memory cache running on a Unix socket (`cache/cache.sock`).
It handles Pub/Sub messaging between feeders and the web layer, temporary data caching, and real-time
event distribution to services such as Meilisearch and the email notification system.

## Backend (Feeders)

The backend consists of 40+ feeders responsible for downloading security advisories from various sources.
All feeders inherit from the `AbstractFeeder` base class (CSAF-based feeders use `CSAFGeneric`).
Each feeder publishes collected data to Valkey Pub/Sub channels, enabling seamless integration with downstream services.

## PostgreSQL database

PostgreSQL stores user accounts, comments, bundles, sightings, watchlists, organizations, and KEV catalogs.
The web application uses SQLAlchemy as its ORM with Alembic for schema migrations.

```{figure} _static/img/model_diagram.png
:alt: Database models diagram
:target: _static/img/model_diagram.png

Database models diagram
```

## Web layer (Flask)

The web interface is a Flask application served by Gunicorn (default port 10001).
Views are organized as Blueprints and the REST API is available under `/api/`.
User authentication is handled by Flask-Login with optional two-factor authentication.

When user accounts are enabled (`user_accounts: true` in `config/generic.json`),
the platform supports comments, bundles, sightings, watchlists, and the CVD workflow.

## Tools

It is possible to extend Vulnerability-Lookup functionalities thanks to its API and various tools.
For example:

- [PyVulnerabilityLookup](https://github.com/vulnerability-lookup/PyVulnerabilityLookup) — Python client library for the REST API.
- [FediVuln](https://pypi.org/project/FediVuln) — publish vulnerability updates to the Fediverse via the Pub/Sub streaming service.
- [ML-Gateway](https://github.com/vulnerability-lookup/ML-Gateway) — AI-powered severity classification, ATT&CK technique suggestions and bi-encoder similarity searches (VLAI).
