# Statistics

Eight figures describe a corpus: records per period, lifecycle status, CVSS versions,
severity bands, known-exploited inclusion, time to exploit, top assigners and top
vendors.

- Each figure is computed **per source** (`?source=`, default `cvelistv5`).
- Each one can be narrowed to **one period**.
- Request and response shapes are in the Swagger UI at `/api/`. The keys are in
  [Architecture](architecture.md). The passes that prepare them are in
  [Prepared statistics figures](command-line-interface.md#prepared-statistics-figures).

## Endpoints

| Figure | Any source | EUVD |
| --- | --- | --- |
| Records per period | `/api/stats/vulnerability/counts` | `/api/euvd/stats/count` |
| Lifecycle status | `/api/stats/vulnerability/status` | `/api/euvd/stats/status` |
| CVSS version distribution | `/api/stats/cvss/versions` | `/api/euvd/stats/cvss/versions` |
| Severity bands | `/api/stats/cvss/scores` | `/api/euvd/stats/cvss/scores` |
| Known-exploited inclusion | `/api/stats/kev` | `/api/euvd/stats/kev` |
| Time to exploit | `/api/stats/time_to_exploit` | `/api/euvd/stats/time_to_exploit` |
| Top assigners | `/api/stats/assigners/ranking` | `/api/euvd/stats/assigners/ranking` |
| Top vendors | `/api/stats/vendors/ranking` | `/api/euvd/stats/vendors/ranking` |

The EUVD namespace is the same code with `source=euvd` fixed (no `source` parameter).
Both are cached for `STATS_CACHE_TIMEOUT` (one hour by default).

## Comparing sources

`GET /api/stats/vulnerability/sources` returns `sources`: one entry per source, with
`source` and `records`, largest first. The statistics page charts the 20 largest and
lists all of them in a table.

- Counts come from `index:<source>`, kept current by the feeders. No reindex is needed.
- Only vulnerability sources are listed, not reference or enrichment feeds.
- Withdrawn records are counted.
- **Counts are not additive.** `cvelistv5`, `nvd`, `fkie_nvd`, `euvd` and the GNA
  mirrors describe the same CVEs, so there is no total or percentage.

## Reading an answer

- **`available: false` is not zero.** It means the figure was never prepared for this
  source. The two rankings are the exception: they return an empty list instead.
- **Most figures date from the last pass.** These are live:
  - lifecycle status;
  - known-exploited membership for EUVD (other sources: as of the last
    `euvd --stats-scan`).

  `time_to_exploit` carries `computed_at`.
- **A year is a publication year**, never the year in the identifier. The one exception
  is known-exploited per period, which uses the listing date (`basis: listed_at`).
- **A period that cannot be applied is a 400**, never silently ignored.

## The eight figures

### Records per period

`GET /api/stats/vulnerability/counts?source=<name>[&granularity=month&period=YYYY][&state=reserved][&basis=...]`

Records per publication year, or per month of one year.

- The years covered come from the source's published index: no leading run of zeros.
- `period` without `granularity=month` is a 400.
- `state=reserved` counts by reservation date, on the same year axis.

`basis` selects which records are counted:

| `basis` | Counts | A past year's total |
| --- | --- | --- |
| `date_published` (default, except for EUVD) | every record with a publication date | never changes |
| `state_published` (default for EUVD) | only records still `PUBLISHED` | falls when one is withdrawn |

- `state_published` is available for `cvelistv5`, the GNA sources and EUVD. Other
  sources answer `available: false`.
- On the CVE list it excludes about 12,000 withdrawn records. It then matches the
  severity and CVSS charts.
- `basis=state_published` with `state=reserved` is a 400.
- `STATS_DEFAULT_BASIS` in `config/website.py` sets the default per source. It is
  `{"euvd": "state_published"}`, and setting the option replaces it. A request that
  names a `basis` gets that one.
- `prepared_at` is set when a figure is only as fresh as the last pass, and is `null`
  when the figure is live. This applies to `state_published` for every source, and to
  both bases for the GNA sources.

**EUVD is different.** EUVD records carry no publication date of their own:

- Minted EUVDs are absent from `date_published`, but counted in `state_published`
  under their CVE's date. So `state_published` can be the *larger* figure.
- Its status comes from the linked CVE when there is one, otherwise from the EUVD
  record itself.
- A past year can grow: an EUVD minted today for a 2017 CVE is added to 2017.
- A published EUVD with no date at all counts in the total but in no period.
- It does not match the EUVD severity and CVSS charts, which skip undated records.
- After upgrading, run the EUVD pass right away: it backfills the stored status.

`/api/stats/kev` also has a `basis` field, with an unrelated meaning.

### Lifecycle status

`GET /api/stats/vulnerability/status?source=<name>[&period=YYYY]`

```json
{
  "source": "cvelistv5",
  "total": 400321,
  "published": 394048,
  "reserved": 400321,
  "state": { "published": 381920, "reserved": 936, "rejected": 17471 }
}
```

- `published` and `reserved` count records that **carry that date**. These are the
  same records as `basis=date_published`.
- `state` gives the **current** lifecycle values. These are the same records as
  `basis=state_published`.
  - For "what does the source publish today", use `state.published`.
  - For "how many were ever published", use `published`.
- The numbers do not sum to `total`: a record can carry both dates, or neither.
- `reserved` is not "awaiting publication". On the CVE list every record has a
  reservation date. Use `state.reserved` for records awaiting publication.
- `state` is absent for a source without a lifecycle index.
- With `period`, only `published` and `reserved` are returned. `total` and `state`
  have no meaning for a past year.

### CVSS version distribution

`GET /api/stats/cvss/versions?source=<name>[&period=YYYY]`

Buckets: `4.0`, `3.1`, `3.0`, `2.0`, `other`, `unscored`.

### Severity bands

`GET /api/stats/cvss/scores?source=<name>[&period=YYYY]`

- Bands: `none`, `low`, `medium`, `high`, `critical` (CVSS v3.1 ratings, applied to
  any version), plus `unscored`.
- The score is from the highest CVSS version of the record, as shown on its page.
  Where that version has several scores, the first is used. For `source=euvd` it is the
  highest, as on the EUVD.
- Both CVSS figures count **published records only, without withdrawn ones**: about
  12,000 on the CVE list. As a result, the per-year bands do not add up to
  `published:count:<source>:<year>`.

### Known-exploited inclusion

```text
GET /api/stats/kev?source=<name>
GET /api/stats/kev?source=<name>&granularity=year
GET /api/stats/kev?source=<name>&granularity=month&period=YYYY
```

Without `granularity` (`basis: membership`), the response gives:

- `exploited`: the records in a known-exploited catalogue;
- the ratio of `exploited` to the total;
- a count per catalogue.

Details:

- A bare `period` is a 400. Membership has no date; use the per-period form.
- **Per-catalogue counts overlap.** A record listed twice is counted twice, so they
  don't sum to `exploited`.
- Every configured catalogue is listed, even with zero.
- `unattributed` counts records whose catalogue can't be named. A non-zero value
  means the catalogue reference data is missing or stale.
- **Scope:** `STATS_KEV_CATALOGS` in `config/website.py` limits a source to some
  catalogues. The default is `{"euvd": ["CISA", "ENISA"]}`, and setting the option
  replaces it.
  - A scoped source answers `scoped: true` and lists only its `catalogs`.
  - Membership counts follow a scope change immediately, except where EUVD's scope
    is widened or removed: its per-catalogue membership is only written for the
    catalogues in scope, so a catalogue brought in is counted from the next
    `euvd --backfill-kev`. The dated figures follow at the next `euvd --stats-scan`.
  - For EUVD, the record itself follows the same scope: its exploited flag, the
    `exploited` search filter, its exploited-since date and the exploited listing. A
    vulnerability listed only by a catalogue out of scope is not exploited for EUVD,
    and stays in that catalogue.
    An entry whose origin maps to no catalogue is out of any scope, as in the figures.
    Run `euvd --backfill-kev` after a change of scope. It refuses to run, naming the
    entry, while the scope names a catalogue that no GCVE KEV reference of the
    instance names, and the KEV sync skips its EUVD updates for as long.
- EUVD maintains its membership during ingestion. For other sources,
  `euvd --stats-scan --stats-source <name>` rebuilds it, which makes sense for
  CVE-keyed sources.
- **Denominator:** the narrowest population the source has, named by `total_basis`.
  The numerator is narrowed the same way, so the ratio never exceeds 1.

| `total_basis` | Counted from | Meaning |
| --- | --- | --- |
| `state_published` | `index:<source>:state:published` | currently published |
| `date_published` | `index:<source>:published` | has a publication date |
| `held` | `index:<source>` | every record held |

#### Per period

With `granularity` (`basis: listed_at`):

- **Buckets use the date a catalogue listed the record**, not its publication date.
  That date is `timestamps.first_seen_at`, else `recorded_at`, else
  `status_updated_at` (never `asserted_at`).
- `catalogs` is `{catalogue: {period: count}}`. It overlaps, like the membership counts.
- `totals` counts each record once, in the period of its first listing. Its sum gives
  `exploited` back.
- `unattributed_periods` is the same series for unnamed catalogues.
- All series share one axis, with real zeros: consecutive years up to now, or the 12
  months of the year asked for.
- `available: false` means the scan never ran for this source. The membership sets
  come from a different pass.

### Time to exploit

`GET /api/stats/time_to_exploit?source=<name>[&period=YYYY | &granularity=year]`

Days from publication to the first known-exploited listing: mean, median, p90, min and
max.

- `period`: the same summary for the records published that year.
- `granularity=year`: every year, under `years`. A year with no exploited record is
  `{"count": 0}`.
- `period` and `granularity` cannot be combined.
- **The first listing among the catalogues in scope** ends the interval. A scoped source
  answers `scoped: true` and `catalogs`, as recorded by the scan.
- `count`, `cves_exploited` and `entries_scanned` give the base: exploited records only.
  Prefer the median: one old CVE listed late moves the mean a lot.
- **Negative intervals are kept**: the catalogue listed it before publication. They are
  counted in `exploited_before_publication`, with their mean in
  `before_publication_mean_days`.
- **Only currently published records are dated.** `excluded_not_published` counts the
  withdrawn and reserved ones left out. It is 0 for sources without a lifecycle index.
- `unmatched_no_record` and `unmatched_no_publication_date` count the entries that
  couldn't be dated. They cover the whole source and are absent from a period summary.
- A period with nothing exploited answers `available: true, count: 0`.

### Top assigners

`GET /api/stats/assigners/ranking?source=<name>&limit=10[&period=YYYY]`

- `limit` defaults to 10, at most 100.
- `source` must be one of `cvelistv5` (default), `certfr_avis`, `certfr_alerte` and
  `euvd`. Any other value is a 400.
- For EUVD, the assigner comes from the linked CVE, not the EUVD record.
- The EUVD rankings leave out records whose CVE was withdrawn. Other sources count
  every record ever published.

### Top vendors

`GET /api/stats/vendors/ranking?source=<name>&limit=10[&period=YYYY]`

Same `limit` and `source` values as the assigner ranking. For alias sources such as
EUVD, vendors come from the linked CVE.

## Other endpoints

These are not per-source figures. All take `?output=markdown` for a Markdown table.

| Endpoint | Returns | Parameters |
| --- | --- | --- |
| `/api/stats/vulnerability/most_sighted` | Vulnerabilities with the most sightings | `date_from`, `date_to`, `sighting_type`, `limit` (10, at most 100) |
| `/api/stats/vulnerability/most_commented` | Vulnerabilities with the most comments | `date_from`, `date_to`, `limit` (10, at most 100) |
| `/api/stats/cwe/most_used` | Most frequent CWEs in `cvelistv5` records | `period`, `limit` (10, at most 100) |
| `/api/stats/credits/ranking` | Most credited people and organisations | `period`, `source`, `limit` (10, at most 1000), `offset` |
| `/api/stats/sighting/exploitation_ratio_cumulative` | Running share of CVEs with an exploitation sighting, by sighting date | `granularity` (`month` or `year`), `window` (30) |

`/api/stats/vulnerability/count` is the older single-period form of `/counts`. It needs
`period` and `source`, and only accepts `cvelistv5`, `github`, `pysec`, `tailscale`,
`ossf_malicious_packages`, `variot` and `euvd`.

## Easy to confuse

- **Known exploited is not the sightings ratio.** `/api/stats/kev` counts catalogue
  membership. `/api/stats/sighting/exploitation_ratio` counts exploitation *sightings*.
- **EUVD is not a second population.** EUVD records are mostly CVE aliases: never add
  EUVD figures to CVE figures.

## Preparing the figures

```bash
# counts, rankings and CVSS distributions (--source <name> to limit it)
$ poetry run index_vulnerabilities

# time to exploit, known-exploited per period, and known-exploited sets for non-EUVD sources
$ poetry run euvd --stats-scan --stats-source cvelistv5
```

Run the reindex once on a new, restored or upgraded instance before trusting the
counters. Cron lines are in
[Prepared statistics figures](command-line-interface.md#prepared-statistics-figures).

See also the [assigner scorecard](scorecard.md), the [catalogue scorecard](kev-scorecard.md)
and [Metrics](metrics.md). `/metrics` exports the small figures, not the rankings.
