Repository navigation
Vulnerability scan: CVE data, software mapping and monitoring - #216
Merged
Merged
Conversation
Name patterns matched as substrings and the first hit won, so a short pattern filed unrelated programs under a foreign product. The matching now works on words: - A pattern matches whole words in order; the pattern with the most words wins. A one-word pattern needs the vendor to agree or the word to be the whole name, and a name that follows "for" is an add-on. - A key match compares vendor and canonical name (no version, architecture or language), which becomes the key for manual mappings. - The fuzzy NVD name search no longer runs during a scan. A program without a mapping stays unmapped. - Mappings load as one immutable snapshot: concurrent readers share one load, a failed reload keeps the last good copy, and an empty result is never served. A scan that cannot read them is put back on the queue instead of filing every program as unknown. - Every program is mapped before the first CVE lookup, so a run that runs out of time has still filed each one. The inventory row takes the computed CPE on every count, set or cleared. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…wn ranges, one result writer CVE data - The NVD fetch asked for one page of 100, which is the oldest 100, and the age filter then dropped them. It now reads a full page and, for a product with more, the last page as well, behind its own HTTP policy and a time limit that covers the body. A timeout fails that request, not the scan. - A cache row keeps only the version ranges of its own product; the filter compares the product again when reading, so rows written earlier stop matching through another product's range. - Known-exploited CVEs are kept ahead of the size cap. - Cache rows and reports are written in replace mode. A merge left the other storage form of an earlier write in place and the reader returned it. - Not judged, and reported as such: Store app stubs with a placeholder version, ranges in another version scheme, and software editions a mapping excludes. The platform check on descriptions matches whole words. - The KEV lookup reads the loaded catalog and never fetches; a failed refresh is reported instead of swallowed. Scan and result - CVE data is read once per product. A product without data is fetched while the run has time, otherwise noted for the nightly refresh; the session is put back once and then reports it as not evaluated instead of clean. - A software entry lists every known-exploited CVE and the most urgent others and carries the full counts; the report is trimmed before it outgrows its table row. Sums and the WhiteGlove merge use the counts. - New counters leave out software marked as self-updating. - The queue handler and the re-scan store through one writer: the report is replaced or removed and the session's CveIndex rows follow it, keeping the detection date of rows that stay. The index takes the five most urgent CVEs per software plus every known-exploited one, and the fleet summary filters by date in the query. - The nightly refresh uses the same fill path as the scan, starts with products waiting for data and rows of the older shape, and removes rows no mapping names. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…, and two fixes they found Tests cover the paged NVD fetch and its policy, the cache fill path, the version filter (own product, editions, version schemes, platform words), report entries and their trimming, the WhiteGlove merge, the scan end to end over a mocked repository, the queue handler's postponements, the re-scan, the per-session index replace, the refresh walk, the KEV catalog and the write modes. Fixes: - A run that is out of time still reads the cache and only skips the fetch. Before, every remaining product was reported as waiting for data even when its data was cached. - An entry of a report stored in the earlier format gets its counts and the most-urgent-first list when it is merged. Before, trimming such an entry left nothing that said how many CVEs it stood for. The NVD client gets a test seam for its request pacing. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
get_software_inventory with scope "unmatched" advanced its cursor by the page size asked for. The backend serves at most 200 rows per page, so a larger page size skipped the difference without a hint. The cursor now advances by the rows that arrived and ends on an empty page. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…-updating products, bounded NVD loops Mapping - The backend keeps a reduced copy of the NVD product dictionary, replaced nightly when NVD publishes a new one and read as a stream. The NVD keyword search is removed: no program name is sent to NVD any more. - A program gets an automatic mapping only when its name is exactly one product's name, the product's cache row holds CVEs, none of them lacks a version, and the program does not occur only as a Store package. Deleting an automatic mapping puts the name on the ignore list. - rules/dist/cpe-mappings.json is the maintained list (410 mappings, editions, default set of self-updating products). An import checks the list, writes, then removes what the list no longer holds; a refused list leaves the mappings in service and is recorded as an ops event. cpe-mapping-seed.json stays until this backend is deployed. - The CPE on every inventory row follows the mappings: after each change and once a night. - The admin routes work on vendor plus canonical name. POST vulnerability/cpe-product-flags lets a Global Admin mark a product as self-updating; the value is stored only while it differs from the list's default. Report and rule - Findings in self-updating software are listed, but the actionable_* counters leave them out. ANALYZE-ID-003 (1.3.0) reads those counters, so a browser no longer triggers it. - A program without a version number and a Store package under an automatic mapping are listed as not evaluated instead of being compared. - The fleet summary leaves self-updating software out unless includeSelfUpdating is set, and says how much that is. NVD - A 403 is a refusal like a 429; a request that got no answer is told apart from a rejected one. - Every loop that asks NVD has a count bound, an unanswered bound and a time bound. The cache walk waits an open cooldown out once before its first request. - The scan filter and the dictionary share one list of the platforms that rule a Windows desktop out; the phone, embedded and ARM-only Windows lines are in it. Monitoring - New unsampled event SoftwareInventoryCpeReconciled; VulnerabilityDataSyncCompleted carries the dictionary, the automatic mappings and the parts that fell short. - Ops events VulnerabilityDataSyncDegraded (also from stored state: walk not finished for two days, dictionary older than a week), CpeMappingUnavailable (mappings unreadable or a list refused; once per UTC day, cause and severity) and CpeAutoMappingsCreated. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…-updating software is shown apart Session report - The headline numbers come from the report's own counts, so a shortened CVE list no longer changes them; each entry says how many CVEs it has in all next to the ones it lists. - Self-updating software forms a group of its own below the findings and is left out of the headline; the number of its CVEs stands next to it. - Programs that could not be judged are listed under "Not evaluated" with the reason. - A report written before these counts existed is read from its lists as before. Software > Vulnerabilities and the dashboard card - Self-updating software is left out by default. A line says how many products on how many devices that is and counts them in on request; their rows carry a mark. - The card says "None to update by hand" when only self-updating software was found. Admin > Software - Mapping and ignoring work on vendor plus canonical name. - Automatic mappings show their source and can be deleted; a product can be marked as self-updating. - A Global Reader gets the lists without the write controls. - The reseed result shows the inventory rows it changed and why a mapping list was refused. Privacy and terms pages - The NVD and EPSS lookups carry only public identifiers (a product identifier, a CVE number); the NVD product dictionary is named as a downloaded source. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…t the index holds - get_vulnerability_summary: new parameter includeSelfUpdating (default false, like the portal). The description names the default, the fields that say how much is left out, and that the index holds per product and device every known exploited CVE plus the five most urgent others. - search_sessions_by_cve: the description names the same limit of the index. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…t the column - Session report: the badge of a row is the worst CVSS band over all CVEs of the entry again, from the entry's counts on a newer report and from its list on an older one. The previous commit had switched it to the report's risk level, which a known exploited CVE raises to critical: rows of existing reports changed from HIGH to CRITICAL and contradicted the severity cards above them. - Admin > Software > Mapped: the actions column is wide enough for both buttons; Delete was cut off at the table's edge. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Its installer keeps the editor current on its own, like a browser. Findings stay listed in the report; the headline numbers and ANALYZE-ID-003 leave them out. A Global Admin can change the flag per product. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…ition Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…dating software was found Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes
CVE data
Mapping
rules/dist/cpe-mappings.jsonis the maintained list (410 mappings).cpe-mapping-seed.jsonstays until the backend is deployed and is removed afterwards.Report, portal, rule
Monitoring
VulnerabilityDataSyncDegraded,CpeMappingUnavailable,CpeAutoMappingsCreated.Measured
Production matcher and version filter over an export of the software inventory (9,864 rows, 216,304 installs):
5,435 installs lose a false match, 29,123 gain a mapping, 4,172 move to the right product.
Verification
errorCodesSyncis red only in a CRLF checkout, the file is untouched).Rollout
Backend, then "Reseed from GitHub" (the list is read from main), then "Refresh NVD Cache", then web, docs and MCP. Customer docs follow in the docs repository.
🤖 Generated with Claude Code