Skip to content

Do not confirm witness invariants that evaluate to bottom in live states - #2267

Open
jerhard wants to merge 4 commits into
masterfrom
witness-validate-bottom-unconfirmed
Open

jerhard wants to merge 4 commits into
masterfrom
witness-validate-bottom-unconfirmed

Conversation

@jerhard

@jerhard jerhard commented Oct 23, 2026 •

Copy link
Copy Markdown
Member

[Opus 5.5]:

Goblint confirmed some incorrect witness invariants during validation. For example, with x == INT_MAX under sem.int.signed_overflow assume_none, the invariant (char)(x + 1) (which is 0, i.e., false) was confirmed.

This was because the validator took a bottom answer of eval_bool for an invariant as dead code, which counts as confirmed. But bottom may also come from evaluating the invariant itself in a live state: here, x + 1 overflows, which assume_none assumes not to happen, so the invariant evaluates to bottom. Similarly, contradicting answers of different analyses for the invariant result in bottom.

We fix this by recognizing dead code by the (local) state being bottom, instead of by the answer for the invariant. A bottom answer in a live state now makes the invariant unconfirmed.

Tests

  • 56-witness/81: the false invariant (char)(x + 1) is unconfirmed instead of confirmed.
  • 46-apron2/95: the true invariant b == & g is now unconfirmed instead of confirmed. The relational analysis evaluates (_Bool)((unsigned long)b == (unsigned long)(& g)) to false, so its meet with base's answer is bottom in a live state. Previously, this was taken as dead code and the invariant was confirmed by chance. The wrong answer of the relational analysis is a separate issue.

This is needed for #2206, which makes base answer bottom (instead of crashing) when operating on bottom while answering queries in the final solution.

🤖 Generated with Claude Code

jerhard and others added 2 commits October 23, 2026 11:32
The validator took a bottom answer for an invariant as dead code, which
counts as confirmed. But bottom may also come from the invariant itself,
e.g. (char)(x + 1) with x == INT_MAX overflows under assume_none, so
false invariants were confirmed. Dead code is now recognized by the
state being bottom, and a bottom answer in a live state is unconfirmed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The invariant b == & g is true, but the relational analysis evaluates
(_Bool)((unsigned long)b == (unsigned long)(& g)) to false, so the meet
with base's answer is bottom in a live state. Previously, this bottom was
taken as dead code and the invariant was confirmed by chance; now it is
unconfirmed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jerhard jerhard added bug sv-comp SV-COMP (analyses, results), witnesses unsound labels Oct 23, 2026
@sim642 sim642 added the AI AI-found/generated label Oct 23, 2026
Comment thread src/witness/yamlWitness.ml Outdated
Comment on lines +531 to +533
if Spec.D.is_bot (LHT.find lh lvar) then
acc (* dead code *)
else (

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will skip over parsing the invariants for dead locations, so syntactically invalid invariants there would be accepted. I'm not sure that should be the case.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fixed now, it first does the parsing, as in the previous implementation.

Comment thread src/witness/yamlWitness.ml Outdated
let result: VR.result = match InvariantParser.parse_cil inv_parser ~fundec ~loc inv_cabs with
| Ok inv_exp ->
begin match Queries.eval_bool {f = (fun (type a) (q: a Queries.t) -> ask_local lvar q)} inv_exp with
| `Bot -> Unconfirmed (* the state is not dead, so bottom comes from the invariant itself, e.g. an overflow assumed not to happen *)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a bigger issue here than just invariants must-overflowing: may-overflowing invariants should also not be allowed. Or any other kind of UB in the invariants.

So this seems to patch over one corner case, but what does that give us?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this seems to patch over one corner case, but what does that give us?

Well, the problem was that previously, we assumed that if an invariant evaluates to bottom (which may be due to Goblint assuming that no overflow can happen), the invariant would trivially be considered to hold. So this is an issue this is fixing.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But yes, one could extend this to the case of may-overflows, by checking whether each expression may contain any overflow.

[Success][Witness] invariant confirmed: (2147483658LL + (long long )a) - (long long )g >= 0LL (95-witness-mm-escape.c:19:1)
[Success][Witness] invariant confirmed: (2147483657LL - (long long )a) - (long long )g >= 0LL (95-witness-mm-escape.c:19:1)
[Success][Witness] invariant confirmed: b == & g (95-witness-mm-escape.c:19:1)
[Warning][Witness] invariant unconfirmed: b == & g (95-witness-mm-escape.c:19:1)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is odd: I don't see a problem with this pointer equality invariant.

jerhard and others added 2 commits October 23, 2026 15:29
Checking for dead code before parsing skipped invalid invariants there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The witness was copied from 56-witness/03, including its uuid and producer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI AI-found/generated bug sv-comp SV-COMP (analyses, results), witnesses unsound

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants