Skip to content

Commit f01556d

Browse files
authored
Merge pull request #205 from VIZ-Blockchain/chain/validate-invariants
chain: implement validate_invariants() and wire --validate-database-invariants to it
2 parents 022dd39 + e445146 commit f01556d

9 files changed

Lines changed: 383 additions & 4 deletions

File tree

‎@l10n/ru/docs/plugins/chain.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,8 @@ json_rpc::plugin
2929
| `--auto-recover-from-snapshot` | По умолчанию `true`. Автоматически восстанавливаться во время работы при обнаружении повреждения shared memory во время обработки или генерации блоков, без перезапуска. Отключить через `--no-auto-recover-from-snapshot`. |
3030
| `--resync-blockchain` | Очистить и shared memory, и block log; начать с genesis или из снимка. Деструктивно — использовать только при восстановлении после полной потери данных. |
3131
| `--check-locks` | Проверить порядок блокировок (только для разработки). |
32-
| `--validate-database-invariants` | Выполнять проверки консистентности базы данных на каждом блоке (очень медленно; только для разработки). |
32+
| `--validate-database-invariants` | Пересчитывать инварианты эмиссии — SHARES (`dgp.total_vesting_shares` против суммы вестинга на аккаунтах) и TOKEN (`dgp.current_supply` против всех пулов, держащих токены вне балансов аккаунтов, включая пулы prediction markets) — один раз при старте и после каждого применённого блока. Расхождение при старте прерывает запуск; после блока оно пишется в лог как `DATABASE INVARIANT VIOLATED`, а нода продолжает работу. Один линейный проход по всему состоянию на каждый блок (все аккаунты и все пулы prediction markets); разбивка по пулам печатается при старте и при любом расхождении, но не на чистом блоке. Только для разработки. Состоянию с замёрзшей до-фикс пылью нужно `--validate-database-invariants-residual` (следующая строка). |
33+
| `--validate-database-invariants-residual` | Ожидаемый остаток TOKEN для `--validate-database-invariants`. Состояние с замёрзшей до-фикс пылью prediction markets (бывает в тестнет-снапшотах) требует то же значение, что `--snapshot-pm-legacy-residual`; `0` (по умолчанию) — строгий, до сатоши, и единственно верный для мейннета. |
3334

3435
### Опции файла конфигурации
3536

‎@l10n/zh-CN/docs/plugins/chain.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,8 @@ json_rpc::plugin
2929
| `--auto-recover-from-snapshot` | 默认 `true`。当区块处理或生成期间检测到共享内存损坏时,无需重启即可自动运行时恢复。通过 `--no-auto-recover-from-snapshot` 禁用。 |
3030
| `--resync-blockchain` | 清除共享内存和区块日志;从 genesis 或快照重新开始。具有破坏性 — 仅在从完全数据损失中恢复时使用。 |
3131
| `--check-locks` | 验证锁排序(仅开发用)。 |
32-
| `--validate-database-invariants` | 在每个区块上运行数据库一致性检查(非常慢;仅开发用)。 |
32+
| `--validate-database-invariants` | 重新计算供应不变量 — SHARES(`dgp.total_vesting_shares` 与各账户 vesting 之和)和 TOKEN(`dgp.current_supply` 与所有持有账户余额之外代币的池,包括预测市场池)— 启动时一次,并在每个已应用区块之后。启动时的偏差会中止启动;区块之后的偏差记录为 `DATABASE INVARIANT VIOLATED`,节点继续运行。每个区块对整个状态做一次线性遍历(所有账户及全部预测市场池);按池的明细在启动时和任何偏差时打印,干净的区块不打印。仅开发用。带有冻结尘埃的状态需要 `--validate-database-invariants-residual`(下一行)。 |
33+
| `--validate-database-invariants-residual` | `--validate-database-invariants` 的预期 TOKEN 供应残差。带有修复前冻结预测市场尘埃的状态(部分测试网快照)使用与 `--snapshot-pm-legacy-residual` 相同的值;`0`(默认)为严格、精确到 satoshi,也是主网唯一正确的值。 |
3334

3435
### 配置文件选项
3536

‎docs/plugins/chain.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,8 @@ These are one-time recovery or maintenance operations; they cause the node to pe
2929
| `--auto-recover-from-snapshot` | Default `true`. Automatically recover at runtime when shared memory corruption is detected during block processing or generation, without a restart. Disable with `--no-auto-recover-from-snapshot`. |
3030
| `--resync-blockchain` | Wipe both shared memory and the block log; start from genesis or from a snapshot. Destructive — use only when recovering from complete data loss. |
3131
| `--check-locks` | Validate lock ordering (development only). |
32-
| `--validate-database-invariants` | Run database consistency checks on every block (very slow; development only). |
32+
| `--validate-database-invariants` | Recompute the supply invariants — SHARES (`dgp.total_vesting_shares` vs the sum of account vesting) and TOKEN (`dgp.current_supply` vs every pool holding tokens outside account balances, prediction-market pools included) — once at startup and after every applied block. A divergence at startup aborts the start; after a block it is logged as `DATABASE INVARIANT VIOLATED` and the node keeps running. One linear pass over the whole state per block (all accounts plus every prediction-market pool); the per-pool breakdown is printed at startup and on any divergence, never on a clean block. Development only. A state carrying frozen pre-fix dust needs `--validate-database-invariants-residual` (next row). |
33+
| `--validate-database-invariants-residual` | Expected TOKEN supply residual for `--validate-database-invariants`. A state carrying frozen pre-fix prediction-market dust (some testnet snapshots) uses the same value as `--snapshot-pm-legacy-residual`; `0` (the default) is strict, satoshi-exact, and the only valid value on mainnet. |
3334

3435
### Config file options
3536

‎libraries/chain/CMakeLists.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ add_library(graphene_chain ${VIZ_LIBRARY_TYPE}
4343
# As database takes the longest to compile, start it first
4444
database.cpp
4545
database_hardfork.cpp
46+
db_invariants.cpp
4647
fork_database.cpp
4748

4849
chain_evaluator.cpp

‎libraries/chain/db_invariants.cpp‎

Lines changed: 216 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,216 @@
1+
/*
2+
* Supply-invariant checks over the live state (issue #204).
3+
*
4+
* database::validate_invariants() was declared in database.hpp but its
5+
* definition was removed in 2018 (faa45ff4 "Remove unused methods") and the
6+
* declaration outlived the removal, so --validate-database-invariants accepted
7+
* the flag and never ran anything. This TU restores the definition.
8+
*
9+
* The accounting mirrors the post-import checks in plugins/snapshot/plugin.cpp
10+
* on purpose: that copy stays where it is, as an independent witness at import
11+
* time (owner decision 2026-10-30 — "let it watch additionally"). If a PM pool
12+
* is ever added or re-scoped, BOTH copies must change together; the numbers in
13+
* the two logs are expected to match object for object.
14+
*/
15+
#include <graphene/chain/database.hpp>
16+
#include <graphene/chain/account_object.hpp>
17+
#include <graphene/chain/chain_objects.hpp>
18+
#include <graphene/chain/global_property_object.hpp>
19+
#include <graphene/chain/invite_objects.hpp>
20+
#include <graphene/chain/pm_objects.hpp>
21+
#include <graphene/chain/validator_objects.hpp>
22+
23+
namespace graphene {
24+
namespace chain {
25+
26+
void database::validate_invariants(int64_t pm_legacy_residual, bool verbose) const {
27+
const auto& dgp = get_dynamic_global_properties();
28+
29+
// ── (1) SHARES ────────────────────────────────────────────────────
30+
// dgp.total_vesting_shares is minted at inflation; account.vesting_shares is the
31+
// only place it lives. Delegation objects redistribute vests that are ALREADY in
32+
// the delegator's account.vesting_shares (delegated_/received_ net to zero
33+
// network-wide), so they are deliberately not summed — adding them would
34+
// double-count.
35+
int64_t summed_vesting_shares = 0;
36+
{
37+
const auto& idx = get_index<account_index>().indices();
38+
for (auto itr = idx.begin(); itr != idx.end(); ++itr)
39+
summed_vesting_shares += itr->vesting_shares.amount.value;
40+
}
41+
const int64_t expected_vesting_shares = dgp.total_vesting_shares.amount.value;
42+
const int64_t shares_delta = summed_vesting_shares - expected_vesting_shares;
43+
44+
// ── (2) TOKEN ─────────────────────────────────────────────────────
45+
// dgp.current_supply must equal every pool that holds TOKEN outside account
46+
// balances. Base (chain-core) pools were verified satoshi-exact against six
47+
// mainnet snapshots in PR #126; PM never mints, it only MOVES tokens from
48+
// account.balance into a PM object field (or back), so the PM-held float must be
49+
// added here or the equality is off by exactly that float.
50+
int64_t acc_balance = 0, acc_reserved = 0;
51+
{
52+
const auto& idx = get_index<account_index>().indices();
53+
for (auto itr = idx.begin(); itr != idx.end(); ++itr) {
54+
acc_balance += itr->balance.amount.value;
55+
acc_reserved += itr->reserved_balance.amount.value;
56+
}
57+
}
58+
int64_t escrow_balance = 0, escrow_fee = 0;
59+
{
60+
const auto& idx = get_index<escrow_index>().indices();
61+
for (auto itr = idx.begin(); itr != idx.end(); ++itr) {
62+
escrow_balance += itr->token_balance.amount.value;
63+
escrow_fee += itr->pending_fee.amount.value;
64+
}
65+
}
66+
int64_t invite_balance = 0;
67+
{
68+
const auto& idx = get_index<invite_index>().indices();
69+
for (auto itr = idx.begin(); itr != idx.end(); ++itr)
70+
invite_balance += itr->balance.amount.value;
71+
}
72+
int64_t validator_pending = 0;
73+
{
74+
// current_supply already includes validator pending_stakeholder_reward
75+
// (credited at inflation, moved into total_vesting_fund at create_vesting).
76+
const auto& idx = get_index<validator_index>().indices();
77+
for (auto itr = idx.begin(); itr != idx.end(); ++itr)
78+
validator_pending += itr->pending_stakeholder_reward.value;
79+
}
80+
81+
const int64_t vesting_fund = dgp.total_vesting_fund.amount.value;
82+
const int64_t reward_fund = dgp.total_reward_fund.amount.value;
83+
const int64_t committee_fund = dgp.committee_fund.amount.value;
84+
const int64_t base_token = acc_balance + acc_reserved
85+
+ escrow_balance + escrow_fee + invite_balance
86+
+ validator_pending + vesting_fund + reward_fund + committee_fund;
87+
88+
// Minimal NON-OVERLAPPING PM holdings (duplicates excluded: market/outcome
89+
// bets_sum and CPMM reserves are aggregates of pm_bet rows; the empty-provider
90+
// lazy LP row duplicates pm_lazy_pool.allocated_balance; pm_lazy_deposit.principal
91+
// is a per-depositor claim on the lazy pool).
92+
int64_t pm_bets = 0;
93+
{
94+
const auto& idx = get_index<pm_bet_index>().indices();
95+
for (auto itr = idx.begin(); itr != idx.end(); ++itr)
96+
if (itr->status == 0 || itr->status == 5 || itr->status == 6) // active/queued/revealed-pending
97+
pm_bets += itr->amount.value;
98+
}
99+
int64_t pm_user_lp = 0;
100+
{
101+
const auto& idx = get_index<pm_liquidity_index>().indices();
102+
for (auto itr = idx.begin(); itr != idx.end(); ++itr)
103+
if (itr->status == 0 && itr->provider.size() > 0)
104+
pm_user_lp += itr->amount.value;
105+
}
106+
int64_t pm_lazy = 0;
107+
{
108+
const auto& idx = get_index<pm_lazy_pool_index>().indices();
109+
for (auto itr = idx.begin(); itr != idx.end(); ++itr)
110+
pm_lazy += itr->free_balance.value + itr->allocated_balance.value;
111+
}
112+
int64_t pm_commit = 0;
113+
{
114+
const auto& idx = get_index<pm_commit_index>().indices();
115+
for (auto itr = idx.begin(); itr != idx.end(); ++itr)
116+
if (itr->status == 0) // escrow held only while committed
117+
pm_commit += itr->escrow_amount.value;
118+
}
119+
int64_t pm_dispute = 0;
120+
{
121+
const auto& idx = get_index<pm_dispute_index>().indices();
122+
for (auto itr = idx.begin(); itr != idx.end(); ++itr)
123+
if (itr->status == 0) // fee held only while dispute open
124+
pm_dispute += itr->dispute_fee.value;
125+
}
126+
int64_t pm_forfeit = 0;
127+
{
128+
const auto& idx = get_index<pm_market_index>().indices();
129+
for (auto itr = idx.begin(); itr != idx.end(); ++itr)
130+
pm_forfeit += itr->forfeit_pool.value; // zeroed post-settlement
131+
}
132+
int64_t pm_insurance = 0;
133+
{
134+
const auto& idx = get_index<pm_oracle_index>().indices();
135+
for (auto itr = idx.begin(); itr != idx.end(); ++itr)
136+
pm_insurance += itr->insurance.value;
137+
}
138+
int64_t pm_leverage = 0;
139+
{
140+
const auto& idx = get_index<pm_leverage_position_index>().indices();
141+
for (auto itr = idx.begin(); itr != idx.end(); ++itr)
142+
if (itr->status == 0) // collateral+loan held only while active
143+
pm_leverage += itr->collateral.value + itr->loan.value;
144+
}
145+
// A settlement spans blocks, so a check can catch one mid-flight. Rows already
146+
// released no longer count above, and payouts already made are in account
147+
// balances — `escrow` is the difference and keeps both sides equal at any block
148+
// boundary. SIGNED: phase 3 pays early-exit claims out of a pot whose rows are
149+
// still standing, so it is legitimately negative until phase 4 releases them.
150+
int64_t pm_settling = 0;
151+
{
152+
const auto& idx = get_index<pm_settlement_index>().indices();
153+
for (auto itr = idx.begin(); itr != idx.end(); ++itr)
154+
pm_settling += itr->escrow.value;
155+
}
156+
const int64_t pm_token = pm_bets + pm_user_lp + pm_lazy + pm_commit
157+
+ pm_dispute + pm_forfeit + pm_insurance + pm_leverage + pm_settling;
158+
159+
const int64_t summed_token = base_token + pm_token;
160+
const int64_t expected_supply = dgp.current_supply.amount.value;
161+
const int64_t token_delta = summed_token - expected_supply;
162+
163+
// The per-pool breakdown is ~600 bytes. At a one-second block interval a silent
164+
// run would put ~50 MB a day into node.log for no information — so it is printed
165+
// when asked (startup, manual call) and on ANY divergence, never on a clean
166+
// per-block pass. A divergence of either sum is "interesting" here regardless of
167+
// the anchor: a delta equal to the anchor still has to be visible.
168+
if (verbose || shares_delta != 0 || token_delta != pm_legacy_residual) {
169+
ilog("Database invariants: head_block_num=${h} SHARES summed=${sv} expected=${ev} "
170+
"delta=${sd}; TOKEN current_supply=${cs} summed=${sm} (base=${bt} + pm=${pm}) "
171+
"delta=${d}. Base: acc_balance=${ab} acc_reserved=${ar} escrow_balance=${eb} "
172+
"escrow_fee=${ef} invite_balance=${ib} validator_pending=${vp} "
173+
"vesting_fund=${vf} reward_fund=${rf} committee_fund=${cf}. "
174+
"PM: bets=${pb} user_lp=${pl} lazy=${plz} commit=${pc} dispute=${pd} "
175+
"forfeit=${pf} insurance=${pi} leverage=${plv} settling=${pst}",
176+
("h", dgp.head_block_number)("sv", summed_vesting_shares)
177+
("ev", expected_vesting_shares)("sd", shares_delta)
178+
("cs", expected_supply)("sm", summed_token)("bt", base_token)("pm", pm_token)
179+
("d", token_delta)
180+
("ab", acc_balance)("ar", acc_reserved)("eb", escrow_balance)("ef", escrow_fee)
181+
("ib", invite_balance)("vp", validator_pending)("vf", vesting_fund)
182+
("rf", reward_fund)("cf", committee_fund)
183+
("pb", pm_bets)("pl", pm_user_lp)("plz", pm_lazy)("pc", pm_commit)
184+
("pd", pm_dispute)("pf", pm_forfeit)("pi", pm_insurance)("plv", pm_leverage)
185+
("pst", pm_settling));
186+
}
187+
188+
if (shares_delta != 0) {
189+
elog("Database invariant FAILED at block ${h}: SHARES conservation violated — "
190+
"summed account.vesting_shares (${sv}) != dgp.total_vesting_shares (${ev}), "
191+
"delta=${sd}",
192+
("h", dgp.head_block_number)("sv", summed_vesting_shares)
193+
("ev", expected_vesting_shares)("sd", shares_delta));
194+
}
195+
if (token_delta != pm_legacy_residual) {
196+
elog("Database invariant FAILED at block ${h}: TOKEN supply divergence — summed "
197+
"(${sm}) != current_supply (${cs}), delta=${d}, expected=${exp}, "
198+
"drift_from_anchor=${dr} (base_delta=${bd})",
199+
("h", dgp.head_block_number)("sm", summed_token)("cs", expected_supply)
200+
("d", token_delta)("exp", pm_legacy_residual)
201+
("dr", token_delta - pm_legacy_residual)
202+
("bd", base_token - expected_supply));
203+
} else if (token_delta != 0) {
204+
wlog("Database invariants: TOKEN delta=${d} matches the configured legacy anchor "
205+
"— tolerated, but the anchor must be 0 on mainnet.", ("d", token_delta));
206+
}
207+
208+
FC_ASSERT(shares_delta == 0 && token_delta == pm_legacy_residual,
209+
"Database invariant violated at block ${h}: SHARES delta=${sd}, TOKEN delta=${td} "
210+
"(expected ${exp}). The state no longer reconciles — refusing to continue.",
211+
("h", dgp.head_block_number)("sd", shares_delta)("td", token_delta)
212+
("exp", pm_legacy_residual));
213+
}
214+
215+
}
216+
} // namespace graphene::chain

0 commit comments

Comments
 (0)