Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
103 commits
Select commit Hold shift + click to select a range
16f299f
Sequence battery: an arm for what a new brain composes
muellerberndt Nov 9, 2026
c5185f9
Heads chamber: composed attention functions in benchmarks/heads (#269)
muellerberndt Nov 9, 2026
509f8cf
Heads chamber: the cortex read at unit norm or scaled, as supports (#…
muellerberndt Nov 9, 2026
7c66acc
Heads chamber: a lesson's step on the maps' cortex columns scaled, as…
muellerberndt Nov 9, 2026
44fc02b
Vocabulary chamber in benchmarks/vocabulary, with the read's logits b…
muellerberndt Nov 9, 2026
864e9e3
Learner: a synapse rate of zero holds the synapse, its moments and it…
muellerberndt Nov 9, 2026
147de47
Add learning_running_error and declare the lesson-step protocol (#258)
muellerberndt Nov 9, 2026
9175d91
Add the device ledger: every host and device crossing of a moment, by…
muellerberndt Nov 9, 2026
2d010c8
Crossing chamber: the keyed rows, the conjunction's decoder and the m…
muellerberndt Nov 9, 2026
af02a87
Declare the exposure-parity protocol and its instrument (#270)
muellerberndt Nov 9, 2026
fda4263
learning_compiled_moment (opt-in): the recall read, its address lesso…
muellerberndt Nov 9, 2026
9bb5bea
Add the goal-route chamber (#257)
muellerberndt Nov 9, 2026
bd8e2e3
Held tokens: R1's six tasks with the read and the vote at answers no …
muellerberndt Nov 9, 2026
968a02b
Declare the goal-route protocol (#257)
muellerberndt Nov 9, 2026
a5c9e93
Recall: the copy pathway leaves (#281)
muellerberndt Nov 9, 2026
d2f075c
Compiled moment: the carried gains' contrast, the lesson's synapse co…
muellerberndt Nov 9, 2026
06bdd98
Learner: learning_surprise_share leaves (#281)
muellerberndt Nov 9, 2026
1eea37d
Say routes with near between places, as the facts say them (#257)
muellerberndt Nov 9, 2026
2fe9008
recall: the maps read the cortex at unit norm, opt-in recall_cortex_n…
muellerberndt Nov 9, 2026
dfb4d95
Declare the route chamber's life at 8,000 batches (#257)
muellerberndt Nov 9, 2026
6f24279
Read the battery's n-back at its taught lengths (#258)
muellerberndt Nov 9, 2026
0173d17
Heads chamber: every port's gain at birth, as an emulation (#269)
muellerberndt Nov 9, 2026
953c229
Crossing chamber: the read's synapses into the cortex frozen as well,…
muellerberndt Nov 9, 2026
28223ab
Withdraw learning_running_error after its declared confirmation (#258)
muellerberndt Nov 9, 2026
9e4677b
Compose: attention=True folds into the settings it named (#281)
muellerberndt Nov 9, 2026
f49dd9b
Heads chamber: the gains set once at their release, a support (#269)
muellerberndt Nov 9, 2026
323eb73
Heads chamber: a task's tokens declared without a gain (#269)
muellerberndt Nov 9, 2026
857be36
Read an outcome's checks once on torch; calibrate an unseen CUDA card…
muellerberndt Nov 9, 2026
25edda4
Recall: recall_next=False leaves; the guides state nothing that left …
muellerberndt Nov 9, 2026
164cb45
Sequence battery: the composed arm in its README
muellerberndt Nov 9, 2026
31b4d78
Time a calibration step that learns from the step before it
muellerberndt Nov 9, 2026
49da78a
Record the goal-route result: planning does not pay (#257)
muellerberndt Nov 9, 2026
cb8d5bd
Guide: many lives at once, the calibration's readings; changelog for …
muellerberndt Nov 9, 2026
684771d
docs: the cortex-norm snippet runs on its own
muellerberndt Nov 9, 2026
079d3b3
Take the memory's device step only where its trace rests on the devic…
muellerberndt Nov 9, 2026
df2e089
Device benchmark readings on an A10G and an M4; benchmark tests
muellerberndt Nov 9, 2026
da0df33
Compiled moment: the read on packed maps through one product, the dri…
muellerberndt Nov 9, 2026
f483996
Heads chamber: the released gains one per port, a support (#269)
muellerberndt Nov 9, 2026
cf9af80
Compiled moment: typed kernel decorators, the memories' flag declared…
muellerberndt Nov 9, 2026
3f585a2
Grammar chamber: cost.py reads a pass's seconds of the default brain,…
muellerberndt Nov 9, 2026
8f00573
Heads chamber: the gains at birth one per port (#269)
muellerberndt Nov 9, 2026
fa585df
Document the lesson-step and goal-route readings (#257, #258)
muellerberndt Nov 9, 2026
bbca62d
Revert "Recall: recall_next=False leaves; the guides state nothing th…
muellerberndt Nov 9, 2026
865c25d
Revert "Compose: attention=True folds into the settings it named (#281)"
muellerberndt Nov 9, 2026
c0d135a
Revert "Learner: learning_surprise_share leaves (#281)"
muellerberndt Nov 9, 2026
e114306
Docs: the memory guide states nothing of the copy pathway; the CHANGE…
muellerberndt Nov 9, 2026
7821a98
cost.py: --parts reads one brain's pass by part from sampled stacks
muellerberndt Nov 9, 2026
0d396e6
recall: the gains released once the read has formed, opt-in recall_ga…
muellerberndt Nov 9, 2026
af98b19
cost.py: the parts timed at their functions, the kernels at their lau…
muellerberndt Nov 9, 2026
d53b9ab
Record the exposure-parity result and its located mechanisms (#270)
muellerberndt Nov 9, 2026
20b943c
Word the rehearsal's replayed moments per row (#270)
muellerberndt Nov 9, 2026
445483f
Readings of the attention lane: heads, crossing and held tokens; docs…
muellerberndt Nov 9, 2026
e948b45
Lives: independent lives over one topology, stepped together on a tor…
muellerberndt Nov 9, 2026
821420d
Public text of the attention lane in the state-only style
muellerberndt Nov 9, 2026
88d37ed
Add memory_fade: store records that fade with disuse, slower the deep…
muellerberndt Nov 9, 2026
c5e8289
docs: the compiled moment in what a moment costs, its seconds and par…
muellerberndt Nov 9, 2026
e19d52c
Lives in the device benchmark and the API reference; refuse MPS
muellerberndt Nov 9, 2026
2ea6b93
Lives: the guide, the readings on an L4 and an M4, reset; the L4's ca…
muellerberndt Nov 9, 2026
73853fb
Document where the default composition's falls come from (#280)
muellerberndt Nov 9, 2026
ee99dd0
Lives take CUDA where there is a card, else the torch CPU device; run…
muellerberndt Nov 9, 2026
3718401
Vocabulary chamber: the declared reading at 1,000 words and the fall …
muellerberndt Nov 9, 2026
95b3c8a
Lives hold the default composition: the recall memory, the carried ga…
muellerberndt Nov 9, 2026
67abec3
Lives tests for the default composition and its variants; teacher mom…
muellerberndt Nov 9, 2026
f38417d
Merge lane-096-simplify: recall_copy removed, a zero synapse_rate hol…
muellerberndt Nov 9, 2026
14e9492
Lives: one batched product per range reached; docs of what Lives holds
muellerberndt Nov 9, 2026
fec8f2b
Merge lane-096-research: the lesson-step, goal-route and exposure-par…
muellerberndt Nov 9, 2026
0909559
Merge lane-096-retention: opt-in memory_fade, and where the compositi…
muellerberndt Nov 9, 2026
903b220
Merge lane-096-attention: opt-in recall_cortex_norm, recall_gain_afte…
muellerberndt Nov 9, 2026
2dad1b3
Cue a fading store by the next observations when learn runs on its ow…
muellerberndt Nov 9, 2026
1df37f1
API reference: the Recall signature without copy, which left with the…
muellerberndt Nov 9, 2026
a5fd507
Merge lane-096-retention at 2dad1b3: a fading store cued by the next …
muellerberndt Nov 9, 2026
e574fb1
AGENTS.md: the second goal epic is saying sentences formed in the bra…
muellerberndt Nov 9, 2026
b042b2d
Compiled moment: a moment in fewer calls
muellerberndt Nov 9, 2026
644f15f
Compiled moment: the position term's indices unsigned
muellerberndt Nov 9, 2026
7a3c214
Lives: the API reference and the benchmark's notes for the default co…
muellerberndt Nov 9, 2026
308cb83
Lives: sweeps that hold no life skip the hold's operations
muellerberndt Nov 9, 2026
4ecc474
docs, CHANGELOG: the compiled moment's seconds alone on a quiet core,…
muellerberndt Nov 9, 2026
6f4c2a0
Merge lane-096-cpu at 4ecc474: opt-in learning_compiled_moment
muellerberndt Nov 9, 2026
a8207c7
Compiled moment: the three sites mypy refuses, typed (#282)
muellerberndt Nov 9, 2026
fd02a02
Compiled moment: a greedy act carries the cortex at unit norm under r…
muellerberndt Nov 9, 2026
f3a26ff
Lives: one configuration per sweep key and mark, so a settle's graphs…
muellerberndt Nov 9, 2026
05b8f98
Lives of the default composition: the guide, the readings on an L4 an…
muellerberndt Nov 9, 2026
6852036
memory_fade: only a row that writes deepens its records; note the are…
muellerberndt Nov 9, 2026
680eecc
Compiled moment: an act, and a lesson's judgment, each in one compile…
muellerberndt Nov 9, 2026
aeb972b
Merge lane-096-retention at 6852036: only a row that writes deepens i…
muellerberndt Nov 9, 2026
d39985e
Merge lane-096-device at 05b8f98: the device ledger, an outcome's che…
muellerberndt Nov 9, 2026
65fa89b
Device benchmarks: the test loads its life module by path under its o…
muellerberndt Nov 9, 2026
62052c1
Lives: a rate of zero holds each life's optimizer moments on a torch …
muellerberndt Nov 9, 2026
9bf5abe
Lives and the backends guide name the options of the other 0.96 lanes…
muellerberndt Nov 9, 2026
5cb1909
memory_fade: a fading store's brain saves checkpoint format cadence-b…
muellerberndt Nov 9, 2026
feb9965
Merge lane-096-retention at 5cb1909: a brain whose store fades saves …
muellerberndt Nov 9, 2026
aea4513
Compiled moment: the streams' first act in the one call, a lesson of …
muellerberndt Nov 9, 2026
80d2219
Compiled moment: a taught lesson's three phases and their checks in o…
muellerberndt Nov 9, 2026
d52f202
Merge release-0.96 into lane-096-cpu: the one-call act carries the co…
muellerberndt Nov 9, 2026
c1ba494
Tests: every compiled path beside each other 0.96 option
muellerberndt Nov 9, 2026
3497b73
docs, CHANGELOG: the compiled moment's one-call paths, their seconds …
muellerberndt Nov 9, 2026
8eb5426
docs: the compiled moment's remaining seconds against the 2.5 bar, st…
muellerberndt Nov 9, 2026
ebf2b12
Compiled moment: the one-call act returns its action without a cast (…
muellerberndt Nov 9, 2026
cf18c25
Compiled moment: a taught lesson in one compiled call, from its phase…
muellerberndt Nov 9, 2026
3a879ef
Compiled moment: less Python around the one-call lesson and the act a…
muellerberndt Nov 9, 2026
637f72f
docs, CHANGELOG: the compiled moment with a taught lesson in one call…
muellerberndt Nov 9, 2026
ed30e3e
Release 0.96.0: many lives on a device, the compiled moment, records …
muellerberndt Nov 9, 2026
fce15d5
CHANGELOG: date 0.96.0 to its release day
muellerberndt Nov 10, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -291,8 +291,8 @@ The plan lives on GitHub; no local plan documents are kept.

- **The roadmap, [#109](https://github.com/muellerberndt/cadence/issues/109):** the two goals, the release line, the order of
work and the parked research.
- **Two goal epics:** [#118](https://github.com/muellerberndt/cadence/issues/118), a Doom player, and [#173](https://github.com/muellerberndt/cadence/issues/173), a
language model that understands sequences and grammar. Each states its definition of
- **Two goal epics:** [#118](https://github.com/muellerberndt/cadence/issues/118), a Doom player, and [#173](https://github.com/muellerberndt/cadence/issues/173),
saying sentences formed in the brain, as a person does. Each states its definition of
done, its stages and the capability contracts folded into it.
- **A milestone per release**, holding the library issues that release delivers, each
with its gate.
Expand Down
248 changes: 248 additions & 0 deletions CHANGELOG.md

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ action is read from that equilibrium, and every synapse learns from the differen
settles, with no backward pass and no training/inference switch.

```sh
python -m pip install cadence-net==0.95.2
python -m pip install cadence-net==0.96.0
```

```python
Expand Down Expand Up @@ -57,7 +57,7 @@ episodic memory, a learned code, images.
The [documentation index](https://github.com/muellerberndt/cadence/blob/main/docs/README.md)
lists the rest: wide inputs, growing and grafting, evaluation, your own anatomy, how the
brain settles, backends; the
[released documentation](https://github.com/muellerberndt/cadence/blob/v0.95.2/docs/README.md)
[released documentation](https://github.com/muellerberndt/cadence/blob/v0.96.0/docs/README.md)
describes the APIs of the installed package. Integrated learned world prediction, repair
localized to what failed and cheap stable operation are work in progress; Cadence is alpha
research software.
Expand Down
91 changes: 47 additions & 44 deletions benchmarks/attention/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,18 @@ ran, archived by content hash under `results/sources/`. A receipt states what wa
code it binds, and a setting it names need not exist in this version. The scripts of a campaign
(`admission.py`, `heads_admission.py`, the `*_campaign.py` files and their protocols, tests and
`verify.py`) are in release 0.94.0, whose tree runs the commands below. Five instruments stay
here, runnable and tested: `copy_tokens.py`, `own_action.py`, `look_back.py`, `held_heads.py` and
here, runnable and tested: `own_action.py`, `look_back.py`, `held_heads.py` and
`contextual_choice.py`.

## Retrieved content becomes the answer: the copy pathway (`recall_copy`, `recall_tokens`)
## Retrieved content becomes the answer: the copy pathway (retired)

The [copy-tokens benchmark](copy_tokens.py) asks two things of eight tokens. A token is heard,
two other tokens follow, and a cue asks for the heard token back: the cue's key names the heard
slot, so the read is right before any lesson, and tokens 5 to 7 are never taught as answers.
And a cue names a token of a sequence of five heard once and asks for the token that followed
it, which the read holds on its next port while its recall port holds the cue's own token.
Lessons at the cue only, 300 and 600 per founder, `teaching="normalized"` with
`learning_surprise=True`, `episodic=False`. Its [receipt](results/copy-tokens.json), six founders
of one module of 24 neurons:
The copy pathway (`recall_copy`) left the library in 0.96.0, and its instrument,
`copy_tokens.py`, with it; [the learned value path](../../docs/memory.md#the-learned-value-path-opt-in)
(`recall_gain`) is the library's path from a read to its token's action. Its
[receipt](results/copy-tokens.json) stays, six founders of one module of 24 neurons, and verifies
on archived bytes with its archived checker:
`python benchmarks/provenance.py --historical benchmarks/attention/results/copy-tokens.json`.
Release 0.95.2 runs the instrument.

| Arm | Untaught produced | Taught | Succession | Untaught successor |
| --- | --- | --- | --- | --- |
Expand All @@ -29,40 +28,6 @@ of one module of 24 neurons:
| `recall_copy=0.5`, no tokens | 0.00 | 0.57 to 0.99 | 0.93 to 1.00 | 0.00 |
| `recall_copy=1.2`, no tokens | 0.00 | 0.22 to 0.87 | 0.86 to 0.97 | 0.00 |

Uniform random answers 0.125. The founder's read names the untaught token at 0.32 to 1.00 (the
founder of seed 2 moved its own read with its lessons) and never produces it. Two founders of a
deep apical brain read the same: untaught 1.00, succession 0.94 and 0.98 against 0.97 and 0.99.
Every declared gate passes; the result is recomputed by `test_copy_tokens.py`.

The design was chosen on these tasks, and the alternatives read as follows (development
readings, outside the receipt unless stated):

- An identity on both ports at once (founder 0, identity 1.0 to 4.0): the next port of the heard
token's slot holds the distractor that followed it, and the newborn produced the heard token on
0.53 to 0.67 of untaught questions; 300 lessons took it to 0.00 to 0.13. The identity starts on
the recall port only.
- No echo suppression (the echo synapses zeroed at birth, six founders): untaught 1.00, but
succession 0.79 to 0.95, mean 0.87, against 0.94 to 0.99 with it: the recall port answers a cue
with its own token.
- Each port's identity tied into one learned gain (`Learner(tie_groups=...)`): on the receipt's
six founders it produced the untaught tokens at 1.00 as the untied pathway does, with succession
0.91 to 0.98 against 0.94 to 0.99, so the ties were left out.
- The apical compartment as a bound was not needed: on the deep apical brain the pathway, which
lands basally, keeps succession, and a bound relative to the motor region's bottom-up drive would
cap the copy of a token whose action that drive never raised. It was not measured.

The first run of this receipt declared a gate that the founder's read of the untaught tokens is
0.80 or better on every founder; founder 2 read them at 0.32, and the gate was replaced by its
production alone (0.00 on every founder), with the reads reported beside it.

The receipt was recorded on the NumPy solver (`CADENCE_FUSED=0`); the compiled dense kernel gave
the same answers, with the reported gains differing in their last digits.

```sh
CADENCE_FUSED=0 PYTHONPATH=src python benchmarks/attention/copy_tokens.py --workers 4 \
--out /tmp/copy-tokens.json
```

## Its own actions in the slots (`recall_action`)

The [own-action benchmark](own_action.py) gives a brain the same observation at every moment, so
Expand Down Expand Up @@ -290,6 +255,44 @@ teaching-policy repair now passes on both development
and all twenty fresh confirmation founders. Numerical checks and that acquisition
control alone cannot admit a library attention option.

## The crossing: a question whose fact depends on the context (`crossing.py`)

[`crossing.py`](crossing.py) reads [#267](https://github.com/muellerberndt/cadence/issues/267): two
records (`dog V0`, `cat V1`, the order drawn per stream), a context `X_c` and a question `Q_k`; the
answer, at a `GO` moment after the question, is the value of the fact `c xor k`. Neither bit names the
fact alone, so a reader that adds what it reads says one of the two values, .50. A life teaches one
lesson a batch on the answer and reads fresh rows; `decode` fits a ridge decoder of `c`, `k` and
`c xor k` on the cortex the answer's read is addressed from, and measures the crossing term of the
four cells' means against the spread of the rows around their cell. The supports `frozen_cortex` and
`frozen_all` restore the synapses into the association and prefrontal regions after every lesson
(the second the read's synapses as well), a diagnostic and no candidate. The matched LSTM and
transformer hear the same rows, one Adam step a batch.

Declared reading, six fresh founders (2670901 to 2670906), one 32-vCPU box, 30,000 batches, the
grammar chamber's brain at the deep teaching point with a held read:

| Learner | Answers at 30,000, per founder | At .90 | Decode of `c`, `k`, `c xor k` at 30,000 | Crossing term / spread: newborn; at 30,000 | Sweeps per act |
| --- | --- | --- | --- | --- | --- |
| The read's rule | .60 .54 .46 .49 .45 .45 | 0 of 6 | .63-.88, .98-1.00, .46-.58 | .02-.03 / .23-.30; .04-.18 / 2.83-3.02 | 50 to 100 |
| The rule, `recall_cortex_norm` | .52 .51 .48 .57 .41 .51 | 0 of 6 | .58-.64, .86-1.00, .44-.55 | .03-.04 / .34-.42; .01-.03 / .54-.61 | 68 to 101 |
| The rule, every synapse into the cortex held (`frozen_all`) | .58 .44 .55 .53 .42 .50 | 0 of 6 | 1.00, 1.00, .73-.90 | .02-.03 / .23-.30; .03-.07 / .46-.71 | 59 to 69 |
| LSTM | 1.00 on 6 (from 10,000 on 5) | 6 of 6 | | | |
| Transformer | .49 .95 .53 1.00 .51 .56 | 2 of 6 | | | |

At every reading of every brain arm the answer sits at the floor; on the two plastic arms erasing the
fact asked takes it to .00 and erasing the other leaves .60 to 1.00, so the answer is lost at the
read's selection. Two terms meet there:

- **The lessons bury the conjunction.** The newborn's cortex holds it linearly (decode .95 to 1.00 at
birth); under lessons the rows' spread around their cell grows about tenfold, from .23-.30 to
2.8-3.0, while the crossing term stays at .04 to .18, and the decode of `c xor k` falls to .46-.58.
The cortex at unit norm keeps the spread at .54-.61 and the term falls with it (.01-.03).
- **A conjunction the cortex holds is not selected.** With every synapse into the cortex held, the
conjunction stays decodable (.73 to .90) at a term of .03 to .07 against a spread of .46 to .71, and
the held address reads .42 to .58: the address lesson's step on the cortex columns, a linear
map driven by the read's own error at `GO`, does not find a direction of that size within 30,000
batches.

## Equations and boundaries

With witnessed keys `K`, contents `V` and a zero-content null slot, let
Expand Down
Loading
Loading