diff --git a/AGENTS.md b/AGENTS.md index 2f66ebbe..80e27483 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -291,8 +291,8 @@ The plan lives on GitHub; no local plan documents are kept. - **The roadmap, [#109](https://github.com/muellerberndt/cadence/issues/109):** the two goals, the release line, the order of work and the parked research. -- **Two goal epics:** [#118](https://github.com/muellerberndt/cadence/issues/118), a Doom player, and [#173](https://github.com/muellerberndt/cadence/issues/173), a - language model that understands sequences and grammar. Each states its definition of +- **Two goal epics:** [#118](https://github.com/muellerberndt/cadence/issues/118), a Doom player, and [#173](https://github.com/muellerberndt/cadence/issues/173), + saying sentences formed in the brain, as a person does. Each states its definition of done, its stages and the capability contracts folded into it. - **A milestone per release**, holding the library issues that release delivers, each with its gate. diff --git a/CHANGELOG.md b/CHANGELOG.md index 83e43a72..fa39d559 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,253 @@ # Changelog +## 0.96.0 — 2026-11-10 + +Many brains of the default composition run as one on a device, and one brain runs its moment in +compiled calls on the CPU: `cadence.Lives(inputs, actions, lives, backend="torch")` steps up to +a thousand independent lives of one anatomy in one call per moment, each life the lone brain of +its seed, and `learning_compiled_moment=True` does the work around a moment's solve in compiled +calls, a grammar-chamber pass in .54 to .56 of the default's seconds on one core. Both leave every +bit of a brain that names neither as it was. Three opt-ins join them: `memory_fade`, under which +place records nobody uses fade and deeply used ones keep, and `recall_cortex_norm`, +`recall_gain_after` and `recall_gain_next`, which let the composed attention circuit form on more +founders. The copy pathway (`recall_copy`) leaves, a `synapse_rate` of zero holds a synapse's +optimizer moments and its neuron's bias, and a brain whose store fades saves its own checkpoint +format. Every default is unchanged: a brain that names none of the new options lives, saves and +describes to the bit as on 0.95.2 on `backend="cpu"`, with `recall_copy` the one refusal. + +### New + +#### Many lives at once, and the lifecycle on the device (#274) + +- **`Lives`: independent lives over one topology, stepped together.** `cd.Lives(inputs, actions, + lives, backend="torch")` holds many brains of the default composition without the store + (`episodic=False`), each with its own synapses, optimizer, critic, eligibility, recall memory + (slots, address and position maps), carried gains, working trace and random generator, one stream + each, and steps them in one call per moment. Every settle reads each stream through its own + life's synapses and stops each life where its own stream meets the tolerance; a lesson judges + each life by its own answer and updates the lives it taught; a reward moves each life on its own + dopamine. Each life follows the lone brain of its seed on that device: in float64 every action + the same and every learned part within 1e-9 over 30 moments of rewards, endings and teaching; on + the CPU it is that lone brain to the bit. On an NVIDIA L4, 128 lives step at 0.118 ms a life and + moment, the speed of one brain's streams, against 0.876 ms in a four-process pool, and 1,024 + lives at 0.049 ms. It holds one or several regions, observers and motor slots at any teaching + point, with or without the recall memory, on CUDA and the torch CPU device, and refuses by name + the store, arousal, a maturity, a code, an apical compartment, rehearsal, the efference copy, the + heard history and the recall options that keep the memory on the host, the cortex at unit norm + among them; MLX has no lives axis ([many lives at + once](docs/backends.md#many-lives-at-once)). A rate of zero holds each life's synapses, biases + and optimizer moments as it holds a lone brain's. +- **An outcome reads its checks once on torch.** Where the critic rests on the device, the + dopamine is made there from the critic's value, and an outcome's checks (the dopamine's, the + critic's, the recall memory's reward step and the update's), the critic's bias step, the + report's steps and the dopamine of every row come to the host in one read before anything is + installed. A torch update and a lesson read their checks and report in one read where they read + five. On MLX the critic's and the recall memory's checks are made in the update's launch. The + CPU path keeps its bits; a device life agrees with the host's as before. +- **A CUDA card the default backend's table has not read is calibrated** once per machine, in + place of the margin of twice the work: the chamber's brain is timed on the card and on one core, + and the crossing scales the table's ([the default + backend](docs/backends.md#the-default-backend)); `CADENCE_CALIBRATE=0` keeps the margin. On an + NVIDIA L4 it reads 50 to 60 rows of the chamber's brain against the A10G's 44. +- **The ledger of a moment.** [`benchmarks/device/ledger.py`](benchmarks/device/ledger.py) lists + every crossing between the host and the device that a `live`, `step`, `lesson` or `act` makes, + by the place in the library that made it, with its bytes and time, beside the seconds of the + moment, on torch and MLX. A moment's bytes are small; its synchronizations are what it pays for. +- The options that keep a moment's memories on the host each state what they add that the device + form does not hold, `memory_fade` and `recall_cortex_norm` among them ([memories on the + device](docs/backends.md#memories-on-the-device)). + +#### The compiled moment (opt-in, #282) + +`learning_compiled_moment=True` (`Brain.compose`, `retune`, `LearnerConfig(compiled_moment=True)`) +does the work around the solve kernels of a moment in compiled calls on the CPU. A greedy act is +one compiled call: the moment's drive (the sensory part, the working trace, the recall memory's +read, with the address and every position term through one product of the maps, which stay packed +until a lesson replaces one, and the place store's read), the free solve, the choice, the working +trace's step and the memory's write; the act that begins its streams is one too. A lesson by +surprise judges in one call, and where it teaches a row the lesson is one call more, from its +phases to its written efficacies: the free phase, the two nudged phases and their checks, the +contrast, the optimizer's step with the tied and bounded efficacies, the address lesson and the +carried gains' contrast and step, and Python stores what the call returns (a learner with frozen +or tied synapses or per-synapse rates, and an update that is refused, take the passes in turn). +With the option the lesson's contrast is one compiled pass in each of its paths, its block +products through the BLAS that Numba calls. An act after a lesson that taught nothing takes the +lesson's read as its own, as it takes the lesson's solve, and writes its moment in the choice's +pass. The equations are the ordinary path's and the sums run in another order, so the bits are +other ones. Off by default; off, every bit is as before: the identity, options and slots batteries +read no difference and the founder parity reads 0 of 24 lives differing against the tree before +the change on one machine, and twelve chamber lives of the default brain read the same panels and +work. Several read heads, a window, the command or when in the slots, the gains, a learned code or +a separator take the ordinary path. Beside each other option of this release +(`recall_cortex_norm`, `memory_fade`, `maturity`, the delayed gains, a synapse rate of zero) a life +of lessons, acts, sampled moments and outcomes keeps its bits through every compiled path and +agrees with the ordinary path to rounding (`tests/test_compiled_moment.py`); with +`recall_cortex_norm` the compiled act carries the cortex at unit norm as `Recall.witness` does. + +On one Xeon 8488C core with nothing else running, in one session, a grammar-chamber pass of the +default brain takes .54 to .56 of its seconds with the option (67.6 to 70.8 s against 36.7 to +39.5 s over three founders) and 2.3 to 2.4 times the matched transformer's 15.9 to 16.2 s, where +the default takes 4.2 to 4.4: under the issue's 2.5 times in that session. The kernels take +26.9 s of the compiled pass, 1.7 times the transformer: the act's and the judgment's call 20.8 s +and the taught lesson's 6.0 s. What stands around them takes 7.2 s, 4.2 s around an act's one +call (about 85 microseconds an act) and 3.0 s around a lesson's (about 130 a lesson), and the +chamber's own loop 1.7 s (`benchmarks/grammar/cost.py`, which reads both the seconds and the +parts and warms every kernel before its clock; +[the compiled moment](docs/tune.md#the-compiled-moment-opt-in)). Seconds differ by a tenth and +more between runs and machines, the transformer's more than the brain's: across sessions the +default read 4.2 to 5.3 times the transformer, and at 4.8 the option's .54 to .56 is 2.6 to 2.7 +times. Within a run the option's share of the default's seconds is steady, and it is the number to +read. The kernels and the chamber's loop are .42 of the default's seconds and the Python around +the calls .11, so 2.5 times at a default of 4.8 takes fewer calls per moment (#291). + +The panels, the option less the default: + +- The grammar chamber over 12 paired fresh lives (founders 0 to 2, seeds 2000 to 2003): clauses + -.012 (s.e. .012), two distractors -.005 (.040), held-out +.004 (.015), roles +.010 (.032), + brackets +.008 (.027), paths -.014 (.010), two hops -.031 (.017), three hops +.022 (.040); over + 36 (seeds 2000 to 2011): -.011 (.007), -.012 (.017), +.002 (.016), -.003 (.020), -.010 (.019), + +.004 (.014), +.003 (.018), -.010 (.036). Every difference is under .07 and within two standard + errors. The one-call paths keep the option's bits, the lesson in one call among them: the + twelve lives read the same panels and work to the bit before and after them. +- The sequence battery (look back and n-back with and without held tokens, two hops, three + founders): -.0013 (s.e. .0042) over 15 paired lives. + +#### Records that fade with disuse (opt-in, #253) + +`Brain.compose(..., memory_fade=1.2e-4)` gives the store's place records the dynamics of a habit: +a record nobody uses fades, and a deeply used one fades far more slowly. Each key coordinate keeps +its depth (the unit cue's magnitude summed over the moments that cue it and the writes that use +it) and its recent use (the moments' sum at `1 - 1/2,000` a moment); every moment of life an idle +record loses `memory_fade / (1 + (depth / 2000) ** 4)` of itself. No count is kept. The default is +0.0, and a brain without the setting is the founder's to the bit on `backend="cpu"`: its arrays, +checkpoints, descriptions and parameter counts are unchanged. A fading store rests on the host. + +A brain with the setting saves checkpoint format `cadence-brain/3`, with a maturity or without. +0.95.2 and earlier refuse the file ("unsupported Brain checkpoint format"), where they would +load it with the fade off. `Brain.load` reads `cadence-brain/1`, `/2` and `/3`; a brain without +the setting saves `cadence-brain/1`, or `/2` with maturity, as before. + +On 24 fresh founders of the competing-skills chamber, with `maturity=True`, a skill practised +for 2,000 moments keeps .36 of itself over 40,000 blank moments and one practised for 40,000 +keeps 1.00, and relearning after disuse beats first learning by +4.3 and +4.9 points (s.e. 1.1). +Without the setting both keep 1.00. The chambers from birth, P4 with a mature skill and the short +ring read as without it, and in the robot arena (48 nurseries from birth, 140 fight runs, 48 +revision lives) the fights, the acquisition and the revision test hold every gate that maturity +alone holds (#253). See +[records that fade with disuse](docs/memory.md#records-that-fade-with-disuse-opt-in). + +#### The composed attention circuit: the cortex at the event's weight, the gains released late (#269) + +- `recall_cortex_norm=True` (opt-in): every map of the read reads the cortex of the moment before + at unit norm per stream, so a lesson's step on the cortex columns weighs no more than its step + on the event's rows. On the indirect-object chamber (`benchmarks/heads`) it moves the lives that + form the composed circuit from 2 to 7 of 12 fresh founders and keeps the taught second hop on 12 + of 12. It keeps the memory off the device path. The default and `backend="cpu"` lives are + unchanged. +- `recall_gain_after=n` and `recall_gain_next=g` (opt-in, with `recall_gain`): the learned gains + drive nothing and learn nothing through the first `n` lessons, then every next port's gain + starts at `g` and every recall port's at zero; `brain.recall.gain_lessons` counts the lessons + and a saved brain keeps it. With the cortex at unit norm, `n=1000` and `g=10`, never-taught + names are said at .97 to 1.00 on 7 of 12 fresh founders with the taught second hop at .99 to + 1.00 on 12 of 12, one life short of #269's bar. At their founders (0 and 0.0) a brain lives as + without them. + +### Changed and removed: the candidates 0.95.0 kept (#281) + +- **`Learner.synapse_rate` of zero holds the whole unit.** A synapse at rate zero takes no step of + its own and the optimizer keeps its running moments (`velocity`, `second_moment`) as they are, + where 0.95.2 kept accumulating its contrast into them. A neuron whose every inbound synapse + steps at zero holds its bias and the bias's moments (`velocity_bias`, `second_moment_bias`) the + same way, where 0.95.2 stepped the bias. `Learner.held()` names the held synapses and neurons by + their rates. A reciprocal pair shares one efficacy and steps by the mean of its two directions' + steps, so the pair holds where both directions are at rate zero; one direction alone at zero + takes half its partner's step (measured: 1.6e-3 on both after three lessons), while `held()` + names it and its moments stay. The rule holds for lessons and rewards on the CPU, torch and MLX, + and in `Lives`. What changes: a learner with a rate of exactly zero on some synapse saves those + moments at the values they held when the rate went to zero (a newborn's zeros), and the held + neurons' biases and bias moments at theirs; every other value of such a learner, and every + learner whose rates are all above zero or absent, is as on 0.95.2 to the bit. No test, benchmark + or composed brain sets a rate of zero, so no recorded bytes change. +- **Removed: the copy pathway, `recall_copy`.** `recall_gain` (0.95.0, #262) is the learned path + from a read to its token's action, and the pathway needed the tokens' label, which cost the + reads only position finds (look back 0.25 to 0.33 against 0.99 to 1.00) and the grammar + chamber's paths (#259). Write `recall_gain=1.0, recall_tokens=...` in its place, or nothing + where the task needs no token no lesson used. `recall_copy` is refused at `compose` and `retune` + with that sentence, and a saved memory with the pathway is refused by name: cadence 0.95.2 runs + it. A saved memory without it loads as before. `recall_tokens` stays, a declaration of which + answers are tokens that adds no synapse. `Recall.copy_synapses` and the sequence battery's + `heads_copy` arm left with it, and the copy-tokens instrument retired to its receipt, which + verifies with its archived checker. `recall_copy_next` left before any release (#273). +- **Kept: `learning_surprise_share`, `teaching="normalized"`, `attention=True` and + `recall_next=False`.** Each was read against what a new brain composes, and each stays; the + readings are on #281. The sequence battery's `composed` arm (`--arms composed --settings ...`) + reads a one-out of the composition. + +### Fixes and the same bits in less time + +- A settle of no sweeps makes no standing drive it would not read, and a lesson whose answer + matches every label makes no targets and reports from one record per batch size. A public + operation enters and leaves the recall memory's scope without a context manager, a lesson of a + brain without a learned code keeps its moment without one, and the labels of one slot are + checked in two reductions. +- **Where the composition's falls come from** (#280). The memory guide lists each fall of the + default composition with the part that causes it, read on fresh founders by silencing one part + at a time: the key-door, the read's drive; binding, lessons teaching only the rows that + surprise; "how long ago?", the read's recency of 0; the competing ring, the read's drive and the + working trace, in part; the hidden route, no single part. It also lists what removing each cause + costs. No setting or default changes. +- The API reference lists the `Recall` signature as the class takes it, and AGENTS.md names the + second goal epic by its title (#173). + +### Instruments + +The library is unchanged by these. Each reading ran under a protocol declared before its first +life, on fresh founders. + +- **The lesson's step** ([`lesson_step.py`](benchmarks/grammar/lesson_step.py), #258). The default + composition settles the grammar chamber's exact-clause reading within .05 over its last three + checkpoints on 4 of 6 lives at width 64 and 5 of 6 at width 128. +- **Routes with a goal** ([`routes.py`](benchmarks/grammar/routes.py), #257). A chamber whose first + hop is said before the goal it must reach, with a sliding planning window beside word by word. +- **Exposure parity** ([`exposure.py`](benchmarks/grammar/exposure.py), #270). Passes of the + chamber's 4,000 batches beside the transformer's offline control at equal compute. Without + rehearsal the default composition sits within .05 of the control on every reading but brackets + (.86 against .97), which reach .99 by the fourth pass. +- The grammar chamber and the sequence battery take a `default` arm: `Brain.compose` with nothing + named but the backend. A chamber life's work counts the rehearsal's replayed moments and rows. +- `benchmarks/heads`: the indirect-object chamber, the head-function panel, causality by silencing + and erasure and the matched controls, with its supports and the declared readings (#269). +- `benchmarks/vocabulary`: the grammar chamber at 200, 1,000 and 5,000 words by class and by + pieces, one hearing, the matched controls, and the read's logits split by source at every word + whose content a heard moment holds (#266). +- `benchmarks/attention/crossing.py`: the crossing on keyed rows, the conjunction's decoder, + supports that hold the cortex's synapses, and the matched LSTM and transformer (#267). +- `benchmarks/sequence/held.py`: R1's six tasks at tokens no lesson used, with the read's + prediction, the vote at each miss and the answer with the association's vote silenced (#272). +- `benchmarks/device`: the ledger, a life's seconds on a device and the lives' per-life cost + against a process pool (#274); `benchmarks/grammar/cost.py`: a pass's seconds and parts for the + default brain, the compiled moment and the transformer on one thread (#282). + +### Measured negatives + +- A lesson's step scaled by the running share of lesson rows answered wrong settles 4 and 3 of 6 + lives and costs held-out clauses .10 to .13, so it stays out of the library (#258). +- A sliding planning window of three reads first hops on held-out start-goal pairs at .312 against + word by word's .334; neither reads the goal there, and the offline controls read .75 to .83: + planning does not pay on that chamber (#257). +- `learning_rehearsal=1.0` costs 5.7 times a pass, since it replays each drawn row's whole + episode, and lowers paths to .82: the recall read delivers fewer hops (#270). +- Recovery after quiet with `memory_fade`: over 40,000 quiet moments after twenty arena fights a + store sheds 12 % of its mass (23 % at most) and the carried change reads 0.134 against 0.139 + right after the fights, a gain of +0.005 (s.e. 0.003), under two standard errors; the routine + does not return (#253). +- The compiled moment reaches 2.3 to 2.4 times the transformer's pass in one session and 2.6 to + 2.7 at a default of 4.8 times across sessions; the issue's 2.5 holds in one session and not + across them (#282, #291). +- With the cortex at unit norm and the gains released at the thousandth lesson the composed + circuit forms on 7 of 12 fresh founders, one life short of #269's bar; the five that miss fail at + the read's selection, before the gains are released. + ## 0.95.2 — 2026-11-09 `Brain.lesson` starts its free phase from the stream's activity, the state the act settles from, diff --git a/README.md b/README.md index 41210455..f9e29651 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,7 @@ action is read from that equilibrium, and every synapse learns from the differen settles, with no backward pass and no training/inference switch. ```sh -python -m pip install cadence-net==0.95.2 +python -m pip install cadence-net==0.96.0 ``` ```python @@ -57,7 +57,7 @@ episodic memory, a learned code, images. The [documentation index](https://github.com/muellerberndt/cadence/blob/main/docs/README.md) lists the rest: wide inputs, growing and grafting, evaluation, your own anatomy, how the brain settles, backends; the -[released documentation](https://github.com/muellerberndt/cadence/blob/v0.95.2/docs/README.md) +[released documentation](https://github.com/muellerberndt/cadence/blob/v0.96.0/docs/README.md) describes the APIs of the installed package. Integrated learned world prediction, repair localized to what failed and cheap stable operation are work in progress; Cadence is alpha research software. diff --git a/benchmarks/attention/README.md b/benchmarks/attention/README.md index 72e8a665..c36ea33b 100644 --- a/benchmarks/attention/README.md +++ b/benchmarks/attention/README.md @@ -7,19 +7,18 @@ ran, archived by content hash under `results/sources/`. A receipt states what wa code it binds, and a setting it names need not exist in this version. The scripts of a campaign (`admission.py`, `heads_admission.py`, the `*_campaign.py` files and their protocols, tests and `verify.py`) are in release 0.94.0, whose tree runs the commands below. Five instruments stay -here, runnable and tested: `copy_tokens.py`, `own_action.py`, `look_back.py`, `held_heads.py` and +here, runnable and tested: `own_action.py`, `look_back.py`, `held_heads.py` and `contextual_choice.py`. -## Retrieved content becomes the answer: the copy pathway (`recall_copy`, `recall_tokens`) +## Retrieved content becomes the answer: the copy pathway (retired) -The [copy-tokens benchmark](copy_tokens.py) asks two things of eight tokens. A token is heard, -two other tokens follow, and a cue asks for the heard token back: the cue's key names the heard -slot, so the read is right before any lesson, and tokens 5 to 7 are never taught as answers. -And a cue names a token of a sequence of five heard once and asks for the token that followed -it, which the read holds on its next port while its recall port holds the cue's own token. -Lessons at the cue only, 300 and 600 per founder, `teaching="normalized"` with -`learning_surprise=True`, `episodic=False`. Its [receipt](results/copy-tokens.json), six founders -of one module of 24 neurons: +The copy pathway (`recall_copy`) left the library in 0.96.0, and its instrument, +`copy_tokens.py`, with it; [the learned value path](../../docs/memory.md#the-learned-value-path-opt-in) +(`recall_gain`) is the library's path from a read to its token's action. Its +[receipt](results/copy-tokens.json) stays, six founders of one module of 24 neurons, and verifies +on archived bytes with its archived checker: +`python benchmarks/provenance.py --historical benchmarks/attention/results/copy-tokens.json`. +Release 0.95.2 runs the instrument. | Arm | Untaught produced | Taught | Succession | Untaught successor | | --- | --- | --- | --- | --- | @@ -29,40 +28,6 @@ of one module of 24 neurons: | `recall_copy=0.5`, no tokens | 0.00 | 0.57 to 0.99 | 0.93 to 1.00 | 0.00 | | `recall_copy=1.2`, no tokens | 0.00 | 0.22 to 0.87 | 0.86 to 0.97 | 0.00 | -Uniform random answers 0.125. The founder's read names the untaught token at 0.32 to 1.00 (the -founder of seed 2 moved its own read with its lessons) and never produces it. Two founders of a -deep apical brain read the same: untaught 1.00, succession 0.94 and 0.98 against 0.97 and 0.99. -Every declared gate passes; the result is recomputed by `test_copy_tokens.py`. - -The design was chosen on these tasks, and the alternatives read as follows (development -readings, outside the receipt unless stated): - -- An identity on both ports at once (founder 0, identity 1.0 to 4.0): the next port of the heard - token's slot holds the distractor that followed it, and the newborn produced the heard token on - 0.53 to 0.67 of untaught questions; 300 lessons took it to 0.00 to 0.13. The identity starts on - the recall port only. -- No echo suppression (the echo synapses zeroed at birth, six founders): untaught 1.00, but - succession 0.79 to 0.95, mean 0.87, against 0.94 to 0.99 with it: the recall port answers a cue - with its own token. -- Each port's identity tied into one learned gain (`Learner(tie_groups=...)`): on the receipt's - six founders it produced the untaught tokens at 1.00 as the untied pathway does, with succession - 0.91 to 0.98 against 0.94 to 0.99, so the ties were left out. -- The apical compartment as a bound was not needed: on the deep apical brain the pathway, which - lands basally, keeps succession, and a bound relative to the motor region's bottom-up drive would - cap the copy of a token whose action that drive never raised. It was not measured. - -The first run of this receipt declared a gate that the founder's read of the untaught tokens is -0.80 or better on every founder; founder 2 read them at 0.32, and the gate was replaced by its -production alone (0.00 on every founder), with the reads reported beside it. - -The receipt was recorded on the NumPy solver (`CADENCE_FUSED=0`); the compiled dense kernel gave -the same answers, with the reported gains differing in their last digits. - -```sh -CADENCE_FUSED=0 PYTHONPATH=src python benchmarks/attention/copy_tokens.py --workers 4 \ - --out /tmp/copy-tokens.json -``` - ## Its own actions in the slots (`recall_action`) The [own-action benchmark](own_action.py) gives a brain the same observation at every moment, so @@ -290,6 +255,44 @@ teaching-policy repair now passes on both development and all twenty fresh confirmation founders. Numerical checks and that acquisition control alone cannot admit a library attention option. +## The crossing: a question whose fact depends on the context (`crossing.py`) + +[`crossing.py`](crossing.py) reads [#267](https://github.com/muellerberndt/cadence/issues/267): two +records (`dog V0`, `cat V1`, the order drawn per stream), a context `X_c` and a question `Q_k`; the +answer, at a `GO` moment after the question, is the value of the fact `c xor k`. Neither bit names the +fact alone, so a reader that adds what it reads says one of the two values, .50. A life teaches one +lesson a batch on the answer and reads fresh rows; `decode` fits a ridge decoder of `c`, `k` and +`c xor k` on the cortex the answer's read is addressed from, and measures the crossing term of the +four cells' means against the spread of the rows around their cell. The supports `frozen_cortex` and +`frozen_all` restore the synapses into the association and prefrontal regions after every lesson +(the second the read's synapses as well), a diagnostic and no candidate. The matched LSTM and +transformer hear the same rows, one Adam step a batch. + +Declared reading, six fresh founders (2670901 to 2670906), one 32-vCPU box, 30,000 batches, the +grammar chamber's brain at the deep teaching point with a held read: + +| Learner | Answers at 30,000, per founder | At .90 | Decode of `c`, `k`, `c xor k` at 30,000 | Crossing term / spread: newborn; at 30,000 | Sweeps per act | +| --- | --- | --- | --- | --- | --- | +| The read's rule | .60 .54 .46 .49 .45 .45 | 0 of 6 | .63-.88, .98-1.00, .46-.58 | .02-.03 / .23-.30; .04-.18 / 2.83-3.02 | 50 to 100 | +| The rule, `recall_cortex_norm` | .52 .51 .48 .57 .41 .51 | 0 of 6 | .58-.64, .86-1.00, .44-.55 | .03-.04 / .34-.42; .01-.03 / .54-.61 | 68 to 101 | +| The rule, every synapse into the cortex held (`frozen_all`) | .58 .44 .55 .53 .42 .50 | 0 of 6 | 1.00, 1.00, .73-.90 | .02-.03 / .23-.30; .03-.07 / .46-.71 | 59 to 69 | +| LSTM | 1.00 on 6 (from 10,000 on 5) | 6 of 6 | | | | +| Transformer | .49 .95 .53 1.00 .51 .56 | 2 of 6 | | | | + +At every reading of every brain arm the answer sits at the floor; on the two plastic arms erasing the +fact asked takes it to .00 and erasing the other leaves .60 to 1.00, so the answer is lost at the +read's selection. Two terms meet there: + +- **The lessons bury the conjunction.** The newborn's cortex holds it linearly (decode .95 to 1.00 at + birth); under lessons the rows' spread around their cell grows about tenfold, from .23-.30 to + 2.8-3.0, while the crossing term stays at .04 to .18, and the decode of `c xor k` falls to .46-.58. + The cortex at unit norm keeps the spread at .54-.61 and the term falls with it (.01-.03). +- **A conjunction the cortex holds is not selected.** With every synapse into the cortex held, the + conjunction stays decodable (.73 to .90) at a term of .03 to .07 against a spread of .46 to .71, and + the held address reads .42 to .58: the address lesson's step on the cortex columns, a linear + map driven by the read's own error at `GO`, does not find a direction of that size within 30,000 + batches. + ## Equations and boundaries With witnessed keys `K`, contents `V` and a zero-content null slot, let diff --git a/benchmarks/attention/copy_tokens.py b/benchmarks/attention/copy_tokens.py deleted file mode 100644 index 990a29da..00000000 --- a/benchmarks/attention/copy_tokens.py +++ /dev/null @@ -1,307 +0,0 @@ -"""Retrieved content becomes the answer: the copy pathway (``recall_copy``, ``recall_tokens``). - -Two tasks on eight tokens, each observation a token one-hot with three flags (HEAR, ASK, -NOISE), eight actions, one per token; the brain declares that observation coordinate ``i`` and -action ``i`` are the same token where the arm says so (``recall_tokens=range(8)``). - -- **Held-out tokens.** A token is heard (HEAR + token), two other tokens follow (NOISE + token), - then a cue (HEAR + ASK) asks for the heard token back. The cue's key names the heard slot, so - the read is right before any lesson; the question is whether the answer follows it. Tokens 0 - to 4 are taught as answers (a lesson at the cue, then the act); tokens 5 to 7 are heard and - follow as distractors but are never taught as answers. Tests ask for every token. -- **Succession.** Five distinct tokens are heard (HEAR + token), then a cue (ASK + token) names - one of them other than the last; the answer is the token that followed it, which the cue's - read holds on its ``next`` port while its ``recall`` port holds the cue's own token. Only - successors among tokens 0 to 4 are taught; tests ask for those (``succession``) and for - successors 5 to 7 (``succession_held``). - -The founder memory is compared with the copy pathway: the recipe (``recall_copy=2.0`` with the -token identities), a weaker identity (``1.0``), and the control without shared identities -(``recall_copy=0.5``, and ``1.2``, the recall projection's own scale, a plain direct projection), -drawn at random as every projection is. Each founder lives -once per arm, at ``teaching="normalized"`` with lessons where the answer differs -(``learning_surprise=True``), ``episodic=False``, on a brain of one module of 24 neurons; the -recipe and the founder also on a deep apical brain (``modules=(32, 24, 24)``, ``apical=0.25`` -at the depth benchmark's teaching point, ``teaching="deep"``). Every reading reports the read's -own prediction -beside the answer: the token most driven on the port that holds the answer (``read_*``). - -Component evidence for the pathway: a token the read delivers drives its own action, untaught -tokens included, and in-context succession is kept. It is not evidence for a composer, a -story or a language task; those keep their own measurements. -""" - -from __future__ import annotations - -import argparse -import json -import runpy -import warnings -from concurrent.futures import ProcessPoolExecutor -from pathlib import Path -from typing import Any - -import numpy as np - -import cadence as cd - -HERE = Path(__file__).resolve().parent -ROOT = HERE.parents[1] -evidence = runpy.run_path(str(ROOT / "benchmarks/provenance.py")) -T = 8 -HEAR, ASK, NOISE = T, T + 1, T + 2 -INPUTS = T + 3 -TAUGHT = 5 # tokens 0..4 are taught as answers; 5..7 never -LENGTH = 5 # tokens in a succession trial -LESSONS = {"heldout": 300, "succession": 600} -TESTS = 160 -UNIFORM = 1 / T -FLAT = {"modules": (24,), "teaching": "normalized"} -DEEP = {"modules": (32, 24, 24), "apical": 0.25, "teaching": "deep"} -# arm -> the copy options -ARMS: dict[str, dict[str, Any]] = { - "founder": {}, - "identity": {"recall_copy": 2.0, "recall_tokens": list(range(T))}, - "identity_weak": {"recall_copy": 1.0, "recall_tokens": list(range(T))}, - "random": {"recall_copy": 0.5}, - "random_strong": {"recall_copy": 1.2}, -} -FLAT_ARMS = tuple(ARMS) -DEEP_ARMS = ("founder", "identity", "random") -FLAT_SEEDS, DEEP_SEEDS = (0, 1, 2, 3, 4, 5), (0, 1) - - -def sources() -> list[tuple[str, Path]]: - """Complete executed library and experiment source identity.""" - return evidence["sources"](ROOT, "benchmarks/attention/copy_tokens.py") - - -def obs(*on: int) -> np.ndarray: - x = np.zeros((1, INPUTS)) - x[0, list(on)] = 1.0 - return x - - -def compose(seed: int, arm: str, depth: str = "flat") -> cd.Brain: - options = dict(FLAT if depth == "flat" else DEEP) - options.update(ARMS[arm]) - return cd.Brain.compose( - INPUTS, T, backend="cpu", seed=seed, episodic=False, recall=True, recall_founder=True, - learning_surprise=True, **options, - ) - - -def answer(brain: cd.Brain, x: np.ndarray, port: str) -> tuple[int, int]: - """The brain's greedy answer to ``x`` and the read's own prediction on ``port``.""" - drive = brain.stimulus(x) - read = int(np.argmax(drive[0, brain.connectome.populations[port]][:T])) - return int(np.asarray(brain.act(x, greedy=True)).reshape(-1)[0]), read - - -def heldout_episode(rng: np.random.Generator, token: int) -> list[np.ndarray]: - others = rng.choice([u for u in range(T) if u != token], size=2, replace=False) - return [obs(HEAR, token), obs(NOISE, int(others[0])), obs(NOISE, int(others[1]))] - - -def heldout(seed: int, arm: str, depth: str) -> dict[str, float]: - brain = compose(seed, arm, depth) - rng = np.random.default_rng([seed, 1]) - cue = obs(HEAR, ASK) - with warnings.catch_warnings(): - warnings.simplefilter("error") # a lesson after an act on the same moment is refused - for _ in range(LESSONS["heldout"]): - token = int(rng.integers(TAUGHT)) - brain.reset() - for x in heldout_episode(rng, token): - brain.act(x, greedy=True) - brain.lesson(cue, np.array([token])) - brain.act(cue, greedy=True) - test = np.random.default_rng([seed, 2]) - hits = {"taught": 0, "held": 0} - reads = {"taught": 0, "held": 0} - count = {"taught": 0, "held": 0} - for i in range(TESTS): - token = i % T - brain.reset() - for x in heldout_episode(test, token): - brain.act(x, greedy=True) - given, read = answer(brain, cue, "recall") - group = "taught" if token < TAUGHT else "held" - hits[group] += given == token - reads[group] += read == token - count[group] += 1 - return { - "taught": hits["taught"] / count["taught"], "held": hits["held"] / count["held"], - "read_taught": reads["taught"] / count["taught"], - "read_held": reads["held"] / count["held"], - } - - -def succession_trial(rng: np.random.Generator, answers: set[int]) -> tuple[np.ndarray, int, int]: - while True: - sequence = rng.permutation(T)[:LENGTH] - places = [k for k in range(LENGTH - 1) if int(sequence[k + 1]) in answers] - if places: - k = int(rng.choice(places)) - return sequence, k, int(sequence[k + 1]) - - -def succession(seed: int, arm: str, depth: str) -> dict[str, float]: - brain = compose(seed, arm, depth) - rng = np.random.default_rng([seed, 3]) - taught = set(range(TAUGHT)) - with warnings.catch_warnings(): - warnings.simplefilter("error") - for _ in range(LESSONS["succession"]): - sequence, k, label = succession_trial(rng, taught) - brain.reset() - for s in sequence: - brain.act(obs(HEAR, int(s)), greedy=True) - cue = obs(ASK, int(sequence[k])) - brain.lesson(cue, np.array([label])) - brain.act(cue, greedy=True) - test = np.random.default_rng([seed, 4]) - out: dict[str, float] = {} - for name, answers in (("succession", taught), ("succession_held", set(range(TAUGHT, T)))): - hits = reads = 0 - for _ in range(TESTS): - sequence, k, label = succession_trial(test, answers) - brain.reset() - for s in sequence: - brain.act(obs(HEAR, int(s)), greedy=True) - given, read = answer(brain, obs(ASK, int(sequence[k])), "next") - hits += given == label - reads += read == label - out[name] = hits / TESTS - out["read_" + name] = reads / TESTS - recall = brain.recall - if recall.copy and recall.tokens is not None: # each port's identity gain after the life - synapses, value, motor = recall.copy_synapses() - pre = np.asarray(brain.connectome.pre)[synapses] - weight = brain.brain.efficacy[synapses] - same = value == motor - following = np.isin(pre, recall.next_neurons) - out["gain_recall"] = float(weight[same & ~following].mean()) - out["gain_next"] = float(weight[same & following].mean()) - return out - - -@evidence["record"](ROOT, "benchmarks/attention/copy_tokens.py") -def run_one(job: tuple[str, str, int]) -> dict[str, Any]: - depth, arm, seed = job - return {"depth": depth, "arm": arm, "seed": seed, - **heldout(seed, arm, depth), **succession(seed, arm, depth)} - - -def jobs() -> list[tuple[str, str, int]]: - return [("flat", arm, seed) for arm in FLAT_ARMS for seed in FLAT_SEEDS] + [ - ("deep", arm, seed) for arm in DEEP_ARMS for seed in DEEP_SEEDS - ] - - -def summary(runs: list[dict[str, Any]]) -> dict[str, Any]: - """Means over founders per operating point and arm.""" - keys = ("taught", "held", "read_taught", "read_held", "succession", "succession_held", - "read_succession", "read_succession_held") - out: dict[str, Any] = {} - for depth in ("flat", "deep"): - for arm in ARMS: - rows = [r for r in runs if r["depth"] == depth and r["arm"] == arm] - if rows: - out[f"{depth}/{arm}"] = {k: float(np.mean([r[k] for r in rows])) for k in keys} - return out - - -def gates_of(runs: list[dict[str, Any]]) -> dict[str, bool]: - """The declared gates, from the readings.""" - def rows(depth: str, arm: str) -> list[dict[str, Any]]: - return sorted((r for r in runs if r["depth"] == depth and r["arm"] == arm), - key=lambda r: r["seed"]) - - out = {} - for depth in ("flat", "deep"): - recipe, founder, control = rows(depth, "identity"), rows(depth, "founder"), rows( - depth, "random") - out[f"{depth}_untaught_tokens_produced"] = all(r["held"] >= 0.8 for r in recipe) - out[f"{depth}_taught_answers_kept"] = all( - r["taught"] >= f["taught"] - 0.05 for r, f in zip(recipe, founder, strict=True)) - out[f"{depth}_succession_kept"] = all( - r["succession"] >= f["succession"] - 0.05 for r, f in zip(recipe, founder, strict=True)) - out[f"{depth}_control_without_identities_produces_none"] = all( - r["held"] <= 0.1 for r in control) - out[f"{depth}_founder_produces_no_untaught_token"] = all(r["held"] <= 0.1 for r in founder) - out[f"{depth}_beats_uniform_random"] = all(r["held"] >= UNIFORM + 0.5 for r in recipe) - return out - - -def configuration() -> dict[str, Any]: - settings = { - "tokens": T, "taught": TAUGHT, "inputs": INPUTS, "length": LENGTH, - "lessons": LESSONS, "tests": TESTS, "teaching": "normalized", - "learning_surprise": True, "episodic": False, "arms": ARMS, - "flat": {"modules": list(FLAT["modules"]), "seeds": list(FLAT_SEEDS)}, - "deep": {**{k: list(v) if isinstance(v, tuple) else v for k, v in DEEP.items()}, - "seeds": list(DEEP_SEEDS), "arms": list(DEEP_ARMS)}, - "uniform_random": UNIFORM, - } - return settings - - -def check(body: dict[str, Any]) -> str | None: - """The decision the readings support, or the reason they do not.""" - problem = evidence["metadata"](body, "cadence-copy-tokens/1", configuration()) - if problem: - return problem - runs = body.get("runs", []) - for row in runs: - problem = evidence["worker"](row, ROOT, "benchmarks/attention/copy_tokens.py") - if problem: - return problem - if sorted((r["depth"], r["arm"], r["seed"]) for r in runs) != sorted(jobs()): - return "the runs are not the declared arms and founders" - if body.get("summary") != summary(runs): - return "the summary does not follow from the runs" - gates = gates_of(runs) - if body.get("gates") != gates or body.get("passed") != all(gates.values()): - return "the gates do not follow from the readings" - return None - - -def historical_check(body: dict[str, Any]) -> str | None: - """Arithmetic at the historical receipt's original source and metric scope.""" - return evidence["historical_check"](ROOT, "benchmarks/attention/results/copy-tokens.json", body) - - -def verify_historical(path: Path) -> tuple[bool, str]: - return evidence["verify_historical"]( - ROOT, path, "benchmarks/attention/results/copy-tokens.json") - - -def main() -> None: - parser = argparse.ArgumentParser() - parser.add_argument("--workers", type=int, default=4) - parser.add_argument("--out", type=Path) - args = parser.parse_args() - with ProcessPoolExecutor(args.workers) as pool: - runs = list(pool.map(run_one, jobs())) - gates = gates_of(runs) - result = { - "schema": "cadence-copy-tokens/1", - "scope": "a token the read delivers drives its own action; component evidence", - "configuration": configuration(), - "execution": evidence["environment"](), - "runs": runs, - "summary": summary(runs), - "gates": gates, - "passed": all(gates.values()), - } - problem = check(result) - if problem is not None: - raise AssertionError(problem) - if args.out is not None: - cd.Receipt.build("cadence-copy-tokens/1", result, sources()).write(args.out) - print(json.dumps({"summary": result["summary"], "gates": gates}, indent=2, sort_keys=True)) - - -if __name__ == "__main__": - main() diff --git a/benchmarks/attention/crossing.py b/benchmarks/attention/crossing.py new file mode 100644 index 00000000..32aa7169 --- /dev/null +++ b/benchmarks/attention/crossing.py @@ -0,0 +1,514 @@ +"""The crossing chamber: a question whose fact depends on an earlier context (cadence#267). + +The answer is the value of the fact ``c xor k``: neither the context ``c`` nor the question ``k`` +names the fact on its own, and a held read's scores add over what it reads (the event, the cortex +of the moment before), so no held address built from main effects selects the fact. The rows +(``keyed``) use the grammar chamber's words on its brain. A stream witnesses two records, a fact's +name and its value (``dog V0``, ``cat V1``: the names fixed, the values drawn from four nouns and +unequal, the order drawn per stream so that a position names no fact), then the context ``X_c`` +(an open bracket) and the question ``Q_k`` (a close bracket). With ``think`` the answer comes at a +``GO`` moment after the question: the crossing can form in the cortex at the question and a held +address read it from there, the second moment a read on a read takes. Without it the question is +answered at its own moment. A third of the taught rows ask a fact by its name in place of the +question, which teaches the association to say what the read holds. A reader that ignores the +crossing says one of the two values: .50. + +Each life teaches ``batches`` batches of sixteen rows, one lesson a batch on the answer moment +before the greedy act on it, and reads at its ``checkpoints`` and at the end, on fresh rows: the +answers (``right``, and ``other_value``, the share answering the other fact's value), the read's +attention on the fact asked for and on the other (a fact's name and value slots together: +``relevant``, ``other``), the facts asked by name (``direct``), and the controls (the fact asked +for erased from the slots before the answer, the other fact erased, every slot erased). + +``decode`` reads where the conjunction stands at the moment the answer's read is addressed from +(the cortex of the question's moment with ``think``, of the context's moment without): a ridge +decoder of ``c``, ``k`` and ``c xor k`` fit on half of a set of fresh rows and read on the other +half, and the crossing term of the four cells' means (``|m00 - m01 - m10 + m11| / 4``) against the +spread of the rows around their cell's mean. A conjunction the cortex holds linearly can be +selected by a held address on it; one decoded at chance cannot. + +Arms (``ARMS``) are settings of the grammar chamber's ``recipe`` brain; ``Fd`` is the read's rule +at the deep teaching point. ``support`` names what the harness does to a life to locate a term, +no option of the library: + +- ``frozen_cortex``: the synapses into the association and prefrontal regions from anything but + the read's ports, and those regions' biases, are restored after every lesson, so the lessons + move the readout, the read's synapses and the read's maps alone; +- ``frozen_all``: the same with the read's synapses into those regions restored as well, so the + cortex the question settles to stays the newborn's and the lessons move the readout and the + read's maps alone; +- ``cortex_unit``: the maps read the cortex of the moment before at unit norm per stream. + +The matched transformer and LSTM (``run_control``) hear the same rows, one Adam step per batch, +and are read on the same fresh rows; independently scored controls, never part of a brain's +answer path. + + PYTHONPATH=src python benchmarks/attention/crossing.py --arms Fd --think --founders 0 1 2 \\ + --batches 3000 --checkpoints 1000 --out DIR/keyed.jsonl --workers 3 + PYTHONPATH=src python benchmarks/attention/crossing.py --table DIR/keyed.jsonl +""" + +from __future__ import annotations + +import argparse +import copy +import importlib.util +import json +import sys +import time +import warnings +from dataclasses import dataclass +from multiprocessing import get_context +from pathlib import Path +from typing import Any + +import numpy as np + +HERE = Path(__file__).resolve().parent + + +def _module(name: str, path: Path) -> Any: + if name in sys.modules: + return sys.modules[name] + spec = importlib.util.spec_from_file_location(name, path) + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + sys.modules[name] = module + spec.loader.exec_module(module) + return module + + +gr = _module("grammar", HERE.parent / "grammar" / "grammar.py") + +ARMS: dict[str, dict[str, Any]] = { + "recipe": {}, + "Fd": {"teaching": "deep"}, +} +SUPPORTS = ("frozen_cortex", "frozen_all", "cortex_unit") +FOUNDERS = (0, 1, 2) +BATCH = 16 +SEED = 228 +NAMES = (0, 1) # the facts' names, ``dog`` and ``cat``: a record's key +VALUES = (2, 3, 6, 7) # the words a fact can hold: ``fox owl foxes owls`` +DIRECT = 3 # one taught row in this many asks a fact by its name +CONTROLS = ("erase_relevant", "erase_other", "memory_off") +FROZEN_REGIONS = ("association", "prefrontal") +READ_PORTS = ("recall", "next") + + +@dataclass +class Ask: + """One stream of a keyed batch: the heard moments, the question and its answer.""" + + pre: list[np.ndarray] + question: np.ndarray + answer: int + c: int + k: int # the question's bit; -1 for a fact asked by its name + fact: int # the fact asked for + values: tuple[int, int] # what fact 0 and fact 1 hold + first: int # the fact witnessed first + think: bool = False # the answer comes at a moment after the question + + +def keyed( + rng: np.random.Generator, n: int = BATCH, direct: float = 0.0, think: bool = False +) -> list[Ask]: + """``n`` streams of the crossing; with ``direct`` that share asks a fact by its name.""" + rows = [] + for _ in range(n): + c, k = int(rng.integers(2)), int(rng.integers(2)) + drawn = rng.permutation(VALUES)[:2] + values = (int(drawn[0]), int(drawn[1])) + first = int(rng.integers(2)) + order = (first, 1 - first) + pre = [gr.heard(word) for fact in order for word in (NAMES[fact], values[fact])] + pre.append(gr.heard(gr.OPENS[c])) + if rng.random() < direct: + fact = int(rng.integers(2)) + rows.append( + Ask(pre, gr.heard(NAMES[fact]), values[fact], c, -1, fact, values, first, think) + ) + else: + fact = c ^ k + question = gr.heard(gr.MATCH[gr.OPENS[k]]) + rows.append(Ask(pre, question, values[fact], c, k, fact, values, first, think)) + return rows + + +def new_work() -> dict[str, float]: + return dict.fromkeys(("lessons", "rows", "lesson_row_sweeps", "acts", "act_sweeps", + "refused_lessons", "refused_acts"), 0.0) + + +# ------------------------------------------------------------------------------- the supports + + +class Support: + """What the harness does to a life to locate a term (``SUPPORTS``).""" + + def __init__(self, brain: Any, names: Any = ()) -> None: + self.names = tuple(names) + if not set(self.names) <= set(SUPPORTS): + raise ValueError(f"unknown support: {self.names}") + self.mask: np.ndarray | None = None + self.neurons: np.ndarray | None = None + if "frozen_cortex" in self.names or "frozen_all" in self.names: + populations = brain.connectome.populations + inside = np.zeros(brain.connectome.n, dtype=bool) + for region in FROZEN_REGIONS: + inside[np.asarray(populations[region])] = True + ports = np.zeros(brain.connectome.n, dtype=bool) + for region in READ_PORTS: + if region in populations: + ports[np.asarray(populations[region])] = True + pre, post = brain.connectome.pre, brain.connectome.post + self.mask = inside[post] & (~ports[pre] | ("frozen_all" in self.names)) + self.neurons = inside + + def before(self, brain: Any) -> None: + """Before a lesson or an act: what the maps read.""" + if "cortex_unit" in self.names and brain.recall is not None: + raw = np.atleast_2d(np.asarray(brain.recall.previous, dtype=float)) + norm = np.linalg.norm(raw, axis=1, keepdims=True) + brain.recall.previous = np.where(norm > 0, raw / np.where(norm > 0, norm, 1.0), raw) + + def lesson(self, brain: Any, x: np.ndarray, labels: np.ndarray) -> Any: + """A lesson, with what a frozen cortex keeps restored after it.""" + self.before(brain) + if self.mask is None: + return brain.lesson(x, labels) + graph = brain.learner.brain + efficacy, bias = np.array(graph.efficacy), np.array(graph.bias) + try: + return brain.lesson(x, labels) + finally: + after = brain.learner.brain + kept = np.where(self.mask, efficacy, np.asarray(after.efficacy)) + kept_bias = np.where(self.neurons, bias, np.asarray(after.bias)) + brain.learner.brain = after.with_parameters(efficacy=kept, bias=kept_bias) + + +def make_brain(arm: str, founder: int, settings: dict[str, Any] | None = None) -> Any: + return gr.make_brain("recipe", founder, {**ARMS[arm], **(settings or {})}) + + +def fact_slots(brain: Any, row: Ask) -> tuple[int, int]: + """The slots of the names of fact 0 and fact 1: the boundary, then the records in order.""" + start = 1 if brain.recall.boundary else 0 + return (start, start + 2) if row.first == 0 else (start + 2, start) + + +def _act(brain: Any, x: np.ndarray, work: dict[str, float], support: Support) -> np.ndarray: + support.before(brain) + try: + action = np.asarray(brain.act(x, greedy=True)).reshape(len(x), -1)[:, 0] + except RuntimeError: + work["refused_acts"] += 1 + return np.full(len(x), -1) + work["acts"] += 1 + work["act_sweeps"] += float((brain.last_settlement or {}).get("steps", 0)) + return action + + +def bring(brain: Any, rows: list[Ask], work: dict[str, float], support: Support) -> np.ndarray: + """Fresh streams brought to their answer: the records and the context heard, and with + ``think`` the question too. Returns the moment that is answered.""" + brain.reset() + for t in range(len(rows[0].pre)): + _act(brain, np.stack([r.pre[t] for r in rows]), work, support) + question = np.stack([r.question for r in rows]) + if not rows[0].think: + return question + _act(brain, question, work, support) + return np.tile(gr.concept("GO"), (len(rows), 1)) + + +def teach(brain: Any, rows: list[Ask], work: dict[str, float], support: Support) -> None: + """One taught batch: a lesson on the answer moment before the greedy act on it.""" + answered = bring(brain, rows, work, support) + try: + report = support.lesson(brain, answered, np.array([r.answer for r in rows])) + work["lessons"] += 1 + work["rows"] += report.get("attempted_presentations", 0.0) + work["lesson_row_sweeps"] += report.get("total_row_sweeps", 0.0) + except RuntimeError: + work["refused_lessons"] += 1 + _act(brain, answered, work, support) + + +def read_rows(brain: Any, rows: list[Ask], support: Support, + control: str | None = None) -> dict[str, list[Any]]: + """The rows answered on a fork of ``brain``, one list per reading, one entry per row.""" + fork = copy.deepcopy(brain) + recall = fork.recall + work = new_work() + answered = bring(fork, rows, work, support) + if control is not None: + keys, values, nexts = recall.keys.copy(), recall.values.copy(), recall.nexts.copy() + for i, r in enumerate(rows): + if control == "memory_off": + slots = list(range(recall.slots)) + else: + fact = r.fact if control == "erase_relevant" else 1 - r.fact + name = fact_slots(fork, r)[fact] + slots = [name, name + 1] # the name, whose next holds the value, and the value + for slot in slots: + keys[i, slot], values[i, slot], nexts[i, slot] = 0.0, 0.0, 0.0 + recall.keys, recall.values, recall.nexts = keys, values, nexts + answer = _act(fork, answered, work, support) + attended = recall.attended_heads[0] if recall.attended_heads else None + out: dict[str, list[Any]] = {k: [] for k in ("right", "other_value", "relevant", "other", + "pair")} + for i, r in enumerate(rows): + out["right"].append(bool(answer[i] == r.answer)) + out["other_value"].append(bool(answer[i] == r.values[1 - r.fact])) + names = fact_slots(fork, r) + mass = (0.0, 0.0) if attended is None else ( + float(attended[i, names[0]] + attended[i, names[0] + 1]), + float(attended[i, names[1]] + attended[i, names[1] + 1]), + ) + out["relevant"].append(mass[r.fact]) + out["other"].append(mass[1 - r.fact]) + out["pair"].append(f"c{r.c}k{r.k}") + return out + + +def panel(brain: Any, batches: list[list[Ask]], direct: list[list[Ask]], support: Support, + controls: bool = True) -> dict[str, Any]: + out: dict[str, Any] = {} + for control in (None, *(CONTROLS if controls else ())): + reads = [read_rows(brain, rows, support, control) for rows in batches] + summary: dict[str, Any] = { + key: round(float(np.mean([v for r in reads for v in r[key]])), 4) + for key in ("right", "other_value", "relevant", "other") + } + pairs: dict[str, list[bool]] = {} + for r in reads: + for pair, right in zip(r["pair"], r["right"], strict=True): + pairs.setdefault(pair, []).append(right) + summary["conditions"] = {k: round(float(np.mean(v)), 4) for k, v in sorted(pairs.items())} + out["intact" if control is None else control] = summary + reads = [read_rows(brain, rows, support) for rows in direct] + out["direct"] = {key: round(float(np.mean([v for r in reads for v in r[key]])), 4) + for key in ("right", "relevant")} + return out + + +# ----------------------------------------------------------------------------- the decoder + + +def cortex_at_address(brain: Any, rows: list[Ask], support: Support) -> np.ndarray: + """The cortex the answer's read is addressed from, per row: the association of the moment + before the answer, as the maps read it.""" + fork = copy.deepcopy(brain) + answered = bring(fork, rows, new_work(), support) + del answered + support.before(fork) + return np.array(fork.recall.previous, dtype=float) + + +def _ridge_accuracy(x: np.ndarray, y: np.ndarray, half: int, alpha: float = 1e-2) -> float: + mean = x[:half].mean(axis=0) + a = np.hstack([x - mean, np.ones((len(x), 1))]) + train, test = a[:half], a[half:] + w = np.linalg.solve(train.T @ train + alpha * np.eye(a.shape[1]), train.T @ y[:half]) + return float(np.mean(np.sign(test @ w) == y[half:])) + + +def decode(brain: Any, rng: np.random.Generator, support: Support, think: bool, + batches: int = 32) -> dict[str, float]: + """Where the conjunction stands at the cortex the answer's read is addressed from.""" + rows = [r for _ in range(batches) for r in keyed(rng, think=think)] + cortex = np.vstack([cortex_at_address(brain, rows[i:i + BATCH], support) + for i in range(0, len(rows), BATCH)]) + c = np.array([r.c for r in rows]) + k = np.array([r.k for r in rows]) + half = len(rows) // 2 + out = {name: round(_ridge_accuracy(cortex, 2.0 * bit - 1.0, half), 4) + for name, bit in (("c", c), ("k", k), ("crossing", c ^ k))} + means = {(i, j): cortex[(c == i) & (k == j)].mean(axis=0) for i in (0, 1) for j in (0, 1)} + cross = (means[0, 0] - means[0, 1] - means[1, 0] + means[1, 1]) / 4.0 + cells = np.array([means[int(ci), int(kj)] for ci, kj in zip(c, k, strict=True)]) + out["crossing_term"] = round(float(np.linalg.norm(cross)), 4) + out["c_term"] = round(float(np.linalg.norm( + (means[1, 0] + means[1, 1] - means[0, 0] - means[0, 1]) / 4.0)), 4) + out["k_term"] = round(float(np.linalg.norm( + (means[0, 1] + means[1, 1] - means[0, 0] - means[1, 0]) / 4.0)), 4) + out["spread"] = round(float(np.linalg.norm(cortex - cells, axis=1).mean()), 4) + out["energy"] = round(float(np.linalg.norm(cortex, axis=1).mean()), 4) + return out + + +# ----------------------------------------------------------------------------- one life + + +def run_life(spec: dict[str, Any]) -> dict[str, Any]: + """One brain's life: ``arm``, ``founder``, ``batches``, ``checkpoints``, ``settings`` (over the + arm's), ``support``, ``think``, ``controls``, ``panel_batches``, ``decode``; ``init_seed`` + (the brain's seed, on the founder's rows).""" + warnings.simplefilter("ignore") + import cadence as cd + + arm, founder = spec["arm"], int(spec["founder"]) + batches = int(spec.get("batches", 3000)) + checkpoints = {int(c) for c in spec.get("checkpoints", [])} + think = bool(spec.get("think", True)) + started = time.monotonic() + brain = make_brain(arm, int(spec.get("init_seed", founder)), spec.get("settings")) + support = Support(brain, spec.get("support", ())) + rng = np.random.default_rng(np.random.SeedSequence([founder, gr.SEEDS["train"], SEED, 1])) + test = np.random.default_rng(np.random.SeedSequence([founder, gr.SEEDS["test"], SEED, 1])) + panels = int(spec.get("panel_batches", 8)) + crossing_rows = [keyed(test, think=think) for _ in range(panels)] + direct_rows = [keyed(test, direct=1.0, think=think) for _ in range(max(panels // 4, 1))] + decoding = bool(spec.get("decode", True)) + + def reading(b: int, controls: bool) -> dict[str, Any]: + out: dict[str, Any] = { + "batches": b, "work": {**work, "seconds": round(time.monotonic() - started, 1)}, + "panels": panel(brain, crossing_rows, direct_rows, support, controls)} + if decoding: + probe = np.random.default_rng(np.random.SeedSequence([founder, 228, 5])) + out["decode"] = decode(brain, probe, support, think) + return out + + work = new_work() + readings = [reading(0, False)] + for b in range(1, batches + 1): + teach(brain, keyed(rng, direct=1.0 / DIRECT, think=think), work, support) + if b in checkpoints or b == batches: + readings.append(reading(b, bool(spec.get("controls", True)) and b == batches)) + return { + "rows": "keyed", "arm": arm, "founder": founder, "think": think, + "init_seed": int(spec.get("init_seed", founder)), "tag": spec.get("tag"), + "settings": {**ARMS[arm], **spec.get("settings", {})}, + "support": list(support.names), "batches": batches, + "parameters": int(brain.parameters()), "cadence": cd.__version__, + "readings": readings, "seconds": round(time.monotonic() - started, 1), + } + + +def run_control(spec: dict[str, Any]) -> dict[str, Any]: + """One online life of a matched control (``model``: ``transformer`` or ``lstm``) on the same + rows, its width the nearest to the brain's parameters, one Adam step per batch at the grammar + chamber's online rate, read by the first word it says on the same fresh rows.""" + import torch + + ct = _module("grammar_controls", HERE.parent / "grammar" / "controls.py") + torch.set_num_threads(1) + model, founder = spec["model"], int(spec["founder"]) + batches = int(spec.get("batches", 3000)) + checkpoints = {int(c) for c in spec.get("checkpoints", [])} + think = bool(spec.get("think", True)) + started = time.monotonic() + target = int(make_brain("Fd", founder).parameters()) + width = ct.matched_width(model, target) + learner = ct.Learner(model, width, founder, float(spec.get("rate", ct.RATES["online"]))) + rng = np.random.default_rng(np.random.SeedSequence([founder, gr.SEEDS["train"], SEED, 1])) + test = np.random.default_rng(np.random.SeedSequence([founder, gr.SEEDS["test"], SEED, 1])) + panels = int(spec.get("panel_batches", 8)) + crossing_rows = [keyed(test, think=think) for _ in range(panels)] + + def rows_of(asks: list[Ask]) -> list[Any]: + return [gr.Row("keyed", [*a.pre, a.question], [a.answer, gr.END]) for a in asks] + + def score(batches_: list[list[Ask]]) -> float: + right = [] + learner.net.eval() + for asks in batches_: + for a, said in zip(asks, learner.speak(rows_of(asks)), strict=True): + right.append(bool(said) and said[0] == a.answer) + learner.net.train() + return round(float(np.mean(right)), 4) + + readings = [] + for b in range(1, batches + 1): + learner.update(rows_of(keyed(rng, direct=1.0 / DIRECT, think=think))) + if b in checkpoints or b == batches: + readings.append({"batches": b, "right": score(crossing_rows), + "seconds": round(time.monotonic() - started, 1)}) + return { + "rows": "keyed_control", "model": model, "founder": founder, "width": width, + "parameters": int(ct.count(learner.net)), "brain_parameters": target, + "batches": batches, "readings": readings, "seconds": round(time.monotonic() - started, 1), + } + + +def run_any(spec: dict[str, Any]) -> dict[str, Any]: + return run_control(spec) if "model" in spec else run_life(spec) + + +def table(rows: list[dict[str, Any]]) -> list[str]: + lines = ["| arm | support | founder | batches | right | other value | relevant | other | " + "direct | erase relevant | erase other | memory off | decode c, k, crossing | " + "crossing term / spread | seconds |", "|" + " --- |" * 15] + for r in sorted((r for r in rows if r.get("rows") == "keyed"), + key=lambda r: (r["arm"], r["support"], r["founder"])): + last = r["readings"][-1] + p = last["panels"] + intact = p["intact"] + d = last.get("decode", {}) + + def cell(name: str, panels: dict[str, Any] = p) -> str: + return f"{panels[name]['right']:.2f}" if name in panels else "" + + lines.append( + f"| {r['arm']} | {'+'.join(r['support']) or '-'} | {r['founder']} | " + f"{last['batches']} | {intact['right']:.2f} | {intact['other_value']:.2f} | " + f"{intact['relevant']:.2f} | {intact['other']:.2f} | {p['direct']['right']:.2f} | " + f"{cell('erase_relevant')} | {cell('erase_other')} | {cell('memory_off')} | " + f"{d.get('c', float('nan')):.2f} {d.get('k', float('nan')):.2f} " + f"{d.get('crossing', float('nan')):.2f} | " + f"{d.get('crossing_term', float('nan')):.3f} / {d.get('spread', float('nan')):.2f} | " + f"{r['seconds']:.0f} |") + for r in sorted((r for r in rows if r.get("rows") == "keyed_control"), + key=lambda r: (r["model"], r["founder"])): + lines.append(f"| {r['model']} | | {r['founder']} | " + ", ".join( + f"{x['batches']}: {x['right']:.2f}" for x in r["readings"]) + " |") + return lines + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__.split("\n", 1)[0]) + parser.add_argument("--arms", nargs="*", default=["Fd"], choices=list(ARMS)) + parser.add_argument("--founders", nargs="*", type=int, default=list(FOUNDERS)) + parser.add_argument("--batches", type=int, default=3000) + parser.add_argument("--checkpoints", nargs="*", type=int, default=[]) + parser.add_argument("--settings", type=json.loads, default={}, help="over the arms'") + parser.add_argument("--support", nargs="*", default=[], choices=SUPPORTS) + parser.add_argument("--own-moment", action="store_true", help="answer at the question") + parser.add_argument("--plan", type=Path, help="a JSON list of life specs in place of the grid") + parser.add_argument("--out", type=Path, help="JSON lines, one row per life, appended") + parser.add_argument("--workers", type=int, default=1) + parser.add_argument("--table", type=Path, help="print the table of a results file") + args = parser.parse_args(argv) + if args.table: + rows = [json.loads(line) for line in args.table.read_text().splitlines() if line.strip()] + print("\n".join(table(rows))) + return 0 + if args.out is None: + parser.error("--out is required") + if args.plan: + specs = json.loads(args.plan.read_text()) + else: + specs = [{"arm": arm, "founder": f, "batches": args.batches, + "checkpoints": args.checkpoints, "settings": args.settings, + "support": args.support, "think": not args.own_moment, + "tag": f"{arm}/{'+'.join(args.support)}/{f}"} + for arm in args.arms for f in args.founders] + done = set() + if args.out.exists(): + done = {json.loads(line).get("tag") for line in args.out.read_text().splitlines() if line} + specs = [s for s in specs if s.get("tag") is None or s.get("tag") not in done] + args.out.parent.mkdir(parents=True, exist_ok=True) + with args.out.open("a") as sink, get_context("spawn").Pool( + max(args.workers, 1), maxtasksperchild=1) as pool: + for row in pool.imap_unordered(run_any, specs, chunksize=1): + sink.write(json.dumps(row) + "\n") + sink.flush() + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/benchmarks/attention/test_copy_tokens.py b/benchmarks/attention/test_copy_tokens.py deleted file mode 100644 index 21b93b0b..00000000 --- a/benchmarks/attention/test_copy_tokens.py +++ /dev/null @@ -1,60 +0,0 @@ -"""Regression gates for the copy pathway (``recall_copy``, ``recall_tokens``).""" - -from __future__ import annotations - -import copy -import importlib.util -import sys -from pathlib import Path - -import numpy as np - -SPEC = importlib.util.spec_from_file_location( - "copy_tokens_tests", Path(__file__).with_name("copy_tokens.py") -) -assert SPEC is not None and SPEC.loader is not None -subject = importlib.util.module_from_spec(SPEC) -sys.modules[SPEC.name] = subject -SPEC.loader.exec_module(subject) - -RECEIPT = Path(__file__).with_name("results") / "copy-tokens.json" - - -def test_the_tasks_ask_what_they_declare() -> None: - rng = np.random.default_rng(0) - for token in range(subject.T): - heard, *others = subject.heldout_episode(rng, token) - assert heard[0, token] == 1 and heard[0, subject.HEAR] == 1 - assert all(x[0, token] == 0 and x[0, subject.NOISE] == 1 for x in others) - for _ in range(32): - sequence, k, label = subject.succession_trial(rng, set(range(subject.TAUGHT))) - assert label == sequence[k + 1] and label < subject.TAUGHT and k < subject.LENGTH - 1 - - -def test_the_arms_differ_only_in_the_pathway() -> None: - founder, recipe, control = (subject.compose(0, arm) for arm in ("founder", "identity", - "random")) - assert founder.recall.copy == 0.0 and recipe.recall.copy == 2.0 - assert recipe.recall.tokens == tuple(range(subject.T)) and control.recall.tokens is None - assert founder.hippocampus is None and recipe.hippocampus is None - # the pathway is appended: every other synapse is the founder's - assert recipe.connectome.synapses > founder.connectome.synapses - - -def test_retained_receipt_is_source_bound_and_recomputed() -> None: - ok, message = subject.verify_historical(RECEIPT) - assert ok, message - body = subject.cd.Receipt.read(RECEIPT).body - assert body["passed"] and all(body["gates"].values()) - flat = body["summary"]["flat/identity"] - assert flat["held"] >= 0.8 and body["summary"]["flat/founder"]["held"] <= 0.1 - - -def test_the_check_refuses_a_decision_its_readings_do_not_support() -> None: - body = copy.deepcopy(subject.cd.Receipt.read(RECEIPT).body) - run = next(r for r in body["runs"] if r["arm"] == "identity" and r["depth"] == "flat") - run["held"] = 0.0 - assert subject.historical_check(body) is not None - body = copy.deepcopy(subject.cd.Receipt.read(RECEIPT).body) - body["runs"] = body["runs"][:-1] - assert subject.historical_check(body) is not None diff --git a/benchmarks/attention/test_crossing.py b/benchmarks/attention/test_crossing.py new file mode 100644 index 00000000..b2383ecf --- /dev/null +++ b/benchmarks/attention/test_crossing.py @@ -0,0 +1,98 @@ +"""The crossing chamber's rows, supports and readings (benchmarks/attention/crossing.py).""" + +from __future__ import annotations + +import importlib.util +import sys +from pathlib import Path + +import numpy as np + +HERE = Path(__file__).resolve().parent + + +def _load() -> object: + spec = importlib.util.spec_from_file_location("crossing_chamber", HERE / "crossing.py") + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + sys.modules["crossing_chamber"] = module + spec.loader.exec_module(module) + return module + + +cr = _load() + + +def test_the_answer_is_the_value_of_the_fact_the_crossing_names() -> None: + rows = cr.keyed(np.random.default_rng(0), n=64, think=True) + for r in rows: + assert r.k in (0, 1) and r.fact == r.c ^ r.k + assert r.answer == r.values[r.fact] and r.values[0] != r.values[1] + assert len(r.pre) == 5 and r.think + heard = [int(np.argmax(x)) for x in r.pre] + assert heard[0] == cr.NAMES[r.first] and heard[1] == r.values[r.first] + assert heard[4] == cr.gr.OPENS[r.c] + assert int(np.argmax(r.question)) == cr.gr.MATCH[cr.gr.OPENS[r.k]] + assert {(r.c, r.k) for r in rows} == {(0, 0), (0, 1), (1, 0), (1, 1)} + direct = cr.keyed(np.random.default_rng(1), n=32, direct=1.0) + assert all(r.k == -1 and int(np.argmax(r.question)) == cr.NAMES[r.fact] for r in direct) + + +def test_the_fact_slots_follow_the_order_witnessed() -> None: + brain = cr.make_brain("Fd", 0) + rows = cr.keyed(np.random.default_rng(2), n=8) + for r in rows: + first, second = cr.fact_slots(brain, r) + start = 1 if brain.recall.boundary else 0 + assert {first, second} == {start, start + 2} + assert (first == start) == (r.first == 0) + + +def test_a_frozen_cortex_moves_only_the_readout_and_the_reads_synapses() -> None: + brain = cr.make_brain("Fd", 0) + support = cr.Support(brain, ["frozen_cortex"]) + before = np.array(brain.learner.brain.efficacy) + bias = np.array(brain.learner.brain.bias) + rng = np.random.default_rng(0) + work = cr.new_work() + for _ in range(3): + cr.teach(brain, cr.keyed(rng, direct=1 / 3, think=True), work, support) + after = np.array(brain.learner.brain.efficacy) + assert np.array_equal(after[support.mask], before[support.mask]) + assert np.array_equal(np.asarray(brain.learner.brain.bias)[support.neurons], + bias[support.neurons]) + assert not np.array_equal(after[~support.mask], before[~support.mask]) + + +def test_a_unit_cortex_is_what_the_maps_read() -> None: + brain = cr.make_brain("Fd", 0) + support = cr.Support(brain, ["cortex_unit"]) + rows = cr.keyed(np.random.default_rng(3), n=4, think=True) + cortex = cr.cortex_at_address(brain, rows, support) + assert np.allclose(np.linalg.norm(cortex, axis=1), 1.0) + + +def test_a_newborn_cortex_holds_the_crossing_and_its_reading_is_bounded() -> None: + brain = cr.make_brain("Fd", 0) + out = cr.decode(brain, np.random.default_rng(1), cr.Support(brain), True) + assert out["c"] > 0.9 and out["k"] > 0.9 + assert out["crossing"] > 0.9 + assert out["crossing_term"] > 0 and out["spread"] > 0 + + +def test_a_life_reads_its_panels_and_leaves_the_brain_as_taught() -> None: + row = cr.run_life({"arm": "Fd", "founder": 0, "batches": 2, "panel_batches": 1, + "decode": False}) + last = row["readings"][-1] + assert last["batches"] == 2 and last["work"]["lessons"] == 2 + intact = last["panels"]["intact"] + assert 0.0 <= intact["right"] <= 1.0 and set(intact["conditions"]) <= { + "c0k0", "c0k1", "c1k0", "c1k1"} + assert {"erase_relevant", "erase_other", "memory_off", "direct"} <= set(last["panels"]) + + +def test_frozen_all_keeps_the_reads_synapses_into_the_cortex_as_well() -> None: + brain = cr.make_brain("Fd", 0) + cortex, every = cr.Support(brain, ["frozen_cortex"]), cr.Support(brain, ["frozen_all"]) + assert every.mask.sum() > cortex.mask.sum() + assert np.array_equal(every.mask & cortex.mask, cortex.mask) diff --git a/benchmarks/attention/test_retained_receipts.py b/benchmarks/attention/test_retained_receipts.py index 16b3bce3..30784045 100644 --- a/benchmarks/attention/test_retained_receipts.py +++ b/benchmarks/attention/test_retained_receipts.py @@ -51,3 +51,17 @@ def test_every_source_a_receipt_binds_is_archived(path: Path) -> None: def test_a_receipt_keeps_its_canonical_form_and_digest(path: Path) -> None: valid, message = cd.Receipt.verify(path) assert valid, message + + +def test_the_copy_pathways_receipt_reads_with_its_archived_checker() -> None: + """The copy pathway left in 0.96.0 with its instrument; its registered receipt's recorded + arithmetic is read by the checker archived with it.""" + import runpy + + root = RESULTS.parents[2] + provenance = runpy.run_path(str(root / "benchmarks/provenance.py")) + receipt = RESULTS / "copy-tokens.json" + valid, message = provenance["verify_historical"]( + root, receipt, "benchmarks/attention/results/copy-tokens.json" + ) + assert valid, message diff --git a/benchmarks/device/README.md b/benchmarks/device/README.md new file mode 100644 index 00000000..14bd2559 --- /dev/null +++ b/benchmarks/device/README.md @@ -0,0 +1,155 @@ +# What a moment costs on a device + +Two instruments for [#274](https://github.com/muellerberndt/cadence/issues/274), the whole lifecycle on the +device: + +- [`ledger.py`](ledger.py), the ledger of a moment: every crossing between the host and the device that a + `live`, `step`, `lesson` or `act` makes, by the place in the library that made it, with its bytes and time, + beside the seconds of the moment. +- [`lives.py`](lives.py), seconds per life: many continuing lives of one topology, stepped in turn in one + process, in a process pool on the CPU, as the streams of one brain in one call per moment, or as + `cadence.Lives`, brains of their own in one call per moment. + +```sh +PYTHONPATH=src python benchmarks/device/ledger.py --backend mlx --brains chamber speech --rows 1 16 +PYTHONPATH=src python benchmarks/device/ledger.py --backend torch:cuda --rows 1 16 256 --out ledger.jsonl +PYTHONPATH=src python benchmarks/device/ledger.py --backend torch:cuda --depth 4 --brains chamber --rows 16 +PYTHONPATH=src python benchmarks/device/ledger.py --table ledger.jsonl +PYTHONPATH=src python benchmarks/device/lives.py --ways serial pool streams --backend torch:cuda --lives 12 128 +PYTHONPATH=src python benchmarks/device/lives.py --composition lives --teach 4 --ways lives streams pool \ + --backend torch:cuda --lives 12 128 1024 +``` + +The brains are the chamber's default brain, `Brain.compose(35, 18)` (251 neurons, 13,120 synapses), and the +speech size, `Brain.compose(3920, 37, modules=(512,))` (12,821 neurons, 6.3 million synapses). A reading of +`ledger.py` is two passes over moments of one brain: the first, untouched, gives the seconds of a moment, and +the second counts the crossings, each named by the innermost function of the library on the stack (`--depth` +names its callers too). A crossing's time includes the wait for the device's queued work, so a `wait` row +(`mx.eval`, `torch.cuda.synchronize`) and the first read after queued work hold the device's own time. + +## Reading a ledger + +A wait or a read of numbers is a synchronization: the host stops until the device has done everything queued +before it, and the overlap of host and device work ends there. On an Apple M4 one costs about 180 +microseconds whatever the work ([which hardware](../../docs/backends.md#which-hardware)). The bytes that cross +are small: a moment of the chamber's brain moves 75 kB at one row and 330 kB at 16, a moment of the speech size +0.9 MB at one row and 14 MB at 16, mostly the observations. A moment pays for its count of synchronizations. + +## On an NVIDIA A10G + +One AWS g5.xlarge: an A10G and four vCPUs of an AMD EPYC 7R32, torch 2.12.1, float32 on the device, one thread +per process. Before is the library before the outcome's one read (main at a513b74), after is with it, and 0.94.0 +is the release, whose default brain composes fewer memories, so its moments are another brain's. + +Host reads per moment (the crossings that bring numbers to the host; each is a synchronization) and the +milliseconds spent in all crossings: + +| Brain | Moment | Rows | 0.94.0 | Before | After | +| --- | --- | ---: | ---: | ---: | ---: | +| Chamber | `live` | 1 | 37, 3.5 ms | 28, 0.82 ms | 24, 0.76 ms | +| Chamber | `step` | 1 | 37, 3.6 ms | 27, 0.77 ms | 23, 0.73 ms | +| Chamber | `step` | 16 | 37, 3.8 ms | 27, 1.0 ms | 23, 0.95 ms | +| Chamber | `step` | 256 | 37, 5.3 ms | 31, 1.0 ms | 25, 0.87 ms | +| Chamber | `lesson` | 256 | 35, 1.4 ms | 30, 1.0 ms | 26, 0.94 ms | +| Speech | `live` | 1 | 37, 4.6 ms | 38, 3.6 ms | 32, 2.0 ms | +| Speech | `step` | 1 | 37, 4.5 ms | 36, 3.5 ms | 30, 2.0 ms | +| Speech | `step` | 16 | 37, 22.7 ms | 36, 15.1 ms | 30, 13.7 ms | +| Speech | `step` | 256 | not read | 36, 82.5 ms | 30, 80.8 ms | +| Speech | `lesson` | 256 | not read | 36, 14.6 ms | 32, 14.1 ms | + +0.94.0's speech-size readings end at 16 rows, where its counting pass ran out of the device's memory. At 256 +speech rows the time in crossings is the device's own work: the outcome's read waits for the reward's step +over 256 streams of 6.3 million eligibilities each. A crossing's bytes stay below 25 MB a moment, mostly the +observations. + +Milliseconds per moment, before and after in turn for three rounds (the median of 30 moments of the chamber's +brain and 20 of the speech size each round, after 5 and 4 to warm), and one core of the same machine: + +| Brain | Moment | Rows | One core | Before, three rounds | After, three rounds | Median, before to after | +| --- | --- | ---: | ---: | --- | --- | --- | +| Chamber | `live` | 1 | 3.01 | 8.43, 7.64, 7.55 | 7.70, 7.87, 7.93 | 7.64 to 7.87 | +| Chamber | `step` | 1 | 2.90 | 8.29, 7.43, 7.67 | 8.07, 7.51, 8.33 | 7.67 to 8.07 | +| Chamber | `step` | 16 | 6.64 | 9.94, 9.21, 9.51 | 9.43, 9.40, 9.33 | 9.51 to 9.40 | +| Chamber | `step` | 256 | 80.0 | 24.66, 23.52, 23.32 | 23.14, 25.04, 23.86 | 23.52 to 23.86 | +| Chamber | `lesson` | 16 | 5.54 | 7.45, 7.22, 7.44 | 6.93, 7.23, 7.04 | 7.44 to 7.04 | +| Chamber | `lesson` | 256 | 70.0 | 12.05, 11.32, 11.35 | 11.70, 11.38, 11.72 | 11.35 to 11.70 | +| Speech | `live` | 1 | 262 | 21.65, 20.99, 21.11 | 20.33, 20.44, 20.56 | 21.11 to 20.44 | +| Speech | `step` | 1 | 266 | 21.93, 20.80, 20.92 | 19.76, 20.51, 19.55 | 20.92 to 19.76 | +| Speech | `step` | 16 | 982 | 34.96, 35.47, 34.99 | 34.47, 35.90, 33.95 | 34.99 to 34.47 | +| Speech | `lesson` | 1 | 302 | 15.42, 14.84, 14.82 | 14.85, 14.23, 14.39 | 14.84 to 14.39 | +| Speech | `lesson` | 16 | 387 | 16.56, 17.06, 16.23 | 16.00, 16.09, 15.85 | 16.56 to 16.00 | + +The speech size, whose critic and memories rest on the device, reads 1.5 to 5.5% faster; the chamber's brain, whose +memories stay on the host below their size, reads within the rounds' spread. + +Whole lives under `backend="auto"` ([`life.py`](life.py)), seconds from before `Brain.compose` to the last moment, +before and after, and with a transport plan calibrated at the life's rows before its second moment: + +| Life | `auto` | `auto` with a plan | One core | +| --- | --- | --- | ---: | +| Chamber, 256 streams, 300 moments of `step` | 9.36 to 9.09 | 9.48 to 9.40 | 23.5 | +| Speech, 1 stream, 100 moments | 8.99 to 8.71 | 9.05 to 8.86 | 9.40 for 20 moments | +| Speech, 16 streams, 50 moments | 8.57 to 8.40 | | | + +A speech-size life on the device spends 5.0 seconds composing (2.9 on the CPU) and 1.7 in its first moment before +its moments run at 20 milliseconds; a chamber life of 256 streams spends 2.1 seconds in its first moment, where the +brain moves to the device. + +## Lives + +Seconds per life and moment of `lives.py` on the same machine, the chamber's default brain with its store +(`--composition default`), 20 moments after 3 to warm: + +| Way | 12 lives | 128 lives | +| --- | ---: | ---: | +| `streams`, one brain on the A10G, one stream per life | 0.77 ms | 0.156 ms | +| `pool`, four CPU processes | 1.25 ms | 1.28 ms | +| `serial`, one core | 2.91 ms | 2.92 ms | +| `serial`, each brain on the A10G in turn | 7.75 ms | 7.59 ms | + +## Lives of their own, on an NVIDIA L4 + +`--composition lives` (`episodic=False`: the default composition without the store, which `cadence.Lives` +holds), one AWS g6.xlarge (an NVIDIA L4 and four vCPUs of an AMD EPYC 7R13), 24 moments after 4 to warm, 12 +after 4 at 512 and 1,024 lives. Every moment a `step` with the reward of the moment before: + +| Way | 12 lives | 128 lives | 512 lives | 1,024 lives | +| --- | ---: | ---: | ---: | ---: | +| `lives`, `cadence.Lives` on the L4 | 1.18 ms | 0.118 ms | 0.043 ms | 0.049 ms | +| `streams`, one brain on the L4 | 0.619 ms | 0.119 ms | 0.045 ms | 0.034 ms | +| `pool`, four CPU processes | 0.874 ms | 0.876 ms | | | +| `serial`, one core | 2.35 ms | 2.37 ms | | | +| `serial`, each brain on the L4 in turn | 6.52 ms | 6.66 ms | | | + +With a teacher's label at every fourth moment (`--teach 4`): + +| Way | 12 lives | 128 lives | 512 lives | 1,024 lives | +| --- | ---: | ---: | ---: | ---: | +| `lives`, `cadence.Lives` on the L4 | 1.45 ms | 0.147 ms | 0.056 ms | 0.067 ms | +| `streams`, one brain on the L4 | 0.789 ms | 0.153 ms | 0.056 ms | 0.040 ms | +| `pool`, four CPU processes | 1.12 ms | 1.13 ms | | | +| `serial`, one core | 2.97 ms | 3.03 ms | | | +| `serial`, each brain on the L4 in turn | 7.86 ms | 7.98 ms | | | + +`--composition plain` (`recall=False` as well) on the same card, `lives` and `streams`: 0.584 ms and 0.451 ms at +12 lives, 0.060 ms and 0.066 ms at 128, 0.021 ms and 0.027 ms at 1,024. + +On an Apple M4 at a load of 4, `--composition lives`, at 12 and 128 lives, without and with the teacher: + +| Way | 12 lives | 128 lives | 12, taught | 128, taught | +| --- | ---: | ---: | ---: | ---: | +| `streams`, one brain on MLX | 0.690 ms | 0.156 ms | 0.572 ms | 0.128 ms | +| `pool`, two CPU processes | 0.595 ms | 0.662 ms | 0.808 ms | 0.727 ms | +| `serial`, one core | 1.42 ms | 0.979 ms | 1.56 ms | 1.48 ms | +| `lives`, `cadence.Lives` on the torch CPU device | 1.17 ms | 1.21 ms | 1.48 ms | 1.32 ms | + +The earlier reading of the plain composition on an L4 (`recall=False, episodic=False`, 20 moments after 3 to +warm) had `lives` at 0.638, 0.063, 0.024 and 0.021 ms and `streams` at 0.468, 0.069, 0.033 and 0.027 ms at 12, +128, 512 and 1,024 lives, a pool of four processes at 0.414 and 0.411 ms and one core at 1.12 ms. + +## On an Apple M4 + +The same ledger on MLX, on an M4 that other work shared at a load of 7 to 10: a speech-size `step` or `live` +waits for the device 9 times a moment after where it waited 11 before (the critic's and the recall memory's +checks join the update's launch), and the chamber's brain, whose memories stay on the host, waits 5 times +before and after. The moments' times there move with the machine's load by more than the change. diff --git a/benchmarks/device/ledger.py b/benchmarks/device/ledger.py new file mode 100644 index 00000000..95b6436c --- /dev/null +++ b/benchmarks/device/ledger.py @@ -0,0 +1,452 @@ +"""The ledger of a moment: every crossing between the host and the device, with its bytes and time. + +A moment of ``live``, ``step``, ``lesson`` or ``act`` on a composed brain runs some of its work on a +device and some on the host. Each time an array goes one way or the other, and each time the host +waits for the device, the moment pays for it. This script lists those crossings by the place in the +library that made them, per moment, beside the seconds of the moment itself. + +Two brains: ``chamber``, the default ``Brain.compose(35, 18)`` (251 neurons, 13,120 synapses), and +``speech``, ``Brain.compose(3920, 37, modules=(512,))`` (12,821 neurons, 6.3 million synapses):: + + python benchmarks/device/ledger.py --backend mlx --brains chamber speech --rows 1 16 + python benchmarks/device/ledger.py --backend torch:cuda --rows 1 16 256 --out ledger.jsonl + python benchmarks/device/ledger.py --backend cpu --rows 1 16 + python benchmarks/device/ledger.py --table ledger.jsonl + +Each reading is two passes over the same moments of one brain. The first, untouched, gives the +seconds of a moment (the median of ``--moments`` after ``--warm`` moments). The second counts: +it wraps the ways an array crosses (on torch a tensor's ``cpu``, ``numpy``, ``item``, ``tolist``, +``to``, ``copy_`` and its conversions to a Python number, ``torch.as_tensor`` and ``torch.tensor`` +onto a device; on MLX ``mx.array`` of a host array, NumPy's ``array``, ``asarray`` and +``ascontiguousarray`` of an MLX array, an array's ``item`` and ``tolist``, and ``mx.eval``) and +names each by the innermost function of the library on the stack. A crossing's time is the time +of its call, which includes the wait for the device's queued work. The counting pass is slower, +so its seconds are not the moment's. The script uses the public API only. + +This is a benchmark of a source checkout: ``PYTHONPATH=src`` from the repository's root. +""" + +from __future__ import annotations + +import argparse +import json +import os +import platform +import sys +import time +import warnings +from collections import defaultdict +from collections.abc import Callable, Iterator +from contextlib import contextmanager +from pathlib import Path +from typing import Any + +import numpy as np + +import cadence as cd + +BRAINS: dict[str, tuple[int, int, dict[str, Any]]] = { + "chamber": (35, 18, {}), + "speech": (3920, 37, {"modules": (512,)}), +} +SOURCE = str(Path(cd.__file__).resolve().parent) + + +def _site() -> str: + """The innermost frames of the library on the stack, ``module.function`` each, the innermost + first: ``DEPTH`` of them.""" + frame = sys._getframe(2) + names: list[str] = [] + while frame is not None and len(names) < DEPTH: + name = frame.f_code.co_filename + if name.startswith(SOURCE): + module = Path(name).stem + names.append(f"{module}.{getattr(frame.f_code, 'co_qualname', frame.f_code.co_name)}") + frame = frame.f_back + return " < ".join(names) or "outside" + + +DEPTH = 1 + + +class Ledger: + """Crossings by site: count, bytes and seconds, in each direction.""" + + def __init__(self) -> None: + self.rows: dict[tuple[str, str], list[float]] = defaultdict(lambda: [0, 0, 0.0]) + self.active = False + + def add(self, way: str, nbytes: int, seconds: float) -> None: + if not self.active: + return + row = self.rows[(way, _site())] + row[0] += 1 + row[1] += nbytes + row[2] += seconds + + def timed(self, way: str, nbytes: Callable[[Any], int], call: Callable[..., Any]) -> Any: + ledger = self + + def wrapped(*args: Any, **kwargs: Any) -> Any: + if not ledger.active: + return call(*args, **kwargs) + start = time.perf_counter() + out = call(*args, **kwargs) + ledger.add(way, nbytes((args, kwargs, out)), time.perf_counter() - start) + return out + + return wrapped + + +def _nbytes(value: Any) -> int: + try: + return int(np.asarray(value).nbytes) if not hasattr(value, "nbytes") else int(value.nbytes) + except Exception: + return 0 + + +@contextmanager +def _torch_counting(ledger: Ledger) -> Iterator[None]: + import torch + + Tensor = torch.Tensor + saved: list[tuple[Any, str, Any]] = [] + + def patch(owner: Any, name: str, new: Any) -> None: + saved.append((owner, name, getattr(owner, name))) + setattr(owner, name, new) + + def on_device(t: Any) -> bool: + return isinstance(t, Tensor) and t.device.type != "cpu" + + def tensor_bytes(t: Any) -> int: + return int(t.numel() * t.element_size()) + + def to_host(name: str) -> None: + original = getattr(Tensor, name) + + def wrapped(self: Any, *args: Any, **kwargs: Any) -> Any: + if not ledger.active or not on_device(self): + return original(self, *args, **kwargs) + start = time.perf_counter() + out = original(self, *args, **kwargs) + ledger.add("to host", tensor_bytes(self), time.perf_counter() - start) + return out + + patch(Tensor, name, wrapped) + + for name in ("cpu", "numpy", "item", "tolist", "__bool__", "__float__", "__int__", + "__index__"): + to_host(name) + + original_to = Tensor.to + + def to(self: Any, *args: Any, **kwargs: Any) -> Any: + if not ledger.active: + return original_to(self, *args, **kwargs) + start = time.perf_counter() + out = original_to(self, *args, **kwargs) + if isinstance(out, Tensor) and out.device.type != self.device.type: + way = "to host" if out.device.type == "cpu" else "to device" + ledger.add(way, tensor_bytes(self), time.perf_counter() - start) + return out + + patch(Tensor, "to", to) + original_copy = Tensor.copy_ + + def copy_(self: Any, src: Any, *args: Any, **kwargs: Any) -> Any: + if not ledger.active or not isinstance(src, Tensor) or src.device.type == self.device.type: + return original_copy(self, src, *args, **kwargs) + start = time.perf_counter() + out = original_copy(self, src, *args, **kwargs) + way = "to host" if self.device.type == "cpu" else "to device" + ledger.add(way, tensor_bytes(src), time.perf_counter() - start) + return out + + patch(Tensor, "copy_", copy_) + + def made(name: str) -> None: + original = getattr(torch, name) + + def wrapped(data: Any, *args: Any, **kwargs: Any) -> Any: + device = kwargs.get("device") + if not ledger.active or device is None or torch.device(device).type == "cpu" or ( + isinstance(data, Tensor) + ): + return original(data, *args, **kwargs) + start = time.perf_counter() + out = original(data, *args, **kwargs) + ledger.add("to device", tensor_bytes(out), time.perf_counter() - start) + return out + + patch(torch, name, wrapped) + + made("as_tensor") + made("tensor") + for module in (getattr(torch, "cuda", None), getattr(torch, "mps", None)): + if module is not None and hasattr(module, "synchronize"): + patch(module, "synchronize", ledger.timed("wait", lambda _: 0, module.synchronize)) + try: + yield + finally: + for owner, name, value in reversed(saved): + setattr(owner, name, value) + + +class _ArrayType(type): + """``mx.array`` in the counting pass: made arrays are counted, ``isinstance`` is MLX's.""" + + real: Any = None + ledger: Ledger | None = None + + def __instancecheck__(cls, obj: Any) -> bool: + return isinstance(obj, cls.real) + + def __call__(cls, data: Any, *args: Any, **kwargs: Any) -> Any: + ledger = cls.ledger + if ledger is None or not ledger.active or isinstance(data, cls.real): + return cls.real(data, *args, **kwargs) + start = time.perf_counter() + out = cls.real(data, *args, **kwargs) + ledger.add("to device", int(out.nbytes), time.perf_counter() - start) + return out + + +@contextmanager +def _mlx_counting(ledger: Ledger) -> Iterator[None]: + import mlx.core as mx + + real = mx.array + saved: list[tuple[Any, str, Any]] = [] + + def patch(owner: Any, name: str, new: Any) -> None: + saved.append((owner, name, getattr(owner, name))) + setattr(owner, name, new) + + proxy = _ArrayType("array", (), {}) + proxy.real, proxy.ledger = real, ledger + patch(mx, "array", proxy) + patch(mx, "eval", ledger.timed("wait", lambda _: 0, mx.eval)) + for name in ("array", "asarray", "ascontiguousarray"): + original = getattr(np, name) + + def wrapped(obj: Any, *args: Any, _original: Any = original, **kwargs: Any) -> Any: + if not ledger.active or not isinstance(obj, real): + return _original(obj, *args, **kwargs) + start = time.perf_counter() + out = _original(obj, *args, **kwargs) + ledger.add("to host", int(obj.nbytes), time.perf_counter() - start) + return out + + patch(np, name, wrapped) + calls: dict[int, float] = {} + + def profile(frame: Any, event: str, arg: Any) -> None: + if event == "c_call" and getattr(arg, "__name__", "") in ("item", "tolist"): + if isinstance(getattr(arg, "__self__", None), real): + calls[id(arg)] = time.perf_counter() + elif event in ("c_return", "c_exception") and id(arg) in calls: + start = calls.pop(id(arg)) + ledger.add("to host", int(arg.__self__.nbytes), time.perf_counter() - start) + + sys.setprofile(profile) + try: + yield + finally: + sys.setprofile(None) + for owner, name, value in reversed(saved): + setattr(owner, name, value) + + +def _counting(backend: str, ledger: Ledger) -> Any: + if backend == "mlx": + return _mlx_counting(ledger) + if backend.startswith("torch"): + return _torch_counting(ledger) + + @contextmanager + def nothing() -> Iterator[None]: + yield + + return nothing() + + +def _place(backend: str) -> dict[str, Any]: + if backend.startswith("torch"): + parts = backend.split(":") + place: dict[str, Any] = {"backend": "torch"} + if len(parts) > 1: + place["device"] = parts[1] + if len(parts) > 2: + place["precision"] = parts[2] + return place + return {"backend": backend} + + +def _sync(brain: cd.Brain) -> None: + kernel = getattr(brain.brain, "_torch", None) + if kernel is not None and kernel.device.type == "cuda": + kernel.torch.cuda.synchronize() + + +def _brain(name: str, backend: str, seed: int, arousal: bool) -> cd.Brain: + inputs, motor, options = BRAINS[name] + if arousal: + options = {**options, "arousal": True} + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + return cd.Brain.compose(inputs, motor, seed=seed, **options, **_place(backend)) + + +def _moments( + brain: cd.Brain, kind: str, rows: int, count: int, seed: int +) -> Iterator[Callable[[], Any]]: + """``count`` moments of ``kind``, each a call with its observations drawn here.""" + inputs, motor = brain.inputs, len(brain.motor_index) + rng = np.random.default_rng(seed) + + def observed() -> np.ndarray: + return rng.random((rows, inputs)) * (rng.random((rows, inputs)) < 0.3) + + for t in range(count): + x = observed() + if kind == "live": + reward = np.array([float(rng.random() < 0.3)]) if t else None + yield lambda x=x, r=reward: brain.live(x, **({} if r is None else {"reward": r})) + elif kind == "step": + reward = (rng.random(rows) < 0.3).astype(float) if t else None + yield lambda x=x, r=reward: brain.step(x, **({} if r is None else {"reward": r})) + elif kind == "lesson": + labels = rng.integers(0, motor, rows) + yield lambda x=x, y=labels: brain.lesson(x, y) + elif kind == "act": + yield lambda x=x: brain.act(x, greedy=True) + else: + raise ValueError(kind) + + +def read(name: str, backend: str, kind: str, rows: int, warm: int, count: int, seed: int, + ledger: bool) -> dict[str, Any]: + """One brain, one kind of moment, one batch: seconds per moment and the crossings.""" + if kind == "live" and rows != 1: + raise ValueError("live follows one stream") + out: dict[str, Any] = {"brain": name, "backend": backend, "kind": kind, "rows": rows} + brain = _brain(name, backend, seed, kind == "live") + seconds = [] + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + for t, call in enumerate(_moments(brain, kind, rows, warm + count, seed)): + start = time.perf_counter() + call() + _sync(brain) + if t >= warm: + seconds.append(time.perf_counter() - start) + placement = getattr(brain, "placement", None) # releases before 0.95.0 have none + out["placement"] = None if placement is None else { + k: placement[k] for k in ("backend", "device", "precision") + } + out["synapses"] = int(brain.connectome.synapses) + out["seconds"] = float(np.median(seconds)) + out["seconds_mean"] = float(np.mean(seconds)) + if not ledger: + return out + book = Ledger() + with warnings.catch_warnings(), _counting(backend, book): + warnings.simplefilter("ignore") + for call in _moments(brain, kind, rows, count, seed + 1): + book.active = True + call() + _sync(brain) + book.active = False + out["crossings"] = sorted( + ( + {"way": way, "site": site, "count": row[0] / count, "bytes": row[1] / count, + "seconds": row[2] / count} + for (way, site), row in book.rows.items() + ), + key=lambda r: -r["seconds"], + ) + return out + + +def environment() -> dict[str, Any]: + env: dict[str, Any] = { + "machine": platform.machine(), "system": platform.system(), + "python": sys.version.split()[0], "cadence": getattr(cd, "__version__", "?"), + "backends": cd.available_backends(), + } + try: + env["load_average"] = os.getloadavg() + except OSError: + pass + try: + import torch + + env["torch"] = torch.__version__ + if torch.cuda.is_available(): + env["card"] = torch.cuda.get_device_name(0) + except ImportError: + pass + return env + + +def table(paths: list[str]) -> None: + for path in paths: + for line in Path(path).read_text().splitlines(): + row = json.loads(line) + if "brain" not in row: + continue + cross = row.get("crossings", []) + count = sum(c["count"] for c in cross if c["way"] != "wait") + waits = sum(c["count"] for c in cross if c["way"] == "wait") + nbytes = sum(c["bytes"] for c in cross) + spent = sum(c["seconds"] for c in cross) + print(f"{row['brain']:8} {row['backend']:12} {row['kind']:7} {row['rows']:>4} " + f"{row['seconds'] * 1e3:9.2f} ms crossings {count:7.1f} waits {waits:5.1f} " + f"{nbytes / 1e3:10.1f} kB {spent * 1e3:8.2f} ms in them") + for c in cross[:12]: + print(f" {c['way']:9} {c['site'][:100]:100} {c['count']:7.2f} x " + f"{c['bytes'] / 1e3:10.2f} kB {c['seconds'] * 1e3:8.3f} ms") + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) + parser.add_argument("--backend", default="cpu", + help="cpu, mlx, torch, torch:cuda, torch:mps, torch:cuda:float64") + parser.add_argument("--brains", nargs="+", default=["chamber", "speech"], choices=BRAINS) + parser.add_argument("--kinds", nargs="+", default=["live", "step", "lesson"], + choices=["live", "step", "lesson", "act"]) + parser.add_argument("--rows", nargs="+", type=int, default=[1, 16]) + parser.add_argument("--warm", type=int, default=5) + parser.add_argument("--moments", type=int, default=20) + parser.add_argument("--seed", type=int, default=0) + parser.add_argument("--no-ledger", action="store_true", help="seconds only") + parser.add_argument("--depth", type=int, default=1, + help="library frames that name a crossing's site, the innermost first") + parser.add_argument("--out") + parser.add_argument("--table", nargs="+") + args = parser.parse_args() + if args.table: + table(args.table) + return + global DEPTH + DEPTH = args.depth + sink = open(args.out, "a") if args.out else None + env = environment() + print(json.dumps({"environment": env}), file=sink or sys.stdout, flush=True) + for name in args.brains: + for kind in args.kinds: + for rows in args.rows: + if kind == "live" and rows != 1: + continue + row = read(name, args.backend, kind, rows, args.warm, args.moments, args.seed, + not args.no_ledger) + print(json.dumps(row), file=sink or sys.stdout, flush=True) + if sink: + print(f"{name} {kind} {rows}: {row['seconds'] * 1e3:.2f} ms", flush=True) + if sink: + sink.close() + + +if __name__ == "__main__": + main() diff --git a/benchmarks/device/life.py b/benchmarks/device/life.py new file mode 100644 index 00000000..edf43630 --- /dev/null +++ b/benchmarks/device/life.py @@ -0,0 +1,84 @@ +"""Seconds of a whole life: compose, place, and step to the end, under a backend or ``auto``. + +A life is a brain composed fresh and ``--moments`` moments of ``step`` on ``--rows`` streams, with +rewards from the second moment on; its seconds run from before ``Brain.compose`` to after the last +moment. ``--plan`` calibrates the torch transport at the life's rows before the first moment +(``NeuralGraph.calibrate_transport``), and its seconds are counted in the life:: + + python benchmarks/device/life.py --brain chamber --rows 256 --moments 300 + python benchmarks/device/life.py --brain speech --rows 1 --moments 100 --plan + python benchmarks/device/life.py --brain speech --rows 16 --backend cpu --moments 20 + +This is a benchmark of a source checkout: ``PYTHONPATH=src`` from the repository's root. +""" + +from __future__ import annotations + +import argparse +import json +import sys +import time +import warnings +from pathlib import Path +from typing import Any + +import numpy as np + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from ledger import BRAINS, _place, _sync, environment # noqa: E402 + +import cadence as cd # noqa: E402 + + +def life(name: str, backend: str, rows: int, moments: int, plan: bool, seed: int) -> dict[str, Any]: + inputs, motor, options = BRAINS[name] + rng = np.random.default_rng(seed) + xs = [rng.random((rows, inputs)) * (rng.random((rows, inputs)) < 0.3) for _ in range(moments)] + rs = [(rng.random(rows) < 0.3).astype(float) for _ in range(moments)] + place = {} if backend == "auto" else _place(backend) + warnings.simplefilter("ignore") + start = time.perf_counter() + brain = cd.Brain.compose(inputs, motor, seed=seed, **options, **place) + composed = time.perf_counter() - start + brain.step(xs[0]) + _sync(brain) + first = time.perf_counter() - start - composed + calibrated = 0.0 + if plan and brain.brain._torch is not None: + began = time.perf_counter() + brain.brain.calibrate_transport(batches=(rows,)) + calibrated = time.perf_counter() - began + for t in range(1, moments): + brain.step(xs[t], reward=rs[t - 1]) + _sync(brain) + total = time.perf_counter() - start + placement = getattr(brain, "placement", {}) or {} + return { + "brain": name, "backend": backend, "rows": rows, "moments": moments, "plan": plan, + "placed": placement.get("backend"), "device": placement.get("device"), + "seconds": total, "composed": composed, "first": first, "calibrated": calibrated, + "per_moment": (total - composed - first) / max(moments - 1, 1), + } + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) + parser.add_argument("--brain", default="chamber", choices=BRAINS) + parser.add_argument("--backend", default="auto") + parser.add_argument("--rows", type=int, default=16) + parser.add_argument("--moments", type=int, default=100) + parser.add_argument("--plan", action="store_true") + parser.add_argument("--seed", type=int, default=0) + parser.add_argument("--out") + args = parser.parse_args() + row = life(args.brain, args.backend, args.rows, args.moments, args.plan, args.seed) + line = json.dumps({**row, "environment": environment()}) + if args.out: + with open(args.out, "a") as sink: + print(line, file=sink) + print(f"{row['brain']} {row['backend']} -> {row['placed']} rows {row['rows']} " + f"plan {row['plan']}: {row['seconds']:.2f} s, {row['per_moment'] * 1e3:.2f} ms a moment") + + +if __name__ == "__main__": + main() diff --git a/benchmarks/device/lives.py b/benchmarks/device/lives.py new file mode 100644 index 00000000..b7ce1a53 --- /dev/null +++ b/benchmarks/device/lives.py @@ -0,0 +1,224 @@ +"""Seconds per life: many continuing lives of one topology, four ways. + +A life is ``Brain.compose(35, 18, seed=k)``, the chamber's default brain, stepping ``--moments`` +moments of ``step`` on observations and rewards of its own. ``--composition lives`` composes it +with ``episodic=False``, the default composition without the store, which ``cadence.Lives`` holds; +``--composition plain`` with ``recall=False`` as well. ``--teach N`` gives every N-th moment a +teacher's label. Four ways to run ``--lives`` of them: + +* ``serial``: one process steps every life in turn on ``--backend``; each life is its own brain. +* ``pool``: ``--workers`` processes on the CPU, one thread each, each stepping its share of the + lives in turn; each life is its own brain. +* ``streams``: one brain on ``--backend`` steps every life as one of its streams, in one call per + moment. Its streams share the brain's parameters and learning, so this is one brain's lives, a + bound on what a batch of independent lives in one call could reach on that device, and not + independent lives. +* ``lives``: ``cadence.Lives`` on ``--backend``, every life a brain of its own with its own + parameters, one call per moment for all of them (``--composition lives`` or ``plain``). + +Each reading states the seconds to build the lives and the seconds per life and moment of the +stepping that follows (after ``--warm`` moments of each life):: + + python benchmarks/device/lives.py --ways serial pool --lives 12 128 --workers 8 + python benchmarks/device/lives.py --ways streams --backend mlx --lives 12 128 + python benchmarks/device/lives.py --ways streams serial --backend torch:cuda --lives 12 128 + python benchmarks/device/lives.py --composition lives --ways lives streams pool \ + --backend torch:cuda --lives 12 128 + +This is a benchmark of a source checkout: ``PYTHONPATH=src`` from the repository's root. +""" + +from __future__ import annotations + +import argparse +import json +import os +import sys +import time +import warnings +from concurrent.futures import ProcessPoolExecutor +from pathlib import Path +from typing import Any + +import numpy as np + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from ledger import _place, _sync, environment # noqa: E402 + +import cadence as cd # noqa: E402 + +INPUTS, MOTOR = 35, 18 +COMPOSITION: dict[str, object] = {} # ``--composition``: lives episodic=False; plain no recall +TEACH = 0 # every ``TEACH``-th moment a teacher's label, 0 none + + +def _world(seed: int, rows: int, moments: int) -> tuple[list[np.ndarray], list[np.ndarray]]: + """Every moment's observations and the rewards of the moment before, for ``rows`` lives.""" + rng = np.random.default_rng(seed) + xs = [rng.random((rows, INPUTS)) * (rng.random((rows, INPUTS)) < 0.3) for _ in range(moments)] + rs = [(rng.random(rows) < 0.3).astype(float) for _ in range(moments)] + return xs, rs + + +def _teacher(t: int, rows: int) -> np.ndarray | None: + """The teacher's labels of moment ``t``, a function of the moment alone; None between.""" + if not TEACH or t % TEACH: + return None + return np.asarray((np.arange(rows) * 7 + t) % MOTOR) + + +def _compose(seed: int, backend: str) -> cd.Brain: + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + return cd.Brain.compose(INPUTS, MOTOR, seed=seed, **_place(backend), **COMPOSITION) + + +def _live(brain: cd.Brain, xs: list[np.ndarray], rs: list[np.ndarray], first: int, last: int, + row: slice) -> None: + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + for t in range(first, last): + teacher = _teacher(t, len(xs[t])) + taught = {} if teacher is None else {"teacher": teacher[row]} + if t: + brain.step(xs[t][row], reward=rs[t][row], **taught) + else: + brain.step(xs[t][row], **taught) + + +def _share( + lives: list[int], backend: str, moments: int, warm: int, + composition: dict[str, Any] | None = None, teach: int | None = None, +) -> tuple[float, float]: + """A worker's lives, in turn: seconds to build them and seconds of their measured moments.""" + global TEACH + if composition is not None: + COMPOSITION.update(composition) + if teach is not None: + TEACH = teach + start = time.perf_counter() + brains = [_compose(k, backend) for k in lives] + built = time.perf_counter() - start + worlds = [_world(1000 + k, 1, moments + warm) for k in lives] + for brain, (xs, rs) in zip(brains, worlds, strict=True): + _live(brain, xs, rs, 0, warm, slice(None)) + _sync(brain) + start = time.perf_counter() + for brain, (xs, rs) in zip(brains, worlds, strict=True): + _live(brain, xs, rs, warm, warm + moments, slice(None)) + for brain in brains: + _sync(brain) + return built, time.perf_counter() - start + + +def serial(lives: int, backend: str, moments: int, warm: int) -> dict[str, Any]: + built, stepped = _share(list(range(lives)), backend, moments, warm) + return {"built": built, "per_life_moment": stepped / (lives * moments)} + + +def pool(lives: int, workers: int, moments: int, warm: int) -> dict[str, Any]: + shares = [list(range(lives))[w::workers] for w in range(workers)] + shares = [s for s in shares if s] + for name in ("OMP_NUM_THREADS", "OPENBLAS_NUM_THREADS", "MKL_NUM_THREADS", + "VECLIB_MAXIMUM_THREADS", "NUMBA_NUM_THREADS"): + os.environ[name] = "1" # the workers' arithmetic, one thread each + with ProcessPoolExecutor(len(shares)) as executor: + executor.submit(int, 0).result() # the workers start before the clock + start = time.perf_counter() + results = list(executor.map(_share, shares, ["cpu"] * len(shares), + [moments] * len(shares), [warm] * len(shares), + [dict(COMPOSITION)] * len(shares), [TEACH] * len(shares))) + wall = time.perf_counter() - start + built = max(r[0] for r in results) + stepped = max(r[1] for r in results) + return {"built": built, "per_life_moment": stepped / (lives * moments), "wall": wall, + "workers": len(shares)} + + +def streams(lives: int, backend: str, moments: int, warm: int) -> dict[str, Any]: + start = time.perf_counter() + brain = _compose(0, backend) + built = time.perf_counter() - start + xs, rs = _world(1000, lives, moments + warm) + _live(brain, xs, rs, 0, warm, slice(None)) + _sync(brain) + start = time.perf_counter() + _live(brain, xs, rs, warm, warm + moments, slice(None)) + _sync(brain) + return {"built": built, "per_life_moment": (time.perf_counter() - start) / (lives * moments), + "placement": brain.placement["backend"]} + + +def batched(lives: int, backend: str, moments: int, warm: int) -> dict[str, Any]: + """``cadence.Lives``: the lives' own parameters, one call per moment for all of them.""" + place = _place(backend) + start = time.perf_counter() + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + held = cd.Lives(INPUTS, MOTOR, lives, **place, **{ + k: v for k, v in COMPOSITION.items() if k != "episodic" + }) + built = time.perf_counter() - start + xs, rs = _world(1000, lives, moments + warm) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + for t in range(warm): + held.step(xs[t], teacher=_teacher(t, lives), **({"reward": rs[t]} if t else {})) + start = time.perf_counter() + for t in range(warm, warm + moments): + held.step(xs[t], reward=rs[t], teacher=_teacher(t, lives)) + if held.brain is not None: + _sync(held.brain) + return {"built": built, "per_life_moment": (time.perf_counter() - start) / (lives * moments)} + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) + parser.add_argument("--ways", nargs="+", default=["serial", "pool", "streams"], + choices=["serial", "pool", "streams", "lives"]) + parser.add_argument("--composition", default="default", + choices=["default", "lives", "plain"]) + parser.add_argument("--teach", type=int, default=0, + help="every N-th moment a teacher's label (0: none)") + parser.add_argument("--backend", default="cpu") + parser.add_argument("--lives", nargs="+", type=int, default=[12, 128]) + parser.add_argument("--workers", type=int, default=os.cpu_count() or 1) + parser.add_argument("--moments", type=int, default=20) + parser.add_argument("--warm", type=int, default=3) + parser.add_argument("--out") + args = parser.parse_args() + global TEACH + TEACH = args.teach + if args.composition == "lives": + COMPOSITION.update(episodic=False) + elif args.composition == "plain": + COMPOSITION.update(recall=False, episodic=False) + elif "lives" in args.ways: + parser.error("the lives way holds --composition lives or plain") + sink = open(args.out, "a") if args.out else sys.stdout + print(json.dumps({"environment": environment()}), file=sink, flush=True) + for lives in args.lives: + for way in args.ways: + if way == "pool": + row = pool(lives, args.workers, args.moments, args.warm) + backend = "cpu" + elif way == "lives": + row = batched(lives, args.backend, args.moments, args.warm) + backend = args.backend + elif way == "serial": + row = serial(lives, args.backend, args.moments, args.warm) + backend = args.backend + else: + row = streams(lives, args.backend, args.moments, args.warm) + backend = args.backend + row = {"way": way, "lives": lives, "backend": backend, + "composition": args.composition, "teach": args.teach, **row} + print(json.dumps(row), file=sink, flush=True) + if sink is not sys.stdout: + print(f"{way:8} {backend:11} {lives:4} lives " + f"{row['per_life_moment'] * 1e3:8.3f} ms per life and moment, " + f"built in {row['built']:.2f} s", flush=True) + + +if __name__ == "__main__": + main() diff --git a/benchmarks/device/test_device_benchmarks.py b/benchmarks/device/test_device_benchmarks.py new file mode 100644 index 00000000..8c985602 --- /dev/null +++ b/benchmarks/device/test_device_benchmarks.py @@ -0,0 +1,58 @@ +"""The device benchmarks run, and the ledger counts a crossing where one is made (cadence#274).""" + +from __future__ import annotations + +import importlib.util +import sys +from pathlib import Path + +import numpy as np +import pytest + +HERE = Path(__file__).resolve().parent +sys.path.insert(0, str(HERE)) +import ledger # noqa: E402 +import lives # noqa: E402 + +# by its path under its own name: benchmarks/vocabulary holds a ``life`` module too, and one +# pytest process imports both +_spec = importlib.util.spec_from_file_location("device_life", HERE / "life.py") +assert _spec is not None and _spec.loader is not None +life = importlib.util.module_from_spec(_spec) +sys.modules["device_life"] = life +_spec.loader.exec_module(life) + + +def test_a_cpu_moment_has_no_crossing(): + row = ledger.read("chamber", "cpu", "step", 2, 1, 2, 0, True) + assert row["seconds"] > 0 and row["crossings"] == [] + assert row["placement"]["backend"] == "cpu" + + +def test_the_ledger_names_the_site_of_a_crossing(): + torch = pytest.importorskip("torch") + book = ledger.Ledger() + tensor = torch.ones(3) + + def crossing(): + book.add("to host", int(tensor.numel() * tensor.element_size()), 0.001) + + book.active = True + crossing() + (way, site), (count, nbytes, seconds) = next(iter(book.rows.items())) + assert way == "to host" and count == 1 and nbytes == 12 and seconds == 0.001 + assert site == "outside" # no library frame made it + + +def test_live_follows_one_stream(): + with pytest.raises(ValueError, match="one stream"): + ledger.read("chamber", "cpu", "live", 2, 0, 1, 0, False) + + +def test_a_life_and_lives_run_on_the_cpu(): + row = life.life("chamber", "cpu", 2, 3, False, 0) + assert row["placed"] == "cpu" and row["seconds"] >= row["composed"] + row["first"] + out = lives.serial(2, "cpu", 2, 1) + assert out["per_life_moment"] > 0 + out = lives.streams(2, "cpu", 2, 1) + assert out["placement"] == "cpu" and np.isfinite(out["per_life_moment"]) diff --git a/benchmarks/grammar/EXPOSURE_PROTOCOL.md b/benchmarks/grammar/EXPOSURE_PROTOCOL.md new file mode 100644 index 00000000..38c0bccd --- /dev/null +++ b/benchmarks/grammar/EXPOSURE_PROTOCOL.md @@ -0,0 +1,100 @@ +# Exposure parity with rehearsal: declared protocol (cadence#270) + +Declared on 2026-11-09, committed to the lane branch before the first life. The founders, arms, +readings and gates below are fixed; a failed gate is reported as failed. + +## The question + +The transformer's offline control replays the grammar chamber's 4,000 training batches in shuffled +epochs until its loss stops falling, and reads 1.00 on the trained panels. A brain hears each batch +once. Does the default composition with its rehearsal (`learning_rehearsal=1.0`: after each lesson as +many rows again, drawn with their episodes from a reservoir of earlier lesson rows), taught the same +batches in passes, reach every panel within .05 of the transformer's offline reading at the +transformer's compute? If not, which mechanism keeps a panel apart? + +## Instrument + +`benchmarks/grammar/exposure.py` (`plan`, `run`, `read`) and `benchmarks/grammar/controls.py`. A +brain life is `Brain.compose(35, 18, modules=(64,), seed=founder, backend="cpu")` with the arm's +settings, nothing else named. The first grammar's 4,000 batches of the founder's stream, taught in +passes: the first in the stream's order, each later one in the order of the transformer's next +offline epoch. After each pass the eight readings below are read on 256-sentence panels; after the +last, `located` for the recall read. A life stops after 4 passes, or after the first pass that raises +no reading's mean by more than .01 over the pass before it. Compute is the module's analytic count. + +The control: `controls.py`'s transformer, offline, its width matched to the brain's `parameters()`, +on the same batches and panels, to convergence (patience 3, at most 30 epochs). + +## Founders and arms + +Founders 9627000 to 9627005 (no life in any lane). Arms `rehearsal` (`learning_rehearsal=1.0`) and +`default` (no rehearsal), twelve brain lives and six control lives. + +## Readings + +Clauses, held-out clauses, clauses at three and four distractors, roles (both right), brackets, +brackets at depths four and five, paths, three hops: the exact sentences of `clause`, `clause_held`, +`clause_long`, `roles`, `dyck`, `dyck_deep`, `path` and `path_long`. + +## Gates + +| Gate | Reading | Pass | +| --- | --- | --- | +| E1 | `rehearsal` at the transformer's offline compute: each founder's last pass whose cumulative compute is at most that founder's control's (the first pass where none is) | every reading's mean at least the control's mean less .05 | +| E2 | `rehearsal` at its last pass (saturation or the fourth) | the same | +| E3 | `default` at E1's rule and at its last pass | reported beside E1 and E2, no bar | + +**Verdict.** The acceptance's parity line is met when E1 passes. Where E1 fails, the mechanism that +keeps each apart panel apart is read from three things recorded before any reading: the panel's mean +per pass (a panel that rises with passes and stops short at the control's compute is apart by +compute; one that stays flat across passes is apart by mechanism), `rehearsal` against `default` at +equal passes (what the rehearsal adds), and `located` on that panel at the last pass (a word not +delivered by the read is lost at the read; one delivered and not said is lost at the readout). The +reading is recorded on the issue whichever way it falls. + +## Compute + +One AWS c7i.16xlarge (or this Mac where the box is unavailable, named with the results), one process +per life, one BLAS thread per process. + +## Result + +Read on one c7i.16xlarge, founders 9627000 to 9627005, 256-sentence panels. The transformer offline +(width 40, 28,658 parameters against the brain's 28,716) converged in 13 to 30 epochs at a mean of +1.30e12 multiply-accumulates. + +| Arm, pass | Compute | Clauses | Held-out | Long clauses | Roles | Brackets | Deep brackets | Paths | Three hops | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | +| Transformer offline | 1.30e12 | 1.000 | .999 | .000 | .995 | .966 | .003 | .997 | .000 | +| Default, pass 1 | 0.95e12 | .995 | .952 | .001 | .966 | .840 | .042 | .965 | .519 | +| Default, pass 2 | | .999 | .971 | .003 | .980 | .896 | .044 | .979 | .439 | +| Default, pass 3 | | .995 | .957 | .002 | .969 | .961 | .057 | .990 | .355 | +| Default, pass 4 | | .980 | .936 | .004 | .941 | .988 | .062 | .987 | .347 | +| Rehearsal, pass 1 | 5.44e12 | .953 | .914 | .003 | .909 | .852 | .051 | .822 | .247 | +| Rehearsal, pass 2 | | .992 | .980 | .003 | .975 | .831 | .014 | .835 | .197 | +| Rehearsal, pass 3 | | .997 | .985 | .002 | .989 | .901 | .020 | .927 | .177 | +| Rehearsal, pass 4 | | .987 | .922 | .002 | .961 | .898 | .008 | .891 | .109 | + +| Gate | Verdict | Readings apart by more than .05 | +| --- | --- | --- | +| E1, rehearsal at the transformer's compute | fail | no pass fits; at pass 1 held-out, roles, brackets, paths | +| E2, rehearsal at its last pass | fail | held-out, brackets, paths | +| E3, default at the transformer's compute | reported | brackets (.862 against .966) | +| E3, default at its last pass | reported | held-out (.934 against .999), brackets (.898) | + +What keeps each panel apart: + +- **The rehearsal's cost.** A rehearsal replays each drawn row's whole episode on fresh streams, about + 3.7 million replayed row moments a pass against the 0.8 million the life hears, so a pass with it costs 5.7 times a pass + without; at the transformer's compute it has less than one pass. +- **Paths under rehearsal: the read.** The recall read delivers a hop's fact on .78 to .89 of hops + with the rehearsal against .98 to 1.00 without it, and says a delivered hop on .90 to .96; paths + fall from .97 to .82 at pass 1 and stay under .93. +- **Brackets: exposure.** Without rehearsal brackets rise with every pass, .84, .90, .96, .99, and + reach the control's .966 at pass 3 (about twice its compute). A closing bracket is said right on .96 to + .99 while the read delivers its open bracket on .32 to .75, so brackets are carried by the trace + and the cortex more than by the read. +- **Held-out clauses: no exposure lever.** Without rehearsal they hold at .94 to .97 over four passes + against the control's .999. +- Long clauses and deep brackets read near zero for every learner, and the brain leads on three hops + (.48 against .00). diff --git a/benchmarks/grammar/LESSON_STEP_PROTOCOL.md b/benchmarks/grammar/LESSON_STEP_PROTOCOL.md new file mode 100644 index 00000000..72f1eb93 --- /dev/null +++ b/benchmarks/grammar/LESSON_STEP_PROTOCOL.md @@ -0,0 +1,95 @@ +# The lesson's step that follows the running error: declared protocol (cadence#258) + +Declared on 2026-11-09, committed to the lane branch before the first life of either stage. The +founders, arms, readings, selection rule and gates below are fixed; a failed gate is reported as +failed. + +## The question + +On the default composition of 0.95.2 a validation reading on the grammar chamber swings between +checkpoints, because a lesson with few wrong rows takes a step of full size. A step that falls once +to a third of the rates late in a life settled it on development lives, and a count of lessons is a +parameter a continuing life does not know the end of. Does a step whose size follows the brain's +running error, with no count, settle the exact-clause reading on fresh founders at widths 64 and 128, +with every chamber reading and every sequence-battery bar holding? + +## The candidate + +`learning_running_error=ρ` (opt-in, `LearnerConfig.running_error`): the learner keeps the running +share of lesson rows answered wrong, `e ← ρ e + (1 − ρ) w`, where `w` is the share of a lesson's rows +whose answer differs from the label (a lesson the brain answers right counts as zero), starting at +one. A lesson's step is scaled by `e` to the power `learning_running_error_power` (0.5 unless named). +It keeps one number and no count. + +## Instruments + +- Grammar chamber: `benchmarks/grammar/grammar.py` through `benchmarks/grammar/lesson_step.py` + (`plan`, `read`). The `default` arm: `Brain.compose(35, 18, modules=(w,), seed=founder, + backend="cpu")` and the arm's settings, nothing else named. The first grammar for 4,000 batches, + 256-sentence panels read every 500 batches and in full at 4,000. +- Sequence battery: `benchmarks/sequence/battery.py`, arm `default` (the same composition, nothing + named but the backend), 1,500 lessons and 100 test episodes per life, the tasks of + `lesson_step.BATTERY`. + +## Readings + +- Swing: the largest minus the smallest exact-clause reading at 3,000, 3,500 and 4,000 batches. +- Clauses, held-out clauses, roles (both right), two hops: the full reading at 4,000. +- Brackets, paths: the means of the readings at 2,000, 2,500, 3,000, 3,500 and 4,000. + +## Stage 1, development (selection) + +Founders 0, 1, 2 at widths 64 and 128; arms `control` (the default composition) and the four +candidates `e99p5`, `e999p5`, `e99p1`, `e999p1` (ρ 0.99 or 0.999, power 0.5 or 1). Thirty lives. + +**Selection rule** (`lesson_step.select`): among the candidates whose mean clauses, held-out clauses, +brackets and paths are each at least the control's less .05 at both widths, the one with the lowest +mean clause swing over the two widths; ties go to the smaller power, then the larger ρ. With none +eligible, the lowest swing overall goes forward, flagged. Development readings select and decide +nothing else. + +## Stage 2, confirmation (fresh founders) + +Grammar chamber: founders 9625800 to 9625805, widths 64 and 128, the control and the selected +candidate (24 lives). Battery: founders 9625810 to 9625815, the control and the candidate (nine +readings from eight lives per founder and arm). No founder of this block has lived in any lane. + +| Gate | Reading | Pass | +| --- | --- | --- | +| C0 | the control's swing at each width | read first and reported, no bar | +| C1 (64), C1 (128) | the candidate's clause swing | at most .05 on at least 5 of 6 founders, at each width | +| C2 (64), C2 (128) | the candidate's mean clauses, held-out, roles, brackets, paths and two hops | each at least the control's less .03 | +| C3 | every battery task: look back first and second, copying unique and repeated, period 40, before, two hops with a think moment, induction, n-back | the candidate passes the task's bar (.90; copying unique .95, period 40 .85) on at least as many founders as the control or on 5 of 6, and its mean is at least the control's less .03 | + +**Verdict.** #258's narrowed acceptance is met when C1, C2 and C3 all pass; the option then ships as +an opt-in, and a default is the owner's decision. A failed C1 is a measured negative for this form of +the step. A failed C2 or C3 with C1 passing records the cost that keeps the form out. + +## Compute + +One AWS c7i.16xlarge, one process per life, one BLAS thread per process; the count of workers enters +no reading. + +## Amendment during the confirmation + +`battery.py`'s n-back task takes no test lengths, so its battery lives raised before reading; the +n-back row reads the lengths it teaches (6 to 12), as the battery reads it. The chamber's lives had +finished; no other row changes. + +## Result + +Read on one c7i.16xlarge. Development selected `e999p5` with the flag (no candidate held brackets +within .05 of the control at width 64). Confirmation, founders 9625800 to 9625805: + +| Gate | Verdict | Reading | +| --- | --- | --- | +| C0, the control | read | swing .042 at 64 (4 of 6 within .05), .024 at 128 (5 of 6) | +| C1 (64) | fail | swing .042, 4 of 6 within | +| C1 (128) | fail | swing .039, 3 of 6 within | +| C2 (64) | fail | held-out -.101, brackets -.113, roles -.060 | +| C2 (128) | fail | held-out -.128, roles -.053 | +| C3 | pass | every battery task within .03 of the control, no bar lost | + +A measured negative for a step that follows the running error: it settles no more lives than the +default composition and costs held-out clauses. `learning_running_error` is withdrawn and the library +is unchanged; the candidate's lives run on commit 968a02b. diff --git a/benchmarks/grammar/README.md b/benchmarks/grammar/README.md index 51256416..f8e7a062 100644 --- a/benchmarks/grammar/README.md +++ b/benchmarks/grammar/README.md @@ -112,6 +112,7 @@ in the same order. | `recipe` | `episodic=False, teaching="normalized"` with the recall memory's recipe for lengths no lesson reached and its relative read (`recall=True, recall_recency=0.0, learning_surprise=True, recall_boundary=True, recall_anchor=True, recall_slope=0.1, recall_position_rate=0.01, recall_relative=True`): the sequence battery's `recipe_relative`, the composition the recall guide gives for stories longer than the lessons and for copying | | `maturity` | `recipe` with `maturity=True` | | `frozen` | the `recipe` newborn, never taught | +| `default` | `Brain.compose` with nothing named but the backend: what a new brain gets (`recall=True` and its settings) | **Persistence and recovery.** The first grammar for 4,000 batches, the second grammar alone for 1,500, then the first again for 500 (recovery), each phase read in full. Lessons leave @@ -683,6 +684,61 @@ These lives say which state lifts this chamber, and none of this is read on fres read beside them stay out of the library because nothing chosen uses them: a brain built with no trace, and the read carried to the next moment on a region of its own. +## Does a life settle? The lesson's step (#258) + +[`lesson_step.py`](lesson_step.py) reads [#258](https://github.com/muellerberndt/cadence/issues/258) +under [its declared protocol](LESSON_STEP_PROTOCOL.md): the `default` arm for 4,000 batches at width +64 or 128, 256-sentence panels every 500 batches. A life's swing is the largest minus the smallest +exact-clause reading at 3,000, 3,500 and 4,000 batches; brackets and paths are the means of the last +five readings. `plan` writes the lives of a stage, `read` the table, the selection and the gates +(`--battery` adds the sequence battery's bars, `battery.py`'s `default` arm). + +On fresh founders 9625800 to 9625805 the default composition settles 9 of 12 lives within .05 with +no new step: + +| Arm | Width | Swing (within .05) | Clauses | Held-out | Roles | Brackets | Paths | Two hops | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | +| Default | 64 | .042 (4 of 6) | .989 | .925 | .953 | .840 | .861 | .771 | +| Default | 128 | .024 (5 of 6) | .993 | .968 | .978 | .755 | .960 | .956 | +| Step by the running error | 64 | .042 (4 of 6) | .974 | .824 | .893 | .727 | .917 | .893 | +| Step by the running error | 128 | .039 (3 of 6) | .979 | .840 | .924 | .764 | .964 | .944 | + +The step scaled by the running share of lesson rows answered wrong (forgetting .999, square root) +settles no more lives and costs held-out clauses by .10 to .13; the battery holds. It is not in the +library, and its lives run on commit 968a02b. + +## Exposure parity with rehearsal (#270) + +[`exposure.py`](exposure.py) reads [#270](https://github.com/muellerberndt/cadence/issues/270) under +[its declared protocol](EXPOSURE_PROTOCOL.md): the `default` arm with and without +`learning_rehearsal=1.0` taught the chamber's 4,000 batches in passes (the first in the stream's +order, the later ones in the transformer's offline epoch order), every first-grammar panel read +after each pass, beside `controls.py`'s transformer offline at the brain's parameter count. Compute +is an analytic count of the row sweeps times the synapses. + +On founders 9627000 to 9627005 the default composition at the transformer's compute (one pass, +0.95e12 against 1.30e12) reads clauses .996, held-out .953, roles .964, brackets .862, paths .969 +and three hops .475, against the control's 1.000, .999, .995, .966, .997 and .000. Brackets rise +with passes to .99 at the fourth; held-out clauses hold at .94 to .97. The rehearsal costs 5.7 times +a pass and lowers paths to .82 at the first pass, where the recall read delivers .78 to .89 of hops +against .98 to 1.00 without it. + +## Routes with a goal: does planning pay? (#257) + +[`routes.py`](routes.py) reads [#257](https://github.com/muellerberndt/cadence/issues/257) under +[its declared protocol](ROUTES_PROTOCOL.md): the world says two branches from a start as `X near Y +end` facts, the meaning names the start and the goal at the end of one branch, and the brain says +`start near a1 near ... near goal end`. The first hop is said before the goal, and only one +neighbour of the start reaches it. Eight start-goal pairs per founder are held out of training. +Speakers: word by word, a sliding window of three (plan three words, say the first, hear it, plan +again), and the matched LSTM and transformer. + +Founders 9625700 to 9625705, 8,000 batches, first hop on held-out pairs: word by word .334, the +sliding window .312 (paired difference -.022, s.e. .050), the LSTM .395 online and .830 offline, the +transformer .540 online and .754 offline. Both speakers say the other neighbour as often as the goal +branch's, so neither reads the goal at the first hop, and settling three words jointly adds no +lookahead. The information suffices, as the offline controls show. + ## What this does and does not establish The chamber measures one composition and its options on one finite language of eighteen words, diff --git a/benchmarks/grammar/ROUTES_PROTOCOL.md b/benchmarks/grammar/ROUTES_PROTOCOL.md new file mode 100644 index 00000000..bfec7551 --- /dev/null +++ b/benchmarks/grammar/ROUTES_PROTOCOL.md @@ -0,0 +1,96 @@ +# Routes with a goal: declared protocol (cadence#257) + +Declared on 2026-11-09, committed to the lane branch before the first life. The founders, speakers, +readings and gate below are fixed; a failed gate is reported as failed. + +## The question + +On the grammar chamber every sentence is a function of the meaning alone, so a word-by-word speaker +that reads well has all it needs, and no row showed planning paying. In the goal-route chamber +(`benchmarks/grammar/routes.py`) the first hop is said before the goal it must lead to: the start +has two neighbours and only one branch reaches the goal. Does a sliding planning window of three +(plan three words, say the first, hear it, plan again) say the first hop right more often than word +by word, on start-goal pairs no training episode joined? + +## Speakers + +- `word`: `Brain.compose(22, 10, modules=(64,), seed=founder, backend="cpu")`, the default + composition, nothing else named. +- `sliding`: the same with `slots=3` and `3 * 10` actions; each lesson teaches the next three + words. +- `lstm`, `transformer`: matched to `word`'s `parameters()`, online (one pass, the brain's order) + and offline (to convergence), reported beside the brains with no gate. + +## Life + +3,000 batches of sixteen episodes, two and three hops in turn, from the founder's training stream; +the panels `held` (the founder's eight held-out start-goal pairs) and `trained` (fresh graphs on +trained pairs), 256 episodes each, read at the end, greedy and closed loop. + +## Founders + +Development (instrument checks only, no reading decides anything): founders 0 and 1, both speakers, +read before the declared run to check that the chamber is learnable within the life. Declared: +founders 9625700 to 9625705, which no lane has used. + +## Gate + +| Gate | Reading | Pass (planning pays) | +| --- | --- | --- | +| P1 | first-hop accuracy on `held` at 3,000 batches, `sliding` minus `word`, paired by founder | mean difference at least .05 and above two standard errors of the paired differences | +| P2 | the same on `trained` | reported, no bar | + +Exact routes, valid routes and first-hop accuracy per hop count are reported for every speaker. + +**Verdict.** P1 passing closes the narrowed acceptance with planning paying on this chamber, and the +sliding form is documented as the planning window's use. P1 failing is a measured negative on this +chamber, recorded on the issue. + +## Compute + +One AWS c7i.16xlarge, one process per life, one BLAS thread per process. + +## Amendment before the declared run + +The first development lives (founders 0 and 1, both speakers, the route said as `start a1 ... goal +end`) read first-hop accuracy .17 to .27 on both panels, below the .5 of a speaker that only picks a +neighbour of the start, so the chamber was not learnable within the life in that form. Each fact is +heard as `X near Y end`, so the place after `X` is two moments on. The route is said as the facts and +the grammar chamber's paths say it, `start near a1 near ... near goal end`, and `first_hop` is the +place after the first `near`. Nothing else changes; the development lives are read again before the +declared run. + +## Second amendment before the declared run + +Read again with `near`, the development lives at 3,000 batches (held-out pairs, founders 0 and 1): +word by word says a neighbour of the start on .74 and .80 of routes, the goal branch's on .32 and +.45, so it chooses between the two neighbours near chance, as the chamber intends; the sliding window +says a neighbour on .37 and .47 and is still rising (first hop .05, .14, .18 and .09, .19, .24 at +1,000, 2,000 and 3,000 batches). The matched transformer online reads .32 and .54. So that the window +is read once it has learned the routes' form, the declared life is 8,000 batches, read at 2,000, +4,000, 6,000 and 8,000, and P1 and P2 read the reading at 8,000. The controls' online pass takes the +same 8,000 batches. Development founders 0 and 1 live the declared life beside the declared founders +and are reported apart. + +## Result + +Read on one c7i.16xlarge, founders 9625700 to 9625705, 8,000 batches, 256 episodes a panel. + +| Gate | Verdict | Reading | +| --- | --- | --- | +| P1, held-out pairs | fail | sliding minus word by word -.022 (s.e. .050): -.047 .090 -.105 -.184 .145 -.027 | +| P2, trained pairs | reported | -.076 (s.e. .028) | + +| Speaker | First hop, held-out | The other neighbour | Exact routes | First hop, trained | +| --- | --- | --- | --- | --- | +| Word by word | .334 | .356 | .240 | .372 | +| Sliding window of three | .312 | .319 | .203 | .296 | +| LSTM online | .395 | .366 | .277 | .419 | +| Transformer online | .540 | .374 | .518 | .548 | +| LSTM offline (30 epochs) | .830 | .106 | .712 | .963 | +| Transformer offline (14 to 30 epochs) | .754 | .246 | .753 | .755 | + +A measured negative: planning does not pay on this chamber. Both speakers say the goal branch's first +place as often as the other neighbour's, so neither reads the goal at the first hop; the window adds no +lookahead to the joint settle. The controls show the information suffices (offline .75 to .83 on +held-out pairs). diff --git a/benchmarks/grammar/cost.py b/benchmarks/grammar/cost.py new file mode 100644 index 00000000..2b3f5985 --- /dev/null +++ b/benchmarks/grammar/cost.py @@ -0,0 +1,235 @@ +"""What a chamber pass costs on one thread: the default brain, the same with +``learning_compiled_moment=True`` and the chamber's matched transformer, in one process. + +One online pass of the first grammar (4,000 batches of 16 rows by default): the brains are +``Brain.compose`` at the library's defaults on the CPU backend, taught by ``grammar.speak``; the +transformer takes one Adam step per batch (``controls.Learner``, online). The three learners +take each batch in turn and every batch's CPU seconds (``time.thread_time``) are summed per +learner, so they share whatever else the machine does. The kernels compile or load on copies +that live the first 20 batches before the clock starts. Pin the process to one core +(``taskset -c 3`` on Linux) and set one thread per library: + + OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 NUMBA_NUM_THREADS=1 PYTHONPATH=src \\ + python benchmarks/grammar/cost.py --founder 0 --out DIR/cost-f0.json + +``torch`` is required for the transformer alone; ``--learners plain,compiled`` leaves it out. + +``--parts plain`` (or ``compiled``) lives one brain's pass alone and reads where its time goes: +the functions of ``PARTS`` are timed, each call's seconds less those of the timed calls inside it +charged to its part; the kernels are timed at the functions that launch them. A timer costs about +a microsecond a call, a few tenths of a second over a pass. +""" + +from __future__ import annotations + +import argparse +import copy +import json +import time +import warnings +from collections import Counter +from pathlib import Path +from typing import Any + +import grammar as gr +import numpy as np + +LEARNERS = ("plain", "compiled", "transformer") +TARGET = 24384 # the parameters the chamber's controls match (the recipe's brain) +WARM = 20 # the batches a copy of each brain lives before the clock starts + + +def make(name: str, founder: int) -> Any: + """One learner of the pass.""" + if name == "transformer": + import controls as ct + + return ct.Learner("transformer", ct.matched_width("transformer", TARGET), founder, + ct.RATES["online"]) + import cadence as cd + + settings = {"learning_compiled_moment": True} if name == "compiled" else {} + return cd.Brain.compose(gr.INPUTS, gr.ACTIONS, modules=(64,), seed=founder, backend="cpu", + **settings) + + +def _warm(brain: Any, stream: list[Any], batches: int = WARM) -> None: + """Copies of ``brain`` live the first batches, so every kernel the pass calls compiles or + loads from Numba's cache before the clock starts: one copy as the pass lives them, one + through the passes a moment falls back on (a compiled moment's general path), and one act + whose solve is refused after its rescue, which a pass meets at a few batches of thousands.""" + from cadence import _moment + + fork = copy.deepcopy(brain) + for batch in stream[:batches]: + gr.speak(fork, batch, teach=True) + flags = _moment.SHORTCUTS, _moment.ONE_CALL, _moment.ONE_LESSON + try: + for modes in ((True, True, False), (True, False, False), (False, False, False)): + _moment.SHORTCUTS, _moment.ONE_CALL, _moment.ONE_LESSON = modes + fork = copy.deepcopy(brain) + for batch in stream[:2]: + gr.speak(fork, batch, teach=True) + finally: + _moment.SHORTCUTS, _moment.ONE_CALL, _moment.ONE_LESSON = flags + fork = copy.deepcopy(brain) + fork.retune(learning_free_steps=2) + try: + fork.act(np.stack([row.pre[0] for row in stream[0]]), greedy=True) + except RuntimeError: + pass + + +def run(founder: int, batches: int, learners: tuple[str, ...] = LEARNERS, + every: int = 500) -> dict[str, Any]: + """One pass of every learner, batch by batch in turn; the seconds per learner, the brains' + work and a mark every ``every`` batches.""" + warnings.simplefilter("ignore") + if "transformer" in learners: + import torch + + torch.set_num_threads(1) + made = {name: make(name, founder) for name in learners} + split = gr.Split.of(founder) + rng = np.random.default_rng(np.random.SeedSequence([founder, gr.SEEDS["train"]])) + stream = [gr.train_batch(rng, split, "g1") for _ in range(batches)] + for name, learner in made.items(): # the kernels compile or load outside the clock + if name != "transformer": + _warm(learner, stream) + seconds = dict.fromkeys(learners, 0.0) + work = {name: gr.new_work() for name in learners if name != "transformer"} + marks = [] + for i, batch in enumerate(stream, 1): + for name, learner in made.items(): + started = time.thread_time() + if name == "transformer": + learner.update(batch) + else: + gr.speak(learner, batch, teach=True, work=work[name]) + seconds[name] += time.thread_time() - started + if i % every == 0 or i == batches: + marks.append({"batches": i, **seconds}) + out: dict[str, Any] = {"founder": founder, "batches": batches, "seconds": seconds, + "work": work, "marks": marks} + if "transformer" in seconds: + out["against_transformer"] = { + name: seconds[name] / seconds["transformer"] for name in seconds + } + return out + + +# The timed functions, each with its part: the kernels at the functions that launch them, then the +# parts of a pass around them. +PARTS = ( + ("cadence._moment", "_moment_kernel", "kernels: a moment in one call"), + ("cadence._moment", "_taught_kernel", "kernels: a taught lesson in one call"), + ("cadence._moment", "_phases_kernel", "kernels: the lessons' phases"), + ("cadence.fused", "_fast_solve", "kernels: the solves"), + ("cadence.fused", "fused_solve", "kernels: the solves"), + ("cadence.fused", "fused_settle", "kernels: the lessons' phases"), + ("cadence.fused", "fused_residual", "kernels: the checks"), + ("cadence.generic", "Brain._stimulus", "the read"), + ("cadence.generic", "Brain._compiled_drive", "the read"), + ("cadence.stream", "Recall.learn", "the address lesson"), + ("cadence.stream", "Hearing.learn", "the carried gains' lesson"), + ("cadence.learning", "Learner.update", "the lessons' update"), + ("cadence.learning", "Learner.step", "the lessons' phases around their kernels"), + ("cadence.generic", "Brain._qualified", "the solves around their kernels"), + ("cadence.generic", "Brain._choose", "the choice and the write"), + ("cadence.generic", "Brain.lesson", "the rest of the lesson"), + ("cadence.generic", "Brain.act", "the rest of the act"), +) +OUTSIDE = "the chamber's own loop" + + +def _timed(spent: Counter[str], stack: list[float], part: str, function: Any) -> Any: + clock = time.perf_counter + # a settle of no sweeps publishes a solve's start and launches no kernel + publish = "the solves around their kernels" if function.__name__ == "fused_settle" else None + + def timed(*args: Any, **kwargs: Any) -> Any: + stack.append(0.0) + started = clock() + try: + return function(*args, **kwargs) + finally: + took = clock() - started + inside = stack.pop() + spent[publish if publish and args[9] == 0 else part] += took - inside + if stack: + stack[-1] += took + + timed.__wrapped__ = function # type: ignore[attr-defined] + return timed + + +def parts(name: str, founder: int, batches: int) -> dict[str, Any]: + """One brain's pass alone, its seconds charged to the parts of ``PARTS``; the functions are + restored after it.""" + import importlib + + warnings.simplefilter("ignore") + brain = make(name, founder) + split = gr.Split.of(founder) + rng = np.random.default_rng(np.random.SeedSequence([founder, gr.SEEDS["train"]])) + stream = [gr.train_batch(rng, split, "g1") for _ in range(batches)] + _warm(brain, stream) + spent: Counter[str] = Counter() + stack: list[float] = [] + held = [] + for module_name, path, part in PARTS: + owner: Any = importlib.import_module(module_name) + *outer, attribute = path.split(".") + for step in outer: + owner = getattr(owner, step) + original = ( + owner.__dict__[attribute] if isinstance(owner, type) else getattr(owner, attribute) + ) + held.append((owner, attribute, original)) + setattr(owner, attribute, _timed(spent, stack, part, original)) + started = time.perf_counter() + try: + for batch in stream: + gr.speak(brain, batch, teach=True) + finally: + for owner, attribute, original in held: + setattr(owner, attribute, original) + wall = time.perf_counter() - started + spent[OUTSIDE] += wall - sum(spent.values()) + return {"learner": name, "founder": founder, "batches": batches, "wall": wall, + "parts": dict(spent.most_common())} + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) + parser.add_argument("--founder", type=int, default=0) + parser.add_argument("--batches", type=int, default=4000) + parser.add_argument("--learners", default=",".join(LEARNERS)) + parser.add_argument("--out", type=Path, default=None) + parser.add_argument("--parts", choices=("plain", "compiled"), default=None, + help="one brain's pass alone, by part") + args = parser.parse_args(argv) + if args.parts: + found = parts(args.parts, args.founder, args.batches) + if args.out is not None: + args.out.parent.mkdir(parents=True, exist_ok=True) + args.out.write_text(json.dumps(found, indent=1)) + for part, seconds in found["parts"].items(): + print(f"{seconds:8.1f} s {part}") + return 0 + learners = tuple(name for name in args.learners.split(",") if name) + unknown = set(learners) - set(LEARNERS) + if unknown: + parser.error(f"unknown learners: {sorted(unknown)}") + result = run(args.founder, args.batches, learners) + text = json.dumps(result, indent=1, default=float) + if args.out is not None: + args.out.parent.mkdir(parents=True, exist_ok=True) + args.out.write_text(text) + print(json.dumps({"seconds": result["seconds"], + "against_transformer": result.get("against_transformer")}, default=float)) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/benchmarks/grammar/exposure.py b/benchmarks/grammar/exposure.py new file mode 100644 index 00000000..78afbaea --- /dev/null +++ b/benchmarks/grammar/exposure.py @@ -0,0 +1,253 @@ +"""Exposure parity on the grammar chamber (cadence#270): the default composition with its +rehearsal, taught the same 4,000 batches the transformer's offline control replays, read after +every pass beside that control at equal compute. + +A life draws the first grammar's 4,000 training batches once, from the stream every arm and +control of a founder hears (``SeedSequence([founder, 11])``), and teaches them in passes: the +first in the stream's order (the chamber's online life), every later one in the order of the +transformer's next offline epoch (``SeedSequence([founder, 99, 0])``, one permutation per +epoch), with lessons as in ``grammar.speak``. After each pass every panel of the first grammar is +read on a fork, with ``located`` (where a word is lost: at the recall read or at the readout) on +the last pass. A life stops after ``passes`` passes, or once a pass raises no reading's mean +(``READINGS``) by more than ``SATURATED`` over the pass before it (saturation). + +Compute is counted analytically: every row sweep of a settle (a lesson's phases, every act's +sweeps times its rows, the rehearsal's replayed moments and lessons at the pass's own sweeps per +moment and per taught row) multiplies each of the brain's synapses once, which counts an act whose +rows stop early at the batch's longest and leaves out the memory's reads; the controls' counts are +``controls.py``'s (a backward pass at twice the forward). + + python benchmarks/grammar/exposure.py plan --out DIR/plan.json + python benchmarks/grammar/exposure.py run --plan DIR/plan.json --out DIR/brain.jsonl --workers 6 + python benchmarks/grammar/controls.py --plan DIR/controls.json --out DIR/controls.jsonl + python benchmarks/grammar/exposure.py read DIR/brain.jsonl DIR/controls.jsonl +""" + +from __future__ import annotations + +import argparse +import json +import sys +import time +import warnings +from multiprocessing import get_context +from pathlib import Path +from typing import Any + +import numpy as np + +HERE = Path(__file__).resolve().parent +if str(HERE) not in sys.path: + sys.path.insert(0, str(HERE)) + +import grammar as gr # noqa: E402 + +BATCHES = 4000 +PASSES = 4 +SATURATED = 0.01 +FOUNDERS = tuple(range(9627000, 9627006)) +ARMS: dict[str, dict[str, Any]] = { + "default": {}, + "rehearsal": {"learning_rehearsal": 1.0}, +} +# panel, score: the readings compared with the control (two hops and three hops are the path +# sentences of two hops and the three-hop panel) +READINGS: dict[str, tuple[str, str]] = { + "clauses": ("clause", "exact"), + "held_out": ("clause_held", "exact"), + "long_clauses": ("clause_long", "exact"), + "roles": ("roles", "both"), + "brackets": ("dyck", "exact"), + "deep_brackets": ("dyck_deep", "exact"), + "paths": ("path", "exact"), + "three_hops": ("path_long", "exact"), +} +MARGIN = 0.05 + + +def synapses(brain: Any) -> int: + return int(brain.describe()["layout"]["synapses"]) + + +def macs(work: dict[str, float], count: int, rows: int = gr.BATCH) -> float: + """The analytic multiply-accumulates of ``work`` on a brain of ``count`` synapses. The + rehearsal reports its replayed moments and taught rows, not its sweeps: each replayed moment + is counted at the pass's sweeps per act and row, each replayed taught row at the pass's + lesson sweeps per taught row.""" + taught = work["rows"] - work["agreed_rows"] + per_moment = work["act_sweeps"] / work["acts"] if work["acts"] else 0.0 + per_row = work["lesson_row_sweeps"] / taught if taught else 0.0 + rehearsal = (work.get("rehearsal_moments", 0.0) * per_moment + + work.get("rehearsal_rows", 0.0) * per_row) + work["rehearsal_row_sweeps"] = rehearsal + return float((work["lesson_row_sweeps"] + rehearsal + work["act_sweeps"] * rows) * count) + + +def order(founder: int, passes: int, count: int = BATCHES) -> list[np.ndarray]: + """The batches' order per pass: the stream's, then the transformer's offline epochs' + (``controls.py`` draws one permutation per epoch from this generator).""" + epochs = np.random.default_rng(np.random.SeedSequence([int(founder), 99, 0])) + return [np.arange(count)] + [epochs.permutation(count) for _ in range(1, passes)] + + +def scores(panels: dict[str, Any]) -> dict[str, float | None]: + return {name: panels[panel][key] if panel in panels else None + for name, (panel, key) in READINGS.items()} + + +def run_life(spec: dict[str, Any]) -> dict[str, Any]: + """One life: ``arm``, ``founder``, ``passes`` (at most), ``module``, ``panel_batches``, + ``batches`` (per pass) and ``settings`` over the arm's.""" + warnings.simplefilter("ignore") + import cadence as cd + + arm, founder = spec["arm"], int(spec["founder"]) + passes, count = int(spec.get("passes", PASSES)), int(spec.get("batches", BATCHES)) + settings = {**ARMS[arm], **spec.get("settings", {})} + started = time.monotonic() + brain = gr.make_brain("default", founder, settings, int(spec.get("module", 64))) + split = gr.Split.of(founder) + tests = gr.panels(founder, int(spec.get("panel_batches", 4))) + rng = np.random.default_rng(np.random.SeedSequence([founder, gr.SEEDS["train"]])) + batches = [gr.train_batch(rng, split, "g1") for _ in range(count)] + size = synapses(brain) + row: dict[str, Any] = { + "arm": arm, "founder": founder, "settings": settings, "batches": count, + "module": int(spec.get("module", 64)), "parameters": int(brain.parameters()), + "synapses": size, "cadence": cd.__version__, "tag": spec.get("tag"), "passes": [], + } + total = 0.0 + previous: dict[str, float | None] | None = None + for index, sequence in enumerate(order(founder, passes, count)): + work = gr.new_work() + clock = time.monotonic() + for b in sequence: + gr.speak(brain, batches[int(b)], teach=True, work=work) + work["seconds"] = round(time.monotonic() - clock, 1) + work["macs"] = macs(work, size) + total += work["macs"] + panels = gr.reading(brain, tests, gr.G1_PANELS, components=False) + now = scores(panels) + saturated = previous is not None and all( + now[k] is None or previous[k] is None or now[k] - previous[k] <= SATURATED + for k in READINGS) + last = saturated or index == passes - 1 + entry = {"pass": index + 1, "work": work, "macs": total, "readings": now, + "panels": {k: {kk: vv for kk, vv in v.items() if kk != "examples"} + for k, v in panels.items()}} + if last and brain.recall is not None: + entry["located"] = {name: gr.locate(brain, tests[name]) for name in gr.LOCATE + if name in tests} + row["passes"].append(entry) + previous = now + if last: + row["saturated"] = saturated + break + row["seconds"] = round(time.monotonic() - started, 1) + return row + + +def plan(founders: tuple[int, ...] = FOUNDERS, passes: int = PASSES, + panel_batches: int = 16) -> list[dict[str, Any]]: + return [{"arm": arm, "founder": f, "passes": passes, "panel_batches": panel_batches, + "tag": f"exposure/{arm}/f{f}"} for arm in ARMS for f in founders] + + +def control_plan(parameters: int, founders: tuple[int, ...] = FOUNDERS, + panel_batches: int = 16) -> list[dict[str, Any]]: + """The transformer's offline control on the same batches, matched to the brain's + parameters (``controls.run_life``).""" + return [{"model": "transformer", "mode": "offline", "founder": f, + "phases": [["g1", BATCHES]], "parameters": parameters, + "panel_batches": panel_batches} for f in founders] + + +def compare(brains: list[dict[str, Any]], controls: list[dict[str, Any]], + arm: str = "rehearsal") -> dict[str, Any]: + """E1: at the transformer's offline compute (each founder's last pass whose compute is + within the control's, else the first), every reading's mean at least the control's less + ``MARGIN``. E2: the same at the last pass read (saturation). Per reading, the means and + each pass's mean.""" + ctrl = {int(r["founder"]): r for r in controls if r["mode"] == "offline"} + lives = {int(r["founder"]): r for r in brains if r["arm"] == arm} + founders = sorted(set(ctrl) & set(lives)) + if not founders: + return {} + + def at(life: dict[str, Any], budget: float) -> dict[str, Any]: + within = [p for p in life["passes"] if p["macs"] <= budget] + return within[-1] if within else life["passes"][0] + + out: dict[str, Any] = {"founders": founders, "arm": arm} + reference = {k: float(np.mean([scores(ctrl[f]["readings"][0]["panels"])[k] + for f in founders])) for k in READINGS} + for gate, pick in (("E1 equal compute", lambda f: at(lives[f], ctrl[f]["readings"][0][ + "work"]["macs"])), ("E2 saturation", lambda f: lives[f]["passes"][-1])): + chosen = {f: pick(f) for f in founders} + mean = {k: float(np.mean([chosen[f]["readings"][k] for f in founders])) + for k in READINGS} + apart = [k for k in READINGS if mean[k] < reference[k] - MARGIN] + out[gate] = {"pass": not apart, "apart": apart, + "brain": {k: round(v, 3) for k, v in mean.items()}, + "control": {k: round(v, 3) for k, v in reference.items()}, + "passes": {f: chosen[f]["pass"] for f in founders}, + "brain_macs": float(np.mean([chosen[f]["macs"] for f in founders])), + "control_macs": float(np.mean([ctrl[f]["readings"][0]["work"]["macs"] + for f in founders]))} + curve: dict[int, dict[str, float]] = {} + for p in range(1, max(len(lives[f]["passes"]) for f in founders) + 1): + have = [lives[f]["passes"][p - 1] for f in founders if len(lives[f]["passes"]) >= p] + curve[p] = {k: round(float(np.mean([h["readings"][k] for h in have])), 3) + for k in READINGS} + curve[p]["lives"] = len(have) + out["curve"] = curve + return out + + +def _rows(path: Path) -> list[dict[str, Any]]: + return [json.loads(line) for line in path.read_text().splitlines() if line.strip()] + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__.split("\n", 1)[0]) + sub = parser.add_subparsers(dest="command", required=True) + p = sub.add_parser("plan") + p.add_argument("--out", type=Path, required=True) + p.add_argument("--controls", type=Path, help="the controls' plan") + r = sub.add_parser("run") + r.add_argument("--plan", type=Path, required=True) + r.add_argument("--out", type=Path, required=True) + r.add_argument("--workers", type=int, default=1) + c = sub.add_parser("read") + c.add_argument("brains", type=Path) + c.add_argument("controls", type=Path) + args = parser.parse_args(argv) + if args.command == "plan": + args.out.parent.mkdir(parents=True, exist_ok=True) + args.out.write_text(json.dumps(plan(), indent=1)) + if args.controls: + parameters = int(gr.make_brain("default", FOUNDERS[0]).parameters()) + args.controls.write_text(json.dumps(control_plan(parameters), indent=1)) + return 0 + if args.command == "run": + specs = json.loads(args.plan.read_text()) + args.out.parent.mkdir(parents=True, exist_ok=True) + with args.out.open("a") as sink: + if args.workers <= 1: + results = map(run_life, specs) + for result in results: + sink.write(json.dumps(result) + "\n") + sink.flush() + else: + with get_context("spawn").Pool(min(args.workers, len(specs))) as pool: + for result in pool.imap_unordered(run_life, specs, chunksize=1): + sink.write(json.dumps(result) + "\n") + sink.flush() + return 0 + brains, controls = _rows(args.brains), _rows(args.controls) + print(json.dumps({arm: compare(brains, controls, arm) for arm in ARMS}, indent=1)) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/benchmarks/grammar/grammar.py b/benchmarks/grammar/grammar.py index 050ce531..5f6b3a8c 100644 --- a/benchmarks/grammar/grammar.py +++ b/benchmarks/grammar/grammar.py @@ -431,6 +431,7 @@ def _module(name: str, path: Path) -> Any: "recipe": {**SEQUENCE, **RECIPE}, "maturity": {**SEQUENCE, **RECIPE, "maturity": True}, "frozen": {**SEQUENCE, **RECIPE}, # the newborn, never taught + "default": {"recall": True}, # what Brain.compose gives a new brain, nothing named } @@ -448,7 +449,8 @@ def make_brain(arm: str, founder: int, settings: dict[str, Any] | None = None, def new_work() -> dict[str, float]: return {"lessons": 0, "rows": 0, "agreed_rows": 0, "lesson_row_sweeps": 0, "acts": 0, - "act_sweeps": 0, "refused": 0, "exhausted": 0, "free_budget_exhausted": 0} + "act_sweeps": 0, "refused": 0, "exhausted": 0, "free_budget_exhausted": 0, + "rehearsal_moments": 0, "rehearsal_rows": 0} def speak(brain: Any, rows: list[Row], *, teach: bool, closed: bool = True, @@ -495,6 +497,9 @@ def speak(brain: Any, rows: list[Row], *, teach: bool, closed: bool = True, work["agreed_rows"] += report.get("agreed_presentations", 0.0) work["lesson_row_sweeps"] += report.get("total_row_sweeps", 0.0) work["free_budget_exhausted"] += report.get("free_budget_exhausted", 0.0) + replayed = brain.last_learning or {} # with ``learning_rehearsal`` + work["rehearsal_moments"] += replayed.get("rehearsal_moments", 0.0) + work["rehearsal_rows"] += replayed.get("rehearsal_accepted_presentations", 0.0) except RuntimeError: work["refused"] += 1 action = _act(brain, x, work) @@ -928,7 +933,8 @@ def run_life(spec: dict[str, Any]) -> dict[str, Any]: tests = panels(founder, int(spec.get("panel_batches", 4))) rng = np.random.default_rng(np.random.SeedSequence([founder, SEEDS["train"]])) row: dict[str, Any] = { - "arm": arm, "founder": founder, "settings": {**ARMS[arm], **spec.get("settings", {})}, + "arm": arm, "founder": founder, "tag": spec.get("tag"), + "settings": {**ARMS[arm], **spec.get("settings", {})}, "phases": [list(p) for p in phases], "forks": forks, "module": int(spec.get("module", 64)), "parameters": int(brain.parameters()), "cadence": cd.__version__, diff --git a/benchmarks/grammar/lesson_step.py b/benchmarks/grammar/lesson_step.py new file mode 100644 index 00000000..158a87f1 --- /dev/null +++ b/benchmarks/grammar/lesson_step.py @@ -0,0 +1,250 @@ +"""The lesson's step on the grammar chamber (cadence#258): does a step that follows the brain's +running error settle a life, with no count of lessons? + +Every life is ``grammar.run_life`` on the default composition (the ``default`` arm: what +``Brain.compose`` gives a new brain, nothing named), the first grammar for 4,000 batches at width +64 or 128, with 256-sentence panels (``panel_batches=16``) read at every 500 batches and in full +at 4,000. The candidates read for #258 (``CANDIDATES``) were ``learning_running_error``: the +lesson's step scaled by the running share of lesson rows answered wrong, to the power +``learning_running_error_power``. That option failed the declared confirmation +(``LESSON_STEP_PROTOCOL.md``) and is not in the library; its lives run on the lane's commit +968a02b. The ``control`` arm, the swing reading and the gates run on any library. + +Readings per life: + +- ``swing``: the largest minus the smallest exact-clause reading at 3,000, 3,500 and 4,000 + batches (the validation reading settles where its swing is small). +- ``clauses``, ``held_out``, ``roles``, ``two_hops``: the full reading at 4,000 (``clause``, + ``clause_held``, ``roles`` both right, ``path`` sentences of two hops). +- ``brackets``, ``paths``: the means of the last five readings (2,000 to 4,000), since either + moves by more than .1 at one reading on last bits alone. + +``plan`` writes the life specifications of a stage, ``read`` the table and the gates of a +results file (``--protocol`` names the stage's gates; see ``LESSON_STEP_PROTOCOL.md``). + + python benchmarks/grammar/lesson_step.py plan --stage dev --out DIR/dev.json + python benchmarks/grammar/grammar.py --plan DIR/dev.json --out DIR/dev.jsonl --workers 30 + python benchmarks/grammar/lesson_step.py read DIR/dev.jsonl --stage dev +""" + +from __future__ import annotations + +import argparse +import json +import sys +from pathlib import Path +from typing import Any + +import numpy as np + +BATCHES = 4000 +CHECKPOINTS = tuple(range(500, BATCHES, 500)) +SWING_AT = (3000, 3500, 4000) +LATE = (2000, 2500, 3000, 3500, 4000) +WIDTHS = (64, 128) +PANEL_BATCHES = 16 + +# the candidate steps read for #258 (on commit 968a02b, whose library held the option): the +# forgetting factor of the running error and its power +ARMS: dict[str, dict[str, Any]] = { + "control": {}, + "e99p5": {"learning_running_error": 0.99, "learning_running_error_power": 0.5}, + "e999p5": {"learning_running_error": 0.999, "learning_running_error_power": 0.5}, + "e99p1": {"learning_running_error": 0.99, "learning_running_error_power": 1.0}, + "e999p1": {"learning_running_error": 0.999, "learning_running_error_power": 1.0}, +} +CANDIDATES = tuple(a for a in ARMS if a != "control") +STAGES = { + "dev": {"founders": (0, 1, 2), "arms": tuple(ARMS)}, + # the confirmation's arms are the control and the candidate the development stage selects + "confirm": {"founders": tuple(range(9625800, 9625806)), "arms": ("control",)}, +} +READINGS = ("swing", "clauses", "held_out", "roles", "brackets", "paths", "two_hops") +LEVELS = ("clauses", "held_out", "brackets", "paths") # the selection's eligibility +HOLD = ("clauses", "held_out", "roles", "brackets", "paths", "two_hops") # the chamber gate + + +def plan(stage: str, arms: tuple[str, ...] | None = None) -> list[dict[str, Any]]: + spec = STAGES[stage] + return [{"arm": "default", "founder": f, "module": w, "phases": [["g1", BATCHES]], + "settings": ARMS[a], "checkpoints": list(CHECKPOINTS), + "panel_batches": PANEL_BATCHES, "tag": f"{stage}/{a}/w{w}/f{f}"} + for a in (arms or spec["arms"]) for w in WIDTHS for f in spec["founders"]] + + +def life(row: dict[str, Any]) -> dict[str, Any]: + """The readings of one life (a row of ``grammar.run_life``).""" + at = {rd["batches"]: rd["panels"] for rd in row["readings"] if rd["phase"] == 0} + final = at[BATCHES] + hops = final["path"]["conditions"].get("hops2") + return { + "swing": max(at[b]["clause"]["exact"] for b in SWING_AT) + - min(at[b]["clause"]["exact"] for b in SWING_AT), + "clauses": final["clause"]["exact"], + "held_out": final["clause_held"]["exact"], + "roles": final["roles"]["both"], + "brackets": float(np.mean([at[b]["dyck"]["exact"] for b in LATE])), + "paths": float(np.mean([at[b]["path"]["exact"] for b in LATE])), + "two_hops": hops, + "seconds": row.get("seconds"), + } + + +def table(rows: list[dict[str, Any]]) -> dict[tuple[str, int], dict[int, dict[str, Any]]]: + """``(arm, width) -> founder -> readings``, the arm and width from each row's tag.""" + out: dict[tuple[str, int], dict[int, dict[str, Any]]] = {} + for row in rows: + _, arm, width, _ = row["tag"].split("/") + out.setdefault((arm, int(width[1:])), {})[int(row["founder"])] = life(row) + return out + + +def means(cell: dict[int, dict[str, Any]]) -> dict[str, float]: + return {k: float(np.mean([v[k] for v in cell.values() if v[k] is not None])) + for k in READINGS} + + +def select(cells: dict[tuple[str, int], dict[int, dict[str, Any]]]) -> dict[str, Any]: + """The development stage's declared selection: among the candidates whose mean clauses, + held-out clauses, brackets and paths are each at least the control's less .05 at both + widths, the lowest mean clause swing over the two widths (ties: the smaller power, then the + larger forgetting factor). With none eligible, the lowest swing overall, flagged.""" + score, eligible = {}, {} + for arm in CANDIDATES: + if not all((arm, w) in cells and ("control", w) in cells for w in WIDTHS): + continue + m = {w: means(cells[(arm, w)]) for w in WIDTHS} + c = {w: means(cells[("control", w)]) for w in WIDTHS} + score[arm] = float(np.mean([m[w]["swing"] for w in WIDTHS])) + eligible[arm] = all(m[w][k] >= c[w][k] - 0.05 for w in WIDTHS for k in LEVELS) + + def key(arm: str) -> tuple[float, float, float]: + s = ARMS[arm] + return (score[arm], s["learning_running_error_power"], -s["learning_running_error"]) + + pool = [a for a in score if eligible[a]] or list(score) + chosen = min(pool, key=key) if pool else None + return {"scores": score, "eligible": eligible, "chosen": chosen, + "flagged": bool(chosen) and not eligible.get(chosen, False)} + + +def gates(cells: dict[tuple[str, int], dict[int, dict[str, Any]]], arm: str) -> dict[str, Any]: + """The confirmation's chamber gates for ``arm`` against the control (the battery's gates + are read by ``battery_gates``).""" + out: dict[str, Any] = {} + for w in WIDTHS: + cand, ctrl = cells[(arm, w)], cells[("control", w)] + within = sum(v["swing"] <= 0.05 for v in cand.values()) + out[f"C1 settles, width {w}"] = { + "pass": within >= 5, "within": f"{within} of {len(cand)}", + "swings": [round(cand[f]["swing"], 3) for f in sorted(cand)], + "control_within": sum(v["swing"] <= 0.05 for v in ctrl.values())} + m, c = means(cand), means(ctrl) + out[f"C2 levels hold, width {w}"] = { + "pass": all(m[k] >= c[k] - 0.03 for k in HOLD), + "differences": {k: round(m[k] - c[k], 3) for k in HOLD}} + return out + + +# the battery's tasks and bars (#256: S1, S2, S4 to S7, T1, U1, U2), as ``battery.py`` reads them +BATTERY: dict[str, tuple[str, dict[str, Any], str, float]] = { + "lookback_first": ("lookback", {"lengths": [5, 12], "test_lengths": [24, 60]}, "first", .90), + "lookback_second": ("lookback", {"lengths": [5, 12], "test_lengths": [24, 60]}, "second", + .90), + "copy_unique": ("repeat", {"unique": True, "lengths": [3, 6], "test_lengths": [8, 12]}, + "accuracy", .95), + "copy_repeated": ("repeat", {"unique": False, "lengths": [3, 6], "test_lengths": [8, 12]}, + "accuracy", .90), + "periodic40": ("periodic", {"periods": [40]}, "accuracy", .85), + "before": ("before", {}, "accuracy", .90), + "twohop": ("twohop", {"think": True}, "accuracy", .90), + "induction": ("induction", {"lengths": [8, 16], "test_lengths": [24, 60]}, "accuracy", .90), + # the battery's n-back reads the lengths it teaches (its task takes no test lengths) + "nback": ("nback", {"lengths": [6, 12]}, "accuracy", .90), +} +BATTERY_FOUNDERS = tuple(range(9625810, 9625816)) + + +def battery_plan(arms: dict[str, dict[str, Any]]) -> list[dict[str, Any]]: + """Battery lives (``benchmarks/sequence/battery.py``) of the default composition: the + tasks a ``lookback`` life reads twice share one life.""" + tasks = {"lookback": BATTERY["lookback_first"][:2]} + tasks.update({k: v[:2] for k, v in BATTERY.items() if v[0] != "lookback"}) + return [{"task": task, "task_args": args, "arm": "default", "founder": f, + "settings": settings, "tag": f"battery/{name}/{key}/f{f}"} + for name, settings in arms.items() for key, (task, args) in tasks.items() + for f in BATTERY_FOUNDERS] + + +def battery_reading(row: dict[str, Any], key: str) -> float: + _, _, reading, _ = BATTERY[key] + test = row["test"] + return float(test["accuracy"] if reading == "accuracy" else test["conditions"][reading]) + + +def battery_gates(rows: list[dict[str, Any]], arm: str) -> dict[str, Any]: + """C3: on every task the arm passes its bar on at least as many founders as the control + (or on 5 of 6), and its mean is at least the control's less .03.""" + found: dict[tuple[str, str], dict[int, float]] = {} + for row in rows: + _, name, key, _ = row["tag"].split("/") + keys = ("lookback_first", "lookback_second") if key == "lookback" else (key,) + for k in keys: + found.setdefault((name, k), {})[int(row["founder"])] = battery_reading(row, k) + out = {} + for key, (_, _, _, bar) in BATTERY.items(): + a, c = found.get((arm, key), {}), found.get(("control", key), {}) + if not a or not c: + continue + pa, pc = sum(v >= bar for v in a.values()), sum(v >= bar for v in c.values()) + ma, mc = float(np.mean(list(a.values()))), float(np.mean(list(c.values()))) + out[key] = {"pass": (pa >= pc or pa >= 5) and ma >= mc - 0.03, + "bar": bar, "arm": f"{pa} of {len(a)}", "control": f"{pc} of {len(c)}", + "means": [round(ma, 3), round(mc, 3)]} + return out + + +def _rows(path: Path) -> list[dict[str, Any]]: + return [json.loads(line) for line in path.read_text().splitlines() if line.strip()] + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__.split("\n", 1)[0]) + sub = parser.add_subparsers(dest="command", required=True) + p = sub.add_parser("plan") + p.add_argument("--stage", choices=list(STAGES), required=True) + p.add_argument("--arms", nargs="*", choices=list(ARMS)) + p.add_argument("--battery", action="store_true", help="the battery lives of the arms") + p.add_argument("--out", type=Path, required=True) + r = sub.add_parser("read") + r.add_argument("results", type=Path) + r.add_argument("--stage", choices=list(STAGES), required=True) + r.add_argument("--arm", choices=CANDIDATES, help="the confirmation's candidate") + r.add_argument("--battery", type=Path, help="the battery's results") + args = parser.parse_args(argv) + if args.command == "plan": + arms = tuple(args.arms) if args.arms else None + specs = (battery_plan({a: ARMS[a] for a in (arms or ("control",))}) if args.battery + else plan(args.stage, arms)) + args.out.parent.mkdir(parents=True, exist_ok=True) + args.out.write_text(json.dumps(specs, indent=1)) + print(f"{len(specs)} lives") + return 0 + cells = table(_rows(args.results)) + for (arm, w), cell in sorted(cells.items()): + m = means(cell) + print(f"{arm:8s} w{w:<4d} n{len(cell)} " + " ".join(f"{k} {m[k]:.3f}" for k in READINGS) + + " | swing " + " ".join(f"{cell[f]['swing']:.3f}" for f in sorted(cell))) + result: dict[str, Any] = {} + if args.stage == "dev": + result["selection"] = select(cells) + elif args.arm: + result["gates"] = gates(cells, args.arm) + if args.battery: + result["battery"] = battery_gates(_rows(args.battery), args.arm) + print(json.dumps(result, indent=1)) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/benchmarks/grammar/routes.py b/benchmarks/grammar/routes.py new file mode 100644 index 00000000..3cc54ee6 --- /dev/null +++ b/benchmarks/grammar/routes.py @@ -0,0 +1,499 @@ +"""Routes with a goal (cadence#257): a chamber where the first word anticipates a constraint no +earlier word settles. + +The world says the facts of a small graph, one fact ``X near Y end`` at a time, heard words with +no ``SPEAK`` flag: from a start, two branches of the same length (two or three hops), each a chain +of distinct places. The meaning names the start (``FROM``) and the goal (``TO``), the last place of +one branch. At ``GO`` the brain says the route as the facts say it, ``start near a1 near ... +near goal end``, one word per moment, hearing its previous word with ``SPEAK``. The first hop is +the decision: the start has two neighbours, and only one of them leads to the goal, which the +brain names only at the route's end. A speaker word by word says the first hop before the +goal; a planning window of three settles the first hop with the words after it. + +Held out (``Split.of``): eight ordered start-goal pairs per founder that no training episode +joins, so ``held`` panels are routes between places training never joined, on graphs drawn +fresh. ``trained`` panels draw from the trained pairs. Every panel is four batches of sixteen +episodes (``panel_batches`` sets the count), two and three hops in turn, from +``SeedSequence([founder, 33, 257])``; the training stream is ``SeedSequence([founder, 11, +257])``. + +Speakers: + +- ``word``: ``Brain.compose(INPUTS, ACTIONS)``, a lesson on each word before the greedy act. +- ``sliding``: ``Brain.compose(INPUTS, 3 * ACTIONS, slots=3)``: at each moment the lesson teaches + the next three words (``end`` past the route's end), the act settles three slots at once, the + brain says the first slot's word and hears it, then plans again. +- ``lstm``, ``transformer``: next-word learners at the brain's parameter count (``controls.py``'s + architectures and rates), taught teacher-forced on the same episodes, online in the brain's + order or offline to convergence, read greedy and closed loop by the same scorer. + +Scores: ``first_hop`` (the place said after the first ``near`` is the goal branch's first place), +``exact`` (the +whole route through ``end``) and ``valid`` (every word said after the start follows the one +before it by a fact, the last is the goal, then ``end``). + + python benchmarks/grammar/routes.py run --founders 0 1 2 --out DIR/r.jsonl + python benchmarks/grammar/routes.py controls --founders 0 1 2 --out DIR/c.jsonl + python benchmarks/grammar/routes.py read DIR/r.jsonl DIR/c.jsonl +""" + +from __future__ import annotations + +import argparse +import copy +import json +import math +import sys +import time +import warnings +from dataclasses import dataclass, field +from multiprocessing import get_context +from pathlib import Path +from typing import Any + +import numpy as np + +PLACES = ("dog", "cat", "fox", "owl", "dogs", "cats", "foxes", "owls") +WORDS = (*PLACES, "near", "end") +NEAR, END = len(PLACES), len(PLACES) + 1 +CONCEPTS = tuple(p.upper() for p in PLACES) + ("FROM", "TO", "GO", "SPEAK") +C = {name: len(WORDS) + i for i, name in enumerate(CONCEPTS)} +ACTIONS, INPUTS = len(WORDS), len(WORDS) + len(CONCEPTS) +BATCH, HOPS, WINDOW = 16, (2, 3), 3 +SEEDS = {"train": (11, 257), "test": (33, 257), "split": (7, 257)} +BATCHES = 3000 +FOUNDERS = tuple(range(9625700, 9625706)) + + +def vector(*names: str) -> np.ndarray: + x = np.zeros(INPUTS) + for name in names: + x[C[name]] = 1.0 + return x + + +def heard(word: int) -> np.ndarray: + x = np.zeros(INPUTS) + x[word] = 1.0 + return x + + +def spoken(word: int) -> np.ndarray: + x = heard(word) + x[C["SPEAK"]] = 1.0 + return x + + +@dataclass +class Row: + pre: list[np.ndarray] # the facts, the meaning; then GO + target: list[int] # the route, start to goal, then end + edges: set[tuple[int, int]] = field(default_factory=set) + meta: dict[str, Any] = field(default_factory=dict) + + +@dataclass(frozen=True) +class Split: + held: frozenset[tuple[int, int]] + + @classmethod + def of(cls, founder: int) -> Split: + rng = np.random.default_rng(np.random.SeedSequence([int(founder), *SEEDS["split"]])) + pairs = [(s, g) for s in range(len(PLACES)) for g in range(len(PLACES)) if s != g] + chosen = rng.choice(len(pairs), size=8, replace=False) + return cls(frozenset(pairs[int(i)] for i in chosen)) + + +def episode(rng: np.random.Generator, hops: int, start: int, goal: int) -> Row: + """A graph of two branches of ``hops`` from ``start``, one ending at ``goal``.""" + others = [p for p in range(len(PLACES)) if p not in (start, goal)] + picked = [int(p) for p in rng.permutation(others)[: 2 * hops - 1]] + good = [start, *picked[: hops - 1], goal] + bad = [start, *picked[hops - 1:]] + edges = [(a, b) for chain in (good, bad) for a, b in zip(chain, chain[1:], strict=False)] + order = rng.permutation(len(edges)) + pre: list[np.ndarray] = [] + for i in order: + a, b = edges[int(i)] + pre += [heard(a), heard(NEAR), heard(b), heard(END)] + pre += [vector(CONCEPTS[start], "FROM"), vector(CONCEPTS[goal], "TO")] + route = [w for place in good for w in (place, NEAR)][:-1] + return Row(pre, [*route, END], set(edges), + {"hops": hops, "start": start, "goal": goal, "first": good[1], "other": bad[1]}) + + +def draw(rng: np.random.Generator, split: Split, hops: int, held: bool) -> Row: + while True: + s, g = (int(v) for v in rng.choice(len(PLACES), size=2, replace=False)) + if ((s, g) in split.held) == held: + return episode(rng, hops, s, g) + + +def batch(rng: np.random.Generator, split: Split, hops: int, held: bool = False) -> list[Row]: + return [draw(rng, split, hops, held) for _ in range(BATCH)] + + +def train_batches(founder: int, count: int) -> list[list[Row]]: + rng = np.random.default_rng(np.random.SeedSequence([int(founder), *SEEDS["train"]])) + split = Split.of(founder) + return [batch(rng, split, HOPS[b % len(HOPS)]) for b in range(count)] + + +def panels(founder: int, batches: int = 4) -> dict[str, list[list[Row]]]: + rng = np.random.default_rng(np.random.SeedSequence([int(founder), *SEEDS["test"]])) + split = Split.of(founder) + return {name: [batch(rng, split, HOPS[b % len(HOPS)], held) for b in range(batches)] + for name, held in (("trained", False), ("held", True))} + + +def score(row: Row, said: list[int]) -> dict[str, Any]: + words = said[: said.index(END) + 1] if END in said else said + places = words[0:-1:2] + valid = (len(words) >= 4 and len(words) % 2 == 0 and words[-1] == END + and all(w == NEAR for w in words[1:-1:2]) and places[0] == row.meta["start"] + and places[-1] == row.meta["goal"] + and all((a, b) in row.edges for a, b in zip(places, places[1:], strict=False))) + hop = words[2] if len(words) > 2 and words[1] == NEAR else None + return {"exact": words == row.target, "first_hop": hop == row.meta["first"], + "other_hop": hop == row.meta["other"], "valid": bool(valid)} + + +def tally(rows: list[Row], said: list[list[int]]) -> dict[str, Any]: + scores = [score(r, s) for r, s in zip(rows, said, strict=True)] + out: dict[str, Any] = {"n": len(scores)} + for key in scores[0]: + out[key] = float(np.mean([s[key] for s in scores])) + for hops in HOPS: + kept = [s for r, s in zip(rows, scores, strict=True) if r.meta["hops"] == hops] + if kept: + out[f"first_hop_{hops}"] = float(np.mean([s["first_hop"] for s in kept])) + out[f"exact_{hops}"] = float(np.mean([s["exact"] for s in kept])) + return out + + +# ------------------------------------------------------------------------------- the brain + + +def make_brain(speaker: str, founder: int, settings: dict[str, Any] | None = None, + module: int = 64) -> Any: + import cadence as cd + + k = WINDOW if speaker == "sliding" else 1 + return cd.Brain.compose(INPUTS, k * ACTIONS, modules=(module,), seed=int(founder), + slots=k, **{"backend": "cpu", **(settings or {})}) + + +def new_work() -> dict[str, float]: + return {"lessons": 0, "rows": 0, "agreed_rows": 0, "lesson_row_sweeps": 0, "acts": 0, + "act_sweeps": 0, "refused": 0, "exhausted": 0} + + +def ahead(target: list[int], j: int, k: int) -> list[int]: + """The ``k`` words from position ``j``, ``end`` past the route's end.""" + part = target[j:j + k] + return part + [END] * (k - len(part)) + + +def _act(brain: Any, x: np.ndarray, k: int, work: dict[str, float]) -> np.ndarray: + try: + action = np.asarray(brain.act(x, greedy=True)).reshape(len(x), k) + work["act_sweeps"] += float((brain.last_settlement or {}).get("steps", 0)) + except RuntimeError: + work["exhausted"] += 1 + action = np.full((len(x), k), -1) + work["acts"] += 1 + return action + + +def speak(brain: Any, rows: list[Row], k: int, *, teach: bool, + work: dict[str, float] | None = None, cap: int = 3) -> list[list[int]]: + """One batch on fresh streams. Taught: a lesson on the next ``k`` words before each act and + the next moment hearing the target's word. Read: greedy, closed loop. Either way the brain + says the first slot's word.""" + work = work if work is not None else new_work() + n = len(rows) + brain.reset() + for t in range(len(rows[0].pre)): + _act(brain, np.stack([r.pre[t] for r in rows]), k, work) + said: list[list[int]] = [[] for _ in rows] + steps = max(len(r.target) for r in rows) + (0 if teach else cap) + for j in range(steps): + if j == 0: + x = np.tile(vector("GO"), (n, 1)) + else: + words = [r.target[j - 1] if teach else said[i][j - 1] for i, r in enumerate(rows)] + x = np.stack([spoken(w) if w >= 0 else vector("SPEAK") for w in words]) + if teach: + labels = np.array([ahead(r.target, j, k) for r in rows]) + try: + report = brain.lesson(x, labels if k > 1 else labels[:, 0]) + work["lessons"] += 1 + work["rows"] += report.get("attempted_presentations", 0.0) + work["agreed_rows"] += report.get("agreed_presentations", 0.0) + work["lesson_row_sweeps"] += report.get("total_row_sweeps", 0.0) + except RuntimeError: + work["refused"] += 1 + action = _act(brain, x, k, work) + for i in range(n): + said[i].append(int(action[i, 0])) + if not teach and all(END in s for s in said): + break + return said + + +def read(speaker: Any, batches: list[list[Row]]) -> dict[str, Any]: + rows = [r for b in batches for r in b] + said = [s for b in batches for s in speaker(b)] + return tally(rows, said) + + +def run_life(spec: dict[str, Any]) -> dict[str, Any]: + """One brain life: ``speaker`` (``word`` or ``sliding``), ``founder``, ``batches``, + ``checkpoints``, ``module``, ``panel_batches``, ``settings``.""" + warnings.simplefilter("ignore") + import cadence as cd + + speaker, founder = spec["speaker"], int(spec["founder"]) + k = WINDOW if speaker == "sliding" else 1 + count = int(spec.get("batches", BATCHES)) + started = time.monotonic() + brain = make_brain(speaker, founder, spec.get("settings"), int(spec.get("module", 64))) + tests = panels(founder, int(spec.get("panel_batches", 4))) + checkpoints = set(int(c) for c in spec.get("checkpoints", [])) + work = new_work() + row: dict[str, Any] = {"speaker": speaker, "founder": founder, "k": k, "batches": count, + "module": int(spec.get("module", 64)), + "parameters": int(brain.parameters()), "cadence": cd.__version__, + "tag": spec.get("tag"), "readings": []} + + def reading(b: int) -> dict[str, Any]: + fork = copy.deepcopy(brain) + return {"batches": b, "work": dict(work), "panels": { + name: read(lambda rows: speak(fork, rows, k, teach=False), tests[name]) + for name in tests}} + + for b, rows in enumerate(train_batches(founder, count), start=1): + speak(brain, rows, k, teach=True, work=work) + if b in checkpoints or b == count: + row["readings"].append(reading(b)) + row["seconds"] = round(time.monotonic() - started, 1) + return row + + +# ---------------------------------------------------------------------------- the controls + + +def encode(rows: list[Row]) -> tuple[np.ndarray, np.ndarray]: + xs, ys = [], [] + for r in rows: + moments = [*r.pre, vector("GO")] + [spoken(w) for w in r.target[:-1]] + xs.append(np.stack(moments)) + ys.append([-100] * len(r.pre) + list(r.target)) + return np.stack(xs).astype(np.float32), np.array(ys, dtype=np.int64) + + +def build(model: str, width: int, seed: int) -> Any: + """``controls.py``'s two architectures at this chamber's inputs and actions.""" + import torch + from torch import nn + + torch.manual_seed(int(seed)) + + class LSTM(nn.Module): # type: ignore[misc] + def __init__(self) -> None: + super().__init__() + self.cell = nn.LSTM(INPUTS, width, batch_first=True) + self.out = nn.Linear(width, ACTIONS) + + def forward(self, x: Any) -> Any: + return self.out(self.cell(x)[0]) + + class Transformer(nn.Module): # type: ignore[misc] + def __init__(self) -> None: + super().__init__() + self.inp = nn.Linear(INPUTS, width) + layer = nn.TransformerEncoderLayer(width, 2, 2 * width, dropout=0.0, + batch_first=True, norm_first=True) + self.body = nn.TransformerEncoder(layer, 2, enable_nested_tensor=False) + self.out = nn.Linear(width, ACTIONS) + position = torch.arange(64)[:, None] + scale = torch.exp(torch.arange(0, width, 2) * (-math.log(10000.0) / width)) + table = torch.zeros(64, width) + table[:, 0::2] = torch.sin(position * scale) + table[:, 1::2] = torch.cos(position * scale)[:, : width // 2] + self.register_buffer("position", table) + + def forward(self, x: Any) -> Any: + n = x.shape[1] + mask = torch.triu(torch.full((n, n), float("-inf")), diagonal=1) + return self.out(self.body(self.inp(x) + self.position[:n], mask=mask, + is_causal=True)) + + return LSTM() if model == "lstm" else Transformer() + + +def matched_width(model: str, target: int) -> int: + best = None + for width in range(4, 257, 2): + n = int(sum(p.numel() for p in build(model, width, 0).parameters())) + if best is None or abs(n - target) < abs(best[1] - target): + best = (width, n) + assert best is not None + return best[0] + + +def control_life(spec: dict[str, Any]) -> dict[str, Any]: + """One control: ``model``, ``mode`` (``online``: one pass, one Adam step per batch at + 0.003; ``offline``: shuffled epochs at 0.001 until the loss stops falling, patience 3, at + most 30), ``founder``, ``batches``, ``parameters`` (to match).""" + import torch + + torch.set_num_threads(1) + model, mode, founder = spec["model"], spec["mode"], int(spec["founder"]) + count = int(spec.get("batches", BATCHES)) + width = matched_width(model, int(spec["parameters"])) + net = build(model, width, founder) + opt = torch.optim.Adam(net.parameters(), lr=0.003 if mode == "online" else 0.001) + data = [encode(rows) for rows in train_batches(founder, count)] + + def update(x: np.ndarray, y: np.ndarray) -> float: + logits = net(torch.from_numpy(x)) + loss = torch.nn.functional.cross_entropy(logits.reshape(-1, ACTIONS), + torch.from_numpy(y).reshape(-1), + ignore_index=-100) + opt.zero_grad() + loss.backward() + torch.nn.utils.clip_grad_norm_(net.parameters(), 1.0) + opt.step() + return float(loss) + + started = time.monotonic() + epochs = 1 + if mode == "online": + for x, y in data: + update(x, y) + else: + order = np.random.default_rng(np.random.SeedSequence([founder, 99, 257])) + best, stale, epochs = math.inf, 0, 0 + while epochs < 30 and stale < 3: + mean = float(np.mean([update(*data[int(i)]) for i in order.permutation(count)])) + epochs += 1 + stale = 0 if mean < best - 1e-4 else stale + 1 + best = min(best, mean) + + def speaker(rows: list[Row]) -> list[list[int]]: + said: list[list[int]] = [[] for _ in rows] + with torch.no_grad(): + for _ in range(max(len(r.target) for r in rows) + 3): + moments = [[*r.pre, vector("GO")] + [spoken(w) for w in s] + for r, s in zip(rows, said, strict=True)] + x = np.stack([np.stack(m) for m in moments]).astype(np.float32) + words = net(torch.from_numpy(x))[:, -1].argmax(dim=1).numpy() + for s, w in zip(said, words, strict=True): + s.append(int(w)) + if all(END in s for s in said): + break + return said + + net.eval() + tests = panels(founder, int(spec.get("panel_batches", 4))) + return {"model": model, "mode": mode, "founder": founder, "width": width, + "parameters": int(sum(p.numel() for p in net.parameters())), "epochs": epochs, + "seconds": round(time.monotonic() - started, 1), "tag": spec.get("tag"), + "panels": {name: read(speaker, tests[name]) for name in tests}} + + +# -------------------------------------------------------------------------------- reading + + +def gate(brains: list[dict[str, Any]], panel: str = "held") -> dict[str, Any]: + """The declared gate: the sliding window's first-hop accuracy on ``panel`` minus word by + word's, paired by founder at the last reading; planning pays where the mean difference is + at least .05 and beyond two standard errors.""" + last = {(r["speaker"], int(r["founder"])): r["readings"][-1]["panels"][panel]["first_hop"] + for r in brains} + founders = sorted(f for s, f in last if s == "word" and ("sliding", f) in last) + diffs = np.array([last[("sliding", f)] - last[("word", f)] for f in founders]) + if len(diffs) < 2: + return {"founders": founders} + se = float(diffs.std(ddof=1) / np.sqrt(len(diffs))) + mean = float(diffs.mean()) + return {"founders": founders, "differences": [round(float(d), 3) for d in diffs], + "mean": round(mean, 4), "se": round(se, 4), + "pays": bool(mean >= 0.05 and mean > 2 * se)} + + +def _rows(path: Path) -> list[dict[str, Any]]: + return [json.loads(line) for line in path.read_text().splitlines() if line.strip()] + + +def _pool(fn: Any, specs: list[dict[str, Any]], out: Path, workers: int) -> None: + out.parent.mkdir(parents=True, exist_ok=True) + with out.open("a") as sink: + if workers <= 1: + for result in map(fn, specs): + sink.write(json.dumps(result) + "\n") + sink.flush() + else: + with get_context("spawn").Pool(min(workers, len(specs))) as pool: + for result in pool.imap_unordered(fn, specs, chunksize=1): + sink.write(json.dumps(result) + "\n") + sink.flush() + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__.split("\n", 1)[0]) + sub = parser.add_subparsers(dest="command", required=True) + r = sub.add_parser("run") + r.add_argument("--speakers", nargs="*", default=["word", "sliding"]) + r.add_argument("--founders", nargs="*", type=int, default=list(FOUNDERS)) + r.add_argument("--batches", type=int, default=BATCHES) + r.add_argument("--checkpoints", nargs="*", type=int, default=[]) + r.add_argument("--panel-batches", type=int, default=16) + r.add_argument("--out", type=Path, required=True) + r.add_argument("--workers", type=int, default=1) + c = sub.add_parser("controls") + c.add_argument("--models", nargs="*", default=["lstm", "transformer"]) + c.add_argument("--modes", nargs="*", default=["online", "offline"]) + c.add_argument("--founders", nargs="*", type=int, default=list(FOUNDERS)) + c.add_argument("--batches", type=int, default=BATCHES) + c.add_argument("--panel-batches", type=int, default=16) + c.add_argument("--out", type=Path, required=True) + c.add_argument("--workers", type=int, default=1) + g = sub.add_parser("read") + g.add_argument("brains", type=Path) + g.add_argument("controls", type=Path, nargs="?") + args = parser.parse_args(argv) + if args.command == "run": + specs = [{"speaker": s, "founder": f, "batches": args.batches, + "checkpoints": args.checkpoints, "panel_batches": args.panel_batches, + "tag": f"routes/{s}/f{f}"} for s in args.speakers for f in args.founders] + _pool(run_life, specs, args.out, args.workers) + return 0 + if args.command == "controls": + warnings.simplefilter("ignore") + target = int(make_brain("word", args.founders[0]).parameters()) + specs = [{"model": m, "mode": mode, "founder": f, "batches": args.batches, + "parameters": target, "panel_batches": args.panel_batches} + for m in args.models for mode in args.modes for f in args.founders] + _pool(control_life, specs, args.out, args.workers) + return 0 + brains = _rows(args.brains) + for b in sorted(brains, key=lambda r: (r["speaker"], r["founder"])): + last = b["readings"][-1]["panels"] + print(f"{b['speaker']:8s} f{b['founder']} " + " ".join( + f"{p} first {v['first_hop']:.2f} exact {v['exact']:.2f} valid {v['valid']:.2f}" + for p, v in last.items())) + if args.controls: + for c_ in sorted(_rows(args.controls), key=lambda r: (r["model"], r["mode"], + r["founder"])): + print(f"{c_['model']}/{c_['mode']} f{c_['founder']} " + " ".join( + f"{p} first {v['first_hop']:.2f} exact {v['exact']:.2f}" + for p, v in c_["panels"].items())) + print(json.dumps({"held": gate(brains, "held"), "trained": gate(brains, "trained")}, + indent=1)) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/benchmarks/grammar/test_exposure.py b/benchmarks/grammar/test_exposure.py new file mode 100644 index 00000000..54bb7d54 --- /dev/null +++ b/benchmarks/grammar/test_exposure.py @@ -0,0 +1,63 @@ +"""The exposure-parity instrument (cadence#270): its orders, its comparison on synthetic rows +and a short life of passes with the rehearsal.""" + +from __future__ import annotations + +import importlib.util +import sys +from pathlib import Path + +import numpy as np + + +def _load(name: str): + if name in sys.modules: + return sys.modules[name] + spec = importlib.util.spec_from_file_location(name, Path(__file__).with_name(f"{name}.py")) + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + sys.modules[name] = module + spec.loader.exec_module(module) + return module + + +gr = _load("grammar") +ex = _load("exposure") + + +def test_later_passes_take_the_transformer_epochs_order(): + passes = ex.order(5, 3, 10) + assert list(passes[0]) == list(range(10)) + epochs = np.random.default_rng(np.random.SeedSequence([5, 99, 0])) + assert list(passes[1]) == list(epochs.permutation(10)) + assert list(passes[2]) == list(epochs.permutation(10)) + + +def _panels(level: float) -> dict: + return {panel: {key: level} for panel, key in ex.READINGS.values()} + + +def test_the_comparison_picks_the_pass_within_the_controls_compute(): + life = {"arm": "rehearsal", "founder": 1, "passes": [ + {"pass": 1, "macs": 1.0, "readings": ex.scores(_panels(0.90))}, + {"pass": 2, "macs": 2.0, "readings": ex.scores(_panels(0.97))}, + {"pass": 3, "macs": 3.0, "readings": ex.scores(_panels(0.99))}]} + control = {"mode": "offline", "founder": 1, + "readings": [{"panels": _panels(1.0), "work": {"macs": 2.5}}]} + out = ex.compare([life], [control], "rehearsal") + assert out["E1 equal compute"]["passes"] == {1: 2} + assert out["E1 equal compute"]["pass"] and out["E2 saturation"]["pass"] + control["readings"][0]["work"]["macs"] = 1.5 + out = ex.compare([life], [control], "rehearsal") + assert not out["E1 equal compute"]["pass"] + assert set(out["E1 equal compute"]["apart"]) == set(ex.READINGS) + + +def test_a_life_of_passes_counts_its_rehearsal(): + row = ex.run_life({"arm": "rehearsal", "founder": 0, "passes": 2, "batches": 2, + "panel_batches": 1}) + assert [p["pass"] for p in row["passes"]] in ([1], [1, 2]) + work = row["passes"][0]["work"] + assert work["rehearsal_row_sweeps"] > 0 and work["macs"] > 0 + assert row["passes"][-1]["macs"] >= work["macs"] + assert "located" in row["passes"][-1] diff --git a/benchmarks/grammar/test_grammar.py b/benchmarks/grammar/test_grammar.py index 0f22e30e..e877e3d1 100644 --- a/benchmarks/grammar/test_grammar.py +++ b/benchmarks/grammar/test_grammar.py @@ -372,3 +372,26 @@ def test_a_window_of_one_is_the_chambers_life_and_wider_windows_say_k_words(): def test_settings_name_a_lifes_backend(): assert gr.make_brain("recipe", 0, {"backend": "cpu"}, module=16).brain.backend == "cpu" + + +def test_the_cost_of_a_pass_is_read_per_learner(tmp_path): + pytest.importorskip("torch") + _load("controls") + cost = _load("cost") + result = cost.run(0, 2, every=1) + assert set(result["seconds"]) == set(cost.LEARNERS) + assert all(seconds > 0 for seconds in result["seconds"].values()) + assert result["work"]["plain"]["acts"] == result["work"]["compiled"]["acts"] > 0 + assert [mark["batches"] for mark in result["marks"]] == [1, 2] + assert result["against_transformer"]["transformer"] == 1.0 + assert cost.main(["--batches", "1", "--learners", "plain,compiled", + "--out", str(tmp_path / "cost.json")]) == 0 + assert "against_transformer" not in (tmp_path / "cost.json").read_text() + found = cost.parts("compiled", 0, 2) + assert found["parts"] and abs(sum(found["parts"].values()) - found["wall"]) < 1e-6 + assert set(found["parts"]) <= {part for _, _, part in cost.PARTS} | {cost.OUTSIDE} + assert found["parts"]["kernels: a moment in one call"] > 0 + assert cost.parts("plain", 0, 2)["parts"]["kernels: the solves"] > 0 + import cadence.fused as fused + + assert not hasattr(fused.fused_fast_solve, "__wrapped__") # restored diff --git a/benchmarks/grammar/test_lesson_step.py b/benchmarks/grammar/test_lesson_step.py new file mode 100644 index 00000000..b23925fb --- /dev/null +++ b/benchmarks/grammar/test_lesson_step.py @@ -0,0 +1,84 @@ +"""The lesson-step instrument (cadence#258): its plans, its readings and gates on synthetic +rows, and a short life of a candidate.""" + +from __future__ import annotations + +import importlib.util +import sys +from pathlib import Path + +import numpy as np + + +def _load(name: str): + if name in sys.modules: + return sys.modules[name] + spec = importlib.util.spec_from_file_location(name, Path(__file__).with_name(f"{name}.py")) + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + sys.modules[name] = module + spec.loader.exec_module(module) + return module + + +gr = _load("grammar") +ls = _load("lesson_step") + + +def _row(tag: str, founder: int, clause: list[float], level: float = 0.9) -> dict: + """A life whose exact clauses at 3,000, 3,500 and 4,000 read ``clause``.""" + readings = [] + for b in range(500, 4001, 500): + c = clause[[3000, 3500, 4000].index(b)] if b >= 3000 else level + panels = {"clause": {"exact": c}, "dyck": {"exact": level}, + "path": {"exact": level, "conditions": {"hops2": level}}} + if b == 4000: + panels.update({"clause_held": {"exact": level}, "roles": {"both": level}}) + readings.append({"phase": 0, "batches": b, "panels": panels}) + return {"tag": tag, "founder": founder, "readings": readings} + + +def test_the_plan_covers_every_arm_width_and_founder(): + specs = ls.plan("dev") + assert len(specs) == len(ls.ARMS) * 2 * 3 + assert {s["module"] for s in specs} == {64, 128} + assert all(s["arm"] == "default" and s["checkpoints"] == list(ls.CHECKPOINTS) + for s in specs) + assert gr.ARMS["default"] == {"recall": True} + + +def test_swing_and_levels_are_read_from_the_checkpoints(): + reading = ls.life(_row("dev/control/w64/f0", 0, [0.8, 0.9, 0.85])) + assert np.isclose(reading["swing"], 0.1) and reading["clauses"] == 0.85 + assert reading["brackets"] == 0.9 and reading["two_hops"] == 0.9 + + +def test_the_selection_takes_the_steadiest_eligible_candidate(): + rows = [] + for w in (64, 128): + for f in (0, 1, 2): + rows.append(_row(f"dev/control/w{w}/f{f}", f, [0.7, 0.9, 0.8])) + rows.append(_row(f"dev/e99p5/w{w}/f{f}", f, [0.88, 0.9, 0.89])) + rows.append(_row(f"dev/e999p5/w{w}/f{f}", f, [0.9, 0.9, 0.9], level=0.5)) + chosen = ls.select(ls.table(rows)) + assert chosen["chosen"] == "e99p5" and not chosen["flagged"] + assert not chosen["eligible"]["e999p5"] + + +def test_the_gates_count_settled_founders(): + rows = [] + for w in (64, 128): + for f in range(6): + rows.append(_row(f"confirm/control/w{w}/f{f}", f, [0.7, 0.9, 0.8])) + swing = [0.9, 0.9, 0.9] if f < 5 else [0.7, 0.9, 0.8] + rows.append(_row(f"confirm/e99p5/w{w}/f{f}", f, swing)) + g = ls.gates(ls.table(rows), "e99p5") + assert g["C1 settles, width 64"]["pass"] and g["C1 settles, width 64"]["within"] == "5 of 6" + assert g["C2 levels hold, width 128"]["pass"] + + +def test_the_control_lives_with_its_tag(): + spec = {"arm": "default", "founder": 0, "phases": [["g1", 3]], "panel_batches": 1, + "settings": ls.ARMS["control"], "tag": "smoke/control/w64/f0"} + row = gr.run_life(spec) + assert row["tag"] == spec["tag"] and row["settings"] == {"recall": True} diff --git a/benchmarks/grammar/test_routes.py b/benchmarks/grammar/test_routes.py new file mode 100644 index 00000000..0fbf69e0 --- /dev/null +++ b/benchmarks/grammar/test_routes.py @@ -0,0 +1,102 @@ +"""Routes with a goal (cadence#257): the graphs, the held-out pairs, the scorer, a short life +of each speaker and a short control.""" + +from __future__ import annotations + +import importlib.util +import sys +from pathlib import Path + +import numpy as np +import pytest + + +def _load(name: str): + if name in sys.modules: + return sys.modules[name] + spec = importlib.util.spec_from_file_location(name, Path(__file__).with_name(f"{name}.py")) + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + sys.modules[name] = module + spec.loader.exec_module(module) + return module + + +rt = _load("routes") + + +def solve(row) -> list[int]: + """The route from the heard facts and the meaning alone.""" + edges, start, goal = [], None, None + facts = row.pre[:-2] + for t in range(0, len(facts), 4): + edges.append((int(np.argmax(facts[t])), int(np.argmax(facts[t + 2])))) + for x in row.pre[-2:]: + place = int(np.argmax(x[rt.C["DOG"]:rt.C["DOG"] + len(rt.PLACES)])) + if x[rt.C["FROM"]]: + start = place + if x[rt.C["TO"]]: + goal = place + nxt: dict[int, list[int]] = {} + for a, b in edges: + nxt.setdefault(a, []).append(b) + for first in nxt[start]: + route = [start, first] + while route[-1] in nxt: + route.append(nxt[route[-1]][0]) + if route[-1] == goal: + return [w for place in route for w in (place, rt.NEAR)][:-1] + [rt.END] + raise AssertionError("no branch reaches the goal") + + +def test_one_first_hop_reaches_the_goal_and_the_heard_moments_answer_it(): + rng = np.random.default_rng(0) + split = rt.Split.of(0) + for hops in rt.HOPS: + for _ in range(50): + row = rt.draw(rng, split, hops, held=False) + assert len(row.target) == 2 * hops + 2 and row.target[-2] == row.meta["goal"] + assert row.meta["first"] != row.meta["other"] + assert len({a for a, _ in row.edges} | {b for _, b in row.edges}) == 2 * hops + 1 + assert solve(row) == row.target + + +def test_held_pairs_never_reach_training_and_fill_the_held_panel(): + split = rt.Split.of(3) + assert len(split.held) == 8 + for rows in rt.train_batches(3, 40): + assert all((r.meta["start"], r.meta["goal"]) not in split.held for r in rows) + tests = rt.panels(3, 2) + assert all((r.meta["start"], r.meta["goal"]) in split.held + for b in tests["held"] for r in b) + assert [b[0].meta["hops"] for b in tests["trained"]] == [2, 3] + + +def test_the_scorer(): + row = rt.episode(np.random.default_rng(1), 2, 0, 1) + assert rt.score(row, row.target) == {"exact": True, "first_hop": True, "other_hop": False, + "valid": True} + wrong = [0, rt.NEAR, row.meta["other"], rt.NEAR, 1, rt.END] + s = rt.score(row, wrong) + assert not s["exact"] and s["other_hop"] and not s["valid"] + + +@pytest.mark.parametrize("speaker", ["word", "sliding"]) +def test_a_short_life_of_each_speaker(speaker): + row = rt.run_life({"speaker": speaker, "founder": 0, "batches": 2, "panel_batches": 1}) + assert row["k"] == (3 if speaker == "sliding" else 1) + reading = row["readings"][-1] + assert reading["work"]["lessons"] == 6 + 8 # routes of two and three hops with near, end + assert set(reading["panels"]) == {"trained", "held"} + + +def test_a_short_control_and_the_gate(): + pytest.importorskip("torch") + row = rt.control_life({"model": "lstm", "mode": "online", "founder": 0, "batches": 2, + "parameters": 5000, "panel_batches": 1}) + assert 0 <= row["panels"]["held"]["first_hop"] <= 1 + brains = [{"speaker": s, "founder": f, "readings": [{"panels": {"held": { + "first_hop": 0.5 + (0.2 if s == "sliding" else 0.0) + 0.01 * f}}}]} + for s in ("word", "sliding") for f in range(6)] + g = rt.gate(brains) + assert g["pays"] and abs(g["mean"] - 0.2) < 1e-9 diff --git a/benchmarks/heads/README.md b/benchmarks/heads/README.md new file mode 100644 index 00000000..de2e64f0 --- /dev/null +++ b/benchmarks/heads/README.md @@ -0,0 +1,416 @@ +# Heads chamber: composed attention functions + +Interpretability work on transformers finds functions composed across heads: one head detects a +duplicate, another suppresses attention to it, another moves the remaining name to the output. +This instrument reads that composition in one brain +([#269](https://github.com/muellerberndt/cadence/issues/269), under the attention tracker +[#272](https://github.com/muellerberndt/cadence/issues/272)): can the brain do it, where does it +fail, and how does it compare? The task that names it: `when mary and john went out john gave a +drink to` answers `mary`, with names no lesson used. Each part is read alone first, so a composed +failure can be traced to its part. Read the +[recall memory](../../docs/memory.md#the-recall-memory-in-full) and the +[grammar chamber](../grammar/README.md), whose scale and controls this chamber takes, before +interpreting results. It establishes no default. + +## The language and the split + +Twenty words, each an input and an action of the brain: twelve names (`mary john anna tom lisa paul +sara mark nina owen ruth sean`) and the frame `when and went out gave a drink to`. Per founder +(`Split.of`) eight names are taught and four are never-taught: never heard and never answered in +training. Four pairs of taught names are never heard together. Every life on a founder hears the +same batches, from `SeedSequence([founder, 11])`, and reads the same frozen panels, from +`SeedSequence([founder, 33])`. A batch is one shape, sixteen streams with their own content, as in +the grammar chamber. + +## The tasks + +Lessons fall on answer moments only, before the act on the same observation (`Brain.lesson`, then +`Brain.act`). Every other moment is heard and acted on greedily. Each task is a life of its own. +The single functions take 1,500 batches (the sequence battery's), the sentences 4,000 (one pass of +the grammar chamber). + +| Task | A batch | The answer | What it reads | +| --- | --- | --- | --- | +| `previous` | a stream of 3 to 8 tokens, the last flagged `ASK` | the token before it | position over content | +| `duplicate` | 3 to 6 distinct tokens, then a token flagged `ASK` | `yes` if it was told, else `no` | the duplicate signal, no route needed | +| `successor` | 3 to 6 distinct tokens told once, each episode in its own order, then one flagged `ASK` | the token told after it | the next port by content | +| `successor_fixed` | the same with one fixed order of the taught tokens in training | the same | whether weights take over from the read | +| `induction` | `... A B ... A`, the query unflagged, 5 to 10 tokens | `B` | the sequence battery's task | +| `twohop` | pairs told, then `a`, then `THINK` | `b`, then `c` | a read on what a read found | +| `copying` | `START`, 3 to 6 distinct tokens, `GO`, each answer heard | each token in turn | closed loop (`test`) and teacher-forced (`forced`) | +| `suppression` | `A B S GO`, `S` one of the two | the one not repeated | inhibition without a frame | +| `which` | the same | `first` or `second`, the place of the one not repeated | suppression with no route needed | +| `ioi` | `when A and B (went (out)) S gave (a drink) to` | the one not repeated | the named task | +| `bare` | `A B (went (out)) S gave (a drink) to` | the one not repeated | the same with no fixed word before the names | +| `mover` | the frame without `S`, `to` flagged `FIRST` or `SECOND` | the name at that place | the name mover alone, inside the frame | +| `dupbit` | the sentence of `ioi`, `to` flagged `WHICH` | `first` or `second`, where the duplicate's earlier mention stood | the duplicate signal through the frame, no name moved | + +The duplicate's position varies by the words before it (none, `went`, `went out`) and after it +(`gave`, `gave a drink`). Training holds five of the six shapes; `went S gave to` is read only. + +## Panels and conditions + +A single function's panel mixes rows by condition. `taught`: every token taught. `key_new`: the +token the read is addressed by is never-taught, the answer taught. `answer_new`: the answer is +never-taught, the key taught. `new`: both. Tokens around them are taught. `copying` reads each +answer's condition from the token heard and the token asked, and its closed loop holds a confound +the panel's forced reading removes: a token the brain cannot say leaves the next moment hearing +another. + +Sentence panels come in twins: one pair of names asked twice, each name repeated once, so a read +of a fixed place reads .50 and `both` (twins both right) reads .00. + +| Panel | Names | +| --- | --- | +| `ioi` | taught, pairs training shows, training's shapes | +| `ioi_pair` | taught, the pairs never heard together | +| `ioi_shape` | taught, pairs training shows, the shape no lesson reached | +| `ioi_half` | one name of each pair never-taught | +| `ioi_new` | both never-taught: the named task | + +**The half panels read no suppression alone.** A pair of one taught and one never-taught name can +be answered by saying the taught one. `which` and `dupbit` read suppression and the duplicate +signal at never-taught names with an answer that needs no route. + +## What each row reads + +- `answer`: the brain's greedy act is the answer. `says_avoid`: it says the duplicate. `pair`: it + says one of the two names. +- `first`: the slot head 0 attends most holds the right token as its value or its next. Where + neighbouring slots hold both names (`suppression`) this reads nothing; `ports` is the reading + there. +- `ports`, `port_best`: per read port, whether the token it drives most is the right one, and the + best port. +- `mass`, `mass_avoid`: the attention on slots that hold the right token, and the duplicate. +- `duplicate_read`, at the duplicate's own moment: whether the first choice's value is the + duplicate (`first_value`) and the attention on its earlier mention (`mass`). +- `terms`: head 0's logits at the duplicate's moment and at the answer, on the slots a suppressing + read tells apart, split by source: the event against the keys (`key`), the address map's event + rows, cortex columns and bias (`event`, `cortex`, `bias`), and the same three of the position + maps (`position_event`, `position_cortex`, `position_bias`). +- `decode` (a life's option): a read-only ridge decoder of which name repeated, fit on taught + sentences, from the cortex and the trace after the duplicate, before the answer and after it. The + trace holds the cortex's activity of the moment before, so the two read alike. + +## Causality + +Each panel is read again on a fork under each intervention, the answer and the read's own reading +kept. + +| Intervention | At | +| --- | --- | +| `silence` | the read's amplitude at zero at every moment | +| `silence_answer`, `silence_dup` | at the answer, at the duplicate's moment | +| `erase`, `erase_dup` | the slot each head attends most emptied (key, value, next) before the act, at the answer, at the duplicate's moment | +| `erase_source` | the answer's own slot emptied, and the next of the slot before it: both routes | +| `erase_other` | a control: a slot neither attended nor holding the answer emptied | +| `trace`, `cortex`, `focus` | at the answer: the working trace, the cortex the address reads, the previous read's focus cleared | +| `swap_trace`, `swap_cortex`, `swap_focus`, `swap_slot`, `swap_all` | at the answer, on twin panels: the twin's carrier in the row's place (`swap_slot`: the duplicate's slot); an answer that follows the twin's names the carrier | + +## Arms + +Each on founders 0 to 2, then on brain seeds (`init_seed`, on the founder's split, lessons and +panels) and last-bit replicas of what moves. Every arm is +`Brain.compose(inputs, actions, modules=(64,), seed=founder)` with the settings below; development. + +| Arm | Settings | +| --- | --- | +| `Fd` | the base composition: `episodic=False, recall=True, recall_recency=0.0, learning_surprise=True, recall_boundary=True, recall_anchor=True, recall_slope=0.1, recall_position_rate=0.01, recall_relative=True, teaching="deep"` (the address lesson as a contrast, the library's one law), no apical | +| `Fd2` | `Fd` with `recall_heads=2` | +| `FdT` | `Fd` with the working trace cleared before every lesson and act, as the grammar harness's cut | +| `Fdg`, `Fd2g`, `FdTg` | each with `recall_gain=30.0` and `recall_tokens` on every word | + +`Support` (`support` in a life's spec) is the location's ladder and no arm: what the harness gives or +takes from a life, to name a term by what restores the composition. + +| Support | What it does | +| --- | --- | +| `no_address_cortex` (address cut) | the address map's cortex columns stay at zero through the life | +| `no_cortex` (cortex cut) | the cortex of the moment before reads zero to every map, at every lesson and act | +| `center_cortex` | the maps read the cortex less its running mean | +| `raw_dup` (oracle) | the read at the duplicate's moment scores the event against the keys alone | +| `gain_at_answer` (oracle) | the gains' drive reaches the motor at answer moments alone | +| `gain_late` | the gains stay at zero through the first thousand lessons; with `gain_late_value` in a life's spec (one number, or one per port) they are set to it then | +| `cortex_unit` | the maps read the cortex of the moment before at unit norm per stream (the emulation of `recall_cortex_norm`) | +| `cortex_scale`, `cortex_rate` (a life's spec) | the maps read that cortex times a number; each lesson's step on every map's cortex columns times a number | +| `gain_start` (a life's spec) | every port's gain at birth, one number or one per port | +| `hold_focus`, `hold_cortex`, `hold_trace` (oracles) | what the duplicate's moment left stands in place up to the answer | + +## Controls + +[`nets.py`](nets.py): the LSTM and the transformer of the +[grammar chamber's controls](../grammar/controls.py), built by that code at a task's inputs and +actions, their widths the nearest to the `Fd` brain's `parameters()` on that task. They hear the +brain's moments and are taught its answer moments by cross-entropy, teacher-forced; the same scorer +reads them. `online`: one pass, one Adam step per batch at 0.003, equal exposure. `offline`: the +same batches in shuffled epochs at 0.001 until the seconds in updates reach the `Fd` life's on the +same box, equal compute. Causality on a control: a moment blanked at test, and on a transformer the +attention layers' output silenced. They are independently scored controls and never part of a +brain's answer path. + +**How a never-taught name reaches the output.** In `lstm` and `transformer` it has no route: its +input weights stay at their random start and its output row is a row of the last linear map that no +lesson raised and every lesson's cross-entropy lowers. The brain without `recall_gain` has none +either. The brain with `recall_gain` has one the two controls lack: a learned scalar per read port +on the token's identity. `tied`, reported apart, gives the transformer the matching route by weight +tying: its logits are the hidden state's product with the same token's input weights. `pointer` adds +one more attention over the moments before, whose read of the heard tokens reaches the logits through +one learned scalar from zero, the brain's route itself, beside the last linear map. Both read .00 at +every never-taught answer: the route exists and goes unused, since the last linear map answers the +taught tokens first. `copy` takes that map from the tokens, so that a token's logit is the log of +that attention's read of it, a pointer network. It is the control whose route is used: a gain reading +above the plain networks at never-taught answers is that route and no more, and a reading against +`copy` compares attention with attention. + +## Readings + +Development, on one shared 64-vCPU box whose load stood near 80, so seconds are inflated and compare work within a row. A life's split, lessons and panels come from its founder. `Fd`'s lives on the library with `recall_gain` equal those on the library without it, panels and work, on all seven single functions. + +**Noise.** A condition holds 96 rows (192 for `duplicate`, 160 for a sentence panel), about .05 at .5 on sampling alone. Last-bit replicas move a function's cell by a mean of .001 and at most .15 (72 lives) and flip no sentence life (12 lives), so the spread that counts is between brains. On six brains `Fd` stays within .07 on every function's cell below; `FdT` spreads by .18 to .22 at a never-taught key on `successor` and `induction`, and `Fd2` by .17 on `duplicate`. Sentence lives are bimodal: a life reads 1.00 or about .50, so a rate over twelve lives is the reading and a mean of three is a screen. + +### Single functions + +Means over founders 0 to 2. `Read` is the first choice of `first` at both-new. `Silenced` and `Erased` are on taught rows; for a network `Erased` blanks the moment that told the answer. `Control` is `erase_other`. + +| Function | Arm | Taught | Key new | Answer new | Both new | Read | Silenced | Erased | Control | Seconds | Parameters | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | +| `previous` | `Fd` | 1.00 | 1.00 | .00 | .00 | 1.00 | .69 | .50 | 1.00 | 63 | 15,326 | +| | `FdT` | 1.00 | 1.00 | .00 | .00 | 1.00 | .11 | .42 | 1.00 | 60 | 15,326 | +| | `Fd2` | 1.00 | 1.00 | .00 | .00 | 1.00 | .52 | .23 | 1.00 | 53 | 24,738 | +| | `Fdg` | 1.00 | 1.00 | .98 | 1.00 | 1.00 | .35 | .48 | 1.00 | 67 | 15,328 | +| | transformer, one pass | 1.00 | 1.00 | .00 | .00 | | .12 | .13 | | 10 | 15,852 | +| | transformer, equal compute | 1.00 | 1.00 | .00 | .00 | | .14 | .14 | | 69 | 15,852 | +| | LSTM, one pass | 1.00 | 1.00 | .00 | .00 | | | .13 | | 4 | 15,564 | +| | `copy`, one pass | 1.00 | 1.00 | 1.00 | 1.00 | | .89 | .16 | | 13 | 15,525 | +| `duplicate` | `Fd` | 1.00 | .92 | | | | .50 | .34 | .86 | 43 | 15,456 | +| | `FdT` | 1.00 | .91 | | | | .50 | .43 | .94 | 41 | 15,456 | +| | `Fd2` | .90 | .94 | | | | .50 | .48 | .88 | 56 | 24,868 | +| | `Fdg` | 1.00 | .98 | | | | .50 | .61 | .74 | 85 | 15,458 | +| | transformer, one pass | .73 | .57 | | | | .52 | .53 | | 11 | 15,914 | +| | transformer, equal compute | 1.00 | .57 | | | | .52 | .52 | | 46 | 15,914 | +| | LSTM, one pass | .50 | .50 | | | | | .50 | | 4 | 15,674 | +| | LSTM, equal compute | 1.00 | .48 | | | | | .50 | | 43 | 15,674 | +| | `copy`, equal compute | 1.00 | .63 | | | | .48 | .51 | | 44 | 15,583 | +| `successor` | `Fd` | 1.00 | .97 | .00 | .00 | 1.00 | .11 | .03 | 1.00 | 40 | 15,326 | +| | `FdT` | 1.00 | .89 | .00 | .00 | 1.00 | .10 | .04 | 1.00 | 40 | 15,326 | +| | `Fd2` | 1.00 | .99 | .00 | .00 | 1.00 | .11 | .04 | 1.00 | 48 | 24,738 | +| | `Fdg` | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | .12 | .01 | 1.00 | 49 | 15,328 | +| | transformer, one pass | 1.00 | .57 | .00 | .00 | | .14 | .09 | | 9 | 15,852 | +| | transformer, equal compute | 1.00 | .65 | .00 | .00 | | .14 | .10 | | 51 | 15,852 | +| | LSTM, equal compute | .89 | .72 | .00 | .00 | | | .01 | | 46 | 15,564 | +| | `copy`, one pass | .98 | .63 | .97 | .56 | | .41 | .06 | | 9 | 15,525 | +| | `copy`, equal compute | 1.00 | .55 | 1.00 | .63 | | .34 | .08 | | 48 | 15,525 | +| `induction` | `Fd` | 1.00 | 1.00 | .00 | .00 | 1.00 | .14 | .12 | 1.00 | 56 | 14,955 | +| | `FdT` | 1.00 | .89 | .00 | .00 | 1.00 | .15 | .11 | 1.00 | 52 | 14,955 | +| | `Fd2` | 1.00 | 1.00 | .00 | .00 | 1.00 | .14 | .14 | 1.00 | 65 | 24,061 | +| | `Fdg` | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | .08 | .09 | 1.00 | 60 | 14,957 | +| | transformer, one pass | .99 | .35 | .00 | .00 | | .13 | .15 | | 13 | 15,822 | +| | transformer, equal compute | 1.00 | .47 | .00 | .00 | | .15 | .11 | | 56 | 15,822 | +| | LSTM, equal compute | .71 | .50 | .00 | .00 | | | .15 | | 55 | 15,348 | +| | `copy`, one pass | .99 | .61 | .93 | .35 | | .30 | .09 | | 11 | 15,497 | +| | `copy`, equal compute | 1.00 | .61 | .99 | .50 | | .36 | .08 | | 57 | 15,497 | +| `copying`, forced | `Fd` | 1.00 | .99 | .00 | .00 | 1.00 | .13 | .10 | 1.00 | 101 | 15,699 | +| | `FdT` | 1.00 | .99 | .00 | .00 | 1.00 | .12 | .06 | 1.00 | 90 | 15,699 | +| | `Fd2` | 1.00 | .98 | .00 | .00 | 1.00 | .12 | .09 | 1.00 | 117 | 25,419 | +| | `Fdg` | 1.00 | 1.00 | 1.00 | .99 | 1.00 | .13 | .00 | 1.00 | 91 | 15,701 | +| | transformer, one pass | 1.00 | .68 | .00 | .00 | | .13 | .79 | | 11 | 15,882 | +| | transformer, equal compute | 1.00 | .71 | .00 | .00 | | .14 | .79 | | 104 | 15,882 | +| | LSTM, equal compute | 1.00 | 1.00 | .00 | .00 | | | .80 | | 96 | 15,780 | +| | `copy`, equal compute | .98 | .69 | .95 | .73 | | .48 | .76 | | 104 | 15,553 | + +- **The read finds what no lesson used.** On `previous`, `successor`, `induction` and forced `copying` the first choice holds the never-taught answer on 1.00 of rows in every arm. Without the gain the answer is .00; the failing part is the route, not the selection. +- **The gain carries it on these functions.** `Fdg` says never-taught answers at .96 to 1.00. Silenced they read .00 to .09 and with the attended slot erased .00 to .19, and the control erasure holds, so the read is what says them. On six brains the reading stays .96 to 1.00. The pointer network reads both-new `induction` at .35 and .50 and `successor` at .56 and .63, one pass and equal compute, and `previous` at 1.00. +- **Where the read carries a function and where weights do.** Trained on one fixed order (`successor_fixed`), every arm says that order at 1.00. Silenced the transformer holds .79 and `Fd` falls to .45, so its weights carry the order and the brain's read stays in use. On told orders of their own `Fd` reads .35 and the transformer .14. +- **The trace is a second carrier of `previous`.** `Fd` holds .69 silenced at every moment and `FdT` .11. + +**`duplicate`** needs no route. The raw key finds the earlier mention on 1.00 of yes rows, and `Fd` reads .92 at a never-taught query where the transformer reads .57 after equal compute. Silenced, `Fd` says one word on every row (.50). With the trace cleared at the answer it falls to .51 and .63, so the signal reads the read and the trace together. + +**Copying in closed loop** reads key new at .04 on `Fd`, a confound and no failure of the read: a token the brain cannot say leaves the next moment hearing another. `Fdg` says it, and reads .99 in closed loop. + +**Suppression**, `A B S GO`, with its answers by name and by place: + +| Task | Arm | Taught | Both new (per founder) | Port, taught | Port, both new (per founder) | Twin's trace | Twin's focus | Rows taught | Sweeps | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | +| `suppression` | `Fd` | 1.00 | .00 .00 .00 | .46 | .17 .88 .00 | .00 | .85 | 542 | .23M | +| | `FdT` | 1.00 | .00 .00 .00 | .93 | 1.00 1.00 .89 | 1.00 | .03 | 2,718 | .84M | +| | `Fd2` | 1.00 | .00 .00 .00 | .73 | .50 1.00 .44 | .01 | .74 | 533 | .23M | +| | `Fdg` | 1.00 | .02 .04 .03 | .51 | .00 .00 .08 | .00 | .24 | 1,218 | .30M | +| | `FdTg` | 1.00 | .00 .50 .11 | .95 | 1.00 .50 1.00 | 1.00 | .01 | 3,653 | .62M | +| | transformer, one pass | 1.00 | .00 .00 .00 | | | | | | | +| | `copy`, one pass | 1.00 | .93 .89 .79 | | | | | | | +| | `copy`, equal compute | 1.00 | .68 .82 .66 | | | | | | | +| `which` | `Fd` | 1.00 | .61 .43 .41 | .47 | .86 .00 .50 | .57 | .53 | 1,322 | .29M | +| | `FdT` | 1.00 | .48 .50 .50 | .25 | .14 .00 .00 | 1.00 | .44 | 1,527 | .33M | +| | transformer, one pass | 1.00 | .78 .91 .86 | | | | | | | +| | transformer, equal compute | 1.00 | .74 .78 .82 | | | | | | | +| | `copy`, one pass | 1.00 | .92 .78 .83 | | | | | | | + +Every arm says taught names at 1.00, by different means. With the trace, a twin's trace in a row's place flips the answer (.00) and the read names the answer on .46 of rows: the cortex holds the sentence. Without it, a twin's focus flips the answer (.03) and the read names both-new names at 1.00, 1.00, .89: the read holds it, at five times the rows taught and 3.7 times the sweeps. + +### The named task + +`ioi`, 4,000 batches, founders 0 to 2 on their own newborns. `copy` is the pointer network below. + +| Arm | Heard together | Never together | Unseen position | Both new: answer | Both new: port | Mass at the duplicate | Seconds | Sweeps | Parameters | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | +| `Fd` | .50 .50 1.00 | .47 .50 .94 | .56 .50 1.00 | .00 .00 .00 | .50 .50 1.00 | .01 .00 .71 | 264 | 3.31M | 18,499 | +| `FdT` | .50 .49 .50 | .54 .48 .50 | .53 .48 .49 | .00 .00 .00 | .51 .49 .50 | .01 .03 .02 | 477 | 6.16M | 18,499 | +| `Fd2` | 1.00 .49 .49 | .84 .50 .46 | 1.00 .48 .50 | .00 .00 .00 | .51 .51 .52 | .01 .01 .01 | 370 | 4.02M | 30,109 | +| `Fdg` | .50 .51 .50 | .49 .48 .50 | .50 .50 .50 | .00 .00 .00 | .51 .47 .50 | .01 .01 .01 | 268 | 4.32M | 18,501 | +| transformer, one pass | 1.00 1.00 1.00 | .65 .69 .83 | 1.00 1.00 1.00 | .00 .00 .00 | | | 21 | | 18,420 | +| transformer, equal compute | 1.00 1.00 1.00 | .70 .88 .69 | 1.00 1.00 1.00 | .00 .00 .00 | | | 187 | | 18,420 | +| LSTM, one pass | 1.00 1.00 1.00 | .40 .38 .21 | 1.00 1.00 1.00 | .00 .00 .00 | | | 9 | | 18,612 | +| LSTM, equal compute | 1.00 1.00 1.00 | .09 .16 .34 | 1.00 1.00 1.00 | .00 .00 .00 | | | 183 | | 18,612 | +| `copy`, one pass | 1.00 1.00 1.00 | .85 .75 .78 | 1.00 1.00 1.00 | .69 .93 .63 | | | 41 | | 18,171 | +| `copy`, equal compute | 1.00 1.00 1.00 | .70 .71 .89 | 1.00 1.00 1.00 | .74 .88 .53 | | | 295 | | 18,171 | + +Lives on twelve brains (founders 0 to 2, each on its own newborn and three more seeds), with the location's cuts. A passing life reads .95 or above on `ioi`; `Both new` counts lives that say both-new names at .80 or above and gives their readings: + +| Life | Passing | Mean, `ioi` | Both new at .80 | Their readings | Rows taught, passing and failing | Sweeps, passing and failing | +| --- | --- | --- | --- | --- | --- | --- | +| `Fd` | 2 of 12 | .58 | 0 | | 1,164, 32,484 | 1.3M, 4.7M | +| `Fd`, without the anchor | 2 of 12 | .58 | 0 | | 1,176, 32,109 | 1.3M, 4.5M | +| `Fd`, the cortex less its mean | 1 of 12 | .54 | 0 | | 1,738, 31,172 | 1.3M, 4.5M | +| `Fd`, address cut | 7 of 12 | .79 | 0 | | 1,233, 32,025 | 1.3M, 4.9M | +| `Fd`, cortex cut | 7 of 12 | .80 | 0 | | 3,444, 31,779 | 1.7M, 4.8M | +| `Fd`, address cut, without the slope | 9 of 12 | .91 | 0 | | 1,461, 25,364 | 1.3M, 4.1M | +| `Fdg` | 0 of 3 | .50 | 0 | | , 32,329 | , 4.3M | +| `Fdg`, cortex cut | 5 of 12 | .69 | 4 | 1.00 1.00 1.00 1.00 | 7,330, 37,997 | 2.0M, 4.6M | +| `FdTg`, cortex cut | 5 of 12 | .71 | 5 | 1.00 1.00 1.00 1.00 .98 | 2,335, 36,542 | 2.0M, 5.4M | +| `Fdg`, address cut, without the slope | 3 of 12 | .59 | 0 | | 1,874, 36,648 | 1.3M, 4.4M | +| `Fdg`, cortex cut, without the slope | 4 of 12 | .65 | 3 | 1.00 1.00 1.00 | 9,955, 36,948 | 2.1M, 4.2M | +| `FdTg`, cortex cut, without the slope | 6 of 12 | .72 | 6 | 1.00 1.00 1.00 1.00 1.00 .93 | 3,457, 39,241 | 2.5M, 5.2M | +| `Fdg`, address cut, drive at answers (oracle) | 6 of 12 | .70 | 6 | 1.00 1.00 .99 .97 .94 .86 | 1,640, 37,743 | 1.3M, 4.7M | +| `Fdg`, address cut, drive at answers (oracle), without the slope | 5 of 12 | .61 | 5 | 1.00 1.00 1.00 .86 .82 | 2,106, 38,326 | 1.4M, 5.0M | + +- **`Fd` reads the named task's circuit on two lives of twelve**, and the cuts raise that. There one head does what the question asks. This is the read's first choice at each moment, taught and both-new names alike, on three lives that form it: + +| Moment | The first name repeats | The second name repeats | Top attention | +| --- | --- | --- | --- | +| `A`, `and`, `B`, and `went`, `out` where heard | `when` | `when` | .47 to .66 | +| the duplicate | the first name's slot | the second name's slot | .63 to .79 | +| `gave` | `and`, or the first name's slot as at the duplicate | `when` | .30 to .45 | +| `a`, `drink` where heard | `and` | `when` | .55 to .67 | +| `to` | `and`, whose next is the second name | `when`, whose next is the first name | .65 to .79 | + + The raw key is the duplicate detector: it finds the earlier mention. The next word's read is the suppression: it steps by the relative map from the earlier mention to the slot whose next holds the other name, and stays there across the words between. The next port is the name mover. It holds with the trace silent and the cortex read as zero, so the read's own place is what crosses the moments, and no lesson falls on any moment but `to`. On the seven lives of `Fd` that form it under the address cut: + +| At test | Taught: answer | Both new: read's port | +| --- | --- | --- | +| The read silenced at every moment (its attention stays) | .09 to .18 | 1.00 | +| Silenced at the answer | .27 to .64 | 1.00 | +| The attended slot erased at the answer | .08 to .21 | .00 to .10 | +| The answer's slot lesioned on both routes | .14 to .30 | .00 | +| A slot neither attended nor holding the answer erased | 1.00 | 1.00 | +| The attended slot erased at the duplicate's moment | .40 to .50 | .40 to .50 | +| Silenced at the duplicate's moment | .99 to 1.00 | 1.00 | +| The focus cleared at the answer | .52 to .67 | .38 to .50 | +| The cortex cleared at the answer | .56 to .86 | .50 to 1.00 | +| The trace cleared at the answer | .72 to 1.00 | 1.00 | +| A twin's focus in the row's place | .00 to .08 | .00 | +| A twin's cortex | 1.00 | 1.00 | +| A twin's trace | .89 to 1.00 | 1.00 | +| A twin's duplicate slot | 1.00 | 1.00 | + +- **The named task, whole, reads at names no lesson used** with no oracle on 4 of 12 lives of `Fdg` under the cortex cut, 5 of 12 of `FdTg` under it and 6 of 12 of `FdTg` under it without the slope, at .93 to 1.00, with twins both right on .96 to 1.00 under the cortex cut. At both-new names those lives read .00 silenced and .00 with the attended slot erased, .50 with the slot erased at the duplicate's moment and .00 with a twin's focus in the row's place, and the control erasure holds their reading. A life that misses reads about .50 on taught names and .00 on new ones. Pairs of one taught and one new name read 1.00 when the new one repeats and .04 to .99 when the taught one does: there the association votes for the taught name. +- **The pointer network reads it at .75 and .71** at one pass and at equal compute, .53 to .93 per founder, with no life at a coin flip on taught names. It has what the brain has not: credit across moments. +- **The plain networks say taught sentences at 1.00 and never-taught names at .00.** On pairs never heard together, the brain's lives that form the circuit read .94 to 1.00 against .65 to .88 for the transformer and .09 to .40 for the LSTM. +- **The parts alone.** The name mover, told by a cue, reads 1.00 on all three founders of `Fd` and names both-new names at 1.00; the failure is the selection, not the moving. + +**How it compares.** A passing life of the brain takes about 1.3M sweeps and 190 seconds on this box, a failing one 4.7M and 310, against 21 seconds for the transformer's pass and 187 at equal compute, at the same parameters. On taught sentences the plain networks read 1.00 on every life where the brain's base composition reads 1.00 on 2 of 12. At names no lesson used they say nothing, since they have no route, so the comparison there is the pointer network: + +| Both new | `Fdg` | `copy`, one pass | `copy`, equal compute | +| --- | --- | --- | --- | +| `previous` | 1.00 | 1.00 | 1.00 | +| `successor` | 1.00 | .56 | .63 | +| `induction` | 1.00 | .35 | .50 | +| `copying`, forced | .99 | .74 | .73 | +| `suppression` | .03 | .87 | .72 | +| `which` | .55 | .84 | .78 | +| `ioi` | .00 on every life; under the cortex cut 1.00 on 4 of 12, .00 on 8 | .75 | .71 | + +The brain's read is the better finder of a token by content: raw keys hold at a token no lesson used, where a learned embedding does not. The pointer network is the better composer: it reads the duplicate at both-new names on every founder, where the brain forms the circuit on some lives and on those says it at 1.00. That reading is the whole of the comparison here: a next-word network trained by gradients across the sentence, against a brain taught by local rules at the answer alone. + +### Where it fails + +Each failure, by the term that fails. + +1. **The read's selection, by the maps' cortex columns.** At `ioi`, lessons name the slots whose next holds the answer. In this frame those are `when` and `and`, the same keys in every sentence. The address map steps on `z = [x, c, 1]`, the cortex holds about twenty times the event's energy (`|c|^2` 18 to 26 against 1 on failing lives), and it stays nearly the same from moment to moment (cosine .86 to .90 between the duplicate's moment and the answer's). So the cortex columns take most of the step, and what they learn at `to` applies at every moment. At the duplicate's moment on a failing life the raw key gives the earlier mention +4.0 and the slot before it gets +4.4 from the address's cortex columns and +3.0 from the position maps', mostly their cortex columns. The first choice leaves the earlier mention (mass .005), and neither the read nor the cortex holds which name repeated (decoder .53). At `to` the two fixed slots read alike, so the answer is a coin flip between the names. Passing lives hold those columns near +1.1. + - The frame without the fixed words (`bare`) reads 1.00 on `Fd` and `FdT` on all three founders, with the read naming both-new names at 1.00. + - Holding the address's cortex columns at zero passes 7 of 12. On the five that miss, the position maps' cortex columns take the pull (+4.5 to +5.8 against +1.2 to +1.9). The cortex read as zero by every map also passes 7 of 12, and there the bias rows take it. Without the slope map the address cut passes 9 of 12. + - Finding the duplicate alone is not enough: an oracle raw read at its moment leaves `ioi` at .51 and .49 where the life fails, since the answer's read keeps its pull. Without the next port the pull goes (mass .97) and so does the name mover (.57). + - The cortex less a running mean passes 1 of 12, so the shared mean alone does not name it. Without the anchor the rate does not move (2 of 12). +2. **The duplicate signal through the frame, as a word.** `dupbit` reads .50 on `Fd`, where the read finds the mention (.68); the bit does not reach `to` for a word to say it. The read's focus carries it where the answer is a slot, and nothing reliably carries it where the answer is a word: under the address cut `dupbit` reads .50, 1.00, .80 and both-new .50, .54, .80. `which` at both-new names reads .41 to .61 on `Fd` where a transformer reads .74 to .91, and `recall_when`, which tells the cortex where a read landed, does not lift it (.61 .43 .50). +3. **The gain's vote.** Where the read selects, the gain's drive decides at never-taught names, since the association reads near zero there. The vote fails in three ways: + - In `suppression`, every slot near the names holds a name on one port and the other on the other, and the duplicate holds two slots' worth: its drive exceeds the answer's (+1.99 against +1.53), and the brain says it on .96 to .98 of rows. `ioi` avoids this: the slot before a name holds a frame word as its value. + - In `bare` the slot the read names holds the answer on its next and, where the duplicate came first, the duplicate as its value: answer and duplicate tie (+2.28 against +2.31) and the brain reads .42 to .51, saying the duplicate on .81 to .83 of those rows. + - At every moment the gain drives the read's token into the motor, and that reaches the cortex the position maps read. On `ioi` with the address cut, the one life of three that forms the circuit then points the read at the duplicate on never-taught names (port .01), and the read returns to 1.00 when it is silent at every moment. With the gains' drive at answer moments alone, an oracle, it reads 1.00, 6 of 12 lives pass and all six say both-new names at .86 to 1.00. The cortex read as zero by the maps gives the same without an oracle. +4. **The life's first lessons.** The gain lowers the rate at which lives form the circuit, and lives that miss are not a little wrong: their association reads +8 to +14 on a taught name and -9 to -16 on the new ones, against about 0 on lives that form it. Holding the gains at zero for the first thousand lessons leaves them near zero for good on lives that form the read, since the surprise rule then teaches no rows. +5. **Two heads.** `Fd2` passes `ioi` on one founder of three by another route: a twin's duplicate slot at `to` flips the answer (.02), and both-new selection stays at chance (.51). The duplicate's slot is read at the answer and the cortex compares by token, which does not reach names no lesson used. Under the address cut with the gain, on the life that selects both-new names (port 1.00), the second head's port names the duplicate on about half the rows, the vote ties (+1.83 against +1.80) and the answer reads .57. + +**A cost of the cuts.** A read on a read goes through the cortex. `twohop`'s second hop at taught tokens reads 1.00, .99, 1.00 on `Fd`, .40, .42, .50 under the address cut and .27, .42, .44 under the cortex cut, with rows taught up from about 2,800 to about 14,300; the first hop holds at 1.00. So the columns that lose the duplicate are the path of a read on a read, and neither cut can be delivered as it stands. On the single functions the address cut costs nothing above noise: `previous` 1.00, `successor` 1.00 and .99, `induction` 1.00, forced `copying` 1.00, `duplicate` .99 and .91. + +**A read on a read at a never-taught key is lost on every arm.** `Fd`'s second hop reads .05 to .12 with the read's first choice at .00 to .02, and `Fdg` .01 to .03: the address reaches a slot through columns taught on taught tokens only. Where the key is taught and the answer new, `Fdg` reads .67 to .79 with the read at 1.00. + +## The cortex at the event's weight and a value path released late + +Two library opt-ins come from the first failure term and the gain's. `recall_cortex_norm=True` gives +every map of the read the cortex of the moment before at unit norm, so the fixed slots a lesson at `to` +names take no step at every moment. `recall_gain_after=n` with `recall_gain_next=g` holds +the learned gains at zero through the first `n` lessons and starts the next port's gain at `g` then, +the recall port's at zero ([the memory guide](../../docs/memory.md#the-value-path-released-once-the-read-has-formed-opt-in)). +On this chamber the harness's emulation of the second (`gain_late` with `gain_late_value`) and the +option read the same panels and work to the bit (`ioi` and `twohop`, two founders each). + +The readings below are declared: a protocol written before any life, twelve fresh founders, each on +its own newborn, one machine (a 32-vCPU box), `causal` readings off. A life passes the gate where +never-taught names (`ioi_new`) are said at .80 or above and the taught second hop (`twohop`, +`hop2_taught`) reads .90 or above on the same founder. + +| Reading | Arm | Taught `ioi` at .95 | The read's port at never-taught names, .80 | Never-taught names said, .80 | Taught second hop, .90 | Lives passing | +| --- | --- | --- | --- | --- | --- | --- | +| First (founders 2691001 to 2691012) | `Fd` | 2 | 2 | 0 | 12 | 0 | +| | `Fd`, cortex at unit norm | 7 | 7 | 0 | 12 | 0 | +| | `Fdg`, cortex at unit norm, both gains released at 30 after 1,000 lessons | 6 | 6 | 6 | 0 (.00 to .05) | 0 | +| Second (founders 2691013 to 2691024) | `Fd`, cortex at unit norm | 7 | 7 | 0 | 12 | 0 | +| | `Fdg`, cortex at unit norm, `recall_gain_after=1000`, `recall_gain_next=10` | 7 | 7 | 7 (.97 to 1.00) | 12 (.99 to 1.00) | 7 | + +The bar of 8 of 12 is missed by one life. What each reading shows: + +- **The cortex at unit norm moves the circuit from 2 to 7 lives of 12** and keeps the taught second hop + on every life (.98 to 1.00). On the lives that form it the read's port names never-taught names at + 1.00; without a value path nothing says them. +- **A gain on the recall port breaks the second hop.** In `twohop` the read lands on the slot of the + query, whose value is the query itself: a recall port's gain says it, and with both gains at 30 the + second hop reads .00 to .05 on all twelve lives and the first .52 to .86. +- **The gain from birth stops the read from forming** (development, founders 0 to 5: the next port's + gain at 10 from birth forms the circuit on 1 or 2 lives of 6, and the second hop reads .00 to .45). A + gain that answers early takes the rows the read's lesson needs. Released after 1,000 lessons it + forms on 6 of 6 there, and on 7 of 12 declared. +- **The five lives that miss fail at the read's selection.** On three of them, at the duplicate's moment + the position maps give the slot before the earlier mention +1.0 to +1.7 against the content's lead of + 1.4 to 1.7 for the mention itself, so the read holds the earlier mention at .27 to .47 where passing + lives hold .53 to .86 (their content's lead 2.1 to 3.3). On the other two the read at `to` lands on the + names' own slots, the content (+2.5 to +3.0) and the position maps (+3.8 to +4.0) both favouring them + over the slot before. A passing life settles an act in 32 to 43 sweeps, a failing one in 39 to 67. + +On the library of this release the chamber reads the earlier rows above as they stand (development, +founders 0 to 2): `ioi` on `Fd` .50, .51, 1.00 on taught sentences; the second hop 1.00, 1.00, .99; the +single functions' cells within .05 of the table above on `Fd` and `Fdg` (copying read in closed +loop). + +## What this does and does not establish + +The chamber reads one composition and its options on one finite language of twenty words. A sentence said right at names no lesson used shows that the life composed the parts on this frame, on its development founders; it shows nothing about open vocabularies, other frames or fresh founders, and the cuts and oracles above are diagnostics that name a term. None is a candidate as it stands: the cortex cut costs the second hop, and the oracle knows which moment is an answer. The rates are twelve lives on one machine and carry the width of twelve bimodal lives. What decides a life's side is not established: replicas keep it and brain seeds change it, and the first lessons differ by the end (rows taught, the cortex's energy) without naming a cause. With tokens declared, `FdTg` reads `duplicate` and `which` at .50 where `Fdg` reads .95 to 1.00 and `FdT` without tokens reads 1.00; that is read and not located. The controls are next-word learners at the brain's parameter count: a control that says a panel right shows that the information suffices at that size, and `tied`, `pointer` and `copy` are additions of this chamber, reported apart, that compare routes and restrict nothing of the brain's. Seconds on the shared box compare work within a row only. + +## Run + +From the repository root, with one BLAS thread per process and `torch` for the controls: + +```sh +PYTHONPATH=src python benchmarks/heads/heads.py --task ioi --arms Fd FdT Fd2 --founders 0 1 2 \ + --out results/ioi.jsonl --workers 9 +PYTHONPATH=src:benchmarks/heads python benchmarks/heads/nets.py --task ioi --models lstm transformer \ + --modes online --founders 0 1 2 --out results/nets.jsonl --workers 3 +python benchmarks/heads/heads.py --table results/ioi.jsonl +``` diff --git a/benchmarks/heads/heads.py b/benchmarks/heads/heads.py new file mode 100644 index 00000000..c49c7670 --- /dev/null +++ b/benchmarks/heads/heads.py @@ -0,0 +1,1562 @@ +"""Composed attention functions (cadence#269): duplicates, suppression and indirect objects. + +Interpretability work on transformers finds functions composed across heads: one head detects a +duplicate, another suppresses attention to it, another moves the remaining name to the output. +This chamber reads that composition in one brain, and each part alone first, at tokens no lesson +used. The named task: ``when mary and john went out john gave a drink to`` answers ``mary``. + +Twenty words, each an input and an action of the brain: twelve names and the frame ``when and +went out gave a drink to``. Per founder (``Split``) eight names are taught and four are never +heard and never answered in training, and four pairs of taught names are never heard together. +The single functions (``TASKS``) use the same twelve tokens, the same split and a cue flag of +their own, each in a life of its own: + +- ``previous``: a stream, its last token flagged ``ASK``; the token before it. +- ``duplicate``: distinct tokens, then a token flagged ``ASK``; ``yes`` if it was told, else ``no``. +- ``successor``: distinct tokens told once, then one flagged ``ASK``; the token told after it. +- ``successor_fixed``: the same, with one fixed order of the taught tokens in training. +- ``induction``: ``... A B ... A``, the query unflagged; ``B``. +- ``twohop``: pairs told, then ``a``, then ``THINK``; ``b`` and ``c``: a read on what a read found. +- ``copying``: ``START``, distinct tokens, ``GO``, each answer heard (closed loop at test); each + token in turn. +- ``suppression``: ``A B S GO``, ``S`` one of the two; the one not repeated. +- ``which``: the same; ``first`` or ``second``, the place of the one not repeated. +- ``ioi``: ``when A and B (went (out)) S gave (a drink) to``; the one not repeated. +- ``bare``: the same with no ``when`` and no ``and``: no fixed word before the names. +- ``mover``: the frame without ``S``, ``to`` flagged ``FIRST`` or ``SECOND``; the name at that + place. +- ``dupbit``: the sentence of ``ioi``, ``to`` flagged ``WHICH``; ``first`` or ``second``, where the + duplicate's earlier mention stood. + +``mover`` and ``dupbit`` are the indirect-object task's two parts inside its own frame. Lessons +fall on answer moments only, before the act on the same observation; every other moment is heard +and acted on greedily. A panel's rows carry conditions: ``taught`` (every token taught), +``key_new`` (the token the read is addressed by is never-taught, the answer taught), +``answer_new`` and ``new`` (both). Beside the answer each row reads the read's own first choice +(``first``: the slot head 0 attends most holds the right token as its value or its next), so a read +that finds a token the motor cannot say is told apart from a read that fails. + +Causality (``CAUSAL``): the read silenced at every moment or at one, the attended slot erased at +the answer or at the duplicate, the answer's own slot lesioned on both routes, a slot neither +attended nor holding the answer erased (a control), the carriers across moments cleared at the +answer (the working trace, the cortex the address reads, the previous read's focus), and on twin +panels each put in a row's place from its twin. ``Support`` names a term by what restores the +composition when the harness gives or takes it from a life; it is no option of the brain. + + python benchmarks/heads/heads.py --task ioi --arms Fd Fd2 FdT --founders 0 1 2 \\ + --out DIR/ioi.jsonl --workers 9 + python benchmarks/heads/heads.py --table DIR/ioi.jsonl +""" + +from __future__ import annotations + +import argparse +import copy +import importlib.util +import json +import sys +import time +import warnings +from dataclasses import dataclass, field, replace +from multiprocessing import get_context +from pathlib import Path +from typing import Any + +import numpy as np + +HERE = Path(__file__).resolve().parent + + +def _module(name: str, path: Path) -> Any: + if name in sys.modules: + return sys.modules[name] + spec = importlib.util.spec_from_file_location(name, path) + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + sys.modules[name] = module + spec.loader.exec_module(module) + return module + + +gr = _module("grammar", HERE.parent / "grammar" / "grammar.py") + +# ----------------------------------------------------------------------------- the language + +BATCH = 16 # the grammar chamber's +FOUNDERS = (0, 1, 2) +SEEDS = {"train": 11, "test": 33, "split": 7} # the grammar chamber's +NAMES = ("mary", "john", "anna", "tom", "lisa", "paul", "sara", "mark", "nina", "owen", "ruth", + "sean") +FRAME = ("when", "and", "went", "out", "gave", "a", "drink", "to") +WORDS = NAMES + FRAME +TOKENS = len(NAMES) +NEW_TOKENS, HELD_PAIRS = 4, 4 +WHEN, AND, WENT, OUT, GAVE, A, DRINK, TO = range(TOKENS, TOKENS + len(FRAME)) +PRE = ((), (WENT,), (WENT, OUT)) # the words between the pair and the duplicate +POST = {1: (GAVE,), 3: (GAVE, A, DRINK)} # the words between the duplicate and ``to`` +TRAIN_SHAPES = ((2, 3), (1, 3), (0, 3), (2, 1), (0, 1)) +HELD_SHAPE = (1, 1) # a duplicate position no lesson reached +KINDS = ("taught", "key_new", "answer_new", "new") +# one pass at the grammar chamber's batches +LESSONS = {"ioi": 4000, "bare": 4000, "mover": 4000, "dupbit": 4000} +SINGLE_LESSONS = 1500 # the sequence battery's + + +def onehot(index: Any, width: int) -> np.ndarray: + index = np.asarray(index) + out = np.zeros((len(index), width)) + known = index >= 0 # a moment that hears nothing is silent + out[np.flatnonzero(known), index[known]] = 1.0 + return out + + +@dataclass(frozen=True) +class Split: + """What one founder's training never shows: four tokens and four pairs of taught tokens.""" + + taught: tuple[int, ...] + new: tuple[int, ...] + pairs: tuple[tuple[int, int], ...] + + @classmethod + def of(cls, founder: int) -> Split: + rng = np.random.default_rng(np.random.SeedSequence([int(founder), SEEDS["split"]])) + order = [int(i) for i in rng.permutation(TOKENS)] + new, taught = sorted(order[:NEW_TOKENS]), sorted(order[NEW_TOKENS:]) + every = [(a, b) for i, a in enumerate(taught) for b in taught[i + 1:]] + chosen = rng.permutation(len(every))[:HELD_PAIRS] + return cls(tuple(taught), tuple(new), tuple(every[int(i)] for i in chosen)) + + def held(self, a: int, b: int) -> bool: + return (min(a, b), max(a, b)) in self.pairs + + +@dataclass +class Moment: + """One moment of one batch. ``label`` is taught and scored when given; ``source`` the token a + right read delivers (the label by default, -1 where no slot holds one); ``avoid`` the token a + suppressing read leaves alone; ``slots`` the moment each named token was heard at; ``mark`` + names a moment other than an answer (``dup``: the duplicate's); ``feedback``: the previous + answer is heard in closed loop; ``tags``: each row's conditions.""" + + x: np.ndarray + label: np.ndarray | None = None + source: np.ndarray | None = None + avoid: np.ndarray | None = None + slots: dict[str, np.ndarray] = field(default_factory=dict) + mark: str = "" + feedback: bool = False + tags: list[tuple[str, ...]] | None = None + + +def _class_of(key: int, answer: int, new: set[int]) -> str: + return KINDS[int(key in new) + 2 * int(answer in new)] + + +class Task: + """A family of batches on one founder's split: ``train(rng)`` a training batch, ``panels(n)`` + the frozen test panels of about ``n`` batches each.""" + + name = "" + flags: tuple[str, ...] = () # cue flags, input coordinates after the words + extra_actions: tuple[str, ...] = () # actions that are no token + words = TOKENS + twins = False # every panel's rows are pairs differing in what the task asks + + def __init__(self, founder: int) -> None: + self.founder = int(founder) + self.split = Split.of(founder) + self.taught = np.array(self.split.taught) + self.new = np.array(self.split.new) + self.new_set = set(self.split.new) + self.inputs = self.words + len(self.flags) + self.actions = self.words + len(self.extra_actions) + self.flag = {name: self.words + i for i, name in enumerate(self.flags)} + + def tokens(self) -> list[int]: + """Per action the observation coordinate of the same token (``recall_tokens``).""" + return [*range(self.words), *([-1] * len(self.extra_actions))] + + def pools(self, kind: str) -> tuple[np.ndarray, np.ndarray]: + """The tokens of a row's key and of its answer, by the row's kind.""" + return (self.new if kind in ("key_new", "new") else self.taught, + self.new if kind in ("answer_new", "new") else self.taught) + + def x(self, tokens: Any, *flags: str) -> np.ndarray: + out = onehot(tokens, self.inputs) + for name in flags: + out[:, self.flag[name]] = 1.0 + return out + + def test_rng(self) -> np.random.Generator: + return np.random.default_rng(np.random.SeedSequence([self.founder, SEEDS["test"]])) + + def kinds_of_a_batch(self) -> list[str]: + return [KINDS[i % len(KINDS)] for i in range(BATCH)] + + def train(self, rng: np.random.Generator) -> list[Moment]: + return self.batch(rng, ["taught"] * BATCH) + + def batch(self, rng: np.random.Generator, kinds: list[str]) -> list[Moment]: + raise NotImplementedError + + def panels(self, batches: int) -> dict[str, list[list[Moment]]]: + rng = self.test_rng() + return {"test": [self.batch(rng, self.kinds_of_a_batch()) for _ in range(batches)]} + + +def _pick(rng: np.random.Generator, pool: Any, avoid: tuple[int, ...] = ()) -> int: + choices = [int(t) for t in pool if int(t) not in avoid] + return choices[int(rng.integers(len(choices)))] + + +class Previous(Task): + """A stream of tokens, its last flagged ``ASK``: the token before it, by position against + content (earlier tokens may be the present one).""" + + name = "previous" + flags = ("ASK",) + lengths = (3, 8) + + def batch(self, rng: np.random.Generator, kinds: list[str]) -> list[Moment]: + length = int(rng.integers(self.lengths[0], self.lengths[1] + 1)) + rows = [] + for kind in kinds: + keys, answers = self.pools(kind) + key = _pick(rng, keys) + answer = _pick(rng, answers, (key,)) + rows.append([*(_pick(rng, self.taught) for _ in range(length - 2)), answer, key]) + seq = np.array(rows) + moments = [Moment(self.x(seq[:, t])) for t in range(length - 1)] + moments.append(Moment(self.x(seq[:, -1], "ASK"), label=seq[:, -2], + slots={"answer": np.full(BATCH, length - 2)}, + tags=[(k, f"L{length}") for k in kinds])) + return moments + + +class Duplicate(Task): + """Distinct tokens, then one flagged ``ASK``: was it told? The answer names no token; half the + rows of a batch say ``yes``.""" + + name = "duplicate" + flags = ("ASK",) + extra_actions = ("yes", "no") + lengths = (3, 6) + + def kinds_of_a_batch(self) -> list[str]: + return [("taught", "key_new")[i % 2] for i in range(BATCH)] + + def batch(self, rng: np.random.Generator, kinds: list[str]) -> list[Moment]: + length = int(rng.integers(self.lengths[0], self.lengths[1] + 1)) + rows, queries, where, tags = [], [], [], [] + yes = rng.permutation(np.arange(BATCH) % 2 == 0) + # the two kinds of a panel's batch are balanced apart: a yes and a no of each + if len(set(kinds)) > 1: + for kind in set(kinds): + at = [i for i, k in enumerate(kinds) if k == kind] + yes[at] = rng.permutation(np.arange(len(at)) % 2 == 0) + for i, kind in enumerate(kinds): + keys, _ = self.pools(kind) + query = _pick(rng, keys) + others = [int(t) for t in rng.permutation(self.taught) if int(t) != query][:length] + place = int(rng.integers(length)) + if yes[i]: + others[place] = query + rows.append(others) + queries.append(query) + where.append(place if yes[i] else -1) + tags.append((kind, "yes" if yes[i] else "no", f"L{length}")) + seq = np.array(rows) + moments = [Moment(self.x(seq[:, t])) for t in range(length)] + label = np.where(yes, self.words, self.words + 1) + queries_a = np.array(queries) + moments.append(Moment(self.x(queries_a, "ASK"), label=label, + source=np.where(yes, queries_a, -1), + slots={"answer": np.array(where)}, tags=tags)) + return moments + + +class Successor(Task): + """Distinct tokens told once, each episode in its own order, then one flagged ``ASK``: the + token told after it.""" + + name = "successor" + flags = ("ASK",) + lengths = (3, 6) + + def told(self, rng: np.random.Generator, kinds: list[str], length: int) -> list[Moment]: + rows, keys_at, tags = [], [], [] + for kind in kinds: + keys, answers = self.pools(kind) + key = _pick(rng, keys) + answer = _pick(rng, answers, (key,)) + place = int(rng.integers(length - 1)) + rest = [int(t) for t in rng.permutation(self.taught) if int(t) not in (key, answer)] + seq = rest[:length - 2] + seq[place:place] = [key, answer] + rows.append(seq) + keys_at.append(place) + tags.append((kind, f"L{length}")) + return self.moments(np.array(rows), np.array(keys_at), tags) + + def moments(self, seq: np.ndarray, keys_at: np.ndarray, tags: list[tuple[str, ...]] + ) -> list[Moment]: + at = np.arange(len(seq)) + moments = [Moment(self.x(seq[:, t])) for t in range(seq.shape[1])] + moments.append(Moment(self.x(seq[at, keys_at], "ASK"), label=seq[at, keys_at + 1], + slots={"answer": keys_at + 1}, tags=tags)) + return moments + + def batch(self, rng: np.random.Generator, kinds: list[str]) -> list[Moment]: + return self.told(rng, kinds, int(rng.integers(self.lengths[0], self.lengths[1] + 1))) + + +class SuccessorFixed(Successor): + """The successor with one fixed order of the taught tokens told in every training episode, so + that weights alone can answer it; read on that order and, as ``successor``, on told orders of + their own.""" + + name = "successor_fixed" + + def __init__(self, founder: int) -> None: + super().__init__(founder) + rng = np.random.default_rng(np.random.SeedSequence([self.founder, SEEDS["split"], 1])) + self.order = rng.permutation(self.taught) + + def fixed(self, rng: np.random.Generator, tags: list[tuple[str, ...]]) -> list[Moment]: + seq = np.tile(self.order, (BATCH, 1)) + return self.moments(seq, rng.integers(seq.shape[1] - 1, size=BATCH), tags) + + def train(self, rng: np.random.Generator) -> list[Moment]: + return self.fixed(rng, [("fixed",)] * BATCH) + + def panels(self, batches: int) -> dict[str, list[list[Moment]]]: + rng = self.test_rng() + return {"test": [self.batch(rng, self.kinds_of_a_batch()) for _ in range(batches)], + "fixed": [self.fixed(rng, [("fixed",)] * BATCH) for _ in range(batches)]} + + +class Induction(Task): + """``... A B ... A``: the token that followed the query's earlier occurrence, the sequence + battery's task.""" + + name = "induction" + lengths = (5, 10) + gap_min = 2 + + def batch(self, rng: np.random.Generator, kinds: list[str]) -> list[Moment]: + length = int(rng.integers(self.lengths[0], self.lengths[1] + 1)) + rows, places, tags = [], [], [] + for kind in kinds: + keys, answers = self.pools(kind) + key = _pick(rng, keys) + answer = _pick(rng, answers, (key,)) + seq = [_pick(rng, self.taught, (key,)) for _ in range(length - 1)] + place = int(rng.integers(0, length - 1 - self.gap_min)) + seq[place], seq[place + 1] = key, answer + rows.append([*seq, key]) + places.append(place) + tags.append((kind, f"L{length}")) + seq_a = np.array(rows) + at = np.array(places) + 1 + moments = [Moment(self.x(seq_a[:, t])) for t in range(length - 1)] + moments.append(Moment(self.x(seq_a[:, -1]), label=seq_a[np.arange(BATCH), at], + slots={"answer": at}, tags=tags)) + return moments + + +class TwoHops(Task): + """Pairs ``first second`` told as consecutive moments, among them ``a b`` and ``b c``; then + ``a`` answers ``b``, and ``THINK`` answers ``c``: a read addressed by what the read before + found, the sequence battery's task. A row's kind names ``b`` as the second hop's key and ``c`` + as its answer.""" + + name = "twohop" + flags = ("THINK",) + pairs = 4 + + def batch(self, rng: np.random.Generator, kinds: list[str]) -> list[Moment]: + rows, firsts, seconds = [], [], [] + for kind in kinds: + keys, answers = self.pools(kind) + a = _pick(rng, self.taught) + b = _pick(rng, keys, (a,)) + c = _pick(rng, answers, (a, b)) + used = (a, b, c) + others = [int(t) for t in rng.permutation(self.taught) if int(t) not in used] + pairs = [(a, b), (b, c)] + pairs += [(others[2 * i], others[2 * i + 1]) for i in range(self.pairs - 2)] + order = rng.permutation(self.pairs) + firsts.append([pairs[int(i)][0] for i in order]) + seconds.append([pairs[int(i)][1] for i in order]) + rows.append((a, b, c)) + a, b, c = (np.array(col) for col in zip(*rows, strict=True)) + first, second = np.array(firsts), np.array(seconds) + moments = [] + for i in range(self.pairs): + moments.append(Moment(self.x(first[:, i]))) + moments.append(Moment(self.x(second[:, i]))) + for keys, answers, hop, x in ((a, b, "hop1", self.x(a)), + (b, c, "hop2", self.x(np.full(len(a), -1), "THINK"))): + kinds_here = [_class_of(int(k), int(v), self.new_set) + for k, v in zip(keys, answers, strict=True)] + moments.append(Moment(x, label=answers, + tags=[(kind, hop, f"{hop}_{kind}") for kind in kinds_here])) + return moments + + +class Copying(Task): + """``START``, distinct tokens, ``GO``; then each token in turn, the brain hearing the token + just answered (its own answer in closed loop at test). A panel holds rows of taught tokens, + of never-taught tokens and of the two mixed; each answer's condition reads the token heard + and the token asked.""" + + name = "copying" + flags = ("START", "GO") + lengths = (3, 6) + + def kinds_of_a_batch(self) -> list[str]: + return [("taught", "mixed", "new", "mixed")[i % 4] for i in range(BATCH)] + + def batch(self, rng: np.random.Generator, kinds: list[str]) -> list[Moment]: + length = int(rng.integers(self.lengths[0], self.lengths[1] + 1)) + if "new" in kinds: # as many as the never-taught tokens allow + length = min(length, NEW_TOKENS) + rows = [] + for kind in kinds: + pool = (self.taught if kind == "taught" else self.new if kind == "new" + else np.arange(TOKENS)) + rows.append([int(t) for t in rng.permutation(pool)[:length]]) + seq = np.array(rows) + n = len(seq) + moments = [Moment(self.x(np.full(n, -1), "START"))] + moments += [Moment(self.x(seq[:, t])) for t in range(length)] + + def tags(keys: np.ndarray | None, answers: np.ndarray, j: int) -> list[tuple[str, ...]]: + out = [] + for r in range(n): + key = int(answers[r]) if keys is None else int(keys[r]) + kind = _class_of(key, int(answers[r]), self.new_set) + out.append((kind, f"p{j}", kinds[r])) + return out + + moments.append(Moment(self.x(np.full(n, -1), "GO"), label=seq[:, 0], + slots={"answer": np.full(n, 1)}, tags=tags(None, seq[:, 0], 0))) + for j in range(length - 1): + moments.append(Moment(self.x(seq[:, j]), label=seq[:, j + 1], feedback=True, + slots={"answer": np.full(n, j + 2)}, + tags=tags(seq[:, j], seq[:, j + 1], j + 1))) + return moments + + def panels(self, batches: int) -> dict[str, list[list[Moment]]]: + """``test`` in closed loop, where a token the brain cannot say leaves the next moment + hearing another, and ``forced``, the same rows hearing the token told: only there does a + moment's key name what was heard.""" + closed = super().panels(batches)["test"] + forced = [[replace(m, feedback=False) for m in moments] for moments in closed] + return {"test": closed, "forced": forced} + + +def _twins(rng: np.random.Generator, firsts: Any, seconds: Any, count: int, + keep: Any = None) -> list[tuple[int, int, int]]: + """``count`` pairs of names in rows ``(a, b, duplicate)``, each pair twice: its first name + repeated and its second, so that a pair's two rows ask for different names.""" + rows: list[tuple[int, int, int]] = [] + while len(rows) < 2 * count: + a, b = _pick(rng, firsts), _pick(rng, seconds) + if a == b or (keep is not None and not keep(a, b)): + continue + rows += [(a, b, a), (a, b, b)] + return rows + + +class Suppression(Task): + """``A B S GO``, ``S`` one of the two: the one not repeated. The parts of the indirect object + without its frame; ``A B A`` holds an induction's answer at the duplicate's moment and + ``A B B`` does not.""" + + name = "suppression" + flags = ("GO",) + twins = True + + def moments(self, rows: list[tuple[int, int, int]], tags: list[tuple[str, ...]] + ) -> list[Moment]: + a, b, s = (np.array(c) for c in zip(*rows, strict=True)) + n = len(rows) + answer = np.where(s == a, b, a) + earlier = np.where(s == a, 0, 1) + return [ + Moment(self.x(a)), Moment(self.x(b)), + Moment(self.x(s), mark="dup", source=s, slots={"dup": earlier}), + Moment(self.x(np.full(n, -1), "GO"), label=answer, avoid=s, + slots={"answer": np.where(s == a, 1, 0), "dup": earlier, + "dup2": np.full(n, 2)}, tags=tags), + ] + + def tagged(self, rows: list[tuple[int, int, int]]) -> list[tuple[str, ...]]: + out = [] + for a, b, s in rows: + answer = b if s == a else a + order = "abb" if s == b else "bab" # abb: the second name repeats, the first answers + out.append((_class_of(s, answer, self.new_set), order)) + return out + + def pair_rows(self, rng: np.random.Generator, panel: str) -> list[tuple[int, int, int]]: + if panel == "taught": + return _twins(rng, self.taught, self.taught, BATCH // 2) + if panel == "new": + return _twins(rng, self.new, self.new, BATCH // 2) + rows: list[tuple[int, int, int]] = [] # one of each pair never-taught, in either place + for i in range(BATCH // 2): + pools = (self.new, self.taught) if i % 2 else (self.taught, self.new) + rows += _twins(rng, *pools, 1) + return rows + + def train(self, rng: np.random.Generator) -> list[Moment]: + rows = [] + for _ in range(BATCH): + a, b = (int(t) for t in rng.permutation(self.taught)[:2]) + rows.append((a, b, a if rng.random() < 0.5 else b)) + return self.moments(rows, self.tagged(rows)) + + def panels(self, batches: int) -> dict[str, list[list[Moment]]]: + rng = self.test_rng() + out = {} + for panel in ("taught", "half", "new"): + batch = [] + for _ in range(batches): + rows = self.pair_rows(rng, panel) + batch.append(self.moments(rows, self.tagged(rows))) + out[panel] = batch + return out + + +class Which(Suppression): + """``A B S GO``: ``first`` or ``second``, the place of the one not repeated. Suppression whose + answer needs no route: a token never taught as an answer can be the one selected, and a brain + without a route to it can say where it stood. ``key_new`` and ``answer_new`` of a suppression + panel read here apart from "say the taught name".""" + + name = "which" + extra_actions = ("first", "second") + + def moments(self, rows: list[tuple[int, int, int]], tags: list[tuple[str, ...]] + ) -> list[Moment]: + moments = super().moments(rows, tags) + asked = moments[-1] + at_first = asked.slots["answer"] == 0 + moments[-1] = replace(asked, label=np.where(at_first, self.words, self.words + 1), + source=asked.label) + return moments + + +class IndirectObject(Suppression): + """``when A and B (went (out)) S gave (a drink) to``: the one not repeated. A batch is one + shape, its rows pairs of names each asked twice. Panels: ``ioi`` (pairs training shows, its + shapes), ``ioi_pair`` (taught names never heard together), ``ioi_shape`` (a duplicate position + no lesson reached), ``ioi_half`` (one name of each pair never-taught), ``ioi_new`` (both, the + named task).""" + + name = "ioi" + flags = () + words = len(WORDS) + twins = True + + def x(self, tokens: Any, *flags: str) -> np.ndarray: + return onehot(tokens, self.inputs) + + def frame(self, rows: list[tuple[int, int, int]], shape: tuple[int, int], + tags: list[tuple[str, ...]]) -> list[Moment]: + a, b, s = (np.array(c) for c in zip(*rows, strict=True)) + n = len(rows) + pre, post = PRE[shape[0]], POST[shape[1]] + stream = [np.full(n, WHEN), a, np.full(n, AND), b, *(np.full(n, w) for w in pre), s, + *(np.full(n, w) for w in post)] + at = 4 + len(pre) # the duplicate's moment + earlier = np.where(s == a, 1, 3) + moments = [Moment(self.x(w)) for w in stream] + moments[at] = Moment(self.x(s), mark="dup", source=s, slots={"dup": earlier}) + moments.append(self.asked(a, b, s, earlier, at, tags)) + return moments + + def asked(self, a: np.ndarray, b: np.ndarray, s: np.ndarray, earlier: np.ndarray, at: int, + tags: list[tuple[str, ...]]) -> Moment: + return Moment(self.x(np.full(len(a), TO)), label=np.where(s == a, b, a), avoid=s, + slots={"answer": np.where(s == a, 3, 1), "dup": earlier, + "dup2": np.full(len(a), at)}, tags=tags) + + def tagged(self, rows: list[tuple[int, int, int]], shape: tuple[int, int] = (2, 3) + ) -> list[tuple[str, ...]]: + tag = (f"pre{shape[0]}", f"post{shape[1]}") + return [(*row, *tag) for row in super().tagged(rows)] + + def train(self, rng: np.random.Generator) -> list[Moment]: + shape = TRAIN_SHAPES[int(rng.integers(len(TRAIN_SHAPES)))] + rows: list[tuple[int, int, int]] = [] + while len(rows) < BATCH: + a, b = (int(t) for t in rng.permutation(self.taught)[:2]) + if not self.split.held(a, b): + rows.append((a, b, a if rng.random() < 0.5 else b)) + return self.frame(rows, shape, self.tagged(rows, shape)) + + def panels(self, batches: int) -> dict[str, list[list[Moment]]]: + rng = self.test_rng() + held = self.split.held + + def seen(a: int, b: int) -> bool: + return not held(a, b) + + out: dict[str, list[list[Moment]]] = {} + + def built(draw: Any, shapes: tuple[tuple[int, int], ...]) -> list[list[Moment]]: + batch = [] + for i in range(batches): + shape = shapes[i % len(shapes)] + rows = draw() + batch.append(self.frame(rows, shape, self.tagged(rows, shape))) + return batch + + out["ioi"] = built(lambda: _twins(rng, self.taught, self.taught, BATCH // 2, seen), + TRAIN_SHAPES) + out["ioi_pair"] = built(lambda: _twins(rng, self.taught, self.taught, BATCH // 2, held), + TRAIN_SHAPES) + out["ioi_shape"] = built(lambda: _twins(rng, self.taught, self.taught, BATCH // 2, seen), + (HELD_SHAPE,)) + out["ioi_half"] = built(lambda: self.pair_rows(rng, "half"), TRAIN_SHAPES) + out["ioi_new"] = built(lambda: self.pair_rows(rng, "new"), TRAIN_SHAPES) + return out + + +class Bare(IndirectObject): + """``A B (went (out)) S gave (a drink) to``: the indirect object with no fixed word before the + names. Every slot that holds a name, or whose next does, then differs from sentence to sentence + but for the boundary, so the address has no constant key to learn for the names' slots.""" + + name = "bare" + + def frame(self, rows: list[tuple[int, int, int]], shape: tuple[int, int], + tags: list[tuple[str, ...]]) -> list[Moment]: + a, b, s = (np.array(c) for c in zip(*rows, strict=True)) + n = len(rows) + pre, post = PRE[shape[0]], POST[shape[1]] + stream = [a, b, *(np.full(n, w) for w in pre), s, *(np.full(n, w) for w in post)] + at = 2 + len(pre) + earlier = np.where(s == a, 0, 1) + moments = [Moment(self.x(w)) for w in stream] + moments[at] = Moment(self.x(s), mark="dup", source=s, slots={"dup": earlier}) + asked = Moment(self.x(np.full(n, TO)), label=np.where(s == a, b, a), avoid=s, + slots={"answer": np.where(s == a, 1, 0), "dup": earlier, + "dup2": np.full(n, at)}, tags=tags) + moments.append(asked) + return moments + + +class Mover(IndirectObject): + """The frame without the duplicate, ``to`` flagged ``FIRST`` or ``SECOND``: the name at that + place. The name mover alone, its selection told by the cue.""" + + name = "mover" + flags = ("FIRST", "SECOND") + + def x(self, tokens: Any, *flags: str) -> np.ndarray: + return Task.x(self, tokens, *flags) + + def frame(self, rows: list[tuple[int, int, int]], shape: tuple[int, int], + tags: list[tuple[str, ...]]) -> list[Moment]: + a, b, s = (np.array(c) for c in zip(*rows, strict=True)) + n = len(rows) + pre, post = PRE[shape[0]], POST[shape[1]] + stream = [np.full(n, WHEN), a, np.full(n, AND), b, *(np.full(n, w) for w in pre), + *(np.full(n, w) for w in post)] + moments = [Moment(self.x(w)) for w in stream] + first = s == b # the name asked is the one a sentence's duplicate leaves + x = self.x(np.full(n, TO)) + x[first, self.flag["FIRST"]] = 1.0 + x[~first, self.flag["SECOND"]] = 1.0 + moments.append(Moment(x, label=np.where(first, a, b), avoid=np.where(first, b, a), + slots={"answer": np.where(first, 1, 3), + "dup": np.where(first, 3, 1)}, tags=tags)) + return moments + + +class DuplicateBit(IndirectObject): + """The sentence of ``ioi``, ``to`` flagged ``WHICH``: ``first`` or ``second``, where the + duplicate's earlier mention stood. No slot holds the answer, so the reading is the duplicate + signal through the frame, apart from moving a name.""" + + name = "dupbit" + flags = ("WHICH",) + extra_actions = ("first", "second") + + def x(self, tokens: Any, *flags: str) -> np.ndarray: + return Task.x(self, tokens, *flags) + + def asked(self, a: np.ndarray, b: np.ndarray, s: np.ndarray, earlier: np.ndarray, at: int, + tags: list[tuple[str, ...]]) -> Moment: + n = len(a) + label = np.where(s == a, self.words, self.words + 1) + return Moment(self.x(np.full(n, TO), "WHICH"), label=label, source=s, + slots={"answer": earlier, "dup": earlier, "dup2": np.full(n, at)}, tags=tags) + + +TASKS: dict[str, type[Task]] = { + "previous": Previous, "duplicate": Duplicate, "successor": Successor, + "successor_fixed": SuccessorFixed, "induction": Induction, "twohop": TwoHops, + "copying": Copying, + "suppression": Suppression, "which": Which, "ioi": IndirectObject, "bare": Bare, + "mover": Mover, "dupbit": DuplicateBit, +} +FUNCTIONS = ("previous", "duplicate", "successor", "induction", "copying", "suppression", "which") + + +def chance(task: Task) -> dict[str, float]: + """What the panel's conditions read without any function: a uniform guess among the tokens, + and for a two-way task or a pair of names the one a fixed place gives.""" + if task.name in ("duplicate", "dupbit", "suppression", "which", "ioi", "bare", "mover"): + return {"guess": 0.5} + return {"guess": 1.0 / len(task.split.taught)} + + +# ------------------------------------------------------------------------------- the arms + +RECIPE = gr.ARMS["recipe"] # the sequence battery's recipe_relative at the normalized point +FD = {**RECIPE, "teaching": "deep"} # the base composition +ARMS: dict[str, dict[str, Any]] = { + "Fd": FD, + "Fd2": {**FD, "recall_heads": 2}, + "FdT": FD, # the trace silent at every lesson and act + "Fdg": {**FD, "recall_gain": 30.0}, # recall_tokens set per task + "Fd2g": {**FD, "recall_heads": 2, "recall_gain": 30.0}, + "FdTg": {**FD, "recall_gain": 30.0}, + "frozen": FD, # the newborn, never taught +} +CUTS = {"FdT": ["trace"], "FdTg": ["trace"]} +MODULE = 64 + + +def make_brain(task: Task, arm: str, founder: int, settings: dict[str, Any] | None = None, + module: int = MODULE, replica: int = 0) -> Any: + import cadence as cd + + gx = _module("gradients", HERE.parent / "grammar" / "gradients.py") + options = {**ARMS[arm], **(settings or {})} + if options.get("recall_tokens") == "task": # the task's tokens declared without a gain + options["recall_tokens"] = task.tokens() + if options.get("recall_gain"): + options.setdefault("recall_tokens", task.tokens()) + options.setdefault("backend", "cpu") # one backend for every life, as the grammar chamber's + brain = cd.Brain.compose(task.inputs, task.actions, modules=(module,), seed=int(founder), + **options) + gx.replicate(brain, int(replica)) + if arm in CUTS: + gx.install_cut(brain, CUTS[arm]) + return brain + + +# ------------------------------------------------------------------------------- the life + + +def new_work() -> dict[str, float]: + return {"lessons": 0, "rows": 0, "agreed_rows": 0, "lesson_row_sweeps": 0, "acts": 0, + "act_sweeps": 0, "refused": 0, "exhausted": 0, "free_budget_exhausted": 0} + + +def _act(brain: Any, x: np.ndarray, work: dict[str, float]) -> np.ndarray: + try: + action = np.asarray(brain.act(x, greedy=True)).reshape(len(x), -1)[:, 0] + settled = brain.last_settlement or {} + work["act_sweeps"] += float(settled.get("steps", 0)) + except RuntimeError: + work["exhausted"] += 1 + action = np.full(len(x), -1) + work["acts"] += 1 + return action + + +class Support: + """What the harness gives a life by oracle, to name the term that fails by what restores the + composition when it is given. It is no option of the brain and no candidate for one: it knows + which moment is the duplicate's. ``raw_dup``: the read at the duplicate's moment scores the + event against the keys alone, every map at zero. ``hold_focus``, ``hold_cortex`` and + ``hold_trace``: what the duplicate's moment left of the previous read's focus (aging as the + slot does), of the cortex the address reads and of the working trace stands in place of what + the words after it would leave, up to the answer. ``no_address_cortex``: the address map's + cortex columns stay at zero through the life, every head's (no oracle, a cut). ``no_cortex``: + the cortex of the moment before reads zero at every lesson and act, so that no map reads it + (no oracle, a cut, the grammar harness's). ``gain_at_answer``: the gains' drive reaches the + motor at answer moments alone. ``gain_late``: the gains stay at zero through the life's first + ``LATE`` lessons (a schedule the harness keeps). ``center_cortex``: the maps read the cortex + less its running mean over the life's moments, so that what moments share is not read (no + oracle). ``cortex_unit``: the maps read the cortex of the moment before at unit norm per + stream, the event's energy (no oracle, an emulation of a balanced read). ``cortex_scale`` + (a number, ``Support(names, scale)``): the maps read that cortex times the number (no oracle; + 0 is ``no_cortex``).""" + + NAMES = ("raw_dup", "hold_focus", "hold_cortex", "hold_trace", "no_address_cortex", + "no_cortex", "gain_at_answer", "gain_late", "center_cortex", "cortex_unit") + LATE = 1000 + CENTER_DECAY = 0.999 + MAPS = ("head_maps", "position_maps", "anchor_maps", "relative_maps", "slope_maps") + + def __init__(self, names: Any = (), scale: float | None = None) -> None: + self.names = tuple(names) + self.scale = None if scale is None else float(scale) + # with ``gain_late``: the gains set to this value (one, or one per port) once released + self.late_value: Any = None + if not set(self.names) <= set(self.NAMES): + raise ValueError(f"unknown support: {self.names}") + self.lessons = 0 + self.mean: np.ndarray | None = None + self.reset() + + def reset(self) -> None: + self.focus: np.ndarray | None = None + self.cortex: np.ndarray | None = None + self.trace: np.ndarray | None = None + + def before(self, brain: Any, teaching: bool = False) -> None: + """Before the moment's lesson and act: what is held stands in place. A running mean moves + on moments of a life's lessons, and a reading leaves it as it is.""" + recall = brain.recall + if "no_cortex" in self.names: + recall.previous = np.zeros_like(recall.previous) + if "cortex_unit" in self.names: + raw = np.atleast_2d(np.asarray(recall.previous, dtype=float)) + norm = np.linalg.norm(raw, axis=1, keepdims=True) + recall.previous = np.where(norm > 0, raw / np.where(norm > 0, norm, 1.0), raw) + if self.scale is not None: + recall.previous = self.scale * np.asarray(recall.previous) + if "center_cortex" in self.names: + raw = np.atleast_2d(np.asarray(recall.previous)) + active = raw.any(axis=1) + if active.any(): + if teaching: + seen = raw[active].mean(axis=0) + self.mean = (seen if self.mean is None else self.CENTER_DECAY * self.mean + + (1.0 - self.CENTER_DECAY) * seen) + if self.mean is not None: + recall.previous = np.where(active[:, None], raw - self.mean, raw) + if self.focus is not None: + recall.focus = self.focus.copy() + if self.cortex is not None: + recall.previous = self.cortex.copy() + if self.trace is not None: + brain.working_memory.trace = self.trace.copy() + + def act(self, brain: Any, m: Moment, x: np.ndarray, work: dict[str, float]) -> np.ndarray: + recall = brain.recall + gains = None + if "gain_at_answer" in self.names and m.label is None: + gains = recall.gains + recall.gains = np.zeros_like(gains) + try: + if m.mark != "dup" or "raw_dup" not in self.names: + return _act(brain, x, work) + saved = {name: getattr(recall, name) for name in self.MAPS} + for name, maps in saved.items(): + setattr(recall, name, [tuple(np.zeros_like(a) for a in head) for head in maps]) + try: + return _act(brain, x, work) + finally: + for name, maps in saved.items(): + setattr(recall, name, maps) + finally: + if gains is not None: + recall.gains = gains + + def taught(self, brain: Any) -> None: + """After a lesson: what a cut keeps at zero.""" + if "no_address_cortex" in self.names: + recall = brain.recall + for h in range(recall.heads): + w_x, w_c, b = recall.head_map(h) + recall._set_map(h, w_x, np.zeros_like(w_c), b) + self.lessons += 1 + if "gain_late" in self.names and self.lessons <= self.LATE: + brain.recall.gains = np.zeros_like(brain.recall.gains) + if self.late_value is not None and self.lessons == self.LATE: + brain.recall.gains = np.broadcast_to( + np.asarray(self.late_value, dtype=float), np.shape(brain.recall.gains)).copy() + + def after(self, brain: Any, m: Moment) -> None: + """After the moment's act: the duplicate's moment gives what is held, and a held focus + ages with the slot it points to.""" + if m.mark == "dup": + if "hold_focus" in self.names: + self.focus = np.array(brain.recall.focus) + if "hold_cortex" in self.names: + self.cortex = np.array(brain.recall.previous) + if "hold_trace" in self.names: + self.trace = np.array(brain.working_memory.trace) + elif self.focus is not None: + self.focus = np.where(self.focus >= 0, self.focus + 1, -1) + + +CORTEX_MAPS = ("head_maps", "position_maps", "anchor_maps", "relative_maps", "slope_maps", + "grid_maps") + + +def _cortex_columns(recall: Any) -> dict[str, list[np.ndarray]]: + """Every map's cortex columns, per kind and head (``w_c`` of ``(w_x, w_c, b)``).""" + return {name: [np.array(head[1]) for head in getattr(recall, name, None) or []] + for name in CORTEX_MAPS} + + +def _scale_cortex_step(recall: Any, before: dict[str, list[np.ndarray]], rate: float) -> None: + """A lesson's step on the maps' cortex columns, times ``rate``: an emulation of a slower + rate on what every moment shares (``cortex_rate`` of a life's spec).""" + for name, columns in before.items(): + maps = getattr(recall, name, None) or [] + setattr(recall, name, [ + (head[0], old + rate * (np.asarray(head[1]) - old), head[2]) + for head, old in zip(maps, columns, strict=True)]) + + +def teach(brain: Any, moments: list[Moment], work: dict[str, float], + support: Support | None = None, cortex_rate: float | None = None) -> None: + """One training batch on fresh streams: a lesson before the act at every answer moment. + ``cortex_rate`` scales each lesson's step on the maps' cortex columns.""" + support = support or Support() + brain.reset() + support.reset() + for m in moments: + support.before(brain, teaching=True) + if m.label is not None: + before = None if cortex_rate is None else _cortex_columns(brain.recall) + try: + report = brain.lesson(m.x, m.label) + work["lessons"] += 1 + work["rows"] += report.get("attempted_presentations", 0.0) + work["agreed_rows"] += report.get("agreed_presentations", 0.0) + work["lesson_row_sweeps"] += report.get("total_row_sweeps", 0.0) + work["free_budget_exhausted"] += report.get("free_budget_exhausted", 0.0) + except RuntimeError: + work["refused"] += 1 + if before is not None: + _scale_cortex_step(brain.recall, before, float(cortex_rate)) + support.taught(brain) + support.act(brain, m, m.x, work) + support.after(brain, m) + + +# ----------------------------------------------------------------------------- the reading + + +class Tally: + """The answers and the reads of a panel, per condition.""" + + def __init__(self) -> None: + self.cells: dict[str, dict[str, Any]] = {} + self.hits: list[np.ndarray] = [] + self.said: list[np.ndarray] = [] + + def cell(self, name: str) -> dict[str, Any]: + return self.cells.setdefault(name, {"n": 0, "answer": 0.0, "says_avoid": 0.0, "pair": 0.0, + "reads": 0, "first": 0.0, "first_any": 0.0, + "first_avoid": 0.0, "mass": 0.0, "mass_avoid": 0.0, + "right_wrong": 0.0, "ports": {}}) + + def add(self, m: Moment, action: np.ndarray, read: dict[str, Any] | None) -> None: + assert m.label is not None + n = len(action) + hit = action == m.label + self.hits.append(hit) + self.said.append(action) + for r in range(n): + for name in ("all", *(m.tags[r] if m.tags else ())): + c = self.cell(name) + c["n"] += 1 + c["answer"] += float(hit[r]) + if m.avoid is not None: + c["says_avoid"] += float(action[r] == m.avoid[r]) + c["pair"] += float(hit[r] or action[r] == m.avoid[r]) + if read is not None and read["held"][r]: + c["reads"] += 1 + c["first"] += float(read["first"][r]) + c["first_any"] += float(read["first_any"][r]) + c["right_wrong"] += float(read["first"][r] and not hit[r]) + c["mass"] += float(read["mass"][r]) + if m.avoid is not None: + c["first_avoid"] += float(read["first_avoid"][r]) + c["mass_avoid"] += float(read["mass_avoid"][r]) + for port, named in read["ports"].items(): + c["ports"][port] = c["ports"].get(port, 0.0) + float(named[r]) + + def summary(self, twins: bool = False) -> dict[str, Any]: + out: dict[str, Any] = {"conditions": {}} + for name, c in sorted(self.cells.items()): + n = max(c["n"], 1) + cell = {"n": c["n"], "answer": round(c["answer"] / n, 4)} + if c["says_avoid"] or c["pair"]: + cell["says_avoid"] = round(c["says_avoid"] / n, 4) + cell["pair"] = round(c["pair"] / n, 4) + if c["reads"]: + reads = c["reads"] + for key in ("first", "first_any", "first_avoid", "mass", "mass_avoid", + "right_wrong"): + cell[key] = round(c[key] / reads, 4) + cell["reads"] = reads + ports = {k: round(v / reads, 4) for k, v in c["ports"].items()} + cell["ports"] = ports + if ports: + cell["port_best"] = max(ports.values()) + out["conditions"][name] = cell + if twins and self.hits: + hits = np.concatenate(self.hits) + said = np.concatenate(self.said) + out["both"] = round(float(np.mean(hits[0::2] & hits[1::2])), 4) + out["differ"] = round(float(np.mean(said[0::2] != said[1::2])), 4) + return out + + +def _delivered(rows: np.ndarray, top: np.ndarray, bank: np.ndarray) -> np.ndarray: + """The token a slot holds on a port (its largest coordinate), -1 where it holds none.""" + held = bank[rows, top] + return np.where(held.max(axis=1) > 0, held.argmax(axis=1), -1) + + +def read_of(heads: tuple[np.ndarray | None, ...], values: np.ndarray, nexts: np.ndarray, + source: np.ndarray, avoid: np.ndarray | None) -> dict[str, Any] | None: + """What the read found at a moment, before that moment's write: per row whether head 0's first + choice holds the right token on its value or its next (``first``), whether any head's does, + whether it holds the token to avoid, the attention on slots that hold each, and per port + whether the token most driven is the right one.""" + p0 = heads[0] if heads else None + if p0 is None or p0.shape[1] != values.shape[1]: + return None + rows = np.arange(len(p0)) + held = source >= 0 + token = np.where(held, source, 0) + out: dict[str, Any] = {"held": held, "ports": {}} + first_any = np.zeros(len(p0), dtype=bool) + mass = np.zeros(len(p0)) + for h, p in enumerate(heads): + if p is None: + continue + top = np.argmax(p, axis=1) + on_value, on_next = _delivered(rows, top, values), _delivered(rows, top, nexts) + right = (on_value == source) | (on_next == source) + first_any |= right + slots_hold = (values[rows, :, token] > 0) | (nexts[rows, :, token] > 0) + mass = np.maximum(mass, np.sum(p * slots_hold, axis=1)) + if h == 0: + out["first"] = right + if avoid is not None: + out["first_avoid"] = (on_value == avoid) | (on_next == avoid) + hold_avoid = (values[rows, :, avoid] > 0) | (nexts[rows, :, avoid] > 0) + out["mass_avoid"] = np.sum(p * hold_avoid, axis=1) + out["ports"][f"recall{h}"] = np.argmax(np.einsum("bs,bsw->bw", p, values), axis=1) == source + out["ports"][f"next{h}"] = np.argmax(np.einsum("bs,bsw->bw", p, nexts), axis=1) == source + out["first_any"] = first_any + out["mass"] = mass + return out + + +def motor_inputs(brain: Any, tokens: dict[str, np.ndarray], actions: int) -> dict[str, Any]: + """What drives each named token's motor neuron at the act just settled, per row: the + association's synapses, the gains' drive and everything else (the motor's own synapses, the + bias). Read after the act; a token below zero reads nothing.""" + graph = brain.connectome + pre, post = np.asarray(graph.pre), np.asarray(graph.post) + motor = np.asarray(brain.motor_index)[:actions] + association = np.asarray(graph.populations["association"]) + weights = np.asarray(brain.brain.weights) + activation = np.atleast_2d(np.asarray(brain.basal_ganglia.state.activation)) + drive = np.atleast_2d(np.asarray(brain.basal_ganglia._drive)) + bias = np.asarray(brain.brain.bias) + from_association = np.isin(pre, association) + rows = np.arange(len(activation)) + out: dict[str, Any] = {} + for name, token in tokens.items(): + known = token >= 0 + neuron = motor[np.where(known, token, 0)] + cortex, other = np.zeros(len(rows)), np.zeros(len(rows)) + for r in rows[known]: + into = post == neuron[r] + parts = weights[into] * activation[r, pre[into]] + cortex[r] = parts[from_association[into]].sum() + other[r] = parts[~from_association[into]].sum() + bias[neuron[r]] + out[name] = {"association": cortex, "gain": drive[rows, neuron], "other": other, + "known": known} + return out + + +def _attended(brain: Any, x: np.ndarray) -> list[np.ndarray]: + """Every head's attention at the moment ``x``, before its settle. ``stimulus`` addresses the + moment and ``act`` addresses it again, so the act that follows is unchanged (a test reads + it).""" + brain.stimulus(x) + heads = brain.recall.attended_heads + return [] if heads is None else [p for p in heads if p is not None] + + +def logit_terms(recall: Any, m: Moment, t: int) -> dict[str, dict[str, np.ndarray]]: + """Head 0's logits at a moment by what they come from, on the slots a suppressing read tells + apart: the slot of each named token and the slot before it (whose next holds it). Read after + ``stimulus``, before the act. ``key`` is the event heard against a slot's key, ``event``, + ``cortex`` and ``bias`` the address map's rows against it, and ``position_*`` every position + map's same rows at the slot's buckets; with ``content`` and ``position`` their sums.""" + moment = recall.last + rows = np.arange(len(m.x)) + x, c = moment.inputs, moment.cortex + w_x, w_c, b = recall.head_map(0) + temperature = recall.temperature + keys = recall.keys + parts: dict[str, np.ndarray] = { + "key": np.einsum("bk,bmk->bm", moment.key, keys) / temperature, + "event": np.einsum("bk,bmk->bm", x @ w_x, keys) / temperature, + "cortex": np.einsum("bk,bmk->bm", c @ w_c, keys) / temperature, + "bias": np.einsum("k,bmk->bm", b, keys) / temperature, + } + position = {"event": 0.0, "cortex": 0.0, "bias": 0.0} + tables = [(recall.position_maps[0], recall.buckets()[None, :].repeat(len(x), axis=0))] + if recall.anchor_maps: + tables.append((recall.anchor_maps[0], recall.anchored_buckets())) + if recall.relative_maps: + tables.append((recall.relative_maps[0], recall.relative_buckets())) + for maps, buckets in tables: + for name, values in zip(("event", "cortex", "bias"), + (x @ maps[0], c @ maps[1], np.tile(maps[2], (len(x), 1))), + strict=True): + position[name] = position[name] + np.take_along_axis(values, buckets, axis=1) + if recall.slope_maps: + lean = (recall.ages() / recall.SLOPE)[None, :] + maps = recall.slope_maps[0] + for name, value in zip(("event", "cortex", "bias"), + (x @ maps[0], c @ maps[1], np.full(len(x), float(maps[2][0]))), + strict=True): + position[name] = position[name] + np.reshape(value, (len(x), 1)) * lean + parts.update({f"position_{name}": value for name, value in position.items()}) + parts["content"] = parts["key"] + parts["event"] + parts["cortex"] + parts["bias"] + parts["position"] = sum(position.values()) + slots = keys.shape[1] + out: dict[str, dict[str, np.ndarray]] = {term: {} for term in parts} + for name, at in m.slots.items(): + place = slots - (t - at) + for suffix, offset in (("", 0), ("_before", -1)): + where = np.clip(place + offset, 0, slots - 1) + for term, values in parts.items(): + out[term][name + suffix] = values[rows, where] + return out + + +def _lesion(recall: Any, rows: np.ndarray, slots: np.ndarray) -> None: + """Zero the key, the value and the next of a slot per row.""" + keep = (slots >= 0) & (slots < recall.keys.shape[1]) + rows, slots = rows[keep], slots[keep] + for name in ("keys", "values", "nexts"): + bank = np.array(getattr(recall, name), copy=True) + bank[rows, slots] = 0.0 + setattr(recall, name, bank) + + +def intervene(brain: Any, m: Moment, plan: dict[str, tuple[str, ...]], t: int, + rng: np.random.Generator) -> bool: + """A causal intervention before the act of a moment named in ``plan``; returns whether the + read is silent at this act.""" + name = "answer" if m.label is not None else m.mark + recall = brain.recall + if recall is None: + return False + if name in plan.get("trace", ()) and brain.working_memory is not None: + brain.working_memory.trace = np.zeros_like(brain.working_memory.trace) + if name in plan.get("cortex", ()): + recall.previous = np.zeros_like(recall.previous) + if name in plan.get("focus", ()): + recall.focus = np.full_like(recall.focus, -1) + rows = np.arange(len(m.x)) + if name in plan.get("erase", ()) and recall.slots: + for p in _attended(brain, m.x): + _lesion(recall, rows, np.argmax(p, axis=1)) + if name in plan.get("erase_other", ()) and recall.slots and m.source is not None: + heads = _attended(brain, m.x) + tokens = np.where(m.source >= 0, m.source, 0) + slots = np.full(len(rows), -1) + for r in rows: + holds = (recall.values[r, :, tokens[r]] > 0) | (recall.nexts[r, :, tokens[r]] > 0) + tops = {int(np.argmax(p[r])) for p in heads} + free = [s for s in range(recall.keys.shape[1]) if not holds[s] and s not in tops] + if free: + slots[r] = free[int(rng.integers(len(free)))] + _lesion(recall, rows, slots) + if name in plan.get("erase_source", ()) and "answer" in m.slots: + at = recall.keys.shape[1] - (t - m.slots["answer"]) # the slot of the answer's moment + at = np.where(m.slots["answer"] >= 0, at, -1) + _lesion(recall, rows, at) + before = np.array(recall.nexts, copy=True) # the next route: the slot before it + known = at >= 1 + before[rows[known], at[known] - 1] = 0.0 + recall.nexts = before + twin = rows ^ 1 # a twin panel's rows come in pairs + if name in plan.get("swap_cortex", ()): + recall.previous = recall.previous[twin] + if name in plan.get("swap_trace", ()) and brain.working_memory is not None: + brain.working_memory.trace = brain.working_memory.trace[twin] + if name in plan.get("swap_focus", ()): + recall.focus = recall.focus[twin] + if name in plan.get("swap_slot", ()) and "dup2" in m.slots: + at = recall.keys.shape[1] - (t - m.slots["dup2"]) # the duplicate's slot, and the next of + for bank_name, offset in (("keys", 0), ("values", 0), ("nexts", 0), ("nexts", -1)): + bank = np.array(getattr(recall, bank_name), copy=True) # the slot before it + bank[rows, at + offset] = bank[twin, at + offset] + setattr(recall, bank_name, bank) + silent = plan.get("silence", ()) + return name in silent or "all" in silent + + +def read_episode(brain: Any, task: Task, moments: list[Moment], tally: Tally, + work: dict[str, float], plan: dict[str, tuple[str, ...]] | None, + rng: np.random.Generator, dup: dict[str, list[float]] | None, + terms: dict[str, Any] | None = None, support: Support | None = None, + vote: dict[str, Any] | None = None) -> list[np.ndarray]: + """One batch on fresh streams, greedy, in closed loop where the task feeds answers back; returns + each answer moment's hits. ``terms`` keeps the logit's terms per moment and row kind, ``vote`` + what drives the answer's, the avoided token's and the winner's motor neurons per row kind.""" + support = support or Support() + brain.reset() + support.reset() + previous = None + hits = [] + recall = brain.recall + for t, m in enumerate(moments): + x = m.x + if m.feedback and previous is not None: + x = onehot(previous, task.inputs) + m = Moment(x, m.label, m.source, m.avoid, m.slots, m.mark, m.feedback, m.tags) + support.before(brain) + silent = bool(plan) and intervene(brain, m, plan or {}, t, rng) + saved = None + if silent: + saved = recall.amplitude + recall.amplitude = 0.0 + values = nexts = None + scored = m.label is not None or m.mark == "dup" + if scored and recall is not None: + values, nexts = np.array(recall.values, copy=True), np.array(recall.nexts, copy=True) + if plan is None and terms is not None and "dup" in m.slots and recall.slots: + brain.stimulus(x) + found = logit_terms(recall, m, t) + answer = moments[-1].tags or [("all",)] * len(x) + for r, row_tags in enumerate(answer): + kind = terms.setdefault("answer" if m.label is not None else m.mark, {}) + kind = kind.setdefault(row_tags[0], {}) + for term, named in found.items(): + cell = kind.setdefault(term, {}) + for name, at in named.items(): + cell.setdefault(name, []).append(float(at[r])) + action = support.act(brain, m, x, work) + support.after(brain, m) + if saved is not None: + recall.amplitude = saved + if (m.mark == "dup" and plan is None and dup is not None and values is not None + and m.source is not None): + heads = recall.attended_heads + if heads and heads[0] is not None and heads[0].shape[1] == values.shape[1]: + rows = np.arange(len(x)) + top = np.argmax(heads[0], axis=1) + dup["first_value"] += list(_delivered(rows, top, values) == m.source) + dup["first_next"] += list(_delivered(rows, top, nexts) == m.source) + at = values.shape[1] - (t - m.slots["dup"]) + dup["mass"] += list(heads[0][rows, at]) + if m.label is not None: + if plan is None and vote is not None and recall is not None and m.tags: + named = {"answer": m.label, "winner": action} + if m.avoid is not None: + named["avoid"] = m.avoid + found = motor_inputs(brain, named, task.actions) + for r, row_tags in enumerate(m.tags): + kind = vote.setdefault(row_tags[0], {}) + for name, parts in found.items(): + if parts["known"][r]: + cell = kind.setdefault(name, {}) + for part in ("association", "gain", "other"): + cell.setdefault(part, []).append(float(parts[part][r])) + read = None + if values is not None: + source = m.label if m.source is None else m.source + heads = recall.attended_heads + if heads: + read = read_of(tuple(heads), values, nexts, source, m.avoid) + tally.add(m, action, read) + hits.append(action == m.label) + previous = action + return hits + + +def read_panel(brain: Any, task: Task, batches: list[list[Moment]], + plan: dict[str, tuple[str, ...]] | None = None, seed: int = 0, + support: Support | None = None, votes: bool = False) -> dict[str, Any]: + """A panel on a fork of ``brain``: answers and, without an intervention, the read's own; with + ``votes`` what drives the motor neurons.""" + fork = copy.deepcopy(brain) + tally = Tally() + work = new_work() + rng = np.random.default_rng(np.random.SeedSequence([task.founder, 77, seed])) + dup: dict[str, list[float]] = {"first_value": [], "first_next": [], "mass": []} + terms: dict[str, Any] = {} + vote: dict[str, Any] | None = {} if votes else None + exact: list[np.ndarray] = [] + for moments in batches: + hits = read_episode(fork, task, moments, tally, work, plan, rng, dup, terms, support, vote) + exact.append(np.all(np.stack(hits), axis=0)) + out = tally.summary(twins=task.twins) + if vote: + out["vote"] = {kind: {name: {part: round(float(np.mean(v)), 3) + for part, v in cell.items()} + for name, cell in named.items()} + for kind, named in vote.items()} + if plan is None and terms: + out["terms"] = {moment: {kind: {term: {name: round(float(np.mean(v)), 3) + for name, v in cell.items()} + for term, cell in named.items()} + for kind, named in kinds.items()} + for moment, kinds in terms.items()} + out["exact"] = round(float(np.mean(np.concatenate(exact))), 4) + if dup["first_value"]: + out["duplicate_read"] = {k: round(float(np.mean(v)), 4) for k, v in dup.items()} + if work["exhausted"]: + out["exhausted"] = work["exhausted"] + return out + + +# a read or a carrier cut at test; ``erase_other`` is the control, a slot neither read nor holding +# the answer +CAUSAL: dict[str, dict[str, tuple[str, ...]]] = { + "silence": {"silence": ("all",)}, + "silence_answer": {"silence": ("answer",)}, + "erase": {"erase": ("answer",)}, + "erase_source": {"erase_source": ("answer",)}, + "erase_other": {"erase_other": ("answer",)}, + "trace": {"trace": ("answer",)}, + "cortex": {"cortex": ("answer",)}, + "focus": {"focus": ("answer",)}, + "silence_dup": {"silence": ("dup",)}, + "erase_dup": {"erase": ("dup",)}, + # a twin's carrier in the row's place at the answer: an answer that follows the twin's + # (``says_avoid``) names the carrier + "swap_cortex": {"swap_cortex": ("answer",)}, + "swap_trace": {"swap_trace": ("answer",)}, + "swap_focus": {"swap_focus": ("answer",)}, + "swap_slot": {"swap_slot": ("answer",)}, + "swap_all": {"swap_cortex": ("answer",), "swap_trace": ("answer",), + "swap_focus": ("answer",), "swap_slot": ("answer",)}, +} +DUP_CAUSAL = ("silence_dup", "erase_dup") +TWIN_CAUSAL = ("swap_cortex", "swap_trace", "swap_focus", "swap_slot", "swap_all") + + +def _ridge(x: np.ndarray, y: np.ndarray, reads: dict[str, tuple[np.ndarray, np.ndarray]], + penalty: float = 1.0) -> dict[str, float]: + """A ridge decoder of the sign of ``y`` fit on ``x``, read on each of ``reads``.""" + mean = x.mean(axis=0) + a = x - mean + w = np.linalg.solve(a.T @ a + penalty * np.eye(a.shape[1]), a.T @ y) + return {name: round(float(np.mean(((tx - mean) @ w > 0) == (ty > 0))), 4) + for name, (tx, ty) in reads.items()} + + +def _bit_states(brain: Any, task: Task, batches: list[list[Moment]], + support: Support | None = None) -> dict[str, Any]: + """Which name repeated, against what each carrier holds: the cortex and the trace right after + the duplicate's moment and just before the answer's, and the cortex the answer settles to.""" + fork = copy.deepcopy(brain) + support = support or Support() + work = new_work() + states: dict[str, list[np.ndarray]] = {} + bits: list[np.ndarray] = [] + for moments in batches: + fork.reset() + support.reset() + for i, m in enumerate(moments): + support.before(fork) + support.act(fork, m, m.x, work) + support.after(fork, m) + names = ["after_dup"] if m.mark == "dup" else [] + if i == len(moments) - 2: + names.append("before_answer") + if m.label is not None: + names.append("after_answer") + for name in names: + states.setdefault(f"cortex_{name}", []).append(np.array(fork.recall.previous)) + if fork.working_memory is not None: + states.setdefault(f"trace_{name}", []).append( + np.array(fork.working_memory.trace)) + asked = moments[-1] + bits.append(np.where(asked.slots["dup"] == min(asked.slots["dup"]), 1.0, -1.0)) + return {"x": {k: np.concatenate(v) for k, v in states.items()}, "y": np.concatenate(bits)} + + +def decoding(brain: Any, task: Task, batches: int = 40, support: Support | None = None + ) -> dict[str, Any]: + """Whether the cortex and the trace hold which name repeated, by a read-only ridge decoder fit + on taught sentences and read on others: the information a learned readout could use. Above + chance it says the bit is there, and says nothing of whether the brain's own weights read it.""" + tests = task.panels(batches) + fit_on, read_on = tests["ioi"][: batches - 10], tests["ioi"][batches - 10:] + fit = _bit_states(brain, task, fit_on, support) + reads = {"taught": _bit_states(brain, task, read_on, support), + "new": _bit_states(brain, task, tests["ioi_new"][:10], support), + "half": _bit_states(brain, task, tests["ioi_half"][:10], support)} + out = {} + for name, x in fit["x"].items(): + if x.std() == 0: + continue + out[name] = _ridge(x, fit["y"], {k: (v["x"][name], v["y"]) for k, v in reads.items()}) + return out + + +def reading(brain: Any, task: Task, tests: dict[str, list[list[Moment]]], causal: bool, + support: Support | None = None, votes: bool = False) -> dict[str, Any]: + out = {} + for name, batches in tests.items(): + panel = read_panel(brain, task, batches, support=support, votes=votes) + if causal and brain.recall is not None: + has_dup = any(m.mark == "dup" for m in batches[0]) + panel["causal"] = {} + for i, (cut, plan) in enumerate(CAUSAL.items()): + if cut in DUP_CAUSAL and not has_dup: + continue + if cut in TWIN_CAUSAL and not task.twins: + continue + read = read_panel(brain, task, batches, plan, seed=i, support=support) + keep = ("answer", "says_avoid", "first", "first_avoid", "port_best", "ports") + panel["causal"][cut] = { + "conditions": {k: {key: v[key] for key in keep if key in v} + for k, v in read["conditions"].items()}, + **{k: read[k] for k in ("both", "differ", "exact") if k in read}} + out[name] = panel + return out + + +def make_task(name: str, founder: int) -> Task: + return TASKS[name](founder) + + +def run_life(spec: dict[str, Any]) -> dict[str, Any]: + """One life: ``task``, ``arm``, ``founder``, with ``lessons`` (batches), ``panel_batches``, + ``module``, ``init_seed`` (the brain's seed, on the founder's split, lessons and panels), + ``replica``, ``settings`` over the arm's, ``checkpoints`` (batches with a lean reading), + ``causal``, ``decode``, ``vote`` (what drives the motor neurons at the answers) and ``support`` + (the oracle a diagnostic life is given), ``cortex_rate`` (each lesson's step on the maps' cortex + columns scaled, an emulation), ``gain_start`` (every port's gain at birth, an emulation).""" + warnings.simplefilter("ignore") + import cadence as cd + + arm, founder = spec["arm"], int(spec["founder"]) + task = make_task(spec["task"], founder) + lessons = int(spec.get("lessons", LESSONS.get(task.name, SINGLE_LESSONS))) + if arm == "frozen": + lessons = 0 + module = int(spec.get("module", MODULE)) + init_seed = int(spec.get("init_seed", founder)) + started = time.monotonic() + brain = make_brain(task, arm, init_seed, spec.get("settings"), module, + int(spec.get("replica", 0))) + if spec.get("gain_start") is not None: # an emulation: every port's gain at birth + brain.recall.gains = np.broadcast_to(np.asarray(spec["gain_start"], dtype=float), + np.shape(brain.recall.gains)).copy() + tests = task.panels(int(spec.get("panel_batches", 10 if task.twins else 24))) + rng = np.random.default_rng(np.random.SeedSequence([founder, SEEDS["train"]])) + checkpoints = {int(c) for c in spec.get("checkpoints", [])} + work = new_work() + curve = [] + support = Support(spec.get("support", ()), spec.get("cortex_scale")) + if spec.get("gain_late_value") is not None: + support.late_value = spec["gain_late_value"] + for b in range(1, lessons + 1): + teach(brain, task.train(rng), work, support, spec.get("cortex_rate")) + if b in checkpoints and b != lessons: + curve.append({"batches": b, "panels": { + name: panel["conditions"]["all"] + for name, panel in reading(brain, task, tests, False, support).items()}}) + seconds_taught = time.monotonic() - started + row: dict[str, Any] = { + "task": task.name, "arm": arm, "founder": founder, "init_seed": init_seed, + "replica": int(spec.get("replica", 0)), "tag": spec.get("tag"), + "settings": {**ARMS[arm], **spec.get("settings", {})}, "cut": CUTS.get(arm, []), + "support": list(support.names), "cortex_rate": spec.get("cortex_rate"), + "gain_start": spec.get("gain_start"), + "cortex_scale": support.scale, + "lessons": lessons, "module": module, "inputs": task.inputs, "actions": task.actions, + "parameters": int(brain.parameters()), "cadence": cd.__version__, + "split": {"taught": list(task.split.taught), "new": list(task.split.new), + "pairs": [list(p) for p in task.split.pairs]}, + "chance": chance(task), "work": work, "curve": curve, + "seconds_taught": round(seconds_taught, 1), + "panels": reading(brain, task, tests, bool(spec.get("causal", True)), support, + bool(spec.get("vote"))), + } + if spec.get("decode"): + row["decode"] = decoding(brain, task, support=support) + if brain.recall is not None and getattr(brain.recall, "gain", 0): + row["gains"] = [round(float(g), 4) for g in brain.recall.gains] + row["seconds"] = round(time.monotonic() - started, 1) + return row + + +# -------------------------------------------------------------------------------- summary + + +def table(rows: list[dict[str, Any]]) -> list[str]: + """One line per life and panel condition: the answer, the read's first choice, the mass on what + it should read, and the answer under the read silenced and the attended slot erased.""" + lines = [] + for r in sorted(rows, key=lambda r: (r["task"], r["arm"], r["founder"], r.get("replica", 0))): + for name, panel in r["panels"].items(): + for cond in ("taught", "key_new", "answer_new", "new", "fixed", "all"): + c = panel["conditions"].get(cond) + if c is None or (cond == "all" and len(panel["conditions"]) > 1 and any( + k in panel["conditions"] for k in KINDS)): + continue + causal = panel.get("causal", {}) + cuts = " ".join( + f"{k[:12]} {v['conditions'][cond]['answer']:.2f}" for k, v in causal.items() + if cond in v["conditions"]) + lines.append( + f"{r['task']:12s} {r['arm']:5s} f{r['founder']}.{r.get('replica', 0)} " + f"{name:9s} {cond:10s} n{c['n']:4d} answer {c['answer']:.2f} " + f"first {c.get('first', float('nan')):.2f} " + f"mass {c.get('mass', float('nan')):.2f} " + f"avoid {c.get('says_avoid', float('nan')):.2f} | {cuts}") + return lines + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__.split("\n", 1)[0]) + parser.add_argument("--task", choices=sorted(TASKS)) + parser.add_argument("--arms", nargs="*", default=["Fd"], choices=list(ARMS)) + parser.add_argument("--founders", nargs="*", type=int, default=list(FOUNDERS)) + parser.add_argument("--plan", type=Path, help="a JSON list of life specs in place of the grid") + parser.add_argument("--out", type=Path, help="JSON lines, one row per life, appended") + parser.add_argument("--workers", type=int, default=1) + parser.add_argument("--table", type=Path, help="print the table of a results file") + args = parser.parse_args(argv) + if args.table is not None: + rows = [json.loads(line) for line in args.table.read_text().splitlines() if line] + print("\n".join(table(rows))) + return 0 + if args.out is None: + parser.error("--out is required") + if args.plan is not None: + specs = json.loads(args.plan.read_text()) + else: + if args.task is None: + parser.error("--task or --plan is required") + specs = [{"task": args.task, "arm": arm, "founder": f, + "tag": f"{args.task}/{arm}/{f}"} for arm in args.arms for f in args.founders] + done = set() + if args.out.exists(): + done = {json.loads(line).get("tag") for line in args.out.read_text().splitlines() if line} + specs = [s for s in specs if s.get("tag") not in done] + args.out.parent.mkdir(parents=True, exist_ok=True) + with args.out.open("a") as sink: + # one process per life: nothing a life patches reaches the next + with get_context("spawn").Pool(max(args.workers, 1), maxtasksperchild=1) as pool: + for row in pool.imap_unordered(run_life, specs, chunksize=1): + sink.write(json.dumps(row) + "\n") + sink.flush() + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/benchmarks/heads/nets.py b/benchmarks/heads/nets.py new file mode 100644 index 00000000..054547dd --- /dev/null +++ b/benchmarks/heads/nets.py @@ -0,0 +1,376 @@ +"""The heads chamber's controls: the grammar chamber's matched networks on its tasks. + +``lstm`` and ``transformer`` are ``controls.py``'s two networks, built by its code at a task's +inputs and actions, their width the one whose parameter count is the nearest to the ``Fd`` brain's +on that task. They hear the brain's moments and are taught its answer moments by cross-entropy, +teacher-forced; they are read on its panels by its scorer, greedy, in closed loop where the task +feeds answers back. They are independently scored controls and never part of a brain's answer path. + +- ``online``: one pass in the brain's order, one Adam step per batch (rate 0.003, gradient norm + clipped at 1): equal exposure. +- ``offline``: the same batches in shuffled epochs (rate 0.001) until the seconds spent in updates + reach ``compute_seconds``, the brain's measured life on the same machine: equal compute. + +**How a never-taught name reaches the output.** In these two networks it has no route. Its input +weights stay at their random start (it is never heard in training), and its output row is a row of +the last linear map that no lesson raised and every lesson's cross-entropy lowers, so the logit of +a never-taught name falls through a life. The brain without ``recall_gain`` has none either: its +motor neuron's synapses took no lesson with it as the label. The brain with ``recall_gain`` has one +the two networks lack: one learned scalar per read port on the token's identity. ``tied`` gives the +transformer the matching route and is reported apart: its logits are the hidden state's product +with the input weights of the same token (weight tying), so a hidden state that holds a token's +embedding names that token, never-taught ones included. It is no restriction of the brain's and no +candidate for it. ``pointer`` gives the transformer the brain's route itself: one more attention +over the moments before, its query and keys learned maps of the hidden state, whose read of the +heard tokens reaches the logits through one learned scalar from zero, beside the last linear map. +``copy`` takes that map away from the tokens: a token's logit is the log of that attention's read of +it, so the network says what it attends, a pointer network. All are reported apart, each at the +brain's parameter count. + +Causality here, on any network: a moment blanked at test (``blank_answer`` the moment that told the +answer, ``blank_dup`` the duplicate's earlier mention, ``blank_dup2`` the duplicate itself), and on +the transformers the attention layers' output silenced. + + python benchmarks/heads/nets.py --task ioi --models lstm transformer tied --modes online \\ + --founders 0 1 2 --out DIR/nets.jsonl --workers 3 +""" + +from __future__ import annotations + +import argparse +import json +import sys +import time +from contextlib import contextmanager +from multiprocessing import get_context +from pathlib import Path +from types import SimpleNamespace +from typing import Any + +import heads as hd +import numpy as np + +ct = hd._module("controls", hd.HERE.parent / "grammar" / "controls.py") +MODELS = ("lstm", "transformer", "tied", "pointer", "copy") +MODES = ("online", "offline") +MAX_EPOCHS = 400 +CAUSAL: dict[str, dict[str, Any]] = { + "silence": {"silence": True}, + "blank_answer": {"blank": "answer"}, + "blank_dup": {"blank": "dup"}, + "blank_dup2": {"blank": "dup2"}, +} + + +@contextmanager +def _dims(task: hd.Task) -> Any: + """``controls.py``'s code at a task's inputs and actions.""" + saved = ct.gr + ct.gr = SimpleNamespace(INPUTS=task.inputs, ACTIONS=task.actions) + try: + yield + finally: + ct.gr = saved + + +def build(model: str, width: int, seed: int, task: hd.Task) -> Any: + import torch + from torch import nn + + with _dims(task): + net = ct.build("transformer" if model in ("tied", "pointer", "copy") else model, width, + seed) + + if model in ("lstm", "transformer"): + return net + + class Pointer(nn.Module): # type: ignore[misc] + """The transformer with a read of the moments before on the tokens' identity: the read's + attention ``p``, from learned maps of the hidden state, adds ``gain * sum_s p_s x_s`` to + the logits of the tokens, ``gain`` one scalar from zero.""" + + def __init__(self) -> None: + super().__init__() + self.inp, self.body, self.out = net.inp, net.body, net.out + self.register_buffer("position", net.position) + self.query, self.key = nn.Linear(width, width), nn.Linear(width, width) + self.gain = nn.Parameter(torch.zeros(1)) + self.pairs = [(j, t) for j, t in enumerate(task.tokens()) if t >= 0] + + def forward(self, x: Any) -> Any: + n = x.shape[1] + mask = torch.triu(torch.full((n, n), float("-inf")), diagonal=1) + h = self.body(self.inp(x) + self.position[:n], mask=mask, is_causal=True) + logits = self.out(h) + before = torch.triu(torch.full((n, n), float("-inf")), diagonal=0) # s < t + before[0, 0] = 0.0 # the first moment reads itself, weighed out below + scores = self.query(h) @ self.key(h).transpose(1, 2) / width ** 0.5 + p = torch.softmax(scores + before, dim=-1) + p = torch.cat([torch.zeros_like(p[:, :1]), p[:, 1:]], dim=1) # nothing before it + actions = [j for j, _ in self.pairs] + tokens = [t for _, t in self.pairs] + read = p @ x[:, :, tokens] + added = torch.zeros_like(logits) + if model == "copy": # the tokens' logits are the read's alone + logits = logits.clone() + logits[:, :, actions] = torch.log(read + 1e-6) + return logits + added[:, :, actions] = self.gain * read + return logits + added + + def macs(self, length: int) -> int: + with _dims(task): + return int(net.macs(length)) + 2 * length * length * width * width + + if model in ("pointer", "copy"): + return Pointer() + + class Tied(nn.Module): # type: ignore[misc] + """The transformer whose logits are the hidden state's product with the same token's input + weights; an action that is no token keeps a row of its own.""" + + def __init__(self) -> None: + super().__init__() + self.inp, self.body = net.inp, net.body + self.register_buffer("position", net.position) + self.coordinates = [t for t in task.tokens() if t >= 0] + others = len(task.tokens()) - len(self.coordinates) + self.others = nn.Parameter(torch.randn(others, width) * 0.1) if others else None + + def forward(self, x: Any) -> Any: + n = x.shape[1] + mask = torch.triu(torch.full((n, n), float("-inf")), diagonal=1) + h = self.body(self.inp(x) + self.position[:n], mask=mask, is_causal=True) + logits = h @ self.inp.weight[:, self.coordinates] + return logits if self.others is None else torch.cat([logits, h @ self.others.T], dim=2) + + def macs(self, length: int) -> int: + with _dims(task): + return int(net.macs(length)) + + return Tied() + + +def count(net: Any) -> int: + return int(sum(p.numel() for p in net.parameters())) + + +def matched_width(model: str, target: int, task: hd.Task) -> int: + """The width whose parameter count is the nearest to ``target`` (even, as the control's).""" + best = None + for width in range(4, 257, 2): + n = count(build(model, width, 0, task)) + if best is None or abs(n - target) < abs(best[1] - target): + best = (width, n) + assert best is not None + return best[0] + + +def encode(moments: list[hd.Moment]) -> tuple[np.ndarray, np.ndarray]: + """``(batch, moments, inputs)`` as heard, and ``(batch, moments)`` with -100 untaught.""" + x = np.stack([m.x for m in moments], axis=1).astype(np.float32) + n = len(moments[0].x) + y = np.stack([np.full(n, -100) if m.label is None else m.label for m in moments], axis=1) + return x, y.astype(np.int64) + + +class Learner: + def __init__(self, model: str, width: int, seed: int, rate: float, task: hd.Task) -> None: + import torch + + self.torch = torch + self.task = task + self.net = build(model, width, seed, task) + self.opt = torch.optim.Adam(self.net.parameters(), lr=rate) + self.work = {"updates": 0, "tokens": 0, "macs": 0, "seconds": 0.0} + + def macs(self, length: int) -> int: + with _dims(self.task): + return int(self.net.macs(length)) + + def update(self, moments: list[hd.Moment]) -> float: + torch = self.torch + started = time.monotonic() + x, y = encode(moments) + logits = self.net(torch.from_numpy(x)) + loss = torch.nn.functional.cross_entropy(logits.reshape(-1, self.task.actions), + torch.from_numpy(y).reshape(-1), + ignore_index=-100) + self.opt.zero_grad() + loss.backward() + torch.nn.utils.clip_grad_norm_(self.net.parameters(), ct.CLIP) + self.opt.step() + self.work["updates"] += 1 + self.work["tokens"] += x.shape[0] * x.shape[1] + self.work["macs"] += 3 * x.shape[0] * self.macs(x.shape[1]) + self.work["seconds"] += time.monotonic() - started + return float(loss.item()) + + def answers(self, moments: list[hd.Moment], plan: dict[str, Any] | None = None + ) -> list[np.ndarray]: + """Each answer moment's greedy answers; a moment that feeds answers back hears the + previous answer. ``plan``: a named moment blanked, the attention silenced.""" + torch = self.torch + plan = plan or {} + hooks = [] + if plan.get("silence"): + layers = getattr(getattr(self.net, "body", None), "layers", ()) + for layer in layers: + hooks.append(layer.self_attn.register_forward_hook( + lambda _m, _i, out: (torch.zeros_like(out[0]), out[1]))) + x, _ = encode(moments) + answer_at = [t for t, m in enumerate(moments) if m.label is not None] + last = moments[answer_at[-1]] + blank = plan.get("blank") + if blank is not None and blank in last.slots: + rows = np.flatnonzero(last.slots[blank] >= 0) + x[rows, last.slots[blank][rows]] = 0.0 + out: list[np.ndarray] = [] + with torch.no_grad(): + if any(m.feedback for m in moments): + for t in answer_at: + if moments[t].feedback and out: + x[:, t] = hd.onehot(out[-1], self.task.inputs).astype(np.float32) + logits = self.net(torch.from_numpy(x[:, : t + 1]))[:, -1] + out.append(logits.argmax(dim=1).numpy()) + else: + logits = self.net(torch.from_numpy(x)) + out = [logits[:, t].argmax(dim=1).numpy() for t in answer_at] + self.work["macs"] += x.shape[0] * self.macs(x.shape[1]) + for hook in hooks: + hook.remove() + return out + + +def read_panel(learner: Learner, task: hd.Task, batches: list[list[hd.Moment]], + plan: dict[str, Any] | None = None) -> dict[str, Any]: + tally = hd.Tally() + exact = [] + for moments in batches: + said = learner.answers(moments, plan) + hits = [] + asked = [m for m in moments if m.label is not None] + for m, action in zip(asked, said, strict=True): + tally.add(m, action, None) + hits.append(action == m.label) + exact.append(np.all(np.stack(hits), axis=0)) + out = tally.summary(twins=task.twins) + out["exact"] = round(float(np.mean(np.concatenate(exact))), 4) + return out + + +def reading(learner: Learner, task: hd.Task, tests: dict[str, list[list[hd.Moment]]], + causal: bool = True) -> dict[str, Any]: + learner.net.eval() + out = {} + for name, batches in tests.items(): + panel = read_panel(learner, task, batches) + if causal: + panel["causal"] = {} + for cut, plan in CAUSAL.items(): + if "blank" in plan and not any(plan["blank"] in m.slots for m in batches[0]): + continue + if "silence" in plan and not hasattr(learner.net, "body"): + continue + read = read_panel(learner, task, batches, plan) + panel["causal"][cut] = { + "conditions": {k: {"answer": v["answer"], **({"says_avoid": v["says_avoid"]} + if "says_avoid" in v else {})} + for k, v in read["conditions"].items()}, + **{k: read[k] for k in ("both", "differ", "exact") if k in read}} + out[name] = panel + learner.net.train() + return out + + +def brain_parameters(task: hd.Task, founder: int, arm: str = "Fd") -> int: + import warnings + + warnings.simplefilter("ignore") + return int(hd.make_brain(task, arm, founder).parameters()) + + +def run_life(spec: dict[str, Any]) -> dict[str, Any]: + """One control life: ``task``, ``model``, ``mode``, ``founder``, with ``lessons``, + ``panel_batches``, ``parameters`` (the brain's by default), ``width`` and for ``offline`` + ``compute_seconds`` and ``epochs`` (the most).""" + import torch + + torch.set_num_threads(1) + model, mode, founder = spec["model"], spec["mode"], int(spec["founder"]) + task = hd.make_task(spec["task"], founder) + lessons = int(spec.get("lessons", hd.LESSONS.get(task.name, hd.SINGLE_LESSONS))) + started = time.monotonic() + target = int(spec.get("parameters") or brain_parameters(task, founder)) + width = int(spec.get("width") or matched_width(model, target, task)) + learner = Learner(model, width, int(spec.get("init_seed", founder)), ct.RATES[mode], task) + tests = task.panels(int(spec.get("panel_batches", 10 if task.twins else 24))) + rng = np.random.default_rng(np.random.SeedSequence([founder, hd.SEEDS["train"]])) + batches = [task.train(rng) for _ in range(lessons)] + curve: list[dict[str, Any]] = [] + epochs = 1 + if mode == "online": + losses = [learner.update(b) for b in batches] + last = float(np.mean(losses[-50:])) + else: + budget = float(spec.get("compute_seconds", 0.0)) + most = int(spec.get("epochs", MAX_EPOCHS)) + order = np.random.default_rng(np.random.SeedSequence([founder, 99, 0])) + epochs, last = 0, float("nan") + while epochs < most and (epochs == 0 or learner.work["seconds"] < budget): + last = float(np.mean([learner.update(batches[int(i)]) + for i in order.permutation(lessons)])) + epochs += 1 + panels = reading(learner, task, tests, causal=False) + curve.append({"epoch": epochs, "update_seconds": round(learner.work["seconds"], 1), + "loss": round(last, 4), + "panels": {k: v["conditions"]["all"]["answer"] + for k, v in panels.items()}}) + row: dict[str, Any] = { + "task": task.name, "arm": f"{model}:{mode}", "model": model, "mode": mode, + "founder": founder, "init_seed": int(spec.get("init_seed", founder)), "replica": 0, + "tag": spec.get("tag"), "width": width, "parameters": count(learner.net), + "brain_parameters": target, "lessons": lessons, "rate": ct.RATES[mode], "epochs": epochs, + "loss_last": last, "curve": curve, "torch": torch.__version__, + "compute_seconds": spec.get("compute_seconds"), "chance": hd.chance(task), + } + work = dict(learner.work) + row["panels"] = reading(learner, task, tests) + row["work"] = {**work, "seconds": round(work["seconds"], 1)} + row["seconds"] = round(time.monotonic() - started, 1) + return row + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__.split("\n", 1)[0]) + parser.add_argument("--task", choices=sorted(hd.TASKS)) + parser.add_argument("--models", nargs="*", default=list(MODELS), choices=MODELS) + parser.add_argument("--modes", nargs="*", default=["online"], choices=MODES) + parser.add_argument("--founders", nargs="*", type=int, default=list(hd.FOUNDERS)) + parser.add_argument("--plan", type=Path, help="a JSON list of life specs in place of the grid") + parser.add_argument("--out", type=Path, required=True) + parser.add_argument("--workers", type=int, default=1) + args = parser.parse_args(argv) + if args.plan is not None: + specs = json.loads(args.plan.read_text()) + else: + if args.task is None: + parser.error("--task or --plan is required") + specs = [{"task": args.task, "model": m, "mode": mode, "founder": f, + "tag": f"{args.task}/{m}:{mode}/{f}"} + for m in args.models for mode in args.modes for f in args.founders] + done = set() + if args.out.exists(): + done = {json.loads(line).get("tag") for line in args.out.read_text().splitlines() if line} + specs = [s for s in specs if s.get("tag") not in done] + args.out.parent.mkdir(parents=True, exist_ok=True) + with args.out.open("a") as sink: + with get_context("spawn").Pool(max(args.workers, 1), maxtasksperchild=1) as pool: + for row in pool.imap_unordered(run_life, specs, chunksize=1): + sink.write(json.dumps(row) + "\n") + sink.flush() + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/benchmarks/heads/test_heads.py b/benchmarks/heads/test_heads.py new file mode 100644 index 00000000..796ed8d7 --- /dev/null +++ b/benchmarks/heads/test_heads.py @@ -0,0 +1,919 @@ +"""The heads chamber (cadence#269): the language, its sealed split, what each panel asks, the +reads and interventions, a short life of each arm and the controls' routes.""" + +from __future__ import annotations + +import copy +import importlib.util +import sys +from pathlib import Path + +import numpy as np +import pytest + + +def _load(name: str): + if name in sys.modules: + return sys.modules[name] + here = Path(__file__).parent + sys.path.insert(0, str(here)) # nets imports heads by name, as the grammar chamber's controls + spec = importlib.util.spec_from_file_location(name, here / f"{name}.py") + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + sys.modules[name] = module + spec.loader.exec_module(module) + return module + + +hd = _load("heads") +SMALL = {"lessons": 3, "panel_batches": 2, "module": 16} + + +def _efficacy(brain): + return brain.learner.brain.efficacy + + +def _tokens(task, moment): + return np.argmax(moment.x[:, : task.words], axis=1) + + +def _stream(task, moments): + return np.stack([_tokens(task, m) for m in moments], axis=1) + + +@pytest.mark.parametrize("founder", [0, 1, 2, 3]) +def test_the_split_is_sealed_and_differs_by_founder(founder): + split = hd.Split.of(founder) + assert split == hd.Split.of(founder) + assert len(split.taught) == 8 and len(split.new) == 4 + assert set(split.taught) | set(split.new) == set(range(hd.TOKENS)) + assert all(a in split.taught and b in split.taught for a, b in split.pairs) + assert hd.Split.of(founder).new != hd.Split.of(founder + 10).new + + +@pytest.mark.parametrize("name", sorted(hd.TASKS)) +def test_training_never_hears_a_never_taught_token(name): + task = hd.TASKS[name](1) + rng = np.random.default_rng(0) + for _ in range(12): + for m in task.train(rng): + assert not m.x[:, list(task.new)].any() + if m.label is not None: + assert not np.isin(m.label, task.new).any() + + +def test_the_named_sentence_is_a_row_of_the_chamber(): + task = hd.IndirectObject(0) + mary, john = hd.NAMES.index("mary"), hd.NAMES.index("john") + rows = [(mary, john, john)] + moments = task.frame(rows, (2, 3), [("x",)]) + said = [hd.WORDS[int(np.argmax(m.x[0]))] for m in moments] + assert " ".join(said) == "when mary and john went out john gave a drink to" + assert hd.WORDS[int(moments[-1].label[0])] == "mary" + assert moments[-1].slots["answer"][0] == 1 and moments[-1].slots["dup"][0] == 3 + + +def test_the_chamber_has_the_grammar_chambers_scale(): + assert len(hd.WORDS) == 20 + assert hd.BATCH == hd.gr.BATCH + assert hd.LESSONS["ioi"] == 4000 + + +def test_a_sentence_asks_the_name_not_repeated_in_both_orders_at_every_position(): + task = hd.IndirectObject(2) + rng = np.random.default_rng(3) + seen_shapes, orders = set(), set() + for _ in range(60): + moments = task.train(rng) + stream = _stream(task, moments) + asked = moments[-1] + at = next(t for t, m in enumerate(moments) if m.mark == "dup") + for r in range(hd.BATCH): + a, b, s = stream[r, 1], stream[r, 3], stream[r, at] + assert a != b and s in (a, b) + assert asked.label[r] == (b if s == a else a) + assert not task.split.held(int(a), int(b)) + assert stream[r, int(asked.slots["answer"][r])] == asked.label[r] + assert stream[r, int(asked.slots["dup"][r])] == s + assert stream[r, int(asked.slots["dup2"][r])] == s + orders.add(asked.tags[r][1]) + seen_shapes.add(at) + assert orders == {"abb", "bab"} + assert seen_shapes == {4, 5, 6} + + +def test_the_panels_ask_what_they_name(): + task = hd.IndirectObject(0) + panels = task.panels(10) + assert set(panels) == {"ioi", "ioi_pair", "ioi_shape", "ioi_half", "ioi_new"} + taught, new = set(task.split.taught), set(task.split.new) + for name, batches in panels.items(): + for moments in batches: + stream = _stream(task, moments) + asked = moments[-1] + for i in range(0, hd.BATCH, 2): # twins: the same pair, the other name repeated + assert (stream[i, 1], stream[i, 3]) == (stream[i + 1, 1], stream[i + 1, 3]) + assert asked.label[i] != asked.label[i + 1] + pairs = {(int(stream[r, 1]), int(stream[r, 3])) for r in range(hd.BATCH)} + for a, b in pairs: + if name in ("ioi", "ioi_shape"): + assert a in taught and b in taught and not task.split.held(a, b) + elif name == "ioi_pair": + assert task.split.held(a, b) + elif name == "ioi_half": + assert (a in new) != (b in new) + else: + assert a in new and b in new + shape = {next(t for t, m in enumerate(b) if m.mark == "dup") for b in panels["ioi_shape"]} + assert shape == {5} and all(len(b) == 7 + 1 for b in panels["ioi_shape"]) + tags = {t for b in panels["ioi_half"] for row in b[-1].tags for t in row} + assert {"key_new", "answer_new"} <= tags + + +def test_previous_asks_the_token_before_the_flagged_one(): + task = hd.Previous(0) + moments = task.batch(np.random.default_rng(0), task.kinds_of_a_batch()) + stream = _stream(task, moments) + assert np.array_equal(moments[-1].label, stream[:, -2]) + assert moments[-1].x[:, task.flag["ASK"]].all() + for r, kind in enumerate(task.kinds_of_a_batch()): + assert (stream[r, -1] in task.new) == (kind in ("key_new", "new")) + assert (stream[r, -2] in task.new) == (kind in ("answer_new", "new")) + + +def test_duplicate_is_balanced_and_true_within_each_kind(): + task = hd.Duplicate(0) + rng = np.random.default_rng(0) + for _ in range(8): + moments = task.batch(rng, task.kinds_of_a_batch()) + stream = _stream(task, moments) + asked = moments[-1] + yes = asked.label == task.words + assert yes.sum() == hd.BATCH // 2 + for r in range(hd.BATCH): + told = stream[r, :-1] + assert (stream[r, -1] in told) == yes[r] + assert len(set(told)) == len(told) + if yes[r]: + assert told[int(asked.slots["answer"][r])] == stream[r, -1] + assert asked.source[r] == stream[r, -1] + else: + assert asked.source[r] == -1 + + +@pytest.mark.parametrize("name", ["successor", "successor_fixed"]) +def test_successor_asks_the_token_told_after_the_flagged_one(name): + task = hd.TASKS[name](0) + moments = task.batch(np.random.default_rng(0), task.kinds_of_a_batch()) + stream = _stream(task, moments) + for r in range(hd.BATCH): + told = list(stream[r, :-1]) + assert len(set(told)) == len(told) + assert moments[-1].label[r] == told[told.index(stream[r, -1]) + 1] + + +def test_a_fixed_successor_tells_one_order_in_training(): + task = hd.SuccessorFixed(1) + rng = np.random.default_rng(0) + first = _stream(task, task.train(rng))[:, :-1] + second = _stream(task, task.train(rng))[:, :-1] + assert np.array_equal(first, second) + assert np.all(first == first[0]) + assert sorted(first[0]) == sorted(task.split.taught) + + +def test_induction_asks_what_followed_the_only_earlier_occurrence(): + task = hd.Induction(0) + moments = task.batch(np.random.default_rng(0), task.kinds_of_a_batch()) + stream = _stream(task, moments) + for r in range(hd.BATCH): + told = list(stream[r, :-1]) + assert told.count(stream[r, -1]) == 1 + assert moments[-1].label[r] == told[told.index(stream[r, -1]) + 1] + + +def test_two_hops_ask_the_second_of_each_pair(): + task = hd.TwoHops(0) + moments = task.batch(np.random.default_rng(0), task.kinds_of_a_batch()) + stream = _stream(task, moments[:-1]) + for r in range(hd.BATCH): + pairs = {int(stream[r, 2 * i]): int(stream[r, 2 * i + 1]) for i in range(task.pairs)} + a = int(stream[r, -1]) + assert moments[-2].label[r] == pairs[a] + assert moments[-1].label[r] == pairs[pairs[a]] + assert moments[-1].x[:, task.flag["THINK"]].all() + assert {t[1] for t in moments[-1].tags} == {"hop2"} + + +def test_copying_asks_each_token_in_turn(): + task = hd.Copying(0) + moments = task.batch(np.random.default_rng(0), task.kinds_of_a_batch()) + length = sum(m.label is not None for m in moments) + told = _stream(task, moments[1:1 + length]) + answers = np.stack([m.label for m in moments if m.label is not None], axis=1) + assert np.array_equal(told, answers) + assert all(m.feedback for m in moments[2 + length:]) + assert {t[0] for m in moments if m.tags for t in m.tags} <= set(hd.KINDS) + + +def test_copying_is_read_in_closed_loop_and_teacher_forced_on_the_same_rows(): + panels = hd.Copying(0).panels(2) + assert set(panels) == {"test", "forced"} + for closed, forced in zip(panels["test"], panels["forced"], strict=True): + assert any(m.feedback for m in closed) and not any(m.feedback for m in forced) + for a, b in zip(closed, forced, strict=True): + assert np.array_equal(a.x, b.x) + + +def test_suppression_asks_the_name_not_repeated(): + task = hd.Suppression(0) + panels = task.panels(3) + assert set(panels) == {"taught", "half", "new"} + for batches in panels.values(): + for moments in batches: + stream = _stream(task, moments[:3]) + a, b, s = stream[:, 0], stream[:, 1], stream[:, 2] + assert np.array_equal(moments[-1].label, np.where(s == a, b, a)) + assert np.array_equal(moments[-1].avoid, s) + + +def test_which_asks_the_place_of_the_one_not_repeated(): + task = hd.Which(0) + assert task.tokens()[-2:] == [-1, -1] + for moments in task.panels(2)["new"]: + stream = _stream(task, moments[:3]) + a, s = stream[:, 0], stream[:, 2] + assert np.array_equal(moments[-1].label, np.where(s == a, task.words + 1, task.words)) + assert np.array_equal(moments[-1].source, np.where(s == a, stream[:, 1], a)) + assert not np.isin(moments[-1].label, task.split.new).any() + + +def test_the_bare_frame_has_no_fixed_word_before_the_names(): + task = hd.Bare(0) + rng = np.random.default_rng(1) + for _ in range(10): + moments = task.train(rng) + stream = _stream(task, moments) + asked = moments[-1] + at = next(t for t, m in enumerate(moments) if m.mark == "dup") + assert hd.WHEN not in stream and hd.AND not in stream + for r in range(hd.BATCH): + a, b, s = stream[r, 0], stream[r, 1], stream[r, at] + assert s in (a, b) and asked.label[r] == (b if s == a else a) + assert stream[r, int(asked.slots["answer"][r])] == asked.label[r] + assert stream[r, int(asked.slots["dup"][r])] == s + + +def test_the_parts_ask_what_the_sentence_needs(): + mover, bit, whole = hd.Mover(0), hd.DuplicateBit(0), hd.IndirectObject(0) + rng = np.random.default_rng(5) + rows = [(0, 1, 0), (0, 1, 1)] + tags = whole.tagged(rows) + moved = mover.frame(rows, (2, 3), tags) + assert not any(m.mark == "dup" for m in moved) + assert moved[-1].x[0, mover.flag["SECOND"]] == 1.0 + assert moved[-1].x[1, mover.flag["FIRST"]] == 1.0 + assert list(moved[-1].label) == list(whole.frame(rows, (2, 3), tags)[-1].label) + asked = bit.frame(rows, (2, 3), tags)[-1] + assert list(asked.label) == [bit.words, bit.words + 1] + assert asked.x[:, bit.flag["WHICH"]].all() + assert bit.tokens()[-2:] == [-1, -1] + assert bit.train(rng)[-1].label.min() >= bit.words + + +def test_the_arms_are_the_base_composition(): + assert hd.FD == {**hd.gr.ARMS["recipe"], "teaching": "deep"} + assert hd.ARMS["Fd2"] == {**hd.FD, "recall_heads": 2} + assert hd.CUTS["FdT"] == ["trace"] + for arm in ("Fd", "Fd2", "FdT"): + assert "apical" not in hd.ARMS[arm] + + +@pytest.fixture(scope="module") +def taught_brain(): + task = hd.IndirectObject(0) + brain = hd.make_brain(task, "Fd", 0, module=16) + rng = np.random.default_rng(0) + work = hd.new_work() + for _ in range(4): + hd.teach(brain, task.train(rng), work) + assert work["lessons"] == 4 + return task, brain + + +def test_addressing_a_moment_leaves_its_act_unchanged(taught_brain): + task, brain = taught_brain + moments = task.train(np.random.default_rng(1)) + plain, probed = copy.deepcopy(brain), copy.deepcopy(brain) + for b in (plain, probed): + b.reset() + work = hd.new_work() + for m in moments: + attended = hd._attended(probed, m.x) + a = hd._act(plain, m.x, work) + c = hd._act(probed, m.x, work) + assert np.array_equal(a, c) + if attended: + assert np.array_equal(attended[0], plain.recall.attended_heads[0]) + assert np.array_equal(plain.recall.keys, probed.recall.keys) + assert np.array_equal(plain.recall.previous, probed.recall.previous) + + +def test_a_reading_never_changes_the_life(taught_brain): + task, brain = taught_brain + before = copy.deepcopy(brain) + hd.reading(brain, task, task.panels(1), True) + assert np.array_equal(_efficacy(before), _efficacy(brain)) + assert np.array_equal(before.recall.head_map(0)[0], brain.recall.head_map(0)[0]) + assert before.recall.amplitude == brain.recall.amplitude + + +def test_a_silent_read_drives_nothing_and_comes_back(taught_brain): + task, brain = taught_brain + fork = copy.deepcopy(brain) + moments = task.train(np.random.default_rng(2)) + fork.reset() + for m in moments[:-1]: + fork.act(m.x, greedy=True) + recall_neurons = fork.connectome.populations["recall"] + loud = fork.stimulus(moments[-1].x)[:, recall_neurons] + assert np.abs(loud).sum() > 0 + fork.recall.amplitude = 0.0 + assert not fork.stimulus(moments[-1].x)[:, recall_neurons].any() + + +def test_a_lesion_empties_the_slot_it_names_and_only_that(taught_brain): + task, brain = taught_brain + fork = copy.deepcopy(brain) + moments = task.train(np.random.default_rng(2)) + fork.reset() + for m in moments[:-1]: + fork.act(m.x, greedy=True) + recall = fork.recall + keys = np.array(recall.keys, copy=True) + rows = np.arange(hd.BATCH) + slots = np.full(hd.BATCH, 2) + hd._lesion(recall, rows, slots) + assert not recall.keys[:, 2].any() and not recall.values[:, 2].any() + assert not recall.nexts[:, 2].any() + assert np.array_equal(recall.keys[:, 1], keys[:, 1]) + + +def test_erasing_the_answers_slot_takes_it_from_both_routes(taught_brain): + task, brain = taught_brain + fork = copy.deepcopy(brain) + moments = task.train(np.random.default_rng(2)) + fork.reset() + for m in moments[:-1]: + fork.act(m.x, greedy=True) + asked = moments[-1] + t = len(moments) - 1 + rows = np.arange(hd.BATCH) + assert fork.recall.values[rows, :, asked.label].any() + hd.intervene(fork, asked, {"erase_source": ("answer",)}, t, np.random.default_rng(0)) + # a name told once: nothing holds it on either port at this moment + assert not fork.recall.values[rows, :, asked.label].any() + assert not fork.recall.nexts[rows, :, asked.label].any() + + +def test_erasing_the_attended_slot_empties_what_the_read_attends(taught_brain): + task, brain = taught_brain + fork = copy.deepcopy(brain) + moments = task.train(np.random.default_rng(2)) + fork.reset() + for m in moments[:-1]: + fork.act(m.x, greedy=True) + top = np.argmax(hd._attended(fork, moments[-1].x)[0], axis=1) + hd.intervene(fork, moments[-1], {"erase": ("answer",)}, len(moments) - 1, + np.random.default_rng(0)) + assert not fork.recall.values[np.arange(hd.BATCH), top].any() + + +def test_the_control_erasure_spares_the_read_and_the_answer(taught_brain): + task, brain = taught_brain + fork = copy.deepcopy(brain) + moments = task.train(np.random.default_rng(2)) + fork.reset() + for m in moments[:-1]: + fork.act(m.x, greedy=True) + asked = moments[-1] + rows = np.arange(hd.BATCH) + top = np.argmax(hd._attended(fork, asked.x)[0], axis=1) + before = np.array(fork.recall.values, copy=True) + hd.intervene(fork, asked, {"erase_other": ("answer",)}, len(moments) - 1, + np.random.default_rng(0)) + assert np.array_equal(fork.recall.values[rows, top], before[rows, top]) + assert fork.recall.values[rows, :, asked.label].any(axis=1).all() + + +@pytest.mark.parametrize("cut", ["swap_cortex", "swap_trace", "swap_focus", "swap_slot"]) +def test_a_swap_puts_the_twins_carrier_in_the_rows_place(taught_brain, cut): + task, brain = taught_brain + fork = copy.deepcopy(brain) + moments = task.panels(1)["ioi"][0] + fork.reset() + for m in moments[:-1]: + fork.act(m.x, greedy=True) + recall = fork.recall + asked = moments[-1] + t = len(moments) - 1 + before = {"swap_cortex": np.array(recall.previous), "swap_focus": np.array(recall.focus), + "swap_trace": np.array(fork.working_memory.trace), + "swap_slot": np.array(recall.values)}[cut] + assert not hd.intervene(fork, asked, {cut: ("answer",)}, t, np.random.default_rng(0)) + twin = np.arange(hd.BATCH) ^ 1 + if cut == "swap_cortex": + assert np.array_equal(recall.previous, before[twin]) + elif cut == "swap_focus": + assert np.array_equal(recall.focus, before[twin]) + elif cut == "swap_trace": + assert np.array_equal(fork.working_memory.trace, before[twin]) + else: + at = recall.keys.shape[1] - (t - asked.slots["dup2"][0]) + assert np.array_equal(recall.values[:, at], before[twin, at]) + assert np.array_equal(recall.values[:, at - 1], before[:, at - 1]) + + +@pytest.mark.parametrize("moment", ["answer", "dup"]) +def test_the_logits_terms_sum_to_the_attention_the_read_uses(taught_brain, moment): + task, brain = taught_brain + fork = copy.deepcopy(brain) + moments = task.panels(1)["ioi"][0] + t = len(moments) - 1 if moment == "answer" else next( + i for i, m in enumerate(moments) if m.mark == "dup") + fork.reset() + for m in moments[:t]: + fork.act(m.x, greedy=True) + asked = moments[t] + fork.stimulus(asked.x) + found = hd.logit_terms(fork.recall, asked, t) + p = fork.recall.attended_heads[0] + rows = np.arange(hd.BATCH) + # the parts sum to the two sums, and a slot's log attention less another's is their difference + for name in found["key"]: + parts = sum(found[k][name] for k in ("key", "event", "cortex", "bias")) + assert np.allclose(parts, found["content"][name]) + parts = sum(found[k][name] for k in ("position_event", "position_cortex", "position_bias")) + assert np.allclose(parts, found["position"][name]) + at = p.shape[1] - (t - asked.slots["dup"]) + before = at - 1 + wanted = np.log(p[rows, at]) - np.log(p[rows, before]) + read = (found["content"]["dup"] + found["position"]["dup"] + - found["content"]["dup_before"] - found["position"]["dup_before"]) + assert np.allclose(wanted, read, atol=1e-8) + + +def test_a_raw_read_at_the_duplicate_finds_its_earlier_mention_and_leaves_the_maps(taught_brain): + task, brain = taught_brain + fork = copy.deepcopy(brain) + # the maps pointing away from every slot, so that only a raw read finds the mention + recall = fork.recall + w_x, w_c, b = recall.head_map(0) + recall._set_map(0, w_x, w_c, b + 50.0 * np.eye(len(b))[hd.WHEN]) + moments = task.panels(1)["ioi"][0] + support = hd.Support(["raw_dup"]) + work = hd.new_work() + fork.reset() + for t, m in enumerate(moments): + before = recall.head_map(0)[2].copy() + slots = recall.keys.shape[1] if m.mark == "dup" else None + support.act(fork, m, m.x, work) + if slots is not None: + at = slots - (t - m.slots["dup"]) + p = recall.attended_heads[0] + assert p[np.arange(hd.BATCH), at].min() > 0.5 + assert np.array_equal(recall.head_map(0)[2], before) + if m.mark == "dup": + break + + +def test_what_is_held_stands_in_place_up_to_the_answer(taught_brain): + task, brain = taught_brain + fork = copy.deepcopy(brain) + moments = task.frame([(0, 1, 0), (0, 1, 1)] * 8, (2, 3), [("x",)] * hd.BATCH) + support = hd.Support(["hold_focus", "hold_cortex", "hold_trace"]) + work = hd.new_work() + fork.reset() + support.reset() + held = held_trace = held_focus = None + for m in moments: + support.before(fork) + if held is not None: + assert np.array_equal(fork.recall.previous, held) + assert np.array_equal(fork.working_memory.trace, held_trace) + assert np.array_equal(fork.recall.focus, held_focus) + held_focus = np.where(held_focus >= 0, held_focus + 1, -1) + support.act(fork, m, m.x, work) + support.after(fork, m) + if m.mark == "dup": + held = np.array(fork.recall.previous) + held_trace = np.array(fork.working_memory.trace) + held_focus = np.array(fork.recall.focus) + + +def test_a_cut_keeps_the_address_maps_cortex_columns_at_zero(): + task = hd.IndirectObject(0) + brain = hd.make_brain(task, "Fd2", 0, module=16) + support = hd.Support(["no_address_cortex"]) + rng = np.random.default_rng(0) + work = hd.new_work() + for _ in range(4): + hd.teach(brain, task.train(rng), work, support) + for h in (0, 1): + w_x, w_c, _ = brain.recall.head_map(h) + assert not w_c.any() and w_x.any() + + +def _taught(cortex_rate, batches=4): + task = hd.IndirectObject(0) + brain = hd.make_brain(task, "Fd", 0, module=16) + rng = np.random.default_rng(0) + work = hd.new_work() + for _ in range(batches): + hd.teach(brain, task.train(rng), work, None, cortex_rate) + return brain + + +def test_a_cortex_rate_of_one_is_the_life_without_it(): + plain, scaled = _taught(None), _taught(1.0) + for name in hd.CORTEX_MAPS: + for a, b in zip(getattr(plain.recall, name) or [], getattr(scaled.recall, name) or [], + strict=True): + for x, y in zip(a, b, strict=True): + np.testing.assert_array_equal(x, y) + + +def test_a_cortex_rate_of_zero_keeps_every_maps_cortex_columns_at_birth(): + brain = _taught(0.0) + assert brain.recall.head_map(0)[0].any() + for name in hd.CORTEX_MAPS: + for head in getattr(brain.recall, name) or []: + assert not np.asarray(head[1]).any() + + +def test_a_cut_of_the_cortex_reads_zero_at_every_act(): + task = hd.IndirectObject(0) + brain = hd.make_brain(task, "Fd", 0, module=16) + support = hd.Support(["no_cortex"]) + moments = task.train(np.random.default_rng(0)) + work = hd.new_work() + brain.reset() + for m in moments[:3]: + support.before(brain) + assert not np.any(brain.recall.previous) + support.act(brain, m, m.x, work) + assert np.any(brain.recall.previous) # what the act leaves, cleared before the next + + +@pytest.mark.parametrize("names, scale", [(["cortex_unit"], None), ([], 0.25)]) +def test_a_balanced_cortex_is_read_at_unit_norm_or_scaled(names, scale): + task = hd.IndirectObject(0) + brain = hd.make_brain(task, "Fd", 0, module=16) + support = hd.Support(names, scale) + moments = task.train(np.random.default_rng(0)) + work = hd.new_work() + brain.reset() + support.act(brain, moments[0], moments[0].x, work) + raw = np.array(brain.recall.previous) + assert np.linalg.norm(raw, axis=1).min() > 0 + support.before(brain) + read = np.array(brain.recall.previous) + if scale is None: + assert np.allclose(np.linalg.norm(read, axis=1), 1.0) + assert np.allclose(read * np.linalg.norm(raw, axis=1, keepdims=True), raw) + else: + assert np.allclose(read, scale * raw) + + +def test_a_centered_cortex_is_read_less_its_mean_and_a_reading_leaves_the_mean(): + task = hd.IndirectObject(0) + brain = hd.make_brain(task, "Fd", 0, module=16) + support = hd.Support(["center_cortex"]) + work = hd.new_work() + rng = np.random.default_rng(0) + hd.teach(brain, task.train(rng), work, support) + mean = np.array(support.mean) + assert mean.any() + moments = task.train(rng) + brain.reset() + brain.act(moments[0].x, greedy=True) + raw = np.array(brain.recall.previous) + support.before(brain) + assert np.allclose(brain.recall.previous, raw - mean) + assert np.array_equal(support.mean, mean) + hd.read_panel(brain, task, task.panels(1)["ioi"], support=support) + assert np.array_equal(support.mean, mean) + + +def _with_gain() -> bool: + import cadence as cd + + recall = cd.Brain.compose(4, 2, modules=(4,), recall=True).recall + return "gain" in getattr(recall, "OPTIONS", ()) + + +needs_gain = pytest.mark.skipif(not _with_gain(), reason="a library with recall_gain") + + +@needs_gain +def test_the_gain_arm_declares_every_word_and_says_a_token_no_lesson_used(): + task = hd.Induction(0) + brain = hd.make_brain(task, "Fdg", 0, module=16) + assert list(brain.recall.tokens) == task.tokens() + row = hd.run_life({"task": "induction", "arm": "Fdg", "founder": 0, **SMALL, "causal": False}) + assert len(row["gains"]) == 2 + both = hd.run_life({"task": "duplicate", "arm": "Fd2g", "founder": 0, **SMALL, "causal": False}) + assert len(both["gains"]) == 4 + + +@needs_gain +def test_the_gains_drive_only_answers_under_the_oracle_and_hold_for_lessons_when_late(): + task = hd.IndirectObject(0) + brain = hd.make_brain(task, "Fdg", 0, module=16) + late = hd.Support(["gain_late"]) + late.LATE = 2 + rng = np.random.default_rng(0) + work = hd.new_work() + hd.teach(brain, task.train(rng), work, late) + hd.teach(brain, task.train(rng), work, late) + assert not brain.recall.gains.any() + hd.teach(brain, task.train(rng), work, late) + assert brain.recall.gains.any() + oracle = hd.Support(["gain_at_answer"]) + moments = task.train(rng) + gains = brain.recall.gains.copy() + brain.reset() + for m in moments: + oracle.act(brain, m, m.x, work) + assert np.array_equal(brain.recall.gains, gains) + out = hd.read_panel(brain, task, task.panels(1)["ioi"], votes=True, support=oracle) + assert out["vote"]["taught"]["answer"]["gain"] != 0.0 + + +def test_a_causal_reading_keeps_what_the_read_found(): + row = hd.run_life({"task": "suppression", "arm": "Fd", "founder": 0, **SMALL}) + cell = row["panels"]["new"]["causal"]["swap_focus"]["conditions"]["new"] + assert {"answer", "first", "port_best", "ports"} <= set(cell) + + +def test_a_life_without_a_support_is_the_life_with_none_named(): + spec = {"task": "suppression", "arm": "Fd", "founder": 0, **SMALL, "causal": False} + plain = hd.run_life(spec) + none = hd.run_life({**spec, "support": []}) + assert plain["panels"] == none["panels"] and none["support"] == [] + with pytest.raises(ValueError): + hd.Support(["nothing"]) + + +@pytest.mark.parametrize("support", [["raw_dup"], ["raw_dup", "hold_focus"], + ["raw_dup", "hold_cortex", "hold_trace"]]) +def test_a_short_life_with_a_support(support): + row = hd.run_life({"task": "ioi", "arm": "Fd", "founder": 0, **SMALL, "causal": False, + "support": support}) + assert row["support"] == support and row["work"]["refused"] == 0 + assert row["panels"]["ioi"]["duplicate_read"]["first_value"] > 0.9 + + +def test_the_decoder_reads_a_bit_a_carrier_holds(): + y = np.array([1.0, -1.0] * 40) + x = np.column_stack([y + np.random.default_rng(0).normal(0, 0.1, 80), + np.random.default_rng(1).normal(0, 1.0, 80)]) + noise = np.random.default_rng(2).normal(0, 1.0, (80, 2)) + read = hd._ridge(x, y, {"same": (x, y), "none": (noise, y)}) + assert read["same"] == 1.0 and read["none"] < 0.8 + + +def test_the_decoding_runs_on_a_sentence_task(taught_brain): + task, brain = taught_brain + out = hd.decoding(brain, task, batches=12) + assert {"taught", "new", "half"} == set(out["cortex_before_answer"]) + assert "cortex_after_dup" in out and "trace_after_answer" in out + + +def test_the_motor_inputs_sum_to_what_the_motor_neuron_receives(taught_brain): + task, brain = taught_brain + fork = copy.deepcopy(brain) + moments = task.panels(1)["ioi"][0] + fork.reset() + for m in moments[:-1]: + fork.act(m.x, greedy=True) + asked = moments[-1] + fork.act(asked.x, greedy=True) + found = hd.motor_inputs(fork, {"answer": asked.label}, task.actions)["answer"] + graph = fork.connectome + pre, post = np.asarray(graph.pre), np.asarray(graph.post) + motor = np.asarray(fork.motor_index) + activation = np.atleast_2d(np.asarray(fork.basal_ganglia.state.activation)) + weights, bias = np.asarray(fork.brain.weights), np.asarray(fork.brain.bias) + for r in range(hd.BATCH): + neuron = motor[asked.label[r]] + into = post == neuron + total = (weights[into] * activation[r, pre[into]]).sum() + bias[neuron] + assert np.isclose(found["association"][r] + found["other"][r], total) + assert found["gain"][r] == 0.0 # this brain holds no gains + out = hd.read_panel(brain, task, task.panels(1)["ioi"], votes=True) + assert {"answer", "avoid", "winner"} == set(out["vote"]["taught"]) + + +def test_the_read_of_a_moment_names_what_it_attends(): + values = np.zeros((2, 3, 5)) + nexts = np.zeros((2, 3, 5)) + values[:, 0, 1], values[:, 1, 2], values[:, 2, 3] = 1.0, 1.0, 1.0 + nexts[:, 0, 2], nexts[:, 1, 3] = 1.0, 1.0 + p = np.array([[0.7, 0.2, 0.1], [0.05, 0.15, 0.8]]) + read = hd.read_of((p,), values, nexts, np.array([2, 2]), np.array([3, 3])) + # row 0 attends the slot whose next is the token; row 1 the slot that holds the other + assert read["first"].tolist() == [True, False] + assert read["first_avoid"].tolist() == [False, True] + assert read["mass"][0] == pytest.approx(0.9) # slot 0 by its next, slot 1 by its value + assert read["mass_avoid"][1] == pytest.approx(0.95) # slot 1 by its next, slot 2 by its value + assert read["ports"]["next0"].tolist() == [True, False] + + +def test_a_row_without_a_token_in_the_slots_takes_no_read(): + values, nexts = np.ones((2, 3, 5)), np.ones((2, 3, 5)) + read = hd.read_of((np.full((2, 3), 1 / 3),), values, nexts, np.array([2, -1]), None) + assert read["held"].tolist() == [True, False] + + +@pytest.mark.parametrize("name", sorted(hd.TASKS)) +def test_a_short_life_of_each_task(name): + row = hd.run_life({"task": name, "arm": "Fd", "founder": 0, **SMALL}) + assert row["lessons"] == 3 and row["work"]["refused"] == 0 + panel = next(iter(row["panels"].values())) + assert panel["conditions"]["all"]["n"] > 0 and "causal" in panel + assert {"silence", "erase", "erase_source", "erase_other"} <= set(panel["causal"]) + assert ("swap_all" in panel["causal"]) == hd.TASKS[name].twins + + +@pytest.mark.parametrize("arm", ["Fd2", "FdT", "frozen"]) +def test_a_short_life_of_each_arm(arm): + row = hd.run_life({"task": "suppression", "arm": arm, "founder": 1, **SMALL}) + assert row["arm"] == arm + assert row["lessons"] == (0 if arm == "frozen" else 3) + conditions = row["panels"]["taught"]["conditions"]["taught"] + assert "first" in conditions + if arm == "Fd2": + assert "next1" in conditions["ports"] + + +def test_the_trace_silent_arm_holds_no_trace(): + task = hd.Suppression(0) + brain = hd.make_brain(task, "FdT", 0, module=16) + moments = task.train(np.random.default_rng(0)) + brain.reset() + brain.act(moments[0].x, greedy=True) + trace = brain.working_memory.trace + assert trace.any() # what a moment leaves, cleared before the next act and lesson + brain.act(moments[1].x, greedy=True) + assert copy.deepcopy(brain).__class__ is brain.__class__ + + +def test_a_replica_is_the_same_newborn_at_the_last_bits(): + task = hd.Suppression(0) + plain = hd.make_brain(task, "Fd", 0, module=16) + replica = hd.make_brain(task, "Fd", 0, module=16, replica=1) + assert not np.array_equal(_efficacy(plain), _efficacy(replica)) + assert np.allclose(_efficacy(plain), _efficacy(replica), rtol=1e-11) + + +@pytest.fixture(scope="module") +def nets(): + pytest.importorskip("torch") + return _load("nets") + + +def test_the_controls_are_the_grammar_chambers_networks_at_the_tasks_size(nets): + task = hd.IndirectObject(0) + for model in ("lstm", "transformer"): + with nets._dims(task): + direct = nets.ct.build(model, 20, 0) + ours = nets.build(model, 20, 0, task) + assert nets.count(direct) == nets.count(ours) + probe = np.zeros((1, 3, task.inputs), dtype=np.float32) + import torch + + out = ours(torch.from_numpy(probe)) + assert out.shape == (1, 3, task.actions) + assert nets.ct.gr is hd.gr + + +def test_a_controls_width_matches_the_brains_parameters(nets): + task = hd.IndirectObject(0) + target = nets.brain_parameters(task, 0) + for model in nets.MODELS: + width = nets.matched_width(model, target, task) + assert abs(nets.count(nets.build(model, width, 0, task)) - target) / target < 0.1 + + +def test_the_tied_control_names_a_token_by_its_own_input_weights(nets): + import torch + + task = hd.DuplicateBit(0) + net = nets.build("tied", 20, 0, task) + x = torch.from_numpy(np.zeros((1, 2, task.inputs), dtype=np.float32)) + assert net(x).shape == (1, 2, task.actions) + h = torch.rand(1, 20) + assert torch.allclose(h @ net.inp.weight[:, : task.words], + h @ net.inp.weight[:, net.coordinates]) + + +def test_the_pointer_control_reads_nothing_at_birth_and_only_earlier_moments(nets): + import torch + + task = hd.IndirectObject(0) + net = nets.build("pointer", 20, 0, task) + plain = nets.ct.gr + assert plain is hd.gr + x = torch.from_numpy(nets.encode(task.train(np.random.default_rng(0)))[0]) + with torch.no_grad(): + assert torch.allclose(net(x), net.out(net.body( + net.inp(x) + net.position[: x.shape[1]], + mask=torch.triu(torch.full((x.shape[1],) * 2, float("-inf")), diagonal=1), + is_causal=True))) + net.gain.data[:] = 5.0 + changed = net(x) + # a later moment's change does not reach an earlier moment's read + y = x.clone() + y[:, -1] = 0.0 + with torch.no_grad(): + same = net(y) + assert torch.allclose(changed[:, :-1], same[:, :-1], atol=1e-5) + + +def test_the_copy_control_says_only_what_it_attends(): + pytest.importorskip("torch") + nets = _load("nets") + import torch + + task = hd.IndirectObject(0) + net = nets.build("copy", 20, 0, task) + x = torch.zeros(1, 2, task.inputs) + x[0, 0, 5] = 1.0 + with torch.no_grad(): + logits = net(x)[0, 1] + # one moment before it, holding one token: that token's logit is the log of its whole attention + assert int(torch.argmax(logits)) == 5 + assert abs(float(logits[5])) < 1e-3 and float(logits[6]) < -10.0 + + +def test_the_pointer_control_copies_a_token_it_attends(): + pytest.importorskip("torch") + nets = _load("nets") + import torch + + task = hd.IndirectObject(0) + net = nets.build("pointer", 20, 0, task) + net.gain.data[:] = 3.0 + x = torch.zeros(1, 3, task.inputs) + x[0, 0, 5] = 1.0 # a token, then nothing + with torch.no_grad(): + with_token = net(x) + y = x.clone() + y[0, 0, 5] = 0.0 + without = net(y) + # at the second moment only the first moment can be read: its token is added at full gain + gap = (with_token - without)[0, 1, 5] - (with_token - without)[0, 1, 6] + assert gap > 1.0 + + +@pytest.mark.parametrize("model", ["lstm", "transformer", "tied", "pointer", "copy"]) +def test_a_short_control_life(nets, model): + row = nets.run_life({"task": "ioi", "model": model, "mode": "online", "founder": 0, + "lessons": 3, "panel_batches": 2}) + assert row["work"]["updates"] == 3 + assert row["parameters"] > 0 and row["panels"]["ioi_new"]["conditions"]["new"]["n"] > 0 + assert "blank_answer" in row["panels"]["ioi"]["causal"] + assert ("silence" in row["panels"]["ioi"]["causal"]) == (model != "lstm") + + +def test_equal_compute_runs_epochs_to_its_seconds(nets): + row = nets.run_life({"task": "suppression", "model": "lstm", "mode": "offline", "founder": 0, + "lessons": 3, "panel_batches": 1, "compute_seconds": 0.05, "epochs": 4}) + assert 1 <= row["epochs"] <= 4 and row["curve"][-1]["epoch"] == row["epochs"] + + +def test_a_closed_loop_control_hears_its_own_answer(nets): + task = hd.Copying(0) + learner = nets.Learner("lstm", 16, 0, 0.001, task) + moments = task.train(np.random.default_rng(0)) + said = learner.answers(moments) + assert len(said) == sum(m.label is not None for m in moments) + + +def test_blanking_a_moment_changes_what_a_control_hears(nets): + import torch + + task = hd.IndirectObject(0) + learner = nets.Learner("transformer", 20, 0, 0.001, task) + learner.net.eval() + moments = task.panels(1)["ioi"][0] + plain = learner.answers(moments)[0] + with torch.no_grad(): + x, _ = nets.encode(moments) + logits = learner.net(torch.from_numpy(x))[:, -1].argmax(dim=1).numpy() + assert np.array_equal(plain, logits) + assert learner.answers(moments, {"silence": True})[0].shape == plain.shape + silenced = learner.answers(moments, {"silence": True})[0] + assert np.array_equal(learner.answers(moments)[0], plain) # the hook comes off + assert isinstance(silenced, np.ndarray) diff --git a/benchmarks/sequence/README.md b/benchmarks/sequence/README.md index 44e9d47c..e252c26c 100644 --- a/benchmarks/sequence/README.md +++ b/benchmarks/sequence/README.md @@ -58,12 +58,12 @@ what do the earlier cues keep? | `recipe64` | `recipe` with `recall_position=64`: the table exact to 64 ages | | `recipe_relative` | `recipe` with `recall_relative=True` | | `heads` | `table` with `recall_heads=2` | -| `heads_copy` | `heads` with the copy pathway as the tokens' identity (`recall_copy=2.0, recall_tokens=range(16)`) | | `every_row` | `recall=True, recall_recency=0.0`: every row taught | | `when` | `every_row` with `recall_when=True` | | `when_anchor` | `when` with `recall_anchor=True` | | `when_sharp` | `when` with `recall_position_rate=0.0, recall_temperature=0.1`: the position term not taught and a sharp softmax, so the read lands on the item heard and stays there | | `rehearsal` | `anchor` with `learning_rehearsal=1.0`: the lessons' episodes replayed | +| `composed` | what a new brain composes: `Brain.compose(inputs, actions, modules=(64,), seed=founder, episodic=False)` with the recall memory at its composition and the composition's teaching point; `--settings` names a one-out, such as `{"teaching": "normalized"}` | ## Ordered recall (`order.py`) @@ -118,6 +118,41 @@ its class. `--memory` adds the recall memory with surprise lessons. Arms: `found width 64 (`fan_in_rates` in `width.py`). `--settings` adds options, such as `{"learning_surprise_share": true}`. +## Answers on tokens no lesson used (`held.py`) + +[`held.py`](held.py) reads R1 of the attention tracker +([#272](https://github.com/muellerberndt/cadence/issues/272)): six of the battery's tasks with some +tokens held out of every answer in training (look back, induction, n-back and two hops hear them and +never answer them; copying and a period of 8 never hear them), on the default composition with the +learned value path (`recall_gain=1.0`, every token declared). Each answer is read beside the read's own +prediction; where the brain misses a held answer, `votes` reads what drives the answer's motor neuron +and the chosen one (the association's synapses, the gains, the rest), and the support +`no_association_vote` reads the test again with the association's synapses onto the tokens at zero. + +Declared reading, six fresh founders (2720901 to 2720906), one 32-vCPU box, 1,500 lessons, bar .80 on +5 of 6 on every task: + +| Task | Held answers, per founder | At .80 | The read's best port at held answers | Association's drive where missed: answer / chosen | Held with the association silenced | +| --- | --- | --- | --- | --- | --- | +| Induction | 1.00 1.00 1.00 .98 .98 1.00 | 6 of 6 | 1.00 | -.23 to -.05 / .25 to .43 | 1.00 on 6 | +| A period of 8 | .98 .97 .98 .99 .99 .98 | 6 of 6 | .98 to .99 | -.63 to -.05 / .88 to 1.22 | .98 to .99 | +| Copying | .89 .83 .94 .88 .92 .93 | 6 of 6 | .89 to .96 | -.09 to .05 / .27 to .38 | 1.00 on 6 | +| Two hops | .76 .78 .78 .79 .78 .72 | 0 of 6 | .89 to .91 | -1.70 to -1.25 / .79 to 1.12 | .90 to .91 | +| N-back | .08 .03 .02 .08 .03 .04 | 0 of 6 | .47 to .65 | -3.25 to -.76 / .62 to .89 | .21 to .48 | +| Look back | .00 on 6 | 0 of 6 | .35 to .97 | -4.85 to -.65 / .42 to 4.73 | .00 .99 .02 .03 .99 .36 | + +Taught answers read .99 to 1.00 on induction, the period and n-back, .86 to .94 on copying, .87 to +.89 on two hops and .96 to 1.00 on look back on five founders (.37 on one). Three terms decide the misses: + +- **The vote.** Lessons on other actions push the association's drive on a held action down, to + -0.8 to -4.9 on n-back and look back, while the action it chooses takes +0.4 to +4.7. On two hops the + read names the held answer on .89 to .91 of moments and the vote keeps it at .72 to .79; silencing + the association's synapses onto the tokens lifts it to .90 and .91. +- **The read on n-back** names the held token on .47 to .65 of moments, so silencing the vote reaches + .21 to .48 only. +- **The read on look back** spreads from .35 to .97 by founder; on the two founders where it names + the held token on .89 and .97 of moments the silenced vote reads .99, and elsewhere .00 to .36. + ## Run From the repository root, with one BLAS thread per process: @@ -291,7 +326,8 @@ Copying repeated tokens (`repeat` with `unique` off) taught at 3 to 6 and read a | `recipe_relative` | 232100 to 232105 | 0.918 to 0.941 | 0.967 to 0.992 | 0.963 to 0.976 | Two hops asked in one moment (`twohop` without `think`; founders 0 to 2): `heads` 0.436 to -0.469 after 1,500 lessons and, with the copy pathway, `heads_copy` 0.473 to 0.502 after 3,000. +0.469 after 1,500 lessons and, with the copy pathway (`heads_copy`, which left with the pathway in +0.96.0), 0.473 to 0.502 after 3,000. A chained second head (its address taking what the first head read), an option tried on the release branch and withdrawn before release, read 0.439 to 0.458 beside `heads` and 0.506 to 0.533 beside `heads_copy` (0.03 higher on each founder). With a `THINK` moment the two heads diff --git a/benchmarks/sequence/battery.py b/benchmarks/sequence/battery.py index d752ca23..30b13c24 100644 --- a/benchmarks/sequence/battery.py +++ b/benchmarks/sequence/battery.py @@ -8,7 +8,9 @@ previous answer, and once more teacher-forced. The brain is ``Brain.compose(inputs, actions, modules=(64,), seed=founder, episodic=False, -teaching="normalized")`` with the arm's settings beside it (``ARMS``). The training episodes +teaching="normalized")`` with the arm's settings beside it (``ARMS``); the arm ``composed`` is +what a new brain composes, ``Brain.compose(inputs, actions, modules=(64,), seed=founder, +episodic=False)`` with the recall memory and its own teaching point. The training episodes come from ``SeedSequence([founder, 11])`` and the test episodes from ``SeedSequence([founder, 33])``, so every arm of a founder is taught and read on the same episodes. @@ -46,7 +48,6 @@ BOUNDARY = {**TABLE, "recall_boundary": True} RECIPE = {**BOUNDARY, "recall_anchor": True, "recall_slope": 0.1, "recall_position_rate": 0.01} HEADS = {**TABLE, "recall_heads": 2} -COPY = {"recall_copy": 2.0, "recall_tokens": list(range(16))} # two hops' 16 tokens WHEN = {"recall": True, "recall_founder": True, "recall_recency": 0.0} # every-row lessons ARMS: dict[str, dict[str, Any]] = { "table": TABLE, @@ -60,7 +61,6 @@ "recipe64": {**RECIPE, "recall_position": 64}, "recipe_relative": {**RECIPE, "recall_relative": True}, "heads": HEADS, - "heads_copy": {**HEADS, **COPY}, "every_row": WHEN, "when": {**WHEN, "recall_when": True}, "when_anchor": {**WHEN, "recall_when": True, "recall_anchor": True}, @@ -68,7 +68,13 @@ "when_sharp": {**WHEN, "recall_when": True, "recall_position_rate": 0.0, "recall_temperature": 0.1}, "rehearsal": {**BOUNDARY, "recall_anchor": True, "learning_rehearsal": 1.0}, + # what a new brain composes: the memory at its composition and the composition's teaching + "composed": {"recall": True}, + # what Brain.compose gives a new brain, nothing named (the battery's composition is not + # applied: ``make_brain`` names nothing but the backend and the arm's settings) + "default": {"recall": True}, } +COMPOSED = ("composed",) # the arms that take the composition's own teaching point @dataclass @@ -390,9 +396,13 @@ def make_brain(task: Task, arm: str, founder: int, settings: dict[str, Any] | No """``Brain.compose`` for ``task`` with the arm's settings, and any ``settings`` over them.""" import cadence as cd + if arm == "default": + return cd.Brain.compose(task.inputs, task.actions, modules=(module,), seed=int(founder), + **{"backend": "cpu", **ARMS[arm], **(settings or {})}) + point = {} if arm in COMPOSED else {"teaching": "normalized"} return cd.Brain.compose(task.inputs, task.actions, modules=(module,), seed=int(founder), - episodic=False, teaching="normalized", - **{"backend": "cpu", "recall": False, + episodic=False, + **{"backend": "cpu", "recall": False, **point, **{**ARMS[arm], **(settings or {})}}) @@ -500,6 +510,7 @@ def test() -> np.random.Generator: row: dict[str, Any] = { "task": spec["task"], "task_settings": task.settings, "arm": arm, "founder": founder, + "tag": spec.get("tag"), "settings": {**ARMS[arm], **spec.get("settings", {})}, "lessons": lessons, "tests": tests, "module": int(spec.get("module", MODULE)), "phases": [list(p) for p in phases], "batch": BATCH, diff --git a/benchmarks/sequence/held.py b/benchmarks/sequence/held.py new file mode 100644 index 00000000..6e8bf92b --- /dev/null +++ b/benchmarks/sequence/held.py @@ -0,0 +1,480 @@ +"""Answers on tokens no lesson used (R1 of the attention tracker, cadence#272). + +Six of the battery's tasks (``battery.py``) with some tokens held out of every answer in +training: ``lookback``, ``induction``, ``nback`` and ``twohop`` hear the held tokens in training and +never answer them (``unanswered``), ``repeat`` (copying unique tokens, closed loop) and +``periodic`` never hear them in training. Tests draw from every token, and a moment whose answer +is a held token is scored under ``held``, one whose answer is taught under ``taught``. A held +answer can only be said through the read: a learned output has never been raised for it. + +Each answer moment is read beside the read's own prediction: every port's read of the slots' +values and successors as the act used it, and the token it drives most (``read_best`` is the port +that names the answer most often). With ``votes`` each scored moment also reads what drives the +answer's motor neuron and the neuron the brain chose: the association's synapses, the read's +learned gains (``recall_gain``) and the rest (the motor's own synapses and its bias), so a held +answer the read names and the brain does not is located in the vote. + +Arms (``ARMS``) are settings of ``Brain.compose(..., recall=True)``, the default composition: +``default`` and ``gain`` (``recall_gain=1.0`` on every token). ``support`` names what the harness +does to a life to locate a term, no option of the library: ``no_association_vote`` reads the tests +with the association's synapses onto the tokens' motor neurons at zero, so the read and its gains +vote alone. ``gain_start`` in a life's settings is an emulation: every port's gain at that value +at birth in place of zero. + + PYTHONPATH=src python benchmarks/sequence/held.py --tasks nback lookback --arms gain \\ + --founders 0 1 2 --out DIR/held.jsonl --workers 6 + PYTHONPATH=src python benchmarks/sequence/held.py --table DIR/held.jsonl +""" + +from __future__ import annotations + +import argparse +import importlib.util +import json +import sys +import time +import warnings +from multiprocessing import get_context +from pathlib import Path +from typing import Any + +import numpy as np + +HERE = Path(__file__).resolve().parent + + +def _module(name: str, path: Path) -> Any: + if name in sys.modules: + return sys.modules[name] + spec = importlib.util.spec_from_file_location(name, path) + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + sys.modules[name] = module + spec.loader.exec_module(module) + return module + + +B = _module("sequence_battery", HERE / "battery.py") + +# the six held tasks of #262's confirmation, at its lengths +TASKS: dict[str, tuple[str, dict[str, Any]]] = { + "lookback": ("lookback", {"lengths": [5, 12], "test_lengths": [24, 60], "held": [3], + "held_mode": "unanswered"}), + "induction": ("induction", {"lengths": [8, 16], "test_lengths": [24, 60], "held": [6, 7], + "held_mode": "unanswered"}), + "nback": ("nback", {"lengths": [6, 12], "test_lengths": [24, 60], "held": [6, 7], + "held_mode": "unanswered"}), + "twohop": ("twohop", {"think": True, "held": [12, 13, 14, 15], "held_mode": "unanswered"}), + "copying": ("repeat", {"unique": True, "lengths": [3, 6], "test_lengths": [8, 12], + "held": [12, 13, 14, 15]}), + "periodic": ("periodic", {"held": [5]}), +} +ARMS: dict[str, dict[str, Any]] = { + "default": {}, + "gain": {"recall_gain": 1.0}, +} +SUPPORTS = ("no_association_vote",) +FOUNDERS = (0, 1, 2) + + +# ------------------------------------------------------------------------ held-out episodes + + +def tokens(task: Any, train: bool) -> np.ndarray: + held = set(task.held) + full = np.arange(task.vocab) + return np.array([t for t in full if t not in held]) if train else full + + +def draw(rng: np.random.Generator, toks: np.ndarray, size: Any) -> np.ndarray: + return toks[rng.integers(len(toks), size=size)] + + +def heard(task: Any, train: bool) -> np.ndarray: + """The tokens an episode may hear: every token where held tokens are heard in training + (``unanswered``), else the answer tokens.""" + return np.arange(task.vocab) if task.held_mode == "unanswered" else tokens(task, train) + + +def ep_lookback(self: Any, rng: np.random.Generator, train: bool) -> list[Any]: + lo, hi = self.lengths if train else self.test_lengths + length = int(rng.integers(lo, hi + 1)) + seq = draw(rng, heard(self, train), (B.BATCH, length)) + c = rng.integers(len(self.cues), size=B.BATCH) + where = {"first": 0, "last": length - 1, "second": 1} + pos = np.array([where[self.cues[i]] for i in c]) + seq[np.arange(B.BATCH), pos] = draw(rng, tokens(self, train), B.BATCH) + moments = [B.Moment(B.onehot(seq[:, t], self.inputs)) for t in range(length)] + label = seq[np.arange(B.BATCH), pos] + moments.append(B.Moment(B.onehot(self.vocab + c, self.inputs), label, length - 1 - pos, + [f"{self.cues[i]}" for i in c])) + return moments + + +def ep_nback(self: Any, rng: np.random.Generator, train: bool) -> list[Any]: + lo, hi = self.lengths if train else getattr(self, "test_lengths", self.lengths) + length = int(rng.integers(lo, hi + 1)) + seq = draw(rng, heard(self, train), (B.BATCH, length)) + k_index = rng.integers(len(self.ks), size=B.BATCH) + ks = np.array(self.ks)[k_index] + seq[np.arange(B.BATCH), length - ks] = draw(rng, tokens(self, train), B.BATCH) + moments = [B.Moment(B.onehot(seq[:, t], self.inputs)) for t in range(length)] + label = seq[np.arange(B.BATCH), length - ks] + moments.append(B.Moment(B.onehot(self.vocab + k_index, self.inputs), label, ks - 1, + [f"k{k}" for k in ks])) + return moments + + +def ep_induction(self: Any, rng: np.random.Generator, train: bool) -> list[Any]: + lo, hi = self.lengths if train else self.test_lengths + length = int(rng.integers(lo, hi + 1)) + toks = heard(self, train) + answers = tokens(self, train) + rows, ages = [], np.zeros(B.BATCH, dtype=int) + for r in range(B.BATCH): + a = int(rng.choice(toks)) + others = [t for t in toks if t != a] + seq = list(rng.choice(others, size=length - 1)) + i = int(rng.integers(0, length - 1 - self.gap_min)) + seq[i] = a + seq[i + 1] = int(rng.choice([t for t in answers if t != a])) + seq.append(a) + rows.append(seq) + ages[r] = (length - 1) - 1 - i + rows_a = np.array(rows) + moments = [B.Moment(B.onehot(rows_a[:, t], self.inputs)) for t in range(length - 1)] + label = np.array([rows_a[r][np.where(rows_a[r][:-1] == rows_a[r][-1])[0][0] + 1] + for r in range(B.BATCH)]) + moments.append(B.Moment(B.onehot(rows_a[:, -1], self.inputs), label, ages - 1, + [f"L{length}"] * B.BATCH)) + return moments + + +def ep_repeat(self: Any, rng: np.random.Generator, train: bool) -> list[Any]: + lo, hi = self.lengths if train else self.test_lengths + length = int(rng.integers(lo, hi + 1)) + toks = tokens(self, train) + if self.unique: + seq = np.stack([toks[rng.permutation(len(toks))[:length]] for _ in range(B.BATCH)]) + else: + seq = draw(rng, toks, (B.BATCH, length)) + moments = [B.Moment(B.onehot(np.full(B.BATCH, self.START), self.inputs))] + moments += [B.Moment(B.onehot(seq[:, t], self.inputs)) for t in range(length)] + moments.append(B.Moment(B.onehot(np.full(B.BATCH, self.GO), self.inputs), seq[:, 0], + np.full(B.BATCH, length - 1), [f"L{length}:p0"] * B.BATCH)) + for j in range(length - 1): + moments.append(B.Moment(B.onehot(seq[:, j], self.inputs), seq[:, j + 1], + np.full(B.BATCH, length - 1), [f"L{length}:p{j + 1}"] * B.BATCH, + feedback=True)) + return moments + + +def ep_twohop(self: Any, rng: np.random.Generator, train: bool) -> list[Any]: + n = self.pairs + toks = tokens(self, train) + hear = heard(self, train) + firsts = np.zeros((B.BATCH, n), dtype=int) + seconds = np.zeros((B.BATCH, n), dtype=int) + a = np.zeros(B.BATCH, dtype=int) + b = np.zeros(B.BATCH, dtype=int) + c = np.zeros(B.BATCH, dtype=int) + for r in range(B.BATCH): + if self.held_mode == "unanswered": + bc = toks[rng.permutation(len(toks))[:2]] + rest = [t for t in hear if t not in bc] + rest = list(np.array(rest)[rng.permutation(len(rest))]) + perm = np.array([rest[0], bc[0], bc[1]] + rest[1:]) + else: + perm = toks[rng.permutation(len(toks))] + a[r], b[r], c[r] = perm[:3] + fs = [a[r], b[r]] + list(perm[3:3 + n - 2]) + ss = [b[r], c[r]] + list(draw(rng, hear, n - 2)) + order = rng.permutation(n) + firsts[r], seconds[r] = np.array(fs)[order], np.array(ss)[order] + moments = [] + for i in range(n): + moments.append(B.Moment(B.onehot(firsts[:, i], self.inputs))) + moments.append(B.Moment(B.onehot(seconds[:, i], self.inputs))) + moments.append(B.Moment(B.onehot(a, self.inputs), b, None, ["hop1"] * B.BATCH)) + moments.append(B.Moment(B.onehot(np.full(B.BATCH, self.THINK), self.inputs), c, None, + ["hop2"] * B.BATCH)) + return moments + + +def ep_periodic(self: Any, rng: np.random.Generator, train: bool) -> list[Any]: + toks = tokens(self, train) + p = int(rng.choice(self.periods)) + motif = draw(rng, toks, (B.BATCH, p)) + length = p * self.repeats + seq = np.tile(motif, (1, self.repeats)) + vary = rng.random((B.BATCH, length)) < self.variation + vary[:, :p] = False + seq = np.where(vary, draw(rng, toks, (B.BATCH, length)), seq) + target = np.tile(motif, (1, self.repeats + 1)) + moments = [] + for t in range(length): + if t < p: + moments.append(B.Moment(B.onehot(seq[:, t], self.inputs))) + continue + moments.append(B.Moment(B.onehot(seq[:, t], self.inputs), target[:, t + 1], + np.full(B.BATCH, p - 2), [f"P{p}"] * B.BATCH)) + return moments + + +EPISODES = {"lookback": ep_lookback, "nback": ep_nback, "induction": ep_induction, + "repeat": ep_repeat, "twohop": ep_twohop, "periodic": ep_periodic} + + +def make_task(name: str) -> Any: + """The battery's task with its held tokens.""" + kind, args = TASKS[name] + args = dict(args) + held = tuple(args.pop("held")) + mode = args.pop("held_mode", "unheard") + longer = args.pop("test_lengths", None) if kind == "nback" else None + task = B.TASKS[kind](**args) + if longer: # n-back's tests at lengths no lesson reached + task.test_lengths = tuple(longer) + task.settings["test_lengths"] = list(longer) + task.held, task.held_mode = held, mode + task.settings.update(held=list(held), held_mode=mode) + task.episode = EPISODES[kind].__get__(task) + return task + + +def make_brain(task: Any, arm: str, founder: int, settings: dict[str, Any] | None = None, + module: int = B.MODULE) -> Any: + """The default composition for ``task``, each action ``j`` the token of observation ``j``.""" + import cadence as cd + + options = {"backend": "cpu", "recall": True, **ARMS[arm], **(settings or {})} + start = options.pop("gain_start", None) # an emulation: every port's gain at birth + if options.get("recall_gain") and "recall_tokens" not in options: + options["recall_tokens"] = list(range(task.actions)) + brain = cd.Brain.compose(task.inputs, task.actions, modules=(module,), seed=int(founder), + **options) + if start is not None: + brain.recall.gains = np.full_like(np.asarray(brain.recall.gains), float(start)) + return brain + + +# ------------------------------------------------------------------------------ the reading + + +def silence_association_vote(brain: Any, actions: int) -> None: + """The association's synapses onto the tokens' motor neurons at zero (a support).""" + graph = brain.connectome + pre, post = np.asarray(graph.pre), np.asarray(graph.post) + motor = np.asarray(graph.populations["motor"])[:actions] + association = np.asarray(graph.populations["association"]) + cut = np.isin(pre, association) & np.isin(post, motor) + learner = brain.learner + efficacy = np.where(cut, 0.0, np.asarray(learner.brain.efficacy)) + learner.brain = learner.brain.with_parameters(efficacy=efficacy) + + +def motor_votes(brain: Any, neurons: dict[str, np.ndarray]) -> dict[str, np.ndarray]: + """What drives each named motor neuron per row at the act just settled: the association's + synapses, the gains' drive and the rest (the motor's other synapses and its bias).""" + graph = brain.connectome + pre, post = np.asarray(graph.pre), np.asarray(graph.post) + association = np.zeros(graph.n, dtype=bool) + association[np.asarray(graph.populations["association"])] = True + weights = np.asarray(brain.brain.weights) + activation = np.atleast_2d(np.asarray(brain.basal_ganglia.state.activation)) + drive = np.atleast_2d(np.asarray(brain.basal_ganglia._drive)) + bias = np.asarray(brain.brain.bias) + out: dict[str, np.ndarray] = {} + rows = np.arange(len(activation)) + for name, neuron in neurons.items(): + cortex, rest = np.zeros(len(rows)), np.zeros(len(rows)) + for r in rows: + into = post == neuron[r] + parts = weights[into] * activation[r, pre[into]] + cortex[r] = parts[association[pre[into]]].sum() + rest[r] = parts[~association[pre[into]]].sum() + bias[neuron[r]] + out[f"{name}_association"] = cortex + out[f"{name}_gain"] = drive[rows, neuron[rows]] + out[f"{name}_rest"] = rest + return out + + +def run_episode(brain: Any, task: Any, moments: list[Any], teach: bool, closed: bool, + stats: dict[str, Any], votes: bool = False) -> None: + """The battery's ``run_episode``, with the read's prediction (and the vote) at scored + moments.""" + brain.reset() + previous = None + held = set(task.held) + vocab = task.vocab + motor = np.asarray(brain.connectome.populations["motor"]) + for m in moments: + x = m.x + if m.feedback and closed and previous is not None: + x = B.onehot(previous, task.inputs) + if teach and m.label is not None: + try: + brain.lesson(x, m.label) + except RuntimeError: + stats["refused"] += 1 + scored = not teach and m.label is not None + if scored: + values = np.array(brain.recall.values, copy=True) + nexts = np.array(brain.recall.nexts, copy=True) + try: + action = np.asarray(brain.act(x, greedy=True)).reshape(len(x), -1)[:, 0] + except RuntimeError: + stats["exhausted"] += 1 + action = np.full(len(x), -1) + if scored: + hit = action == m.label + conds = [("held" if int(m.label[r]) in held else "taught") for r in range(len(x))] + heads = brain.recall.attended_heads or () + reads = {} + for h, p in enumerate(heads): + if p is None or p.shape[1] != values.shape[1]: + continue + reads[f"recall{h}"] = np.argmax(np.einsum("bs,bsw->bw", p, values)[:, :vocab], 1) + reads[f"next{h}"] = np.argmax(np.einsum("bs,bsw->bw", p, nexts)[:, :vocab], 1) + voted = None + if votes: + chosen = np.where(action >= 0, action, 0) + voted = motor_votes(brain, {"answer": motor[m.label], "chosen": motor[chosen]}) + for r, cond in enumerate(conds): + cell = stats["cells"].setdefault(cond, {"n": 0, "answer": 0, "ports": {}, + "right_wrong": {}, "votes": {}}) + cell["n"] += 1 + cell["answer"] += int(hit[r]) + for port, pred in reads.items(): + ok = int(pred[r] == m.label[r]) + cell["ports"][port] = cell["ports"].get(port, 0) + ok + cell["right_wrong"][port] = cell["right_wrong"].get(port, 0) + int( + ok and not hit[r]) + if voted is not None and not hit[r]: + for key, value in voted.items(): + cell["votes"][key] = cell["votes"].get(key, 0.0) + float(value[r]) + cell["votes"]["missed"] = cell["votes"].get("missed", 0) + 1 + previous = action if m.label is not None else previous + + +def evaluate(brain: Any, task: Any, rng: np.random.Generator, episodes: int, closed: bool, + votes: bool = False) -> dict[str, Any]: + stats: dict[str, Any] = {"cells": {}, "refused": 0, "exhausted": 0} + for _ in range(episodes): + run_episode(brain, task, task.episode(rng, train=False), teach=False, closed=closed, + stats=stats, votes=votes) + out: dict[str, Any] = {"exhausted": stats["exhausted"]} + for cond, cell in sorted(stats["cells"].items()): + n = max(cell["n"], 1) + ports = {k: v / n for k, v in cell["ports"].items()} + best = max(ports, key=lambda k: ports[k]) if ports else None + row: dict[str, Any] = { + "n": cell["n"], "answer": round(cell["answer"] / n, 4), "best_port": best, + "read_best": round(ports[best], 4) if best else None, + "read_right_answer_wrong": round(cell["right_wrong"][best] / n, 4) if best else None, + } + missed = cell["votes"].pop("missed", 0) + if missed: + row["vote_missed"] = {k: round(v / missed, 3) for k, v in cell["votes"].items()} + row["missed"] = missed + out[cond] = row + return out + + +def run_life(spec: dict[str, Any]) -> dict[str, Any]: + """One life: ``task`` (a key of ``TASKS``), ``arm``, ``founder``, with ``settings`` over the + arm's, ``lessons``, ``tests``, ``module``, ``support`` and ``votes``.""" + warnings.simplefilter("ignore") + import copy + + import cadence as cd + + task = make_task(spec["task"]) + arm, founder = spec["arm"], int(spec["founder"]) + lessons, tests = int(spec.get("lessons", B.LESSONS)), int(spec.get("tests", B.TESTS)) + module = int(spec.get("module", B.MODULE)) + started = time.monotonic() + brain = make_brain(task, arm, founder, spec.get("settings"), module) + rng = np.random.default_rng(np.random.SeedSequence([founder, B.SEEDS["train"]])) + stats: dict[str, Any] = {"cells": {}, "refused": 0, "exhausted": 0} + for _ in range(lessons): + run_episode(brain, task, task.episode(rng, train=True), teach=True, closed=False, + stats=stats) + closed = task.name == "repeat" + votes = bool(spec.get("votes", True)) + + def test() -> np.random.Generator: + return np.random.default_rng(np.random.SeedSequence([founder, B.SEEDS["test"]])) + + row: dict[str, Any] = { + "task": spec["task"], "task_settings": task.settings, "arm": arm, "founder": founder, + "settings": {**ARMS[arm], **spec.get("settings", {})}, "lessons": lessons, + "tests": tests, "module": module, "cadence": cd.__version__, "tag": spec.get("tag"), + "refused_lessons": stats["refused"], "exhausted_train": stats["exhausted"], + "test": evaluate(copy.deepcopy(brain), task, test(), tests, closed, votes), + } + if brain.recall is not None and getattr(brain.recall, "gain", 0): + row["gains"] = [round(float(g), 4) for g in brain.recall.gains] + if "no_association_vote" in spec.get("support", ()): + fork = copy.deepcopy(brain) + silence_association_vote(fork, task.actions) + row["no_association_vote"] = evaluate(fork, task, test(), tests, closed) + row["seconds"] = round(time.monotonic() - started, 1) + return row + + +def table(rows: list[dict[str, Any]]) -> list[str]: + lines = ["| task | arm | founder | held | read held | taught | association vote, answer / " + "chosen | gain vote, answer / chosen | without the association's vote: held, " + "taught |", "|" + " --- |" * 9] + for r in sorted(rows, key=lambda r: (r["task"], r["arm"], r["founder"])): + held, taught = r["test"].get("held", {}), r["test"].get("taught", {}) + v = held.get("vote_missed") or {} + cut = r.get("no_association_vote", {}) + lines.append( + f"| {r['task']} | {r['arm']} | {r['founder']} | {held.get('answer', float('nan')):.2f}" + f" | {held.get('read_best') or 0:.2f} | {taught.get('answer', float('nan')):.2f} | " + f"{v.get('answer_association', float('nan')):.2f} / " + f"{v.get('chosen_association', float('nan')):.2f} | " + f"{v.get('answer_gain', float('nan')):.2f} / {v.get('chosen_gain', float('nan')):.2f}" + f" | {cut.get('held', {}).get('answer', float('nan')):.2f}, " + f"{cut.get('taught', {}).get('answer', float('nan')):.2f} |") + return lines + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__.split("\n", 1)[0]) + parser.add_argument("--tasks", nargs="*", default=list(TASKS), choices=list(TASKS)) + parser.add_argument("--arms", nargs="*", default=["gain"], choices=list(ARMS)) + parser.add_argument("--founders", nargs="*", type=int, default=list(FOUNDERS)) + parser.add_argument("--settings", type=json.loads, default={}, help="over the arms'") + parser.add_argument("--support", nargs="*", default=[], choices=SUPPORTS) + parser.add_argument("--plan", type=Path, help="a JSON list of life specs in place of the grid") + parser.add_argument("--out", type=Path) + parser.add_argument("--workers", type=int, default=1) + parser.add_argument("--table", type=Path) + args = parser.parse_args(argv) + if args.table: + rows = [json.loads(line) for line in args.table.read_text().splitlines() if line.strip()] + print("\n".join(table(rows))) + return 0 + if args.out is None: + parser.error("--out is required") + specs = json.loads(args.plan.read_text()) if args.plan else [ + {"task": t, "arm": a, "founder": f, "settings": args.settings, "support": args.support, + "tag": f"{t}/{a}/{f}"} for t in args.tasks for a in args.arms for f in args.founders] + done = set() + if args.out.exists(): + done = {json.loads(line).get("tag") for line in args.out.read_text().splitlines() if line} + specs = [s for s in specs if s.get("tag") is None or s.get("tag") not in done] + args.out.parent.mkdir(parents=True, exist_ok=True) + with args.out.open("a") as sink, get_context("spawn").Pool( + max(args.workers, 1), maxtasksperchild=1) as pool: + for row in pool.imap_unordered(run_life, specs, chunksize=1): + sink.write(json.dumps(row) + "\n") + sink.flush() + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/benchmarks/sequence/test_held.py b/benchmarks/sequence/test_held.py new file mode 100644 index 00000000..bb9a93d2 --- /dev/null +++ b/benchmarks/sequence/test_held.py @@ -0,0 +1,66 @@ +"""Answers on tokens no lesson used (benchmarks/sequence/held.py).""" + +from __future__ import annotations + +import importlib.util +import sys +from pathlib import Path + +import numpy as np +import pytest + +HERE = Path(__file__).resolve().parent +spec = importlib.util.spec_from_file_location("held_tokens", HERE / "held.py") +assert spec is not None and spec.loader is not None +held = importlib.util.module_from_spec(spec) +sys.modules["held_tokens"] = held +spec.loader.exec_module(held) + + +@pytest.mark.parametrize("name", list(held.TASKS)) +def test_no_training_answer_is_a_held_token_and_tests_ask_them(name: str) -> None: + task = held.make_task(name) + rng = np.random.default_rng(0) + hidden = set(task.held) + heard = set() + for _ in range(30): + for m in task.episode(rng, train=True): + heard |= {int(i) for i in np.argmax(m.x, axis=1) if i < task.vocab} + if m.label is not None: + assert not hidden & {int(v) for v in m.label} + if task.held_mode == "unanswered": + assert hidden <= heard + else: + assert not hidden & heard + asked = set() + for _ in range(60): + for m in task.episode(rng, train=False): + if m.label is not None: + asked |= {int(v) for v in m.label} + assert hidden & asked + + +def test_the_cut_silences_the_association_onto_the_tokens_alone() -> None: + task = held.make_task("nback") + brain = held.make_brain(task, "gain", 0) + graph = brain.connectome + pre, post = np.asarray(graph.pre), np.asarray(graph.post) + motor = np.asarray(graph.populations["motor"])[:task.actions] + cut = np.isin(pre, np.asarray(graph.populations["association"])) & np.isin(post, motor) + before = np.array(brain.learner.brain.efficacy) + held.silence_association_vote(brain, task.actions) + after = np.asarray(brain.learner.brain.efficacy) + assert cut.any() and not after[cut].any() + assert np.array_equal(after[~cut], before[~cut]) + + +def test_a_life_reads_held_and_taught_answers_and_the_vote() -> None: + row = held.run_life({"task": "nback", "arm": "gain", "founder": 0, "lessons": 5, + "tests": 3, "support": ["no_association_vote"]}) + test = row["test"] + assert {"held", "taught"} <= set(test) + assert len(row["gains"]) == 2 and test["held"]["best_port"] in ("recall0", "next0") + missed = test["held"].get("vote_missed") + if missed is not None: + assert {"answer_association", "chosen_association", "answer_gain"} <= set(missed) + assert "held" in row["no_association_vote"] diff --git a/benchmarks/vocabulary/README.md b/benchmarks/vocabulary/README.md new file mode 100644 index 00000000..47a444eb --- /dev/null +++ b/benchmarks/vocabulary/README.md @@ -0,0 +1,426 @@ +# Vocabulary chamber: the read's rule on thousands of words + +This chamber reads [#266](https://github.com/muellerberndt/cadence/issues/266): the +[grammar chamber](../grammar/README.md) holds 18 words and every reading of the read's rule stands +there. Here the same sentences, panels and scorer run at 200, 1,000 and 5,000 words, with raw codes, +a fixed code and a learned code; a word heard once is asked in another role beside the matched +networks; and a word of several pieces is said. It establishes no default. + +## The language + +`language.py` holds `Language(nouns, verbs)`, the chamber's three families and two grammars at any +size. Sizes go by class, two noun stems to a verb, each in two forms, with the chamber's six function +words unchanged. + +| Words | Noun stems | Verbs | Function words | Inputs | Actions | +| --- | --- | --- | --- | --- | --- | +| 18 | 4 | 2 | 6 | 35 | 18 | +| 200 | 65 | 32 | 6 | 308 | 200 | +| 1,000 | 331 | 166 | 6 | 1,508 | 1,000 | +| 5,000 | 1,665 | 832 | 6 | 7,508 | 5,000 | + +`Language(4, 2)` is the grammar chamber to the bit: panels, training batches and a composed life +(`test_vocabulary.py`). At any larger size the split holds the chamber's shares: about one +agent-number-verb triple in eight and one noun pair in six, by a hash of the founder's salt. A +word's own coordinate, a concept per stem and the chamber's eleven flags make the input. + +## The composition + +Every brain is the read's rule, `Fd`: `Brain.compose(inputs, actions, modules=(64,), episodic=False, +recall=True, recall_recency=0.0, learning_surprise=True, recall_boundary=True, recall_anchor=True, +recall_slope=0.1, recall_position_rate=0.01, recall_relative=True, teaching="deep", +backend="cpu")`, with the memory `Brain.compose` gives a new brain beside those settings (its grid, +capacity of 128 and carried read) and the address lesson as a contrast, the library's one law. The +readings of 6 November below were taken before the composed memory and the contrast law were the +library's, with a capacity of 64 and no grid. The working trace is built in every arm. Its block is +`w^2` synapses of `w^2 + (3 I + 2 A) w`, `I` the key width and `A` the motor neurons. + +| Arm | Key width | Setting | +| --- | --- | --- | +| raw | the inputs | none | +| fixed code | 128, 256, 512 at 200, 1,000, 5,000 words | `code=units` | +| learned code | 64, 128, 256 at 200, 1,000, 5,000 words | `learned_code=units`; an observation wider than the code keys the memories on the code alone | +| coded maps | the inputs; the maps read `units` | an emulation (`mapcode.py`): the read's maps read a fixed sparse code of the event, keys and values stay raw, a map holds `units * inputs` in place of `inputs ** 2` | + +## Words by their pieces + +`pieces.py` holds `PieceLanguage`: the chamber's clauses, brackets and paths with one motor neuron +per piece in place of one per word. A word is a stem of pieces and a number piece (`ref ##act ##or +##0`, `dog ##s`); a verb's last piece agrees with its agent. `##0` is a silent piece, so that every +form of a stem has one length and a batch's streams keep in step. `lexicon.py` makes the forms from +a coder's words and a ranked English list by byte-pair merges, words of at most four pieces. The +sentences carry no English meaning: only the forms and how they share pieces are real. + +| Lexicon | Stems | Pieces | Inputs (heard) | Actions | Brain's parameters | +| --- | --- | --- | --- | --- | --- | +| 200 words | 200 | 143 | 162 | 150 | 101,925 | +| 1,000 words | 1,000 | 391 | 410 | 398 | 345,709 | +| 5,000 words | 5,000 | 994 | 1,013 | 1,001 | 1,451,611 | + +A meaning names a word in one of two ways (`route`). `heard`: the word's pieces, one moment each, +every moment with its role flag, so a word the brain has never been taught can be asked as long as +its pieces are known. `concept`: one moment, the word's own concept coordinate and flags, as in the +grammar chamber, so only taught words can be asked and the brain has to know their pieces. + +A tenth of the stems are novel per founder: they stand in no training sentence, and every piece of +them does. `swap` adds a family: the world says `A verb B end` with no flags, the brain says `B verb +A end`, each noun in the role it was not heard in and the verb agreeing with its new agent. + +| Panel | Asks | +| --- | --- | +| `taught` | stems from training, zero to two distractors | +| `long` | three and four distractors | +| `novel_agent`, `novel_patient` | one novel stem in that role (`heard` only) | +| `novel_all` | every noun and the verb novel (`heard` only) | +| `swap`, `swap_novel` | heard in one role and said in another; taught stems, novel stems | +| `dyck` | the chamber's brackets | +| `path`, `path_novel`, `path_long` | chains over taught stems, chains over novel stems, three hops | + +## One hearing across episodes + +`hearing.py` reads the word language with `fresh` noun stems out of training. On a fork of a taught +life sixteen fresh stems are heard, one batch of sixteen taught sentences, eight as the agent and +eight as the patient. Then they are asked in new sentences in the same role and in the other one, +after 1, 2, 4, ... 512 hearings, with zero hearings as the baseline. `said` is the fresh word in its +place, closed loop; `exact` the whole sentence; `rank` the fresh word's rank among all outputs, +teacher-forced. `mention` asks the word in the episode that first says it, a path `X near wug end`. +A network hears the same batches: one Adam step per batch (equal exposure), or sixteen (nearer +equal compute). The tied form shares its embedding with its output, and the pretrained form, a +labelled control, is handed a table in which a stem's words and concept share a vector from the start. + +## Controls + +`controls.py` holds the chamber's LSTM and causal transformer over any language, their size the one +whose parameter count is nearest the brain's. One pass is one Adam step per batch at rate 0.003, the +brain's exposure. Replay runs the same stored batches in shuffled epochs at rate 0.001 until the +brain's seconds are spent, equal compute. They are independently scored controls and never part of +any brain's answer path. + +## Run + +```bash +python benchmarks/vocabulary/life.py --words 200 --founder 0 --checkpoints 2000 8000 --out DIR/raw.jsonl +python benchmarks/vocabulary/life.py --summary DIR/raw.jsonl +python benchmarks/vocabulary/lexicon.py --ranked RANKED.txt --words 5000 --pieces 400 --longest 6 --out LEX.json +python benchmarks/vocabulary/life.py --lexicon LEX.json --swap --pseudo 0.5 --founder 0 \ + --checkpoints 2000 8000 --settings '{"recall_capacity": 128}' --save DIR --out DIR/pieces.jsonl +python benchmarks/vocabulary/controls.py --model transformer --lexicon LEX.json --swap --pseudo 0.5 \ + --founder 0 --parameters 357837 --checkpoints 2000 8000 --seconds 7000 --out DIR/controls.jsonl +python benchmarks/vocabulary/hearing.py --speaker brain --words 200 --founder 0 --checkpoints 8000 --out DIR/h.jsonl +python benchmarks/vocabulary/diagnose.py --brain DIR/BRAIN.npz --lexicon LEX.json --swap --pseudo 0.5 --out DIR/d.json +python -m pytest benchmarks/vocabulary/test_vocabulary.py +``` + +`recall_capacity=128` holds a path of long stems: at the founder's 64 slots a path of six-piece words +drops its first fact before the sentence ends. + +`mapcode.py`, `chamber_gain`, `address_scale` and `cortex` (`install_cortex`) in `life.py` are +emulations on one process per life: they change no library file and a brain with them is not saved. +`terms` in a life's spec splits the first head's logits at every word whose content a heard moment +holds by source (`logit_terms`): the event against a slot's key, the address map's event rows, cortex +columns and bias, and the same three of the position maps, each at the right slot less the best +other slot, per kind of word and over the words the read misses. + +## Readings (6 November 2026) + +Development unless declared: founders 0 to 2, one process per life on a shared 64-thread box at a +load of about 80, so seconds are of that load. A row of one life is a screen. On three lives a +difference under about .1 is noise, on six lives under about .07. `Read` is the share where the +first head attends most to the word's own slot or the one before it, teacher-forced; `said` is the +share of those words said, teacher-forced. Brackets and paths are bimodal, and the tables give them +per life where they spread. + +**The read's rule on whole words.** It holds at 200 words and does not hold at 1,000. At 200 raw +words the read lands right at 1.00, 1.00, .98 and .98 by 8,000 batches and the nouns are said at .97 +to .99. What is left is the verb's number: its identity reads .98 and its agreement .65, and clauses +read .62 against the matched transformer's .96. At 1,000 raw words the read is right at 500 batches +and then falls and stays down, with words said at .00 at 8,000 batches, while the matched transformer +reads clauses 1.00 at the same exposure. At 5,000 raw words a life costs 11.4 s per batch and reads +nothing at 250 batches; the matched transformer reads .86 at 8,000 batches. + +**Words by their pieces.** With one motor neuron per piece the read holds on later and number pieces +at every size read here (.98 to 1.00), on first pieces at 400 tokens and fewer, and not at 994 (.00 +to .69 at 2,000 batches), and a word is said piece by piece in one composition. On six fresh founders, with half the training stems +made up on the spot, later pieces of a word never taught are said at .97 to .98 and its first pieces +at .93 to .98, teacher-forced. Whole sentences stand below the transformer on every panel but +the ones both read at .00: .32 against .87 on taught words, .21 against .58 on words never taught, .15 +against .84 on words heard in one role and said in another, .68 against .80 on brackets. On a saved +development brain the largest single cause is the verb's agreeing piece, which reads at chance. + +**One hearing.** Neither speaker says a word after one hearing at equal exposure. The brain takes +16 to 32 times the hearings of the transformer in the role heard, and more in another role. + +**The learned code.** At 64, 128 and 256 units every noun concept takes one code (cosine .9997 to +.9999): it keeps the class and loses the word, so the read and the lookup fail. With 308 units the +key joins the identity, and the code then shows no class (.008 within nouns) at 6.3 s per batch. + +### Whole words, by size + +| Words | Arm | Params | Batches | Lives | s/batch | Sweeps/act | Clauses | Held | Roles | Agree | Brackets | Paths | Read: agent, distractor, patient, verb | Said | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | +| 18 | raw | 24,384 | 2,000 | 1 | .12 | 61 | .98 | .89 | .94 | 1.00 | .41 | 1.00 | 1.00 1.00 1.00 .66 | 1.00 .98 1.00 1.00 | +| 200 | raw | 224,311 | 2,000 | 3 | .83 | 102 | .01 | .00 | .00 | .07 | .00 | .00 | .99 .96 .90 .17 | .85 .65 .59 .02 | +| 200 | raw | 224,311 | 8,000 | 3 | .73 | 91 | .62 | .68 | .57 | .65 | .00 .16 .64 | .00 .30 .05 | 1.00 1.00 .98 .98 | .99 .98 .97 .66 | +| 200 | raw | 224,311 | 16,000 | 1 | .61 | 79 | .56 | .50 | .50 | .66 | .48 | .00 | 1.00 1.00 .91 .92 | 1.00 .98 .88 .66 | +| 200 | first synapses at their 18-word magnitude | 224,311 | 2,000 | 3 | .59 | 135 | .10 | .11 | .14 | .16 | .33 .00 .09 | .01 | 1.00 .99 1.00 .94 | .97 .87 .90 .15 | +| 200 | first synapses at their 18-word magnitude | 224,311 | 4,000 | 3 | .53 | 127 | .51 | .58 | .52 | .53 | .39 .11 .42 | .02 | 1.00 1.00 1.00 1.00 | 1.00 .99 .99 .52 | +| 200 | fixed `code=128` | 83,551 | 8,000 | 1 | .58 | 82 | .36 | .19 | .23 | .75 | .25 | .00 | 1.00 .98 .88 1.00 | .89 .90 .52 .77 | +| 200 | coded maps, 64 units | 128,175 | 8,000 | 1 | .75 | 93 | .53 | .50 | .44 | .66 | .25 | .00 | 1.00 1.00 1.00 1.00 | .88 .88 .97 .66 | +| 200 | two heads | 411,313 | 8,000 | 1 | .56 | 90 | .59 | .50 | .66 | .61 | .12 | .00 | 1.00 1.00 1.00 1.00 | 1.00 1.00 .98 .61 | +| 200 | learned code, 64 | 73,235 | 2,000 | 2 | 1.1 | 81 | .00 | .00 | .00 | .01 | .00 | .00 | .66 .67 .52 .00; .25 .00 .00 .95 | .02 .00 .00 .00; .00 | +| 200 | learned code, 308, identity joined | 710,335 | 2,000 | 1 | 6.3 | 103 | .00 | .00 | .00 | .12 | .03 | .00 | .98 .98 .94 .42 | .80 .58 .33 .00 | +| 1,000 | raw | 2,870,711 | 8,000 | 1 | 1.68 | 39 | .00 | .00 | .00 | .00 | .03 | .00 | .11 .08 .17 .81 | .00 .00 .02 .00 | +| 1,000 | first synapses at their 18-word magnitude | 2,870,711 | 2,000 | 1 | 1.40 | 43 | .00 | .00 | .00 | .00 | .00 | .00 | .00 .00 .00 .55 | .00 .02 .00 .00 | +| 1,000 | fixed `code=256` | 228,991 | 2,000 | 1 | 1.17 | 40 | .00 | .00 | .00 | .00 | .00 | .00 | .00 .25 .28 .50 | .00 .00 .02 .00 | +| 1,000 | learned code, 128 | 338,275 | 1,000 | 1 | 1.87 | 61 | .00 | .00 | .00 | .00 | .00 | .00 | .14 .00 .41 .50 | .00 | +| 5,000 | raw | 59,302,711 | 250 | 1 | 11.4 | 32 | .00 | .00 | .00 | .00 | .00 | .00 | .30 .00 .00 .75 | .00 | +| 5,000 | fixed `code=512` | 774,175 | 1,000 | 1 | 2.91 | 32 | .00 | .00 | .00 | .00 | .00 | .00 | .00 .00 .00 1.00 | .00 | +| 5,000 | coded maps, 256 units | 4,231,023 | 250 | 1 | 4.05 | 32 | .00 | .00 | .00 | .00 | .00 | .00 | .30 .00 .00 .77 | .00 | +| 5,000 | learned code, 256 | 2,434,931 | 250 | 1 | 6.77 | 32 | .00 | .00 | .00 | .00 | .00 | .00 | .00 .00 .00 1.00 | .00 | + +The trace block is built in every row: it is `w^2` = 4,096 of the synapses, and at raw 5,000 words +the address map alone holds 56.4 M of the 59.3 M parameters. Memory is .3 GB at 200 and 1,000 words +(.42 GB raw at 1,000) and 1.9 GB raw at 5,000. Raw 5,000 words stop being practical on one process: a +batch of 16 takes 11.4 s, 2,000 batches 6.3 h. + +| Words | Control, one pass | Params | s/batch | Batches | Clauses | Held | Roles | Agree | Brackets | Paths | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | +| 200 | transformer, 3 lives | 220,826 | .014 | 2,000 | .90 | .92 | .88 | .95 | .09 | .04 | +| 200 | transformer, 3 lives | 220,826 | .015 | 8,000 | .96 | .97 | .96 | .97 | .61 .00 .73 | .13 | +| 200 | transformer, 3 lives | 220,826 | .018 | 32,000 | .99 | .99 | .99 | 1.00 | 1.00 .23 .92 | .19 | +| 200 | LSTM, 3 lives | 225,816 | .011 | 8,000 | .97 | .95 | .94 | 1.00 | 1.00 | .03 | +| 1,000 | transformer, 1 life | 668,940 | .025 | 8,000 | 1.00 | 1.00 | .98 | 1.00 | .56 | .16 | +| 1,000 | LSTM, 1 life | 667,000 | .020 | 8,000 | .58 | .48 | .45 | .98 | .77 | .00 | +| 5,000 | transformer, 1 life | 785,622 | .034 | 8,000 | .86 | .84 | .80 | .94 | .75 | .02 | + +The controls at 1,000 and 5,000 words are matched to the coded-map brain and the fixed-code brain, +far smaller than the raw brains. Equal compute: for its 8,000 batches a raw 200-word brain takes about +5,800 s and the transformer's one pass 75 to 120 s. Replaying the same stored batches for 5,000 s on +founder 0, the transformer reads clauses 1.00, roles 1.00 and brackets 1.00 from the second epoch +(143 s) and paths .78 from the sixteenth (1,467 s). + +**Where raw 1,000 words go.** The read on the agent reads 1.00 at 500 batches, .08 at 2,000 and .11 +at 8,000. Three causes were read, one life each, and none holds it: + +| Arm at raw 1,000 | Read: agent, distractor, patient, verb, at 2,000 batches | +| --- | --- | +| plain | .08 .15 .34 .48 | +| the chamber's gain on the first synapses (`chamber_gain`) | .00 .00 .00 .55 | +| a vocabulary that grows, 200 words for 2,000 batches, all by 5,000 | at 5,000: .00 .00 .00 .00 (.83 .73 .78 .00 at 2,000) | +| the address lesson's temperature by the key width (`address_scale`) | .95 .35 .77 .00 (.39 .10 .75 .00 at 1,000; .14 .00 .73 .11 at 1,500) | + +Three more cuts, one life each: `teaching="normalized"`, no relative floor, holds the read fixed at +1.00 .67 .20 .00 from 500 to 2,000 batches with words said at .00; a module of 128 (3.34 M) reads .53 +.02 .78 .00 at 1,000 with words said at .00; words named by their own coordinate, with no concept +coordinates and 1,008 motor neurons, read .00 on the nouns at 1,000 with words said at .00, where the +same form at 200 words reads 1.00 .90 .91 .00 and says .75 .31 .58 .00 at 2,000. None of the seven +brings a word said at 1,000 tokens above .02, where at 400 tokens or fewer words form by 2,000 batches, +so the failure follows the number of tokens, and it needs no concept coordinates. Whole-word brains at +1,000 words stay at 33 to 44 sweeps per act, a newborn's, where brains at 400 tokens or fewer reach 90 +to 150 and the 994-piece brain 80. The cause is open. + +**Steps and gain at 200 words.** Three times the deep point's steps is much worse (said .03 .04 .00 +against .85 .65 .59 at 2,000 batches, one life). A projection's first synapses scale by its fan, +`sqrt(6 / (pre + post))`, and at a one-hot word only one presynaptic neuron is active, so the drive +through it falls with the square root of the vocabulary. Restored to its magnitude at 18 words +(`chamber_gain`), on the same three founders at 2,000 batches, the read lands right at 1.00 .99 1.00 .94 +against .99 .96 .90 .17 and the words are said at .97 .87 .90 .15 against .85 .65 .59 .02. At 4,000 +batches clauses read .51 and the nouns are said at 1.00 .99 .99, where raw takes 8,000 for .62 and +.99 .98 .97. The verb stays at .52, so agreement is what the gain leaves. It adds no parameter and reads 127 +sweeps per act against 91. + +### Words by their pieces + +| Lexicon | Training | Arm | Params | s/batch | Batches | Lives | Taught | Never taught: agent, patient, all | Swap, never-taught swap | Brackets | Paths | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | +| 200 stems, 143 pieces | plain | brain | 102,379 | 1.24 | 8,000 | 1 | .23 | .02 .05 .00 | .20 .06 | .02 | .16 | +| | plain | transformer | 102,000 | .022 | 8,000 | 2 | .81 | .02 .04 .00 | .95 .00 | .66 | .23 | +| | half made up | brain, declared | 102,379 | 1.08 | 8,000 | 6 | .32 | .28 .28 .21 | .24 .15 | .68 | .08 | +| | half made up | transformer, declared | 102,000 | .016 | 8,000 | 6 | .86 | .75 .76 .58 | .88 .84 | .80 | .32 | +| | half made up | tied transformer, declared | 101,456 | .018 | 8,000 | 6 | .79 | .70 .68 .60 | .90 .87 | .62 | .25 | +| | half made up | LSTM, declared | 102,054 | .009 | 8,000 | 6 | .00 | .00 .00 .00 | .01 .00 | .91 | .01 | +| | half made up | brain, two heads | 180,189 | 1.18 | 8,000 | 1 | .30 | .39 .27 .42 | .44 .42 | .67 | .16 | +| 1,000 stems, 391 pieces | plain | brain | 346,659 | 3.43 | 2,000 | 1 | .03 | .00 .00 .00 | .00 .00 | .33 | .05 | +| | plain | transformer | 349,458 | .029 | 8,000 | 1 | .67 | .56 .52 .23 | .86 .22 | .64 | .83 | +| | half made up | brain | 346,659 | 2.09 | 3,000 | 1 | .00 | .00 .00 .00 | .00 .00 | .00 | .02 | +| 5,000 stems, 994 pieces | plain | brain | 1,453,767 | 5.17 | 2,000 | 1 | .00 | .00 .00 .00 | .00 .00 | .00 | .00 | +| | plain | transformer | 1,450,605 | .067 | 4,000 | 1 | .27 | .23 .12 .12 | .45 .22 | .25 | .00 | +| 5,000 stems, 400 pieces, up to six | plain | brain | 357,837 | 2.52 | 3,000 | 1 | .00 | .00 .02 .00 | .00 .00 | .23 | .06 | +| | plain | transformer | 361,397 | .028 | 3,000 | 1 | .20 | .00 .06 .00 | .42 .47 | .56 | .11 | +| | plain | transformer | 361,397 | .030 | 8,000 | 1 | .75 | .50 .61 .36 | .95 .95 | .86 | .20 | + +`Taught` at 5,000 stems over 400 pieces reads .00 with the agent's word said right on .70 of sentences +at 3,000 batches (the transformer .91). Five thousand words, `kubernetes` as `k ##ub ##er ##n ##et +##es`, fit in 407 motor neurons and 358 k parameters, against 5,000 motor neurons and 59.3 M at raw +codes. The read holds there on later and number pieces at 1.00 and on the nouns' first pieces at .81 to +.92, on taught words and much the same on words never taught. + +Teacher-forced, read then said: + +| Lexicon, batches | Agent first | Agent later | Number | Distractor first | Patient first | Patient later | Verb first | Verb later | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | +| 200, plain, 8,000, taught | 1.00 1.00 | .99 1.00 | 1.00 1.00 | .98 .96 | 1.00 1.00 | 1.00 .99 | .89 .84 | 1.00 1.00 | +| 200, plain, 8,000, never taught | .95 .91 | .96 .61 | .95 .97 | .92 .85 | .91 .80 | .95 .57 | .95 .06 | .99 .41 | +| 200, half made up, 8,000, never taught, 6 lives | .97 .98 | .99 .98 | .98 .98 | .91 .90 | .89 .93 | .99 .98 | .63 .54 | .95 .93 | +| 1,000, plain, 2,000, taught | .97 .86 | 1.00 .96 | 1.00 1.00 | .88 .73 | .94 .64 | 1.00 1.00 | .70 .09 | .99 .92 | +| 1,000, half made up, 3,000, never taught | .97 .28 | 1.00 .99 | 1.00 1.00 | .67 .19 | .64 .13 | 1.00 .98 | .23 .02 | 1.00 .98 | +| 5,000 over 994, plain, 2,000, taught | .31 .00 | .99 .01 | 1.00 .14 | .69 .00 | .17 .00 | 1.00 .01 | .00 .00 | .98 .02 | +| 5,000 over 400, plain, 3,000, taught | .92 .70 | 1.00 1.00 | 1.00 1.00 | .92 .56 | .81 .59 | 1.00 .99 | .41 .27 | .99 .99 | +| 5,000 over 400, plain, 3,000, never taught | .95 .56 | 1.00 .94 | 1.00 1.00 | 1.00 .88 | .58 .38 | .99 .92 | .41 .19 | .97 .91 | + +**Made-up stems.** With plain training the read lands right on a word never taught and the later +pieces are said at .41 to .65 at 8,000 batches: the cortex answers with the piece sequences it has +memorized. With half +the stems made up on the spot they are said at .97 to .98, and the transformer learns the same from +them (never-taught sentences .00 plain, .58 made up). The change is to training and not to the brain. + +**The value path's gain.** With `recall_gain` the later pieces are said at .82 to .99 at 2,000 batches +and first pieces at .02 to .36, with taught sentences .00 (one life): at a word's first piece the read +lands on that piece's own slot, and the next port's token, the second piece, outvotes the value port's. + +**Words by concept.** A meaning that names a word by its own concept asks the brain to know the word's +pieces. At 200 stems, 4,000 batches, one life (276,779 parameters): the agent's word reads .42 with its +first piece said at 1.00 and later pieces at .99 teacher-forced; the patient's first piece at .03 and +the verb's at .02; taught sentences .00. The transformer on the same route reads taught .62 and the +agent's word 1.00 at 8,000, and .50 at 4,000. + +**Where whole sentences go wrong.** On a saved brain (founder 0, made-up stems, 4,000 batches, 128 +sentences a panel, `diagnose.py`): + +| Panel | Right | First wrong word: agreeing piece | `near` | Verb first | Distractor first | Patient first | Teacher-forced: agreeing piece | +| --- | --- | --- | --- | --- | --- | --- | --- | +| taught | 31 | 39 | 28 | 15 | 11 | 0 | .53 | +| never taught | 62 | 44 | 7 | 6 | 5 | 1 | .61 | +| never-taught swap | 27 | 53 | 0 | 21 | 0 | 16 | .48 | + +Every other kind reads .71 to 1.00 teacher-forced. A verb that agrees with a singular agent says `##s`, +which the slots of plural nouns hold, and the read's lesson names a slot by what it holds, so the +shared token could mislead the lesson. On one life it does not: with tokens of its own (`##vs`, `##v0`) +agreement on taught sentences reads .22 at 3,000 batches against .22 with the shared piece at 4,000. +With two heads as well it reads .50 at 2,000 and .36 at 3,000, taught .42 and .34, never-taught .27 and +.22, never-taught swap .34 and .34, one life, against the declared transformer's taught .15 at 2,000 +and .73 at 4,000. On raw words a second head brings the read and the nouns to 1.00 and leaves agreement +at .61. Agreement is open. + +### The declared confirmation + +Declared before any run on them: founders 266000 to 266005, one box, 200 stems over 143 pieces, half +the training stems made up. A failed gate is reported as failed. + +| Gate, at 8,000 batches | Requirement | Reading | Verdict | +| --- | --- | --- | --- | +| 1. The read on a word never taught | each of four at least .90; their mean at least .90 on 5 of 6 | agent first .966, agent later .988, patient first .888, patient later .989; the mean .92 to .98 on 6 of 6 | failed, by .012 on the patient's first piece | +| 2. Said, teacher-forced | later pieces at least .90, first at least .80 | later .981 .969 .984; first .982 .932 | passed | +| 3. Never-taught whole sentences | within .05 of the transformer | brain .21 (.27 .19 .17 .27 .28 .11), transformer .58 (.53 .19 .64 .62 .81 .66) | failed | +| 4. Heard in one role, said in another, never taught | within .05 | brain .15 (.02 .06 .28 .08 .16 .28), transformer .84 (.69 .80 .84 .92 .86 .94) | failed | +| 5. Taught words | within .05 | brain .32 (.22 .36 .23 .36 .39 .33), transformer .87 (.75 .86 .91 .88 .95 .84) | failed | + +A declared brain took about 8,600 s for its 8,000 batches, at 113 sweeps per act and .034 s per +moment, with no refusal and no exhausted act. Agreement on taught sentences reads .28 to .41 against +the transformer's .89 to .97. + +At equal compute the gap widens. Replaying the same stored batches on two of the founders, the +transformer reads taught .98 and .98, never-taught .95 and .77, never-taught swap .92 and 1.00 at its +second epoch (239 s and 269 s), and 1.00 on all three, with paths at .98, at 7,000 s (62 and 60 epochs), +under the brain's 8,600 s. These replays are read beside the gates and carry none. + +### One hearing across episodes + +Sixteen fresh stems, on a fork of a taught life, founder 0 for the brain and founders 0 and 1 for the +networks, at a base of 8,000 batches (clauses: brain .75, transformer .99). Rank is the fresh word's +place among 200 outputs after one hearing, from its place before any. + +| Speaker | Rank at 1 hearing | Same role: 1, 16, 64, 256, 512 hearings | Other role: 1, 16, 64, 256, 512 hearings | +| --- | --- | --- | --- | +| brain | 164 to 166 | .00 .00 .09 .78 .94 | .00 .00 .03 .06 .19 | +| transformer | 130 to 98 | .00 1.00 .98 1.00 1.00 | .00 .12 .36 .59 .66 | +| tied transformer | 133 to 115 | .00 .62 .97 1.00 1.00 | .00 .00 .25 .36 .41 | +| pretrained embedding, a control | 18 to 17 | .03 .97 1.00 1.00 1.00 | .00 .28 .59 .70 .70 | +| LSTM | 184 to 182 | .00 .17 .98 1.00 1.00 | .00 .00 .00 .00 .00 | +| transformer, 16 steps a hearing | 130 to 6 | .64 1.00 .98 1.00 .98 | .14 .38 .53 .86 .97 | + +Sixteen Adam steps a hearing is about the brain's compute for that hearing. A word mentioned in the +episode that first says it, a path `X near wug end` with no lesson on the word, reads .00 for every +speaker at no hearing. A pretrained table places a fresh word near its class (rank 18 against 130) and +does not say it at one hearing either, so the embedding alone is not what is missing at this size. + +### The learned code + +| Units, words | Noun concepts: within-class cosine, pairs the key cannot tell apart | Noun words: within-class cosine | Active units, concept | +| --- | --- | --- | --- | +| 64, 200 | .9998, all | .81 | 13 | +| 128, 1,000 | .9999, all | .97 | 37 | +| 256, 5,000 | .9997, all | .35 | 86 | +| 308, 200, identity joined | .54 (the flag), none | .008 | 152 | + +A code narrower than the input keys the memories on the code alone, and there every noun concept reads +as one: the read cannot find a noun and the cortex cannot tell which one it read. That is the group +kept and the member lost, at every size read. Keeping the member takes a code at least as wide as the +input, where the key joins the identity. It then reads about as raw does at 2,000 batches at eight +times the seconds, and shows no class, so it gives a fresh word nothing. At this size and exposure the +learned code adds cost and no gain, on words and on similar words. + +## Declared reading at 1,000 words (9 November 2026) + +Six fresh founders (2660901 to 2660906), one 32-vCPU box, 1,000 raw words by class, the first grammar, +4,000 batches, the protocol written before any life. The brain is `Fd` on the composed memory, with +and without `recall_cortex_norm`; the matched transformer (2,929,812 parameters against the brain's +2,928,914) takes one Adam step per batch on the same batches. The gate, taught whole sentences within +.05 of the transformer on 5 of 6, is not met: + +| Learner | Taught clauses at 4,000, per founder | Within .05 of the transformer | +| --- | --- | --- | +| Transformer | .80 .92 .83 .97 .61 .38 | | +| `Fd` | .00 on 6 | 0 of 6 | +| `Fd`, cortex at unit norm | .00 on 6 | 0 of 6 | + +`locate` and `terms` read every word whose content a heard moment holds, teacher-forced. The ranges are +over the six founders; a term's gap is its logit at the right slot less at the best other slot, the +mean over the words the read misses. + +| Arm | Batches | The read lands right: agent, mod, patient, verb | Said | Misses | The two terms that carry the misses | Sweeps per act | +| --- | --- | --- | --- | --- | --- | --- | +| `Fd` | 500 | .30-1.00, .10-.54, .02-.73, .00 | .00 | .43-.75 | the position maps' cortex columns -1.74, the address map's cortex columns -1.57 | 21-26 | +| | 1,000 | .22-1.00, .25-.48, .64-.81, .00-.05 | .00 | .42-.68 | position cortex -2.46, cortex -.85 | 23-27 | +| | 2,000 | .09-1.00, .00-.48, .55-.81, .00-.42 | .00-.02 | .43-.71 | position cortex -1.28, cortex -1.05 | 26-30 | +| | 4,000 | .00-.80, .00-.40, .09-.70, .00-.81 | .00-.02 | .47-.83 | the raw key -1.40, cortex -1.09 | 28-38 | +| `Fd`, unit norm | 500 | .88-1.00, .25-.48, .56-.81, .00 | .00 | .43-.49 | the position maps' bias -.99, position cortex -.80 | 22-29 | +| | 1,000 | .06-.66, .00-.48, .44-.81, .00 | .00-.02 | .50-.85 | the position maps' event rows -.65, the raw key -.54 | 24-29 | +| | 2,000 | .00-.03, .00-.38, .02-.70, .00-.50 | .00-.02 | .71-.85 | the raw key -1.39, position event -.58 | 28-31 | +| | 4,000 | .00, .00, .00-.72, .00-.53 | .00 | .78-.87 | the raw key -1.42, position event -.50 | 30-33 | + +**The fall above 1,000 tokens, located.** + +- **No word is said.** Through 4,000 batches the brain says .00 to .02 of the words, on every founder + and both arms, where the read lands right on the agent's word on up to 1.00 of rows. An act settles in + 21 to 38 sweeps, near a newborn's, against 116 to 128 at 200 words from 2,000 batches (development), + so the motor + readout of a one-hot word over 1,000 actions is not formed at this exposure. +- **The read's selection falls to shared terms.** Each word's own rows of the address and position + maps learn only from that word's lessons, five times fewer at 1,000 words than at 200; the shared + columns learn from every lesson. The misses are carried first by the cortex columns of the position + and address maps (-1.6 to -2.5), and from 2,000 batches by the raw key itself: the event heard matches + an earlier slot of the same word (-1.4), which the word's own rows would have to outweigh. The cortex + at unit norm moves the early misses to the position maps' bias and event rows and the late ones to the + raw key, and the read falls faster: the agent's word lands right at .88 to 1.00 at 500 batches and at + .00 to .03 at 2,000. + +At 200 words (development, one life) the cortex at unit norm brings the verb's read to 1.00 by 2,000 +batches, where the read at raw codes reads .00 there, and clauses to .61 at 4,000 batches, against +.62 at raw codes at 8,000 in the readings of 6 November. + +## What this does and does not establish + +It establishes, on six fresh founders, that a word never taught is said piece by piece with its +previous pieces given, at .93 to .98, in one composition with one motor neuron per piece. In development +it reads that 5,000 words fit in about 400 motor neurons with the read holding on later and number +pieces at 1.00, that the read's rule at raw codes holds at 200 words and fails at 1,000, that the learned +code loses the word when it is narrower than the input, and that a pretrained table does not by itself +give a word in one hearing. + +It establishes no parity with the transformer: whole sentences stand below it wherever it reads above +.00, and at equal compute the gap widens. It leaves open the cause of the collapse at raw 1,000 words, +the verb's number at any size, and whether lookups form at 994 tokens, where they read .00 to .14 at +2,000 batches. It holds no reading of real English: the sentences are the chamber's and only the forms +and how they share pieces are real. Most rows are one life. Nothing here reads a spoken word: the pieces +are given as input coordinates, and hearing them from audio is another question. diff --git a/benchmarks/vocabulary/controls.py b/benchmarks/vocabulary/controls.py new file mode 100644 index 00000000..65dcbf67 --- /dev/null +++ b/benchmarks/vocabulary/controls.py @@ -0,0 +1,348 @@ +"""Next-word learners at matched information, at any vocabulary: the chamber's controls. + +The grammar chamber's LSTM and causal transformer (``benchmarks/grammar/controls.py``) over a +``Language``: they hear the brain's moments and are trained to predict each word, teacher-forced, +by cross-entropy, and are read on the brain's panels by the same scorer, closed loop, greedy. +Their size is the one whose parameter count is nearest the brain's ``parameters()``. + +- one pass (``online``): one Adam step per batch the brain is taught, rate 0.003, the brain's + exposure exactly; a reading at each of the brain's checkpoints. +- replay (``--seconds``): after the pass, the same stored batches in shuffled epochs at rate 0.001 + until the given seconds of training are spent, the brain's seconds for the same batches: equal + compute. A reading at the end, and at each doubling of the epochs. + +``--tied`` shares the embedding table between the input and the output for the words (the +output of a word is the dot product with its input row), the transformer's usual form for a +language model. ``--pretrained`` (with the tied form, a labelled control) fixes the embedding of +every word and concept to a table made before the life: a stem's words and concept share a +vector, each class a vector of its own, so a word no sentence used sits with its class and its +concept from the start. It is what a pretrained embedding would hand a network. + +They are independently scored controls and never part of the brain's answer path. + + python benchmarks/vocabulary/controls.py --model transformer --words 200 --founder 0 \\ + --parameters 224311 --checkpoints 500 2000 --seconds 3000 --out DIR/controls.jsonl +""" + +from __future__ import annotations + +import argparse +import json +import math +import sys +import time +from pathlib import Path +from typing import Any + +import numpy as np +from language import EVERYTHING, LEAN, SEEDS, TWINS, Language, Row, by_size +from pieces import PieceLanguage + +RATES = {"online": 0.003, "offline": 0.001} +CLIP = 1.0 +MAX_LENGTH = 128 + + +def encode(lang: Language, rows: list[Row]) -> tuple[np.ndarray, np.ndarray]: + """The moments the brain hears and the words it is taught, teacher-forced: ``(batch, moments, + inputs)`` and ``(batch, moments)``, with -100 where nothing is taught.""" + xs, ys = [], [] + for r in rows: + moments = [*r.pre, (lang.go,)] + [lang.spoken(w) for w in r.target[:-1]] + xs.append(lang.dense(moments)) + ys.append([-100] * len(r.pre) + list(r.target)) + return np.stack(xs).astype(np.float32), np.array(ys, dtype=np.int64) + + +def build(model: str, inputs: int, actions: int, width: int, seed: int, tied: bool = False) -> Any: + import torch + from torch import nn + + torch.manual_seed(int(seed)) + + class LSTM(nn.Module): # type: ignore[misc] + def __init__(self) -> None: + super().__init__() + self.cell = nn.LSTM(inputs, width, batch_first=True) + self.out = nn.Linear(width, actions) + + def forward(self, x: Any) -> Any: + h, _ = self.cell(x) + return self.out(h) + + def macs(self, length: int) -> int: + return length * (4 * width * (inputs + width) + width * actions) + + class Transformer(nn.Module): # type: ignore[misc] + LAYERS, HEADS = 2, 2 + + def __init__(self) -> None: + super().__init__() + self.inp = nn.Linear(inputs, width) + layer = nn.TransformerEncoderLayer(width, self.HEADS, 2 * width, dropout=0.0, + batch_first=True, norm_first=True) + self.body = nn.TransformerEncoder(layer, self.LAYERS, enable_nested_tensor=False) + self.tied = tied + self.out = None if tied else nn.Linear(width, actions) + self.bias = nn.Parameter(torch.zeros(actions)) if tied else None + position = torch.arange(MAX_LENGTH)[:, None] + scale = torch.exp(torch.arange(0, width, 2) * (-math.log(10000.0) / width)) + table = torch.zeros(MAX_LENGTH, width) + table[:, 0::2] = torch.sin(position * scale) + table[:, 1::2] = torch.cos(position * scale)[:, : width // 2] + self.register_buffer("position", table) + + def forward(self, x: Any) -> Any: + n = x.shape[1] + mask = torch.triu(torch.full((n, n), float("-inf")), diagonal=1) + h = self.body(self.inp(x) + self.position[:n], mask=mask, is_causal=True) + if self.tied: + # an action's output row is its word's input row (the first coordinates) + return h @ self.inp.weight[:, :actions] + self.bias + return self.out(h) + + def macs(self, length: int) -> int: + per_token = (inputs * width + self.LAYERS * 8 * width * width + width * actions) + attention = self.LAYERS * width * length * (length + 1) + return length * per_token + attention + + return LSTM() if model == "lstm" else Transformer() + + +def count(net: Any) -> int: + return int(sum(p.numel() for p in net.parameters())) + + +def matched_width(model: str, inputs: int, actions: int, target: int, tied: bool = False) -> int: + """The width whose parameter count is nearest ``target`` (even, for the transformer), by a + closed-form search with the count of a built network at the end.""" + + def params(width: int) -> int: + if model == "lstm": + return 4 * (inputs + width + 1) * width + (width + 1) * actions + layer = 8 * width * width + 4 * width + 2 * width * 2 + 2 * width + width * 2 + 2 * width + io = (inputs + 1) * width + (actions if tied else (width + 1) * actions) + return 2 * layer + io + + best = min(range(4, 4098, 2), key=lambda w: abs(params(w) - target)) + return best + + +def pretrained_table(lang: Language, width: int, seed: int) -> tuple[list[int], np.ndarray]: + """The fixed columns of a pretrained control over the word language: which coordinates, and + each one's vector, the stem's own vector and its class's, at unit length.""" + rng = np.random.default_rng(np.random.SeedSequence([seed, 266, 3])) + + def unit(v: np.ndarray) -> np.ndarray: + return v / np.linalg.norm(v) + + noun, verb, number = (unit(rng.normal(size=width)) for _ in range(3)) + coordinates: list[int] = [] + vectors: list[np.ndarray] = [] + for i in range(lang.nouns): + stem = unit(rng.normal(size=width)) + for coordinate, vector in ((lang.noun(i, 0), stem + noun), + (lang.noun(i, 1), stem + noun + number), + (lang.noun_concept0 + i, stem + noun)): + coordinates.append(coordinate) + vectors.append(unit(vector)) + for v in range(lang.verbs): + stem = unit(rng.normal(size=width)) + for coordinate, vector in ((lang.verb(v, 0), stem + verb), + (lang.verb(v, 1), stem + verb + number), + (lang.verb_concept0 + v, stem + verb)): + coordinates.append(coordinate) + vectors.append(unit(vector)) + return coordinates, np.stack(vectors, axis=1) + + +class Learner: + def __init__(self, lang: Language, model: str, width: int, seed: int, rate: float, + tied: bool = False, pretrained: bool = False) -> None: + self.lang = lang + self.net = build(model, lang.inputs, lang.actions, width, seed, tied) + self.opt = self.torch.optim.Adam(self.net.parameters(), lr=rate) + self.work = {"updates": 0, "tokens": 0, "macs": 0, "seconds": 0.0} + self.fixed: tuple[list[int], Any] | None = None + if pretrained: + coordinates, table = pretrained_table(lang, width, seed) + self.fixed = (coordinates, self.torch.from_numpy(table.astype(np.float32))) + self._hold() + + def _hold(self) -> None: + if self.fixed is not None: + with self.torch.no_grad(): + self.net.inp.weight[:, self.fixed[0]] = self.fixed[1] + + @property + def torch(self) -> Any: + import torch + + return torch + + def set_rate(self, rate: float) -> None: + for group in self.opt.param_groups: + group["lr"] = rate + + def update(self, rows: list[Row]) -> float: + torch = self.torch + started = time.monotonic() + x, y = encode(self.lang, rows) + logits = self.net(torch.from_numpy(x)) + loss = torch.nn.functional.cross_entropy(logits.reshape(-1, self.lang.actions), + torch.from_numpy(y).reshape(-1), + ignore_index=-100) + self.opt.zero_grad() + loss.backward() + torch.nn.utils.clip_grad_norm_(self.net.parameters(), CLIP) + self.opt.step() + self._hold() + self.work["updates"] += 1 + self.work["tokens"] += x.shape[0] * x.shape[1] + self.work["macs"] += 3 * x.shape[0] * self.net.macs(x.shape[1]) + self.work["seconds"] += time.monotonic() - started + return float(loss.item()) + + def speak(self, rows: list[Row], cap: int = 4) -> list[list[int]]: + """Closed loop, greedy: each row hears its own previous word after its ``force``.""" + torch, lang = self.torch, self.lang + n = len(rows) + said: list[list[int]] = [[] for _ in rows] + done = [False] * n + moments: list[list[tuple[int, ...]]] = [[*r.pre, (lang.go,)] for r in rows] + longest = max(len(r.target) for r in rows) + cap + with torch.no_grad(): + for j in range(longest): + dense = np.stack([lang.dense(m) for m in moments]).astype(np.float32) + action = self.net(torch.from_numpy(dense))[:, -1].argmax(dim=1).numpy() + self.work["macs"] += n * self.net.macs(dense.shape[1]) + for i, r in enumerate(rows): + said[i].append(int(action[i])) + if j >= r.force and action[i] == lang.end: + done[i] = True + given = j + 1 <= r.force and j < len(r.target) + moments[i].append(lang.spoken(r.target[j] if given else int(action[i]))) + if all(done): + break + out = [] + for i, r in enumerate(rows): + words = said[i][r.force:] + out.append(words[:words.index(lang.end) + 1] if lang.end in words else words) + return out + + +def reading(learner: Learner, tests: dict[str, list[list[Row]]], + names: tuple[str, ...]) -> dict[str, Any]: + learner.net.eval() + out = {name: learner.lang.read_speaker(learner.speak, None, tests[name], twins=name in TWINS) + for name in names} + learner.net.train() + return out + + +def run_life(spec: dict[str, Any]) -> None: + import torch + + torch.set_num_threads(1) + if spec.get("lexicon"): + lang: Any = PieceLanguage(spec["lexicon"], str(spec.get("route", "heard")), + bool(spec.get("swap")), None, bool(spec.get("start")), + float(spec.get("pseudo", 0.0)), bool(spec.get("verb_tokens"))) + else: + nouns, verbs = (int(spec["nouns"]), int(spec["verbs"])) if "nouns" in spec else by_size( + int(spec["words"])) + lang = Language(nouns, verbs, int(spec.get("fresh", 0)), bool(spec.get("verb_flag"))) + model, founder = spec["model"], int(spec["founder"]) + tied = bool(spec.get("tied")) + width = int(spec.get("width") or matched_width(model, lang.inputs, lang.actions, + int(spec["parameters"]), tied)) + learner = Learner(lang, model, width, founder, RATES["online"], tied) + split = lang.split(founder) + tests = lang.panels(founder, int(spec.get("panel_batches", 4))) + rng = np.random.default_rng(np.random.SeedSequence([founder, SEEDS["train"]])) + full = {int(c) for c in spec.get("full", [])} + sink = Path(spec["out"]) + sink.parent.mkdir(parents=True, exist_ok=True) + head = {"model": model, "tied": tied, "founder": founder, "words": lang.words, + "fresh": getattr(lang, "fresh", 0), "width": width, "parameters": count(learner.net), + "brain_parameters": int(spec.get("parameters", 0)), "torch": torch.__version__, + "tag": spec.get("tag", "")} + + def write(mode: str, batches: int, epochs: float, names: tuple[str, ...]) -> None: + begun = time.monotonic() + row = {**head, "mode": mode, "batches": batches, "epochs": round(epochs, 2), + "work": dict(learner.work), "panels": reading(learner, tests, names), + "reading_seconds": round(time.monotonic() - begun, 1), + "seconds_per_batch": round(learner.work["seconds"] + / max(1, learner.work["updates"]), 5)} + with sink.open("a") as handle: + handle.write(json.dumps(row) + "\n") + + def panel_names(everything: bool) -> tuple[str, ...]: + if isinstance(lang, PieceLanguage): + return tuple(tests) + return EVERYTHING if everything else LEAN + + stored: list[list[Row]] = [] + batch = 0 + for checkpoint in sorted({int(c) for c in spec.get("checkpoints", [2000])}): + while batch < checkpoint: + rows = lang.train_batch(rng, split) + if spec.get("seconds"): + stored.append(rows) + learner.update(rows) + batch += 1 + write("online", batch, 1.0, panel_names(checkpoint in full)) + if spec.get("seconds"): + limit = float(spec["seconds"]) + learner.set_rate(RATES["offline"]) + order = np.random.default_rng(np.random.SeedSequence([founder, 99])) + epochs, next_read = 1.0, 2.0 + while learner.work["seconds"] < limit: + for i in order.permutation(len(stored)): + learner.update(stored[int(i)]) + if learner.work["seconds"] >= limit: + break + epochs += 1 + if epochs >= next_read: + write("replay", batch, epochs, panel_names(False)) + next_read *= 2 + write("replay_end", batch, epochs, panel_names(True)) + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__.split("\n", 1)[0]) + parser.add_argument("--model", default="transformer", choices=("lstm", "transformer")) + parser.add_argument("--words", type=int, default=200) + parser.add_argument("--fresh", type=int, default=0) + parser.add_argument("--lexicon", default="") + parser.add_argument("--verb-flag", action="store_true") + parser.add_argument("--route", default="heard", choices=("heard", "concept")) + parser.add_argument("--swap", action="store_true") + parser.add_argument("--start", action="store_true") + parser.add_argument("--pseudo", type=float, default=0.0) + parser.add_argument("--verb-tokens", action="store_true") + parser.add_argument("--founder", type=int, default=0) + parser.add_argument("--parameters", type=int, default=0) + parser.add_argument("--width", type=int, default=0) + parser.add_argument("--tied", action="store_true") + parser.add_argument("--checkpoints", nargs="*", type=int, default=[2000]) + parser.add_argument("--full", nargs="*", type=int, default=[]) + parser.add_argument("--seconds", type=float, default=0.0) + parser.add_argument("--panel-batches", type=int, default=4) + parser.add_argument("--tag", default="") + parser.add_argument("--out", type=Path, required=True) + args = parser.parse_args(argv) + run_life({"model": args.model, "words": args.words, "fresh": args.fresh, + "lexicon": args.lexicon, "route": args.route, "swap": args.swap, + "start": args.start, "pseudo": args.pseudo, "verb_tokens": args.verb_tokens, + "verb_flag": args.verb_flag, + "founder": args.founder, "parameters": args.parameters, "width": args.width, + "tied": args.tied, "checkpoints": args.checkpoints, "full": args.full, + "seconds": args.seconds, "panel_batches": args.panel_batches, "tag": args.tag, + "out": str(args.out)}) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/benchmarks/vocabulary/diagnose.py b/benchmarks/vocabulary/diagnose.py new file mode 100644 index 00000000..14d1b97f --- /dev/null +++ b/benchmarks/vocabulary/diagnose.py @@ -0,0 +1,117 @@ +"""Where a saved brain's whole sentences go wrong, on the wordpiece language. + +Closed loop, each sentence's first wrong word: its kind (a word's first or later piece, its number +piece, ``near``, ``end``) and its slot, and what was said instead. Teacher-forced, every word by +kind, the ones the read has no slot for among them (``near``, ``end``, a verb's agreeing piece). + + python benchmarks/vocabulary/diagnose.py --brain DIR/brain --lexicon LEX --swap --founder 0 +""" + +from __future__ import annotations + +import argparse +import json +import sys +import warnings +from collections import Counter +from pathlib import Path + +import life +import numpy as np +from pieces import PieceLanguage + + +def kinds_of(lang: PieceLanguage, row: object) -> list[str]: + out = [] + target = row.target # type: ignore[attr-defined] + meta = row.meta # type: ignore[attr-defined] + for j, word in enumerate(target): + name = "grammar" + for slot, a, b in meta.get("slots", []): + if a <= j < b: + place = "number" if j == b - 1 else "first" if j == a else "later" + name = f"{slot}_{place}" + if word == lang.near: + name = "near" + if word == lang.end: + name = "end" + out.append(name) + return out + + +def said_as(lang: PieceLanguage, words: list[int], at: int) -> str: + """What kind of word stands where a sentence first goes wrong.""" + if at >= len(words): + return "(stopped)" + word = words[at] + if word == lang.near: + return "near" + if word == lang.end: + return "end" + if word in (lang.s, lang.zero): + return "##s or ##0" + if word >= lang.tokens_pieces: + return "bracket" + return "later piece" if lang.piece_names[word].startswith("##") else "first piece" + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__.split("\n", 1)[0]) + parser.add_argument("--brain", type=Path, required=True) + parser.add_argument("--lexicon", required=True) + parser.add_argument("--swap", action="store_true") + parser.add_argument("--pseudo", type=float, default=0.0) + parser.add_argument("--founder", type=int, default=0) + parser.add_argument("--panels", nargs="*", default=["taught", "novel_all", "swap_novel"]) + parser.add_argument("--out", type=Path, required=True) + args = parser.parse_args(argv) + warnings.simplefilter("ignore") + import cadence as cd + + lang = PieceLanguage(args.lexicon, "heard", args.swap, None, False, args.pseudo) + tests = lang.panels(args.founder, 8) + report: dict[str, object] = {} + for name in args.panels: + brain = cd.Brain.load(args.brain) + first: Counter[str] = Counter() + instead: Counter[str] = Counter() + by_shape: dict[str, list[int]] = {} + for rows in tests[name]: + said = life.speak(lang, brain, rows, teach=False) + for row, words in zip(rows, said, strict=True): + kinds = kinds_of(lang, row) + key = f"L{row.meta.get('L', 0)}" + by_shape.setdefault(key, []).append(int(words == row.target)) + wrong = next((j for j, w in enumerate(row.target) + if j >= len(words) or words[j] != w), None) + if wrong is None: + first["right"] += 1 + continue + first[kinds[wrong]] += 1 + instead[f"{kinds[wrong]} -> {said_as(lang, words, wrong)}"] += 1 + forced: dict[str, list[int]] = {} + for rows in tests[name]: + brain.reset() + for t in range(len(rows[0].pre)): + brain.act(lang.dense([r.pre[t] for r in rows]), greedy=True) + for j in range(len(rows[0].target)): + x = (lang.dense([(lang.go,)] * len(rows)) if j == 0 + else lang.dense([lang.spoken(r.target[j - 1]) for r in rows])) + action = np.asarray(brain.act(x, greedy=True)).reshape(len(x), -1)[:, 0] + for i, r in enumerate(rows): + k = kinds_of(lang, r)[j] + forced.setdefault(k, []).append(int(action[i] == r.target[j])) + report[name] = { + "sentences": sum(first.values()), + "first_wrong": first.most_common(), + "instead": instead.most_common(16), + "exact_by_shape": {k: round(float(np.mean(v)), 3) for k, v in sorted(by_shape.items())}, + "forced": {k: [round(float(np.mean(v)), 3), len(v)] for k, v in sorted(forced.items())}, + } + args.out.write_text(json.dumps(report, indent=1)) + print(json.dumps(report, indent=1)) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/benchmarks/vocabulary/hearing.py b/benchmarks/vocabulary/hearing.py new file mode 100644 index 00000000..1c87bbce --- /dev/null +++ b/benchmarks/vocabulary/hearing.py @@ -0,0 +1,279 @@ +"""One hearing: a word heard once in one role, used in another, against the matched networks. + +A life is taught on every stem but ``fresh`` of them. On a fork of it, sixteen fresh noun stems are +heard: one batch of sixteen taught sentences (``agent verb patient end``), each with its own fresh +stem in the singular, eight as the agent and eight as the patient, the other nouns taught ones. +Then the same stems are asked in new sentences, in the role they were heard in (``same``) and in the +other (``other``), by the same scorer, closed loop, greedy. The reading repeats after 2, 4, ... 512 +such batches, on new sentences of the same stems in the same roles, so it gives the exposures a +fresh word takes as well as what one hearing does; zero exposures is the baseline. + +- ``said``: the fresh word said in its place, closed loop. +- ``exact``: the whole sentence. +- ``rank``: teacher-forced, the fresh word's rank among all the motor neurons or output rows at its + place, 1 the best: what the reading moved where a word is unsaid. + +``mention`` asks a fresh word in the episode it is first heard in: a path ``X near wug end`` whose +fact the world says once. The word's own motor neuron or output row has had only the hearings of +the ladder, so at zero hearings a speaker says it only by a route that needs no lesson on that word. + +A network hears the same batches: one Adam step per batch at the pass's rate (equal exposure), or +``steps`` per batch (equal compute, by the seconds a brain's batch takes). + + python benchmarks/vocabulary/hearing.py --speaker brain --words 200 --fresh 16 --founder 0 \\ + --checkpoints 2000 8000 --out DIR/hearing.jsonl + python benchmarks/vocabulary/hearing.py --speaker transformer --parameters 224311 \\ + --words 200 --fresh 16 --founder 0 --checkpoints 2000 8000 --out DIR/hearing.jsonl +""" + +from __future__ import annotations + +import argparse +import copy +import json +import sys +import time +import warnings +from pathlib import Path +from typing import Any + +import controls +import life +import numpy as np +from language import SEEDS, Language, Row, by_size + +LADDER = (1, 2, 4, 8, 16, 32, 64, 128, 256, 512) +ROLES = ("agent", "patient") + + +def sentences(lang: Language, rng: np.random.Generator, plan: list[tuple[int, str]]) -> list[Row]: + """A batch of ``agent verb patient end``, each row with its fresh stem in its role.""" + rows = [] + for stem, role in plan: + other = (int(rng.integers(lang.trained)), int(rng.integers(2))) + agent = (stem, 0) if role == "agent" else other + patient = (stem, 0) if role == "patient" else other + rows.append(lang.clause(rng, 0, agent=agent, patient=patient, + v=int(rng.integers(lang.verbs)), mods=[])) + return rows + + +def mentioned(lang: Language, rng: np.random.Generator, plan: list[tuple[int, str]]) -> list[Row]: + """A batch of one-hop paths whose chain ends in the row's fresh word, heard once in a fact.""" + rows = [] + entities = 2 * lang.trained + for stem, _ in plan: + draws = [lang._entity(int(e)) for e in rng.permutation(entities)[:5]] + start, firsts, seconds = draws[0], draws[1:3], draws[3:5] + word = lang.noun(stem, 0) + facts = [(start, word), *zip(firsts, seconds, strict=True)] + pre: list[tuple[int, ...]] = [] + for i in rng.permutation(len(facts)): + x, y = facts[int(i)] + pre += [lang.heard(x), lang.heard(lang.near), lang.heard(y), lang.heard(lang.end)] + pre.append(lang.noun_moment(start % lang.nouns, start // lang.nouns, lang.c_from)) + rows.append(Row("path", pre, [start, lang.near, word, lang.end], + {"hops": 1, "chain": [start, word], "facts": len(facts)})) + return rows + + +def place(role: str) -> int: + """Where a fresh noun stands in ``agent verb patient end``, or in ``X near wug end``.""" + return 0 if role == "agent" else 2 + + +def plans(lang: Language) -> tuple[list[tuple[int, str]], list[tuple[int, str]]]: + """The heard stems with the roles they are heard in, and the same stems in the other role.""" + stems = list(range(lang.trained, lang.nouns))[:16] + heard = [(stem, ROLES[i % 2]) for i, stem in enumerate(stems)] + return heard, [(stem, ROLES[1 - i % 2]) for i, stem in enumerate(stems)] + + +def score(lang: Language, rows: list[Row], said: list[list[int]], + plan: list[tuple[int, str]]) -> dict[str, float]: + right, exact = [], [] + for r, s, (stem, role) in zip(rows, said, plan, strict=True): + at = place(role) + right.append(len(s) > at and s[at] == lang.noun(stem, 0)) + exact.append(s == r.target) + return {"said": float(np.mean(right)), "exact": float(np.mean(exact))} + + +class BrainSpeaker: + def __init__(self, lang: Language, brain: Any) -> None: + self.lang, self.brain = lang, copy.deepcopy(brain) + + def fork(self) -> BrainSpeaker: + return BrainSpeaker(self.lang, self.brain) + + def hear(self, rows: list[Row]) -> None: + life.speak(self.lang, self.brain, rows, teach=True) + + def say(self, rows: list[Row]) -> list[list[int]]: + return life.speak(self.lang, copy.deepcopy(self.brain), rows, teach=False) + + def ranks(self, rows: list[Row], plan: list[tuple[int, str]]) -> list[int]: + lang, brain = self.lang, copy.deepcopy(self.brain) + brain.reset() + for t in range(len(rows[0].pre)): + brain.act(lang.dense([r.pre[t] for r in rows]), greedy=True) + out = [0] * len(rows) + motor = np.asarray(brain.connectome.populations["motor"]) + for j in range(len(rows[0].target)): + x = (lang.dense([(lang.go,)] * len(rows)) if j == 0 + else lang.dense([lang.spoken(r.target[j - 1]) for r in rows])) + brain.act(x, greedy=True) + activity = np.asarray(brain.basal_ganglia.state.activation)[:, motor] + for i, (stem, role) in enumerate(plan): + if place(role) == j: + word = lang.noun(stem, 0) + out[i] = int((activity[i] > activity[i, word]).sum()) + 1 + return out + + +class NetworkSpeaker: + def __init__(self, lang: Language, learner: Any, steps: int = 1) -> None: + self.lang, self.learner, self.steps = lang, copy.deepcopy(learner), steps + + def fork(self) -> NetworkSpeaker: + return NetworkSpeaker(self.lang, self.learner, self.steps) + + def hear(self, rows: list[Row]) -> None: + for _ in range(self.steps): + self.learner.update(rows) + + def say(self, rows: list[Row]) -> list[list[int]]: + self.learner.net.eval() + said = self.learner.speak(rows) + self.learner.net.train() + return said + + def ranks(self, rows: list[Row], plan: list[tuple[int, str]]) -> list[int]: + torch = self.learner.torch + x, _ = controls.encode(self.lang, rows) + self.learner.net.eval() + with torch.no_grad(): + logits = self.learner.net(torch.from_numpy(x)).numpy() + self.learner.net.train() + out = [] + for i, (stem, role) in enumerate(plan): + at = len(rows[i].pre) + place(role) + word = self.lang.noun(stem, 0) + out.append(int((logits[i, at] > logits[i, at, word]).sum()) + 1) + return out + + +def ladder(lang: Language, speaker: Any, founder: int, tag: dict[str, Any], + rounds: tuple[int, ...] = LADDER, test_batches: int = 2) -> dict[str, Any]: + """The reading on a fork of ``speaker``, at no hearing and after each count of hearings.""" + heard, other = plans(lang) + fork = speaker.fork() + rng = np.random.default_rng(np.random.SeedSequence([founder, 266, 1])) + test_rng = np.random.default_rng(np.random.SeedSequence([founder, 266, 2])) + tests = {name: [sentences(lang, test_rng, plan) for _ in range(test_batches)] + for name, plan in (("same", heard), ("other", other))} + tests["mention"] = [mentioned(lang, test_rng, heard) for _ in range(test_batches)] + told = [(stem, "mention") for stem, _ in heard] + readings = [] + given, seconds = 0, 0.0 + + def read() -> None: + row: dict[str, Any] = {"hearings": given, "seconds": round(seconds, 3)} + for name, plan in (("same", heard), ("other", other), ("mention", told)): + said = [fork.say(b) for b in tests[name]] + scores = [score(lang, b, s, plan) for b, s in zip(tests[name], said, strict=True)] + ranks = [x for b in tests[name] for x in fork.ranks(b, plan)] + row[name] = {"said": float(np.mean([s["said"] for s in scores])), + "exact": float(np.mean([s["exact"] for s in scores])), + "rank": float(np.median(ranks)), "rank_mean": float(np.mean(ranks))} + readings.append(row) + + read() + for count in rounds: + begun = time.monotonic() + while given < count: + fork.hear(sentences(lang, rng, heard)) + given += 1 + seconds += time.monotonic() - begun + read() + return {**tag, "actions": lang.actions, "readings": readings} + + +def run_life(spec: dict[str, Any]) -> None: + warnings.simplefilter("ignore") + nouns, verbs = by_size(int(spec["words"])) + lang = Language(nouns, verbs, int(spec.get("fresh", 16))) + founder = int(spec["founder"]) + split = lang.split(founder) + tests = lang.panels(founder) + rng = np.random.default_rng(np.random.SeedSequence([founder, SEEDS["train"]])) + sink = Path(spec["out"]) + sink.parent.mkdir(parents=True, exist_ok=True) + kind = spec["speaker"] + checkpoints = sorted(int(c) for c in spec["checkpoints"]) + panels = ("clause", "roles", "path") + batch = 0 + if kind == "brain": + brain = life.make_brain(lang, "Fd", founder, spec.get("settings"), + int(spec.get("module", 64))) + parameters = int(brain.parameters()) + for checkpoint in checkpoints: + while batch < checkpoint: + life.speak(lang, brain, lang.train_batch(rng, split), teach=True) + batch += 1 + tag = {"speaker": kind, "settings": spec.get("settings", {}), "founder": founder, + "words": lang.words, "fresh": lang.fresh, "batches": batch, + "parameters": parameters, "steps": 1, "tag": spec.get("tag", "")} + base = {n: life.read_panel(lang, brain, tests[n], twins=n == "roles")["exact"] + for n in panels} + row = ladder(lang, BrainSpeaker(lang, brain), founder, {**tag, "base": base}) + with sink.open("a") as handle: + handle.write(json.dumps(row) + "\n") + return + import torch + + torch.set_num_threads(1) + tied = kind in ("tied", "pretrained") + model = "lstm" if kind == "lstm" else "transformer" + width = controls.matched_width(model, lang.inputs, lang.actions, int(spec["parameters"]), + tied) + learner = controls.Learner(lang, model, width, founder, controls.RATES["online"], tied, + kind == "pretrained") + for checkpoint in checkpoints: + while batch < checkpoint: + learner.update(lang.train_batch(rng, split)) + batch += 1 + base = {n: controls.reading(learner, tests, (n,))[n]["exact"] for n in panels} + for steps in spec.get("steps", [1]): + tag = {"speaker": kind, "width": width, "founder": founder, "words": lang.words, + "fresh": lang.fresh, "batches": batch, "parameters": controls.count(learner.net), + "steps": int(steps), "tag": spec.get("tag", ""), "base": base} + row = ladder(lang, NetworkSpeaker(lang, learner, int(steps)), founder, tag) + with sink.open("a") as handle: + handle.write(json.dumps(row) + "\n") + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__.split("\n", 1)[0]) + parser.add_argument("--speaker", default="brain", + choices=("brain", "transformer", "tied", "pretrained", "lstm")) + parser.add_argument("--words", type=int, default=200) + parser.add_argument("--fresh", type=int, default=16) + parser.add_argument("--founder", type=int, default=0) + parser.add_argument("--parameters", type=int, default=0) + parser.add_argument("--checkpoints", nargs="*", type=int, default=[2000]) + parser.add_argument("--settings", type=json.loads, default={}) + parser.add_argument("--steps", nargs="*", type=int, default=[1]) + parser.add_argument("--module", type=int, default=64) + parser.add_argument("--tag", default="") + parser.add_argument("--out", type=Path, required=True) + args = parser.parse_args(argv) + run_life({"speaker": args.speaker, "words": args.words, "fresh": args.fresh, + "founder": args.founder, "parameters": args.parameters, + "checkpoints": args.checkpoints, "settings": args.settings, "steps": args.steps, + "module": args.module, "tag": args.tag, "out": str(args.out)}) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/benchmarks/vocabulary/language.py b/benchmarks/vocabulary/language.py new file mode 100644 index 00000000..761bee0c --- /dev/null +++ b/benchmarks/vocabulary/language.py @@ -0,0 +1,625 @@ +"""The grammar chamber's language at any vocabulary, by class. + +``Language(nouns, verbs)`` holds ``nouns`` noun stems in the singular and the plural, ``verbs`` +verbs in the singular and the plural and the chamber's six function words (``near``, four +brackets, ``end``). Its input holds one coordinate per word and the concepts of the meaning: one +per noun stem and verb, and the chamber's eleven flags and bracket concepts. ``Language(4, 2)`` is +the chamber of ``benchmarks/grammar/grammar.py`` to the bit: the same coordinates and, from the +same seeds, the same sentences (a test holds it). + +Moments are kept as tuples of the coordinates that are on, because at thousands of words a dense +copy of every moment of every panel holds gigabytes; ``dense`` gives a batch's array. + +``fresh`` stems stay out of training and out of the ordinary panels: the one-hearing reading +(``hearing.py``) names them. ``verb_flag`` adds a ``VERB`` flag to the meaning's verb moment, the +class flag the meaning's nouns carry through their roles. + +Sizes by class, ``by_size``: the words are the chamber's mix, two noun stems to a verb. + +===== ===== ===== ========== ====== ========================= +words nouns verbs function inputs actions (one per word) +===== ===== ===== ========== ====== ========================= +18 4 2 6 35 18 +200 65 32 6 308 200 +1000 331 166 6 1508 1000 +5000 1665 832 6 7508 5000 +===== ===== ===== ========== ====== ========================= +""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from typing import Any + +import numpy as np + +BATCH = 16 +SEEDS = {"train": 11, "test": 33, "split": 7} +TRAIN_L, TEST_L = (0, 1, 2), (3, 4) # distractor phrases in a clause +TRAIN_DEPTH, TEST_DEPTH = 3, (4, 5) # the deepest nesting of a Dyck sentence +TRAIN_HOPS, TEST_HOPS = (1, 2), (3,) # facts chained by a path sentence +FAMILIES = {"clause": 0.5, "dyck": 0.25, "path": 0.25} + +Moment = tuple[int, ...] + + +def by_size(words: int) -> tuple[int, int]: + """The noun stems and verbs of a language of ``words`` words: two stems to a verb.""" + if words == 18: + return 4, 2 + pairs = (words - 6) // 2 + verbs = pairs // 3 + return pairs - verbs, verbs + + +def mix(*parts: int) -> int: + """A deterministic integer hash, for the split's membership at any size.""" + h = 1469598103934665603 + for part in parts: + h ^= int(part) & 0xFFFFFFFF + h = (h * 1099511628211) & 0xFFFFFFFFFFFFFFFF + h ^= h >> 29 + return h + + +@dataclass(frozen=True) +class Split: + """What one founder's training never shows. ``Language(4, 2)``: the chamber's two + agent-number-verb triples and one noun pair, which hold about one triple in eight and one + pair in six. Any larger language holds the same shares by a hash of the founder's salt.""" + + triples: tuple[tuple[int, int, int], ...] = () + pair: tuple[int, int] = (0, 0) + salt: int | None = None # None: the chamber's explicit form + + def held_triple(self, agent: tuple[int, int], v: int) -> bool: + if self.salt is None: + return (agent[0], agent[1], v) in self.triples + return mix(self.salt, 1, agent[0], agent[1], v) % 8 == 0 + + def held_pair(self, p: int, q: int) -> bool: + if p == q: + return False + if self.salt is None: + return {p, q} == set(self.pair) + return mix(self.salt, 2, min(p, q), max(p, q)) % 6 == 0 + + def held(self, agent: tuple[int, int], patient: tuple[int, int], v: int) -> bool: + return self.held_triple(agent, v) or self.held_pair(agent[0], patient[0]) + + +@dataclass +class Row: + """One stream's episode: heard moments, then the sentence from ``GO``.""" + + family: str + pre: list[Moment] + target: list[int] + meta: dict[str, Any] = field(default_factory=dict) + force: int = 0 # the target's first words given as heard (the read starts after them) + + +class Language: + """The chamber's three families and two grammars over a vocabulary of this size.""" + + def __init__(self, nouns: int = 4, verbs: int = 2, fresh: int = 0, + verb_flag: bool = False) -> None: + if not 0 <= fresh < nouns - 1: + raise ValueError("a language trains at least two noun stems") + self.nouns, self.verbs, self.fresh = int(nouns), int(verbs), int(fresh) + self.trained = self.nouns - self.fresh # the stems training and the panels draw from + self.spoken_verbs = self.verbs # the verbs they draw from (``grown``) + n, v = self.nouns, self.verbs + self.verb0 = 2 * n + self.near = 2 * n + 2 * v + self.open_round, self.close_round = self.near + 1, self.near + 2 + self.open_square, self.close_square = self.near + 3, self.near + 4 + self.end = self.near + 5 + self.words = self.end + 1 + self.opens = (self.open_round, self.open_square) + self.match = {self.open_round: self.close_round, self.open_square: self.close_square} + base = self.words + self.noun_concept0 = base + self.plural = base + n + self.agent, self.patient, self.mod = base + n + 1, base + n + 2, base + n + 3 + self.verb_concept0 = base + n + 4 + after = base + n + 4 + v + self.c_open_round, self.c_open_square, self.c_close = after, after + 1, after + 2 + self.c_from, self.dialect, self.go, self.speak = after + 3, after + 4, after + 5, after + 6 + self.inputs = after + 7 + self.verb_flag = None + if verb_flag: + self.verb_flag = self.inputs + self.inputs += 1 + self.actions = self.words + self.c_opens = (self.c_open_round, self.c_open_square) + + def grown(self, nouns: int, verbs: int) -> None: + """The stems and verbs a sentence draws from from here on: a vocabulary that grows in a + life, its words on their coordinates from the start.""" + self.trained = min(int(nouns), self.nouns - self.fresh) + self.spoken_verbs = min(int(verbs), self.verbs) + + # ------------------------------------------------------------------------------ moments + + def noun(self, i: int, plural: int) -> int: + return int(i) + self.nouns * int(plural) + + def verb(self, v: int, plural: int) -> int: + return self.verb0 + 2 * int(v) + int(plural) + + def is_noun(self, w: int) -> bool: + return 0 <= w < self.verb0 + + def is_verb(self, w: int) -> bool: + return self.verb0 <= w < self.near + + def heard(self, word: int) -> Moment: + return (int(word),) + + def spoken(self, word: int) -> Moment: + return (int(word), self.speak) + + def noun_moment(self, i: int, plural: int, *flags: int) -> Moment: + return (self.noun_concept0 + int(i), *((self.plural,) if plural else ()), *flags) + + def dense(self, moments: list[Moment]) -> np.ndarray: + x = np.zeros((len(moments), self.inputs)) + for row, moment in enumerate(moments): + x[row, list(moment)] = 1.0 + return x + + def say(self, words: list[int]) -> str: + return " ".join(self.word_name(w) for w in words) + + def word_name(self, w: int) -> str: + if self.is_noun(w): + return f"n{w % self.nouns}{'s' if w >= self.nouns else ''}" + if self.is_verb(w): + k = w - self.verb0 + return f"v{k // 2}{'p' if k % 2 else ''}" + return {self.near: "near", self.open_round: "(", self.close_round: ")", + self.open_square: "[", self.close_square: "]", self.end: "end"}.get(w, "?") + + # -------------------------------------------------------------------------------- split + + def split(self, founder: int) -> Split: + rng = np.random.default_rng(np.random.SeedSequence([int(founder), SEEDS["split"]])) + if self.trained == 4 and self.verbs == 2: + a, b = (int(i) for i in rng.permutation(4)[:2]) + triples = ((a, 1, 0), (b, 0, 1)) + p, q = (int(i) for i in rng.permutation(4)[:2]) + return Split(triples, (min(p, q), max(p, q))) + return Split(salt=int(rng.integers(2**31))) + + def held_pair(self, rng: np.random.Generator, split: Split) -> tuple[int, int]: + """A noun pair training never shows together: the chamber's one, or a draw.""" + if split.salt is None: + return split.pair + while True: + p, q = (int(i) for i in rng.integers(self.trained, size=2)) + if split.held_pair(p, q): + return min(p, q), max(p, q) + + # ------------------------------------------------------------------------------ families + + def clause(self, rng: np.random.Generator, L: int, *, agent: tuple[int, int] | None = None, + patient: tuple[int, int] | None = None, v: int | None = None, + mods: list[tuple[int, int]] | None = None, inverted: bool | None = None, + dialect: bool = False) -> Row: + agent = agent or (int(rng.integers(self.trained)), int(rng.integers(2))) + patient = patient or (int(rng.integers(self.trained)), int(rng.integers(2))) + v = int(rng.integers(self.spoken_verbs)) if v is None else v + mods = mods if mods is not None else [(int(rng.integers(self.trained)), + int(rng.integers(2))) for _ in range(L)] + inverted = bool(rng.integers(2)) if inverted is None else inverted + act: Moment = (self.verb_concept0 + v, *((self.dialect,) if dialect else ()), + *(() if self.verb_flag is None else (self.verb_flag,))) + a = self.noun_moment(*agent, self.agent) + p = self.noun_moment(*patient, self.patient) + m = [self.noun_moment(*mod, self.mod) for mod in mods] + pre = [act, p, a, *m] if inverted else [act, a, *m, p] + words = [self.noun(*agent)] + for mod in mods: + words += [self.near, self.noun(*mod)] + tail = ([self.noun(*patient), self.verb(v, agent[1])] if dialect + else [self.verb(v, agent[1]), self.noun(*patient)]) + meta = {"agent": list(agent), "patient": list(patient), "verb": v, + "mods": [list(x) for x in mods], "L": len(mods), "inverted": inverted, + "dialect": dialect, + "attract": any(x[1] != agent[1] for x in mods), + "patient_attract": patient[1] != agent[1]} + return Row("clause", pre, words + tail + [self.end], meta) + + def trained_clause(self, rng: np.random.Generator, split: Split, L: int, + dialect: bool) -> Row: + while True: + row = self.clause(rng, L, dialect=dialect) + m = row.meta + if not split.held(tuple(m["agent"]), tuple(m["patient"]), m["verb"]): + return row + + @staticmethod + def dyck_pattern(rng: np.random.Generator, length: int, max_depth: int, + min_depth: int = 1, nested: bool | None = None) -> list[int]: + """A prefix of opens (1) and closes (0) whose depth stays within ``max_depth``, ends + open, and reaches ``min_depth``.""" + while True: + if nested: + return [1] * length + depth, seq, deepest = 0, [], 0 + for _ in range(length): + can_open, can_close = depth < max_depth, depth > 0 + step = 1 if not can_close else 0 if not can_open else int(rng.random() < 0.6) + depth += 1 if step else -1 + deepest = max(deepest, depth) + seq.append(step) + if depth > 0 and deepest >= min_depth and (nested is None or (0 in seq) != nested): + return seq + + def dyck(self, rng: np.random.Generator, pattern: list[int], + types: list[int] | None = None) -> Row: + types = types if types is not None else [int(rng.integers(2)) for _ in pattern] + pre: list[Moment] = [] + words: list[int] = [] + stack: list[int] = [] + for step, t in zip(pattern, types, strict=True): + if step: + pre.append((self.c_opens[t],)) + words.append(self.opens[t]) + stack.append(self.opens[t]) + else: + pre.append((self.c_close,)) + words.append(self.match[stack.pop()]) + depth, deepest = 0, 0 + for step in pattern: + depth += 1 if step else -1 + deepest = max(deepest, depth) + meta = {"prefix": len(pattern), "depth": deepest, "open": len(stack), + "nested": 0 not in pattern, "types": [int(t) for t in types]} + return Row("dyck", pre, words + [self.match[o] for o in reversed(stack)] + [self.end], + meta) + + def path(self, rng: np.random.Generator, hops: int, distractors: int = 2) -> Row: + """Facts ``X near Y end`` with one chain of ``hops`` from the start to a sink.""" + entities = 2 * self.trained + order = [int(i) for i in rng.permutation(entities)] + order = [self._entity(e) for e in order] + chain, rest = order[:hops + 1], order[hops + 1:] + facts = [(chain[i], chain[i + 1]) for i in range(hops)] + firsts = [int(i) for i in rng.permutation(rest)[:distractors]] + for f in firsts: + others = [self._entity(e) for e in range(entities) if self._entity(e) != f] + facts.append((f, int(rng.choice(others)))) + pre: list[Moment] = [] + for i in rng.permutation(len(facts)): + x, y = facts[int(i)] + pre += [self.heard(x), self.heard(self.near), self.heard(y), self.heard(self.end)] + start = chain[0] + pre.append(self.noun_moment(start % self.nouns, start // self.nouns, self.c_from)) + words = [chain[0]] + for e in chain[1:]: + words += [self.near, e] + return Row("path", pre, words + [self.end], + {"hops": hops, "chain": chain, "facts": len(facts)}) + + def _entity(self, e: int) -> int: + """The word of the ``e``-th trained noun form: singulars, then plurals.""" + return e if e < self.trained else e - self.trained + self.nouns + + def train_batch(self, rng: np.random.Generator, split: Split, grammar: str = "g1", + size: int = BATCH) -> list[Row]: + """One training batch: one family and one shape, streams of their own content.""" + if grammar == "g2": + L = int(rng.choice(TRAIN_L)) + return [self.trained_clause(rng, split, L, True) for _ in range(size)] + family = str(rng.choice(list(FAMILIES), p=list(FAMILIES.values()))) + if family == "clause": + L = int(rng.choice(TRAIN_L)) + return [self.trained_clause(rng, split, L, False) for _ in range(size)] + if family == "dyck": + pattern = self.dyck_pattern(rng, int(rng.integers(1, 7)), TRAIN_DEPTH) + return [self.dyck(rng, pattern) for _ in range(size)] + hops = int(rng.choice(TRAIN_HOPS)) + return [self.path(rng, hops) for _ in range(size)] + + def solve(self, pre: list[Moment]) -> list[int]: + """The sentence an exact solver says from the heard moments alone: the instrument's + check that every sentence is answerable from what the brain hears.""" + on = [set(moment) for moment in pre] + + def named(moment: set[int]) -> int: + stem = next(c - self.noun_concept0 for c in moment + if self.noun_concept0 <= c < self.plural) + return self.noun(stem, self.plural in moment) + + if any(self.c_from in o for o in on): + here = named(on[-1]) + words = [next(iter(o)) for o in on[:-1]] + facts = {words[i]: words[i + 2] for i in range(0, len(words), 4)} + said = [here] + while here in facts: + here = facts[here] + said += [self.near, here] + return said + [self.end] + if any(o & {*self.c_opens, self.c_close} for o in on): + said, stack = [], [] + for o in on: + if self.c_close in o: + said.append(self.match[stack.pop()]) + else: + stack.append(self.open_round if self.c_open_round in o else self.open_square) + said.append(stack[-1]) + return said + [self.match[o] for o in reversed(stack)] + [self.end] + v = next(c - self.verb_concept0 for c in on[0] + if self.verb_concept0 <= c < self.verb_concept0 + self.verbs) + nouns, mods = {}, [] + for o in on[1:]: + word = named(o) + if self.mod in o: + mods.append(word) + else: + nouns["agent" if self.agent in o else "patient"] = word + agent, patient = nouns["agent"], nouns["patient"] + said = [agent] + for m in mods: + said += [self.near, m] + form = self.verb(v, agent >= self.nouns) + tail = [patient, form] if self.dialect in on[0] else [form, patient] + return said + tail + [self.end] + + # -------------------------------------------------------------------------------- panels + + def twin(self, row: Row, **change: Any) -> Row: + """The clause of ``row`` with one part of its meaning changed.""" + m = row.meta + spec = {"agent": tuple(m["agent"]), "patient": tuple(m["patient"]), "v": m["verb"], + "mods": [tuple(x) for x in m["mods"]], "inverted": m["inverted"], + "dialect": m["dialect"]} + spec.update(change) + return self.clause(np.random.default_rng(0), len(spec["mods"]), **spec) + + @staticmethod + def removed(row: Row, keep: int) -> Row: + """The context removed: no meaning, ``GO``, then the recent words as heard.""" + recent = row.target[row.force - keep:] + return Row(row.family, [], recent, {**row.meta, "removed": True}, force=keep) + + def panels(self, founder: int, batches: int = 4) -> dict[str, list[list[Row]]]: + """The founder's frozen test panels, each ``batches`` batches of one shape.""" + split = self.split(founder) + rng = np.random.default_rng(np.random.SeedSequence([int(founder), SEEDS["test"]])) + out: dict[str, list[list[Row]]] = {} + + def held_clause(L: int) -> Row: + while True: + row = self.clause(rng, L) + m = row.meta + if split.held(tuple(m["agent"]), tuple(m["patient"]), m["verb"]): + return row + + def train_shape(i: int) -> int: + return TRAIN_L[i % len(TRAIN_L)] + + out["clause"] = [[self.trained_clause(rng, split, train_shape(i), False) + for _ in range(BATCH)] for i in range(batches)] + out["clause_held"] = [[held_clause(train_shape(i)) for _ in range(BATCH)] + for i in range(batches)] + out["clause_long"] = [[self.trained_clause(rng, split, TEST_L[i % len(TEST_L)], False) + for _ in range(BATCH)] for i in range(batches)] + roles = [] # a held pair in both roles, side by side: the same words, the roles swapped + for i in range(batches): + rows = [] + for _ in range(BATCH // 2): + p, q = self.held_pair(rng, split) + base = self.clause(rng, train_shape(i), agent=(p, int(rng.integers(2))), + patient=(q, int(rng.integers(2)))) + if rng.random() < 0.5: + base = self.twin(base, agent=tuple(base.meta["patient"]), + patient=tuple(base.meta["agent"])) + rows += [base, self.twin(base, agent=tuple(base.meta["patient"]), + patient=tuple(base.meta["agent"]))] + roles.append(rows) + out["roles"] = roles + out["g2"] = [[self.trained_clause(rng, split, train_shape(i), True) + for _ in range(BATCH)] for i in range(batches)] + out["g2_long"] = [[self.trained_clause(rng, split, TEST_L[i % len(TEST_L)], True) + for _ in range(BATCH)] for i in range(batches)] + out["dyck"] = [] + out["dyck_deep"] = [] + for i in range(batches): + nested = i % 2 == 0 + length = int(rng.integers(1 if nested else 3, 4 if nested else 7)) + pattern = self.dyck_pattern(rng, length, TRAIN_DEPTH, nested=nested) + out["dyck"].append([self.dyck(rng, pattern) for _ in range(BATCH)]) + depth = TEST_DEPTH[(i // 2) % len(TEST_DEPTH)] + length = depth if nested else int(rng.integers(depth + 2, depth + 5)) + pattern = self.dyck_pattern(rng, length, depth, min_depth=depth, nested=nested) + out["dyck_deep"].append([self.dyck(rng, pattern) for _ in range(BATCH)]) + out["path"] = [[self.path(rng, TRAIN_HOPS[i % len(TRAIN_HOPS)]) for _ in range(BATCH)] + for i in range(batches)] + out["path_long"] = [[self.path(rng, TEST_HOPS[0]) for _ in range(BATCH)] + for _ in range(batches)] + + # the context: twins share their recent words and differ in an earlier part + for name, change in (("ctx_agent", "agent"), ("ctx_verb", "verb")): + rows_all = [] + for i in range(batches): + L = 1 + i % 2 + rows = [] + for _ in range(BATCH // 2): + base = self.trained_clause(rng, split, L, False) + m = base.meta + for _ in range(64): + other = (self.twin(base, agent=(m["agent"][0], 1 - m["agent"][1])) + if change == "agent" else + self.twin(base, v=self._other_verb(m["verb"]))) + om = other.meta + if not split.held(tuple(om["agent"]), tuple(om["patient"]), om["verb"]): + break + base = self.trained_clause(rng, split, L, False) + m = base.meta + for r in (base, other): + r.force = 1 + 2 * L # the agent and the distractors are given + rows += [base, other] + rows_all.append(rows) + out[name] = rows_all + out[name + "_removed"] = [[self.removed(r, 2 * (1 + i % 2)) for r in rows] + for i, rows in enumerate(rows_all)] + rows_all = [] + for i in range(batches): + depth = 2 + i % 2 + rows = [] + for _ in range(BATCH // 2): + types = [int(t) for t in rng.integers(2, size=depth)] + for first in (types[0], 1 - types[0]): + r = self.dyck(rng, [1] * depth, [first, *types[1:]]) + r.force = depth # the opens are given; the closes are the brain's + rows.append(r) + rows_all.append(rows) + out["ctx_dyck"] = rows_all + out["ctx_dyck_removed"] = [[self.removed(r, r.force - 1) for r in rows] + for rows in rows_all] + return out + + def _other_verb(self, v: int) -> int: + """The verb a context twin says instead: the chamber's other verb; at any size the + next one (the twins differ by one meaning coordinate, whichever verbs they are).""" + return 1 - v if self.verbs == 2 else (v + 1) % self.verbs + + # ------------------------------------------------------------------------------- scoring + + def score(self, row: Row, said: list[int]) -> dict[str, Any]: + """The whole sentence said, from its first word through ``end``.""" + target = row.target[row.force:] + out: dict[str, Any] = {"exact": said == target} + if row.meta.get("removed"): + return out + end = self.end + words = said[:-1] if said and said[-1] == end else said + ended = bool(said) and said[-1] == end + v0, n = self.verb0, self.nouns + if row.family == "clause" and row.force == 0: + m = row.meta + verbs = [i for i, w in enumerate(words) if self.is_verb(w)] + one = len(verbs) == 1 + k = verbs[0] if one else -1 + out["agree"] = one and (words[k] - v0) % 2 == m["agent"][1] + out["verb"] = one and (words[k] - v0) // 2 == m["verb"] + subject = words[0] if words and self.is_noun(words[0]) else None + out["agree_own"] = (one and subject is not None + and (words[k] - v0) % 2 == subject // n) + before = words[k - 1] if one and k > 0 and self.is_noun(words[k - 1]) else None + out["attracted"] = (None if before is None or before // n == m["agent"][1] + else (words[k] - v0) % 2 == before // n) + out["subject"] = subject == self.noun(*m["agent"]) + patient_at = (k - 1) if m["dialect"] else (k + 1) + out["patient"] = one and 0 <= patient_at < len(words) and words[patient_at] == ( + self.noun(*m["patient"])) + out["roles"] = out["subject"] and out["patient"] + mods = [w for pair in m["mods"] for w in (self.near, self.noun(*pair))] + out["mods"] = words[1:1 + len(mods)] == mods if mods else None + out["ended"] = ended + elif row.family == "clause": # a context continuation: the verb, the patient, end + out["agree"] = (bool(said) and self.is_verb(said[0]) + and (said[0] - v0) % 2 == row.meta["agent"][1]) + out["verb"] = (bool(said) and self.is_verb(said[0]) + and (said[0] - v0) // 2 == row.meta["verb"]) + elif row.family == "dyck": + stack: list[int] = [] + valid = ended + for w in row.target[:row.force] + words: + if w in self.opens: + stack.append(w) + elif w in (self.close_round, self.close_square) and stack and ( + self.match[stack[-1]] == w): + stack.pop() + else: + valid = False + out["valid"] = valid and not stack and len(words) == len(target) - 1 + if row.force == 0: + out["prefix"] = words[:row.meta["prefix"]] == row.target[:row.meta["prefix"]] + elif row.family == "path": + chain = row.meta["chain"] + out["hops"] = sum(1 for h in range(1, len(chain)) + if len(words) > 2 * h and words[2 * h] == chain[h] + and words[2 * h - 1] == self.near and all( + words[2 * g] == chain[g] for g in range(h))) + out["first_hop"] = len(words) > 2 and words[2] == chain[1] + return out + + def tally(self, rows: list[Row], said: list[list[int]]) -> dict[str, Any]: + scores = [self.score(r, s) for r, s in zip(rows, said, strict=True)] + out: dict[str, Any] = {"n": len(scores)} + for key in scores[0]: + kept = [s[key] for s in scores if s.get(key) is not None] + out[key] = float(np.mean(kept)) if kept else None + return out + + @staticmethod + def conditions(r: Row) -> list[str]: + m = r.meta + if r.family == "clause" and "L" in m: + keys = [f"L{m['L']}"] + if r.force == 0: + keys += ["attract" if m["attract"] else "plain", + "inverted" if m["inverted"] else "canonical"] + return keys + if r.family == "dyck": + return [f"depth{m['depth']}", "nested" if m["nested"] else "mixed"] + if r.family == "path": + return [f"hops{m['hops']}"] + return [] + + def read_speaker(self, speaker: Any, next_words: Any, batches: list[list[Row]], *, + twins: bool = False) -> dict[str, Any]: + """The panel's scores for any speaker: ``speaker(rows)`` returns the words each row + says from its ``force`` on, ``next_words(rows)`` (or None) the teacher-forced hits.""" + rows_all: list[Row] = [] + said_all: list[list[int]] = [] + hits: list[float] = [] + conds: dict[str, list[int]] = {} + for rows in batches: + said = speaker(rows) + rows_all += rows + said_all += said + if next_words is not None: + hits += next_words(rows) + out = self.tally(rows_all, said_all) + if next_words is not None: + out["next_word"] = float(np.mean(hits)) + agree: dict[str, list[int]] = {} + for r, s in zip(rows_all, said_all, strict=True): + scored = self.score(r, s) + for key in self.conditions(r): + c = conds.setdefault(key, [0, 0]) + c[0] += int(scored["exact"]) + c[1] += 1 + if scored.get("agree") is not None: + a = agree.setdefault(key, [0, 0]) + a[0] += int(scored["agree"]) + a[1] += 1 + out["conditions"] = {k: round(v[0] / v[1], 4) for k, v in sorted(conds.items())} + if agree: + out["agree_conditions"] = {k: round(v[0] / v[1], 4) + for k, v in sorted(agree.items())} + if twins: + pairs = list(zip(rows_all[0::2], said_all[0::2], rows_all[1::2], said_all[1::2], + strict=True)) + out["both"] = float(np.mean([self.score(a, sa)["exact"] + and self.score(b, sb)["exact"] + for a, sa, b, sb in pairs])) + out["differ"] = float(np.mean([sa != sb for _, sa, _, sb in pairs])) + out["examples"] = [[self.say(r.target[r.force:]), self.say(s)] for r, s in + list(zip(rows_all, said_all, strict=True))[:4]] + return out + + +TWINS = ("roles", "ctx_agent", "ctx_agent_removed", "ctx_verb", "ctx_verb_removed", + "ctx_dyck", "ctx_dyck_removed") +G1_PANELS = ("clause", "clause_held", "clause_long", "roles", "dyck", "dyck_deep", "path", + "path_long") +CONTEXT_PANELS = ("ctx_agent", "ctx_agent_removed", "ctx_verb", "ctx_verb_removed", "ctx_dyck", + "ctx_dyck_removed") +EVERYTHING = G1_PANELS + CONTEXT_PANELS + ("g2", "g2_long") +LEAN = ("clause", "clause_held", "clause_long", "roles", "dyck", "path", "path_long") diff --git a/benchmarks/vocabulary/lexicon.py b/benchmarks/vocabulary/lexicon.py new file mode 100644 index 00000000..1c4a6604 --- /dev/null +++ b/benchmarks/vocabulary/lexicon.py @@ -0,0 +1,131 @@ +"""A lexicon of real word forms and its wordpieces, for ``pieces.py``. + +The pieces come from byte-pair merges over the word list, each word weighed once, in the +``##`` form of a word's later pieces: ``kube ##rnet ##es``. The words' grammar is the chamber's and +carries no English meaning: only the forms and how they share pieces are real. ``TECH`` is a +coder's words, always in the list ahead of the ranked words. The ranked list is passed as a file, +one word a line, most frequent first. + + python benchmarks/vocabulary/lexicon.py --ranked google10k.txt --words 5000 --pieces 1000 \\ + --out lexicon-5000.json +""" + +from __future__ import annotations + +import argparse +import json +import sys +from collections import Counter +from pathlib import Path + +TECH = """ +api apis app apps array arrays async await auth authenticate authentication authorization backend +backup bash batch binary bit bitwise bool boolean branch breakpoint browser buffer bug bugs build +builder byte bytes cache caching callback class classes cli client clone cloud cluster code codebase +coder commit commits compile compiler component config configure console constant container +containers context cookie core coroutine cpu crash cron css csv cursor daemon dashboard data +database dataset debug debugger decorator default dependency deploy deployment deserialize dev +developer devops dictionary diff directory distributed dns docker dockerfile docs driver dropdown +encoding encrypt encryption endpoint enum env environment error errors exception executable export +extension fallback fetch field file filesystem firewall float flutter folder fork framework +frontend function functions generic generics getter git github gitlab golang gpu grafana graphql +hash hashmap header heap helm hook hooks host hostname http https ide immutable import index +inference inheritance init inline input instance integer interface interpreter iterator java +javascript jenkins json jwt kafka kernel key keyword kotlin kubernetes lambda latency layout lint +linter linux list listener localhost lock log logger logging login loop macos malloc markdown +merge metadata method methods microservice microservices middleware migration mock module modules +mutex mutable namespace neovim network nginx node nodejs null numpy oauth object observability +offset opcode operator optimizer orm output overflow package parser parameter payload pipeline +plugin pointer polymorphism postgres prefetch process processor production prometheus prompt +protobuf proxy pull pytest python queue react readme rebase recursion recursive redis refactor +regex registry release repo repository request response rest retry rollback router runtime rust +sandbox scala schema script sdk semaphore serialize server serverless session setter shader shell +snapshot socket sql stack staging stateful stateless struct subnet sudo swift syntax tensor +terraform thread threads timeout token tokenizer tokens traceback transaction transformer tuple +typescript ubuntu unicode unittest upstream url utf variable vector virtual vscode webhook +websocket webpack wifi workflow worktree yaml zip +""".split() + + +def read_ranked(path: Path) -> list[str]: + words = [] + for line in path.read_text().splitlines(): + w = line.strip().lower() + if len(w) >= 2 and w.isalpha() and w.isascii(): + words.append(w) + return words + + +def merges(words: list[str], pieces: int, keep: tuple[str, ...] = ("##s",)) -> dict[str, list[str]]: + """Each word as its pieces after merging the most frequent adjacent pair until the inventory + holds ``pieces`` pieces (``keep`` pieces are in it from the start and are never merged away).""" + split = {w: [w[0]] + ["##" + c for c in w[1:]] for w in words} + inventory = {p for parts in split.values() for p in parts} | set(keep) + while len(inventory) < pieces: + pairs: Counter[tuple[str, str]] = Counter() + for parts in split.values(): + for a, b in zip(parts, parts[1:], strict=False): + pairs[(a, b)] += 1 + if not pairs: + break + (a, b), _ = pairs.most_common(1)[0] + joined = a + b.removeprefix("##") + for w, parts in split.items(): + if a not in parts: + continue + out, i = [], 0 + while i < len(parts): + if i + 1 < len(parts) and parts[i] == a and parts[i + 1] == b: + out.append(joined) + i += 2 + else: + out.append(parts[i]) + i += 1 + split[w] = out + inventory = {p for parts in split.values() for p in parts} | set(keep) + return split + + +def build(ranked: list[str], words: int, pieces: int, longest: int = 3) -> dict[str, object]: + """The first ``words`` forms of TECH and the ranked list that take at most ``longest`` pieces + and do not end in the plural piece. Merges run on a list some way longer than the lexicon, + so that a word's pieces are shared the way a larger vocabulary shares them.""" + pool = list(dict.fromkeys(TECH + ranked))[:int(words * 1.5) + 200] + split = merges(pool, pieces) + chosen = [w for w in pool if len(split[w]) <= longest and split[w][-1] != "##s"][:words] + used = sorted({p for w in chosen for p in split[w]} | {"##s"}) + lengths = Counter(len(split[w]) for w in chosen) + return {"words": chosen, "pieces": used, "split": {w: split[w] for w in chosen}, + "lengths": {str(k): v for k, v in sorted(lengths.items())}, + "tech": sum(1 for w in chosen if w in set(TECH))} + + +def whole(ranked: list[str], words: int) -> dict[str, object]: + """The same forms with every word its own piece: one motor neuron per word, by the wordpiece + language's grammar. A novel stem then has no taught piece at all (``untaught``).""" + chosen = list(dict.fromkeys(TECH + ranked))[:words] + return {"words": chosen, "pieces": [*chosen, "##s"], "split": {w: [w] for w in chosen}, + "lengths": {"1": len(chosen)}, "tech": sum(1 for w in chosen if w in set(TECH)), + "untaught": True} + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__.split("\n", 1)[0]) + parser.add_argument("--ranked", type=Path, required=True) + parser.add_argument("--words", type=int, default=200) + parser.add_argument("--pieces", type=int, default=100) + parser.add_argument("--longest", type=int, default=3) + parser.add_argument("--whole", action="store_true", help="every word its own piece") + parser.add_argument("--out", type=Path, required=True) + args = parser.parse_args(argv) + ranked = read_ranked(args.ranked) + lexicon = (whole(ranked, args.words) if args.whole + else build(ranked, args.words, args.pieces, args.longest)) + args.out.write_text(json.dumps(lexicon)) + print(len(lexicon["words"]), "words", len(lexicon["pieces"]), "pieces", lexicon["lengths"], + "tech", lexicon["tech"]) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/benchmarks/vocabulary/life.py b/benchmarks/vocabulary/life.py new file mode 100644 index 00000000..ebf17a67 --- /dev/null +++ b/benchmarks/vocabulary/life.py @@ -0,0 +1,694 @@ +"""One continuing life of the composed brain in the grammar chamber at a vocabulary size. + +The life, the lessons and the scorer are the grammar chamber's (``benchmarks/grammar/``): a +meaning arrives as concept moments, at ``GO`` the brain says the sentence word by word, a lesson +comes before the act on every word it is taught, panels are read on forks, greedy, closed loop. +``Language(4, 2)`` lives the chamber's life to the bit (a test holds it). Each checkpoint writes +a line at once, so a life cut short keeps what it read. + + python benchmarks/vocabulary/life.py --words 200 --founder 0 --arm Fd \\ + --checkpoints 500 2000 8000 --out DIR/raw200.jsonl + python benchmarks/vocabulary/life.py --summary DIR/raw200.jsonl +""" + +from __future__ import annotations + +import argparse +import copy +import dataclasses +import json +import resource +import sys +import tempfile +import time +import warnings +from pathlib import Path +from typing import Any + +import numpy as np +from language import EVERYTHING, LEAN, SEEDS, TWINS, Language, Row, by_size +from pieces import PieceLanguage + +RECIPE: dict[str, Any] = { + "recall": True, "recall_recency": 0.0, "learning_surprise": True, "recall_boundary": True, + "recall_anchor": True, "recall_slope": 0.1, "recall_position_rate": 0.01, + "recall_relative": True, +} +SEQUENCE: dict[str, Any] = {"episodic": False, "teaching": "normalized"} +ARMS: dict[str, dict[str, Any]] = { + "recipe": {**SEQUENCE, **RECIPE}, + # the read's rule: the deep teaching point, no apical (the address lesson is the library's + # contrast) + "Fd": {**SEQUENCE, **RECIPE, "teaching": "deep"}, +} + + +def make_brain(lang: Any, arm: str, founder: int, settings: dict[str, Any] | None = None, + module: int = 64) -> Any: + import cadence as cd + + options = {**ARMS[arm], **(settings or {})} + options.setdefault("backend", "cpu") # one backend for every life and size + if options.get("recall_tokens") == "words": # action j and observation j are one token + options["recall_tokens"] = list(range(lang.actions)) + scale = options.pop("eta_scale", None) # the teaching point's steps, scaled, as an expert + map_code = options.pop("map_code", None) # the read's maps on a fixed code (``mapcode``) + gain = options.pop("init_gain", None) # ``both``, ``in`` or ``motor``: see ``chamber_gain`` + sharp = options.pop("address_scale", None) # the lesson's temperature follows the key width + cortex = options.pop("cortex", None) # how the maps take the cortex (``install_cortex``) + if gain is not None: + chamber_gain(lang, str(gain), module, options) + brain = cd.Brain.compose(lang.inputs, lang.actions, modules=(module,), seed=int(founder), + **options) + if scale is not None: + config = brain.learner.config + brain.retune(learning=dataclasses.replace(config, eta=config.eta * float(scale), + eta_bias=config.eta_bias * float(scale))) + if sharp is not None: # an emulation: a slot's share of the read-back falls with the width + brain.recall.address_temperature = 0.1 * float(np.sqrt(35 / lang.inputs)) + if map_code is not None: + import mapcode + + mapcode.install(brain, int(map_code), int(founder)) + if cortex is not None: + install_cortex(brain, cortex) + return brain + + +CORTEX_MAPS = ("head_maps", "position_maps", "anchor_maps", "relative_maps", "slope_maps", + "grid_maps") + + +def install_cortex(brain: Any, how: dict[str, Any]) -> None: + """A development emulation of how the read's maps take the cortex of the moment before, on + one brain and its forks: ``unit`` reads it at unit norm per stream, ``scale`` times a + number, and ``rate`` scales each lesson's step on every map's cortex columns. The brain's + class becomes a subclass that does this around ``act`` and ``lesson``.""" + unit, scale = bool(how.get("unit")), how.get("scale") + rate = how.get("rate") + + def read(recall: Any) -> None: + if unit: + raw = np.atleast_2d(np.asarray(recall.previous, dtype=float)) + norm = np.linalg.norm(raw, axis=1, keepdims=True) + recall.previous = np.where(norm > 0, raw / np.where(norm > 0, norm, 1.0), raw) + if scale is not None: + recall.previous = float(scale) * np.asarray(recall.previous) + + base = type(brain) + + class Balanced(base): # type: ignore[misc, valid-type] + def act(self, *args: Any, **kwargs: Any) -> Any: + read(self.recall) + return base.act(self, *args, **kwargs) + + def lesson(self, *args: Any, **kwargs: Any) -> Any: + read(self.recall) + before = None if rate is None else { + name: [np.array(h[1]) for h in getattr(self.recall, name, None) or []] + for name in CORTEX_MAPS} + try: + return base.lesson(self, *args, **kwargs) + finally: + if before is not None: + for name, columns in before.items(): + maps = getattr(self.recall, name, None) or [] + setattr(self.recall, name, [ + (h[0], old + float(rate) * (np.asarray(h[1]) - old), h[2]) + for h, old in zip(maps, columns, strict=True)]) + + Balanced.__name__ = base.__name__ + Balanced.__qualname__ = base.__qualname__ + brain.__class__ = Balanced + + +def chamber_gain(lang: Any, gain: str, module: int, options: dict[str, Any]) -> None: + """A development emulation, one process per life: a projection's initial synapses are scaled + by its fan, ``sqrt(6 / (pre + post))``, which suits dense activity, and at a one-hot word a + large vocabulary's single active neuron then drives far less than at the chamber's size. This + sets the sensory, recall and next projections (``in``) and the association's with the motor + region (``motor``) to the per-synapse magnitude they have at 18 words: their scales times + ``sqrt((pre + post) / (pre + post at 18 words))``.""" + from cadence import generic + + if gain in ("both", "in"): + factor = float(np.sqrt((lang.inputs + module) / (35 + module))) + options["scales"] = {**options.get("scales", {}), "sensory": factor} + generic._RECALL_SCALE = 1.2 * factor + if gain in ("both", "motor"): + factor = float(np.sqrt((lang.actions + module) / (18 + module))) + original = generic.Projection + + def scaled(pre: str, post: str, **kw: Any) -> Any: + if (pre, post) == ("association", "motor"): + kw["scale"] = factor + return original(pre, post, **kw) + + generic.Projection = scaled + + +def new_work() -> dict[str, float]: + return {"lessons": 0, "rows": 0, "agreed_rows": 0, "lesson_row_sweeps": 0, "acts": 0, + "act_sweeps": 0, "refused": 0, "exhausted": 0, "free_budget_exhausted": 0} + + +def speak(lang: Language, brain: Any, rows: list[Row], *, teach: bool, closed: bool = True, + work: dict[str, float] | None = None, cap: int = 4, + intervene: Any = None) -> list[list[int]]: + """One batch on fresh streams (``grammar.speak``). Heard moments are acted on greedily and + never taught. From ``GO`` each moment hears the previous word: the target's while teaching, + within a row's ``force`` and when ``closed`` is off; else the brain's own. With ``teach`` a + lesson on the target word comes before every act from ``GO`` on.""" + work = work if work is not None else new_work() + n = len(rows) + lengths = {len(r.pre) for r in rows} + if len(lengths) != 1: + raise ValueError("a batch's rows hear the same number of moments") + if teach and len({len(r.target) for r in rows}) != 1: + raise ValueError("a taught batch's sentences have one length") + brain.reset() + for t in range(lengths.pop()): + act(brain, lang.dense([r.pre[t] for r in rows]), work) + said: list[list[int]] = [[] for _ in rows] + done = [False] * n + longest = max(len(r.target) for r in rows) + (0 if teach else cap) + for j in range(longest): + if j == 0: + x = lang.dense([(lang.go,)] * n) + else: + moments: list[tuple[int, ...]] = [] + for i, r in enumerate(rows): + given = teach or not closed or j <= r.force + w = r.target[j - 1] if given and j - 1 < len(r.target) else said[i][j - 1] + moments.append(lang.spoken(w) if w >= 0 else (lang.speak,)) + x = lang.dense(moments) + if intervene is not None and j == rows[0].force: + intervene(brain, rows) + if teach: + labels = np.array([r.target[j] for r in rows]) + try: + report = brain.lesson(x, labels) + work["lessons"] += 1 + work["rows"] += report.get("attempted_presentations", 0.0) + work["agreed_rows"] += report.get("agreed_presentations", 0.0) + work["lesson_row_sweeps"] += report.get("total_row_sweeps", 0.0) + work["free_budget_exhausted"] += report.get("free_budget_exhausted", 0.0) + except RuntimeError: + work["refused"] += 1 + action = act(brain, x, work) + for i, r in enumerate(rows): + said[i].append(int(action[i])) + if j >= r.force and action[i] == lang.end: + done[i] = True + if not teach and all(done): + break + out = [] + for i, r in enumerate(rows): + words = said[i][r.force:] + out.append(words[:words.index(lang.end) + 1] if lang.end in words else words) + return out + + +def act(brain: Any, x: np.ndarray, work: dict[str, float]) -> np.ndarray: + try: + action = np.asarray(brain.act(x, greedy=True)).reshape(len(x), -1)[:, 0] + settled = brain.last_settlement or {} + work["act_sweeps"] += float(settled.get("steps", 0)) + except RuntimeError: + work["exhausted"] += 1 + action = np.full(len(x), -1) + work["acts"] += 1 + return action + + +def erase(where: str) -> Any: + """``grammar.erase``: the earlier context emptied from the recall slots, or the trace.""" + + def apply(brain: Any, rows: list[Row]) -> None: + if where == "trace": + brain.working_memory.trace = np.zeros_like(brain.working_memory.trace) + return + recall = brain.recall + if recall is None: + return + written = len(rows[0].pre) + rows[0].force + ages = np.asarray(recall.ages()) + meaning = ages >= written - len(rows[0].pre) - 2 + recall.keys = np.where(meaning[None, :, None], 0.0, recall.keys) + recall.values = np.where(meaning[None, :, None], 0.0, recall.values) + recall.nexts = np.where(meaning[None, :, None], 0.0, recall.nexts) + + return apply + + +def forced(lang: Language, brain: Any, rows: list[Row]) -> list[float]: + """Next-word accuracy with every previous word given (component evidence).""" + brain.reset() + for t in range(len(rows[0].pre)): + brain.act(lang.dense([r.pre[t] for r in rows]), greedy=True) + hits = [] + for j in range(len(rows[0].target)): + x = (lang.dense([(lang.go,)] * len(rows)) if j == 0 + else lang.dense([lang.spoken(r.target[j - 1]) for r in rows])) + action = np.asarray(brain.act(x, greedy=True)).reshape(len(x), -1)[:, 0] + if j >= rows[0].force: + hits += [float(action[i] == r.target[j]) for i, r in enumerate(rows)] + return hits + + +def sources(lang: Language, row: Row) -> list[int | None]: + """For each target word, the heard moment that holds its content, or None for a word the + grammar alone decides (``near``, ``end``).""" + if "sources" in row.meta: + return list(row.meta["sources"]) + if row.family == "clause": + m = row.meta + L = m["L"] + agent, mods, patient = ((2, list(range(3, 3 + L)), 1) if m["inverted"] + else (1, list(range(2, 2 + L)), 2 + L)) + out: list[int | None] = [agent] + for i in mods: + out += [None, i] + tail: list[int | None] = [patient, 0] if m["dialect"] else [0, patient] + return out + tail + [None] + if row.family == "dyck": + out, stack = [], [] + for i, x in enumerate(row.pre): + if lang.c_close in x: + out.append(stack.pop()) + else: + out.append(i) + stack.append(i) + return out + list(reversed(stack)) + [None] + out = [len(row.pre) - 1] # the start, named by the meaning + heard_words = [x[0] for x in row.pre[:-1]] + for a in row.meta["chain"][:-1]: + at = next(i for i in range(0, len(heard_words), 4) if heard_words[i] == a) + out += [None, at + 2] # the fact's second word as heard + return out + [None] + + +def kind(lang: Language, r: Row, j: int) -> str: + if "slots" in r.meta: + return lang.kind(r, j) + if r.family == "dyck" and "sources" in r.meta: + return "dyck_" + ("open" if r.target[j] in lang.opens else "close") + if r.family == "clause": + m = r.meta + k = next(i for i, w in enumerate(r.target) if lang.is_verb(w)) + if j == 0: + name = "agent" + elif j == k: + name = "verb" + elif r.target[j] == lang.noun(*m["patient"]) and ( + j == k + 1 or (m["dialect"] and j == k - 1)): + name = "patient" + else: + name = "mod" + return f"clause_{name}" + if r.family == "dyck": + return "dyck_" + ("open" if r.target[j] in lang.opens else "close") + return "path_" + ("start" if j == 0 else "hop") + + +def locate(lang: Language, brain: Any, batches: list[list[Row]]) -> dict[str, Any]: + """Where the read lands and what is said, teacher-forced, on a fork: for every word whose + content a heard moment holds, whether the recall memory's first head attends most to that + moment or to the moment before it (``delivered``), and whether the brain says the word. A + word delivered and not said is lost at the readout, one not delivered at the read.""" + fork = copy.deepcopy(brain) + groups: dict[str, list[int]] = {} + for rows in batches: + fork.reset() + written = 0 + for t in range(len(rows[0].pre)): + fork.act(lang.dense([r.pre[t] for r in rows]), greedy=True) + written += 1 + expected = [sources(lang, r) for r in rows] + for j in range(len(rows[0].target)): + x = (lang.dense([(lang.go,)] * len(rows)) if j == 0 + else lang.dense([lang.spoken(r.target[j - 1]) for r in rows])) + ages = None if fork.recall is None else np.asarray(fork.recall.ages()) + action = np.asarray(fork.act(x, greedy=True)).reshape(len(x), -1)[:, 0] + heads = fork.recall.attended_heads if fork.recall is not None else None + p = heads[0] if heads else None + for i, r in enumerate(rows): + source = expected[i][j] + if source is None: + continue + landed = -2 + if p is not None and ages is not None: + landed = written - 1 - int(ages[int(np.argmax(p[i]))]) + g = groups.setdefault(kind(lang, r, j), [0, 0, 0, 0]) + found = landed in (source, source - 1) + g[0] += int(found) + g[1] += int(action[i] == r.target[j]) + g[2] += int(found and action[i] == r.target[j]) + g[3] += 1 + written += 1 + return {k: {"n": v[3], "delivered": round(v[0] / v[3], 4), "said": round(v[1] / v[3], 4), + "said_when_delivered": round(v[2] / v[0], 4) if v[0] else None} + for k, v in sorted(groups.items())} + + +TERMS = ("key", "event", "cortex", "bias", "position_event", "position_cortex", "position_bias") + + +def logit_terms(brain: Any, x: np.ndarray) -> dict[str, np.ndarray]: + """Head 0's logits over each stream's slots at the moment ``x``, by what they come from, + read before the act: the event against a slot's key (``key``), the address map's event rows, + cortex columns and bias against it (``event``, ``cortex``, ``bias``), and the same three of + every position map at the slot's buckets (``position_*``). They sum to the logits of the read + (``total``), whose softmax is the attention the act reads with.""" + recall = brain.recall + moment = recall._moment(brain.observation_code(x)) + inputs, c = moment.inputs, moment.cortex + w_x, w_c, b = recall.head_map(0) + keys, temperature = recall.keys, recall.temperature + out = { + "key": np.einsum("bk,bmk->bm", moment.key, keys) / temperature, + "event": np.einsum("bk,bmk->bm", inputs @ w_x, keys) / temperature, + "cortex": np.einsum("bk,bmk->bm", c @ w_c, keys) / temperature, + "bias": np.einsum("k,bmk->bm", b, keys) / temperature, + } + zero_x, zero_c = np.zeros_like(inputs), np.zeros_like(c) + whole = recall.position_term(0, inputs, c) + if whole is None: + whole = np.zeros_like(out["key"]) + event_only = bias_only = whole + else: + event_only = recall.position_term(0, inputs, zero_c) + bias_only = recall.position_term(0, zero_x, zero_c) + out["position_cortex"] = whole - event_only + out["position_event"] = event_only - bias_only + out["position_bias"] = bias_only + total = sum(out[name] for name in TERMS) + if recall.recency: + total = total - recall.recency * recall.ages()[None, :] + out["total"] = np.where(recall.held, total, -np.inf) + return out + + +def terms(lang: Language, brain: Any, batches: list[list[Row]]) -> dict[str, Any]: + """Where the read's selection is decided, teacher-forced on a fork: at every word whose + content a heard moment holds, each term's logit at the right slot (the source moment or the + one before it, whichever the read scores higher) less its logit at the best other slot, per + kind of word, and over the rows the read misses. ``check`` is the largest gap between the + softmax of the summed terms and the attention the act read with.""" + fork = copy.deepcopy(brain) + sums: dict[str, dict[str, Any]] = {} + check = 0.0 + for rows in batches: + fork.reset() + written = 0 + for t in range(len(rows[0].pre)): + fork.act(lang.dense([r.pre[t] for r in rows]), greedy=True) + written += 1 + expected = [sources(lang, r) for r in rows] + for j in range(len(rows[0].target)): + x = (lang.dense([(lang.go,)] * len(rows)) if j == 0 + else lang.dense([lang.spoken(r.target[j - 1]) for r in rows])) + parts = logit_terms(fork, x) + ages = np.asarray(fork.recall.ages()).astype(int) + fork.act(x, greedy=True) + heads = fork.recall.attended_heads + if heads: + total = parts["total"] + p = np.exp(total - total.max(axis=1, keepdims=True)) + p /= p.sum(axis=1, keepdims=True) + check = max(check, float(np.abs(p - heads[0]).max())) + for i, r in enumerate(rows): + source = expected[i][j] + if source is None: + continue + places = [int(s) for s in np.flatnonzero(np.isin( + written - 1 - ages, (source, source - 1)) & fork_held(parts, i))] + if not places: + continue + right = max(places, key=lambda s: parts["total"][i, s]) + others = np.array([s for s in range(len(ages)) if s not in places + and np.isfinite(parts["total"][i, s])]) + if not len(others): + continue + rival = int(others[np.argmax(parts["total"][i, others])]) + missed = parts["total"][i, rival] > parts["total"][i, right] + for group in (kind(lang, r, j), "all"): + g = sums.setdefault(group, {"n": 0, "missed": 0, "terms": dict.fromkeys( + (*TERMS, "total"), 0.0), "missed_terms": dict.fromkeys( + (*TERMS, "total"), 0.0)}) + g["n"] += 1 + g["missed"] += int(missed) + for name in (*TERMS, "total"): + gap = float(parts[name][i, right] - parts[name][i, rival]) + g["terms"][name] += gap + if missed: + g["missed_terms"][name] += gap + written += 1 + out: dict[str, Any] = {"check": round(check, 6)} + for group, g in sorted(sums.items()): + n, m = g["n"], g["missed"] + out[group] = { + "n": n, "missed": round(m / n, 4), + "gap": {k: round(v / n, 3) for k, v in g["terms"].items()}, + "gap_missed": {k: round(v / m, 3) for k, v in g["missed_terms"].items()} if m else None, + } + return out + + +def fork_held(parts: dict[str, np.ndarray], row: int) -> np.ndarray: + """The slots a stream holds, from the terms' total.""" + return np.isfinite(parts["total"][row]) + + +def read_panel(lang: Language, brain: Any, batches: list[list[Row]], *, twins: bool = False, + components: bool = False, intervene: Any = None) -> dict[str, Any]: + fork = copy.deepcopy(brain) + return lang.read_speaker( + lambda rows: speak(lang, fork, rows, teach=False, intervene=intervene), + (lambda rows: forced(lang, fork, rows)) if components else None, + batches, twins=twins) + + +ERASED = ("ctx_agent", "ctx_verb", "ctx_dyck") + + +def reading(lang: Language, brain: Any, tests: dict[str, list[list[Row]]], + names: tuple[str, ...], components: bool = False) -> dict[str, Any]: + """Each named panel; with the context panels, the same twins with the context erased from + the recall memory or from the working trace at the continuation's start.""" + out = {name: read_panel(lang, brain, tests[name], twins=name in TWINS, components=components) + for name in names} + for name in ERASED: + if name in names: + for where in ("recall", "trace"): + out[f"{name}_erased_{where}"] = read_panel( + lang, brain, tests[name], twins=True, intervene=erase(where)) + return out + + +def code_read(lang: Language, brain: Any, sample: int = 48) -> dict[str, Any]: + """The learned code itself: how words of a class lie together and whether a word keeps its + identity. Cosines of the keys of noun concepts (with the agent's flag), the nouns' own heard + words and the verbs' concepts; ``same`` the share of a class's pairs at cosine .98 or above + (two words the memories cannot tell apart); ``units`` the mean active units of a key.""" + + def keys(moments: list[tuple[int, ...]]) -> np.ndarray: + rows = [brain.observation_code(lang.dense(moments[i:i + 16])) + for i in range(0, len(moments), 16)] + k = np.concatenate(rows) + return k / np.maximum(np.linalg.norm(k, axis=1, keepdims=True), 1e-12) + + def within(k: np.ndarray) -> tuple[float, float]: + c = k @ k.T + off = c[~np.eye(len(k), dtype=bool)] + return float(off.mean()), float((off >= 0.98).mean()) + + fork = copy.deepcopy(brain) + brain = fork + stems = range(min(sample, lang.trained)) + concepts = keys([lang.noun_moment(i, 0, lang.agent) for i in stems]) + words = keys([lang.heard(lang.noun(i, 0)) for i in stems]) + verbs = keys([(lang.verb_concept0 + v,) for v in range(min(sample, lang.verbs))]) + out: dict[str, Any] = {} + for name, k in (("noun_concepts", concepts), ("noun_words", words), ("verb_concepts", verbs)): + mean, same = within(k) + out[name] = {"within": round(mean, 4), "same": round(same, 4), + "units": round(float((k > 0).sum(axis=1).mean()), 2)} + out["concept_to_verb"] = round(float((concepts @ verbs.T).mean()), 4) + out["word_to_verb"] = round(float((words @ verbs.T).mean()), 4) + out["concept_to_own_word"] = round(float(np.mean(np.sum(concepts * words, axis=1))), 4) + return out + + +def rss_gb() -> float: + usage = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss + return round(usage / (1024**3 if sys.platform == "darwin" else 1024**2), 2) + + +def language_of(spec: dict[str, Any]) -> Any: + """The spec's language: by wordpiece with ``lexicon``, else by word at ``words``.""" + if spec.get("lexicon"): + families = spec.get("families") + return PieceLanguage(spec["lexicon"], str(spec.get("route", "heard")), + bool(spec.get("swap")), families, bool(spec.get("start")), + float(spec.get("pseudo", 0.0)), bool(spec.get("verb_tokens"))) + nouns, verbs = (int(spec["nouns"]), int(spec["verbs"])) if "nouns" in spec else by_size( + int(spec["words"])) + return Language(nouns, verbs, int(spec.get("fresh", 0)), bool(spec.get("verb_flag"))) + + +def run_life(spec: dict[str, Any]) -> list[dict[str, Any]]: + """One life: ``words`` (or ``nouns`` and ``verbs``, or ``lexicon``, ``route`` and ``swap`` + for the wordpiece language), ``fresh``, ``arm``, ``founder``, + ``settings``, ``module``, ``checkpoints`` (batches at which a lean reading is taken and, + with ``save`` (a folder), the brain saved), ``full`` (batches read on every panel), ``out`` + (JSON lines, appended at each checkpoint), ``panel_batches``, ``locate``.""" + warnings.simplefilter("ignore") + import cadence as cd + + lang = language_of(spec) + arm, founder = str(spec.get("arm", "Fd")), int(spec["founder"]) + module = int(spec.get("module", 64)) + started = time.monotonic() + brain = make_brain(lang, arm, founder, spec.get("settings"), module) + split = lang.split(founder) + tests = lang.panels(founder, int(spec.get("panel_batches", 4))) + rng = np.random.default_rng(np.random.SeedSequence([founder, SEEDS["train"]])) + checkpoints = sorted({int(c) for c in spec.get("checkpoints", [2000])}) + full = {int(c) for c in spec.get("full", [])} + folder = Path(spec["save"]) if spec.get("save") else None + sink = Path(spec["out"]) if spec.get("out") else None + out: list[dict[str, Any]] = [] + head = {"arm": arm, "founder": founder, "words": lang.words, "nouns": lang.nouns, + "verbs": lang.verbs, "fresh": getattr(lang, "fresh", 0), "inputs": lang.inputs, + "actions": lang.actions, "module": module, "settings": spec.get("settings", {}), + "parameters": int(brain.parameters()), "cadence": cd.__version__, + "tag": spec.get("tag", "")} + work = new_work() + teaching = 0.0 + batch = 0 + grow = spec.get("grow") # {"words": at the start, "hold": batches, "ramp": batches to all} + whole = (lang.nouns, lang.verbs) + for checkpoint in checkpoints: + begun = time.monotonic() + while batch < checkpoint: + if grow: + start = by_size(int(grow["words"])) + share = min(1.0, max(0.0, (batch - int(grow["hold"])) / max(1, int(grow["ramp"])))) + lang.grown(round(start[0] + share * (whole[0] - start[0])), + round(start[1] + share * (whole[1] - start[1]))) + speak(lang, brain, lang.train_batch(rng, split), teach=True, work=work) + batch += 1 + teaching += time.monotonic() - begun + begun = time.monotonic() + names = (tuple(tests) if isinstance(lang, PieceLanguage) + else EVERYTHING if checkpoint in full else LEAN) + row = {**head, "batches": batch, "work": dict(work), "teaching_seconds": round(teaching, 1), + "panels": reading(lang, brain, tests, tuple(names), components=False)} + if spec.get("locate", True) and brain.recall is not None: + row["located"] = {name: locate(lang, brain, tests[name]) + for name in (("taught", "novel_all", "path", "dyck") + if isinstance(lang, PieceLanguage) + else ("clause", "path", "dyck")) if name in tests} + if spec.get("terms") and brain.recall is not None: + row["terms"] = {name: terms(lang, brain, tests[name][:2]) + for name in ("clause", "path") if name in tests} + if getattr(brain, "learned_code", None) is not None and isinstance(lang, Language): + row["code"] = code_read(lang, brain) + moments = work["acts"] + row["seconds_per_batch"] = round(teaching / max(1, batch), 4) + row["seconds_per_moment"] = round(teaching / max(1, moments), 5) + row["sweeps_per_act"] = round(work["act_sweeps"] / max(1, moments), 1) + row["reading_seconds"] = round(time.monotonic() - begun, 1) + row["rss_gb"] = rss_gb() + row["seconds"] = round(time.monotonic() - started, 1) + if folder is not None: + folder.mkdir(parents=True, exist_ok=True) + row["saved"] = str(brain.save(folder / f"{arm}-w{lang.words}-f{founder}-b{batch}")) + out.append(row) + if sink is not None: + sink.parent.mkdir(parents=True, exist_ok=True) + with sink.open("a") as handle: + handle.write(json.dumps(row) + "\n") + if spec.get("continuation"): + out[-1]["continuation"] = continuation(lang, brain, rng, split, tests["clause"][0]) + return out + + +def continuation(lang: Language, brain: Any, rng: np.random.Generator, split: Any, + rows: list[Row]) -> dict[str, Any]: + """Save and load a fork; both copies take the same three lessons and say the same panel.""" + import cadence as cd + + live = copy.deepcopy(brain) + with tempfile.TemporaryDirectory() as folder: + loaded = cd.Brain.load(live.save(Path(folder) / "brain")) + stream = copy.deepcopy(rng) + batches = [lang.train_batch(stream, split) for _ in range(3)] + for b in batches: + speak(lang, live, b, teach=True) + speak(lang, loaded, b, teach=True) + said = [speak(lang, x, rows, teach=False) for x in (live, loaded)] + return {"identical": said[0] == said[1], "sentences": len(rows)} + + +COLUMNS = (("clause", "exact", "cl"), ("clause", "L2", "L2"), ("clause_held", "exact", "held"), + ("clause_long", "exact", "long"), ("roles", "exact", "roles"), + ("clause", "agree", "agree"), ("dyck", "exact", "dyck"), ("path", "exact", "path"), + ("path", "hops2", "2hop"), ("path_long", "exact", "3hop")) + + +def line(row: dict[str, Any]) -> str: + cells = [] + for panel, key, name in COLUMNS: + p = row["panels"].get(panel, {}) + value = p.get(key, p.get("conditions", {}).get(key)) + cells.append(f"{name} {value:.2f}" if value is not None else f"{name} --") + return (f"{row['arm']:6s} w{row['words']} f{row['founder']} b{row['batches']:6d} " + f"p{row['parameters']:9d} {row['seconds_per_moment']:.4f}s/m " + f"{row['sweeps_per_act']:5.1f}sw " + " ".join(cells)) + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__.split("\n", 1)[0]) + parser.add_argument("--words", type=int, default=200) + parser.add_argument("--fresh", type=int, default=0) + parser.add_argument("--verb-flag", action="store_true") + parser.add_argument("--grow", type=json.loads, default=None, + help='{"words": 200, "hold": 2000, "ramp": 3000}: a vocabulary that grows') + parser.add_argument("--lexicon", default="", help="the wordpiece language's lexicon (JSON)") + parser.add_argument("--route", default="heard", choices=("heard", "concept")) + parser.add_argument("--swap", action="store_true") + parser.add_argument("--start", action="store_true", help="a flag on a word's first piece") + parser.add_argument("--pseudo", type=float, default=0.0, help="share of made-up stems") + parser.add_argument("--verb-tokens", action="store_true", help="a verb's own agreeing pieces") + parser.add_argument("--arm", default="Fd", choices=list(ARMS)) + parser.add_argument("--founder", type=int, default=0) + parser.add_argument("--module", type=int, default=64) + parser.add_argument("--checkpoints", nargs="*", type=int, default=[2000]) + parser.add_argument("--full", nargs="*", type=int, default=[]) + parser.add_argument("--settings", type=json.loads, default={}) + parser.add_argument("--panel-batches", type=int, default=4) + parser.add_argument("--save", default="") + parser.add_argument("--tag", default="") + parser.add_argument("--out", type=Path) + parser.add_argument("--summary", type=Path) + args = parser.parse_args(argv) + if args.summary is not None: + rows = [json.loads(x) for x in args.summary.read_text().splitlines() if x] + for row in sorted(rows, key=lambda r: (r["arm"], r["words"], r["founder"], r["batches"])): + print(line(row)) + return 0 + run_life({"words": args.words, "fresh": args.fresh, "lexicon": args.lexicon, + "verb_flag": args.verb_flag, "grow": args.grow, + "route": args.route, "swap": args.swap, "start": args.start, + "pseudo": args.pseudo, "verb_tokens": args.verb_tokens, "arm": args.arm, + "founder": args.founder, + "module": args.module, "checkpoints": args.checkpoints, "full": args.full, + "settings": args.settings, "panel_batches": args.panel_batches, + "save": args.save, "tag": args.tag, "out": str(args.out) if args.out else ""}) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/benchmarks/vocabulary/mapcode.py b/benchmarks/vocabulary/mapcode.py new file mode 100644 index 00000000..9c86707d --- /dev/null +++ b/benchmarks/vocabulary/mapcode.py @@ -0,0 +1,63 @@ +"""A development emulation: the read's maps reading a fixed code of the event. + +At raw codes the address map is ``inputs`` squared and holds nearly all of a large brain's +parameters and much of its seconds: 56 M of 59.3 M at 5,000 words. Its rows mostly say what to +look for after a kind of moment, and a kind needs far fewer rows than there are words. Here the +address, position, anchored, relative and slope maps read ``z = x R`` in place of the event ``x``: +``R`` fixed, sparse and random, ``per_input`` units per input coordinate, each at ``1 / +sqrt(per_input)``, so a coordinate keeps its energy. The keys and values stay the raw events, and +the rule's form is unchanged: the same contrast on the same inputs ``[z, c, 1]``. A map holds +``units * inputs`` in place of ``inputs ** 2``. A newborn reads as a brain without it, since the +maps start at zero. + +This changes no library file; it is read in development, on one process per life, and a brain with +it is not saved. +""" + +from __future__ import annotations + +from typing import Any + +import numpy as np +from scipy import sparse + +from cadence.stream import Recall, _Moment, _of + + +class MapCoded(Recall): + """``Recall`` whose maps read the fixed code ``self.map_code`` of the event.""" + + def _moment(self, event: np.ndarray) -> _Moment: + x = np.asarray(event, dtype=float) + n = len(x) + c = _of(self.previous, n) + key = self.heard * x + inputs = np.asarray(self.map_code.T.dot(x.T)).T + w_x, w_c, b = self.head_maps[0] + q = key + inputs @ w_x + c @ w_c + b + return _Moment(x, c, q, key, inputs) + + +def install(brain: Any, units: int, seed: int, per_input: int = 4) -> None: + """Make ``brain``'s recall memory read its maps through a code of ``units`` units.""" + recall = brain.recall + if recall.window != 1 or recall.action or recall.heads != 1: + raise ValueError("the emulation covers one head, no window and no command") + rng = np.random.default_rng(np.random.SeedSequence([int(seed), 266, 7])) + inputs = recall.key_width + rows = np.repeat(np.arange(inputs), per_input) + cols = np.concatenate([rng.choice(units, size=per_input, replace=False) + for _ in range(inputs)]) + code = sparse.csr_matrix((np.full(len(rows), 1.0 / np.sqrt(per_input)), (rows, cols)), + shape=(inputs, units)) + recall.__class__ = MapCoded + recall.map_code = code + recall.map_width = int(units) + count = recall.position_buckets + recall.head_maps = recall._zero_maps(recall.heads, recall.address_width) + recall.position_maps = recall._zero_maps(recall.heads if count else 0, count) + recall.anchor_maps = recall._zero_maps(recall.heads if recall.anchor else 0, count + 1) + recall.relative_maps = recall._zero_maps(recall.heads if recall.relative else 0, + 2 * recall.RELATIVE + 2) + recall.slope_maps = recall._zero_maps(recall.heads if recall.slope else 0) + recall.grid_maps = recall._zero_maps(recall.heads if recall.grid else 0, sum(recall.grid)) diff --git a/benchmarks/vocabulary/pieces.py b/benchmarks/vocabulary/pieces.py new file mode 100644 index 00000000..88479187 --- /dev/null +++ b/benchmarks/vocabulary/pieces.py @@ -0,0 +1,589 @@ +"""The chamber's sentences said by wordpiece: one motor neuron per piece, not per word. + +A word is a stem of pieces (``ref ##act ##or``) and a number piece: ``##0`` for the singular, +``##s`` for the plural of a noun. A verb's last piece agrees with its agent: ``##s`` with a singular +agent, ``##0`` with a plural one. ``##0`` is a silent piece, so that every form of a stem takes +one length and a batch's streams keep in step. ``near``, the brackets and ``end`` stay words. + +Two ways of naming a word in a meaning (``route``): + +- ``heard``: the meaning says the word's pieces, one moment each, every moment with its role flag + (``AGENT``, ``PATIENT``, ``MOD``, ``VERB``; ``PLURAL`` on a plural noun), so the brain can say any + word it hears whose pieces it knows. Input coordinates: the tokens and 11 flags. +- ``concept``: one moment per word, the word's own concept coordinate and flags, as in the + grammar chamber: the brain has to know a word's pieces. Only taught words can be asked. + +The grammar is the chamber's: ``clause`` (``agent (near mod)* verb patient end``), ``dyck`` and +``path`` (``X near Y end`` heard, the chain said), with ``swap`` beside them when ``swap`` is on: +the world says ``A verb B end``, the brain says ``B verb A end``, every noun used in the role it +was not heard in, and the verb agreeing with its new agent. Novel stems, a share of the stems per +founder, never appear in a training sentence and every piece of them does. + +``pseudo`` is the share of the stems in a training sentence that are made up on the spot, one taught +first piece and taught later pieces drawn at random, the way a name or a new word arrives. Their +sentences teach the brain to say what it hears and leave no word to memorize. The ``pseudo`` panel +asks such stems at test. + +``verb_tokens`` gives a verb's agreeing piece tokens of its own, ``##vs`` and ``##v0``. With the +shared pieces the read can only name a slot by what it holds, and a verb that agrees with a singular +agent says ``##s``, which the slots of plural nouns hold. + +``start`` adds a ``START`` flag to the moment that hears a word's first piece: what the ``##`` of a +later piece says, as a coordinate all first pieces share, so an address can look for a word's start. + +Forms and sharing of pieces are real (``lexicon.py``); the sentences carry no English meaning. +""" + +from __future__ import annotations + +import json +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +import numpy as np +from language import BATCH, SEEDS, TEST_HOPS, TRAIN_DEPTH, TRAIN_HOPS, TRAIN_L, Row, mix + +Moment = tuple[int, ...] +NOVEL = 0.1 # the share of the stems held out as novel, per founder +LONG_L = (3, 4) + + +@dataclass(frozen=True) +class PieceSplit: + """A founder's stems: taught and novel, noun and verb, by stem length.""" + + nouns: dict[int, list[int]] # stem length -> taught noun stems + verbs: dict[int, list[int]] + novel_nouns: dict[int, list[int]] + novel_verbs: dict[int, list[int]] + salt: int + firsts: tuple[int, ...] = () # the first pieces of the taught stems + laters: tuple[int, ...] = () # their later pieces + + def held(self, agent: int, patient: int) -> bool: + return (mix(self.salt, 3, min(agent, patient), max(agent, patient)) % 6 == 0 + and agent != patient) + + +class PieceLanguage: + """The chamber's families said by wordpiece, over a lexicon of real forms.""" + + def __init__(self, lexicon: dict[str, Any] | str | Path, route: str = "heard", + swap: bool = False, families: dict[str, float] | None = None, + start: bool = False, pseudo: float = 0.0, verb_tokens: bool = False) -> None: + if isinstance(lexicon, (str, Path)): + lexicon = json.loads(Path(lexicon).read_text()) + self.route, self.swap, self.pseudo = route, swap, float(pseudo) + self.untaught = bool(lexicon.get("untaught")) # a novel stem may have no taught piece + pieces = [p for p in lexicon["pieces"] if p not in ("##s", "##0")] + self.piece_names = [*pieces, "##s", "##0", *(("##vs", "##v0") if verb_tokens else ())] + self.piece_id = {p: i for i, p in enumerate(self.piece_names)} + self.s, self.zero = self.piece_id["##s"], self.piece_id["##0"] + self.verb_s, self.verb_zero = self.s, self.zero + if verb_tokens: + self.verb_s, self.verb_zero = self.piece_id["##vs"], self.piece_id["##v0"] + self.tokens_pieces = len(self.piece_names) + order = np.random.default_rng(np.random.SeedSequence([2026, 266])).permutation( + len(lexicon["words"])) + words = [lexicon["words"][int(i)] for i in order] + self.word_names = words + stems = [tuple(self.piece_id[p] for p in lexicon["split"][w]) for w in words] + verbs = len(stems) // 3 + self.verb_stems, self.noun_stems = stems[:verbs], stems[verbs:] + self.nouns, self.verbs = len(self.noun_stems), len(self.verb_stems) + self.words = self.nouns + self.verbs # stems: every stem has two forms + P = self.tokens_pieces + self.near = P + self.open_round, self.close_round = P + 1, P + 2 + self.open_square, self.close_square = P + 3, P + 4 + self.end = P + 5 + self.actions = P + 6 + self.opens = (self.open_round, self.open_square) + self.match = {self.open_round: self.close_round, self.open_square: self.close_square} + f = self.actions + self.plural, self.agent, self.patient = f, f + 1, f + 2 + self.mod, self.verb_flag = f + 3, f + 4 + self.c_open_round, self.c_open_square, self.c_close = f + 5, f + 6, f + 7 + self.c_from, self.go, self.speak, self.c_swap = f + 8, f + 9, f + 10, f + 11 + self.c_opens = (self.c_open_round, self.c_open_square) + base = f + (12 if swap else 11) + if not swap: + self.c_swap = -1 + self.concept0 = base + self.inputs = base + (len(stems) if route == "concept" else 0) + self.start: int | None = None + if start: + self.start = self.inputs + self.inputs += 1 + self.families = dict(families) if families else ( + {"clause": 0.4, "dyck": 0.2, "path": 0.2, "swap": 0.2} if swap + else {"clause": 0.5, "dyck": 0.25, "path": 0.25}) + + # ------------------------------------------------------------------------------ moments + + def dense(self, moments: list[Moment]) -> np.ndarray: + x = np.zeros((len(moments), self.inputs)) + for row, moment in enumerate(moments): + x[row, list(moment)] = 1.0 + return x + + def spoken(self, word: int) -> Moment: + return (int(word), self.speak) + + def say(self, words: list[int]) -> str: + names = {self.near: "near", self.open_round: "(", self.close_round: ")", + self.open_square: "[", self.close_square: "]", self.end: "end"} + return " ".join(names.get(w, self.piece_names[w] if 0 <= w < self.tokens_pieces else "?") + for w in words) + + def noun_form(self, stem: Any, plural: int) -> list[int]: + pieces = self.noun_stems[stem] if isinstance(stem, int) else stem + return [*pieces, self.s if plural else self.zero] + + def verb_form(self, stem: Any, agent_plural: int) -> list[int]: + pieces = self.verb_stems[stem] if isinstance(stem, int) else stem + return [*pieces, self.verb_zero if agent_plural else self.verb_s] + + @staticmethod + def made_up(rng: np.random.Generator, split: PieceSplit, length: int) -> tuple[int, ...]: + """A stem of ``length`` taught pieces drawn at random.""" + rest = rng.choice(split.laters, size=length - 1) if length > 1 else () + return (int(rng.choice(split.firsts)), *(int(p) for p in rest)) + + def noun_moments(self, stem: Any, plural: int, *flags: int) -> list[Moment]: + number = (self.plural,) if plural else () + if self.route == "concept": + return [(self.concept0 + self.verbs + stem, *number, *flags)] + return self.pieces_heard(self.noun_form(stem, plural), *number, *flags) + + def verb_moments(self, stem: Any) -> list[Moment]: + if self.route == "concept": + return [(self.concept0 + stem, self.verb_flag)] + pieces = self.verb_stems[stem] if isinstance(stem, int) else stem + return self.pieces_heard(pieces, self.verb_flag) + + def pieces_heard(self, form: Any, *flags: int) -> list[Moment]: + """A word's pieces as heard, one moment each, the first with ``START`` when it is on.""" + first = () if self.start is None else (self.start,) + return [(p, *flags, *(first if i == 0 else ())) for i, p in enumerate(form)] + + def kind(self, r: Row, j: int) -> str: + """What the ``j``-th word of a sentence is, for ``life.locate``.""" + for name, a, b in r.meta["slots"]: + if a <= j < b: + place = "number" if j == b - 1 else "first" if j == a else "later" + return f"{r.family}_{name}_{place}" + return f"{r.family}_grammar" + + # -------------------------------------------------------------------------------- split + + def split(self, founder: int) -> PieceSplit: + """Stems by length; a tenth of them novel, every piece of a novel stem kept in a taught + stem. ``##s`` and ``##0`` are taught by every sentence.""" + rng = np.random.default_rng(np.random.SeedSequence([int(founder), SEEDS["split"]])) + count: dict[int, int] = {} + for stem in (*self.noun_stems, *self.verb_stems): + for p in stem: + count[p] = count.get(p, 0) + 1 + novel: set[tuple[str, int]] = set() + everything = [("n", i) for i in range(self.nouns)] + [("v", i) for i in range(self.verbs)] + target = int(NOVEL * len(everything)) + for k in rng.permutation(len(everything)): + kind, i = everything[int(k)] + stem = self.noun_stems[i] if kind == "n" else self.verb_stems[i] + if len(novel) < target and (self.untaught + or all(count[p] > stem.count(p) for p in stem)): + novel.add((kind, i)) + for p in stem: + count[p] -= 1 + + def buckets(kind: str, stems: list[tuple[int, ...]], held: bool) -> dict[int, list[int]]: + out: dict[int, list[int]] = {} + for i, stem in enumerate(stems): + if ((kind, i) in novel) == held: + out.setdefault(len(stem), []).append(i) + return out + + nouns, verbs = buckets("n", self.noun_stems, False), buckets("v", self.verb_stems, False) + taught = [self.noun_stems[i] for b in nouns.values() for i in b] + taught += [self.verb_stems[i] for b in verbs.values() for i in b] + return PieceSplit(nouns, verbs, + buckets("n", self.noun_stems, True), + buckets("v", self.verb_stems, True), + int(rng.integers(2**31)), + tuple(sorted({stem[0] for stem in taught})), + tuple(sorted({p for stem in taught for p in stem[1:]}))) + + @staticmethod + def _length(rng: np.random.Generator, stems: dict[int, list[int]], low: int = 2) -> int: + """A stem length, in proportion to the stems of that length, among lengths with enough + stems for a batch's streams to differ.""" + lengths = [k for k, v in stems.items() if len(v) >= low] or list(stems) + lengths.sort() + weights = np.array([len(stems[k]) for k in lengths], dtype=float) + return int(rng.choice(lengths, p=weights / weights.sum())) + + # ------------------------------------------------------------------------------ families + + def clause_shape(self, rng: np.random.Generator, split: PieceSplit, L: int, + novel: tuple[str, ...] = ()) -> dict[str, Any]: + def pick(role: str, kind: str) -> int: + taught = split.nouns if kind == "n" else split.verbs + held = split.novel_nouns if kind == "n" else split.novel_verbs + return self._length(rng, held if role in novel else taught) + + return {"agent": pick("agent", "n"), "patient": pick("patient", "n"), + "verb": pick("verb", "v"), + "mods": [pick("mod", "n") for _ in range(L)], "novel": novel, + "inverted": bool(rng.integers(2)), "pseudo": 0.0} + + def clause(self, rng: np.random.Generator, split: PieceSplit, shape: dict[str, Any]) -> Row: + novel, pseudo = shape["novel"], shape["pseudo"] + + def noun_of(role: str, length: int) -> tuple[Any, int]: + if pseudo and rng.random() < pseudo: + return self.made_up(rng, split, length), int(rng.integers(2)) + stems = split.novel_nouns if role in novel else split.nouns + return int(rng.choice(stems[length])), int(rng.integers(2)) + + while True: + agent, patient = noun_of("agent", shape["agent"]), noun_of("patient", shape["patient"]) + if (novel or pseudo or not isinstance(agent[0], int) or not isinstance(patient[0], int) + or not split.held(agent[0], patient[0])): + break + verbs = split.novel_verbs if "verb" in novel else split.verbs + v: Any = (self.made_up(rng, split, shape["verb"]) if pseudo and rng.random() < pseudo + else int(rng.choice(verbs[shape["verb"]]))) + mods = [noun_of("mod", k) for k in shape["mods"]] + act = self.verb_moments(v) + a = self.noun_moments(*agent, self.agent) + p = self.noun_moments(*patient, self.patient) + m = [self.noun_moments(*mod, self.mod) for mod in mods] + heard: list[list[Moment]] = [act, p, a, *m] if shape["inverted"] else [act, a, *m, p] + names = (["verb", "patient", "agent"] + ["mod"] * len(m) if shape["inverted"] + else ["verb", "agent"] + ["mod"] * len(m) + ["patient"]) + pre = [moment for part in heard for moment in part] + where: dict[str, list[list[int]]] = {} + at = 0 + for name, part in zip(names, heard, strict=True): + where.setdefault(name, []).append(list(range(at, at + len(part)))) + at += len(part) + target: list[int] = [] + slots: list[tuple[str, int, int]] = [] + src: list[int | None] = [] + + def add(name: str, form: list[int], heard_at: list[int]) -> None: + start = len(target) + target.extend(form) + slots.append((name, start, len(target))) + for i in range(len(form)): + src.append(heard_at[i] if i < len(heard_at) else + (heard_at[-1] if heard_at and self.route == "concept" else None)) + + add("agent", self.noun_form(*agent), where["agent"][0]) + for k, mod in enumerate(mods): + target.append(self.near) + src.append(None) + add("mod", self.noun_form(*mod), where["mod"][k]) + add("verb", self.verb_form(v, agent[1]), where["verb"][0]) + add("patient", self.noun_form(*patient), where["patient"][0]) + target.append(self.end) + src.append(None) + meta = {"agent": list(agent), "patient": list(patient), "verb": v, "L": len(mods), + "mods": [list(x) for x in mods], "slots": slots, "sources": src, + "inverted": shape["inverted"], "novel": list(novel), + "attract": any(x[1] != agent[1] for x in mods)} + return Row("clause", pre, target, meta) + + def swapped(self, rng: np.random.Generator, split: PieceSplit, shape: dict[str, Any]) -> Row: + """The world says ``A verb B end`` by pieces with no flags; ``SWAP``; the brain says ``B + verb A end``, the verb agreeing with B.""" + novel, pseudo = shape["novel"], shape["pseudo"] + + def noun_of(role: str, length: int) -> tuple[Any, int]: + if pseudo and rng.random() < pseudo: + return self.made_up(rng, split, length), int(rng.integers(2)) + stems = split.novel_nouns if role in novel else split.nouns + return int(rng.choice(stems[length])), int(rng.integers(2)) + + a, b = noun_of("agent", shape["agent"]), noun_of("patient", shape["patient"]) + verbs = split.novel_verbs if "verb" in novel else split.verbs + v: Any = (self.made_up(rng, split, shape["verb"]) if pseudo and rng.random() < pseudo + else int(rng.choice(verbs[shape["verb"]]))) + heard_a, heard_b = self.noun_form(*a), self.noun_form(*b) + heard_v = self.verb_form(v, a[1]) + said_v = self.verb_form(v, b[1]) + pre: list[Moment] = [*self.pieces_heard(heard_a), *self.pieces_heard(heard_v), + *self.pieces_heard(heard_b), (self.end,), (self.c_swap,)] + at_a = list(range(len(heard_a))) + at_v = list(range(len(heard_a), len(heard_a) + len(heard_v))) + at_b = list(range(at_v[-1] + 1, at_v[-1] + 1 + len(heard_b))) + target = [*heard_b, *said_v, *heard_a, self.end] + slots = [("agent", 0, len(heard_b)), ("verb", len(heard_b), len(heard_b) + len(said_v)), + ("patient", len(heard_b) + len(said_v), len(target) - 1)] + src: list[int | None] = [*at_b, *at_v[:-1], None, *at_a, None] + meta = {"agent": list(b), "patient": list(a), "verb": v, "L": 0, "mods": [], + "slots": slots, "sources": src, "inverted": False, "novel": list(novel), + "attract": False, "swap": True} + return Row("swap", pre, target, meta) + + def swap_shape(self, rng: np.random.Generator, split: PieceSplit, + novel: tuple[str, ...] = ()) -> dict[str, Any]: + return self.clause_shape(rng, split, 0, novel) + + def dyck(self, rng: np.random.Generator, pattern: list[int]) -> Row: + types = [int(rng.integers(2)) for _ in pattern] + pre: list[Moment] = [] + words: list[int] = [] + src: list[int | None] = [] + stack: list[tuple[int, int]] = [] + for i, (step, t) in enumerate(zip(pattern, types, strict=True)): + if step: + pre.append((self.c_opens[t],)) + words.append(self.opens[t]) + stack.append((self.opens[t], i)) + src.append(i) + else: + pre.append((self.c_close,)) + o, j = stack.pop() + words.append(self.match[o]) + src.append(j) + depth, deepest = 0, 0 + for step in pattern: + depth += 1 if step else -1 + deepest = max(deepest, depth) + for o, j in reversed(stack): + words.append(self.match[o]) + src.append(j) + meta = {"prefix": len(pattern), "depth": deepest, "open": len(stack), + "nested": 0 not in pattern, "sources": [*src, None]} + return Row("dyck", pre, [*words, self.end], meta) + + def path(self, rng: np.random.Generator, split: PieceSplit, hops: int, length: int, + novel: bool = False, distractors: int = 2, pseudo: float = 0.0) -> Row: + """Facts heard by pieces; the chain from the named start said by pieces. Every entity of + a row has a stem of ``length`` pieces. With ``novel``, the chain's stems are novel and the + distractors' taught; with ``pseudo``, that share of the chain's stems is made up.""" + taught = split.nouns[length] + held = split.novel_nouns.get(length, []) if novel else taught + need = hops + 1 + if len(held) >= need: + stems: list[Any] = [int(i) for i in rng.permutation(held)[:need]] + else: + stems = [int(i) for i in rng.permutation(taught)[:need]] + stems = [self.made_up(rng, split, length) if pseudo and rng.random() < pseudo else s + for s in stems] + chain = [(s, int(rng.integers(2))) for s in stems] + rest = [s for s in taught if s not in set(stems)] + firsts = [int(i) for i in rng.permutation(rest)[:distractors]] + others = [int(i) for i in rng.permutation(rest)[:distractors * 2 + 2]] + facts = [(chain[i], chain[i + 1]) for i in range(hops)] + for k, f in enumerate(firsts): + other = next(o for o in others[k:] if o != f) + facts.append(((f, int(rng.integers(2))), (other, int(rng.integers(2))))) + pre: list[Moment] = [] + spots: dict[tuple[int, int], list[int]] = {} + for i in rng.permutation(len(facts)): + x, y = facts[int(i)] + fx, fy = self.noun_form(*x), self.noun_form(*y) + at = len(pre) + pre += [*self.pieces_heard(fx), (self.near,)] + spots[y] = list(range(len(pre), len(pre) + len(fy))) + pre += [*self.pieces_heard(fy), (self.end,)] + spots.setdefault(x, list(range(at, at + len(fx)))) + start_at = len(pre) + start = chain[0] + pre += (self.pieces_heard(self.noun_form(*start), self.c_from) if self.route == "heard" + else [(self.concept0 + self.verbs + int(start[0]), + *((self.plural,) if start[1] else ()), self.c_from)]) + words: list[int] = [] + src: list[int | None] = [] + slots: list[tuple[str, int, int]] = [] + for k, e in enumerate(chain): + if k: + words.append(self.near) + src.append(None) + form = self.noun_form(*e) + slots.append(("start" if k == 0 else "hop", len(words), len(words) + len(form))) + words += form + if k == 0: + src += [(start_at + i if self.route == "heard" else start_at) + for i in range(len(form))] + else: + src += list(spots[e]) + return Row("path", pre, [*words, self.end], + {"hops": hops, "chain": [list(e) for e in chain], "facts": len(facts), + "slots": slots, "sources": [*src, None], "novel": ["chain"] if novel else []}) + + def train_batch(self, rng: np.random.Generator, split: PieceSplit, grammar: str = "g1", + size: int = BATCH) -> list[Row]: + family = str(rng.choice(list(self.families), p=list(self.families.values()))) + if family == "clause": + shape = self.clause_shape(rng, split, int(rng.choice(TRAIN_L))) + shape["pseudo"] = self.pseudo + return [self.clause(rng, split, shape) for _ in range(size)] + if family == "swap": + shape = self.swap_shape(rng, split) + shape["pseudo"] = self.pseudo + return [self.swapped(rng, split, shape) for _ in range(size)] + if family == "dyck": + pattern = dyck_pattern(rng, int(rng.integers(1, 7)), TRAIN_DEPTH) + return [self.dyck(rng, pattern) for _ in range(size)] + hops = int(rng.choice(TRAIN_HOPS)) + length = self._length(rng, split.nouns, low=8) + return [self.path(rng, split, hops, length, pseudo=self.pseudo) for _ in range(size)] + + # -------------------------------------------------------------------------------- panels + + def panels(self, founder: int, batches: int = 4) -> dict[str, list[list[Row]]]: + split = self.split(founder) + rng = np.random.default_rng(np.random.SeedSequence([int(founder), SEEDS["test"]])) + out: dict[str, list[list[Row]]] = {} + + def clauses(Ls: tuple[int, ...], novel: tuple[str, ...] = (), + pseudo: float = 0.0) -> list[list[Row]]: + rows = [] + for i in range(batches): + shape = self.clause_shape(rng, split, Ls[i % len(Ls)], novel) + shape["pseudo"] = pseudo + rows.append([self.clause(rng, split, shape) for _ in range(BATCH)]) + return rows + + out["taught"] = clauses(TRAIN_L) + out["long"] = clauses(LONG_L) + if self.route == "heard": + out["novel_agent"] = clauses(TRAIN_L, ("agent",)) + out["novel_patient"] = clauses(TRAIN_L, ("patient",)) + out["novel_all"] = clauses(TRAIN_L, ("agent", "patient", "mod", "verb")) + out["dyck"] = [] + for i in range(batches): + nested = i % 2 == 0 + length = int(rng.integers(1 if nested else 3, 4 if nested else 7)) + pattern = dyck_pattern(rng, length, TRAIN_DEPTH, nested=nested) + out["dyck"].append([self.dyck(rng, pattern) for _ in range(BATCH)]) + + def paths(hops: tuple[int, ...], novel: bool = False) -> list[list[Row]]: + rows = [] + for i in range(batches): + stems = split.novel_nouns if novel else split.nouns + need = max(hops) + 1 if novel else 8 + length = self._length(rng, {k: v for k, v in stems.items() if len(v) >= need} + or stems, low=need) + rows.append([self.path(rng, split, hops[i % len(hops)], length, novel) + for _ in range(BATCH)]) + return rows + + out["path"] = paths(TRAIN_HOPS) + out["path_long"] = paths(TEST_HOPS) + if self.route == "heard": + out["path_novel"] = paths(TRAIN_HOPS, True) + if self.pseudo: + pseudo_rng = np.random.default_rng(np.random.SeedSequence([int(founder), 266, 5])) + kept, rng = rng, pseudo_rng + out["pseudo"] = clauses(TRAIN_L, (), 1.0) + rng = kept + if self.swap: + out["swap"] = [[self.swapped(rng, split, shape) for _ in range(BATCH)] + for shape in (self.swap_shape(rng, split) for _ in range(batches))] + out["swap_novel"] = [[self.swapped(rng, split, shape) for _ in range(BATCH)] + for shape in (self.swap_shape(rng, split, ("agent", "patient")) + for _ in range(batches))] + return out + + # ------------------------------------------------------------------------------- scoring + + def score(self, row: Row, said: list[int]) -> dict[str, Any]: + target = row.target[row.force:] + out: dict[str, Any] = {"exact": said == target} + ended = bool(said) and said[-1] == self.end + if row.family in ("clause", "swap"): + slots = row.meta["slots"] + hit = {name: all(said[a:b] == target[a:b] for n, a, b in slots if n == name) + for name, _, _ in slots} + out["agent_word"] = hit.get("agent") + out["patient_word"] = hit.get("patient") + out["verb_word"] = hit.get("verb") + out["mod_words"] = hit.get("mod") if row.meta["L"] else None + verb = next((a, b) for n, a, b in slots if n == "verb") + out["agree"] = (len(said) >= verb[1] and said[verb[1] - 1] == target[verb[1] - 1]) + out["ended"] = ended + out["first_piece"] = bool(said) and said[0] == target[0] + out["words_right"] = float(np.mean([said[a:b] == target[a:b] for _, a, b in slots])) + elif row.family == "dyck": + stack: list[int] = [] + valid = ended + for w in (said[:-1] if ended else said): + if w in self.opens: + stack.append(w) + elif w in (self.close_round, self.close_square) and stack and ( + self.match[stack[-1]] == w): + stack.pop() + else: + valid = False + out["valid"] = valid and not stack and len(said) == len(target) + elif row.family == "path": + slots = row.meta["slots"] + run = 0 + for _, a, b in slots: + if said[a:b] != target[a:b]: + break + run += 1 + out["hops"] = max(0, run - 1) + out["first_hop"] = run >= 2 + out["words_right"] = float(np.mean([said[a:b] == target[a:b] for _, a, b in slots])) + return out + + def tally(self, rows: list[Row], said: list[list[int]]) -> dict[str, Any]: + scores = [self.score(r, s) for r, s in zip(rows, said, strict=True)] + out: dict[str, Any] = {"n": len(scores)} + for key in scores[0]: + kept = [s[key] for s in scores if s.get(key) is not None] + out[key] = float(np.mean(kept)) if kept else None + return out + + @staticmethod + def conditions(r: Row) -> list[str]: + m = r.meta + if r.family == "clause": + return [f"L{m['L']}", "inverted" if m["inverted"] else "canonical"] + if r.family == "dyck": + return [f"depth{m['depth']}", "nested" if m["nested"] else "mixed"] + if r.family == "path": + return [f"hops{m['hops']}"] + return [] + + def read_speaker(self, speaker: Any, next_words: Any, batches: list[list[Row]], *, + twins: bool = False) -> dict[str, Any]: + rows_all: list[Row] = [] + said_all: list[list[int]] = [] + hits: list[float] = [] + conds: dict[str, list[int]] = {} + for rows in batches: + rows_all += rows + said_all += speaker(rows) + if next_words is not None: + hits += next_words(rows) + out = self.tally(rows_all, said_all) + if next_words is not None: + out["next_word"] = float(np.mean(hits)) + for r, s in zip(rows_all, said_all, strict=True): + for key in self.conditions(r): + c = conds.setdefault(key, [0, 0]) + c[0] += int(s == r.target[r.force:]) + c[1] += 1 + out["conditions"] = {k: round(v[0] / v[1], 4) for k, v in sorted(conds.items())} + out["examples"] = [[self.say(r.target[r.force:]), self.say(s)] for r, s in + list(zip(rows_all, said_all, strict=True))[:4]] + return out + + +def dyck_pattern(rng: np.random.Generator, length: int, max_depth: int, + nested: bool | None = None) -> list[int]: + from language import Language + + return Language.dyck_pattern(rng, length, max_depth, nested=nested) + + +PANELS = ("taught", "long", "novel_agent", "novel_patient", "novel_all", "pseudo", "dyck", "path", + "path_long", "path_novel", "swap", "swap_novel") diff --git a/benchmarks/vocabulary/test_vocabulary.py b/benchmarks/vocabulary/test_vocabulary.py new file mode 100644 index 00000000..01f16d05 --- /dev/null +++ b/benchmarks/vocabulary/test_vocabulary.py @@ -0,0 +1,352 @@ +"""The scaled language is the grammar chamber at the chamber's size, and sound at any size.""" + +from __future__ import annotations + +import copy +import importlib.util +import sys +import warnings +from pathlib import Path + +import numpy as np +import pytest + +HERE = Path(__file__).resolve().parent +sys.path.insert(0, str(HERE)) +sys.path.insert(0, str(HERE.parent / "grammar")) + + +def _load(name: str, path: Path): + if name in sys.modules: + return sys.modules[name] + spec = importlib.util.spec_from_file_location(name, path) + module = importlib.util.module_from_spec(spec) + sys.modules[name] = module + spec.loader.exec_module(module) + return module + + +gr = _load("grammar", HERE.parent / "grammar" / "grammar.py") +language = _load("language", HERE / "language.py") +life = _load("life", HERE / "life.py") + + +def same_row(lang, ours, theirs) -> None: + assert ours.family == theirs.family + assert ours.target == theirs.target + assert ours.force == theirs.force + assert len(ours.pre) == len(theirs.pre) + for a, b in zip(ours.pre, theirs.pre, strict=True): + assert np.array_equal(lang.dense([a])[0], b) + + +def test_the_chamber_at_its_own_size_is_the_chamber() -> None: + lang = language.Language(4, 2) + assert (lang.inputs, lang.actions) == (gr.INPUTS, gr.ACTIONS) + assert lang.near == gr.NEAR and lang.end == gr.END and lang.opens == gr.OPENS + assert lang.go == gr.C["GO"] and lang.speak == gr.C["SPEAK"] + for founder in range(4): + ours, theirs = lang.panels(founder), gr.panels(founder) + assert set(ours) == set(theirs) + for name in theirs: + for a, b in zip(ours[name], theirs[name], strict=True): + for x, y in zip(a, b, strict=True): + same_row(lang, x, y) + s = gr.Split.of(founder) + r1 = np.random.default_rng(founder) + r2 = np.random.default_rng(founder) + ours_split = lang.split(founder) + assert ours_split.triples == s.triples and ours_split.pair == s.pair + for _ in range(30): + for x, y in zip(lang.train_batch(r1, ours_split), gr.train_batch(r2, s, "g1"), + strict=True): + same_row(lang, x, y) + + +def test_the_life_at_the_chamber_size_is_the_chamber_life() -> None: + warnings.simplefilter("ignore") + lang = language.Language(4, 2) + settings = life.ARMS["Fd"] + gr.ARMS["vocab_fd"] = settings + ours = life.make_brain(lang, "Fd", 3) + theirs = gr.make_brain("vocab_fd", 3) + assert ours.parameters() == theirs.parameters() == 28086 + split_ours, split_theirs = lang.split(3), gr.Split.of(3) + r1 = np.random.default_rng(np.random.SeedSequence([3, 11])) + r2 = np.random.default_rng(np.random.SeedSequence([3, 11])) + for _ in range(6): + said_ours = life.speak(lang, ours, lang.train_batch(r1, split_ours), teach=True) + said_theirs = gr.speak(theirs, gr.train_batch(r2, split_theirs, "g1"), teach=True) + assert said_ours == said_theirs + rows_ours, rows_theirs = lang.panels(3)["path"][0], gr.panels(3)["path"][0] + assert life.speak(lang, ours, rows_ours, teach=False) == gr.speak( + theirs, rows_theirs, teach=False) + + +@pytest.mark.parametrize("words", [200, 1000, 5000]) +def test_sizes_by_class(words: int) -> None: + nouns, verbs = language.by_size(words) + lang = language.Language(nouns, verbs) + assert lang.words == words and lang.actions == words + assert lang.inputs == words + nouns + verbs + 11 + + +@pytest.mark.parametrize("words", [18, 200, 1000]) +def test_every_panel_and_batch_is_answerable_from_what_is_heard(words: int) -> None: + lang = language.Language(*language.by_size(words)) + tests = lang.panels(1, 2) + for name in language.G1_PANELS + ("ctx_agent", "ctx_verb", "ctx_dyck", "g2", "g2_long"): + for batch in tests[name]: + for row in batch: + assert lang.solve(row.pre) == row.target, name + rng = np.random.default_rng(5) + split = lang.split(1) + for _ in range(20): + for row in lang.train_batch(rng, split): + assert lang.solve(row.pre) == row.target + + +def test_the_split_holds_its_shares_and_training_keeps_out() -> None: + lang = language.Language(*language.by_size(1000)) + split = lang.split(0) + rng = np.random.default_rng(0) + pairs = rng.integers(lang.trained, size=(20000, 2)) + share = np.mean([split.held_pair(int(p), int(q)) for p, q in pairs if p != q]) + triples = np.mean([split.held_triple((int(rng.integers(lang.trained)), + int(rng.integers(2))), int(rng.integers(lang.verbs))) + for _ in range(20000)]) + assert abs(share - 1 / 6) < 0.02 and abs(triples - 1 / 8) < 0.02 + for _ in range(200): + row = lang.trained_clause(rng, split, 1, False) + m = row.meta + assert not split.held(tuple(m["agent"]), tuple(m["patient"]), m["verb"]) + for batch in lang.panels(0, 2)["clause_held"]: + for row in batch: + m = row.meta + assert split.held(tuple(m["agent"]), tuple(m["patient"]), m["verb"]) + + +def test_fresh_stems_stay_out_of_training_and_panels() -> None: + lang = language.Language(65, 32, fresh=16) + rng = np.random.default_rng(0) + split = lang.split(0) + fresh_words = {lang.noun(i, p) for i in range(lang.trained, lang.nouns) for p in (0, 1)} + fresh_concepts = {lang.noun_concept0 + i for i in range(lang.trained, lang.nouns)} + for _ in range(100): + for row in lang.train_batch(rng, split): + assert not fresh_words & set(row.target) + assert not any(fresh_concepts & set(m) or fresh_words & set(m) for m in row.pre) + for batches in lang.panels(0, 2).values(): + for batch in batches: + for row in batch: + assert not fresh_words & set(row.target) + assert not any(fresh_concepts & set(m) or fresh_words & set(m) for m in row.pre) + + +def test_a_mention_is_a_path_the_solver_can_answer() -> None: + hearing = _load("hearing", HERE / "hearing.py") + lang = language.Language(65, 32, fresh=16) + heard, other = hearing.plans(lang) + assert {s for s, _ in heard} == {s for s, _ in other} == set(range(49, 65)) + assert all(a[1] != b[1] for a, b in zip(heard, other, strict=True)) + rng = np.random.default_rng(0) + for row, (stem, _) in zip(hearing.mentioned(lang, rng, heard), heard, strict=True): + assert lang.solve(row.pre) == row.target + assert row.target[2] == lang.noun(stem, 0) + for row in hearing.sentences(lang, rng, other): + assert lang.solve(row.pre) == row.target + + +def test_heard_pieces_say_their_word_and_novel_stems_have_taught_pieces(tmp_path: Path) -> None: + pieces = _load("pieces", HERE / "pieces.py") + words = [f"w{a}{b}" for a in "abcdefgh" for b in "ijklmnop"] + split = {w: [w[:2], "##" + w[2]] for w in words} + names = sorted({p for w in words for p in split[w]}) + lang = pieces.PieceLanguage({"words": words, "pieces": names, "split": split}, swap=True) + s = lang.split(0) + taught = {p for table in (s.nouns, s.verbs) for stems in table.values() for i in stems + for p in (lang.noun_stems[i] if table is s.nouns else lang.verb_stems[i])} + for table, stems in ((s.novel_nouns, lang.noun_stems), (s.novel_verbs, lang.verb_stems)): + for bucket in table.values(): + for i in bucket: + assert set(stems[i]) <= taught + rng = np.random.default_rng(0) + for _ in range(40): + batch = lang.train_batch(rng, s) + assert len({len(r.pre) for r in batch}) == 1 + assert len({len(r.target) for r in batch}) == 1 + for r in batch: + for m in r.pre: + assert all(0 <= c < lang.inputs for c in m) + assert all(0 <= w < lang.actions for w in r.target) + assert len(r.meta["sources"]) == len(r.target) + for name, batches in lang.panels(0, 2).items(): + for batch in batches: + assert len({len(r.pre) for r in batch}) == 1, name + if name.startswith("novel") or name.startswith("swap_novel"): + for r in batch: + held = [lang.noun_stems[stem] for stem, _ in ( + [tuple(r.meta["agent"])] if "agent" in r.meta["novel"] else [])] + assert all(stem in [lang.noun_stems[i] for b in s.novel_nouns.values() + for i in b] for stem in held) + row = lang.panels(0, 2)["taught"][0][0] + hit = lang.score(row, list(row.target)) + assert hit["exact"] and hit["agent_word"] and hit["patient_word"] and hit["agree"] + + +def test_the_verb_flag_adds_one_input_and_keeps_every_sentence() -> None: + plain, flagged = language.Language(65, 32), language.Language(65, 32, verb_flag=True) + assert flagged.inputs == plain.inputs + 1 + a, b = np.random.default_rng(3), np.random.default_rng(3) + for _ in range(4): + x, y = plain.clause(a, 1), flagged.clause(b, 1) + assert x.target == y.target + assert set(y.pre[0]) == {*x.pre[0], flagged.verb_flag} + assert flagged.solve(y.pre) == y.target + + +def test_the_coded_maps_start_as_a_brain_without_them_and_learn() -> None: + warnings.simplefilter("ignore") + lang = language.Language(*language.by_size(200)) + plain = life.make_brain(lang, "Fd", 2) + coded = life.make_brain(lang, "Fd", 2, {"map_code": 32}) + assert coded.parameters() < plain.parameters() + rows = lang.panels(2, 1)["clause"][0] + assert life.speak(lang, plain, rows, teach=False) == life.speak(lang, coded, rows, teach=False) + w_before = coded.recall.head_maps[0][0].copy() + life.speak(lang, coded, lang.train_batch(np.random.default_rng(1), lang.split(2)), teach=True) + assert coded.recall.head_maps[0][0].shape == (32, lang.inputs) + assert np.abs(coded.recall.head_maps[0][0] - w_before).max() > 0 + fork = __import__("copy").deepcopy(coded) + assert life.speak(lang, fork, rows, teach=False) == life.speak(lang, coded, rows, teach=False) + + +def test_the_start_flag_marks_each_heard_first_piece_and_changes_no_sentence() -> None: + pieces = _load("pieces", HERE / "pieces.py") + words = [f"w{a}{b}" for a in "abcdefgh" for b in "ijklmnop"] + split = {w: [w[:2], "##" + w[2]] for w in words} + names = sorted({p for w in words for p in split[w]}) + lexicon = {"words": words, "pieces": names, "split": split} + plain = pieces.PieceLanguage(lexicon, swap=True) + flagged = pieces.PieceLanguage(lexicon, swap=True, start=True) + assert flagged.inputs == plain.inputs + 1 + a, b = np.random.default_rng(1), np.random.default_rng(1) + s = plain.split(0) + for _ in range(30): + for x, y in zip(plain.train_batch(a, s), flagged.train_batch(b, s), strict=True): + assert x.target == y.target + assert [set(m) - {flagged.start} for m in y.pre] == [set(m) for m in x.pre] + if x.family == "clause": + heard = [i for i, m in enumerate(y.pre) if flagged.start in m] + assert len(heard) == 3 + x.meta["L"] # one first piece a word + + +def test_made_up_stems_come_from_taught_pieces_and_leave_the_panels_of_no_share_alone() -> None: + pieces = _load("pieces", HERE / "pieces.py") + words = [f"w{a}{b}" for a in "abcdefgh" for b in "ijklmnop"] + split = {w: [w[:2], "##" + w[2]] for w in words} + names = sorted({p for w in words for p in split[w]}) + lexicon = {"words": words, "pieces": names, "split": split} + plain = pieces.PieceLanguage(lexicon, swap=True) + made = pieces.PieceLanguage(lexicon, swap=True, pseudo=0.5) + s = plain.split(0) + assert made.split(0) == s + assert set(s.firsts) and set(s.laters) + for name, batches in plain.panels(0, 2).items(): + for x_batch, y_batch in zip(batches, made.panels(0, 2)[name], strict=True): + for x, y in zip(x_batch, y_batch, strict=True): + assert x.target == y.target, name + rng = np.random.default_rng(0) + found = False + for _ in range(60): + for row in made.train_batch(rng, s): + assert all(0 <= w < made.actions for w in row.target) + assert len(row.meta["sources"]) == len(row.target) + if row.family == "clause": + stem = row.meta["agent"][0] + if not isinstance(stem, int): + found = True + assert stem[0] in s.firsts and all(p in s.laters for p in stem[1:]) + assert found + for batch in made.panels(0, 2)["pseudo"]: + assert len({len(r.pre) for r in batch}) == 1 + for r in batch: + assert not isinstance(r.meta["agent"][0], int) + + +def test_a_growing_vocabulary_draws_from_what_it_holds_and_the_panels_stay_whole() -> None: + lang = language.Language(*language.by_size(1000)) + tests = lang.panels(0, 1) + split = lang.split(0) + lang.grown(65, 32) + rng = np.random.default_rng(0) + for _ in range(40): + for row in lang.train_batch(rng, split): + if row.family == "clause": + assert row.meta["agent"][0] < 65 and row.meta["patient"][0] < 65 + assert row.meta["verb"] < 32 + assert lang.solve(row.pre) == row.target + if row.family == "path": + assert all(lang.is_noun(w) and w % lang.nouns < 65 for w in row.meta["chain"]) + assert any(r.meta["agent"][0] >= 65 for r in tests["clause"][0]) + lang.grown(10**6, 10**6) + assert lang.trained == lang.nouns and lang.spoken_verbs == lang.verbs + + +@pytest.mark.parametrize("extra", [{}, {"full": [1]}, + {"grow": {"words": 200, "hold": 0, "ramp": 1}}]) +def test_a_life_reads_at_its_checkpoints(extra: dict, tmp_path: Path) -> None: + out = tmp_path / "life.jsonl" + rows = life.run_life({"words": 1000 if "grow" in extra else 200, "founder": 0, + "checkpoints": [1], "locate": False, "panel_batches": 1, + "out": str(out), **extra}) + assert rows[0]["batches"] == 1 and "clause" in rows[0]["panels"] + assert out.read_text().count("\n") == 1 + + +def _taught_small(settings: dict | None = None, batches: int = 3) -> tuple: + warnings.simplefilter("ignore") + lang = language.Language(4, 2) + brain = life.make_brain(lang, "Fd", 1, settings) + rng = np.random.default_rng(np.random.SeedSequence([1, 11])) + split = lang.split(1) + for _ in range(batches): + life.speak(lang, brain, lang.train_batch(rng, split), teach=True) + return lang, brain + + +def test_the_terms_sum_to_the_attention_the_read_uses() -> None: + lang, brain = _taught_small() + tests = lang.panels(1, 1) + out = life.terms(lang, brain, tests["clause"]) + assert out["check"] < 1e-6 + whole = out["all"] + assert whole["n"] > 0 and 0.0 <= whole["missed"] <= 1.0 + parts = sum(whole["gap"][name] for name in life.TERMS) + assert abs(parts - whole["gap"]["total"]) < 0.01 + + +def test_the_cortex_emulation_reads_unit_norm_and_a_fork_keeps_it() -> None: + warnings.simplefilter("ignore") + lang = language.Language(4, 2) + plain = life.make_brain(lang, "Fd", 1) + unit = copy.deepcopy(life.make_brain(lang, "Fd", 1, {"cortex": {"unit": True}})) + assert type(unit).__name__ == "Brain" and type(unit) is not type(plain) + x = lang.dense([(lang.go,)] * 2) + for brain in (plain, unit): + brain.reset() + brain.act(x, greedy=True) + raw = np.asarray(plain.recall.previous) + assert np.linalg.norm(raw, axis=1).min() > 0 + plain.recall.previous = raw / np.linalg.norm(raw, axis=1, keepdims=True) + for brain in (plain, unit): + brain.act(x, greedy=True) + np.testing.assert_allclose(plain.recall.attended_heads[0], unit.recall.attended_heads[0]) + + +def test_a_cortex_rate_of_zero_keeps_every_maps_cortex_columns_at_birth() -> None: + lang, brain = _taught_small({"cortex": {"rate": 0.0}}, batches=2) + assert brain.recall.head_map(0)[0].any() + for name in life.CORTEX_MAPS: + for head in getattr(brain.recall, name, None) or []: + assert not np.asarray(head[1]).any() diff --git a/docs/README.md b/docs/README.md index a3c70591..7a1af331 100644 --- a/docs/README.md +++ b/docs/README.md @@ -25,7 +25,7 @@ compares it with networks trained by backpropagation. ## Install ```sh -python -m pip install cadence-net==0.95.2 +python -m pip install cadence-net==0.96.0 ``` Cadence needs Python 3.11 or later and NumPy. Optional extras add speed and hardware: @@ -36,10 +36,10 @@ Cadence needs Python 3.11 or later and NumPy. Optional extras add speed and hard | `[accel]` | PyTorch | NVIDIA GPUs (CUDA) | | `[apple]` | MLX | Apple silicon GPUs | -For example `python -m pip install "cadence-net[fast]==0.95.2"`. From a checkout of the +For example `python -m pip install "cadence-net[fast]==0.96.0"`. From a checkout of the repository, run `python -m pip install -e .` at its root. A new brain picks the fastest backend the machine has for its size by itself ([backends](backends.md)). The -[v0.95.2 documentation](https://github.com/muellerberndt/cadence/blob/v0.95.2/docs/README.md) +[v0.96.0 documentation](https://github.com/muellerberndt/cadence/blob/v0.96.0/docs/README.md) is pinned to the release tag. ## Your first brain diff --git a/docs/api.md b/docs/api.md index d938bd64..086d5ff8 100644 --- a/docs/api.md +++ b/docs/api.md @@ -107,7 +107,7 @@ brain](anatomy.md#genome-development-brain)). Brain(connectome, *, episodic=True, consolidation=0.05, memory_rate=1.0, memory_amplitude=1.0, memory_choice=False, memory_context=0.0, memory_extinction=0.0, - memory_context_hold=None, maturity=False, maturity_scale=10.0, + memory_context_hold=None, memory_fade=0.0, maturity=False, maturity_scale=10.0, maturity_recovery=200000.0, working_memory_decay=0.0, working_memory_amplitude=3.0, efference_decay=0.2, efference_amplitude=0.0, @@ -135,6 +135,7 @@ of 8 decisions, demonstrations recorded at 1.0, 24 active separation cells) are | Rule | Raised | | --- | --- | | `memory_choice` needs `episodic=True` | `ValueError` | +| `memory_fade` in `[0, 1)`, and above 0 needs `episodic=True` ([records that fade](memory.md#records-that-fade-with-disuse-opt-in)) | `ValueError` | | `memory_context`, `memory_extinction` need `memory_choice`; `memory_context_hold` (`"unchanged"` or `"punished"`) needs `memory_choice` and `memory_context` | `ValueError` | | `learned_code` at least 2; `learned_code_fields` without it, `code` beside it, an image brain, the `torch` backend, or a connectome without the region's populations and complete synapses | `ValueError` | | `memory_context` in `[0, 1)`; `memory_extinction` in `[0, 1]`; `working_memory_decay`, `efference_decay` in `[0, 1)` | `ValueError` | @@ -204,7 +205,12 @@ their shape. Keep row identities fixed until `reset()`. writes `cadence-brain/2`, which 0.93.1 and earlier refuse: the same members, with the maturity (`scale`, `recovery`, `use`) under `maturity` in the metadata and the store's noise scales (`memories/hippocampus/gate_noise`). `Brain.load` refuses a `cadence-brain/1` file with a -maturity and a `cadence-brain/2` file without one. +maturity and a `cadence-brain/2` file without one. A brain whose store fades +([`memory_fade`](memory.md#records-that-fade-with-disuse-opt-in)) writes `cadence-brain/3`, which +0.95.2 and earlier refuse, with a maturity or without: the same members, with the fade under +the store's settings and its two sums (`memories/hippocampus/fade_depth`, `fade_recent`). +`Brain.load` refuses a `cadence-brain/3` file whose store does not fade, and a fading store +under either other name. | Members | What they hold | | --- | --- | @@ -217,10 +223,10 @@ maturity and a `cadence-brain/2` file without one. | `hearing/` | with `carried_gain`: the actor's eligibility of the gains (`trace`) and a pending act's contrast (`pending`); the gains, their second moments and their count are in the metadata, the read's route among them with `recall_drive_gain` | | `memories/history/` | the heard history's `trace` | | `memories/recall/` | the slots, `held`, `previous`, the address map, and where set the actor's `trace` and `pending` contrast of the map, `ended` and the window's `past`; the per-stream `told` and `focus` and the anchored, relative and slope maps of the options that have them | -| `memories/hippocampus/` | the associative store's ports, strengths, masses, consolidated records and separator; with maturity its noise scales (`gate_noise`) | +| `memories/hippocampus/` | the associative store's ports, strengths, masses, consolidated records and separator; with maturity its noise scales (`gate_noise`); with `memory_fade` its depths and recent traces (`fade_depth`, `fade_recent`) | | `memories/episodic/` | the episodic choice (`memory_choice`): the context store (`context_memory/`), the stream's `context` and `decision`, the replay `path/`, with the hold the context and code of the last decision (`before`, `seen`), and with extinction the direct payments (`paid_memory/`, sharing the place records' separator) | -`Brain.load` reads `cadence-brain/1` and `/2`, the formats of 0.85.0 on. It refuses the earlier +`Brain.load` reads `cadence-brain/1`, `/2` and `/3`, the formats of 0.85.0 on. It refuses the earlier `cadence-generic/1` to `/10` by name: cadence 0.94.0 loads those and saves `cadence-brain/1`. A retired setting loads at the one value this version keeps for it and is refused by name at any other ([CHANGELOG](../CHANGELOG.md)). @@ -236,6 +242,39 @@ population, so action indices keep their meaning after loading. A saved maturity 1e64), with the store's noise scales as nonnegative finite numbers under a boolean `surprise_gate`; anything else is refused with `ValueError` naming the field. +## Lives (`cadence`) + +```text +Lives(inputs, actions, lives, *, seeds=None, backend="cpu", device=None, precision=None, **options) +``` + +Many continuing brains of one anatomy, each composed by `Brain.compose(inputs, actions, seed=seeds[i], +episodic=False, **options)`, the default composition without the store (seeds `range(lives)` by default), +stepped together, one stream each. Every life has its own parameters, optimizer, critic, eligibility, recall +memory (its slots, its address and position maps), carried gains, working trace and random state. +`backend="cpu"` holds lone brains and steps them in turn; `backend="torch"` holds one brain whose parameters +carry a lives axis, on CUDA where there is a card and else on the torch CPU device (or the `device` named), +one call per moment for all of them ([many lives at once](backends.md#many-lives-at-once)). + +| Method | Meaning | +| --- | --- | +| `act(observations, *, greedy=False)` | each life's `Brain.act` on its own row: `(lives, inputs)` in, `(lives,)` out | +| `step(observations, *, reward=None, done=None, teacher=None)` | each life's `Brain.step`, its own reward, ending and teacher label | +| `learn(reward, done, next_observations)` | each life's `Brain.learn` | +| `lesson(observations, labels)` | each life's `Brain.lesson`; under `learning_surprise` each life is judged by its own answer | +| `reset()` | every life's stream starts afresh; what the lives learned stays | +| `parameters(i)` | life `i`'s learned state on the host: `efficacy`, `bias`, `w_critic`, `b_critic`, the optimizer's four moments and its counts `updates` and `contrast_updates`, with a recall memory `recall` (its maps by kind), with carried gains `carried_gains`, `carried_second` and `carried_count` | +| `life(i)` | life `i` as a lone `Brain` on the CPU with all of that and its random state; its stream starts afresh | +| `lives`, `seeds`, `brain`, `brains` | the count, the seeds, the batched brain (torch) or the lone brains (CPU) | + +`Lives` holds the default composition without the store, with or without the recall memory (`recall=False`) +and the carried gains, at any teaching point, with one or several regions, observers and motor slots. It +refuses, naming the part: the store (`episodic=True`) and `memory_choice`, arousal (so no `live`), a +maturity, a learned or fixed code, an apical compartment, rehearsal, qualified teaching, the efference copy, +the heard history, a recall memory with several heads, a window, the command, the read of when or tokens, +image inputs, and MPS, which has no float64 for the settles' checks. A moment any life refuses is refused for +every life. + ## Arousal (`cadence.arousal`) ```text @@ -342,9 +381,9 @@ Recall(connectome, amplitude=3.0, temperature=0.25, recency=0.25, address_reward=0.0, position_reward=0.0, window=1, heads=1, source="sensory", target="recall", inputs=0, writes=0, position=32, position_rate=0.1, - boundary=False, copy=0.0, tokens=None, action=False, anchor=False, + boundary=False, tokens=None, action=False, anchor=False, relative=False, slope=0.0, when=False, gain=0.0, grid=(), actions=0, drive_gain=False, - hearing=1.0) + hearing=1.0, cortex_norm=False, gain_after=0, gain_next=0.0, gain_lessons=0) ``` The held-input index memory of each stream's recent moments. Each constructor argument is the @@ -360,7 +399,6 @@ retired setting is refused by name with a sentence saying what the memory does i | --- | --- | | `stimulate(drive, observations)` | Addresses the moment, fixes the softmax over the slots (`attended_heads`) and writes the read into `recall` and, with `next`, `next`; with `heads` above one each head reads with its own address into its own port. | | `witness(observations, state, command=None)` | Writes the moment as a slot after its settle; with `action` the `(batch, actions)` command executed at it joins the slot's key and value (zero when None). | -| `copy_synapses()` | The copy pathway's synapses into the motor region, with the value coordinate and the motor neuron of each. | | `learn(graph, learned) -> dict` | The address lesson after a teaching step; returns `address_rows`, `address_error`, `address_refused`, and with a position term `position_rows` and `position_refused` (zero when nothing moved). A nonfinite step is refused per row (per head for the position maps). With several heads each head steps its own maps: rows and refusals are summed over the heads, the error averaged over the heads that moved. The position term learns whatever `address` is. | | `contrast_rows(graph, plus, minus)`, `propose(step)`, `accept(proposal)` | Reward training of the address map at `address_reward > 0` and of the position maps (and the anchored ones) at `position_reward > 0`, on the rows of the event heard; `ActorCritic` calls them. `contrast_rows` is `None` when neither trains the read. It holds the event's rows of the columns that can step (`reward_blocks`); `full_eligibility(kept)` lays it out over every row of the maps and all `reward_columns`, as a checkpoint keeps it, and `propose` takes either. | | `end(batch, rows=None)` | Ends episodes and keeps the slots, as `Brain.reset(keep=("slots",))` does; the latest slot learns no successor. With `boundary` an end of every stream opens one boundary slot. | @@ -455,7 +493,7 @@ LearnerConfig(beta=0.1, eta=0.2, eta_bias=None, free_steps=100, nudged_steps=50, normalize_floor=1e-3, momentum=0.0, scale_cap=8.0, qualified=False, damping=3, surprise=False, normalize_relative=0.0, nudged_fixed=False, rehearsal=0.0, rehearsal_capacity=512, - surprise_share=False, fast_settle=False) + surprise_share=False, fast_settle=False, compiled_moment=False) Learner(brain, outputs, config=LearnerConfig(), plastic_synapses=None, plastic_neurons=None, reciprocal=True, tie_groups=None, synapse_rate=None, slots=1, updates=0, contrast_updates=0) diff --git a/docs/backends.md b/docs/backends.md index 74a01aaa..116e75fe 100644 --- a/docs/backends.md +++ b/docs/backends.md @@ -8,7 +8,7 @@ NumPy float64 unless you pick another. Start from the NumPy-only [installation](../README.md#get-started) and add an extra: ```bash -python -m pip install "cadence-net[fast]==0.95.2" +python -m pip install "cadence-net[fast]==0.96.0" ``` `[fast]` installs Numba and SciPy, `[accel]` PyTorch and `[apple]` MLX (0.21.1 or later); from a checkout, use @@ -139,10 +139,26 @@ of 12 to 1,024 inputs crosses over there, within about 10% on either side, and s inputs on an Apple M4. On MLX a moment's reads join the launch that settles on them, and a slot is written in place in unified memory. -A memory with several heads, a window, `recall_action`, `recall_when`, `recall_gain` or -`recall_tokens`, a store with a separator or a surprise gate, `memory_choice` and a learned code keep -the moment's memories on the host at every size. On MLX a `maturity` reads the efficacies on the host at each -outcome, since it measures the step applied. +Some options keep the moment's memories on the host at every size. The device form in +[`cadence/resident.py`](../src/cadence/resident.py) holds one read port over one bank of slots keyed on one +event, and each of these adds a part that form does not hold, so the whole moment's memories stay with it on the +host, where the same equations run in NumPy: + +| Option | What it adds that the device form does not hold | +| --- | --- | +| `recall_heads=2` or more | several read ports over the one bank, each reading in turn from the share of every slot the heads before it left, each with its own address map | +| `recall_window=k` | a per-stream window of the last k events that the key and the address read in order | +| `recall_action=True` | a bank of the executed commands beside the slots, written from the host's action choice | +| `recall_when=True` | the read's age features, delivered on a region of their own | +| `recall_tokens`, `recall_gain` | the tokens' identity map into the motor region and one learned gain per port, taught by the motor's movement | +| a store with a separator, a surprise gate or `memory_fade` | the separator's expanded code of each key, the gate's noise over the consolidated strengths, or the fade's depth and recent use per key coordinate, read at every moment | +| `recall_cortex_norm=True` | the cortex of the moment before set to unit norm per stream before the maps read it | +| `memory_choice=True` | the episodic records: place and context records, their vote in the choice and the replay of a path | +| `learned_code=units` | a region that settles to its code before the moment's settle, with its own lessons | + +These brains take a device from four times the work ([the default backend](#the-default-backend)), so their +crossings count in the choice. On MLX a `maturity` reads the efficacies on the host at each outcome, since it +measures the step applied. Reading one of these arrays on the host (`brain.recall.keys`, `brain.working_memory.trace`, `brain.hippocampus.strength`, `brain.basal_ganglia.w_critic`), saving a checkpoint, copying the brain, @@ -154,10 +170,17 @@ but MPS, which is how the tests read this path on a machine without a GPU. `CADE same on MLX. Each moment crosses: to the device the observations, a sampled action's target, the outcome's -reward, key and dopamine; to the host the motor columns, a value per stream, one number per stream and +reward and key; to the host the motor columns, a value per stream, one number per stream and carried route, one small read per residual check and per settle, and one read of the lesson's or -update's report and finite checks. The memories, the eligibility and the optimizer's state do not -cross ([`tests/test_moment_on_device.py`](../tests/test_moment_on_device.py)). +update's report and finite checks. On torch the dopamine is made on the device from the value the +critic makes there, and an outcome reads its checks once before anything is installed: the dopamine's, +the critic's, the recall memory's reward step and the update's, with the critic's bias step, the +report's steps and the dopamine of every row, which the carried routes' gains take on the host. On MLX +the dopamine is the host's and goes to the device. The memories, the eligibility and the optimizer's +state do not cross ([`tests/test_moment_on_device.py`](../tests/test_moment_on_device.py)). +[`benchmarks/device/ledger.py`](../benchmarks/device/ledger.py) lists every crossing of a moment by the place +in the library that made it, with its bytes and time, and [its readings](../benchmarks/device/README.md) on an +A10G and an M4 give what each moment pays: its bytes are small, and its synchronizations are what it pays for. A stream's synapse eligibility takes one float per synapse, 25 MB at the speech size in float32, so `step` takes its streams from the device's memory: 256 of them fit an A10G's 22 GB in float32 and in float64, and 22 GB @@ -229,11 +252,28 @@ On an Apple M4, the same, single readings on a shared machine at a load of 12 to | The same | | 16 | 210, 938, 683 | 13.3, 89.2, 74.7 | MPS and other torch devices are not on the list: MPS reads slower than MLX and the CPU, and the others -have no reading; name them to use them. A CUDA card this table has not read takes the device from twice the -work, a stated margin that no measurement sets. Options that keep the moment's memories on the +have no reading; name them to use them. Options that keep the moment's memories on the host take a device from four times the work, and `precision="float64"` named on CUDA from four times, both reasoned from the readings above and not read at that size. A learned code stays off torch. +A CUDA card this table has not read is measured once. The first brain whose work reaches a quarter of +the table's crossing times the chamber's default brain on the card in float32 and on one core of the +machine: a greedy `act`, a `lesson` and a `step` at 16, 64 and 256 rows, two of each after one to warm. +Where the two cross, in `rows * synapses`, sets the card's crossings: the table's, scaled by that crossing over +the A10G's, which the same reading of the A10G table puts at 573,568 (43.7 rows of the chamber's brain). A card +half as fast as the A10G on that brain takes the device from twice the table's work, and a card twice as fast from +half of it. The calibration composes six brains and runs 72 moments, 54 of them timed. On an A10G it took +2.6 seconds and read the crossing at 43.8 and at 44.3 rows in two runs, within 1.4% of the table's. On an NVIDIA +L4, a card the table has not read, it read 59.8, 53.6 and 50.4 rows in three runs, 2.2 seconds each after the first, +whose 8.0 seconds held the device's start; the cached reading puts the L4's crossings at 1.15 times the table's, so +the chamber's brain takes the CPU at 64 rows and the L4 at 256 there. +Its result is kept beside the [stored transport plans](#choosing-kernels-by-measurement), in +`CADENCE_PLAN_CACHE` or `~/.cache/cadence/transport_plans.json` under `card `, so each card is calibrated +once per machine. `cadence.resident.calibrate_card()` runs it by hand and returns the readings, the crossing +and the seconds it took. `CADENCE_CALIBRATE=0` skips it, and a card without a calibration, or one whose +calibration fails, takes the device from twice the table's work, a stated margin +([`tests/test_default_backend.py`](../tests/test_default_backend.py)). + A choice is made once, at the first moment: rows that grow later do not move the brain. `brain.placement` states the `backend`, `device`, `precision`, `chosen_by` (`"auto"`, `"named"` or `"environment"`), `why`, `refused` and `pending`. `brain.describe()` and a checkpoint state the same wherever the brain is not on the CPU by name, whose @@ -347,6 +387,94 @@ Cap BLAS and Numba threads for parallel workers. To compare two implementations, dtype, steps and tolerances over several seeds, warm up, alternate the order, synchronize around each timed call, and compare residuals and states too. +## Many lives at once + +Search and self-play settle many positions or games each second. Which way to run them depends on whether +they are one brain's streams, sharing what they learn, or brains of their own. + +| You want | Use | +| --- | --- | +| One player in many games, a search over positions: one brain, many streams | `Brain.compose`, one row per stream in `step` or `act`, on a device | +| Many brains of their own on a CUDA card: a population, a league, independent seeds | `Lives(inputs, actions, lives, backend="torch")` | +| Many brains of their own on a Mac or a CPU machine | a process pool, one thread per process, or `Lives(..., backend="cpu")` in one process | + +```python +import numpy as np + +lives = cd.Lives(35, 18, 8, backend="torch") # 8 brains of seeds 0 to 7, on CUDA where there is one +observations = np.random.default_rng(0).random((8, 35)) +actions = lives.step(observations) # one row per life in, one action per life out +actions = lives.step(observations, reward=np.ones(8)) # each life learns from its own outcome +actions = lives.step(observations, reward=np.ones(8), teacher=np.arange(8)) # and from its own label +lone = lives.life(5) # life 5 as a lone Brain on the CPU, to save +``` + +`Lives` composes each life as `Brain.compose(inputs, actions, seed=s, episodic=False, **options)`: the default +composition without the store. Each life has its own synapses and biases, its own optimizer (momentum, second +moments and their counts), critic, eligibility, recall memory (its slots, its address and position maps), carried +gains, working trace and random generator. On a torch device the lives are the streams of one brain whose +parameters carry a lives axis: + +- a settle reads every stream through its own life's synapses and holds each life where its own stream meets the + tolerance, and a rescue on the damping ladder runs life by life; +- a lesson judges each life by its own answer under `learning_surprise`, runs its phases on every life and + updates the lives it taught, each at its own optimizer's count; +- the recall memory's address and position lessons, the carried gains' steps and a reward's update move each life + on its own row and its own dopamine. + +Each life follows the lone brain of its seed on that device ([`tests/test_lives.py`](../tests/test_lives.py)). In +float64 every action is the same and every learned part (synapses, biases, critic, optimizer, recall maps, carried +gains) agrees within 1e-9 over 30 moments of rewards, endings and teaching, and over 160 moments of the chamber's +composition within 1e-8; on an NVIDIA L4 over 48 moments within 1e-10. In float32 every action is the same over +those moments and the parts agree within the spread float32 gives a lone brain's own two memory paths (its +memories on the host against on the device: 1e-2 in the recall maps over 48 moments, where a float32 life on the +L4 reads 4e-3). On the CPU each life is the lone brain itself, to the bit. + +Seconds per life and moment of that composition stepping its own observations and rewards, read with +[`benchmarks/device/lives.py`](../benchmarks/device/lives.py) on one AWS g6.xlarge (an NVIDIA L4 and four vCPUs +of an AMD EPYC 7R13) and on an Apple M4 at a load of 4: + +| Way | Machine | 12 lives | 128 lives | 512 | 1,024 | +| --- | --- | ---: | ---: | ---: | ---: | +| `Lives`, `backend="torch"` | L4 | 1.18 ms | 0.118 ms | 0.043 ms | 0.049 ms | +| One brain's streams (shared parameters) | L4 | 0.619 ms | 0.119 ms | 0.045 ms | 0.034 ms | +| Process pool, four CPU processes | L4 box | 0.874 ms | 0.876 ms | | | +| Each brain in turn, one core | L4 box | 2.35 ms | 2.37 ms | | | +| Each brain in turn on the L4 | L4 | 6.52 ms | 6.66 ms | | | +| One brain's streams on MLX (shared parameters) | M4 | 0.690 ms | 0.156 ms | | | +| Process pool, two CPU processes | M4 | 0.595 ms | 0.662 ms | | | +| Each brain in turn, one core | M4 | 1.42 ms | 0.979 ms | | | +| `Lives` on the torch CPU device | M4 | 1.17 ms | 1.21 ms | | | + +The same with a teacher's label at every fourth moment (`--teach 4`): + +| Way | Machine | 12 lives | 128 lives | 512 | 1,024 | +| --- | --- | ---: | ---: | ---: | ---: | +| `Lives`, `backend="torch"` | L4 | 1.45 ms | 0.147 ms | 0.056 ms | 0.067 ms | +| One brain's streams (shared parameters) | L4 | 0.789 ms | 0.153 ms | 0.056 ms | 0.040 ms | +| Process pool, four CPU processes | L4 box | 1.12 ms | 1.13 ms | | | +| Each brain in turn, one core | L4 box | 2.97 ms | 3.03 ms | | | +| Each brain in turn on the L4 | L4 | 7.86 ms | 7.98 ms | | | +| One brain's streams on MLX (shared parameters) | M4 | 0.572 ms | 0.128 ms | | | +| Process pool, two CPU processes | M4 | 0.808 ms | 0.727 ms | | | +| Each brain in turn, one core | M4 | 1.56 ms | 1.48 ms | | | +| `Lives` on the torch CPU device | M4 | 1.48 ms | 1.32 ms | | | + +On the L4, `Lives` runs at the speed of one brain's streams at 128 and 512 brains and at 1.4 times their time at +1,024, where every life updates synapses of its own and the streams share one update. Against the four-process pool +it is 7.4 times as fast at 128 brains and 17.8 times at 1,024 (7.7 and 17.0 times with the teacher). At 12 lives +the pool is faster. A lesson of lives runs its phases on every life, where one brain's streams run them on the rows +it teaches. MLX has no lives axis in the library; one brain's streams there bound what one could reach. + +`Lives` holds the default composition without the store, with or without the recall memory and the carried +gains, at any teaching point, with one or several regions, observers and motor slots. It refuses, naming the +part: the store (`episodic=True`, whose consolidated records have no lives axis) and `memory_choice`; arousal, so +`live` stays a lone brain's; a maturity; a learned or fixed code and image inputs; an apical compartment; +rehearsal and qualified teaching; the efference copy and the heard history; a recall memory with several heads, a +window, the command, the read of when or tokens; MPS and MLX. A moment any life refuses is refused for every +life. Each life holds its own synapses, their dense blocks, two optimizer moments and an eligibility, five arrays +of one float per synapse, and in float32 a float64 copy of the blocks for the settles' checks. + ## Precision matters Near multiple attractors a rounding difference can change the selected state, and float32 diff --git a/docs/grow.md b/docs/grow.md index 775ead37..86839138 100644 --- a/docs/grow.md +++ b/docs/grow.md @@ -39,7 +39,7 @@ bigger.step(x, teacher=labels) | Setting | Adds | Starts at zero | | --- | --- | --- | -| `recall=True`, with any `recall_*` setting `compose` takes but `recall_tokens` | The [recall memory](memory.md#recall-memory) at [what a new brain's memory holds](memory.md#what-recalltrue-gives-a-new-brain) (`recall_founder=True` for its founders): its read ports, empty slots, and address and position maps at zero. The brain keeps its teaching point and its lesson settings, and the read's route comes only with the brain's own `carried_gain` | The ports' synapses into the association region (and a copy pathway's into the motor region) | +| `recall=True`, with any `recall_*` setting `compose` takes but `recall_tokens` | The [recall memory](memory.md#recall-memory) at [what a new brain's memory holds](memory.md#what-recalltrue-gives-a-new-brain) (`recall_founder=True` for its founders): its read ports, empty slots, and address and position maps at zero. The brain keeps its teaching point and its lesson settings, and the read's route comes only with the brain's own `carried_gain` | The ports' synapses into the association region | | `recall_heads=k` | More read heads of the brain's recall memory, over its bank | The new ports' synapses | | `history_decays=(...)` | A heard history (with `history_amplitude`), or the brain's decays followed by new ones | The history's synapses into the association region | | `efference_amplitude=a` (above zero) | An efference copy (with `efference_decay`) | The copy's synapses into the association region | @@ -267,8 +267,8 @@ new slot what the other brain knows. - **Devices.** The new brain lives on the backend, device and precision of the brain whose method made it. - **Checkpoints.** A grown or grafted brain is an ordinary brain: it saves and loads like any - other, in `cadence-brain/1` (with maturity `cadence-brain/2`; [checkpoint - format](api.md#checkpoint-format)). + other, in `cadence-brain/1` (with maturity `cadence-brain/2`, with `memory_fade` + `cadence-brain/3`; [checkpoint format](api.md#checkpoint-format)). - **Determinism.** The same brain grows, and grafts, to the same brain every time. What does not grow or graft: @@ -277,8 +277,8 @@ What does not grow or graft: `lateral`, `apical`, other `inputs`); compose the larger layout and graft instead; - `arousal`, `maturity` or the episodic memory (`memory_choice`), choices of the brain's construction; -- `recall_tokens`, which start the copy pathway as an identity onto the motor neurons: no recall - memory with them, and no heads or slots for a brain with them; +- `recall_tokens`, laid out per motor neuron and per read port: no recall memory with them, and + no heads or slots for a brain with them; - new slots where a memory is laid out per action (`recall_action`, `memory_choice`), or where the motor region is a deep brain's [apical](tune.md#deep-brains) pool (observers above it): a new neuron would join the pool's normalization; diff --git a/docs/memory.md b/docs/memory.md index 85fdf4e9..89547a6d 100644 --- a/docs/memory.md +++ b/docs/memory.md @@ -450,10 +450,12 @@ only where they differ from their founder. A saved brain keeps its genes, slots | `recall_slope` | `0.0` | construction | Above 0, every head's position term also leans by age at every length, taught at that rate ([the slope](#positions-at-lengths-no-lesson-reached)); needs `recall_position` | | `recall_grid` | `()` | construction | Past `recall_position`, every head's position term also reads each slot's age modulo each of these coprime numbers, and the slope takes no step ([long context](#reads-over-long-context)); needs `recall_position` | | `recall_relative` | `False` | construction | Every head's position term also reads each slot's age relative to the slot the previous read attended most ([the relative read](#positions-at-lengths-no-lesson-reached)); needs `recall_position` | +| `recall_gain_after` | `0` | construction | With `recall_gain`, the gains drive nothing and learn nothing through the memory's first this many lessons, and start then ([the value path released late](#the-value-path-released-once-the-read-has-formed-opt-in)) | +| `recall_gain_next` | `0.0` | construction | With `recall_gain`, every next port's gain starts at this value when the gains start, every recall port's at zero ([the value path released late](#the-value-path-released-once-the-read-has-formed-opt-in)) | +| `recall_cortex_norm` | `False` | construction | Every map reads the cortex of the moment before at unit norm per stream, so what every moment shares cannot outweigh the event in the address lesson ([the cortex at the event's weight](#the-cortex-at-the-events-weight-opt-in)) | | `recall_anchor` | `False` | construction | Every head's position term also reads each slot's place in its episode, so "the first told" holds at every story length ([the anchor](#positions-at-lengths-no-lesson-reached)); needs `recall_position` | | `recall_boundary` | `False` | retune | Each stream that begins holds a silent boundary slot whose next the first event teaches ([the episode's boundary](#episodes-keys-and-windows)); a retune applies from the next stream that begins | -| `recall_copy` | `0.0` | construction | Above 0, plastic synapses from every read port into the motor region at that initial magnitude ([the copy pathway](#the-copy-pathway-opt-in)) | -| `recall_tokens` | `None` | construction | Per action the observation coordinate of the same token (`-1` for none). With `recall_copy` the pathway starts as their identity, echo suppressed; without it no synapse is added, and a lesson whose answer has no token leaves the read's maps as they are | +| `recall_tokens` | `None` | construction | Per action the observation coordinate of the same token (`-1` for none). No synapse is added: a lesson whose answer has no token leaves the read's maps as they are, and `recall_gain` reads the tokens ([declared tokens](#declared-tokens)) | | `recall_action` | `False` | construction | Every slot carries the executed command in its key and value, and the maps read the command of the moment before ([its own actions](#its-own-actions-in-the-slots-opt-in)) | The [API reference](api.md#recall-memory-genes) gives the `Recall` class signature. @@ -781,6 +783,37 @@ development founders and 9 to 10 of 24 on fresh ones, at delay 5 reads 3 to 3 of wins 27 to 26 blocks of 30. Pass `recall=False` for a world paid by reward alone that gives the brain nothing to look up, and `recall_founder=True` for the founder memory. +#### Where each fall comes from + +A declared run on fresh founders (#280; one machine, 0.95.2) silenced one part of the +composition at a time and read the falls again. A part is a fall's cause when silencing it +brings the composition back to its control: + +| Fall | Control | Composition | Cause | Composition with the cause silenced | +| --- | --- | --- | --- | --- | +| Key-door, delay 2, rule A acquired | 14 of 24 | 7 of 24 | the read's drive | 12 of 24; with the working trace silenced too, 14 | +| Competing ring, hard nursery, skill A | .79 | .50 | the read's drive and the working trace, in part | .67 with both silenced | +| Binding continuing, held-out | .98 | .78 | lessons teaching only the rows that surprise | .98 with every row taught | +| Binding token, a name restated | .69 | .49 | the same | .83 | +| "How long ago?" | .69 | .54 | the read's recency of 0 | .70 at the founder's .25 | +| Hidden route, guided, sampled and greedy | .64, .58 | .19, .05 | none of the four parts alone or the two lesson parts together | at most .23, .15 | + +The control is `recall=False` on the first two rows and `recall_founder=True` on the rest. On +these founders the observed chain (34 and 38 of 48), the odour nursery `step` at 1,000 (16 and +18 of 30), the chain whose stages look alike (5 and 9 of 48) and binding token's held-out answers +(.96 and .92) do not fall. + +No cause can be removed without a larger cost. Each of the three removals was read against the +gates the composition holds: + +- **Every row taught** (`learning_surprise=False`) gives binding back and fails the grammar + chamber: paths .29 and two hops .05 against bars of .90 and .85. +- **The founder's recency** (`recall_recency=0.25`) gives "how long ago?" back and fails induction + at unseen lengths (.66 to .73 against .90), "was A told before B?" and copying. +- **The read starting closed**, its learned gain at 0 in place of 1, does not give the key-door + back (7 of 24). It costs the bandit (5 to 2 lives passing of 6) and the odour nursery `step` + at 300 (3 to 0). + ### Two heads on this composition (opt-in) Two heads at about one head's parameters: width 33, the read's temperature at .15. The first block read this @@ -1026,7 +1059,7 @@ for token in range(TOKENS): # no lesson used any of the - **A newborn is the founder.** The gains start at zero, so a newborn settles, acts and saves as one without the path: the motor drive gains exactly nothing. They are saved as the array `gains`, counted by `parameters()`, kept by `graft` and `grow`; `retune` refuses the setting. `grow` adds no - recall memory with tokens, as for the copy pathway. + recall memory with tokens. - **Limit.** The drive is a term held through the settle, which is sound for a held read. A read revised by the live association inside the settle would have to carry it in the reciprocal force, or the settle leaves the descent of one energy; the two are not composed. @@ -1061,7 +1094,7 @@ recipe in the place of the composition): the path answers early. On copying repeated tokens the read's lesson takes 406 rows against 971 on a development founder, and each moves the association a third as far or less. The read then falls at positions no lesson reached, and the answer follows it. On a path's hop the read's delivery falls the - same way. This is the copy pathway's failure beside the recipe, smaller. + same way. - **An action no lesson taught loses the vote.** On look back the read names the held token (0.46 to 0.98) and the answer is 0.00. Lessons on other actions push the association's drive on that action down, and the read is spread over the slots it holds, so the path's edge for the right token @@ -1071,53 +1104,42 @@ Use it for a brain that has to say tokens no lesson used where the answer is wha continuing, copying and inducing from what was heard. Leave it out of a brain that relies on repeated tokens at lengths no lesson reached, on Dyck sentences or on chained facts. -### The copy pathway (opt-in) +### The value path released once the read has formed (opt-in) -A held read reaches the motor region only through the association region, which learns which -answer each read deserves. A token taught as an answer is then produced from its read; a token -never taught as an answer is read as well as any and never produced, because no lesson ever -raised its action. `recall_copy` adds a direct pathway from every head's read ports (`recall`, -and `next` where it is on) to the motor region: plastic synapses learned by the same settle -contrast as every other synapse, inside the one settle. +The learned gains answer for the read from the first lesson. A gain that answers early takes the +lessons the read itself needs: a surprise lesson teaches only rows that are wrong, so once the gain +says the answer the read is taught less, and a read that depends on a read (a second hop) never forms. +`recall_gain_after=n` holds the gains at zero through the memory's first `n` lessons: they drive +nothing and learn nothing, and the read is taught on every row it gets wrong. At the `n`-th lesson +every next port's gain starts at `recall_gain_next` and every recall port's at zero, and they learn at +`recall_gain` from there. The next port is the one that carries a successor, which is what a read +lands one slot before; a recall port's gain would say the token the read is addressed by. ```python -TOKENS, HEAR, ASK, NOISE = 6, 6, 7, 8 -copier = Brain.compose( - inputs=9, actions=TOKENS, modules=(16,), seed=3, episodic=False, recall=True, recall_founder=True, - recall_copy=2.0, recall_tokens=range(TOKENS), # observation i and action i: one token +import cadence as cd + +brain = cd.Brain.compose( + 20, 12, recall=True, recall_cortex_norm=True, + recall_gain=1.0, recall_tokens=list(range(12)), + recall_gain_after=1000, recall_gain_next=10.0, ) -for token in range(TOKENS): # no lesson: every token is untaught - copier.reset() - copier.act(np.eye(9)[[token]] + np.eye(9)[[HEAR]], greedy=True) # a token is heard - copier.act(np.eye(9)[[NOISE]], greedy=True) # something else - cue = np.eye(9)[[HEAR]] + np.eye(9)[[ASK]] # asked for it back - assert copier.act(cue, greedy=True)[0] == token ``` -- **Without tokens** (`recall_copy=s` alone) the synapses are drawn at magnitude `s` as every - projection is, appended last, so the brain with the pathway is the brain without it plus these - synapses. Learning shapes them like any synapse; nothing lets a read drive an action that was - never taught. -- **Shared identities** (`recall_tokens`). Where an observation coordinate and an action are the - same token, declare it: `recall_tokens[j]` is the observation coordinate of action `j` (`-1` - for none). The pathway then starts as that identity: from each recall port, the read's own - value, at `recall_copy`; from each next port at zero; and from the present observation (the - sensory region) at `-recall_copy`, so a token the moment holds is not copied back. - Every other synapse of the pathway starts at zero. The tokens must be in the recall values: a - brain without a code, or a learned code joined to an observation no wider than it; a fixed - code is refused. -- **Every synapse learns on its own.** A lesson moves the synapses of the tokens the read and - the observation held; the identity of a token never read stays where it started, so a token - never taught as an answer keeps its route to its action. -- **Why the echo.** A read addressed by the present moment holds that moment's own token in its - value: in a succession task (a cue names a token, the answer is the token that followed it) - the recall port holds the cue's token and the next port the answer. Copying the recall port - alone answers the cue with itself; the echo removes what the observation holds. -- **Why the next port starts at zero.** Which port holds the answer depends on the task: the - heard token is on the recall port in a copy-back question, the successor on the next port in - a succession. The pathway starts from the recall port, the token the read delivers, and the - next port's gain is learned. Started equal, the two ports would drive the heard token and the - distractor that followed it alike. +`brain.recall.gain_lessons` counts the lessons the gains have seen, and a saved brain keeps it. + +Declared on the indirect-object chamber with `recall_cortex_norm`, twelve fresh founders: never-taught +names are said at .97 to 1.00 on 7 lives of 12, every life whose circuit forms, and the taught second +hop reads .99 to 1.00 on 12 of 12. With the gains from birth the circuit forms on 1 or 2 lives of 6 and +the second hop reads .00 to .45 (development); with both ports released at 30 the second hop reads +.00 to .05. The five lives that miss fail at the read's selection, before the gains are released. + +### Declared tokens + +Where an observation coordinate and an action are the same token, declare it: +`recall_tokens[j]` is the observation coordinate of action `j` (`-1` for none). The declaration +adds no synapse. It tells the read's lessons which answers a slot can hold, and +[the learned value path](#the-learned-value-path-opt-in) (`recall_gain`) reads it. + - **Answers that are no token.** The read's lessons take their target from the slots' contents. With tokens declared, a lesson whose answer is an action without a token (`-1`: a yes or a no, a count) leaves the address and position maps as they are; the cortex learns @@ -1128,45 +1150,11 @@ for token in range(TOKENS): # no lesson: every token is classes carry one class at a time and never the comparison that separates them ("before" stays at chance after 4,500 lessons, against 0.97 at every row). Lessons at every row collapse the token answers of induction, copying and two hops (0.11 to 0.68 on development founders), - so those stay judged by surprise. The tokens declare this without the pathway as well - (`recall_tokens` without `recall_copy`: no synapse is added), and an action list without any - token (`recall_tokens=[-1] * actions`) declares that no answer is a slot's content: no lesson - moves the read and every row is taught. -- **Beside the positions recipe.** The address and position lessons name the slots to read by - the association's movement read back through the read port. With the pathway the answer - reaches the motor region directly, the association moves little toward it, and a read that - only position finds (look back) stops learning where to look: taught at 5 to 12 symbols and - read at 24 to 60, the recipe with the pathway answers 0.25 to 0.33 on five fresh founders of - six (the recipe alone 0.99 to 1.00), its position, anchored and slope maps at a third, a - quarter and a sixth of their norms. Reading the motor's movement back through the pathway - restores look back (0.99) and misdirects reads whose answer is a successor (two hops 0.27 to - 0.28): the recall port's identity names the slot holding the answer itself, not the slot it - follows. Leave the pathway out of a brain that relies on the recipe. - -In the [copy-tokens benchmark](../benchmarks/attention/copy_tokens.py) a token is heard, two -other tokens follow, and a cue asks for the heard token back; tokens 0 to 4 are taught as -answers and 5 to 7 never are. A second task cues a token of a sequence of five heard once and -asks for the token that followed it. On six founders of one module -([receipt](../benchmarks/attention/results/copy-tokens.json)): - -| Arm | Untaught tokens produced | Taught answers | Succession | Read of the untaught | -| --- | --- | --- | --- | --- | -| founder memory | 0.00 | 0.72 to 0.98 | 0.95 to 0.99 | 0.32 to 1.00 | -| `recall_copy=2.0` with tokens | 1.00 on every founder | 1.00 | 0.94 to 0.99 | 1.00 | -| `recall_copy=1.0` with tokens | 0.17 to 1.00, mean 0.78 | 0.86 to 1.00 | 0.95 to 1.00 | 0.98 to 1.00 | -| `recall_copy=0.5` without tokens | 0.00 | 0.57 to 0.99 | 0.93 to 1.00 | 0.58 to 1.00 | -| `recall_copy=1.2` without tokens | 0.00 | 0.22 to 0.87 | 0.86 to 0.97 | 0.70 to 1.00 | - -Uniform random answers 0.125. Use `recall_copy=2.0` with tokens. With it the answer follows its -read on every untaught and taught question, and a deep apical brain (`modules=(32, 24, 24)`, -`apical=0.25`) reads the same. A weaker identity loses the untaught tokens on some founders, the -association having learned the taught answers first; a pathway drawn without tokens costs taught -answers and succession. A successor never taught as an answer stays at 0.00 in every arm, since -the next port's gain grows slowly. - -Use it when the answer is a token the read can deliver, and compare it with the founder memory -and with the pathway drawn without tokens on the task. The options are construction settings, -described and saved only where they are on. + so those stay judged by surprise. An action list without any token + (`recall_tokens=[-1] * actions`) declares that no answer is a slot's content: no lesson moves + the read and every row is taught. +- **The tokens must be in the recall values:** a brain without a code, or a learned code joined + to an observation no wider than it. A fixed code is refused where the gains read them. ### Its own actions in the slots (opt-in) @@ -1263,15 +1251,35 @@ interpretation cannot revise the query while that settle forms. The benchmark es bounded contextual lookup. Compare it on the task with attention off, episodic memory alone and uniform random. Several facts in one answer take [several heads](#several-heads-opt-in). +### The cortex at the event's weight (opt-in) + +The read's address and position maps step on what they read: the event heard and the cortex of the +moment before, `z = [x, c, 1]`. A one-hot event has an energy of 1, and the cortex can hold twenty +times that while it changes little from one moment to the next. A lesson's step then lands mostly on +the cortex columns, and what it teaches at the answer applies at every moment, since every moment +shares most of that cortex. `recall_cortex_norm=True` gives every map the cortex at unit norm per +stream, so the event and the cortex weigh alike in the lesson and in the read. The keys, the values +and the settle are unchanged; the option keeps the memory off the device path. + +On the [indirect-object chamber](../benchmarks/heads/README.md#the-cortex-at-the-events-weight-and-a-value-path-released-late) +("when Mary and John went out, John gave a drink to"), declared on twelve fresh founders, the read's +circuit forms on 7 lives of 12 with the option against 2 without it, and the taught second hop of a +read on a read holds on 12 of 12. It does not carry the crossing of a question with a context +([the crossing chamber](../benchmarks/attention/README.md#the-crossing-a-question-whose-fact-depends-on-the-context-crossingpy)), +and at 1,000 raw words the read falls ([the vocabulary chamber](../benchmarks/vocabulary/README.md)). + +```python +import cadence as cd + +brain = cd.Brain.compose(20, 12, recall=True, recall_cortex_norm=True) +``` + ### Recall cost The two regions add one neuron per value coordinate each: `compose(250, 2)` grows from 380 to 880 neurons and from 20,352 to 52,352 synapses. `recall_action` adds one neuron per motor neuron to every read port region, their synapses into the association, and as many rows to the -address and position maps and columns to the address; `recall_copy` adds the value width times -the motor neurons per port region (and with tokens one synapse per sensory neuron and action): -with two heads over a code of 128 and 323 actions the pathway adds 4 x 128 x 323 = 165,376 -synapses. Each further head adds its two regions, their projections and one more address map. +address and position maps and columns to the address. Each further head adds its two regions, their projections and one more address map. The address map is `code` by `address_width` plus the association width by `address_width`; with compose's 64 association neurons it is 0.66 MB at 256 inputs, 1,322 MB at 12,824 inputs on the uncoded observation and 8.9 MB with `code=1024`. One stream's slots hold `capacity` @@ -1288,9 +1296,7 @@ with `recall_next=False`), each with one neuron per value coordinate, `Genome.observation_width(inputs, code)`, and with `recall_heads=k` the regions `recall_h` and `next_h` for h from 1 to k - 1 (`Recall.ports(k)` names them). With `recall_action` each of those regions has a `_action` region of one neuron per motor neuron -(`Recall.action_ports(k)`); with `recall_copy` synapses from every read port neuron into -`motor`, and with `recall_tokens` from every token's sensory neuron into its action, which the -brain starts as the declared identity. Without `recall=True` populations of those names are +(`Recall.action_ports(k)`). Without `recall=True` populations of those names are ordinary ones. `tests/test_recall_memory.py` checks that the library with `recall_position=0` reproduces the reference memory `benchmarks/binding/index_read.py` moment for moment. To check that the memory does the work, erase a copy's slots with `brain.recall.reset(batch)` before the @@ -1303,11 +1309,6 @@ query ([ablation](#check-that-a-memory-does-the-work)). reaches the linear map only through conjunctions held in the cortex of the moment before. - The slots are a record within one stream. Nothing consolidates across streams except through the plastic synapses that read them. -- The copy pathway's identity starts on the recall port. A task whose answer is on the next - port (a successor) has to learn that port's gain, which grows slowly; the association learns - the taught successors first. The identity reaches only tokens in the recall values: a learned - code narrower than the observation keys on the code alone, and there the pathway is drawn at - random. - With `recall_action` the commands reach the answer through synapses that start at zero; under reward alone the address and position maps do not learn (`recall_address_reward` 0), so only the values' commands are read. @@ -1660,6 +1661,71 @@ credit and demonstrations. The context records (`brain.context_memory`) and the payments of `memory_extinction` write as without maturity. The noise scales are saved and loaded with the store and counted by `parameters()`; `clear()` erases them with the records. +### Records that fade with disuse (opt-in) + +A memory nobody uses should fade, and a skill practised for a long time should fade far more +slowly. `memory_fade` gives the store's place records that dynamic. It is off at the founder +(0.0), and a brain without it is the founder's to the bit. + +```python +brain = Brain.compose(inputs=6, actions=3, seed=0, recall=False, maturity=True, memory_fade=1.2e-4) +assert brain.hippocampus.fade == 1.2e-4 +``` + +Each key coordinate `k` of the consolidated records keeps two sums of activity, saved with the +store (`brain.hippocampus.fade_depth`, `fade_recent`): + +- **its depth**, the magnitude of the unit cue on `k` summed over every moment that cues it and + every write that uses it; +- **its recent use**, the same sum over the moments, kept at `1 - 1/2,000` each moment. + +Every moment of life (`live`, `step`) the records along `k` lose a share of themselves: + +```text +idle_k = 1 / (1 + (recent_k / 0.01) ** 2) +lost_k = memory_fade * idle_k / (1 + (depth_k / 2000) ** 4) +``` + +A record cued every few hundred moments has a recent use far above 0.01 and loses under a +billionth of itself a moment. One left alone for about fifteen thousand moments starts to fade, at `memory_fade` a moment while it +is shallow and far more slowly once its depth passes 2,000. Each stream keeps its residual +around the shared records. No count is kept anywhere: both sums are leaky integrals of +activity, as [maturity](reward.md#maturity-opt-in)'s use is. The constants are +`SynapticMemory.FADE_SCALE`, `FADE_ORDER`, `FADE_IDLE` and `FADE_FLOOR`. `memory_fade` is a +construction setting: `retune` refuses it, and `grow`, `graft`, `save` and `load` keep it with +its sums. A brain with the setting saves checkpoint format `cadence-brain/3`, which 0.95.2 and +earlier refuse ([checkpoint format](api.md#checkpoint-format)); without it a brain saves the +format it saved before. + +**What it does** (#253's declared run, 24 fresh founders of the competing-skills chamber, with +`maturity=True` on every arm). A brain practises skill A under the certain pay, lives 40,000 +blank moments with nothing paid, then returns to A. Retention is the right action's probability +at the window's end, `(p - .25) / (p_end - .25)`; relearning is the return's first 500 moments +less the first practice's: + +| Practice | Retention with `memory_fade=1.2e-4` | Without | Relearning with it, points (s.e.) | +| --- | --- | --- | --- | +| 2,000 moments (shallow) | .36 | 1.00 | +4.3 (1.1) | +| 40,000 moments (deep) | 1.00 | 1.00 | +4.9 (1.1) | + +The shallow skill fades within the window and the deep one keeps it, and relearning after +disuse is faster than first learning at both depths. On the same founders every chamber from +birth reads as the brain without the fade (the key-door at delays 2 and 5, the odour nursery +`live` and `step` at 300 and 1,000, the bandit, the contextual bandit and delayed response), and +so do P4 with a mature skill A (A gains 2.4 points over the ring, B .24) and the short ring +(A loses 28.8 points, B .35), each to the same counts and means as without it. + +A sense that is active at every moment keeps its records in use. In a body whose senses stay +busy, only the records of the senses that fall silent fade. Twelve brains of the +[Robot Arena demo](https://github.com/muellerberndt/cadence-demos/tree/main/robot-arena) read +this way ([#253](https://github.com/muellerberndt/cadence/issues/253)): over a nursery the +fade takes records of "outside the ring", a sense that falls silent as a brain learns to stay +inside, and little else. The fights, the acquisition from birth and the revision test there +hold every gate that maturity alone holds. + +Use it for a continuing brain whose skills come and go over long lives. In lives shorter than +about fifteen thousand moments it changes nothing. + ## Learned weights and eligibility The learned weights (synapse scales and biases, the actor's readout and the critic) are the diff --git a/docs/reward.md b/docs/reward.md index a7ad4b21..16f9c6b4 100644 --- a/docs/reward.md +++ b/docs/reward.md @@ -173,8 +173,8 @@ counted by `parameters()` with the use, a pass over the synapses on each actor u measure the step, and the gate's arithmetic on each write of the store. The variable (`brain.basal_ganglia.maturity`, a `Maturity`) and the noise scales are saved and loaded with the brain in checkpoint format `cadence-brain/2`, which 0.93.1 and earlier refuse -([checkpoint format](api.md#checkpoint-format)). Off, the brain is the founder's to the bit and -saves `cadence-brain/1`. +([checkpoint format](api.md#checkpoint-format); `cadence-brain/3` where the store also fades). +Off, the brain is the founder's to the bit and saves `cadence-brain/1`. ## Decision and reset contract diff --git a/docs/settling.md b/docs/settling.md index 195d7073..1584863e 100644 --- a/docs/settling.md +++ b/docs/settling.md @@ -553,12 +553,13 @@ Select rates on development data and freeze them. | `surprise` | `LearnerConfig` | [teach only differing rows](teach.md#lessons-only-where-the-answer-differs) | `False` | | `surprise_share` | `LearnerConfig` | with `surprise`, scale a lesson's step by the share of its rows it taught ([in proportion to the surprise](teach.md#in-proportion-to-the-surprise)) | `False` | | `fast_settle` | `LearnerConfig` | a composed brain's free solves on the compiled CPU path: rows that stop apart, the activation in vectorizing arithmetic ([settling](#settling-and-equilibrium)) | `False`; `True` in a new brain with the memory | +| `compiled_moment` | `LearnerConfig` | a composed brain's work around the solve on the CPU in compiled calls: a greedy act in one (read, solve, choice, write), a lesson's judgment in one and a taught lesson in one more (its phases with their checks, the contrast, the update, the address lesson and the carried gains' step); other bits ([the compiled moment](tune.md#the-compiled-moment-opt-in)) | `False` | | `rehearsal`, `rehearsal_capacity` | `LearnerConfig` | in a composed brain, after each lesson replay that many times its rows again, each with its episode, drawn from a reservoir of that many lesson rows ([rehearsal](teach.md#teach-several-types-in-sequence-rehearsal)) | 0.0 (off), 512 | | `outputs`, `config` | `Learner` | output neuron indices; the `LearnerConfig` | required, `LearnerConfig()` | | `plastic_synapses`, `plastic_neurons` | `Learner` | bool masks of moving efficacies and biases | all | | `reciprocal` | `Learner` | tie each synapse and its reverse into one efficacy | `True` | | `tie_groups` | `Learner` | int per synapse, `-1` untied; a group shares one step | `None` | -| `synapse_rate` | `Learner` | nonnegative step multiplier per synapse, before tying. A rate of zero holds that synapse's efficacy; the biases have no rate and step, and the optimizer's running moments of a held synapse keep reading its contrast, so a rate raised later steps on that history | `None` (1); a new brain's [width](tune.md#wide-brains) sets it above 64 | +| `synapse_rate` | `Learner` | nonnegative step multiplier per synapse, before tying. A rate of zero holds that synapse: it takes no step of its own and the optimizer keeps its running moments as they are. A neuron whose every inbound synapse steps at zero holds its bias and the bias's moments too (`Learner.held()` names both), so a rate of zero on every synapse into a region freezes that region. A reciprocal pair shares one efficacy and steps by the mean of its two directions' steps, so the pair holds where both directions are at rate zero; one direction alone at zero takes half its partner's step, while `held()` names it and its moments stay | `None` (1); a new brain's [width](tune.md#wide-brains) sets it above 64 | | `slots` | `Learner` | outputs split into softmax groups: a count of equal groups or a size per group | 1 | | `gain`, `slope`, `leak`, `dt`, `stimulus_amplitude` | `learning_neuron_model` | synaptic gain, activation slope, sub-rest response, integration step, input scale | 1.0, 1.0, 0.1, 0.5 (`Brain.compose` uses 1.0), 1.0 | | `density`, `feedback`, `lateral`, `seed` | `layered` | input-to-hidden density, feedback scale, output-to-output weight, wiring seed | 0.3, 1.0, 0.0, 0 | diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 3195dba4..0a298741 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -5,7 +5,7 @@ Short answers to the problems builders meet first, each linking the page that ow ## Install and run -**`pip install cadence-net==0.95.2`, then `import cadence`.** Python 3.11 or newer and NumPy are +**`pip install cadence-net==0.96.0`, then `import cadence`.** Python 3.11 or newer and NumPy are the only requirements. **Do I need a GPU?** No. A new brain runs on the CPU unless its size makes a GPU faster and one diff --git a/docs/tune.md b/docs/tune.md index 68c85f92..568f2d6b 100644 --- a/docs/tune.md +++ b/docs/tune.md @@ -55,6 +55,7 @@ step size. The defaults of a new brain suit most work; change one at a time and | --- | --- | --- | --- | | Fewer neuron updates per answer, the same equations | `learning_fast_settle=True` | on (off with `recall=False`) | rows stop apart and the activation vectorizes; other bits than the ordinary solve ([settling](settling.md#settling-and-equilibrium)) | | The ordinary solve | `learning_fast_settle=False` | | every row runs to the slowest row's check | +| The work around each solve compiled, the same equations | `learning_compiled_moment=True` | off | a greedy act in one compiled call on the CPU, a lesson's judgment in one and its phases in one more; other bits than the ordinary path ([the compiled moment](#the-compiled-moment-opt-in)) | | Fewer sweeps, looser answers | `learning_tolerance=0.01` | `0.003` | about a fifth fewer sweeps per act ([what a moment costs](#what-a-moment-costs)) | | Lessons refused unless every phase satisfies the equations | `learning_qualified=True, learning_nudged_steps=1024` | off | stricter and slower lessons ([qualified teaching](settling.md#qualified-teaching-budgets)) | | Larger or smaller lesson steps | `teaching="founder"`, `"normalized"` or `"deep"` | `"deep"` (`"founder"` with `recall=False`) | the size and form of each lesson's step ([which teaching point](teach.md#which-teaching-point)) | @@ -87,7 +88,7 @@ The composed graph, in the order its regions are developed: | `observer_0` ... | `observers` | Reciprocal with every module, `motor` and earlier observers | below | | `efference` | `actions` | With `efference_amplitude` above 0 | [memory](memory.md#efference-copy) | | `history` | sensory width per decay | With a nonempty `history_decays` | [memory](memory.md#heard-history) | -| `recall`, `next` | observation-code width | With `recall=True`; `next` unless `recall_next=False`; with `recall_copy` also into `motor` | [recall memory](memory.md#the-recall-memory-in-full) | +| `recall`, `next` | observation-code width | With `recall=True`; `next` unless `recall_next=False` | [recall memory](memory.md#the-recall-memory-in-full) | | `recall_action`, `next_action` | `actions` each | With `recall_action=True`, one per read port region; synapses into the association start at zero | [its own actions](memory.md#its-own-actions-in-the-slots-opt-in) | | `recall_when`, `recall_1_when` ... | 2 each, 4 with `recall_anchor` | With `recall_when=True`, one per head; synapses into the association start at zero | [when a read item happened](memory.md#when-a-read-item-happened-opt-in) | @@ -147,6 +148,7 @@ settings and counts), `pending` (whether an issued action owns the next outcome) | `learning_scale_cap`, `learning_surprise`, `learning_surprise_share` | `8.0`, `False` (`True` in a new brain with the memory), `False` | retune | [learning](settling.md#7-every-knob) | | `learning_rehearsal` | `0.0` (off) | retune | [rehearsal](teach.md#teach-several-types-in-sequence-rehearsal) | | `learning_fast_settle` | `False`, `True` in a new brain with the memory | retune | [settling](settling.md#settling-and-equilibrium) | +| `learning_compiled_moment` | `False` | retune | [the compiled moment](#the-compiled-moment-opt-in) | | `actor_gamma`, `actor_lam`, `actor_eta`, `actor_eta_bias`, `actor_eta_critic` | `0.9`, `0.8`, `1.0`, `0.05`, `0.3` | retune | [reward](reward.md#eligibility-traces-and-dopamine) | | `maturity`, `maturity_scale`, `maturity_recovery` | `False`, `10.0`, `200000.0` moments | `maturity` at construction; the scale and the recovery retune | [maturity](reward.md#maturity-opt-in) | | `arousal` | off; `True` is `ArousalConfig()` | construction (switch), retune (fields) | [continuous](start.md#routine-and-repair-live) | @@ -275,6 +277,14 @@ sharing across positions is a second form, read only if this one holds. The foun and is unchanged. The [grammar chamber](../benchmarks/grammar/README.md#a-planning-window) reads the window on its sentences at `k` of 1, 3, 4 and 8. +When to use it: the window pays where two actions inside it constrain each other and the trace +alone does not carry the constraint. It is no lookahead. On +[routes with a goal](../benchmarks/grammar/README.md#routes-with-a-goal-does-planning-pay-257), +where the first step must lead to a goal two steps on, a sliding window of three (plan three, take +the first, plan again) chooses the first step no better than word by word (.31 against .33 on +held-out pairs; each says the wrong neighbour as often as the right one), so a plan that must search ahead needs +more than a joint settle. + Every bias starts at zero and is plastic: `brain.brain.bias` holds the learned values, and loading restores them. To start the processing neurons at a bias of your own, set `brain.learner.brain = brain.learner.brain.with_parameters(bias=bias)` before the first lesson. @@ -336,21 +346,62 @@ take about three tenths and the memory's read a sixth, and most of the rest is P many small steps around the kernels that a network's compiled batch does not make. What a pass reads at these costs is in the [chamber's tables](../benchmarks/grammar/README.md). -| Work | Brain | One thread, Apple M4 | -| --- | --- | ---: | -| One stream of speech: 3,920 inputs, association 512, one act per 500 ms of audio | founder settings | 0.9 ms an act, 25 ms a taught lesson | -| The same | the recipe | 10 ms, 190 ms | -| The same | the recipe with `history_decays=(0.5, 0.9)` | 14 ms, 270 ms | -| The same | a new brain | 14 ms, 291 ms | -| Long context: 16 streams, 256 inputs, 64, 1,024 and 4,096 slots | the chamber's recipe | 1, 6 and 21 ms an act, 7, 27 and 99 ms a moment with its lesson | - -At the speech size the read's address map, 3,920 by 3,920 entries, is most of an act, and with -thousands of slots the read is: at 4,096 slots it is 20 of the act's 21 ms, and the write 0.3 ms. A -realtime loop's limit is the act: at the speech size a lesson that teaches costs about twenty times -an act, and only the act has to keep up with the stream. +### The compiled moment (opt-in) + +`learning_compiled_moment=True` does the work around the solve kernels in compiled calls on the +CPU. A greedy act is one call: the moment's drive (the sensory part, the working trace, the +memory's read, with the address and every position term through one product of the maps, kept +packed until a lesson replaces one, and the place store's read), the free solve, the choice, the +trace's step and the memory's write. A lesson by surprise judges in one call, the read and the +solve its answer is read from, and where it teaches a row the lesson is one call more, from its +phases to its written efficacies: the free phase, the two nudged phases and their checks, the +contrast, the optimizer's step with the tied and bounded efficacies, the address lesson and the +carried gains' contrast and step; Python stores what the call returns. An act after a lesson that +taught nothing takes the lesson's read and solve and writes its moment in the choice's pass. The +equations are the same and the sums run in another order, so the bits are other ones; off, every +bit is the ordinary path's. A brain with several read heads, a window, the command or when in its +slots, the gains, a learned code or a separator reads through the ordinary path, and a moment the +one call does not take whole runs as the passes it is made of: a solve that needs its rescue, a +sampled act, a lesson of a learner with frozen or tied synapses or per-synapse rates, an update +that is refused. + +[`cost.py`](../benchmarks/grammar/cost.py) times a pass of the default brain, the same brain with +the option and the chamber's transformer in one process, batch by batch in turn, on one thread. On +one core of a Xeon 8488C with nothing else running, one founder at a time, in one session: + +| Founder | Default brain | With `learning_compiled_moment` | Transformer | Against the transformer: default, compiled | +| --- | ---: | ---: | ---: | ---: | +| 0 | 67.6 s | 36.7 s | 16.0 s | 4.2, 2.3 | +| 1 | 70.5 s | 38.5 s | 15.9 s | 4.4, 2.4 | +| 2 | 70.8 s | 39.5 s | 16.2 s | 4.4, 2.4 | + +The option takes .54 to .56 of the default's seconds. The seconds of one machine differ from run +to run and from machine to machine by a tenth and more, the transformer's more than the brain's: +across sessions the default read 4.2 to 5.3 times the transformer, so compare within one run. At +a default of 4.8, .54 to .56 of its seconds is 2.6 to 2.7 times the transformer. + +`cost.py --parts` reads one brain's pass by part, each function's seconds less those of the timed +functions inside it, the kernels at the functions that launch them (founder 0, alone on the core): + +| Part | Default brain | Compiled moment | +| --- | ---: | ---: | +| Kernels: the solves; with the option the act's and the judgment's one call (read, solve, choice, write) | 18.0 s | 20.8 s | +| Kernels: the lessons' free and nudged phases and their checks; with the option a taught lesson's one call (phases, checks, contrast, update, address lesson, gains) | 5.3 s | 6.0 s | +| The read, as a pass of its own | 20.8 s | 0.0 s | +| Around the kernels of an act: the solve's layers, the choice and the write, the rest | 12.5 s | 4.2 s | +| Around the kernels of a lesson: the phases, the update, the address lesson, the gains, the rest | 9.3 s | 3.0 s | +| The chamber's own loop | 2.0 s | 1.7 s | +| The pass | 67.9 s | 35.9 s | + +The kernels take 26.9 seconds, 1.7 times the transformer's pass. What stands around them takes +7.2 seconds with the option: about 85 microseconds of Python around an act's call and 130 around +a lesson's, which check the public call's arguments, hand the compiled call its arrays and store +what it returns. The kernels and the chamber's loop are .42 of the default's seconds and the +Python around the calls .11, which fewer calls per moment would lower. | To spend less | Setting | What changes | | --- | --- | --- | +| The work around the solve | `learning_compiled_moment=True` | An act in one compiled call, a lesson's judgment in one and a taught lesson in one more: .54 to .56 of a chamber pass of the default brain on one Xeon core, 2.3 to 2.4 times the transformer's seconds where the default takes 4.2 to 4.4 ([the compiled moment](#the-compiled-moment-opt-in)). Other bits. | | Sweeps per solve | `learning_fast_settle=True`, a new brain's | Another solve of the same equations, in which rows stop apart and the activation vectorizes: about half of a chamber pass on the Xeon above, and .74 of one on an Apple M4 (229 to 169 CPU seconds, four pairs at the default brain on one thread of a shared machine), whose exponential costs less. `learning_fast_settle=False` names the ordinary solve, with other bits. | | Sweeps per solve | a larger `learning_tolerance` | At 0.01 in place of 0.003, about a fifth fewer sweeps per act and an eighth of a chamber pass. | | Slots | a smaller `recall_capacity` | A read grows with the slots a stream holds; a write does not. | @@ -398,7 +449,7 @@ full = Brain.compose( recall_heads=2, # two facts in one settle recall_boundary=True, # the start of an episode has a slot recall_action=True, # the slots keep what it did - recall_copy=2.0, recall_tokens=range(4), # observations 0-3 name the 4 actions + recall_gain=1.0, recall_tokens=range(4), # observations 0-3 name the 4 actions observers=(16,), # optional System 2 ) assert full.live(np.eye(16)[[3]]).shape == (1,) @@ -412,10 +463,10 @@ assert full.live(np.eye(16)[[3]]).shape == (1,) ([the episode's boundary](memory.md#episodes-keys-and-windows)). - `recall_action=True`: every slot keeps the command executed at its moment ([its own actions](memory.md#its-own-actions-in-the-slots-opt-in)). -- `recall_copy=2.0, recall_tokens=range(4)`: a token the read delivers drives the action of the - same token, including tokens never taught as answers - ([the copy pathway](memory.md#the-copy-pathway-opt-in)). Declare only identities the task - has; without shared tokens leave it out. +- `recall_gain=1.0, recall_tokens=range(4)`: a learned gain per read port carries a token the read + delivers to the action of the same token, including tokens never taught as answers + ([the learned value path](memory.md#the-learned-value-path-opt-in)). Declare only identities the + task has; without shared tokens leave it out. - `observers=(16,)`: a System 2 region inside the same settle, bounded by the compartment. The recall memory reads into the top region; @@ -440,7 +491,10 @@ For modules of several hundred neurons and more `w`, with no setting: above 64 the projection from prefrontal to association and the association-motor pair step at `64 / w` (`Learner.synapse_rate`), and `recall_address` is .01 times `64 / w`. At 64 and under every step is whole. Every other projection keeps its step: the sensory one, a chain between modules, an observer's and the memory's reads. - The rate multiplies every step on those synapses, a lesson's and a reward's. + The rate multiplies every step on those synapses, a lesson's and a reward's. A rate of zero holds a synapse and its + optimizer history, and a neuron whose every inbound synapse steps at zero holds its bias too: zero on every synapse + into a region, both directions of each pair, freezes the region. A reciprocal pair shares one efficacy and steps by + the mean of its two directions' steps, so one direction alone at zero takes half its partner's step. A rate you name wins over its part: diff --git a/pyproject.toml b/pyproject.toml index a96d7d89..5e1d2f73 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "hatchling.build" [project] name = "cadence-net" -version = "0.95.2" +version = "0.96.0" description = "Research toward general intelligence through local repair, equilibrium detuning and self-reflection." readme = "README.md" requires-python = ">=3.11" @@ -69,7 +69,10 @@ testpaths = [ "benchmarks/sequence", "benchmarks/extinction", "benchmarks/grammar", + "benchmarks/heads", + "benchmarks/vocabulary", "benchmarks/long_context", + "benchmarks/device", ] [tool.ruff] diff --git a/src/cadence/__init__.py b/src/cadence/__init__.py index 021f8e00..316d12cc 100644 --- a/src/cadence/__init__.py +++ b/src/cadence/__init__.py @@ -38,6 +38,7 @@ preflight, seam_report, ) +from .lives import Lives from .memory import SynapticMemory from .neuron import Adaptation, NeuronModel from .plasticity import ActorCritic, ActorCriticConfig, Maturity @@ -70,6 +71,7 @@ "Adaptation", "Region", "Brain", + "Lives", "regions", "Projection", "Genome", @@ -111,4 +113,4 @@ "save", ] -__version__ = "0.95.2" +__version__ = "0.96.0" diff --git a/src/cadence/_moment.py b/src/cadence/_moment.py new file mode 100644 index 00000000..87f0d5c0 --- /dev/null +++ b/src/cadence/_moment.py @@ -0,0 +1,2412 @@ +"""The compiled moment (``learning_compiled_moment``, opt-in): the work around the solve kernels +of a moment on the CPU in compiled passes. + +A moment's settle runs in the kernels of ``cadence.fused``. Around it the recall memory reads its +slots, the brain chooses, the memory writes the moment and a lesson steps the read's maps. On the +CPU each of these is many small NumPy operations at a few rows and slots, and at that size the cost +of a call is most of the moment. Here each is one compiled pass of the same equations: + +- the read (``recall_read``): the address and every position term of every row through one product + of the inputs with the maps packed side by side (``packed_maps``, made again only where a lesson + replaces a map), then each slot's logit, the attention and the reads of the values and nexts; +- the address lesson (``address_lesson``): the scores of the slots against the lesson's movement, + the prior, the attention the lesson names and one least-mean-squares step of every packed column; +- the place store's read (``store_read``) and the carried gains' contrast (``route_contrast``); +- the write (``recall_write``) into the room behind the slots, with the ordinary write's bits; +- a greedy act and a lesson's judgment each as one call of these passes around the solve + (``one_call``), and a taught lesson as one more, from its phases to its written efficacies + (``taught_lesson``), each with the bits of its passes in turn. + +The sums of the read, the lesson and the contrasts run in another order than NumPy's and BLAS's, so +those bits differ from the ordinary path, which stays the default. The kernels allocate their +outputs; a caller checks what it keeps, as the ordinary path does. +""" + +from __future__ import annotations + +from typing import Any + +import numpy as np + +from . import brain as _graphs +from . import fused +from .brain import BrainState, _keep_products +from .fused import _compiled, _inlined, _vectorized, njit + +if njit is not None: + from .fused import _index + +__all__ = [ + "address_lesson", "available", "recall_read", "recall_write", "route_contrast", "store_read", +] + + +# The compiled moment's shortcuts that keep its bits: a read of what the latest read read is its +# drive, and an act and a lesson call their passes without the general path's layers. False runs +# the compiled passes through the general path (the tests compare the two to the bit). +SHORTCUTS = True +# An act, and a lesson's judgment, each as one compiled call (``one_call``), with the bits of the +# shortcuts above. False runs them as the passes they are made of (the tests compare the two). +ONE_CALL = True + + +def available() -> bool: + """Whether the compiled moment can run here: Numba is installed and the compiled passes of + ``cadence.fused`` are in use (``fused.compiled``, read at call time).""" + return njit is not None and bool(_graphs._FUSED) + + +if njit is not None: + + @_inlined + def _term( + z: np.ndarray, b: int, age: int, columns: np.ndarray, numbers: np.ndarray, + slope_unit: float, told: int, focus: int, table: np.ndarray, grid: np.ndarray, + ) -> float: + """A slot's position term from row ``b`` of the packed products ``z`` + (``Recall.position_term``): the table over its age bucket, the anchored term over its + place in the episode, the relative term over its offset from the previous read's focus, + the slope along its age and past the table the grid's residues.""" + if numbers[0] == 0: + return 0.0 + last = table.shape[0] - 1 + row = _index(b) + term = z[row, _index(columns[1] + table[min(age, last)])] + if numbers[1] != 0: + place = told - 1 - age + bucket = numbers[7] if place < 0 else table[min(place, last)] + term += z[row, _index(columns[2] + bucket)] + if numbers[2] != 0: + reach = numbers[6] + if focus < 0: + bucket = 2 * reach + 1 + else: + bucket = max(min(age - focus, reach), -reach) + reach + term += z[row, _index(columns[3] + bucket)] + if numbers[3] != 0: + term += z[row, _index(columns[4])] * (age / slope_unit) + if numbers[4] != 0 and age >= numbers[5]: + total = 0.0 + start = 0 + for i in range(grid.shape[0]): + total += z[row, _index(columns[5] + start + age % grid[i])] + start += grid[i] + term += total + return float(term) + + @_vectorized + def _products( + x: np.ndarray, c: np.ndarray, w_x: np.ndarray, w_c: np.ndarray, w: np.ndarray + ) -> np.ndarray: + """``x w_x + c w_c + w``: every row's inputs through every packed column.""" + z: np.ndarray = np.dot(x, w_x) + np.dot(c, w_c) + for b in range(z.shape[0]): + for j in range(z.shape[1]): + z[b, j] += w[j] + return z + + @_vectorized + def _packed_read( + x: np.ndarray, c: np.ndarray, w_x: np.ndarray, w_c: np.ndarray, w: np.ndarray, + columns: np.ndarray, numbers: np.ndarray, scalars: np.ndarray, keys: np.ndarray, + values: np.ndarray, nexts: np.ndarray, has_next: bool, held: np.ndarray, first: int, + slots: int, told: np.ndarray, focus: np.ndarray, table: np.ndarray, grid: np.ndarray, + ) -> tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray, np.ndarray]: + """Every row's key, address, attention over its slots and reads of the values and the + nexts (``Recall.stimulate`` with one head), the maps packed (``packed_maps``): one + product of the rows' inputs with every column, then each slot's logit. The slots are + ``slots`` entries of the arrays from ``first`` on (the room behind a memory's slots). + A slot's dot product (four running sums, which one sum cannot overlap) and its position + term (``_term``'s arithmetic) are written in place: a helper's arrays cost their + reference counts per slot.""" + rows, width = x.shape + address_width = columns[1] + value_width = values.shape[2] + heard, temperature, recency, slope_unit = scalars[0], scalars[1], scalars[2], scalars[3] + has_position, has_anchor = numbers[0] != 0, numbers[1] != 0 + has_relative, has_slope, has_grid = numbers[2] != 0, numbers[3] != 0, numbers[4] != 0 + position, reach, outside = numbers[5], numbers[6], numbers[7] + c2, c3, c4, c5 = columns[2], columns[3], columns[4], columns[5] + last = table.shape[0] - 1 + moduli = grid.shape[0] + z = _products(x, c, w_x, w_c, w) + key = np.empty((rows, width)) + q = np.empty((rows, address_width)) + attention = np.empty((rows, slots)) + read = np.zeros((rows, value_width)) + following = np.zeros((rows, value_width if has_next else 0)) + # per slot, the same in every row: its table column and its age over the slope's unit + column_of = np.zeros(slots, dtype=np.int64) + lean_of = np.empty(slots) + for m in range(slots): + age = slots - 1 - m + if has_position: + column_of[m] = columns[1] + table[min(age, last)] + lean_of[m] = age / slope_unit + quads = address_width - address_width % 4 + for b in range(rows): + for k in range(width): + key[b, k] = heard * x[b, k] + for j in range(address_width): + q[b, j] = key[b, j] + z[b, j] + episode, focused = told[b], focus[b] + top = -np.inf + for m in range(slots): + at = first + m + if not held[b, at]: + attention[b, m] = -np.inf + continue + age = slots - 1 - m + s0 = 0.0 + s1 = 0.0 + s2 = 0.0 + s3 = 0.0 + j = 0 + while j < quads: + s0 += q[b, j] * keys[b, at, j] + s1 += q[b, j + 1] * keys[b, at, j + 1] + s2 += q[b, j + 2] * keys[b, at, j + 2] + s3 += q[b, j + 3] * keys[b, at, j + 3] + j += 4 + while j < address_width: + s0 += q[b, j] * keys[b, at, j] + j += 1 + logit = ((s0 + s1) + (s2 + s3)) / temperature + if recency != 0.0: + logit -= recency * age + term = 0.0 + if has_position: + term = z[b, column_of[m]] + if has_anchor: + place = episode - 1 - age + if place < 0: + term += z[b, c2 + outside] + else: + term += z[b, c2 + table[min(place, last)]] + if has_relative: + if focused < 0: + term += z[b, c3 + 2 * reach + 1] + else: + term += z[b, c3 + max(min(age - focused, reach), -reach) + reach] + if has_slope: + term += z[b, c4] * lean_of[m] + if has_grid and age >= position: + total = 0.0 + start = 0 + for i in range(moduli): + total += z[b, c5 + start + age % grid[i]] + start += grid[i] + term += total + logit += term + attention[b, m] = logit + top = max(top, logit) + if top == -np.inf: # a stream that holds no slot reads silence + for m in range(slots): + attention[b, m] = 0.0 + continue + total = 0.0 + for m in range(slots): + p = np.exp(attention[b, m] - top) + attention[b, m] = p + total += p + for m in range(slots): + p = attention[b, m] / total + attention[b, m] = p + if p != 0.0: + row, at = _index(b), _index(first + m) + for j in range(_index(value_width)): + read[row, j] += p * values[row, at, j] + if has_next: + for j in range(_index(value_width)): + following[row, j] += p * nexts[row, at, j] + return key, q, attention, read, following + + @_vectorized + def _readback( + movement: np.ndarray, weights: np.ndarray, synapses: np.ndarray, sources: np.ndarray, + targets: np.ndarray, width: int, + ) -> np.ndarray: + """``movement @ W.T`` for a port's synapses into the association (``Recall.readback``): + per source neuron, its synapses' weights times their association neuron's movement.""" + rows = movement.shape[0] + out = np.zeros((rows, width)) + for b in range(rows): + for e in range(synapses.shape[0]): + out[b, sources[e]] += weights[synapses[e]] * movement[b, targets[e]] + return out + + @_vectorized + def _finite(a: np.ndarray) -> bool: + for v in a.ravel(): + if not (v - v == 0.0): + return False + return True + + @_vectorized + def _lesson_kernel( + rows: np.ndarray, movement: np.ndarray, weights: np.ndarray, + value_synapses: np.ndarray, value_sources: np.ndarray, value_targets: np.ndarray, + next_synapses: np.ndarray, next_sources: np.ndarray, next_targets: np.ndarray, + has_next: bool, values: np.ndarray, nexts: np.ndarray, keys: np.ndarray, + held: np.ndarray, masked: bool, inputs: np.ndarray, cortex: np.ndarray, + address: np.ndarray, key: np.ndarray, w_x: np.ndarray, w_c: np.ndarray, w: np.ndarray, + columns: np.ndarray, numbers: np.ndarray, scalars: np.ndarray, told: np.ndarray, + focus: np.ndarray, table: np.ndarray, grid: np.ndarray, positions: bool, + ) -> tuple[int, float, np.ndarray, np.ndarray, np.ndarray, bool, bool]: + """The address lesson of head 0 (``Recall.learn``), the maps packed (``packed_maps``): + the scores of each moving row's slots against the lesson's movement, the prior its + address gave, the attention the lesson names, and one least-mean-squares step of every + packed column: the address columns at the address rate, the table, anchored, relative + and grid columns at the position rate and the slope column at its own. Returns the rows + that moved, the address error, the new packed maps and whether the address step and the + position step are finite (the candidate addresses with the address step).""" + slots = keys.shape[1] + key_width = keys.shape[2] + value_width = values.shape[2] + temperature, recency, slope_unit = scalars[1], scalars[2], scalars[3] + address_temperature, address_rate = scalars[4], scalars[5] + position_rate, slope_rate = scalars[6], scalars[7] + d_value = _readback(movement, weights, value_synapses, value_sources, value_targets, + value_width) + d_next = ( + _readback(movement, weights, next_synapses, next_sources, next_targets, value_width) + if has_next else d_value + ) + count = rows.shape[0] + moving = np.zeros(count, dtype=np.bool_) + norms = np.zeros(count) + moved = 0 + for i in range(count): + total = 0.0 + for j in range(value_width): + total += d_value[i, j] * d_value[i, j] + if has_next: + for j in range(value_width): + total += d_next[i, j] * d_next[i, j] + norms[i] = np.sqrt(total) + go = norms[i] > 1e-12 + if masked and go: + go = False + for m in range(slots): + go = go or held[rows[i], m] + moving[i] = go + moved += go + width = columns[6] + if moved == 0: + return 0, 0.0, w_x, w_c, w, False, False + streams = np.empty(moved, dtype=np.int64) + x = np.empty((moved, inputs.shape[1])) + c = np.empty((moved, cortex.shape[1])) + k = 0 + for i in range(count): + if moving[i]: + streams[k] = i + x[k] = inputs[rows[i]] + c[k] = cortex[rows[i]] + k += 1 + z = _products(x, c, w_x, w_c, w) + step = np.zeros((moved, width)) # every packed column's step, rate aside + prior = np.empty(slots) + logits = np.empty(slots) + absolute = 0.0 + stepped_slope = numbers[3] and not (numbers[4] and slots > numbers[5]) + reach = numbers[6] + last = table.shape[0] - 1 + for k in range(moved): + i = streams[k] + b = rows[i] + top = -np.inf + for m in range(slots): + if masked and not held[b, m]: + prior[m] = -np.inf + logits[m] = -np.inf + continue + score = 0.0 + for j in range(value_width): + score += d_value[i, j] * values[b, m, j] + if has_next: + for j in range(value_width): + score += d_next[i, j] * nexts[b, m, j] + logits[m] = score / norms[i] / address_temperature + dot = 0.0 + for j in range(key_width): + dot += address[b, j] * keys[b, m, j] + logit = dot / temperature + if recency != 0.0: + logit -= recency * (slots - 1 - m) + logit += _term(z, k, slots - 1 - m, columns, numbers, slope_unit, told[b], + focus[b], table, grid) + prior[m] = logit + top = max(top, logit) + total = 0.0 + for m in range(slots): + prior[m] -= top + total += np.exp(prior[m]) + shift = np.log(total) + best = -np.inf + for m in range(slots): + prior[m] -= shift + logits[m] += prior[m] + best = max(best, logits[m]) + total = 0.0 + for m in range(slots): + logits[m] = np.exp(logits[m] - best) + total += logits[m] + energy = 1.0 + for j in range(x.shape[1]): + energy += x[k, j] * x[k, j] + for j in range(c.shape[1]): + energy += c[k, j] * c[k, j] + bound = min(1.0, 1.0 / (address_rate * energy)) + near = min(1.0, 1.0 / (position_rate * energy)) + peak = 0.0 + for m in range(slots): + logits[m] /= total # the attention the lesson names + peak = max(peak, logits[m], np.exp(prior[m]) if prior[m] > -np.inf else 0.0) + lean = 0.0 + for m in range(slots): + attended = np.exp(prior[m]) if prior[m] > -np.inf else 0.0 + gradient = logits[m] - attended + if gradient == 0.0: + continue + for j in range(key_width): + step[k, j] += gradient * keys[b, m, j] # the error, bounded below + if not numbers[0]: + continue + age = slots - 1 - m + step[k, columns[1] + table[min(age, last)]] += gradient * near + if numbers[1]: + place = told[b] - 1 - age + bucket = numbers[7] if place < 0 else table[min(place, last)] + step[k, columns[2] + bucket] += gradient * near + if numbers[2]: + if focus[b] < 0: + bucket = 2 * reach + 1 + else: + bucket = max(min(age - focus[b], reach), -reach) + reach + step[k, columns[3] + bucket] += gradient * near + lean += gradient * (age / slope_unit) + if numbers[4] and age >= numbers[5]: + far = gradient / peak if peak > 0 else gradient + start = 0 + for i2 in range(grid.shape[0]): + step[k, columns[5] + start + age % grid[i2]] += far * near + start += grid[i2] + for j in range(key_width): + absolute += abs(step[k, j]) + step[k, j] *= bound + if stepped_slope: + step[k, columns[4]] = lean * min(1.0, 1.0 / (slope_rate * energy)) + mean_error = absolute / (moved * key_width) + # one least-mean-squares step of every column, each at its part's rate, averaged + s_x = np.dot(x.T.copy(), step) + s_c = np.dot(c.T.copy(), step) + rates = np.empty(width) + for j in range(width): + rates[j] = position_rate if positions else 0.0 + for j in range(columns[1]): + rates[j] = address_rate + if numbers[3]: + rates[columns[4]] = slope_rate if (positions and stepped_slope) else 0.0 + new_x = np.empty_like(w_x) + new_c = np.empty_like(w_c) + new = np.empty_like(w) + for j in range(width): + mean = 0.0 + for k in range(moved): + mean += step[k, j] + new[j] = w[j] + rates[j] * (mean / moved) + for a in range(w_x.shape[0]): + for j in range(width): + new_x[a, j] = w_x[a, j] + rates[j] * s_x[a, j] / moved + for a in range(w_c.shape[0]): + for j in range(width): + new_c[a, j] = w_c[a, j] + rates[j] * s_c[a, j] / moved + address_ok = ( + _finite(new_x[:, : columns[1]]) and _finite(new_c[:, : columns[1]]) + and _finite(new[: columns[1]]) + ) + if address_ok: # the candidate addresses of the moving rows + candidate = _products(x, c, new_x[:, : columns[1]].copy(), + new_c[:, : columns[1]].copy(), new[: columns[1]].copy()) + for k in range(moved): + b = rows[streams[k]] + for j in range(columns[1]): + v = key[b, j] + candidate[k, j] + if not (v - v == 0.0): + address_ok = False + position_ok = ( + _finite(new_x[:, columns[1]:]) and _finite(new_c[:, columns[1]:]) + and _finite(new[columns[1]:]) + ) + return moved, mean_error, new_x, new_c, new, address_ok, position_ok + + +_NONE2 = np.zeros((1, 1)) + + +def _settings(recall: Any) -> tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]: + """The read's settings as the kernels take them: which position terms it has and their + sizes, its scalars, the age buckets' table and the grid's moduli; made again where one + changes.""" + key = ( + bool(recall.position_maps), bool(recall.anchor_maps), bool(recall.relative_maps), + bool(recall.slope_maps), bool(recall.grid_maps), recall.position, recall.capacity, + recall.heard, recall.temperature, recall.recency, recall.address_temperature, + recall.address, recall.position_rate, recall.slope, tuple(recall.grid or ()), + ) + held = recall.__dict__.get("_kernel_settings") + if held is not None and held[0] == key: + return held[1] # type: ignore[no-any-return] + numbers = np.array([ + *key[:5], int(recall.position), int(recall.RELATIVE), int(recall.position_buckets), + ], dtype=np.int64) + scalars = np.array([ + float(recall.heard), float(recall.temperature), float(recall.recency), float(recall.SLOPE), + float(recall.address_temperature), float(recall.address), float(recall.position_rate), + float(recall.slope), + ]) + table = recall._age_buckets() if recall.position_maps else _NO_TABLE + grid = np.asarray(recall.grid if recall.grid_maps else (1,), dtype=np.int64) + settings = (numbers, scalars, np.ascontiguousarray(table, dtype=np.int64), grid) + within = numbers.copy() + within[4] = 0 # the numbers of a read whose slots do not reach the grid's columns + recall.__dict__["_kernel_settings"] = (key, settings, within) + return settings + + +def _behind(recall: Any, name: str) -> tuple[np.ndarray, int]: + """The slot array ``name`` as the room it is a view of and its first slot there + (``Recall._longer``), or the array itself, C-ordered, from slot 0.""" + array = getattr(recall, name) + held = recall._rooms.get(name) + if held is not None and held[1] is array: + return held[0], int(held[2]) + return np.ascontiguousarray(array), 0 + + +def recall_read( + recall: Any, x: np.ndarray, c: np.ndarray +) -> tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray, np.ndarray]: + """``(key, address, attention, read of the values, read of the nexts)`` of head 0 at the + moments ``x`` with the cortex ``c`` of the moment before (``_packed_read``).""" + w_x, w_c, w, columns = packed_maps(recall) + numbers, scalars, table, grid = _settings(recall) + slots = recall.slots + if numbers[4] and slots <= recall.position: # no slot reaches the grid's columns + w_x, w_c, w = recall.__dict__["_packed"][3] + numbers = numbers.copy() + numbers[4] = 0 + keys, first = _behind(recall, "keys") + values, at_values = _behind(recall, "values") + nexts, at_nexts = _behind(recall, "nexts") if recall.next else (values, at_values) + held, at_held = _behind(recall, "held") + if not first == at_values == at_nexts == at_held: # rooms of another history: every array + keys, values, held = (np.ascontiguousarray(a) for a in (recall.keys, recall.values, + recall.held)) + nexts = np.ascontiguousarray(recall.nexts) if recall.next else values + first = 0 + return _packed_read( + np.ascontiguousarray(x), np.ascontiguousarray(c), w_x, w_c, w, columns, numbers, + scalars, keys, values, nexts, bool(recall.next), held, first, slots, recall.told, + recall.focus, table, grid, + ) + + +_NO_TABLE = np.zeros(1, dtype=np.int64) +# the maps of a read in their order in the packed columns: the address map, then the position term's +PACKED = ("position_maps", "anchor_maps", "relative_maps", "slope_maps", "grid_maps") + + +def packed_parts(recall: Any) -> tuple[Any, ...]: + """Head 0's address map and position maps, a tuple each (None for a map the memory does not + hold), in their order in the packed columns.""" + position, anchor, relative = recall.position_maps, recall.anchor_maps, recall.relative_maps + slope, grid = recall.slope_maps, recall.grid_maps + return ( + recall.head_maps[0], position[0] if position else None, anchor[0] if anchor else None, + relative[0] if relative else None, slope[0] if slope else None, + grid[0] if grid else None, + ) + + +def packed_key(recall: Any) -> tuple[int, ...]: + """The identity of head 0's maps: a lesson replaces a map's whole tuple.""" + return tuple(id(part) for part in packed_parts(recall)) + + +def packed_maps(recall: Any) -> tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]: + """Head 0's address map and position maps as one map ``(P_x, P_c, p)`` of every column in + turn (the address, the table, the anchored, the relative, the slope and the grid columns) and + where each part starts (``columns``, with the end last), made again only where a map is + replaced: a lesson replaces the maps it steps and never writes into them.""" + parts = packed_parts(recall) + held = recall.__dict__.get("_packed") + if held is not None: + made = held[2] # the maps the packed ones were made from, held, so compared by identity + if ( + parts[0] is made[0] and parts[1] is made[1] and parts[2] is made[2] + and parts[3] is made[3] and parts[4] is made[4] and parts[5] is made[5] + ): + return held[1] # type: ignore[no-any-return] + identity = None + blocks_x, blocks_c, blocks, columns = [], [], [], [0] + for part in parts: + if part is None: + columns.append(columns[-1]) + continue + m_x, m_c, m = part + m_x, m_c = np.asarray(m_x, dtype=float), np.asarray(m_c, dtype=float) + if m_x.ndim == 1: # the slope maps: one column + m_x, m_c = m_x[:, None], m_c[:, None] + blocks_x.append(m_x) + blocks_c.append(m_c) + blocks.append(np.atleast_1d(np.asarray(m, dtype=float))) + columns.append(columns[-1] + m_x.shape[1]) + packed = ( + np.ascontiguousarray(np.concatenate(blocks_x, axis=1)), + np.ascontiguousarray(np.concatenate(blocks_c, axis=1)), + np.concatenate(blocks), np.asarray(columns, dtype=np.int64), + ) + end = columns[-2] # without the grid's columns, which only slots past the table read + within = ( + np.ascontiguousarray(packed[0][:, :end]), np.ascontiguousarray(packed[1][:, :end]), + packed[2][:end].copy(), + ) + # the arrays are held with their identities, so an identity is never a freed array's + recall.__dict__["_packed"] = (identity, packed, parts, within) + return packed + + +# -- the address lesson + + +def address_lesson( + recall: Any, graph: Any, rows: np.ndarray, movement: np.ndarray, positions: bool +) -> tuple[Any, ...]: + """The kernel's address lesson of head 0 (``_lesson_kernel``) for ``recall`` at the streams + ``rows`` with the association's ``movement``, through the synapses of ``graph``.""" + (v_syn, v_src, v_dst), (n_syn, n_src, n_dst) = recall._head_synapses[0] + moment = recall.last + # the slots as arrays of one kind, whether or not they are views of the room behind them: + # the kernel compiles once + held = np.ascontiguousarray(recall.held) + values = np.ascontiguousarray(recall.values) + w_x, w_c, w, columns = packed_maps(recall) + numbers, scalars, table, grid = _settings(recall) + return _lesson_kernel( + np.asarray(rows, dtype=np.int64), np.ascontiguousarray(movement, dtype=float), + np.ascontiguousarray(graph._weights, dtype=float), v_syn, v_src, v_dst, n_syn, n_src, + n_dst, bool(recall.next), values, + np.ascontiguousarray(recall.nexts) if recall.next else values, + np.ascontiguousarray(recall.keys), held, + not bool(held.all()), moment.inputs, moment.cortex, moment.address, moment.key, + w_x, w_c, w, columns, numbers, scalars, recall.told, recall.focus, table, grid, + bool(positions), + ) + + +# -- the place store's read + +if njit is not None: + + @_vectorized + def _store_kernel( + cue: np.ndarray, strength: np.ndarray, mass: np.ndarray, shared: bool, normalize: bool, + amplitude: float, + ) -> np.ndarray: + """``FastSynapses.recall`` of the delta rule: each row's cue scaled by its largest entry + and set to unit length, read through the row's strengths (``shared``: one baseline for + every row, with no writes), over the decayed count of writes with ``normalize``, at + ``amplitude``.""" + rows, width = cue.shape + post = strength.shape[2] + out = np.zeros((rows, post)) + unit = np.empty(width) + for b in range(rows): + held = 0 if shared else b + scale = 0.0 + for k in range(width): + scale = max(scale, abs(cue[b, k])) + if scale <= 0.0: + scale = 1.0 + norm = 0.0 + for k in range(width): + unit[k] = cue[b, k] / scale + norm += unit[k] * unit[k] + norm = np.sqrt(norm) + if norm <= 0.0: + norm = 1.0 + for k in range(width): + v = unit[k] / norm + if v != 0.0: + for j in range(post): + out[b, j] += v * strength[held, k, j] + divisor = max(0.0 if shared else mass[b], 1e-12) if normalize else 1.0 + for j in range(post): + out[b, j] = amplitude * (out[b, j] / divisor) + return out + + +def store_read(store: Any, cue: np.ndarray) -> np.ndarray: + """``FastSynapses.recall`` of a delta-rule store without a separator: at its own batch the + streams' strengths, at another the baseline every new stream reads (``_baseline``).""" + shared = len(store.strength) != len(cue) + strength = store._baseline()[None, :, :] if shared else store.strength + return _store_kernel( + cue, np.ascontiguousarray(strength, dtype=float), store.mass, shared, + bool(store.normalize), float(store.amplitude), + ) + + +# -- the carried routes' gains + +if njit is not None: + + @_vectorized + def _route_kernel( + movement: np.ndarray, weights: np.ndarray, synapses: np.ndarray, sources: np.ndarray, + targets: np.ndarray, held: np.ndarray, scale: float, amplitude: float, slope: float, + threshold: float, rest: float, leak: float, + ) -> np.ndarray: + """One route's ``c`` per row (``Hearing._contrast``): the movement read back through the + route's synapses, weighed by the slope of the activation at the held potential + ``scale * held`` times ``amplitude * held``.""" + rows, width = held.shape + out = np.zeros(rows) + back = np.empty(width) + for b in range(rows): + for j in range(width): + back[j] = 0.0 + for e in range(synapses.shape[0]): + back[sources[e]] += weights[synapses[e]] * movement[b, targets[e]] + total = 0.0 + for j in range(width): + v = scale * held[b, j] + sig = 1.0 / (1.0 + np.exp(-slope * (v - threshold))) + d = slope * sig * (1.0 - sig) + d = d / (1.0 - rest) if sig > rest else leak * d / rest + total += d * amplitude * held[b, j] * back[j] + out[b] = total + return out + + +def route_contrast( + graph: Any, route: dict[str, Any], movement: np.ndarray, held: np.ndarray, scale: float, + amplitude: float, model: Any, +) -> np.ndarray: + """``_route_kernel`` for one route of ``Hearing`` through the synapses of ``graph``.""" + return _route_kernel( + np.ascontiguousarray(movement, dtype=float), + np.ascontiguousarray(graph._weights, dtype=float), route["synapses"], route["rows"], + route["cols"], np.ascontiguousarray(held, dtype=float), float(scale), float(amplitude), + float(model.slope), float(model.threshold), float(model.rest_emission), + float(model.leak), + ) + + +# -- the recall memory's write + +if njit is not None: + + @_compiled + def _write_kernel( + key: np.ndarray, value: np.ndarray, held: np.ndarray, ended: np.ndarray, + attended: np.ndarray, has_attended: bool, focus: np.ndarray, + keys_room: np.ndarray, values_room: np.ndarray, nexts_room: np.ndarray, + held_room: np.ndarray, first: int, drop: int, stop: int, + ) -> np.ndarray: + """One moment per stream written into the rooms behind the slots (``Recall._observe``): + each stream's latest held slot that closed no episode learns ``value`` as its next where + it survives the drop, the new slot at column ``stop`` holds ``key`` and ``value`` and no + next, and the focus moves to the slot this moment's read attended most. Returns the + new focus.""" + rows, slots = held.shape + out = np.empty(rows, dtype=np.int64) + for b in range(rows): + for j in range(nexts_room.shape[2]): + nexts_room[b, stop, j] = 0.0 + if not ended[b]: + latest = -1 + for m in range(slots): + if held[b, m]: + latest = m + if latest >= drop: + for j in range(value.shape[1]): + nexts_room[b, first + latest, j] = value[b, j] + for j in range(key.shape[1]): + keys_room[b, stop, j] = key[b, j] + for j in range(value.shape[1]): + values_room[b, stop, j] = value[b, j] + held_room[b, stop] = True + if has_attended: + best = attended[b, 0] + at = 0 + for m in range(1, slots): + if attended[b, m] > best: + best = attended[b, m] + at = m + out[b] = slots - 1 - at + 1 if best > 0 else -1 + else: + out[b] = focus[b] + 1 if focus[b] >= 0 else -1 + return out + + +def recall_write(recall: Any, key: np.ndarray, value: np.ndarray) -> bool: + """``Recall._observe`` at the bank's own batch where every slot array has room behind it: + the write in one compiled pass, the same bits. False where it does not apply (the ordinary + write then runs).""" + if len(key) != recall.streams: + return False + rooms = recall._rooms + names = ("keys", "values", "nexts", "held") + held_rooms = [rooms.get(name) for name in names] + for name, room in zip(names, held_rooms, strict=True): + old = getattr(recall, name) + if room is None or room[1] is not old or old.base is not room[0] or ( + room[3] >= room[0].shape[1] + ): + return False + if len({(room[2], room[3]) for room in held_rooms if room is not None}) != 1: + return False + kept = recall.keys.shape[1] + drop = max(kept + 1 - recall.capacity, 0) + first, stop = held_rooms[0][2], held_rooms[0][3] + attended = recall._attended + has_attended = ( + attended is not None and attended.shape == (len(key), kept) and kept > 0 + ) + focus = _write_kernel( + key, value, recall.held, recall.ended, attended if has_attended else _NONE2, + has_attended, recall.focus, held_rooms[0][0], held_rooms[1][0], + held_rooms[2][0], held_rooms[3][0], first, drop, stop, + ) + if recall.ended.any(): + recall.ended = np.zeros(len(key), dtype=bool) + new_first = first + drop + for name, room in zip(names, held_rooms, strict=True): + assert room is not None + view = room[0][:, new_first : stop + 1] + rooms[name] = (room[0], view, new_first, stop + 1) + setattr(recall, name, view) + recall._contents = [] + recall._content_keys = None + recall.told = recall.told + 1 + recall.focus = focus + return True + + +# -- the moment's drive in one pass + +if njit is not None: + + @_vectorized + def _drive_kernel( + x: np.ndarray, c: np.ndarray, w_x: np.ndarray, w_c: np.ndarray, w: np.ndarray, + columns: np.ndarray, numbers: np.ndarray, scalars: np.ndarray, keys: np.ndarray, + values: np.ndarray, nexts: np.ndarray, has_next: bool, held: np.ndarray, first: int, + slots: int, told: np.ndarray, focus: np.ndarray, table: np.ndarray, grid: np.ndarray, + neurons: int, spans: np.ndarray, gains: np.ndarray, flags: np.ndarray, + trace: np.ndarray, strength: np.ndarray, mass: np.ndarray, + ) -> tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray, np.ndarray]: + """The drive of a moment in the order ``Memories.stimulus`` reads the memories: the + sensory part, the working trace, the recall read (``_packed_read``) at its gain and the + place store's read of the sensory part (``_store_kernel``). ``spans`` are the column + ranges (sensory, trace, recall, next, ports, the store's key and value), ``gains`` the + scales (the stimulus amplitude, the trace's, the read's, the ports' and the store's + amplitude) and ``flags`` which parts there are (a trace of this batch, the read's drive + gain, a store, a store read at its baseline, a normalized store). Returns the drive, + the read's key, address and attention, and what the ports held at the read's gain.""" + rows = x.shape[0] + drive = np.zeros((rows, neurons)) + for b in range(rows): + for k in range(spans[1] - spans[0]): + drive[b, spans[0] + k] = x[b, k] * gains[0] + if flags[0]: + for b in range(rows): + for k in range(spans[3] - spans[2]): + drive[b, spans[2] + k] = gains[1] * trace[b, k] + ports = np.zeros((rows, spans[7] - spans[6]) if flags[1] else (rows, 0)) + if flags[5]: + key, q, attention, read, following = _packed_read( + x, c, w_x, w_c, w, columns, numbers, scalars, keys, values, nexts, has_next, + held, first, slots, told, focus, table, grid, + ) + value_width = read.shape[1] + for b in range(rows): + for j in range(value_width): + drive[b, spans[4] + j] = gains[2] * read[b, j] + if has_next: + for j in range(value_width): + drive[b, spans[5] + j] = gains[2] * following[b, j] + if flags[1]: + for b in range(rows): + for j in range(spans[7] - spans[6]): + ports[b, j] = drive[b, spans[6] + j] + drive[b, spans[6] + j] = gains[3] * ports[b, j] + else: # streams that begin: no slot to read, the ports held silent + key = np.empty((0, 0)) + q = np.empty((0, 0)) + attention = np.empty((0, 0)) + if flags[2]: + cue = np.empty((rows, spans[9] - spans[8])) + for b in range(rows): + for k in range(spans[9] - spans[8]): + cue[b, k] = drive[b, spans[8] + k] + stored = _store_kernel(cue, strength, mass, flags[3] != 0, flags[4] != 0, gains[4]) + for b in range(rows): + for j in range(spans[11] - spans[10]): + drive[b, spans[10] + j] += stored[b, j] + return drive, key, q, attention, ports + + +# -- the free solve of a moment + +_RULES: dict[int, tuple[Any, tuple[float, float, float, float]]] = {} + + +def _rule_of(model: Any) -> tuple[float, float, float, float]: + """The neuron rule as the solve kernels read it (``fused._rule``), made once per model.""" + held = _RULES.get(id(model)) + if held is None or held[0] is not model: + held = (model, (model.slope, model.threshold, model.rest_emission, model.leak)) + if len(_RULES) > 64: + _RULES.clear() + _RULES[id(model)] = held + return held[1] + + +def free_solve(graph: Any, drive: np.ndarray, state: Any, budget: int, tolerance: float) -> Any: + """``graph.equilibrate(drive, state=state, budget=budget, tolerance=tolerance, fast=True)`` + of a host state on the compiled path, in the kernels that path calls (the activations of the + warm potentials, ``fused._fast_solve``): the same state, report and bits; None where the + solve takes another path (a compartment, adaptation, another step, a wrapped settle).""" + from .brain import Equilibrium + + model = graph.neuron_model + if ( + model.adaptation is not None or model.dt != 1.0 or graph.backend != "cpu" + or not graph._solves_compiled() or budget < 0 or drive.ndim != 2 + or drive.dtype != np.float64 or not drive.flags.c_contiguous + ): + return None + batch, n = drive.shape + if n != graph.connectome.n or n == 0 or not fused.rows_finite(drive): + return None # the general path refuses it + if state is None: + v, a, products = np.zeros((batch, n)), np.zeros((batch, n)), None + else: + v = np.array(state.v, dtype=np.float64, copy=True, ndmin=2) + a = np.array(state.adaptation, dtype=np.float64, copy=True, ndmin=2) + if v.shape != (batch, n) or a.shape != (batch, n) or not ( + fused.rows_finite(v) and fused.rows_finite(a) + ): + return None + products = state.__dict__.get("_products") + flat = graph._fused_blocks()[0] + slope, threshold, rest, leak = _rule_of(model) + s = np.empty_like(v) + fused._publish(v, s, *fused._held(products), slope, threshold, rest, leak) + layout = graph.layout + out = np.empty(batch) + used, checks, taken, row_sweeps, kept = fused._fast_solve( + v, s, fused._standing(drive, graph.bias), flat, layout.starts, layout.pair_pre, + layout.pair_post, layout.offset, slope, threshold, rest, leak, budget, 16, + float(tolerance), min(float(tolerance), fused._FAST_SETTLED), out, + ) + result = BrainState(v=v, activation=s, adaptation=a, steps=taken) + _keep_products(result, fused._kept(flat, flat, batch, kept)) + return Equilibrium(result, out, used, tolerance, residual_checks=checks, row_sweeps=row_sweeps) + + +_NO_ROWS = np.zeros((1, 1)) +_NO_HELD = np.zeros((1, 1), dtype=np.bool_) +_NO_STORE = np.zeros((1, 1, 1)) +_NO_MASS = np.zeros(1) + + +def drive_spans(brain: Any) -> np.ndarray | None: + """The column ranges of a brain's moment drive (``_drive_kernel``): the sensory region, the + working trace's, the recall read's ports and the place store's key and value; None where + one is not a contiguous range. Made once per brain and its memories.""" + recall, trace, store = brain.recall, brain.working_memory, brain.hippocampus + held = brain.__dict__.get("_drive_spans") + if ( + held is not None and held[0] is recall and held[1] is trace and held[2] is store + and held[3] is brain.sensory_index + ): + return held[4] # type: ignore[no-any-return] + parts = ( + brain._sensory_columns, None if trace is None else trace._glow_columns, + *recall._head_columns[0], recall._port_columns, + None if store is None else store._pre_columns, + None if store is None else store._post_columns, + ) + spans: np.ndarray | None = np.zeros(12, dtype=np.int64) + for k, part in enumerate(parts): + if part is None: + continue + if not isinstance(part, slice) or spans is None: + spans = None + continue + if k in (2, 3): # the recall and next ports: their first column + spans[4 + (k - 2)] = part.start + else: + at = {0: 0, 1: 2, 4: 6, 5: 8, 6: 10}[k] + spans[at], spans[at + 1] = part.start, part.stop + brain.__dict__["_drive_spans"] = (recall, trace, store, brain.sensory_index, spans) + return spans + + +def moment_drive(brain: Any, x: np.ndarray, spans: np.ndarray) -> np.ndarray: + """The drive of a moment of the compiled moment in one pass (``_drive_kernel``), the recall + memory left holding its read as ``Recall.stimulate`` leaves it: the same drive and memory to + the bit as the read of each memory in turn.""" + from .resident import _Moment + from .stream import _of + + recall, trace, store = brain.recall, brain.working_memory, brain.hippocampus + rows = len(x) + c = _of(recall.previous, rows) + w_x, w_c, w, columns = packed_maps(recall) + numbers, scalars, table, grid = _settings(recall) + slots = recall.slots + if numbers[4] and slots <= recall.position: # no slot reaches the grid's columns + w_x, w_c, w = recall.__dict__["_packed"][3] + numbers = numbers.copy() + numbers[4] = 0 + keys, first = _behind(recall, "keys") + values, at_values = _behind(recall, "values") + nexts, at_nexts = _behind(recall, "nexts") if recall.next else (values, at_values) + held, at_held = _behind(recall, "held") + if not first == at_values == at_nexts == at_held: + keys, values, held = (np.ascontiguousarray(a) for a in (recall.keys, recall.values, + recall.held)) + nexts = np.ascontiguousarray(recall.nexts) if recall.next else values + first = 0 + has_trace = trace is not None and len(trace.trace) == rows + shared = store is not None and len(store.strength) != rows + strength = ( + _NO_STORE if store is None + else np.ascontiguousarray(store._baseline()[None, :, :]) if shared else store.strength + ) + gains = np.array([ + brain.brain.neuron_model.stimulus_amplitude, + trace.amplitude * trace.hearing if has_trace else 0.0, + 1.0 if recall.drive_gain else recall.amplitude, + recall.amplitude * recall.hearing, + 0.0 if store is None else float(store.amplitude), + ]) + flags = np.array([ + has_trace, recall.drive_gain, store is not None, shared, + store is not None and bool(store.normalize), True, has_trace, + ], dtype=np.int64) + drive, key, q, attention, ports = _drive_kernel( + x, np.ascontiguousarray(c), w_x, w_c, w, columns, numbers, scalars, keys, values, nexts, + bool(recall.next), held, first, slots, recall.told, recall.focus, table, grid, + brain.connectome.n, spans, gains, flags, trace.trace if has_trace else _NO_ROWS, + strength, _NO_MASS if store is None else store.mass, + ) + recall.last = _Moment(x, c, q, key, x) + attention.flags.writeable = False + recall._attended = attention + recall._head_reads = None + if recall.drive_gain: + recall.trace = ports + return drive + + +# -- the act's choice and write in one pass + +if njit is not None: + from .fused import _clear_choice + + @_compiled + def _after_kernel( + s: np.ndarray, spans: np.ndarray, temperature: float, decay: float, trace: np.ndarray, + has_trace: bool, write: bool, key: np.ndarray, value: np.ndarray, ended: np.ndarray, + attended: np.ndarray, has_attended: bool, focus: np.ndarray, keys_room: np.ndarray, + values_room: np.ndarray, nexts_room: np.ndarray, held_room: np.ndarray, first: int, + slots: int, drop: int, stop: int, + ) -> tuple[np.ndarray, bool, np.ndarray, np.ndarray, np.ndarray, np.ndarray]: + """What a greedy act does after its solve, from the settled activations ``s``: the + choice of the motor span where every row's largest stands clear (``_clear_choice``), the + working trace's step toward the source span (``Trace.update``), the cortex the recall + memory carries (the association span) and, with ``write``, the moment's write + (``_write_kernel``). ``spans`` are the motor, the source and the association ranges. + Returns the action, whether it stands clear, the trace, the source activation, the + cortex and the focus.""" + rows = s.shape[0] + motor = np.empty((rows, spans[1] - spans[0])) + for b in range(rows): + for j in range(spans[1] - spans[0]): + motor[b, j] = s[b, spans[0] + j] + action, clear = _clear_choice(motor, temperature) + source = np.empty((rows, spans[3] - spans[2])) + for b in range(rows): + for j in range(spans[3] - spans[2]): + source[b, j] = s[b, spans[2] + j] + stepped = np.empty_like(source) + if has_trace: + for b in range(rows): + for j in range(source.shape[1]): + stepped[b, j] = decay * trace[b, j] + (1.0 - decay) * source[b, j] + cortex = np.empty((rows, spans[5] - spans[4])) + for b in range(rows): + for j in range(spans[5] - spans[4]): + cortex[b, j] = s[b, spans[4] + j] + moved = focus.copy() + if write: + moved = _write_kernel( + key, value, held_room[:, first:first + slots], ended, attended, has_attended, + focus, keys_room, values_room, nexts_room, held_room, first, drop, stop, + ) + return action, clear, stepped, source, cortex, moved + + +def _rooms(recall: Any, rows: int) -> tuple[Any, ...] | None: + """Where the next write goes (``recall_write``): the rooms behind the slots, the first slot, + the slots dropped and the column of the new slot; None where the ordinary write runs.""" + if rows != recall.streams: + return None + held = recall._rooms + keys, values = held.get("keys"), held.get("values") + nexts, slots = held.get("nexts"), held.get("held") + if keys is None or values is None or nexts is None or slots is None: + return None + # every slot array is the view of its room that the memory made, with room behind it, and + # the four share one history (the first slot and the column of the next) + history = keys[2:4] + for room, old in ( + (keys, recall.keys), (values, recall.values), (nexts, recall.nexts), + (slots, recall.held), + ): + if ( + room[1] is not old or old.base is not room[0] or room[3] >= room[0].shape[1] + or room[2:4] != history + ): + return None + return keys, values, nexts, slots, max(recall.keys.shape[1] + 1 - recall.capacity, 0) + + +def act_after(brain: Any, x: np.ndarray, free: Any, spans: np.ndarray) -> tuple[np.ndarray, bool]: + """A greedy act's choice, its working trace's step and its write after the solve, in one pass + (``_after_kernel``): the same action, trace and slots to the bit as the actor's greedy act, + ``Trace.update`` and ``Recall.witness`` in turn. Returns the action and whether it stands + clear (where it does not, the caller takes the softmax's argmax).""" + recall, trace = brain.recall, brain.working_memory + s = free.activation + rows = len(s) + moment = recall.last + rooms = _rooms(recall, rows) if moment is not None and moment.event is x else None + has_trace = trace is not None and len(trace.trace) == rows + if trace is not None and not has_trace: + trace.reset(rows) # as ``Trace.update`` begins streams of another batch + has_trace = True + attended = recall._attended + has_attended = ( + attended is not None and attended.shape == (rows, recall.slots) and recall.slots > 0 + ) + if rooms is None: + write_args: tuple[Any, ...] = ( + False, _NONE2, _NONE2, recall.ended, _NONE2, False, recall.focus, _NO_STORE, + _NO_STORE, _NO_STORE, np.zeros((1, 1), dtype=np.bool_), 0, 0, 0, 0, + ) + else: + keys, values, nexts, held, drop = rooms + first, stop = keys[2], keys[3] + write_args = ( + True, moment.key, moment.event, recall.ended, + attended if has_attended else _NONE2, has_attended, recall.focus, keys[0], + values[0], nexts[0], held[0], first, recall.slots, drop, stop, + ) + action, clear, stepped, source, cortex, focus = _after_kernel( + s, spans, float(brain.learner.config.temperature), + 0.0 if trace is None else float(trace.decay), + trace.trace if trace is not None else _NONE2, has_trace, *write_args, + ) + if trace is not None: + trace.trace = stepped + trace.last = source + trace.cold[:] = False + if rooms is None: + recall.witness(x, free, None) + return action, clear + if recall.ended.any(): + recall.ended = np.zeros(rows, dtype=bool) + new_first = first + drop + for name, room in zip(("keys", "values", "nexts", "held"), rooms[:4], strict=True): + view = room[0][:, new_first : stop + 1] + recall._rooms[name] = (room[0], view, new_first, stop + 1) + setattr(recall, name, view) + recall._contents = [] + recall._content_keys = None + recall.told = recall.told + 1 + recall.focus = focus + recall.writes += rows + if recall.cortex_norm: # as ``Recall.witness``: the maps read it at unit norm + norm = np.linalg.norm(cortex, axis=1, keepdims=True) + cortex = np.where(norm > 0, cortex / np.where(norm > 0, norm, 1.0), cortex) + recall.previous = cortex + recall.last = None + return action, clear + + +def after_spans(brain: Any) -> np.ndarray | None: + """The motor, the working trace's source and the association ranges of a brain, for + ``act_after``; None where one is not a contiguous range. Made once per brain.""" + from .stream import columns as contiguous + + trace = brain.working_memory + output = brain.learner.output_index + held = brain.__dict__.get("_after_spans") + if ( + held is not None and held[0] is output and held[1] is trace + and held[2] is brain.recall + ): + return held[3] # type: ignore[no-any-return] + parts: tuple[Any, ...] = ( + contiguous(np.asarray(output, dtype=np.int64)), + trace._hidden_columns if trace is not None else slice(0, 0), + brain.recall._association_columns, + ) + spans: np.ndarray | None = None + if all(isinstance(part, slice) for part in parts): + spans = np.array([v for part in parts for v in (part.start, part.stop)], dtype=np.int64) + brain.__dict__["_after_spans"] = (output, trace, brain.recall, spans) + return spans + + +def written(recall: Any, rooms: tuple[Any, ...], rows: int, focus: np.ndarray, + cortex: np.ndarray) -> None: + """The memory after a compiled write into ``rooms`` (``_rooms``), as ``Recall.witness`` + leaves it: the slots one longer, the episode and the focus moved, the cortex carried.""" + first, stop, drop = rooms[0][2], rooms[0][3], rooms[4] + if recall.ended.any(): + recall.ended = np.zeros(rows, dtype=bool) + new_first = first + drop + for name, room in zip(("keys", "values", "nexts", "held"), rooms[:4], strict=True): + view = room[0][:, new_first : stop + 1] + recall._rooms[name] = (room[0], view, new_first, stop + 1) + setattr(recall, name, view) + recall._contents = [] + recall._content_keys = None + recall.told = recall.told + 1 + recall.focus = focus + recall.writes += rows + if recall.cortex_norm: # as ``Recall.witness``: the maps read it at unit norm + norm = np.linalg.norm(cortex, axis=1, keepdims=True) + cortex = np.where(norm > 0, cortex / np.where(norm > 0, norm, 1.0), cortex) + recall.previous = cortex + recall.last = None + + +# -- an act, and a lesson's judgment, in one call + +if njit is not None: + from .fused import _fast_solve, _publish, _standing, rows_finite + + @_compiled + def _moment_kernel( + x: np.ndarray, c: np.ndarray, w_x: np.ndarray, w_c: np.ndarray, w: np.ndarray, + columns: np.ndarray, numbers: np.ndarray, scalars: np.ndarray, keys: np.ndarray, + values: np.ndarray, nexts: np.ndarray, has_next: bool, held: np.ndarray, first: int, + slots: int, told: np.ndarray, focus: np.ndarray, table: np.ndarray, grid: np.ndarray, + neurons: int, spans: np.ndarray, gains: np.ndarray, flags: np.ndarray, + trace: np.ndarray, strength: np.ndarray, mass: np.ndarray, warm: np.ndarray, + warm_a: np.ndarray, has_warm: bool, kept_in: np.ndarray, has_kept: bool, + bias: np.ndarray, flat: np.ndarray, starts: np.ndarray, pair_pre: np.ndarray, + pair_post: np.ndarray, offset: np.ndarray, rule: np.ndarray, budget: int, chunk: int, + tolerance: float, settled: float, mode: int, after: np.ndarray, temperature: float, + decay: float, write: bool, ended: np.ndarray, drop: int, stop: int, + ) -> tuple[Any, ...]: + """A moment in one call, each part the kernel the passes in turn call: the drive + (``_drive_kernel``), the free solve from the warm state (the activations of its + potentials, ``fused._fast_solve``) and, where every row qualifies, for an act (``mode`` + 0) its choice, trace step and write (``_after_kernel``), for a lesson's judgment + (``mode`` 1) each row's most active motor neuron. ``status`` is 0 where the moment is + whole, 1 where the drive or the warm state is not finite and 2 where a row does not + qualify; nothing of the memory is written then.""" + rows = x.shape[0] + drive, key, q, attention, ports = _drive_kernel( + x, c, w_x, w_c, w, columns, numbers, scalars, keys, values, nexts, has_next, held, + first, slots, told, focus, table, grid, neurons, spans, gains, flags, trace, + strength, mass, + ) + v = np.zeros((rows, neurons)) + a = np.zeros((rows, neurons)) + s = np.empty((rows, neurons)) + out = np.empty(rows) + used = 0 + checks = 0 + taken = 0 + row_sweeps = 0 + kept = kept_in + status = 0 + if not rows_finite(drive): + status = 1 + if status == 0 and has_warm: + if rows_finite(warm) and rows_finite(warm_a): + for b in range(rows): + for i in range(neurons): + v[b, i] = warm[b, i] + a[b, i] = warm_a[b, i] + else: + status = 1 + if status == 0: + _publish(v, s, kept_in, has_kept, rule[0], rule[1], rule[2], rule[3]) + used, checks, taken, row_sweeps, kept = _fast_solve( + v, s, _standing(drive, bias), flat, starts, pair_pre, pair_post, offset, + rule[0], rule[1], rule[2], rule[3], budget, chunk, tolerance, settled, out, + ) + for b in range(rows): + if not out[b] <= tolerance: + status = 2 + action = np.zeros(rows, dtype=np.int64) + clear = True + stepped = np.empty((0, 0)) + source = np.empty((0, 0)) + cortex = np.empty((0, 0)) + moved = focus + if status == 0 and mode == 0: + action, clear, stepped, source, cortex, moved = _after_kernel( + s, after, temperature, decay, trace, flags[6] != 0, write, key, x, ended, + attention, slots > 0, focus, keys, values, nexts, held, first, slots, drop, + stop, + ) + if status == 0 and mode == 1: + start, width = after[0], after[1] - after[0] + for b in range(rows): + best = s[b, start] + at = 0 + for j in range(1, width): + if s[b, start + j] > best: + best = s[b, start + j] + at = j + action[b] = at + return ( + status, drive, key, q, attention, ports, v, a, s, out, used, checks, taken, + row_sweeps, kept, action, clear, stepped, source, cortex, moved, + ) + + +_RULE_ARRAYS: dict[int, tuple[Any, np.ndarray]] = {} + + +def _rule_array(model: Any) -> np.ndarray: + """``_rule_of`` as the array the one-call kernel reads, made once per model.""" + held = _RULE_ARRAYS.get(id(model)) + if held is None or held[0] is not model: + held = (model, np.array(_rule_of(model), dtype=float)) + if len(_RULE_ARRAYS) > 64: + _RULE_ARRAYS.clear() + _RULE_ARRAYS[id(model)] = held + return held[1] + + +def motor_spans(brain: Any) -> np.ndarray | None: + """The motor region's range as the one-call kernel's judgment reads it (``after`` of + ``_moment_kernel``); None where it is not one contiguous range. Made once per brain.""" + from .stream import columns as contiguous + + index = brain.motor_index + held = brain.__dict__.get("_motor_spans") + if held is not None and held[0] is index: + return held[1] # type: ignore[no-any-return] + part: Any = contiguous(np.asarray(index, dtype=np.int64)) + spans = ( + np.array([part.start, part.stop, 0, 0, 0, 0], dtype=np.int64) + if isinstance(part, slice) else None + ) + brain.__dict__["_motor_spans"] = (index, spans) + return spans + + +def one_call( + brain: Any, x: np.ndarray, state: Any, mode: int, spans: np.ndarray, after: np.ndarray +) -> tuple[Any, ...] | None: + """A moment of ``brain`` at the observations ``x`` from the warm ``state`` in one compiled + call (``_moment_kernel``): for an act (``mode`` 0) with ``after`` from ``after_spans``, for + a lesson's judgment (``mode`` 1) with ``after`` from ``motor_spans``. Returns the cortex of + the moment before, the rooms a write went into (None without a compiled write), the blocks + the solve read and the kernel's values; None where an array is not the kernel's kind (the + caller takes the passes in turn). The caller installs what the kernel returns.""" + recall, trace, store = brain.recall, brain.working_memory, brain.hippocampus + graph = brain.brain + rows = len(x) + n = graph.connectome.n + if not (x.flags.c_contiguous and x.flags.writeable): + return None + if state is None: + warm = warm_a = _NO_ROWS + has_warm = False + kept_in, has_kept = fused._NO_PRODUCTS + else: + warm, warm_a = state.v, state.adaptation + for array in (warm, warm_a): + if ( + type(array) is not np.ndarray or array.shape != (rows, n) + or array.dtype != np.float64 or not array.flags.c_contiguous + or not array.flags.writeable + ): + return None + has_warm = True + kept_in, has_kept = fused._held(state.__dict__.get("_products")) + c = recall.previous # ``stream._of``: streams of another batch begin from a silent cortex + c = np.ascontiguousarray(c if len(c) == rows else np.zeros((rows, *c.shape[1:]))) + w_x, w_c, w, columns = packed_maps(recall) + numbers, scalars, table, grid = _settings(recall) + slots = recall.slots + begun = bool(slots) and recall.streams == rows + if numbers[4] and slots <= recall.position: # no slot reaches the grid's columns + w_x, w_c, w = recall.__dict__["_packed"][3] + numbers = recall.__dict__["_kernel_settings"][2] + rooms = _rooms(recall, rows) if mode == 0 and begun else None + told, focus, ended = recall.told, recall.focus, recall.ended + if rooms is not None: + keys, values, nexts, held = rooms[0][0], rooms[1][0], rooms[2][0], rooms[3][0] + first, stop, drop = rooms[0][2], rooms[0][3], rooms[4] + elif not begun: # streams that begin read silence, and the general write begins them + stop = drop = first = slots = 0 + keys = values = nexts = _NO_STORE + held = _NO_HELD + told = focus = np.zeros(rows, dtype=np.int64) + ended = np.zeros(rows, dtype=np.bool_) + else: + stop = drop = 0 + keys, first = _behind(recall, "keys") + values, at_values = _behind(recall, "values") + nexts, at_nexts = _behind(recall, "nexts") if recall.next else (values, at_values) + held, at_held = _behind(recall, "held") + if not first == at_values == at_nexts == at_held: + keys, values, held = ( + np.ascontiguousarray(a) for a in (recall.keys, recall.values, recall.held) + ) + nexts = np.ascontiguousarray(recall.nexts) if recall.next else values + first = 0 + steps = trace is not None # a trace of another batch reads silence and begins at zero + has_trace = steps and len(trace.trace) == rows + traced = _NO_ROWS + if steps: + traced = trace.trace if has_trace else np.zeros((rows, trace.trace.shape[1])) + shared = store is not None and len(store.strength) != rows + if store is None: + strength, mass = _NO_STORE, _NO_MASS + elif shared: + strength, mass = np.ascontiguousarray(store._baseline()[None, :, :]), store.mass + else: + strength, mass = store.strength, store.mass + scales = ( + graph.neuron_model.stimulus_amplitude, + trace.amplitude * trace.hearing if has_trace else 0.0, + 1.0 if recall.drive_gain else recall.amplitude, + recall.amplitude * recall.hearing, + 0.0 if store is None else float(store.amplitude), + has_trace, bool(recall.drive_gain), store is not None, shared, + store is not None and bool(store.normalize), begun, steps, + ) + made = brain.__dict__.get("_moment_scales") + if made is None or made[0] != scales: + made = (scales, np.array(scales[:5], dtype=float), np.array(scales[5:], dtype=np.int64)) + brain.__dict__["_moment_scales"] = made + cfg = brain.learner.config + budget = cfg.free_steps + tolerance = float(cfg.tolerance) + flat = graph._fused_blocks()[0] + layout = graph.layout + out = _moment_kernel( + x, c, w_x, w_c, w, columns, numbers, scalars, keys, values, nexts, bool(recall.next), + held, first, slots, told, focus, table, grid, n, spans, made[1], made[2], + traced, strength, mass, warm, warm_a, has_warm, + kept_in, has_kept, graph.bias, flat, layout.starts, layout.pair_pre, layout.pair_post, + layout.offset, _rule_array(graph.neuron_model), + budget if budget < 2 else (budget + 1) // 2, 16, # the fast solve's chunk + tolerance, min(tolerance, fused._FAST_SETTLED), mode, after, + float(cfg.temperature), 0.0 if trace is None else float(trace.decay), + rooms is not None, ended, drop, stop, + ) + return c, rooms, flat, out, begun + + +def solved_state( + flat: np.ndarray, rows: int, v: np.ndarray, a: np.ndarray, s: np.ndarray, taken: int, + kept: np.ndarray, +) -> Any: + """The state a one-call solve ends on, with what it keeps, as ``free_solve`` returns it.""" + result = BrainState(v=v, activation=s, adaptation=a, steps=taken) + _keep_products(result, fused._kept(flat, flat, rows, kept)) + return result + + +# -- a taught lesson's phases and their checks in one call + +if njit is not None: + from .fused import _kernel, _residual_kernel + + @_compiled + def _phases_kernel( + warm_v: np.ndarray, warm_a: np.ndarray, kept_in: np.ndarray, has_kept: bool, + has_given: bool, drive: np.ndarray, bias: np.ndarray, flat: np.ndarray, + pairs: np.ndarray, apical_pairs: np.ndarray, starts: np.ndarray, pair_pre: np.ndarray, + pair_post: np.ndarray, offset: np.ndarray, freezable: np.ndarray, + freezable_nudged: np.ndarray, keep: np.ndarray, rule: np.ndarray, target0: np.ndarray, + nmask0: np.ndarray, gid0: np.ndarray, softmax0: np.ndarray, weight0: np.ndarray, + anchor0: np.ndarray, gain0: np.ndarray, target: np.ndarray, nmask: np.ndarray, + gid: np.ndarray, ngroups: int, beta: float, softmax_t: np.ndarray, weight: np.ndarray, + anchor: np.ndarray, anchor_gain: np.ndarray, pool_of: np.ndarray, + pool_index: np.ndarray, pool_starts: np.ndarray, pool_run: np.ndarray, + no_held: np.ndarray, free_steps: int, nudged_steps: int, tolerance: float, + stops: bool, nudged_tolerance: float, nudged_stops: bool, + ) -> tuple[Any, ...]: + """A finite lesson's three phases and their checks, each the kernel ``Brain.settle_batch`` + and ``Brain.residual`` launch, in their order: the free phase from the warm state and its + check, the phases nudged toward and away from the target from the free state, and their + checks. A phase starts from a copy of its state's potentials, publishes them from what + the state keeps and reads its products; a check reads and renews them.""" + dt, slope, threshold, rest, leak = rule[0], rule[1], rule[2], rule[3], rule[4] + strength, tau = rule[5], rule[6] + rows = drive.shape[0] + v0 = warm_v.copy() + a0 = warm_a.copy() + s0 = np.empty_like(v0) + _publish(v0, s0, kept_in, has_kept, slope, threshold, rest, leak) + taken0, change0, kept0 = _kernel( + v0, a0, s0, _standing(drive, bias), flat, pairs, flat, apical_pairs, starts, + pair_pre, pair_post, offset, freezable, keep, False, dt, slope, threshold, rest, + leak, False, strength, tau, False, target0, nmask0, gid0, 0, 0.0, softmax0, weight0, + anchor0, gain0, False, pool_of, pool_index, pool_starts, pool_run, 1.0, 1.0, 0.0, + no_held, False, free_steps, tolerance, stops, kept_in, has_given, + ) + read = np.empty_like(v0) + published = _publish(v0, read, kept0, True, slope, threshold, rest, leak) + out0 = np.empty(rows) + kept0 = _residual_kernel( + v0, a0, read, _standing(drive, bias), flat, pairs, flat, apical_pairs, starts, + pair_pre, pair_post, offset, keep, False, dt, slope, threshold, rest, leak, False, + strength, False, target0, nmask0, gid0, 0, 0.0, softmax0, weight0, anchor0, gain0, + False, pool_of, pool_index, pool_starts, pool_run, 1.0, 1.0, 0.0, no_held, False, + out0, kept0, True, published, + ) + v1 = v0.copy() + a1 = a0.copy() + s1 = np.empty_like(v1) + _publish(v1, s1, kept0, True, slope, threshold, rest, leak) + taken1, change1, kept1 = _kernel( + v1, a1, s1, _standing(drive, bias), flat, pairs, flat, apical_pairs, starts, + pair_pre, pair_post, offset, freezable_nudged, keep, False, dt, slope, threshold, + rest, leak, False, strength, tau, True, target, nmask, gid, ngroups, beta, + softmax_t, weight, anchor, anchor_gain, False, pool_of, pool_index, pool_starts, + pool_run, 1.0, 1.0, 0.0, no_held, False, nudged_steps, nudged_tolerance, + nudged_stops, kept0, True, + ) + v2 = v0.copy() + a2 = a0.copy() + s2 = np.empty_like(v2) + _publish(v2, s2, kept0, True, slope, threshold, rest, leak) + taken2, change2, kept2 = _kernel( + v2, a2, s2, _standing(drive, bias), flat, pairs, flat, apical_pairs, starts, + pair_pre, pair_post, offset, freezable_nudged, keep, False, dt, slope, threshold, + rest, leak, False, strength, tau, True, target, nmask, gid, ngroups, -beta, + softmax_t, weight, anchor, anchor_gain, False, pool_of, pool_index, pool_starts, + pool_run, 1.0, 1.0, 0.0, no_held, False, nudged_steps, nudged_tolerance, + nudged_stops, kept0, True, + ) + read = np.empty_like(v1) + published = _publish(v1, read, kept1, True, slope, threshold, rest, leak) + out1 = np.empty(rows) + kept1 = _residual_kernel( + v1, a1, read, _standing(drive, bias), flat, pairs, flat, apical_pairs, starts, + pair_pre, pair_post, offset, keep, False, dt, slope, threshold, rest, leak, False, + strength, True, target, nmask, gid, ngroups, beta, softmax_t, weight, anchor, + anchor_gain, False, pool_of, pool_index, pool_starts, pool_run, 1.0, 1.0, 0.0, + no_held, False, out1, kept1, True, published, + ) + read = np.empty_like(v2) + published = _publish(v2, read, kept2, True, slope, threshold, rest, leak) + out2 = np.empty(rows) + kept2 = _residual_kernel( + v2, a2, read, _standing(drive, bias), flat, pairs, flat, apical_pairs, starts, + pair_pre, pair_post, offset, keep, False, dt, slope, threshold, rest, leak, False, + strength, True, target, nmask, gid, ngroups, -beta, softmax_t, weight, anchor, + anchor_gain, False, pool_of, pool_index, pool_starts, pool_run, 1.0, 1.0, 0.0, + no_held, False, out2, kept2, True, published, + ) + return ( + v0, a0, s0, taken0, change0, kept0, out0, v1, a1, s1, taken1, change1, kept1, out1, + v2, a2, s2, taken2, change2, kept2, out2, + ) + + +def lesson_phases(learner: Any, drive: Any, warm: Any, target: np.ndarray) -> Any: + """The free phase, the two nudged phases and their checks of a finite lesson of ``learner`` + at the rows of ``drive`` from the ``warm`` state toward ``target``, in one compiled call + (``_phases_kernel``): ``(free, nudged, opposite, checks)``, the states with what they keep + and each phase's residual, to the bit as ``Learner.free``, ``Learner._nudged`` and + ``Brain.residual`` make them in turn; None where a phase takes another path (a device, a + compartment, adaptation, a recording, a state that is not finite), which the caller takes.""" + + cfg, graph = learner.config, learner.brain + model = graph.neuron_model + if ( + graph.backend != "cpu" or graph.apical is not None or model.adaptation is not None + or not graph._solves_alone() or cfg.free_steps < 1 or cfg.nudged_steps < 1 + or cfg.qualified or type(drive) is not np.ndarray or drive.ndim != 2 + or drive.dtype != np.float64 or not drive.flags.c_contiguous + ): + return None + blocks = graph._fused_blocks() + rows, n = drive.shape + if blocks is None or blocks[1] is not None or n != graph.connectome.n or not rows: + return None + flat = blocks[0] + if not fused.rows_finite(drive): + return None + if warm is None: + warm_v = warm_a = np.zeros((rows, n)) + products = None + else: + if warm.device is not None: + return None + warm_v, warm_a = warm.v, warm.adaptation + for array in (warm_v, warm_a): + if ( + type(array) is not np.ndarray or array.shape != (rows, n) + or array.dtype != np.float64 or not fused.rows_finite(array) + ): + return None + warm_v, warm_a = np.ascontiguousarray(warm_v), np.ascontiguousarray(warm_a) + products = warm.__dict__.get("_products") + kept_in, has_kept = fused._held(products) + has_given = fused._given(products, flat, flat, rows)[1] + layout = graph.layout + pull = learner.nudge_for(target, cfg.beta, None, None) + target_k, nmask, gid, ngroups, beta, softmax_t, weight, anchor, anchor_gain = ( + fused._nudge_arrays(pull, rows, n) + ) + target0, nmask0, gid0, _, _, softmax0, weight0, anchor0, gain0 = fused._no_nudge(rows, n) + freezable = ~fused._fed_ranges(layout) + freezable_nudged = freezable.copy() + freezable_nudged[ + np.searchsorted( + layout.starts, np.flatnonzero((nmask > 0) | (anchor_gain > 0)), side="right" + ) - 1 + ] = False + pairs, _, apical_pairs, _, pool_of, pool_index, pool_starts, pool_run, _, _, _, no_held, _ = ( + fused._compartment(None, layout, flat, None, rows, n) + ) + tolerance = cfg.tolerance if cfg.tolerance else None # a tolerance of zero never stops + nudged_tolerance = None if cfg.nudged_fixed else tolerance + out = _phases_kernel( + warm_v, warm_a, kept_in, has_kept, has_given, drive, graph.bias, flat, pairs, + apical_pairs, layout.starts, layout.pair_pre, layout.pair_post, layout.offset, + freezable, freezable_nudged, fused._NO_MASK, np.array(fused._rule(model), dtype=float), + target0, nmask0, gid0, softmax0, weight0, anchor0, gain0, target_k, nmask, gid, + int(ngroups), float(beta), softmax_t, weight, anchor, anchor_gain, pool_of, pool_index, + pool_starts, pool_run, no_held, int(cfg.free_steps), int(cfg.nudged_steps), + float(tolerance) if tolerance is not None else 0.0, tolerance is not None, + float(nudged_tolerance) if nudged_tolerance is not None else 0.0, + nudged_tolerance is not None, + ) + states, checks = [], [] + for k in range(3): + v, a, s, taken, change, kept, residual = out[7 * k : 7 * k + 7] + state = BrainState( + v=v, activation=s, adaptation=a, steps=int(taken), activity_change=change, + ) + _keep_products(state, fused._kept(flat, flat, rows, kept)) + states.append(state) + checks.append(residual) + return states[0], states[1], states[2], checks + + +# -- a taught lesson in one call: its phases, the update, the address lesson, the carried gains + +# A taught lesson of the compiled moment as one compiled call from its judgment to its written +# efficacies (``taught_lesson``), with the bits of the passes in turn. False runs them as those +# passes (the tests compare the two). +ONE_LESSON = True + +if njit is not None: + from .fused import _divisor, _group_power, _moments, _pick, _plain, _step, _tied, rows_finite + + @_vectorized + def _contrast_kernel( + s_plus: np.ndarray, s_minus: np.ndarray, starts: np.ndarray, pair_pre: np.ndarray, + pair_post: np.ndarray, offset: np.ndarray, edge_index: np.ndarray, span: float, + ) -> tuple[np.ndarray, np.ndarray]: + """A lesson's contrast between two phases over ``span`` (``Learner.contrast`` by blocks, + ``blocks.block_contrast``): every synapse's ``sum_b s+[b, pre] s+[b, post] - s-[b, pre] + s-[b, post]`` over the rows times ``span``, and every neuron's mean movement over + ``span``. A block is ``A+.T (B+ - B-)``, plus ``(A+ - A-).T B-`` where its source range + moved, each one matrix product as ``block_contrast`` makes it.""" + rows, n = s_plus.shape + g = np.empty(offset[offset.shape[0] - 1]) + for k in range(pair_pre.shape[0]): + a0, a1 = starts[pair_pre[k]], starts[pair_pre[k] + 1] + b0, b1 = starts[pair_post[k]], starts[pair_post[k] + 1] + a_plus = np.ascontiguousarray(s_plus[:, a0:a1]) + change = np.empty((rows, b1 - b0)) + moved = False + for b in range(rows): + for j in range(b1 - b0): + change[b, j] = s_plus[b, b0 + j] - s_minus[b, b0 + j] + for i in range(a1 - a0): + if s_plus[b, a0 + i] != s_minus[b, a0 + i]: + moved = True + block = np.dot(a_plus.T, change) + if moved: + source = np.empty((rows, a1 - a0)) + for b in range(rows): + for i in range(a1 - a0): + source[b, i] = s_plus[b, a0 + i] - s_minus[b, a0 + i] + block = block + np.dot(source.T, np.ascontiguousarray(s_minus[:, b0:b1])) + g[offset[k]:offset[k + 1]] = block.ravel() + raw = np.empty(edge_index.shape[0]) + _pick(g, edge_index, rows * span, raw) + neuron = np.empty(n) + for j in range(n): + total = s_plus[0, j] - s_minus[0, j] + for b in range(1, rows): + total += s_plus[b, j] - s_minus[b, j] + neuron[j] = total / rows / span + return raw, neuron + + @_vectorized + def _all_finite(a: np.ndarray) -> bool: + """Whether every entry of the vector ``a`` is finite (a finite number less itself is + zero), without a branch per entry: the loop vectorizes.""" + bad = False + for i in range(a.shape[0]): + bad |= not (a[i] - a[i] == 0.0) + return not bad + + @_vectorized + def _update_kernel( + raw: np.ndarray, raw_neuron: np.ndarray, velocity: np.ndarray, second: np.ndarray, + velocity_bias: np.ndarray, second_bias: np.ndarray, group: np.ndarray, groups: int, + neuron_group: np.ndarray, neuron_groups: int, rates: np.ndarray, partner: np.ndarray, + efficacy: np.ndarray, gain_pre: np.ndarray, plastic: np.ndarray, bias: np.ndarray, + order: np.ndarray, edge_index: np.ndarray, size: int, + ) -> tuple[Any, ...]: + """A lesson's update from its contrasts (``Learner._update`` for an all-plastic learner + without per-synapse rates or tied groups): the optimizer's moments and steps + (``_compiled_step``: ``fused.optimizer_step``'s passes for the synapses, the same + arithmetic for the neurons), the tied and bounded step with the effective weights + (``_compiled_apply``: ``fused.tied_step``'s pass) and the flat blocks of the new weights + (``Layout.flat``). ``rates`` holds the momentum, one less it and its correction, the + same three of the second moment, the rates of the synapses and the biases, the absolute + floor, the relative floor's share and its tiny term, the lesson's share and the bound. + Returns a status (0, or which check of the passes in turn refuses the update: 1 the + contrasts, 2 the moments, 3 the steps, 4 the tied steps, 5 the efficacies, 6 the + weights, 7 the biases), the four moments, the synapses' tied steps, the neurons' + steps, the efficacies, the weights, the biases and the blocks. Nothing is written in + place.""" + momentum, momentum_rest, momentum_correction = rates[0], rates[1], rates[2] + rho, rho_rest, rho_correction = rates[3], rates[4], rates[5] + eta, eta_bias, floor, relative, tiny = rates[6], rates[7], rates[8], rates[9], rates[10] + share, cap = rates[11], rates[12] + count, n = raw.shape[0], raw_neuron.shape[0] + has_momentum, has_normalize = momentum > 0, rho > 0 + status = 0 + if not (_all_finite(raw) and _all_finite(raw_neuron)): + status = 1 + # the neurons' momentum + term_bias = np.empty(n) + new_velocity_bias = velocity_bias + if has_momentum: + new_velocity_bias = np.empty(n) + for j in range(n): + moving = momentum * velocity_bias[j] + momentum_rest * raw_neuron[j] + new_velocity_bias[j] = moving + term_bias[j] = moving / momentum_correction + else: + for j in range(n): + term_bias[j] = raw_neuron[j] + # the synapses' moments and steps + new_velocity = np.empty(count) if has_momentum else velocity + new_second = np.empty(count) if has_normalize else second + step = np.empty(count) + if not (has_normalize and relative != 0.0): + _plain( + raw, velocity, second, momentum, momentum_rest, momentum_correction, rho, + rho_rest, rho_correction, has_momentum, has_normalize, eta, floor, new_velocity, + new_second, step, + ) + else: + term, root = np.empty(count), np.empty(count) + _moments( + raw, velocity, second, momentum, momentum_rest, momentum_correction, rho, + rho_rest, rho_correction, has_momentum, new_velocity, new_second, term, root, + ) + power, members = _group_power(root, group, groups) + shared = np.empty(groups) + for k in range(groups): + shared[k] = np.sqrt(power[k] / members[k]) + _step(term, root, group, shared, eta, relative, tiny, step) + # the neurons' second moment and steps + new_second_bias = second_bias + if has_normalize: + new_second_bias = np.empty(n) + root_bias = np.empty(n) + for j in range(n): + held = rho * second_bias[j] + rho_rest * (raw_neuron[j] * raw_neuron[j]) + new_second_bias[j] = held + root_bias[j] = np.sqrt(held / rho_correction) + rms = np.empty(n) + if relative != 0.0: + power, members = _group_power(root_bias, neuron_group, neuron_groups) + shared = np.empty(neuron_groups) + for k in range(neuron_groups): + shared[k] = np.sqrt(power[k] / members[k]) + _divisor(root_bias, neuron_group, shared, relative, tiny, rms) + else: + for j in range(n): + rms[j] = root_bias[j] + floor + for j in range(n): + term_bias[j] = term_bias[j] / rms[j] + if status == 0 and not ( + _all_finite(new_velocity) and _all_finite(new_velocity_bias) and _all_finite(new_second) + and _all_finite(new_second_bias) + ): + status = 2 + # the lesson's share of the steps, tied and bounded + for e in range(count): + step[e] = share * step[e] + delta_bias = np.empty(n) + for j in range(n): + delta_bias[j] = share * (eta_bias * term_bias[j]) + if status == 0 and not (_all_finite(step) and _all_finite(delta_bias)): + status = 3 + delta, scale, weights = np.empty(count), np.empty(count), np.empty(count) + bad = _tied(step, partner, efficacy, gain_pre, cap, delta, scale, weights) + if status == 0: + if bad[0] != 0: + status = 4 + elif bad[1] != 0: + status = 5 + elif bad[2] != 0: + status = 6 + new_bias = np.empty(n) + for j in range(n): + if not plastic[j]: + delta_bias[j] = 0.0 + new_bias[j] = bias[j] + delta_bias[j] + if status == 0 and not _all_finite(new_bias): + status = 7 + # the blocks of the new weights, a synapse at each of its entries + if order.shape[0] == size: + flat = np.empty(size) + for i in range(size): + flat[i] = weights[order[i]] + else: + flat = np.zeros(size) + for e in range(count): + flat[edge_index[e]] = weights[e] + return ( + status, new_velocity, new_second, new_velocity_bias, new_second_bias, delta, + delta_bias, scale, weights, new_bias, flat, + ) + + @_vectorized + def _gains_kernel( + s_plus: np.ndarray, s_minus: np.ndarray, rows: np.ndarray, heard: np.ndarray, + span: float, weights: np.ndarray, routes: int, synapses: Any, sources: Any, + targets: Any, traces: Any, live: np.ndarray, spans: np.ndarray, scales: np.ndarray, + rule: np.ndarray, second: np.ndarray, gains: np.ndarray, bounds: np.ndarray, + stepping: np.ndarray, + ) -> tuple[np.ndarray, np.ndarray, np.ndarray]: + """The carried routes' ``c`` per taught row and route (``Hearing._contrast`` after a + lesson, ``_route_kernel`` per route): the association's movement between the nudged + phases over ``span`` read back through each route's synapses of ``weights``, against + what the route's memory holds at the streams ``rows``. A route whose memory holds + another batch reads silence (``live``). Then the gains' step on the mean of ``c`` over + the rows (``Hearing.step``), with ``stepping`` its forgetting factor, one less it, the + correction of this step's count, the rate and the floor: the new second moments and the + new gains within their ``bounds``. Returns ``c`` too; with one route the caller steps + (NumPy sums one column of eight rows and more in another order than row by row).""" + count, width = rows.shape[0], heard.shape[0] + movement = np.empty((count, width)) + for i in range(count): + for j in range(width): + movement[i, j] = (s_plus[i, heard[j]] - s_minus[i, heard[j]]) / span + out = np.zeros((count, max(routes, 1))) + for k in range(routes): + if not live[k]: + continue + start, stop = spans[k, 0], spans[k, 1] + trace = traces[k] + held = np.empty((count, stop - start)) + for i in range(count): + for j in range(stop - start): + held[i, j] = trace[rows[i], start + j] + c = _route_kernel( + movement, weights, synapses[k], sources[k], targets[k], held, scales[k, 0], + scales[k, 1], rule[1], rule[2], rule[3], rule[4], + ) + for i in range(count): + out[i, k] = c[i] + rho, rho_rest, correction, rate, floor = ( + stepping[0], stepping[1], stepping[2], stepping[3], stepping[4] + ) + new_second = np.empty(max(routes, 1)) + new_gains = np.empty(max(routes, 1)) + for k in range(routes): + total = out[0, k] + for i in range(1, count): + total += out[i, k] + mean = total / count + held_second = rho * second[k] + rho_rest * (mean * mean) + new_second[k] = held_second + gain = gains[k] + rate * mean / (np.sqrt(held_second / correction) + floor) + if gain == gain: # ``np.clip``: the larger of it and zero, then the smaller of that + if not gain > 0.0: # and the bound; a NaN stays + gain = 0.0 + if not gain < bounds[k]: + gain = bounds[k] + new_gains[k] = gain + return out, new_second, new_gains + + @_vectorized + def _peak(residual: np.ndarray) -> float: + """``np.max``: the largest entry, NaN where one is.""" + best = float(residual[0]) + for i in range(residual.shape[0]): + r = float(residual[i]) + if r != r: + return r + if r > best: + best = r + return best + + @_vectorized + def _cut( + w_x: np.ndarray, w_c: np.ndarray, w: np.ndarray, columns: np.ndarray, k: int + ) -> tuple[np.ndarray, np.ndarray, np.ndarray]: + """Part ``k`` of the packed maps as arrays of its own (``packed_maps``'s columns).""" + start, stop = columns[k], columns[k + 1] + return ( + np.ascontiguousarray(w_x[:, start:stop]), np.ascontiguousarray(w_c[:, start:stop]), + w[start:stop].copy(), + ) + + @_compiled + def _taught_kernel( + differs: np.ndarray, labels: np.ndarray, drive: np.ndarray, warm_v: np.ndarray, + warm_a: np.ndarray, has_warm: bool, kept_in: np.ndarray, has_kept: bool, + has_given: bool, bias: np.ndarray, flat: np.ndarray, efficacy: np.ndarray, + gain_pre: np.ndarray, velocity: np.ndarray, second: np.ndarray, + velocity_bias: np.ndarray, second_bias: np.ndarray, rates: np.ndarray, + addressed: bool, keys: np.ndarray, values: np.ndarray, nexts: np.ndarray, + held: np.ndarray, first: int, slots: int, has_next: bool, inputs: np.ndarray, + cortex: np.ndarray, address: np.ndarray, key: np.ndarray, w_x: np.ndarray, + w_c: np.ndarray, w: np.ndarray, columns: np.ndarray, numbers: np.ndarray, + scalars: np.ndarray, told: np.ndarray, focus: np.ndarray, table: np.ndarray, + grid: np.ndarray, positions: bool, routes: int, traces: Any, live: np.ndarray, + scales: np.ndarray, heard_second: np.ndarray, gains: np.ndarray, stepping: np.ndarray, + output_index: np.ndarray, pairs: np.ndarray, apical_pairs: np.ndarray, + starts: np.ndarray, pair_pre: np.ndarray, pair_post: np.ndarray, offset: np.ndarray, + freezable: np.ndarray, freezable_nudged: np.ndarray, keep: np.ndarray, + rule: np.ndarray, target0: np.ndarray, nmask0: np.ndarray, gid0: np.ndarray, + softmax0: np.ndarray, anchor0: np.ndarray, gain0: np.ndarray, nmask: np.ndarray, + gid: np.ndarray, ngroups: int, beta: float, softmax_t: np.ndarray, anchor: np.ndarray, + anchor_gain: np.ndarray, pool_of: np.ndarray, pool_index: np.ndarray, + pool_starts: np.ndarray, pool_run: np.ndarray, no_held: np.ndarray, free_steps: int, + nudged_steps: int, tolerance: float, stops: bool, nudged_tolerance: float, + nudged_stops: bool, edge_index: np.ndarray, span: float, group: np.ndarray, + groups: int, neuron_group: np.ndarray, neuron_groups: int, partner: np.ndarray, + plastic: np.ndarray, order: np.ndarray, association: np.ndarray, + value_synapses: np.ndarray, value_sources: np.ndarray, value_targets: np.ndarray, + next_synapses: np.ndarray, next_sources: np.ndarray, next_targets: np.ndarray, + heard: np.ndarray, route_synapses: Any, route_sources: Any, route_targets: Any, + route_spans: np.ndarray, bounds: np.ndarray, + ) -> tuple[Any, ...]: + """A taught lesson from its judgment to its written efficacies, each part the kernel + the passes in turn launch, in their order: the rows whose answer ``differs`` from their + label and their targets (``Learner.step``), the free phase, the nudged phases and their + checks (``_phases_kernel``), the contrast (``_contrast_kernel``), the update + (``_update_kernel``), the address lesson through the new weights (``_lesson_kernel``) + and the carried routes' contrasts through them with the gains' step + (``_gains_kernel``). Nothing is written in place: the caller stores what comes back + where the status is 0, and takes the passes in turn where it is not (1: a taught row's + drive or warm state is not finite; 10 and above: the update refuses, + ``_update_kernel``'s status plus 10). The stepped maps come back packed, without the + grid's columns (``packed_maps``) and each of their six parts apart (``Recall``'s + maps).""" + batch, n = drive.shape + count = 0 + for b in range(batch): + if differs[b]: + count += 1 + rows = np.empty(count, dtype=np.int64) + taught = np.empty((count, n)) + v = np.zeros((count, n)) + a = np.zeros((count, n)) + target = np.zeros((count, n)) + i = 0 + for b in range(batch): + if differs[b]: + rows[i] = b + for j in range(n): + taught[i, j] = drive[b, j] + if has_warm: + for j in range(n): + v[i, j] = warm_v[b, j] + a[i, j] = warm_a[b, j] + target[i, output_index[labels[b]]] = 1.0 + i += 1 + status = 0 + if not (rows_finite(taught) and rows_finite(v) and rows_finite(a)): + status = 1 # the passes in turn take this lesson: the phases here run from rest + for i in range(count): + for j in range(n): + taught[i, j] = 0.0 + v[i, j] = 0.0 + a[i, j] = 0.0 + weight = np.ones(count) + ( + _, _, _, taken0, _, _, out0, _, _, s1, taken1, _, _, out1, _, _, s2, taken2, _, _, + out2, + ) = _phases_kernel( + v, a, kept_in, has_kept, has_given, taught, bias, flat, pairs, apical_pairs, starts, + pair_pre, pair_post, offset, freezable, freezable_nudged, keep, rule, target0, + nmask0, gid0, softmax0, weight, anchor0, gain0, target, nmask, gid, ngroups, beta, + softmax_t, weight, anchor, anchor_gain, pool_of, pool_index, pool_starts, pool_run, + no_held, free_steps, nudged_steps, tolerance, stops, nudged_tolerance, nudged_stops, + ) + raw, raw_neuron = _contrast_kernel( + s1, s2, starts, pair_pre, pair_post, offset, edge_index, span + ) + ( + refused, new_velocity, new_second, new_velocity_bias, new_second_bias, delta, + delta_bias, scale, weights, new_bias, new_flat, + ) = _update_kernel( + raw, raw_neuron, velocity, second, velocity_bias, second_bias, group, groups, + neuron_group, neuron_groups, rates, partner, efficacy, gain_pre, plastic, bias, + order, edge_index, flat.shape[0], + ) + if status == 0 and refused != 0: + status = 10 + refused + moved = 0 + error = 0.0 + new_x, new_c, new = w_x, w_c, w + address_ok = False + position_ok = False + if addressed and status == 0: + width = association.shape[0] + movement = np.empty((count, width)) + for i in range(count): + for j in range(width): + movement[i, j] = s1[i, association[j]] - s2[i, association[j]] + slot_held = np.ascontiguousarray(held[:, first:first + slots]) + masked = False + for b in range(batch): + for m in range(slots): + if not slot_held[b, m]: + masked = True + slot_values = np.ascontiguousarray(values[:, first:first + slots, :]) + slot_nexts = ( + np.ascontiguousarray(nexts[:, first:first + slots, :]) if has_next + else slot_values + ) + moved, error, new_x, new_c, new, address_ok, position_ok = _lesson_kernel( + rows, movement, weights, value_synapses, value_sources, value_targets, + next_synapses, next_sources, next_targets, has_next, slot_values, slot_nexts, + np.ascontiguousarray(keys[:, first:first + slots, :]), slot_held, masked, + inputs, cortex, address, key, w_x, w_c, w, columns, numbers, scalars, told, + focus, table, grid, positions, + ) + contrasts, new_heard_second, new_gains = _gains_kernel( + s1, s2, rows, heard, span, weights, routes, route_synapses, route_sources, + route_targets, traces, live, route_spans, scales, rule, heard_second, gains, bounds, + stepping, + ) + end = columns[5] + return ( + status, count, taken0, taken1, taken2, _peak(out0), _peak(out1), _peak(out2), + new_velocity, new_second, new_velocity_bias, new_second_bias, delta, delta_bias, + scale, weights, new_bias, new_flat, moved, error, new_x, new_c, new, address_ok, + position_ok, contrasts, new_heard_second, new_gains, + np.ascontiguousarray(new_x[:, :end]), np.ascontiguousarray(new_c[:, :end]), + new[:end].copy(), + ) + ( + _cut(new_x, new_c, new, columns, 0) + _cut(new_x, new_c, new, columns, 1) + + _cut(new_x, new_c, new, columns, 2) + _cut(new_x, new_c, new, columns, 3) + + _cut(new_x, new_c, new, columns, 4) + _cut(new_x, new_c, new, columns, 5) + ) + + +def contrast( + layout: Any, s_plus: Any, s_minus: Any, span: float +) -> tuple[np.ndarray, np.ndarray] | None: + """``Learner.contrast`` by blocks between the activations of two phases on the host, in one + compiled pass (``_contrast_kernel``): the per-synapse and per-neuron terms over ``span``. + None where the arrays are not the kernel's kind (the caller reads them with NumPy).""" + if not available(): + return None + for s in (s_plus, s_minus): + if ( + type(s) is not np.ndarray or s.dtype != np.float64 or s.ndim != 2 + or not s.flags.c_contiguous + ): + return None + if s_plus.shape != s_minus.shape or not len(s_plus): + return None + found: tuple[np.ndarray, np.ndarray] = _contrast_kernel( + s_plus, s_minus, layout.starts, layout.pair_pre, layout.pair_post, layout.offset, + layout.edge_index, float(span), + ) + return found + + +_NO_INDEX = np.zeros(0, dtype=np.int64) +_NO_TRACE = np.zeros((1, 1)) +_PLAIN_LESSONS: dict[type, bool] = {} + + +def _plain_lesson(kind: type) -> bool: + """Whether a learner of ``kind`` teaches a lesson by the passes ``_taught_kernel`` composes: + none of them is overridden.""" + from .learning import Learner + + plain = _PLAIN_LESSONS.get(kind) + if plain is None: + plain = all( + getattr(kind, name) is getattr(Learner, name) + for name in ( + "step", "free", "_nudged", "_diagnostic", "nudge_for", "targets", "update", + "_update", "contrast", "apply", "_compiled_apply", "_compiled_step", "held", + "_share", "_differs", "_projections", "_mlx_update_tables", "_device_kernel", + ) + ) + _PLAIN_LESSONS[kind] = plain + return plain + + +def _lesson_plan(brain: Any) -> tuple[Any, ...] | None: + """What ``_taught_kernel`` reads that a lesson does not change, made once per learner, + layout, settings and wiring: the layout's tables, the nudge's arrays, the optimizer's + groups and pairs, the read's synapses and the carried routes'; with them the arrays the + caller fills per lesson (the update's numbers, the routes' scales and gains). None where + the learner teaches another way (a subclass, tied groups, a sparse or parallel layout, a + budget of no sweep).""" + from .learning import _RELATIVE_TINY + + learner, recall, hearing = brain.learner, brain.recall, brain.hearing + graph = learner.brain + cfg, layout, model = learner.config, graph.layout, graph.neuron_model + routes = hearing.routes if hearing is not None else None + key = ( + cfg, layout, model, learner.output_mask, learner.output_groups, learner.output_index, + learner.plastic_synapses, learner.plastic_neurons, learner._paired, learner._members, + recall, hearing, routes, type(learner), + ) + held = brain.__dict__.get("_lesson_plan") + if held is not None: + made = held[0] # the objects the plan was made from, held, so compared by identity + for k in range(14): + if key[k] is not made[k]: + break + else: + if held[1] == (0 if routes is None else len(routes)): + return held[2] # type: ignore[no-any-return] + plan: tuple[Any, ...] | None = None + w = graph.connectome + n = w.n + if ( + _plain_lesson(type(learner)) and cfg.free_steps >= 1 and cfg.nudged_steps >= 1 + and not cfg.qualified and cfg.tolerance is not None and learner.slot_count == 1 + and learner.plastic_synapses is not None and learner.plastic_neurons is not None + and learner.plastic_neurons.flags.c_contiguous and not len(learner._members) + and graph._blocked and w.synapses > 4 * n and not layout.parallel_synapses + and not fused._elements_in_parallel(w.synapses) + ): + pull = learner.nudge_for(np.zeros((1, n)), cfg.beta, None, None) + _, nmask, gid, ngroups, beta, softmax_t, _, anchor, anchor_gain = fused._nudge_arrays( + pull, 1, n + ) + target0, nmask0, gid0, _, _, softmax0, _, anchor0, gain0 = fused._no_nudge(1, n) + freezable = ~fused._fed_ranges(layout) + freezable_nudged = freezable.copy() + freezable_nudged[ + np.searchsorted( + layout.starts, np.flatnonzero((nmask > 0) | (anchor_gain > 0)), side="right" + ) - 1 + ] = False + compartment = fused._compartment(None, layout, np.zeros(0), None, 1, n) + pairs, apical_pairs, no_held = compartment[0], compartment[2], compartment[11] + pool_of, pool_index, pool_starts, pool_run = compartment[4:8] + tolerance = cfg.tolerance if cfg.tolerance else None # a tolerance of zero never stops + nudged_tolerance = None if cfg.nudged_fixed else tolerance + group, neuron_group = learner._projections() + partner = np.full(w.synapses, -1, dtype=np.int64) + partner[learner._paired] = learner._paired_reverse + order = layout._flat_order + m, rho = cfg.momentum, cfg.normalize + rates = np.array([ + m, 1 - m, 1.0, rho, 1 - rho, 1.0, cfg.eta, cfg.eta_bias, cfg.normalize_floor, + cfg.normalize_relative, _RELATIVE_TINY, 1.0, cfg.scale_cap, + ], dtype=float) + if recall is not None: + association = np.ascontiguousarray(recall.association, dtype=np.int64) + (v_syn, v_src, v_dst), (n_syn, n_src, n_dst) = ( + tuple(np.ascontiguousarray(a, dtype=np.int64) for a in part) + for part in recall._head_synapses[0] + ) + else: + association = v_syn = v_src = v_dst = n_syn = n_src = n_dst = _NO_INDEX + count = len(routes) if routes else 0 + if routes: + assert hearing is not None + heard = np.ascontiguousarray(hearing.association, dtype=np.int64) + carried = tuple( + tuple(np.ascontiguousarray(route[name], dtype=np.int64) for route in routes) + for name in ("synapses", "rows", "cols") + ) + spans = np.array([[route["start"], route["stop"]] for route in routes], + dtype=np.int64) + bounds = np.array([route["bound"] for route in routes], dtype=float) + else: + heard = _NO_INDEX + carried = ((_NO_INDEX,), (_NO_INDEX,), (_NO_INDEX,)) + spans = np.zeros((1, 2), dtype=np.int64) + bounds = np.zeros(1) + constants = ( + np.ascontiguousarray(learner.output_index, dtype=np.int64), pairs, apical_pairs, + layout.starts, layout.pair_pre, layout.pair_post, layout.offset, freezable, + freezable_nudged, fused._NO_MASK, np.array(fused._rule(model), dtype=float), + target0, nmask0, gid0, softmax0, anchor0, gain0, nmask, gid, int(ngroups), + float(beta), softmax_t, anchor, anchor_gain, pool_of, pool_index, pool_starts, + pool_run, no_held, int(cfg.free_steps), int(cfg.nudged_steps), + float(tolerance) if tolerance is not None else 0.0, tolerance is not None, + float(nudged_tolerance) if nudged_tolerance is not None else 0.0, + nudged_tolerance is not None, layout.edge_index, 2.0 * cfg.beta, + np.ascontiguousarray(group, dtype=np.int64), int(group.max()) + 1, + np.ascontiguousarray(neuron_group, dtype=np.int64), int(neuron_group.max()) + 1, + partner, learner.plastic_neurons, + _NO_INDEX if order is None or len(order[0]) != layout.size else order[0], + association, v_syn, v_src, v_dst, n_syn, n_src, n_dst, heard, *carried, spans, + bounds, + ) + size = max(count, 1) + plan = ( + constants, rates, np.zeros((size, 2)), np.zeros(size, dtype=np.bool_), + np.zeros(size), np.zeros(5), + ) + # the key's objects are held with the plan, so an identity is never a freed object's + brain.__dict__["_lesson_plan"] = (key, 0 if routes is None else len(routes), plan) + return plan + + +# The report of a taught lesson's phases, in the order ``Learner.step`` writes it +# (``learning._phase_report``, then the rows' counts). +_PHASE_REPORT = tuple( + tuple( + f"{name}_{entry}" + for entry in ("steps", "residual", "residual_checks", "damping_halvings", + "stagnation_checks", "row_sweeps", "row_residual_checks") + ) + for name in ("free", "nudged", "opposite") +) + + +def taught_lesson( + brain: Any, drive: np.ndarray, state: Any, labels: np.ndarray, differs: np.ndarray +) -> dict[str, float] | None: + """A taught lesson of the compiled moment in one compiled call (``_taught_kernel``), after + its judgment: the rows of ``drive`` whose answer ``differs`` from their ``labels`` settle + their phases from the stream's ``state``, the update steps the efficacies and the biases, + the address lesson steps the read's maps and the carried gains take their step, and what + the call returns is stored. The same parameters, moments, maps, gains and report to the bit + as ``Learner.step``, ``Recall.learn`` and ``Hearing.learn`` in turn; None where the brain, + the learner or the lesson takes those (nothing has changed then): another learner or + layout, synapses that are frozen, tied in groups or stepped at their own rates, a carried + route off the host, a lesson the update refuses.""" + + plan = _lesson_plan(brain) + learner = brain.learner + if plan is None or learner.synapse_rate is not None or not learner.plastic_synapses.all(): + return None + constants, rates, scales, live, gains, stepping = plan + graph, cfg = learner.brain, learner.config + recall, hearing = brain.recall, brain.hearing + batch = len(drive) + flat = graph._fused_blocks()[0] + if graph._efficacy is None or graph._torch is not None or graph._mlx is not None: + return None + moments = ( + learner.velocity, learner.second_moment, learner.velocity_bias, + learner.second_moment_bias, + ) + for array in moments: + if type(array) is not np.ndarray or not array.flags.c_contiguous: + return None + if not labels.flags.c_contiguous: + labels = np.ascontiguousarray(labels) + kept_in, has_kept = fused._NO_PRODUCTS + has_given = False + if state is None: + warm_v = warm_a = _NO_ROWS + else: + warm_v, warm_a = state.v, state.adaptation + if differs.all(): # every row is taught: the phases start from the state itself + products = state.__dict__.get("_products") + kept_in, has_kept = fused._held(products) + has_given = fused._given(products, flat, flat, batch)[1] + # the address lesson (``Recall.learn``): the moment the lesson's drive addressed + moment: Any = None if recall is None else recall.last + positions = False + addressed = False + if moment is not None: + positions = bool(recall.position_maps) and (recall.position_rate > 0 or recall.slope > 0) + addressed = bool( + (recall.address or positions) and recall.slots + and recall.streams == len(moment.address) == batch + ) + read: tuple[Any, ...] = _NO_READ + columns = None + if addressed: + keys, first = _behind(recall, "keys") + values, at_values = _behind(recall, "values") + nexts, at_nexts = _behind(recall, "nexts") if recall.next else (values, at_values) + held, at_held = _behind(recall, "held") + if not first == at_values == at_nexts == at_held: + keys, values, held = ( + np.ascontiguousarray(a) for a in (recall.keys, recall.values, recall.held) + ) + nexts = np.ascontiguousarray(recall.nexts) if recall.next else values + first = 0 + w_x, w_c, w, columns = packed_maps(recall) + numbers, scalars, table, grid = _settings(recall) + read = ( + True, keys, values, nexts, held, first, recall.slots, bool(recall.next), + moment.inputs, moment.cortex, moment.address, moment.key, w_x, w_c, w, columns, + numbers, scalars, recall.told, recall.focus, table, grid, positions, + ) + # the carried routes (``Hearing.learn``): what each holds at this batch, and its gain + routes = hearing.routes if hearing is not None else () + carried: tuple[Any, ...] = (_NO_TRACE,) + heard_second = gains + if routes: + traces = [] + for k, route in enumerate(routes): + memory = route["memory"] + trace = memory.trace + if ( + type(trace) is not np.ndarray or trace.dtype != np.float64 or trace.ndim != 2 + or not trace.flags.c_contiguous or not trace.flags.writeable + ): + return None + traces.append(trace) + amplitude = float(memory.amplitude) + gain = hearing.gain(route) + gains[k] = gain + scales[k, 0] = amplitude * gain + scales[k, 1] = amplitude + live[k] = len(trace) == batch + carried = tuple(traces) + heard_second = hearing.second + if type(heard_second) is not np.ndarray or not heard_second.flags.c_contiguous: + return None + rho = hearing.RHO + stepping[0], stepping[1] = rho, 1.0 - rho + stepping[2] = 1.0 - rho ** (hearing.count + 1) + stepping[3], stepping[4] = hearing.rate, hearing.FLOOR + m, rho = cfg.momentum, cfg.normalize + count = learner.contrast_updates + 1 # only this optimizer's own history + rates[2] = 1.0 - m**count if m > 0 else 1.0 + rates[5] = 1.0 - rho**count if rho > 0 else 1.0 + rates[11] = learner._share(differs) + kernel: Any = _taught_kernel # its arguments are the lesson's, the read's and the plan's + out = kernel( + differs, labels, drive, warm_v, warm_a, state is not None, kept_in, has_kept, has_given, + graph.bias, flat, graph._efficacy, graph._gain_pre, *moments, rates, *read, + len(routes), carried, live, scales, heard_second, gains, stepping, *constants, + ) + if out[0] != 0: + return None + (_, taught, taken0, taken1, taken2, peak0, peak1, peak2, velocity, second, velocity_bias, + second_bias, delta, delta_bias, scale, weights, bias, new_flat, moved, error, new_x, + new_c, new, address_ok, position_ok, contrasts, new_heard_second, new_gains) = out[:28] + # the update (``Learner._update``): the moments, then the graph of the new parameters + if m > 0: + learner.velocity_bias = velocity_bias + learner.velocity = velocity + if rho > 0: + learner.second_moment = second + learner.second_moment_bias = second_bias + updated = graph._with_update(scale, weights, bias, (True, True)) + updated._flat = new_flat + learner.brain = updated + learner.updates += 1 + learner.contrast_updates += 1 + rows = float(taught) + report = { + "scale_step": float(np.abs(delta).mean()), + "bias_step": float(np.abs(delta_bias).mean()), + "accepted": 1.0, "qualified": 0.0, "qualification_required": 0.0, + "attempted_presentations": rows, "accepted_presentations": rows, + } + steps = taken0 + taken1 + taken2 + for names, taken, peak in zip(_PHASE_REPORT, (taken0, taken1, taken2), + (peak0, peak1, peak2), strict=True): + report[names[0]] = float(taken) + report[names[1]] = peak + report[names[2]] = 1.0 + report[names[3]] = 0.0 + report[names[4]] = 0.0 + report[names[5]] = float(taught * taken) + report[names[6]] = rows + report["total_steps"] = float(steps) + report["total_residual_checks"] = 3.0 + report["total_stagnation_checks"] = 0.0 + report["total_row_sweeps"] = float(taught * steps) + report["total_row_residual_checks"] = float(3 * taught) + report["attempted_presentations"] = float(batch) + report["accepted_presentations"] = rows + report["agreed_presentations"] = float(batch) - rows + exhausted = taken0 >= cfg.free_steps > 0 + report["free_budget_exhausted"] = float(exhausted) + learner._warn_exhausted(exhausted) + # the address lesson's maps (``Recall.learn``) + if recall is not None: + quiet = {"address_rows": 0.0, "address_error": 0.0, "address_refused": 0.0} + if recall.position_maps: + quiet.update(position_rows=0.0, position_refused=0.0) + if addressed: + parts = [out[k:k + 3] for k in range(31, 49, 3)] + report.update(recall._learned_compiled( + int(moved), error, new_x, new_c, new, address_ok, position_ok, positions, + quiet, parts, + )) + assert columns is not None + if ( + moved and recall.address and address_ok and ( + columns[6] == columns[1] or (positions and position_ok and not ( + recall.slope_maps and recall.grid_maps and recall.slots > recall.position + )) + ) + ): + # every part of the packed maps was stepped and taken: the next read's packed + # maps are the kernel's, the arrays ``packed_maps`` would make from the parts + recall.__dict__["_packed"] = ( + None, (new_x, new_c, new, columns), packed_parts(recall), out[28:31], + ) + else: + report.update(quiet) + # the carried gains' step (``Hearing.learn``) + if len(routes) == 1: + hearing.step(contrasts.mean(axis=0)) + elif routes: + hearing.count += 1 + hearing.second = new_heard_second + for route, gain in zip(routes, new_gains.tolist(), strict=True): + hearing.set(route, gain) + return report + + +_NO_READ: tuple[Any, ...] = ( + False, _NO_STORE, _NO_STORE, _NO_STORE, _NO_HELD, 0, 0, False, _NO_TRACE, _NO_TRACE, + _NO_TRACE, _NO_TRACE, _NO_TRACE, _NO_TRACE, np.zeros(1), np.zeros(7, dtype=np.int64), + np.zeros(8, dtype=np.int64), np.zeros(8), _NO_INDEX, _NO_INDEX, np.zeros(1, dtype=np.int64), + np.zeros(0, dtype=np.int64), False, +) diff --git a/src/cadence/brain.py b/src/cadence/brain.py index 5d3c896a..8ba98978 100644 --- a/src/cadence/brain.py +++ b/src/cadence/brain.py @@ -880,14 +880,17 @@ def calibrate_transport( raise ValueError("a transport plan belongs to the torch backend") return cast("dict[str, Any]", self._torch.calibrate(batches, name)) - def _with_device_parameters(self, scale: Any, bias: Any) -> Brain: - """The brain with new parameters that stay on the torch device. + def _with_device_parameters( + self, scale: Any, bias: Any, checked: tuple[Any, ...] | None = None + ) -> Brain: + """The brain with new parameters that stay on the torch device; ``checked`` is their + ``parameter_check``, read finite by the caller. The kernel is shared and updated in place, so a state settled on it continues on it; the brain this was called on is superseded (its host copies, if any, are stale). """ assert self._torch is not None - self._torch.set_parameters(scale, bias) + self._torch.set_parameters(scale, bias, checked=checked) new = copy.copy(self) new._efficacy = None new._bias = None @@ -1086,6 +1089,9 @@ def settle( _keep_products(single, out.__dict__.get("_products")) # the same row return single + # a subclass whose numerical copies settle on the kernel of their step, as this class's do + _steps_on_kernel = False + def _with_dt(self, dt: float) -> Brain: """A numerical copy on current parameters; boundary subclasses must override. @@ -1096,7 +1102,7 @@ def _with_dt(self, dt: float) -> Brain: """ if ( self._torch is not None - and type(self) is Brain + and (type(self) is Brain or self._steps_on_kernel) and "settle_batch" not in self.__dict__ and "residual" not in self.__dict__ ): @@ -1300,6 +1306,54 @@ def _equilibrate_local( stagnation_checks=stagnation_checks, ) + def _published(self, drive: np.ndarray, state: BrainState | None) -> BrainState: + """``settle_batch(drive, steps=0, state=state)`` on the compiled CPU path of a graph + that solves in one compiled call (``_solves_compiled``), without the checks of a mask, a + nudge, a trajectory or a device: the same arrays and refusals of the drive and the + state.""" + from .fused import fused_settle + + drive = np.asarray(drive, float) + if drive.ndim == 1: + drive = drive[None, :] + batch, n = drive.shape + if n != self.connectome.n: + raise ValueError("drive must have one column per neuron") + if n == 0 or not _rows_finite(drive): + raise ValueError("drive must be finite and the brain must contain neurons") + if state is None: + v, a = np.zeros((batch, n)), np.zeros((batch, n)) + else: + v = np.array(state.v, dtype=np.float64, copy=True, ndmin=2) + a = np.array(state.adaptation, dtype=np.float64, copy=True, ndmin=2) + if v.shape != (batch, n) or a.shape != (batch, n): + raise ValueError("state batch does not match the drive batch") + if not (_rows_finite(v) and _rows_finite(a)): + raise ValueError("warm potentials and adaptation must be finite") + compiled = self._fused_blocks() + assert compiled is not None + s, taken, change, products = fused_settle( + v, a, drive, self.bias, self.layout, compiled[0], None, self.neuron_model, None, 0, + None, apical=compiled[1], + products=None if state is None else state.__dict__.get("_products"), + ) + result = BrainState(v=v, activation=s, adaptation=a, steps=taken, activity_change=change) + _keep_products(result, products) + return result + + def _fast_equilibrate( + self, drive: np.ndarray, state: BrainState | None, budget: int, tolerance: float + ) -> Equilibrium | None: + """``equilibrate(drive, state=state, budget=budget, tolerance=tolerance, fast=True)`` in + the calls its compiled path makes (``_published``, ``_solve_compiled``): the same + arrays to the bit; None where it takes another path (the caller then calls it).""" + if self.backend != "cpu" or not self._solves_compiled() or budget < 0: + return None + if _resident(drive): + return None + return self._solve_compiled(drive, self._published(drive, state), budget, 32, + tolerance, True) + def _solves_compiled(self) -> bool: """Whether a free solve can run in one compiled call: no compartment, no recording, on the compiled CPU path. A wrapped ``settle_batch`` or ``residual`` sees every launch, and @@ -3329,23 +3383,44 @@ def __deepcopy__(self, memo: dict[int, Any]) -> _TorchKernel: # -- parameters - def set_parameters(self, scale: Any, bias: Any) -> None: - """New parameters, as device tensors: the transport weights follow, in place.""" + def parameter_check(self, scale: Any, bias: Any) -> tuple[Any, ...]: + """What ``set_parameters`` checks before it writes, not read: the transport's weights, the + bias in the kernel's precision, the plan's summed values (None for a plan without them) + and two device flags, the parameters and weights finite, and the summed values finite.""" torch = self.torch if tuple(scale.shape) != tuple(self.gain_pre.shape) or tuple(bias.shape) != (self.n,): raise ValueError("device parameters must match synapse and neuron shapes") weights = (self.gain_pre * scale).to(self.dtype) live_bias = bias.to(self.dtype) values = self.plan.scatter(weights) if self.plan.parallel else None - checked = [scale, bias, weights, live_bias] + ([values] if self.plan.parallel else []) - if not bool(torch.stack([torch.isfinite(value).all() for value in checked]).all()): - if self.plan.parallel and bool( - torch.stack([torch.isfinite(v).all() for v in checked[:4]]).all() - ): - raise ValueError("summed effective synaptic weights must be finite") - raise ValueError( - "device parameters and effective weights must be finite at runtime precision" - ) + finite = torch.stack( + [torch.isfinite(value).all() for value in (scale, bias, weights, live_bias)] + ).all() + summed = torch.isfinite(values).all() if values is not None else finite + return weights, live_bias, values, finite, summed + + @staticmethod + def parameter_refusal(finite: bool, summed: bool) -> None: + """The refusal of a ``parameter_check`` whose flags read ``finite`` and ``summed``.""" + if finite and summed: + return + if finite: + raise ValueError("summed effective synaptic weights must be finite") + raise ValueError( + "device parameters and effective weights must be finite at runtime precision" + ) + + def set_parameters( + self, scale: Any, bias: Any, *, checked: tuple[Any, ...] | None = None + ) -> None: + """New parameters, as device tensors: the transport weights follow, in place. ``checked`` + is the ``parameter_check`` of these parameters whose flags the caller read finite.""" + torch = self.torch + if checked is None: + checked = self.parameter_check(scale, bias) + flags = torch.stack([checked[3], checked[4]]).cpu().numpy() + self.parameter_refusal(bool(flags[0]), bool(flags[1])) + weights, live_bias, values = checked[:3] # Validation precedes every write to the shared live kernel and its weights. self.scale = scale self.bias_param = bias @@ -5907,6 +5982,7 @@ def rewarded( self, plus: Mapping[str, Any], minus: Mapping[str, Any], trace: Sequence[Any] | None, trace_bias: Any, decay: np.ndarray, span: float, delta: np.ndarray, eta: float, eta_bias: float, scale: Sequence[Any], tables: dict[str, Any], cap: float, + also: Sequence[Any] = (), before: Any = None, ) -> tuple[_MlxKernel, Any, float, float]: """A reward's step, all of it on the device: the dopamine ``delta``'s mean over the rows of each row's eligibility after this moment's contrast (``decay`` its own factor), at ``eta`` @@ -5914,7 +5990,9 @@ def rewarded( synapses' eligibility itself moves after it, with nothing left to refuse (``eligible``): a row's contrast is an outer product, so the step needs no array of them. Returns the new kernel, the neurons' eligibility and the mean steps' sizes; a value that is not finite - raises ``ValueError`` with nothing changed.""" + raises ``ValueError`` with nothing changed. ``also`` is made in the same launch and + ``before()`` runs after it, before this step's own check, for the checks a caller reads + of what ``also`` made (the critic's and the memory's, refused first).""" mx, lay = self.mx, self.layout rows = len(delta) plus_of, minus_of = self._ranges_of(plus), self._ranges_of(minus) @@ -5954,8 +6032,10 @@ def rewarded( ok = mx.isfinite(total) & mx.isfinite(mx.sum(bias_step)) mx.eval( new.blocks, new.scale, total, ok, new_bias_trace, new.bias, bias_size, - new.w if plan else [], + new.w if plan else [], *also, ) + if before is not None: + before() if not bool(ok.item()): raise ValueError("steps must be finite vectors, one per synapse and one per neuron") return ( diff --git a/src/cadence/fused.py b/src/cadence/fused.py index 1d806b71..5b28f707 100644 --- a/src/cadence/fused.py +++ b/src/cadence/fused.py @@ -2026,7 +2026,6 @@ def fused_settle( """ assert njit is not None batch, n = v.shape - standing = _standing(np.ascontiguousarray(drive), bias) if keep is None: masked = False keep = _NO_MASK @@ -2043,6 +2042,7 @@ def fused_settle( if apical is not None: # a held gain's shape is checked at no sweeps too _compartment(apical, layout, flat, nudge, batch, n) return s, 0, np.zeros(batch), products + standing = _standing(np.ascontiguousarray(drive), bias) has_adapt = neuron_model.adaptation is not None target, nmask, gid, ngroups, beta, softmax_t, weight, anchor, anchor_gain = _nudge_arrays( nudge, batch, n diff --git a/src/cadence/generic.py b/src/cadence/generic.py index 55f97945..c0f9c318 100644 --- a/src/cadence/generic.py +++ b/src/cadence/generic.py @@ -50,6 +50,7 @@ import numpy as np +from . import _moment from ._configuration import ( CONSTANTS, POINT, @@ -74,6 +75,7 @@ from .connectome import Connectome from .genome import Genome, Projection, develop from .learning import ( + LearnedState, Learner, LearnerConfig, _reservoir_cut, @@ -90,6 +92,8 @@ _Candidate, _fastest_backends, _memory_on_device, + _Moment, + _on_host, arrays_of, ) from .stream import ( @@ -112,6 +116,11 @@ # A brain with maturity (#253) saves under its own format name, which 0.93.1 and earlier refuse; # every other brain keeps BRAIN_FORMAT. MATURITY_FORMAT = "cadence-brain/2" +# A brain whose store fades (``memory_fade``, #253) saves under a format name of its own, which +# 0.95.2 and earlier refuse, with or without maturity: the highest name a file needs is its one +# name, and a ``cadence-brain/3`` file with a maturity carries it as a ``cadence-brain/2`` file +# does. +FADE_FORMAT = "cadence-brain/3" # A memory's group in BRAIN_FORMAT and the prefix the reader validates its arrays under; the # episodic memory's arrays are the context store's and the stream's (``_flat_layout``). _MEMORY_PREFIXES = { @@ -196,10 +205,16 @@ def _operation(method: _Operation) -> _Operation: def operated(self: Brain, *args: Any, **kwargs: Any) -> Any: if self._unplaced: self._place_for(args[0] if args else kwargs.get("observations")) - if self.recall is None: + recall = self.recall + if recall is None: return method(self, *args, **kwargs) - with self.recall._scope(): + token = recall._enter() + if token is None: # within an operation in progress, which this one shares return method(self, *args, **kwargs) + try: + return method(self, *args, **kwargs) + finally: + recall._leave(token) return cast(_Operation, operated) @@ -213,7 +228,9 @@ def lived(self: Brain, *args: Any, **kwargs: Any) -> Any: if self._unplaced: # before anything of this brain is held across the moment self._place_for(args[0] if args else kwargs.get("observations")) maturity = self.basal_ganglia.maturity - if maturity is None: + store = self.hippocampus + fading = isinstance(store, SynapticMemory) and bool(store.fade) + if maturity is None and not fading: return method(self, *args, **kwargs) self._life_depth += 1 try: @@ -221,7 +238,15 @@ def lived(self: Brain, *args: Any, **kwargs: Any) -> Any: finally: self._life_depth -= 1 if self._life_depth == 0: - maturity.relax() + if maturity is not None: + maturity.relax() + if fading: # the moment's place key cues the records it uses (``memory_fade``) + assert isinstance(store, SynapticMemory) + if method.__name__ == "learn": # ``learn(reward, done, next_observations)`` + x = args[2] if len(args) > 2 else kwargs.get("next_observations") + else: + x = args[0] if args else kwargs.get("observations") + store.fade_moment(None if x is None else self._fade_key(x)) return result return cast(_Operation, lived) @@ -410,62 +435,6 @@ def _recall_genes(recall: Any, genes: dict[str, Any]) -> dict[str, Any]: return {**founders, **Recall.options(), **settings} -def _copy_identity( - connectome: Connectome, recalled: Mapping[str, Any], offset: int -) -> Connectome: - """The connectome whose copy pathway starts from the declared tokens (``recall_tokens``). - - For motor neuron ``j`` of token ``tokens[j]``: the synapse from every head's ``recall`` - port at value coordinate ``offset + tokens[j]`` (the observation's own coordinates in the - value) starts at ``recall_copy``, so the token the read delivers drives its own action; and - the synapse from sensory neuron ``tokens[j]`` starts at ``-recall_copy``, so a token the - present observation holds, the read's echo of the moment that addressed it, is not - copied back. Every other synapse into the motor region from a read port, or from the sensory - region, starts at zero. The same connectome when it already starts so (a saved brain). - Every one of these synapses learns on its own, as every synapse does.""" - populations = connectome.populations - n = connectome.n - value = np.full(n, -1, dtype=np.int64) # each port neuron's value coordinate - names = [name for port in Recall.ports(recalled["heads"], recalled["next"]) for name in port] - for name in names: - neurons = np.asarray(populations[name], dtype=np.int64) - value[neurons] = np.arange(len(neurons)) - if recalled["action"]: # the command's part: no token, starts at zero - value[np.asarray(populations[f"{name}_action"], dtype=np.int64)] = -2 - sensory = np.asarray(populations["sensory"], dtype=np.int64) - heard = np.full(n, -1, dtype=np.int64) - heard[sensory] = np.arange(len(sensory)) - motor = np.asarray(populations["motor"], dtype=np.int64) - local_motor = np.full(n, -1, dtype=np.int64) - local_motor[motor] = np.arange(len(motor)) - tokens = np.asarray(recalled["tokens"], dtype=np.int64) - pre, post = np.asarray(connectome.pre), np.asarray(connectome.post) - token = np.where(local_motor[post] >= 0, tokens[np.maximum(local_motor[post], 0)], -1) - pathway = (value[pre] != -1) & (local_motor[post] >= 0) - identity = pathway & (token >= 0) & (value[pre] == offset + token) - echoes = (heard[pre] >= 0) & (token >= 0) & (heard[pre] == token) - declared = int((tokens >= 0).sum()) - if int(identity.sum()) != declared * len(names) or int(echoes.sum()) != declared: - raise ValueError( - "recall_tokens need a synapse from every read port's token coordinate and from the " - "token's sensory neuron into its motor neuron; Brain.compose adds them" - ) - reading = np.isin(pre, np.concatenate([ - np.asarray(populations[name], dtype=np.int64) for name in names - if name.startswith("recall") - ])) - copy = float(recalled["copy"]) - sign = np.where(pathway | ((heard[pre] >= 0) & (local_motor[post] >= 0)), 0.0, connectome.sign) - sign[identity & reading] = copy - sign[echoes] = -copy - if np.array_equal(sign, connectome.sign): - return connectome - return Connectome( - n, connectome.pre, connectome.post, connectome.count, sign, - dict(populations), connectome.label, - ) - - def _learned_code_units(value: Any) -> int: """The units of the brain's learned code: ``learned_code`` as given, 0 or None for none.""" if value is None: @@ -611,7 +580,9 @@ def _restore_traces(held: list[tuple[Trace | History, dict[str, np.ndarray]]]) - # a known outcome moves a record halfway: remembered at once, revised by repetition "memory_rate": 0.5, } -_EPISODIC_SETTINGS = ("memory_choice", "memory_context", "memory_extinction", "memory_context_hold") +_EPISODIC_SETTINGS = ( + "memory_choice", "memory_context", "memory_extinction", "memory_context_hold", "memory_fade", +) # The constants of the stores the brain builds, the founders every reading used (``RETIRED``): # the share of a stream's record each later write keeps (``memory_decay``) and the cells of the # episodic separation and those active per key (``memory_expansion``, ``memory_winners``). A @@ -947,6 +918,18 @@ def array(name: str, shape: tuple[int, ...] | None = None) -> np.ndarray: memory.gate_noise = noise.astype(float) elif prefix + "gate_noise" in data: raise ValueError("saved noise scales require the surprise gate's metadata") + fade = real("saved memory fade", metadata.get("fade", 0.0), low=0, high=1) + if fade: # a fading store's depths and recent traces (``memory_fade``) + if fade >= 1: + raise ValueError("saved memory fade must lie in [0, 1)") + memory.fade = fade + depth = array("fade_depth", (memory.key_width,)) + recent = array("fade_recent", (memory.key_width,)) + if (depth < 0).any() or (recent < 0).any(): + raise ValueError("saved fade depths and traces must be nonnegative") + memory.fade_depth, memory.fade_recent = depth.astype(float), recent.astype(float) + elif prefix + "fade_depth" in data or prefix + "fade_recent" in data: + raise ValueError("saved fade depths require the fade's metadata") if ( metadata.get("persistent_parameters", memory.consolidated.size) != memory.consolidated.size @@ -1270,7 +1253,7 @@ def _validate_recall_state( names = {*Recall.RUNTIME, *Recall.CONSTRUCTION} # a retired setting is saved where the format of its release wrote it: like the options, # only where it differed from its founder - optional = {"inputs", "hearing", *Recall.OPTIONS, *Recall.RETIRED} + optional = {"inputs", "hearing", "gain_lessons", *Recall.OPTIONS, *Recall.RETIRED} if ( not isinstance(recalled, dict) or set(recalled) - optional @@ -1299,6 +1282,9 @@ def _validate_recall_state( writes = recalled["writes"] if isinstance(writes, bool) or not isinstance(writes, int) or writes < 0: raise ValueError("invalid saved recall write count") + seen = recalled.get("gain_lessons", 0) + if isinstance(seen, bool) or not isinstance(seen, int) or seen < 0: + raise ValueError("invalid saved recall gain lesson count") inputs = recalled.get("inputs", sensory) if isinstance(inputs, bool) or not isinstance(inputs, int) or inputs != width: raise ValueError( @@ -1454,6 +1440,14 @@ def _actor_config(values: Any) -> ActorCriticConfig: raise ValueError("invalid saved actor config") from exc +def _saved_fade(saved: Mapping[str, Any]) -> bool: + """Whether a checkpoint's metadata names a fading store (``memory_fade`` above zero): the + brains that save ``FADE_FORMAT``. The store's reader validates the value itself.""" + memories = saved.get("memories") + store = memories.get("hippocampus") if isinstance(memories, dict) else None + return isinstance(store, dict) and bool(store.get("fade")) + + def _flat_layout( saved: dict[str, Any], data: Mapping[str, Any] ) -> tuple[dict[str, Any], dict[str, Any]]: @@ -1859,29 +1853,9 @@ def size(value: Any) -> int: projections.append( Projection(f"{name}_action", "association", scale=0.0, reciprocal=False) ) - if recalled["copy"]: - # the copy pathway from every read port into the motor region, appended last: the - # brain with it is the brain without it plus these synapses, drawn at the initial - # magnitude ``recall_copy`` (the command's part at zero); declared tokens start it - # as their identity once developed (``_copy_identity``) - for name in ports: - projections.append( - Projection(name, "motor", scale=recalled["copy"], reciprocal=False) - ) - if recalled["action"]: - for name in ports: - projections.append( - Projection(f"{name}_action", "motor", scale=0.0, reciprocal=False) - ) - if recalled["tokens"] is not None: - # the present observation's tokens into their actions, the copy's echo - # suppression, drawn at zero and started at ``-recall_copy`` once developed - projections.append( - Projection(regions[0].name, "motor", scale=0.0, reciprocal=False) - ) if recalled["when"]: # every head's read of when, two neurons of the age and with the anchor two of the - # place, appended after everything else (the copy pathway's draws included): the + # place, appended after everything else: the # brain that delivers when is the brain without it plus these regions, whose # synapses into the association start at zero for name in Recall.when_regions(recalled["heads"]): @@ -1930,8 +1904,8 @@ class Brain: and, with ``recall=True``, ``recall`` and ``next`` (one neuron per value coordinate each) for the index memory of the stream's moments, and with ``recall_heads=k`` also ``recall_h`` and ``next_h`` for each further head (with ``recall_action`` each of them - with a ``_action`` region of one neuron per motor neuron, and with - ``recall_copy`` synapses from them into ``motor``); ``Genome.composed`` lays one out. Without + with a ``_action`` region of one neuron per motor neuron); ``Genome.composed`` lays + one out. Without ``recall=True`` populations of those names are ordinary ones. ``Brain.compose`` builds the default brain with working memory, and with ``connectome`` gives a custom anatomy its founders. @@ -1986,6 +1960,7 @@ def __init__( memory_context: float = 0.0, memory_extinction: float = 0.0, memory_context_hold: str | None = None, + memory_fade: float = 0.0, maturity: bool = False, maturity_scale: float = _MATURITY_FOUNDERS["maturity_scale"], maturity_recovery: float = _MATURITY_FOUNDERS["maturity_recovery"], @@ -2073,10 +2048,8 @@ def __init__( # than on the observation itself self._coded_keys = bool(units) or bool(learned) recalled = _recall_genes(recall, genes) # the recall_* genes leave ``genes`` here - # declared tokens start the copy pathway as their identity, the echo included, and the - # gains read them (``recall_gain``) - tokened = bool(recalled.get("copy")) and recalled.get("tokens") is not None - if recalled.get("tokens") is not None and (tokened or recalled.get("gain")): + # the gains read the declared tokens (``recall_gain``) + if recalled.get("tokens") is not None and recalled.get("gain"): # the observation's own coordinates in the recall values: the observation itself, # or a learned code joined to it, after the code's units joined = bool(learned) and LearnedCode.joins(self.inputs, learned["units"]) @@ -2097,17 +2070,6 @@ def __init__( raise ValueError( f"recall_tokens name observation coordinates below {self.inputs}" ) - ports = [ - name for port in Recall.ports(recalled["heads"], recalled["next"]) for name in port - ] - if recalled["action"]: - ports += [f"{name}_action" for name in ports] - # the pathway's identity needs its synapses (else the memory names them) - if tokened and all(name in populations for name in (*ports, "motor")): - connectome = _copy_identity( - connectome, recalled, learned["units"] if learned else 0 - ) - self.connectome = connectome learning_genes, actor_genes, arousal_genes = split_genes(genes) learning = configured(_learning() if learning is None else learning, learning_genes, "learning") @@ -2142,6 +2104,11 @@ def __init__( raise ValueError("memory_choice needs the episodic memory") if not memory_choice and (memory_context or memory_extinction): raise ValueError("memory_context and memory_extinction need memory_choice") + memory_fade = real("memory_fade", memory_fade, low=0, high=1) + if memory_fade >= 1: + raise ValueError("memory_fade must lie in [0, 1)") + if memory_fade and not episodic: + raise ValueError("memory_fade needs the episodic memory") working_memory_decay = real("working_memory_decay", working_memory_decay, low=0) efference_decay = real("efference_decay", efference_decay, low=0) if working_memory_decay >= 1 or efference_decay >= 1: @@ -2197,15 +2164,7 @@ def __init__( # the first module reads the code alone: the sensory neurons project to the code # region and nowhere else pre, post = np.asarray(connectome.pre), np.asarray(connectome.post) - # the copy pathway's echo suppression (``recall_tokens``) is the one exception: - # the present observation's tokens reach their own actions - echo = ( - np.isin(post, self.motor_index) if tokened - else np.zeros(len(post), dtype=bool) - ) - if ( - np.isin(pre, self.sensory_index) & ~np.isin(post, self.code_index) & ~echo - ).any(): + if (np.isin(pre, self.sensory_index) & ~np.isin(post, self.code_index)).any(): raise ValueError( "a learned code's sensory neurons project to the code region alone" ) @@ -2318,6 +2277,10 @@ def store(keys: np.ndarray, offset: int) -> SynapticMemory: coded = self._coded_keys places = np.arange(self.code_width) if coded else self.sensory_index self.hippocampus = records = store(places, 0) + if memory_fade: # records nobody uses fade, slower the deeper their use (#253) + records.fade = memory_fade + records.fade_depth = np.zeros(records.key_width) + records.fade_recent = np.zeros(records.key_width) if maturity: # surprise-gated writes engaged by the same maturity, noise from zero records.gate_noise = np.zeros(records.consolidated.shape) records.maturity = self.basal_ganglia.maturity @@ -3251,6 +3214,8 @@ def moved(prefix: str, name: str, value: Any) -> bool: genes.update({f"{prefix}_{field}": getattr(component, field) for field in fields}) if isinstance(self.hippocampus, SynapticMemory): genes["consolidation"] = self.hippocampus.consolidation + if self.hippocampus.fade: + genes["memory_fade"] = self.hippocampus.fade maturity = self.basal_ganglia.maturity if maturity is not None: genes.update(maturity=True, maturity_scale=maturity.scale, @@ -3573,9 +3538,112 @@ def _moment_drive(self, x: np.ndarray) -> np.ndarray: raise ValueError("drive must be finite and the brain must contain neurons") return cast(np.ndarray, kernel.observed(drive)) + def _compiled_moment(self) -> bool: + """Whether the moment's work around the solve runs compiled (``learning_compiled_moment``): + the option, on the CPU backend.""" + return bool(self.learner.config.compiled_moment) and self.brain.backend == "cpu" + + def _compiled_drive(self, x: np.ndarray, code: np.ndarray) -> np.ndarray | None: + """The drive of the compiled moment (``learning_compiled_moment``) on the host, written + in one array: the sensory part, the working trace, the recall memory's compiled read and + the place store's, each as ``memories.stimulus`` reads them in turn; None where the brain + holds another memory or a part the compiled read does not take (``_stimulus`` then reads + them in turn). A read of the same observations from the same memories as the latest read + (a lesson's, and the act that follows a lesson that taught nothing) is that read's drive, + and the memories hold that read.""" + recall, trace, store = self.recall, self.working_memory, self.hippocampus + if recall is not None: + recall.compiled_moment = True + if ( + self.learned_code is not None or recall is None or type(recall) is not Recall + or recall._device is not None or not recall._compiles(len(x)) + or self.efference is not None or self.history is not None or self.memory_choice + or (trace is not None and (type(trace) is not Trace or trace._device is not None)) + or (store is not None and ( + type(store) is not SynapticMemory or store._device is not None + or store.separator is not None or self.memory_choice or self._coded_keys + )) + ): + return None + if self.hearing is not None: + self.hearing.compiled_moment = True + if store is not None: + store.compiled_moment = True + self.basal_ganglia.reside(None) + key = self._read_key(x, code, recall, trace, store) if _moment.SHORTCUTS else None + held = self.__dict__.get("_read_held") + if ( + key is not None and held is not None and held[0] == key and recall.last is held[2] + and recall._attended is held[3] and recall.trace is held[4] + ): + return cast(np.ndarray, held[1].copy()) + spans = ( + _moment.drive_spans(self) + if _moment.SHORTCUTS and code is x and not self.coded_cortex else None + ) + if spans is not None: # every part in one pass, as the parts below read in turn + drive = _moment.moment_drive(self, np.ascontiguousarray(x), spans) + else: + drive = np.zeros((len(x), self.connectome.n)) + drive[:, self._sensory_columns] = ( + self._sensed(x, code) * self.brain.neuron_model.stimulus_amplitude + ) + if trace is not None: + drive[:, trace._glow_columns] = ( + trace.amplitude * trace.hearing * trace.trace if len(trace.trace) == len(x) + else 0.0 + ) + recall._stimulate_compiled(drive, x, inplace=True) + if store is not None: + drive[:, store._post_columns] += _moment.store_read( + store, np.ascontiguousarray(drive[:, store._pre_columns]) + ) + self.__dict__["_read_held"] = ( + key, drive.copy(), recall.last, recall._attended, recall.trace, + self._read_named(recall, trace, store), + ) + return drive + + @property + def _sensory_columns(self) -> slice | np.ndarray: + """The sensory neurons as a slice where they are one range (``stream.columns``).""" + held = self.__dict__.get("_sensory_held") + if held is None or held[0] is not self.sensory_index: + from .stream import columns as contiguous + + held = (self.sensory_index, contiguous(self.sensory_index)) + self.__dict__["_sensory_held"] = held + return held[1] + + def _read_key( + self, x: np.ndarray, code: np.ndarray, recall: Recall, trace: Trace | None, store: Any + ) -> tuple[Any, ...]: + """What a compiled read reads, by value or by the identity of arrays the memories replace + when they change: the observations and their code, the slots, the cortex of the moment + before, the episode and the focus, the maps (``_moment.packed_maps``) and the settings, + the working trace and the place store.""" + return ( + x.tobytes(), code is x or code.tobytes(), id(recall.keys), id(recall.values), + id(recall.nexts), id(recall.held), id(recall.previous), id(recall.told), + id(recall.focus), *map(id, _moment.packed_parts(recall)), recall.hearing, + recall.amplitude, + recall.heard, recall.temperature, recall.recency, recall.drive_gain, + self.brain.neuron_model.stimulus_amplitude, + # a trace's rows begin again in place (``Trace.reset``), so its values are the key + None if trace is None else (trace.trace.tobytes(), trace.hearing, trace.amplitude), + None if store is None else ( + id(store.strength), id(store.mass), id(store.consolidated), store.amplitude, + store.mass.tobytes(), store.writes, + ), + ) + def _stimulus(self, x: np.ndarray, code: np.ndarray, *, memory: bool = True) -> np.ndarray: """The drive of validated observations and their code.""" kernel = self._memories_kernel(len(x)) if memory else None + if memory and kernel is None and self._compiled_moment(): + fused = self._compiled_drive(x, code) + if fused is not None: + return fused drive: Any if kernel is not None: drive = self._resident_drive(kernel, x, code) @@ -3594,6 +3662,11 @@ def _stimulus(self, x: np.ndarray, code: np.ndarray, *, memory: bool = True) -> self.basal_ganglia.reside(kernel) if self.recall is not None: self.recall.place(self.brain._mlx) + self.recall.compiled_moment = self._compiled_moment() + if self.hippocampus is not None: + self.hippocampus.compiled_moment = self._compiled_moment() + if self.hearing is not None: + self.hearing.compiled_moment = self._compiled_moment() # every memory in order; the recall memory's address, before the settle, reads the # stream's slots drive = self.memories.stimulus(drive, self._read_moment(x, code)) @@ -3628,13 +3701,14 @@ def placement(self) -> dict[str, Any]: def _memories_stay_on_host(self) -> bool: """Whether an option keeps the moment's memories on the host at every size: several heads, a window, the read of the command or of when, the gains, tokens, a separator, a surprise - gate, an episodic vote or a learned code.""" + gate, a fade, an episodic vote or a learned code.""" recall, store = self.recall, self.hippocampus return bool( self._episodic is not None or self.learned_code is not None or (recall is not None and not recall.resides) or (store is not None and (store.separator is not None - or getattr(store, "gate_noise", None) is not None)) + or getattr(store, "gate_noise", None) is not None + or bool(getattr(store, "fade", 0.0)))) ) def _backends_at(self, rows: int) -> list[_Candidate]: @@ -3773,10 +3847,14 @@ def _equilibrate( # torch state stays apart, as it is own = type(brain).equilibrate is not NeuralGraph.equilibrate # a subclass's own solve options: dict[str, Any] = {"fast": True} if cfg.fast_settle and not own else {} - phase = brain.equilibrate( - drive, state=state if brain.backend != "torch" else _copy_state(state), - budget=first_budget, tolerance=tolerance, **options, - ) + phase = None + if options and _moment.SHORTCUTS and self._compiled_moment(): + phase = brain._fast_equilibrate(drive, state, first_budget, tolerance) + if phase is None: + phase = brain.equilibrate( + drive, state=state if brain.backend != "torch" else _copy_state(state), + budget=first_budget, tolerance=tolerance, **options, + ) if np.all(phase.qualified) or phase.steps >= budget: return phase if not all( @@ -3826,6 +3904,270 @@ def _read_judged( return cast(BrainState, held[8]) return None + def _settlement( + self, operation: str, residual: np.ndarray, steps: int, checks: int, row_sweeps: int + ) -> dict[str, Any]: + """The report of a fast solve that qualified every row (``last_settlement``).""" + cfg = self.learner.config + rows = tuple(residual.tolist()) + assert cfg.tolerance is not None + return { + "operation": operation, + "qualified": True, + "row_qualified": (True,) * len(rows), + "residual": rows, + "max_residual": max(rows), + "steps": int(steps), + "budget": int(cfg.free_steps), + "tolerance": float(cfg.tolerance), + "residual_checks": int(checks), + "damping_halvings": 0, + "stagnation_checks": 0, + "row_sweeps": int(row_sweeps), + } + + def _one_call(self, rows: int, begun: bool = True) -> bool: + """Whether a greedy act and a lesson's judgment of ``rows`` streams each run as one + compiled call (``_moment.one_call``): the compiled moment's brain as ``_compiled_drive``, + ``_lean_qualified``, ``_moment.free_solve`` and ``_greedy_choice`` take it, each of its + memories on the host. Without ``begun`` the streams may begin at this moment: the recall + memory then holds no slot of this batch, reads silence and is begun by its write.""" + recall, trace, store = self.recall, self.working_memory, self.hippocampus + agent, cfg, graph = self.basal_ganglia, self.learner.config, self.brain + model = graph.neuron_model + if ( + recall is None or type(recall) is not Recall or recall._device is not None + or self.learned_code is not None or self.sparse_code is not None + or self.efference is not None or self.history is not None or self.memory_choice + or self._episodic is not None or self.coded_cortex + or cfg.tolerance is None or cfg.qualified or not cfg.fast_settle or cfg.rehearsal + or self.learner.slot_count != 1 or type(agent).act is not ActorCritic.act + or type(graph).equilibrate is not NeuralGraph.equilibrate + or model.adaptation is not None or model.dt != 1.0 + or (trace is not None and (type(trace) is not Trace or trace._device is not None)) + or (store is not None and ( + type(store) is not SynapticMemory or store._device is not None + or store.separator is not None or self._coded_keys + )) + ): + return False + recall.compiled_moment = True + if not recall._compiles(rows, begun) or not graph._solves_compiled(): + return False + if self.hearing is not None: + self.hearing.compiled_moment = True + if store is not None: + store.compiled_moment = True + agent.reside(None) + return True + + def _unjudged(self, state: BrainState | None) -> bool: + """Before a one-call solve: whether no solve a lesson judged by can be this one, as + ``_read_judged`` finds it, short of comparing the drive. A kept solve of another graph, + setting or warm state is dropped, as ``_read_judged`` drops it; one only its drive + decides stays for the passes in turn to read (False).""" + held = self._judged + if held is None: + return True + graph = self.brain + if ( + held[0] is graph and held[1] is graph._efficacy and held[2] is graph._bias + and held[3] is self.learner.config and held[4] == "act" and held[5] is state + ): + return False + self._judged = None + return True + + def _act_compiled(self, x: np.ndarray) -> np.ndarray | None: + """A greedy act of the compiled moment in one compiled call (``_moment.one_call``): the + read, the free solve, the choice, the working trace's step and the write. The same + action, state, report and memories to the bit as ``_settled`` and ``_greedy_choice`` in + turn; None where the brain, the moment or the solve takes those (nothing of the stream + has changed then). An act on what the latest read read (after a lesson) reads that + read's drive and the lesson's solve, as those do.""" + rows = len(x) + if not self._one_call(rows, begun=False): + return None + recall, trace, agent = self.recall, self.working_memory, self.basal_ganglia + assert recall is not None + spans, after = _moment.drive_spans(self), _moment.after_spans(self) + if spans is None or after is None: + return None + state = self._activity() + held = self.__dict__.get("_read_held") + if held is not None and recall.last is held[2]: + # the latest read may be this one: its drive, and the solve a lesson judged by + key = self._read_key(x, x, recall, trace, self.hippocampus) + if held[0] == key and recall._attended is held[3] and recall.trace is held[4]: + free = self._read_judged(held[1], state, "act") + if free is None: + return None + # the read's own observations, which these equal to the bit (the key): the + # write then takes the read's moment as it is, in the choice's pass + if held[2] is not None: + x = held[2].event + self._coded_moment = (x, x, None, x, None) + return self._greedy_choice(x, held[1], free) + if not self._unjudged(state): + return None + found = _moment.one_call(self, x, state, 0, spans, after) + if found is None or found[3][0] != 0: + return None + c, rooms, flat, out, begun = found + (_, drive, key, q, attention, ports, v, a, s, residual, used, checks, taken, row_sweeps, + kept, action, clear, stepped, source, cortex, focus) = out + if begun: + attention.flags.writeable = False + recall._attended = attention + else: # streams that begin read nothing + recall._attended = None + recall._head_reads = None + if recall.drive_gain: + recall.trace = ports + free = _moment.solved_state(flat, rows, v, a, s, taken, kept) + self._last_settlement = MappingProxyType( + self._settlement("act", residual, used, checks, row_sweeps) + ) + self._coded_moment = (x, x, None, x, None) + agent._free = free + agent._free_brain = self.brain + agent._drive = drive + self._prepared = None + self._awaiting = None + agent._pending = None + agent._pending_memory = None + agent._pending_hearing = None + if not clear: + action = np.argmax(agent.probabilities(free, None, bias=None), axis=-1) + chosen: np.ndarray = np.asarray(action, dtype=np.int64) + if trace is not None: + if len(trace.cold) != rows: # a trace of another batch began at zero + trace.cold = np.ones(rows, dtype=bool) + trace.trace = stepped + trace.last = source + trace.cold[:] = False + if rooms is not None: + _moment.written(recall, rooms, rows, focus, cortex) + else: # the general write: it begins the streams, or makes the room behind the slots + recall.last = _Moment(x, c, q, key, x) if begun else None + recall.witness(x, free, None) + if self._episode_moments: + self._episode_moments, self._episode_told = [], np.zeros(0, dtype=np.int64) + self._moment = None + self._moment_code = None + return chosen + + def _judge_compiled( + self, x: np.ndarray, labels: np.ndarray, whole: bool = False + ) -> tuple[np.ndarray, np.ndarray, np.ndarray | None] | None: + """A surprise lesson's read and judging solve of the compiled moment in one compiled + call (``_moment.one_call``): the drive, the brain's own answer, each row's most + active motor neuron on the settle its act reads, and the rows whose answer differs from + their ``labels`` (None for labels of another kind than the answer's). The same drive, + answer, report and kept solve to the bit as ``_compiled_drive`` and ``_qualified`` in + turn; None where the brain, the moment or the solve takes those (nothing has changed + then). With ``whole`` a lesson that teaches a row runs in one call next + (``_moment.taught_lesson``): it changes what the read and the solve were made from, so + neither is kept for the act that follows, which makes its own.""" + rows = len(x) + recall, trace, store = self.recall, self.working_memory, self.hippocampus + if ( + not self.learner.config.surprise or not self._one_call(rows) + or recall is None or recall.tokens is not None + ): + return None + spans, motor = _moment.drive_spans(self), _moment.motor_spans(self) + if spans is None or motor is None: + return None + state = self._activity() + held = self.__dict__.get("_read_held") + if held is not None and recall.last is held[2]: + return None # the latest read may be this one: the passes in turn read it + if not self._unjudged(state): + return None + found = _moment.one_call(self, x, state, 1, spans, motor) + if found is None or found[3][0] != 0: + return None + c, _, flat, out, _ = found + _, drive, key, q, attention, ports, v, a, s, residual, used, checks, taken = out[:13] + row_sweeps, kept, answer = out[13:16] + recall.last = _Moment(x, c, q, key, x) + attention.flags.writeable = False + recall._attended = attention + recall._head_reads = None + if recall.drive_gain: + recall.trace = ports + report = self._settlement("act", residual, used, checks, row_sweeps) + self._last_settlement = MappingProxyType(report) + self._coded_moment = (x, x, None, x, None) + differs = None + if ( + isinstance(labels, np.ndarray) and labels.dtype == np.int64 + and labels.shape == answer.shape + ): + differs = answer != labels # ``Learner._differs`` of one slot + if whole and differs.any(): + return drive, answer, differs + self.__dict__["_read_held"] = ( + self._read_key(x, x, recall, trace, store), drive.copy(), recall.last, + recall._attended, recall.trace, self._read_named(recall, trace, store), + ) + free = _moment.solved_state(flat, rows, v, a, s, taken, kept) + graph = self.brain + self._judged = ( + graph, graph._efficacy, graph._bias, self.learner.config, "act", state, + drive.tobytes(), + None if state is None else (state.v.tobytes(), state.adaptation.tobytes()), + free, report, + ) + return drive, answer, differs + + @staticmethod + def _read_named(recall: Recall, trace: Trace | None, store: Any) -> Any: + """What ``_read_key`` names by identity, held with the key so that an identity is never + a freed object's.""" + return ( + recall.keys, recall.values, recall.nexts, recall.held, recall.previous, recall.told, + recall.focus, _moment.packed_parts(recall), None if trace is None else trace.trace, + None if store is None else (store.strength, store.mass, store.consolidated), + ) + + def _lean_qualified( + self, drive: np.ndarray, state: BrainState | None, operation: str, judges: bool = False + ) -> BrainState | None: + """``_qualified`` of the compiled moment where its first solve qualifies every row, in + the calls its compiled path makes (``_moment.free_solve``): the same state, report and + bits; None where the solve takes another path or a row needs the rescue (the caller + then qualifies the drive, whose first solve is this one).""" + cfg = self.learner.config + graph = self.brain + if ( + cfg.tolerance is None or cfg.qualified or not cfg.fast_settle + or self.learned_code is not None + or type(graph).equilibrate is not NeuralGraph.equilibrate + ): + return None + read = self._read_judged(drive, state, operation) + if read is not None: + return read + budget = cfg.free_steps + phase = _moment.free_solve( + graph, drive, state, budget if budget < 2 else (budget + 1) // 2, cfg.tolerance + ) + if phase is None or not (phase.residual <= phase.tolerance).all(): + return None + report = self._settlement( + operation, phase.residual, phase.steps, phase.residual_checks, phase.row_sweeps + ) + self._last_settlement = MappingProxyType(report) + if judges: + self._judged = ( + graph, graph._efficacy, graph._bias, cfg, operation, state, drive.tobytes(), + None if state is None else (state.v.tobytes(), state.adaptation.tobytes()), + phase.state, report, + ) + return cast(BrainState, phase.state) + def _qualified( self, drive: np.ndarray, state: BrainState | None = None, *, operation: str = "answer", judges: bool = False, @@ -3844,6 +4186,10 @@ def _qualified( cfg = self.learner.config if cfg.tolerance is None: raise ValueError("Brain answers require a finite residual tolerance") + if _moment.SHORTCUTS and self._compiled_moment(): + lean = self._lean_qualified(drive, state, operation, judges) + if lean is not None: + return lean read = self._read_judged(drive, state, operation) if read is not None: return read @@ -4132,6 +4478,13 @@ def _lesson( ) -> dict[str, float]: """The lesson of validated observations and labels (or targets, and weights); ``last_learning`` takes its report with the ``demonstration_`` prefix.""" + if self.learned_code is None: # ``_code_attempt`` without a code: the moment alone + moment = self._coded_moment + try: + return self._teach(x, labels, targets=targets, weights=weights) + except BaseException: + self._coded_moment = moment + raise with self._code_attempt(): return self._teach(x, labels, targets=targets, weights=weights) @@ -4165,14 +4518,33 @@ def _teach( from .learning import LearningPhaseError batch = len(x) - self._prepare(x) - drive = self._moment_drive(x) + drive = None + judged = None + if ( + _moment.SHORTCUTS and self.learned_code is None and self.sparse_code is None + and self._compiled_moment() + ): # the compiled moment's read without the general path's layers (``_settled``) + if _moment.ONE_CALL and labels is not None: + # with the judging solve, in one call + judged = self._judge_compiled( + x, labels, _moment.ONE_LESSON and weights is None + ) + if judged is not None: + drive = judged[0] + else: + self._coded_moment = (x, x, None, x, None) + drive = self._compiled_drive(x, x) + if drive is None: + self._prepare(x) + drive = self._moment_drive(x) code = self._coded(x) place = self._placed(x) key = self._decision_key(code) # of the decision the lesson is shown for answer = None tokens = None if labels is None else self._token_rows(labels) - if self.learner.config.surprise and labels is not None: + if judged is not None: + answer = judged[1] + elif self.learner.config.surprise and labels is not None: # the brain's own answer, the act it would issue: its complete greedy choice on the # settle its act reads, warm from the stream, with the episodic vote under the # decision's own key; a row that answer already gives as shown is no surprise @@ -4187,6 +4559,20 @@ def _teach( answer = np.where( tokens.reshape((-1,) + (1,) * (np.ndim(answer) - 1)), answer, -1 ) + agreed = None + whole = None + differs = None if judged is None else judged[2] + if differs is not None: + # the one call judged every row: where none differs the lesson is the learner's + # lesson of no row (``Learner.step`` returns it for these validated labels) + if not differs.any(): + agreed = self.learner._untaught(differs) + elif _moment.ONE_LESSON and weights is None and labels is not None: + # the rows that differ are taught in one call, from the phases to the written + # efficacies, the read's maps and the carried gains + report = _moment.taught_lesson(self, drive, self._activity(), labels, differs) + if report is not None: + whole = (LearnedState(None, None, None, differs), report) try: # the lesson's free phase starts where the act settles: warm from the stream's activity # (rest after a reset, and for streams that have not acted), with a learned code's @@ -4198,7 +4584,7 @@ def _teach( options["weight"] = weights if targets is not None: options["targets"] = targets - lesson, teaching = self.learner.step( + lesson, teaching = agreed or whole or self.learner.step( drive, labels, warm=self._warm(drive, self._activity()), answer=answer, **options ) except LearningPhaseError as error: @@ -4207,17 +4593,25 @@ def _teach( ) self.last_learning["demonstrations"] = 0.0 raise - if self.recall is not None: + # a lesson in one call taught the read's maps and the carried gains too + if whole is None and self.recall is not None: # the address lesson: the slots scored against the lesson's movement step the map, # with declared tokens at the rows whose answer is one teaching.update(self.recall.learn(self.learner.brain, lesson, tokens)) # the gains' lesson (``recall_gain``): the motor's movement steps them teaching.update(self.recall.learn_gain(lesson)) - if self.hearing is not None: + if whole is None and self.hearing is not None: + self.hearing.compiled_moment = self._compiled_moment() self.hearing.learn(self.learner.brain, lesson, self.learner.config.beta) - self.last_learning.update( - {"demonstration_" + name: value for name, value in teaching.items()} - ) + named = self.__dict__.get("_agreed_report") if agreed is not None else None + if named is None or named[0] != teaching: + named = ( + dict(teaching), + {"demonstration_" + name: value for name, value in teaching.items()}, + ) + if agreed is not None: # a lesson of no row reports the same at every such lesson + self.__dict__["_agreed_report"] = named + self.last_learning.update(named[1]) # A demonstration teaches the slow policy, not an invented reward value. An # episodic memory also records the shown action for the situation it was shown in, # under the observation code; under the learner's surprise rule, for the rows the @@ -4982,15 +5376,96 @@ def act( raise ValueError("action batch must match the live streams; reset for new streams") if temperature is not None: temperature = self.basal_ganglia._temperature(temperature) - drive, free = self._settled(x) - action = self._choose(x, drive, free, greedy=greedy, temperature=temperature) + compiled = ( + greedy and temperature is None and _moment.SHORTCUTS and self._compiled_moment() + ) + action: np.ndarray | None = ( + self._act_compiled(x) if compiled and _moment.ONE_CALL else None + ) + if action is None: + drive, free = self._settled(x) + if compiled: + action = self._greedy_choice(x, drive, free) + if action is None: + action = self._choose(x, drive, free, greedy=greedy, temperature=temperature) self._acted = (x.copy(), self.basal_ganglia.state) return action + def _greedy_choice( + self, x: np.ndarray, drive: np.ndarray, free: BrainState + ) -> np.ndarray | None: + """``_choose(x, drive, free, greedy=True)`` of the compiled moment in the calls it makes + for a brain of one motor slot without an episodic vote, a learned code, an efference + copy, a heard history or rehearsal: the same action and state to the bit; None for any + other brain (the caller then chooses).""" + agent = self.basal_ganglia + if ( + self._episodic is not None or self.learned_code is not None + or self.efference is not None or self.history is not None + or self.learner.slot_count != 1 or self.learner.config.rehearsal + or type(agent).act is not ActorCritic.act or _resident(drive) + or not np.isfinite(drive).all() + ): + return None + agent._free = free + agent._free_brain = self.brain + agent._drive = drive.copy() + self._prepared = None + self._awaiting = None + code = self._coded(x) + # the actor's greedy act on the free state it holds (``ActorCritic.act``) + agent._pending = None + agent._pending_memory = None + agent._pending_hearing = None + spans = ( + _moment.after_spans(self) + if self.recall is not None and not self.recall.action and code is x + and self.recall._compiles(len(x)) and isinstance(free.activation, np.ndarray) + and (self.working_memory is None or self.working_memory._device is None) + else None + ) + action: Any + if spans is not None: # the choice, the trace's step and the write in one pass + action, clear = _moment.act_after(self, x, free, spans) + if not clear: + action = np.argmax(agent.probabilities(free, None, bias=None), axis=-1) + action = np.asarray(action, dtype=np.int64) + else: + cleared = agent._clear_greedy(free) + if cleared is None: + cleared = np.argmax(agent.probabilities(free, None, bias=None), axis=-1) + action = np.asarray(cleared, dtype=np.int64) + if self.working_memory is not None: + self.working_memory.update(free) + if self.recall is not None: + self.recall.witness( + code, free, self._command(action) if self.recall.action else None + ) + if self._episode_moments: + self._episode_moments, self._episode_told = [], np.zeros(0, dtype=np.int64) + self._moment = None + self._moment_code = None + return cast(np.ndarray, action) + def _settled(self, x: np.ndarray) -> tuple[np.ndarray, BrainState]: """The drive of validated observations and its qualified free state, warm from the stream's last state; the observations' code is computed here once for the moment. Refusal raises before anything changes.""" + if ( + _moment.SHORTCUTS and self.learned_code is None and self.sparse_code is None + and self._compiled_moment() + ): + # the compiled moment's read and solve without the general path's layers: the code + # of the moment is the observations, and the drive the compiled read's + held = self._coded_moment + self._coded_moment = (x, x, None, x, None) + try: + drive = self._compiled_drive(x, x) + if drive is not None: + return drive, self._qualified(drive, self._activity(), operation="act") + except BaseException: + self._coded_moment = held + raise with self._code_attempt(): self._prepare(x) drive = self._moment_drive(x) @@ -5002,6 +5477,15 @@ def _coded(self, x: np.ndarray) -> np.ndarray: coded = self._coded_moment return coded[1] if coded is not None and coded[0] is x else self._code(x) + def _fade_key(self, x: Any) -> np.ndarray | None: + """The place key a fading store's records are cued by this moment (``memory_fade``): + the observation code where the store keys on it, else the observations.""" + try: + raw = np.atleast_2d(np.asarray(_on_host(x), dtype=float)) + return np.atleast_2d(np.asarray(self._code(raw))) if self._coded_keys else raw + except (TypeError, ValueError): # no key of this moment: it cues no record + return None + def _placed(self, x: np.ndarray) -> np.ndarray: """The code the place stores key on (the associative store, the episodic place records): the moment's, kept by ``_prepare``; of any other observations, the @@ -5305,8 +5789,8 @@ def parameters(self) -> int: """Actor/critic and consolidated weights (the episodic records' paid store with ``memory_extinction``), the recall memory's address and position maps and the maps of its options (anchor, relative, slope, gain), a learned code's tables, and with ``maturity`` - its use and the place records' noise scales; per-stream transient storage is - additional.""" + its use and the place records' noise scales, with ``memory_fade`` the records' depths and + recent traces; per-stream transient storage is additional.""" recall = self.recall paid = None if self._episodic is None else self._episodic.paid return ( @@ -5314,6 +5798,7 @@ def parameters(self) -> int: + ( self.hippocampus.consolidated.size + (0 if self.hippocampus.gate_noise is None else self.hippocampus.gate_noise.size) + + (2 * self.hippocampus.key_width if self.hippocampus.fade else 0) if isinstance(self.hippocampus, SynapticMemory) else 0 ) @@ -5340,7 +5825,9 @@ def save(self, path: str | Path, *, compressed: bool = True) -> Path: """Save the complete composition, including an action awaiting its real outcome. The file is ``cadence-brain/1`` (with maturity ``cadence-brain/2``, which adds the - maturity and the store's noise scales): the learner's checkpoint, the actor (its config, + maturity and the store's noise scales; with ``memory_fade`` ``cadence-brain/3``, which + adds the records' depths and recent traces, beside a maturity where the brain has + one): the learner's checkpoint, the actor (its config, critic, eligibility and random generator), the brain's random generator, the free phase, a pending action's nudged states, the state ``wait`` sensed since that action was chosen, the arousal, the observation code or the learned code, and one group @@ -5358,8 +5845,11 @@ def save(self, path: str | Path, *, compressed: bool = True) -> Path: agent = self.basal_ganglia data = _learner_data(self.learner) memories: dict[str, Any] = {} + store = self.hippocampus + fading = isinstance(store, SynapticMemory) and bool(store.fade) metadata: dict[str, Any] = { - "format": BRAIN_FORMAT if agent.maturity is None else MATURITY_FORMAT, + "format": FADE_FORMAT if fading + else BRAIN_FORMAT if agent.maturity is None else MATURITY_FORMAT, "actor": agent.config.to_dict(), "rng": self.rng.bit_generator.state, "actor_rng": agent.rng.bit_generator.state, @@ -5499,7 +5989,8 @@ def load( chose takes it again at its first moment (``backend="auto"``), and any other runs on the CPU, portable inference and learning. - Reads ``cadence-brain/1`` and ``cadence-brain/2`` (a brain with maturity), the formats + Reads ``cadence-brain/1``, ``cadence-brain/2`` (a brain with maturity) and + ``cadence-brain/3`` (a brain whose store fades, ``memory_fade``), the formats of 0.85.0 on, and refuses the earlier ``cadence-generic/1`` to ``/10`` by name: cadence 0.94.0 loads those and saves ``cadence-brain/1``. Keep the saved batch row identities, or call ``reset`` for new episodes. Cross-backend results are subject to floating-point @@ -5518,10 +6009,14 @@ def load( raise ValueError("not a Brain checkpoint; use Learner.load for a learner") saved = json.loads(str(archive["brain"])) if not isinstance(saved, dict) or saved.get("format") not in ( - BRAIN_FORMAT, MATURITY_FORMAT + BRAIN_FORMAT, MATURITY_FORMAT, FADE_FORMAT ): raise ValueError("unsupported Brain checkpoint format") - if (saved["format"] == MATURITY_FORMAT) != ("maturity" in saved): + fading = _saved_fade(saved) + if (saved["format"] == FADE_FORMAT) != fading: + raise ValueError(f"a fading store belongs to checkpoint format {FADE_FORMAT}") + # a fading store's file holds a maturity or none; any other file is named by it + if not fading and (saved["format"] == MATURITY_FORMAT) != ("maturity" in saved): raise ValueError( f"a brain's maturity belongs to checkpoint format {MATURITY_FORMAT}" ) @@ -5695,6 +6190,7 @@ def saved_maps(kind: str) -> list[tuple[np.ndarray, np.ndarray, np.ndarray]]: if recall.action: # validated above: saved with the command recall.acted = data["recall/acted"].astype(float) recall.writes = int(recalled["writes"]) + recall.gain_lessons = int(recalled.get("gain_lessons", 0)) result.learner = learner result._composition = composition result.attention = attention @@ -5928,9 +6424,8 @@ def saved_maps(kind: str) -> list[tuple[np.ndarray, np.ndarray, np.ndarray]]: ) # settings of the brain's own loop: a new part there cannot start invisible and learn _GROW_TOKENS = ( - "recall_tokens start the copy pathway as an identity onto the motor neurons, which grow " - "starts at zero on a trained brain: grow adds no recall memory with tokens and no heads to " - "a brain with them" + "recall_tokens are laid out per motor neuron and per read port: grow adds no recall memory " + "with tokens and no heads to a brain with them" ) _GROW_EPISODIC = frozenset({"episodic", *_EPISODIC_SETTINGS}) _GROW_LOOP = frozenset( @@ -6612,7 +7107,7 @@ def _grafted_checkpoint( data = dict(host) # the donor's contact counts, efficacies and plasticity masks, so a grafted synapse drives # its neuron as it did in the donor; this brain's signs, the construction its efficacies - # started from, which a recall copy pathway's tokens lay out again on load; an optimizer + # started from; an optimizer # history afresh, as a new synapse's: the donor's moments belong to the donor's count of # lessons for name in ("count", "efficacy", "plastic_synapses"): diff --git a/src/cadence/learning.py b/src/cadence/learning.py index fd8eba31..a5605bba 100644 --- a/src/cadence/learning.py +++ b/src/cadence/learning.py @@ -137,6 +137,10 @@ class LearnerConfig: # it qualifies and the activation is made in arithmetic that vectorizes # (``Brain.equilibrate``); the same equations, order and tolerance, other bits fast_settle: bool = False + # opt-in, a composed brain's on the CPU (``cadence._moment``): the work around the solve + # kernels of a moment (the recall memory's read, write and address lesson, the place store's + # read, the carried gains' contrast) in compiled passes; the same equations, other bits + compiled_moment: bool = False # Fields this library no longer takes, each with the release that retired it, the one # value at which a saved config still loads (the law of this version) and what the learner @@ -206,6 +210,8 @@ def __post_init__(self) -> None: raise ValueError("nudged_fixed must be boolean") if not isinstance(self.fast_settle, bool): raise ValueError("fast_settle must be boolean") + if not isinstance(self.compiled_moment, bool): + raise ValueError("compiled_moment must be boolean") if self.tolerance is not None and (not np.isfinite(self.tolerance) or self.tolerance < 0): raise ValueError("tolerance must be finite and nonnegative, or None") if not isinstance(self.qualified, bool): @@ -272,7 +278,7 @@ def to_dict(self) -> dict[str, Any]: _NEUTRAL: dict[str, Any] = { "normalize_relative": 0.0, "nudged_fixed": False, "rehearsal": 0.0, "rehearsal_capacity": 512, - "surprise_share": False, "fast_settle": False, + "surprise_share": False, "fast_settle": False, "compiled_moment": False, } @@ -367,6 +373,11 @@ def _restrict(state: BrainState, taught: np.ndarray | None, drive: Any = None) - return state if taught is None or bool(taught.all()) else _rows(state, taught, drive) +# The report of a lesson whose caller-read answer matches every label, by batch size and +# qualification (``Learner._untaught``): the same numbers at every such lesson. +_UNTAUGHT: dict[tuple[int, bool], dict[str, float]] = {} + + def _taught_report(taught: np.ndarray | None, batch: int) -> dict[str, float]: """How many rows were offered, how many the lesson taught, and how many it agreed with.""" count = float(batch if taught is None else int(taught.sum())) @@ -469,7 +480,11 @@ class Learner: reciprocal: bool = True # a synapse and its reverse share one efficacy: a reciprocal pair tie_groups: np.ndarray | None = None # int per synapse (-1: none); a group shares one scale # float per synapse multiplying its step (default one): a fan-in scale, a per-projection - # rate; a wide population's inbound synapses need a smaller step than a narrow one's + # rate; a wide population's inbound synapses need a smaller step than a narrow one's. A rate + # of zero holds the synapse: it takes no step of its own and the optimizer keeps its running + # moments as they are, and a neuron whose every inbound synapse steps at zero holds its bias + # the same way (``held``). A reciprocal pair steps by the mean of its two directions' steps, + # so the pair holds where both are at zero; one alone takes half its partner's step synapse_rate: np.ndarray | None = None # the outputs as groups, each its own softmax choice, all nudged together: a count of # equal groups, or one size per group (a body's controller: a move of nine, a grip of two) @@ -724,6 +739,11 @@ def _qualified_phase( def _labels(self, labels: np.ndarray) -> np.ndarray: labels = np.asarray(labels) + if labels.dtype == np.int64 and labels.ndim == 1 and self.slot_count == 1 and labels.size: + # one slot's labels of the kind returned: the checks below in two reductions + if labels.min() < 0 or labels.max() >= self.slot_size: + raise ValueError("a label lies outside its output slot's choices") + return labels if self.slot_count > 1: if labels.ndim != 2 or labels.shape[1] != self.slot_count: raise ValueError("a slotted learner takes labels shaped (batch, slots)") @@ -812,6 +832,14 @@ def contrast( return edges / (batch * span), neurons / (batch * span) s_plus, s_minus = nudged.activation, minus_state.activation if self.brain._blocked and w.synapses > 4 * w.n: + if self.config.compiled_moment and _compiled_passes(): + # the compiled moment's contrast: the same products in one compiled pass, which + # its lesson in one call makes too (``_moment.taught_lesson``) + from . import _moment + + found = _moment.contrast(self.brain.layout, s_plus, s_minus, span) + if found is not None: + return found gram = block_contrast(self.brain.layout, s_plus, s_minus, len(s_plus) * span) return gram, (s_plus - s_minus).mean(axis=0) / span if not self.brain._blocked: @@ -849,6 +877,38 @@ def contrast_rows( hebb = a_plus * (b_plus - b_minus) + (a_plus - a_minus) * b_minus return hebb / span, (s_plus - s_minus) / span + def held(self) -> tuple[np.ndarray | None, np.ndarray | None]: + """What a rate of zero holds (``synapse_rate``): the synapses at rate zero, and the neurons + whose every inbound synapse steps at zero, whose biases take no step either. A neuron no + synapse reaches is held by none. A held synapse or bias keeps its optimizer moments as they + are. A synapse is named by its rate alone: one direction of a reciprocal pair at zero is + named and keeps its moments, and takes half its partner's step through the pair's mean. + ``(None, None)`` where no rate is zero.""" + rate = self.synapse_rate + if rate is None or not len(rate) or float(np.min(rate)) > 0: + return None, None + zero = np.asarray(rate) == 0 + w = self.brain.connectome + inbound = np.bincount(w.post, minlength=w.n) + stopped = np.bincount(w.post[zero], minlength=w.n) + neurons = (inbound > 0) & (stopped == inbound) + return zero, (neurons if neurons.any() else None) + + def _stepping_neurons(self) -> np.ndarray: + """The neurons whose biases move: the plastic ones a rate of zero does not hold.""" + assert self.plastic_neurons is not None + neurons = self.held()[1] + return self.plastic_neurons if neurons is None else self.plastic_neurons & ~neurons + + def _hold_moments(self, before: Mapping[str, np.ndarray]) -> None: + """Give the synapses and neurons a rate of zero holds (``held``) their moments of + ``before`` back, after an update wrote the others.""" + synapses, neurons = self.held() + for name, mask in (("velocity", synapses), ("second_moment", synapses), + ("velocity_bias", neurons), ("second_moment_bias", neurons)): + if mask is not None and getattr(self, name) is not before[name]: + setattr(self, name, np.where(mask, before[name], getattr(self, name))) + def apply(self, delta_scale: np.ndarray, delta_bias: np.ndarray) -> dict[str, float]: """Apply a per-synapse and per-neuron step: masks, tying, decay, bounds; then set the brain. @@ -889,8 +949,7 @@ def apply(self, delta_scale: np.ndarray, delta_bias: np.ndarray) -> dict[str, fl if not np.isfinite(delta_scale).all(): raise ValueError("scaled and tied steps must remain finite") scale = self.brain.efficacy + delta_scale - assert self.plastic_neurons is not None - delta_bias = np.where(self.plastic_neurons, delta_bias, 0.0) + delta_bias = np.where(self._stepping_neurons(), delta_bias, 0.0) bias = self.brain.bias + delta_bias cap = cfg.scale_cap scale = np.where(self.plastic_synapses, np.clip(scale, -cap, cap), scale) @@ -1033,6 +1092,12 @@ def _adaptive_device(self, kernel: Any, edges: Any, neurons: Any) -> tuple[Any, neurons = neurons / ( (held["second_moment_bias"] / correction).sqrt() + cfg.normalize_floor ) + ix = self._device_indices(kernel) + for name, mask in (("velocity", "held_synapses"), ("second_moment", "held_synapses"), + ("velocity_bias", "held_neurons"), + ("second_moment_bias", "held_neurons")): + if ix[mask] is not None and held[name] is not previous[name]: + held[name] = kernel.torch.where(ix[mask], previous[name], held[name]) if not bool(kernel.torch.stack([ kernel.torch.isfinite(held[name]).all() for name in _MOMENTS ]).all()): @@ -1101,7 +1166,9 @@ def to(x: np.ndarray, dtype: Any = None) -> Any: return torch.from_numpy(np.ascontiguousarray(x)).to(dev, dtype) all_synapses = bool(self.plastic_synapses.all()) - all_neurons = bool(self.plastic_neurons.all()) + held_synapses, held_neurons = self.held() + stepping = self._stepping_neurons() + all_neurons = bool(stepping.all()) cache.clear() cache.update( key=key, @@ -1119,19 +1186,35 @@ def to(x: np.ndarray, dtype: Any = None) -> Any: if len(self._members) else None, synapses=None if all_synapses else to(self.plastic_synapses, kernel.param_dtype), - neurons=None if all_neurons else to(self.plastic_neurons, kernel.param_dtype), + neurons=None if all_neurons else to(stepping, kernel.param_dtype), + held_synapses=None if held_synapses is None else to(held_synapses), + held_neurons=None if held_neurons is None else to(held_neurons), ) return cast(dict[str, Any], cache) def _apply_device(self, kernel: Any, delta_scale: Any, delta_bias: Any) -> dict[str, float]: """``apply`` on the device: the same masks, tying, decay and bounds, no host array.""" + candidate = self._device_candidate(kernel, delta_scale, delta_bias) + flags = kernel.torch.stack( + [flag.to(kernel.dtype) for flag in candidate["flags"]] + + [step.to(kernel.dtype) for step in candidate["steps"]] + ).cpu().numpy() + self._device_refusal(flags[:4]) + return self._install_device(candidate, flags[4:]) + + def _device_candidate(self, kernel: Any, delta_scale: Any, delta_bias: Any) -> dict[str, Any]: + """The parameters ``_apply_device`` would install, with its checks and its report, all on + the device and none read: ``flags`` (the steps finite, the scaled and tied steps finite, the + parameters and weights finite, their summed values finite, in the order ``apply`` refuses + them) and ``steps`` (the report's synapse and bias steps). ``_device_refusal`` reads the + flags, ``_install_device`` installs.""" torch, cfg = kernel.torch, self.config if ( tuple(delta_scale.shape) != (self.brain.connectome.synapses,) or tuple(delta_bias.shape) != (self.brain.connectome.n,) - or not bool(torch.isfinite(delta_scale).all() & torch.isfinite(delta_bias).all()) ): raise ValueError("steps must be finite vectors, one per synapse and one per neuron") + given = torch.isfinite(delta_scale).all() & torch.isfinite(delta_bias).all() ix = self._device_indices(kernel) d = delta_scale.clone() if ix["synapses"] is not None: @@ -1150,8 +1233,7 @@ def _apply_device(self, kernel: Any, delta_scale: Any, delta_bias: Any) -> dict[ ] if ix["synapses"] is not None: # tying never moves a frozen synapse d = d * ix["synapses"] - if not bool(torch.isfinite(d).all()): - raise ValueError("scaled and tied steps must remain finite") + tied = torch.isfinite(d).all() scale = kernel.scale + d db = delta_bias if ix["neurons"] is None else delta_bias * ix["neurons"] bias = kernel.bias_param + db @@ -1161,13 +1243,34 @@ def _apply_device(self, kernel: Any, delta_scale: Any, delta_bias: Any) -> dict[ if ix["synapses"] is None else torch.where(ix["synapses"].bool(), bounded, kernel.scale) ) - self.brain = self.brain._with_device_parameters(scale, bias) - self.updates += 1 + checked = kernel.parameter_check(scale, bias) + zero = torch.zeros((), dtype=d.dtype, device=d.device) return { - "scale_step": float(d.abs().mean()) if d.numel() else 0.0, - "bias_step": float(db.abs().mean()), + "scale": scale, "bias": bias, "checked": checked, + "flags": (given, tied, checked[3], checked[4]), + "steps": (d.abs().mean() if d.numel() else zero, db.abs().mean()), } + @staticmethod + def _device_refusal(flags: Any) -> None: + """The refusal of a ``_device_candidate`` whose four flags were read as ``flags``.""" + if not flags[0]: + raise ValueError("steps must be finite vectors, one per synapse and one per neuron") + if not flags[1]: + raise ValueError("scaled and tied steps must remain finite") + from .brain import _TorchKernel + + _TorchKernel.parameter_refusal(bool(flags[2]), bool(flags[3])) + + def _install_device(self, candidate: dict[str, Any], steps: Any) -> dict[str, float]: + """Install a ``_device_candidate`` whose flags read clear; ``steps`` its report's two + numbers, read.""" + self.brain = self.brain._with_device_parameters( + candidate["scale"], candidate["bias"], candidate["checked"] + ) + self.updates += 1 + return {"scale_step": float(steps[0]), "bias_step": float(steps[1])} + def _projections(self) -> tuple[np.ndarray, np.ndarray]: """Each synapse's projection (its block of the layout) and each neuron's range.""" layout = self.brain.layout @@ -1252,13 +1355,21 @@ def _mlx_update_tables(self, plus: BrainState, minus: BrainState) -> dict[str, A ): tables = kernel.update_tables(self.brain._gain_pre, self._paired, self._paired_reverse) if tables is not None: - assert self.plastic_neurons is not None tables["plastic_neurons"] = kernel.mx.array( - np.asarray(self.plastic_neurons, dtype=np.float32) + np.asarray(self._stepping_neurons(), dtype=np.float32) ) if rate is not None: rate.flags.writeable = False tables["rates"] = kernel.block_rates(rate) + synapses, neurons = self.held() + if synapses is not None: # by block, where a block holds any + flat = kernel.layout.flat(synapses.astype(float)) + tables["held_blocks"] = [ + (k, kernel.mx.array(np.ascontiguousarray(part) > 0)) + for k, part in enumerate(kernel.layout.blocks(flat)) if part.any() + ] + if neurons is not None: + tables["held_neurons"] = kernel.mx.array(neurons) held = (kernel.root, self._paired, tables, rate) self.__dict__["_mlx_tables"] = held return cast("dict[str, Any] | None", held[2]) @@ -1301,6 +1412,13 @@ def _update_on_mlx( ), share * cfg.eta_bias, cfg.scale_cap, m > 0, rho > 0, bool(cfg.normalize_relative), ) + mx = kernel.mx + for k, mask in tables.get("held_blocks", ()): # a rate of zero keeps the moments it held + for name in ("velocity", "second_moment"): + moments[name][k] = mx.where(mask, held[name][k], moments[name][k]) + if "held_neurons" in tables: + for name in ("velocity_bias", "second_moment_bias"): + moments[name] = mx.where(tables["held_neurons"], held[name], moments[name]) self.__dict__["_device_moments"] = moments self.brain = self.brain._with_mlx_parameters(new) self.updates += 1 @@ -1359,8 +1477,10 @@ def _update( raise ValueError("phase contrasts must be finite") raw_synapse, raw_neuron = synapse_term, neuron_term count = self.contrast_updates + 1 # only this optimizer's own history + before = {name: getattr(self, name) for name in _MOMENTS} if len(raw_synapse) and _compiled_passes(): delta_scale, delta_bias = self._compiled_step(raw_synapse, raw_neuron, count) + self._hold_moments(before) if not all(np.isfinite(getattr(self, name)).all() for name in _MOMENTS): raise ValueError("optimizer moments must remain finite") report = self.apply(share * delta_scale, share * delta_bias) @@ -1388,6 +1508,7 @@ def _update( synapse_term, neuron_term = synapse_term / rms, neuron_term / rms_bias delta_scale = share * cfg.eta * synapse_term delta_bias = share * cfg.eta_bias * neuron_term + self._hold_moments(before) if not all(np.isfinite(getattr(self, name)).all() for name in _MOMENTS): raise ValueError("optimizer moments must remain finite") report = self.apply(delta_scale, delta_bias) @@ -1426,28 +1547,33 @@ def step( label does. A distribution has no single answer to agree with: ``surprise`` judges labels, and every row of targets is taught. """ + target: np.ndarray | None = None if labels is None: if targets is None: raise ValueError("a lesson needs labels or targets") target = self.soft_targets(targets) + batch = len(target) else: if targets is not None: raise ValueError("a lesson takes labels or targets, not both") labels = self._labels(labels) - target = self.targets(labels) + batch = len(labels) if not _resident(drive): drive = np.asarray(drive, dtype=float) if drive.ndim == 1: drive = drive[None, :] - if drive.ndim != 2 or drive.shape[0] != target.shape[0]: + if drive.ndim != 2 or drive.shape[0] != batch: raise ValueError("drive and labels must have the same batch size") cfg = self.config - batch = len(target) judged = None if cfg.surprise and answer is not None and labels is not None: judged = self._differs(np.asarray(answer), labels) if not judged.any(): return self._untaught(judged) + if target is None: # the labels' targets, made where a phase reads them + assert labels is not None + target = self.targets(labels) + if judged is not None: drive, target, weight = self._lesson(judged, drive, target, weight) warm = None if warm is None else _restrict(warm, judged) if cfg.qualified: @@ -1476,6 +1602,18 @@ def step( report.update(_phase_report(phases, accepted=True, qualified=True)) report.update(_taught_report(taught, batch)) return LearnedState(free, nudged, opposite, taught), report + if judged is not None and weight is None and cfg.compiled_moment: + # the compiled moment's lesson: the three phases and their checks in one call + found = self._phases_compiled(drive, warm, target) + if found is not None: + free, nudged, opposite, phases = found + exhausted = cfg.tolerance is not None and free.steps >= cfg.free_steps > 0 + report = self.update(free, nudged, opposite, share=self._share(judged)) + report.update(_phase_report(phases, accepted=True, qualified=False)) + report.update(_taught_report(judged, batch)) + report["free_budget_exhausted"] = float(exhausted) + self._warn_exhausted(exhausted) + return LearnedState(free, nudged, opposite, judged), report free = self.free(drive, warm) # Finite phases get post-solve residual diagnostics. They count additional # work and do not certify the lesson. @@ -1521,6 +1659,36 @@ def step( self._warn_exhausted(exhausted) return LearnedState(free, nudged, opposite, taught), report + def _phases_compiled( + self, drive: Any, warm: BrainState | None, target: np.ndarray + ) -> tuple[BrainState, BrainState, BrainState, dict[str, Equilibrium]] | None: + """A finite lesson's free phase, nudged phases and their residual checks in one compiled + call (``_moment.lesson_phases``, under ``compiled_moment``): the states and the phases' + records ``step`` makes through ``free``, ``_nudged`` and ``_diagnostic``, to the bit; None + where a phase takes another path or one of them is overridden.""" + from . import _moment + + kind = type(self) + if ( + not (_moment.ONE_CALL and _moment.SHORTCUTS) or not _compiled_passes() + or kind.free is not Learner.free or kind._nudged is not Learner._nudged + or kind._diagnostic is not Learner._diagnostic + or kind.nudge_for is not Learner.nudge_for + ): + return None + found = _moment.lesson_phases(self, drive, warm, target) + if found is None: + return None + free, nudged, opposite, checks = found + tolerance = self.config.tolerance if self.config.tolerance is not None else 0.0 + phases = { + name: Equilibrium(state, check, state.steps, tolerance, residual_checks=1) + for name, state, check in zip( + ("free", "nudged", "opposite"), (free, nudged, opposite), checks, strict=True + ) + } + return free, nudged, opposite, phases + def _differs(self, answer: np.ndarray, labels: np.ndarray) -> np.ndarray: """True per row where an answer differs from its label in any slot.""" if answer.shape != labels.shape: @@ -1558,22 +1726,25 @@ def _untaught(self, taught: np.ndarray) -> tuple[LearnedState, dict[str, float]] """The lesson of a batch whose caller-read answer matches every label: no phase runs, nothing changes; every phase counts zero work.""" cfg = self.config - report = { - "accepted": 1.0, - "qualified": float(cfg.qualified), - "qualification_required": float(cfg.qualified), - "attempted_presentations": float(len(taught)), - } - for name in ("free", "nudged", "opposite"): - for key in _PHASE_KEYS: - report[f"{name}_{key}"] = 0.0 - for key in ("steps", "residual_checks", "stagnation_checks", "row_sweeps", - "row_residual_checks"): - report[f"total_{key}"] = 0.0 - report.update(_taught_report(taught, len(taught))) - if not cfg.qualified: - report["free_budget_exhausted"] = 0.0 - return LearnedState(None, None, None, taught), report + held = _UNTAUGHT.get((len(taught), cfg.qualified)) + if held is None: + report = { + "accepted": 1.0, + "qualified": float(cfg.qualified), + "qualification_required": float(cfg.qualified), + "attempted_presentations": float(len(taught)), + } + for name in ("free", "nudged", "opposite"): + for key in _PHASE_KEYS: + report[f"{name}_{key}"] = 0.0 + for key in ("steps", "residual_checks", "stagnation_checks", "row_sweeps", + "row_residual_checks"): + report[f"total_{key}"] = 0.0 + report.update(_taught_report(taught, len(taught))) + if not cfg.qualified: + report["free_budget_exhausted"] = 0.0 + held = _UNTAUGHT[(len(taught), cfg.qualified)] = report + return LearnedState(None, None, None, taught), dict(held) def _agreed( self, free: BrainState, phases: Mapping[str, Equilibrium], taught: np.ndarray diff --git a/src/cadence/lives.py b/src/cadence/lives.py new file mode 100644 index 00000000..76475b03 --- /dev/null +++ b/src/cadence/lives.py @@ -0,0 +1,1453 @@ +"""Independent lives over one topology, stepped together (cadence#274, #167). + +``Lives`` holds many composed brains of one anatomy, each with its own parameters, optimizer, +critic, eligibility, recall memory, carried gains, working trace and random state, one stream +each. On the CPU each life is a lone ``Brain``, stepped in turn. On a torch device the lives are +the streams of one brain whose parameters carry a lives axis: every settle reads each stream +through its own life's synapses, and every update moves each life on its own contrast and +dopamine, so one call per moment settles and teaches all of them. Each life follows the lone +brain of its seed on the same device within the solver's tolerance. + +The lives run the composition's own equations through these parts: + +- ``_LivesTorchKernel``: the settle's transport as one batched product per block of the layout, + each life's dense block of its own synapses, each life held where its own stream meets the + tolerance, and the contrast per stream; +- ``_LivesGraph``: a solve that holds each life where its own equations hold, and a rescue on + the damping ladder life by life; +- ``_LivesLearner``: a lesson's update and a reward's update per life; the momentum, the second + moments and the counts of their bias correction per life; a lesson judged by surprise updates + the lives whose own answer differed; +- ``_LivesRecall``: the recall memory's address and position maps per life (a stream's slots are + its own already), the address lesson and the reward's step on each life's own row; +- ``_LivesHearing``: the carried gains, their second moments and counts per life; +- ``_LivesActorCritic`` and ``_LivesCritic``: the critic's weights and bias per life, the + dopamine and the actor's step per stream; +- ``_LivesRandom``: each life's own random generator, drawn as its lone brain draws it. + +A composition holds the parts these cover: the default composition without the store +(``episodic=False``: the store's consolidated records are a brain's own and have no lives axis), +with or without the recall memory and the carried gains, at any teaching point. ``Lives`` refuses +arousal, maturity, a learned or fixed code, an apical compartment, rehearsal, qualified teaching, +the efference copy, the heard history and a recall memory with several heads, a window, the +command, the read of when or tokens, each by name. A moment any life refuses is refused for +every life. +""" + +from __future__ import annotations + +import warnings +from collections.abc import Sequence +from dataclasses import replace +from typing import Any, cast + +import numpy as np + +from .brain import ( + _MLX_CHECK_FLOOR, + BrainState, + Equilibrium, + _check_ladder, + _plan_structure, + _SweepConfig, + _TorchKernel, + _transport_override, + _TransportPlan, +) +from .brain import Brain as NeuralGraph +from .learning import _MOMENTS, _RELATIVE_TINY, LearnedState, Learner, _taught_report +from .plasticity import ActorCritic, _held_drive +from .resident import ( + _MAP_KINDS, + TorchArrays, + _CriticOnDevice, + _forgotten, + _Moment, + _RecallOnDevice, + arrays_of, +) +from .stream import Hearing + +__all__ = ["Lives"] + + +class _RowBias: + """A kernel's biases, one row per life, as its sweeps read them: ``bias[:, None]``, the drive + that never moves, is ``(neurons, lives)`` there.""" + + def __init__(self, rows: Any) -> None: + self.rows = rows # (lives, neurons) + + def __getitem__(self, key: Any) -> Any: + return self.rows.T + + +class _LivesTorchKernel(_TorchKernel): + """``_TorchKernel`` whose parameters carry a lives axis, one stream per life. + + ``scale`` is ``(lives, synapses)`` and ``bias_param`` ``(lives, neurons)``. A sweep's synaptic + input is one batched product per block of the layout, each life's stream through its own + dense block, and the contrast of a lesson or a reward is each stream's own. Every settle + holds exactly one stream per life.""" + + def __init__( + self, connectome: Any, efficacy: Any, gain_pre: Any, bias: Any, neuron_model: Any, + device: str | None, layout: Any, precision: str | None = None, *, + reference: bool = False, parent: _TorchKernel | None = None, + ) -> None: + self.lives = int(efficacy.shape[0]) + self._dense: Any = None + super().__init__( + connectome, efficacy[0], gain_pre, bias[0], neuron_model, device, layout, precision, + None, reference=reference, parent=parent, + ) + torch = self.torch + self._fused = None # the neuron update operation by operation: it reads one bias per row + self.plan = _TransportPlan( # a plan without held products: the lives' blocks move each + torch, + _plan_structure(torch, connectome, None, self.device, layout, _transport_override(), + False), + connectome.n, self.device, self.dtype, + ) + self.flat = self.plan.values + self.set_parameters(self._placed(efficacy), self._placed(bias)) + + # -- the blocks + + def _blocks(self) -> list[tuple[Any, int, int, Any]]: + """The layout's blocks grouped by the range they reach: per group the presynaptic neurons + of its blocks side by side (a slice where they are one run, else their indices), the + range reached and the group's view, ``(lives, pre, post)``, of the dense buffer the + parameters are written into in place (a captured sweep reads the same storage). A sweep + makes one batched product per range reached.""" + if self._dense is None: + torch, lay = self.torch, self.layout + assert lay is not None + connectome = self._connectome + by_post: dict[tuple[int, int], list[int]] = {} + for k in range(lay.pairs): + _, _, b0, b1 = lay.bounds(k) + by_post.setdefault((b0, b1), []).append(k) + block_of = np.asarray(lay.block_of_edge) + entry = np.zeros(connectome.synapses, dtype=np.int64) + spans, at = [], 0 + for (b0, b1), blocks in by_post.items(): + width, row, pre = b1 - b0, 0, [] + for k in blocks: + a0, a1, _, _ = lay.bounds(k) + inside = np.flatnonzero(block_of == k) + entry[inside] = ( + at + (row + connectome.pre[inside] - a0) * width + + connectome.post[inside] - b0 + ) + pre.append(np.arange(a0, a1)) + row += a1 - a0 + index = np.concatenate(pre) + spans.append((index, b0, b1, at, row)) + at += row * width + self._dense = torch.zeros(self.lives, at, dtype=self.dtype, device=self.device) + self._entry = torch.from_numpy(entry).to(self.device) + views = [] + for index, b0, b1, first, rows in spans: + run = len(index) and bool((np.diff(index) == 1).all()) + place = slice(int(index[0]), int(index[-1]) + 1) if run else torch.from_numpy( + index + ).to(self.device) + view = self._dense[:, first : first + rows * (b1 - b0)] + views.append((place, b0, b1, view.view(self.lives, rows, b1 - b0))) + self._views = views + return self._views + + def _synaptic_input(self, s: Any, out: Any) -> Any: + """The transport of a neuron-major activation ``(neurons, lives)``: per range reached, + every life's presynaptic row through its own blocks.""" + if s.shape[1] != self.lives: + raise ValueError(f"a settle of these lives holds one stream per life ({self.lives})") + torch = self.torch + out.zero_() + rows = s.T + for place, b0, b1, block in self._blocks(): + product = torch.bmm(rows[:, place].unsqueeze(1), block).squeeze(1) + out[b0:b1].add_(product.T) + return out + + # -- each life stops where its own stream meets the tolerance + + def _configure( + self, ws: Any, drive: Any, keep: Any, nudge: Any, tolerance: float | None, batch: int, + ) -> Any: + cfg = super()._configure(ws, drive, keep, nudge, tolerance, batch) + held = getattr(ws, "lives_met", None) + if held is None or held.shape[0] != batch: + ws.lives_met = self.torch.zeros(batch, dtype=self.torch.bool, device=self.device) + hold = getattr(self.root, "_hold", None) # the lives a solve found settled already + if hold is None: + ws.lives_met.zero_() + else: + ws.lives_met.copy_(hold) + # sweeps that hold a life are other operations than sweeps that hold none: another + # captured graph, and ``_update`` reads the mark. The kernel keeps one configuration per + # key; this is the lives' own of it, one per configuration and mark + holds = hold is not None or tolerance is not None + kept = ws.__dict__.setdefault("lives_configs", {}) + mine = kept.get((id(cfg), holds)) + if mine is None or mine[0] is not cfg: + if len(kept) >= 64: + kept.clear() + mine = kept[(id(cfg), holds)] = (cfg, _SweepConfig( + (cfg.key, holds), cfg.masked, cfg.keep, cfg.nudge, cfg.held, cfg.tolerance, + cfg.quiet, + )) + return mine[1] + + def _update( + self, ws: Any, cfg: Any, i: int, j: int, synaptic: Any, stage: int, standing: Any, + ) -> None: + """The sweep of every stream, then each life whose stream met the tolerance at an earlier + sweep held where it met it, as its lone brain's settle stops there: a stream's own + movement under the tolerance meets it, and the settle meets it when every life has.""" + super()._update(ws, cfg, i, j, synaptic, stage, standing) + if not cfg.key[-1]: # a fixed count of sweeps with no life settled: every life sweeps + return + torch = self.torch + met = ws.lives_met # (lives,): met at an earlier sweep, or settled before the solve + for slots in ((ws.v[i], ws.v[j]), (ws.s[i], ws.s[j]), + *(((ws.a[i], ws.a[j]),) if ws.a is not None else ())): + torch.where(met[None, :], slots[0], slots[1], out=slots[1]) + torch.where(met, ws.change[i], ws.change[j], out=ws.change[j]) + if cfg.tolerance is None: + return + moved = ws.total.amax(dim=0) # each stream's largest movement in this sweep + met |= moved < cfg.tolerance + torch.all(met, out=ws.flag[j]) + + # -- parameters + + def _spread(self, values: Any, width: int) -> Any: + """A parameter of one life given to every life, or the lives' own.""" + if values.ndim == 1: + return values.unsqueeze(0).expand(self.lives, width).clone() + return values + + def parameter_check(self, scale: Any, bias: Any) -> tuple[Any, ...]: + torch = self.torch + if tuple(scale.shape) != (self.lives, *self.gain_pre.shape) or tuple(bias.shape) != ( + self.lives, self.n, + ): + raise ValueError("lives' parameters are (lives, synapses) and (lives, neurons)") + weights = (self.gain_pre * scale).to(self.dtype) + live_bias = bias.to(self.dtype) + finite = torch.stack( + [torch.isfinite(value).all() for value in (scale, bias, weights, live_bias)] + ).all() + return weights, live_bias, None, finite, finite + + def set_parameters( + self, scale: Any, bias: Any, *, checked: tuple[Any, ...] | None = None + ) -> None: + scale = self._spread(scale, int(self.gain_pre.shape[0])) + bias = self._spread(bias, self.n) + if checked is None: + checked = self.parameter_check(scale, bias) + flags = self.torch.stack([checked[3], checked[4]]).cpu().numpy() + self.parameter_refusal(bool(flags[0]), bool(flags[1])) + weights, live_bias = checked[:2] + self.scale = scale + self.bias_param = bias + self._blocks() + self._dense.zero_() + self._dense.index_add_(1, self._entry, weights) + self.bias = _RowBias(live_bias) + self._version += 1 + + def _float64(self) -> _TorchKernel: + if self.dtype == self.torch.float64: + return self + ref = self._reference + if ref is None: + synapses = self._connectome.synapses + ref = self._reference = _LivesTorchKernel( + self._connectome, np.zeros((self.lives, synapses)), np.zeros(synapses), + np.zeros((self.lives, self.n)), self.neuron_model, str(self.device), + self.layout, "float64", reference=True, + ) + ref._version = -1 + if ref._version != self._version: + ref.set_parameters_from(self) + return ref # type: ignore[no-any-return] + + def stepped(self, neuron_model: Any) -> _TorchKernel: + root = self.root + if root is not self: + return root.stepped(neuron_model) + kernel = self._stepped.get(neuron_model) + if kernel is None: + kernel = _LivesTorchKernel( + self._connectome, self.scale, self.gain_pre, self.bias_param, neuron_model, + str(self.device), self.layout, + "float32" if self.dtype == self.torch.float32 else "float64", parent=self, + ) + kernel._version = self._version + self._stepped[neuron_model] = kernel + elif kernel._version != self._version or kernel.gain_pre is not self.gain_pre: + kernel.set_parameters_from(self) + return kernel + + def __deepcopy__(self, memo: dict[int, Any]) -> _LivesTorchKernel: + twin = _LivesTorchKernel( + self._connectome, self.host_scale(), self.gain_pre.cpu().double().numpy(), + self.host_bias(), self.neuron_model, str(self.device), self.layout, + "float32" if self.dtype == self.torch.float32 else "float64", + reference=self._reference_only, + ) + memo[id(self)] = twin + return twin + + # -- contrasts, one per stream and so per life + + def contrast_tensors(self, s_plus: Any, s_minus: Any) -> tuple[Any, Any]: + edges, neurons = self.contrast_rows(s_plus, s_minus) + return edges.to(self.param_dtype), neurons.to(self.param_dtype) + + def contrast_edges(self, s_plus: Any, s_minus: Any, weights: Any = None) -> Any: + edges = self.contrast_rows(s_plus, s_minus)[0] + return edges if weights is None else edges * weights[:, None] + + +class _LivesGraph(NeuralGraph): + """The lives' graph: a solve holds each life where its own stream's equations hold, as each + lone brain stops its own solve there, and a rescue on the damping ladder runs life by life.""" + + _steps_on_kernel = True # a numerical copy at another step keeps this class and its kernel + + def _equilibrate_local( + self, drive: Any, *, budget: int, chunk: int, tolerance: float, + state: BrainState | None, mask: Any, nudge: Any, stop_if_stalled: bool = False, + fast: bool = False, + ) -> Equilibrium: + """``NeuralGraph._equilibrate_local`` with every life whose residual the check found + within the tolerance held through the next chunk of sweeps. With ``root._only`` set, the + solve is that one life's: every other life is held as it is.""" + kernel = self._torch + assert isinstance(kernel, _LivesTorchKernel) + root: Any = kernel.root + only = getattr(root, "_only", None) + current = self.settle_batch(drive, steps=0, state=state, mask=mask, nudge=nudge) + reference: dict[str, bool] = ( + {"own_precision": True} if tolerance >= _MLX_CHECK_FLOOR else {} + ) + + def settled(error: np.ndarray) -> np.ndarray: + done = np.asarray(error <= tolerance) + if only is not None: + done = done.copy() + done[np.arange(len(done)) != only] = True + return done + + error = self.residual(drive, current, mask=mask, nudge=nudge, **reference) + used, checks, stagnation_checks, repeats = 0, 1, 0, 0 + while used < budget and not settled(error).all(): + if not np.isfinite(error).all() and any( + not np.isfinite(value).all() + for value in (current.v, current.activation, current.adaptation) + ): + break + previous = current + done = settled(error) + root._hold = kernel.torch.as_tensor(done, device=kernel.device) if done.any() else None + try: + current = self.settle_batch( + drive, steps=min(chunk, budget - used), state=current, mask=mask, nudge=nudge + ) + finally: + root._hold = None + used += current.steps + error = self.residual(drive, current, mask=mask, nudge=nudge, **reference) + checks += 1 + if stop_if_stalled and used < budget and not settled(error).all(): + stagnation_checks += 1 + repeats = repeats + 1 if self._repeated_state(previous, current) else 0 + if repeats >= 2: + break + return Equilibrium( + current, error, used, tolerance, residual_checks=checks, + stagnation_checks=stagnation_checks, + ) + + def equilibrate( + self, drive: Any, *, budget: int = 512, chunk: int = 32, tolerance: float = 1e-5, + state: BrainState | None = None, mask: Any = None, nudge: Any = None, damping: int = 0, + fast: bool = False, + ) -> Equilibrium: + """``NeuralGraph.equilibrate``. With ``damping``, each life whose equations do not hold + takes the ladder of halved steps by itself, every other life held, as its lone brain + takes it: a life that stalls or qualifies moves on at its own sweep.""" + if not damping: + return super().equilibrate( + drive, budget=budget, chunk=chunk, tolerance=tolerance, state=state, mask=mask, + nudge=nudge, + ) + kernel = self._torch + assert isinstance(kernel, _LivesTorchKernel) + root: Any = kernel.root + budget, chunk, damping = int(budget), int(chunk), int(damping) + attempts = min(damping + 1, max(1, budget)) + _check_ladder( + self.neuron_model, attempts - 1, + "float32" if kernel.dtype == kernel.torch.float32 else "float64", + ) + current = self.settle_batch(drive, steps=0, state=state, mask=mask, nudge=nudge) + error = self.residual(drive, current, mask=mask, nudge=nudge) + steps = checks = stagnation_checks = halvings = 0 + for life in np.flatnonzero(~np.asarray(error <= tolerance)): + used = 0 + for halving in range(attempts): + portion = (budget - used + attempts - halving - 1) // (attempts - halving) + candidate = self if halving == 0 else self._with_dt( + float(np.ldexp(self.neuron_model.dt, -halving))) + root._only = int(life) + try: + phase = candidate._equilibrate_local( + drive, state=current, budget=portion, chunk=chunk, tolerance=tolerance, + mask=mask, nudge=nudge, stop_if_stalled=halving < attempts - 1, + ) + finally: + root._only = None + used += phase.steps + checks += phase.residual_checks + stagnation_checks += phase.stagnation_checks + current = phase.state + found = phase.residual + if halving: + found = self.residual(drive, current, mask=mask, nudge=nudge) + checks += 1 + halvings = max(halvings, halving) + if found[life] <= tolerance or used >= budget or not np.isfinite(found[life]): + break + steps = max(steps, used) + error = self.residual(drive, current, mask=mask, nudge=nudge) + return Equilibrium( + current, error, steps, tolerance, residual_checks=checks + 2, + damping_halvings=halvings, stagnation_checks=stagnation_checks, + ) + + +class _LivesLearner(Learner): + """``Learner`` whose device update moves each life on its own contrast: the momentum, the + second moments and their counts are per life, a lesson judged by surprise runs its phases + on every life and updates the lives it taught, and the masks, ties and bounds read the last + axis.""" + + def _counts(self, lives: int) -> np.ndarray: + """Each life's own count of lessons that updated it, the optimizer's bias correction.""" + held = self.__dict__.get("lives_contrast_updates") + if held is None or len(held) != lives: + held = self.__dict__["lives_contrast_updates"] = np.zeros(lives, dtype=np.int64) + return cast(np.ndarray, held) + + def _applied(self, lives: int) -> np.ndarray: + """Each life's own count of updates applied, lessons and outcomes.""" + held = self.__dict__.get("lives_updates") + if held is None or len(held) != lives: + held = self.__dict__["lives_updates"] = np.zeros(lives, dtype=np.int64) + return cast(np.ndarray, held) + + def step( + self, drive: Any, labels: np.ndarray | None, warm: BrainState | None = None, + weight: np.ndarray | None = None, *, answer: np.ndarray | None = None, + targets: np.ndarray | None = None, + ) -> tuple[LearnedState, dict[str, float]]: + """``Learner.step`` for one row per life. Under ``surprise`` the caller's ``answer`` + judges each life by its own row; the phases run on every life, since a settle of these + lives holds one stream per life, and the update moves the lives whose answer differed. + ``LearnedState.taught`` marks them, and the states hold every life's row.""" + cfg = self.config + judged = None + if cfg.surprise and labels is not None: + if answer is None: + raise ValueError("a lesson of lives judged by surprise takes its caller's answer") + judged = self._differs(np.asarray(answer), self._labels(labels)) + if not judged.any(): + return self._untaught(judged) + self.__dict__["_taught_lives"] = judged + self.config = replace(cfg, surprise=False) # every row runs; the update reads the mark + try: + learned, report = super().step(drive, labels, warm, weight, targets=targets) + finally: + self.config = cfg + self.__dict__["_taught_lives"] = None + if judged is not None: + learned = LearnedState(learned.free, learned.nudged, learned.opposite, judged) + report.update(_taught_report(judged, len(judged))) + return learned, report + + def _update( + self, free: BrainState, nudged: BrainState, opposite: BrainState | None = None, + share: float = 1.0, + ) -> dict[str, float]: + self._validate_phases(free, nudged, opposite) + cfg = self.config + assert cfg.eta_bias is not None + minus_state, span = (free, cfg.beta) if opposite is None else (opposite, 2.0 * cfg.beta) + kernel = self._device_kernel(nudged, minus_state) + if kernel is None: + raise RuntimeError("the lives' phases rest on their device") + edges, neurons = kernel.contrast_tensors(nudged.device["s"], minus_state.device["s"]) + taught = self.__dict__.get("_taught_lives") + mask = None + if taught is not None and not taught.all(): + mask = kernel.torch.as_tensor(np.asarray(taught, dtype=bool), device=kernel.device) + edges, neurons = self._adaptive_lives(kernel, edges / span, neurons / span, mask) + step, step_bias = share * cfg.eta * edges, share * cfg.eta_bias * neurons + if mask is not None: # a life the lesson did not teach moves nothing + zero = kernel.torch.zeros((), dtype=step.dtype, device=step.device) + step = kernel.torch.where(mask[:, None], step, zero) + step_bias = kernel.torch.where(mask[:, None], step_bias, zero) + applied = self._applied(kernel.lives).copy() + report = self._apply_device(kernel, step, step_bias) + rows = slice(None) if taught is None else np.asarray(taught, dtype=bool) + self._counts(kernel.lives)[rows] += 1 + self.__dict__["lives_updates"] = applied # an update of every life counted the taught + applied[rows] += 1 + self.contrast_updates += 1 + return report + + def _install_device(self, candidate: dict[str, Any], steps: Any) -> dict[str, float]: + report = super()._install_device(candidate, steps) + kernel = self.brain._torch + assert kernel is not None + self._applied(kernel.lives)[:] += 1 + return report + + def _group_power(self, kernel: Any, values: Any, group: tuple[Any, ...], which: int) -> Any: + """Each life's sum of ``values`` squared per group of a projection table: every group in + a row of its own, summed in a fixed order, as the lone update sums it.""" + torch, lives = kernel.torch, kernel.lives + index, size, _, place, matrix = group + if place is None: + power = torch.zeros(lives, int(size), dtype=values.dtype, device=values.device) + return power.index_add_(1, index, values * values) + rooms = self.__dict__.setdefault("_lives_rooms", {}) + key = (id(kernel), which) + room = rooms.get(key) + if room is None or room.shape != (lives, matrix.numel()): + room = rooms[key] = torch.zeros( + lives, matrix.numel(), dtype=values.dtype, device=values.device + ) + room[:, place] = values * values + return room.view(lives, *matrix.shape).sum(dim=2) + + def _adaptive_lives(self, kernel: Any, edges: Any, neurons: Any, mask: Any) -> tuple[Any, Any]: + """``Learner._adaptive_device`` per life: each life's momentum and second moments, the + bias correction at its own count, the relative floor over its own projections. A life + ``mask`` leaves out keeps its history.""" + cfg = self.config + if not (cfg.momentum or cfg.normalize): + return edges, neurons + torch = kernel.torch + previous = self._moments_on_device(kernel) + held = {"holder": kernel, **{name: previous[name] for name in _MOMENTS}} + self.__dict__["_device_moments"] = held + count = self._counts(kernel.lives) + 1 + + def correction(factor: float) -> Any: + return torch.as_tensor( + 1.0 - np.power(factor, count), dtype=kernel.param_dtype, device=kernel.device + )[:, None] + + def kept(name: str, new: Any) -> Any: + held[name] = new if mask is None else torch.where(mask[:, None], new, previous[name]) + return new + + raw_edges, raw_neurons = edges, neurons + if cfg.momentum: + m = cfg.momentum + velocity = kept("velocity", previous["velocity"].mul(m).add_(edges, alpha=1.0 - m)) + velocity_bias = kept( + "velocity_bias", previous["velocity_bias"].mul(m).add_(neurons, alpha=1.0 - m) + ) + edges, neurons = velocity / correction(m), velocity_bias / correction(m) + if cfg.normalize: + rho = cfg.normalize + second = kept("second_moment", previous["second_moment"].mul(rho).addcmul_( + raw_edges, raw_edges, value=1.0 - rho + )) + second_bias = kept("second_moment_bias", previous["second_moment_bias"].mul( + rho + ).addcmul_(raw_neurons, raw_neurons, value=1.0 - rho)) + rms = (second / correction(rho)).sqrt() + rms_bias = (second_bias / correction(rho)).sqrt() + if cfg.normalize_relative: + divisors = [] + for which, (values, group) in enumerate(zip( + (rms, rms_bias), self._projections_on_device(kernel), strict=True + )): + power = self._group_power(kernel, values, group, which) + floor = (power / group[2]).sqrt()[:, group[0]] + divisors.append(values + cfg.normalize_relative * floor + _RELATIVE_TINY) + edges, neurons = edges / divisors[0], neurons / divisors[1] + else: + edges = edges / (rms + cfg.normalize_floor) + neurons = neurons / (rms_bias + cfg.normalize_floor) + ix = self._device_indices(kernel) # a rate of zero keeps the moments it held (``held``) + for name, stopped in (("velocity", "held_synapses"), ("second_moment", "held_synapses"), + ("velocity_bias", "held_neurons"), + ("second_moment_bias", "held_neurons")): + if ix[stopped] is not None and held[name] is not previous[name]: + held[name] = torch.where(ix[stopped], previous[name], held[name]) + if not bool(torch.stack([torch.isfinite(held[name]).all() for name in _MOMENTS]).all()): + raise ValueError("optimizer moments must remain finite") + return edges, neurons + + def _device_candidate(self, kernel: Any, delta_scale: Any, delta_bias: Any) -> dict[str, Any]: + torch, cfg = kernel.torch, self.config + lives = kernel.lives + if ( + tuple(delta_scale.shape) != (lives, self.brain.connectome.synapses) + or tuple(delta_bias.shape) != (lives, self.brain.connectome.n) + ): + raise ValueError("steps must be one per life and synapse and one per life and neuron") + given = torch.isfinite(delta_scale).all() & torch.isfinite(delta_bias).all() + ix = self._device_indices(kernel) + d = delta_scale.clone() + if ix["synapses"] is not None: + d = d * ix["synapses"] + if ix["rate"] is not None: + d = d * ix["rate"] + if ix["paired"] is not None: + d[:, ix["paired"]] = 0.5 * (d[:, ix["paired"]] + d[:, ix["paired_reverse"]]) + if ix["members"] is not None: + total = torch.zeros(lives, len(ix["member_count"]), dtype=d.dtype, device=d.device) + total.index_add_(1, ix["member_groups"], d[:, ix["members"]]) + d[:, ix["members"]] = (total / torch.clamp(ix["member_count"], min=1.0))[ + :, ix["member_groups"] + ] + if ix["synapses"] is not None: + d = d * ix["synapses"] + tied = torch.isfinite(d).all() + scale = kernel.scale + d + db = delta_bias if ix["neurons"] is None else delta_bias * ix["neurons"] + bias = kernel.bias_param + db + bounded = torch.clamp(scale, -cfg.scale_cap, cfg.scale_cap) + scale = ( + bounded if ix["synapses"] is None + else torch.where(ix["synapses"].bool(), bounded, kernel.scale) + ) + checked = kernel.parameter_check(scale, bias) + zero = torch.zeros((), dtype=d.dtype, device=d.device) + return { + "scale": scale, "bias": bias, "checked": checked, + "flags": (given, tied, checked[3], checked[4]), + "steps": (d.abs().mean() if d.numel() else zero, db.abs().mean()), + } + + +class _LivesArrays(TorchArrays): + """The array namespace of a lives kernel: a read back through the synapses reads each + stream through its own life's.""" + + def read_back( + self, graph: Any, ports: Sequence[tuple[np.ndarray, np.ndarray, np.ndarray, int]], + movement: Any, width: int, + ) -> list[Any]: + """``movement @ W.T`` per port, each row through its own life's ``W``.""" + kernel = graph._torch + torch, lives = self.torch, kernel.lives + if movement.shape[0] != lives: + raise ValueError("a read back of these lives takes one row per life") + out = [] + for port in ports: + placed = self.placed.get(id(port[0])) + if placed is None: + placed = self.placed[id(port[0])] = ( + port[0], self.long(port[0]), self.long(port[1]), self.long(port[2]), + ) + _, synapses, on_rows, on_cols = placed + values = (kernel.gain_pre[synapses] * kernel.scale[:, synapses]).to(movement.dtype) + w = torch.zeros((lives, port[3] * width), dtype=movement.dtype, device=self.device) + if graph.layout.parallel_synapses: + w.index_add_(1, on_rows * width + on_cols, values) + else: + w[:, on_rows * width + on_cols] = values + w = w.view(lives, port[3], width) + out.append(torch.bmm(w, movement[:, :, None])[:, :, 0]) + return out + + +def _finite_rows(torch: Any, *arrays: Any) -> Any: + """Per life, whether every entry of its row of every array is finite.""" + return torch.stack([ + torch.isfinite(a).reshape(a.shape[0], -1).all(dim=1) for a in arrays + ]).all(dim=0) + + +def _chosen(torch: Any, rows: Any, new: Any, old: Any) -> Any: + """``new`` at the lives ``rows`` marks, ``old`` at the others.""" + return torch.where(rows.reshape(-1, *([1] * (new.ndim - 1))), new, old) + + +class _LivesRecall(_RecallOnDevice): + """A recall memory's slots, maps and reads on the device, the maps one per life. + + A stream's slots, the cortex of the moment before and its reads are a stream's already + (``_RecallOnDevice``); here every address and position map carries a lives axis first, a + moment's address and position terms read each stream through its own life's maps, and the + address lesson and the reward's step move each life's maps on its own row. The arrays rest + on the device alone: ``Lives.parameters`` reads a life's maps.""" + + def __init__(self, recall: Any, kernel: Any, lone: Sequence[Any]) -> None: + super().__init__(recall, kernel) + here = self.xp.here + for name in _MAP_KINDS: + if getattr(self, name): + setattr(self, name, [tuple( + here(np.stack([np.asarray(getattr(one, name)[0][part]) for one in lone])) + for part in range(3) + )]) + + def give_back(self) -> None: + raise RuntimeError( + "the lives' recall memory rests on their device; Lives.parameters(i) and " + "Lives.life(i) read a life's maps" + ) + + def maps_of(self, life: int) -> dict[str, tuple[np.ndarray, ...]]: + """Life ``life``'s maps on the host, by kind.""" + host = self.xp.host + return { + name: tuple(host(a[life]) for a in getattr(self, name)[0]) + for name in _MAP_KINDS if getattr(self, name) + } + + # -- each stream through its own life's maps + + def _term(self, maps: Any, x: Any, c: Any) -> Any: + m_x, m_c, m = maps + if m_x.ndim == 2: # one number per life (the slope's lean), its bias ``(lives, 1)`` + return (x * m_x).sum(axis=1) + (c * m_c).sum(axis=1) + m[:, 0] + bmm = self.xp.bmm + return bmm(x[:, None, :], m_x)[:, 0] + bmm(c[:, None, :], m_c)[:, 0] + m + + def _moment(self, x: np.ndarray) -> _Moment: + recall, xp = self.recall, self.xp + inputs = self.kernel.observed(x) + n = len(x) + if n != self.kernel.root.lives: + raise ValueError("a moment of these lives takes one row per life") + c = self.previous if len(self.previous) == n else xp.zeros((n, len(recall.association))) + key = recall.heard * inputs + return _Moment(x, c, key + self._term(self.head_maps[0], inputs, c), key, inputs) + + def _lms(self, maps: Any, step: Any, rate: float, x: Any, c: Any) -> Any: + """``_lms`` per life: each life's maps on its own row's step, no mean over the rows.""" + m_x, m_c, m = maps + if m_x.ndim == 2: + step = step[:, None] + return (m_x + rate * x * step, m_c + rate * c * step, m + rate * step) + return ( + m_x + rate * x[:, :, None] * step[:, None, :], + m_c + rate * c[:, :, None] * step[:, None, :], + m + rate * step, + ) + + def _position_step(self, found: tuple[Any, ...]) -> dict[str, Any]: + """``_RecallOnDevice._position_step`` with the slope's rule read per life: past the + table a life's lean takes no step, and a life's own slots decide it (the bank is as wide + as the longest stream's).""" + recall, xp = self.recall, self.xp + stepped = super()._position_step(found) + if not (self.slope_maps and self.grid_maps and self.slots > recall.position): + return stepped + _, x, c, _, _, _, target, prior, _, _ = found + finite = xp.isfinite(prior) + gradient = xp.where(finite, target - xp.where(finite, xp.exp(prior), 0.0), 0.0) + energy = (x * x).sum(axis=1) + (c * c).sum(axis=1) + 1.0 + lean = gradient @ (self._ages() / recall.SLOPE) + lean = lean * xp.minimum(xp.ones_like(energy), 1.0 / (recall.slope * energy)) + new = self._lms(self.slope_maps[0], lean, recall.slope, x, c) + short = xp.flags(recall.held.sum(axis=1) <= recall.position) + torch = self.kernel.torch + stepped["slope_maps"] = tuple( + _chosen(torch, short, a, b) for a, b in zip(new, self.slope_maps[0], strict=True) + ) + return stepped + + def learn(self, graph: Any, learned: Any) -> dict[str, float]: + """``Recall.learn`` per life: the phases hold every life's row, each taught life whose + row moves steps its own maps, and a step that is not finite is refused for that life.""" + recall, xp = self.recall, self.xp + torch = self.kernel.torch + quiet = {"address_rows": 0.0, "address_error": 0.0, "address_refused": 0.0} + if self.position_maps: + quiet.update(position_rows=0.0, position_refused=0.0) + moment = recall.last + positions = bool(self.position_maps) and (recall.position_rate > 0 or recall.slope > 0) + if ( + not (recall.address or positions) + or moment is None + or learned.free is None + or learned.nudged is None + or learned.nudged is learned.free + ): + return quiet + lives = len(moment.address) + taught = np.ones(lives, dtype=bool) if learned.taught is None else np.asarray( + learned.taught, dtype=bool + ) + if not self.slots or not taught.any() or self.streams != lives: + return quiet + minus = learned.free if learned.opposite is None else learned.opposite + movement = self._movement(learned.nudged, minus) + d_value, d_next = self._readbacks(graph, movement) + found = self._row_steps(moment, np.arange(lives), d_value, d_next) + moving = found[0] & xp.flags(taught) + _, x, c, key, error, step = found[:6] + zero = torch.zeros((), dtype=error.dtype, device=error.device) + scalars = [moving.sum()] + head = moved = None + if recall.address: + w_x, w_c, b = self._lms(self.head_maps[0], step, recall.address, x, c) + address = key + self._term((w_x, w_c, b), x, c) + moved = moving & _finite_rows(torch, w_x, w_c, b, address) + head = tuple( + _chosen(torch, moved, a, old) + for a, old in zip((w_x, w_c, b), self.head_maps[0], strict=True) + ) + scalars += [moved.sum(), torch.where(moved, error.abs().mean(dim=1), zero).sum()] + placed: dict[str, tuple[Any, ...]] = {} + if positions: + stepped = self._position_step(found) + sound = moving & _finite_rows(torch, *[a for maps in stepped.values() for a in maps]) + for name, maps in stepped.items(): + placed[name] = tuple( + _chosen(torch, sound, a, old) + for a, old in zip(maps, getattr(self, name)[0], strict=True) + ) + scalars.append(sound.sum()) + read = xp.read(*scalars) + report = dict(quiet) + at = 1 + if head is not None: + self.head_maps = [head] + report["address_rows"] = float(read[at]) + report["address_error"] = float(read[at + 1] / read[at]) if read[at] else 0.0 + report["address_refused"] = float(read[0] - read[at]) + at += 2 + if positions: + for name, maps in placed.items(): + setattr(self, name, [maps]) + report["position_rows"] = float(read[at]) + report["position_refused"] = float(read[0] - read[at]) + return report + + # -- reward trains each life's read + + def weighted(self, trace: Any, delta: Any) -> Any: + """Each life's eligibility weighted by its own dopamine.""" + xp = self.xp + d = xp.floats(delta) if isinstance(delta, type(trace)) else xp.here( + np.asarray(delta, dtype=float) + ) + return d[:, None, None] * trace + + def proposal_unread(self, step: Any) -> tuple[Any, ...] | None: + """``_RecallOnDevice.proposal_unread`` of a step per life, ``(lives, inputs, columns)``.""" + recall, xp = self.recall, self.xp + positions = bool(recall.position_reward) and bool(self.position_maps) + if not (recall.address_reward or positions): + return None + w_x, w_c, b = self.head_maps[0] + stepped, changed = [], [] + start = recall.address_width if recall._address_held or recall.address_reward else 0 + if recall.address_reward: + moved = recall.address_reward * step[:, :, :recall.address_width] + w_x = w_x + moved + stepped.append(moved) + changed.append(w_x) + position_maps = anchor_maps = None + if positions: + count = recall.position_buckets + part = recall.position_reward * step[:, :, start : start + count] + on_x, on_c, on_b = self.position_maps[0] + position_maps = [(on_x + part, on_c, on_b)] + stepped.append(part) + changed.append(position_maps[0][0]) + start += count + if self.anchor_maps: + part = recall.position_reward * step[:, :, start : start + count + 1] + on_x, on_c, on_b = self.anchor_maps[0] + anchor_maps = [(on_x + part, on_c, on_b)] + stepped.append(part) + changed.append(anchor_maps[0][0]) + every = [*changed, *stepped] + norm = xp.sqrt(sum((part * part).sum() for part in stepped)) + return (w_x, w_c, b, xp.all_finite(*every), norm, every, position_maps, anchor_maps) + + +class _LivesHearing(Hearing): + """``Hearing`` with one gain per life and route. A route's memory holds its gains as a + ``(lives, 1)`` array on the device, which the memory's drive and the route's contrast read + as they read one gain; the second moments and the counts are each life's own.""" + + lives_gains: np.ndarray + lives_second: np.ndarray + lives_count: np.ndarray + kernel: Any + + @classmethod + def take(cls, hearing: Hearing, lone: Sequence[Hearing], kernel: Any) -> None: + """Make ``hearing`` the lives': each life's gains, second moments and count from its + lone brain's.""" + for route in hearing.routes: + if route["slot"] is not None: + raise ValueError("Lives holds the carried gains of the working trace and the read") + hearing.__class__ = cls + assert isinstance(hearing, cls) + hearing.kernel = kernel + hearing.lives_gains = np.array( + [[one.gain(route) for route in one.routes] for one in lone], dtype=float + ) + hearing.lives_second = np.stack([np.asarray(one.second, dtype=float) for one in lone]) + hearing.lives_count = np.array([one.count for one in lone], dtype=np.int64) + hearing._place() + + def _place(self) -> None: + """The gains where the device paths read them: ``(lives, 1)`` per route.""" + here = arrays_of(self.kernel).here + for k, route in enumerate(self.routes): + route["memory"].hearing = here(self.lives_gains[:, k : k + 1]) + + def gain(self, route: dict[str, Any]) -> Any: + return route["memory"].hearing + + def gains_of(self, life: int) -> dict[str, float]: + """Life ``life``'s gains by route.""" + return { + route["name"]: float(self.lives_gains[life, k]) for k, route in enumerate(self.routes) + } + + def step(self, contrast: np.ndarray) -> None: + raise RuntimeError("the lives' gains step per life (step_lives)") + + def step_lives(self, contrast: np.ndarray, rows: np.ndarray) -> None: + """``Hearing.step`` for the lives ``rows``, each on its own row of ``contrast``.""" + self.lives_count[rows] += 1 + second = self.RHO * self.lives_second[rows] + (1.0 - self.RHO) * contrast**2 + correction = 1.0 - np.power(self.RHO, self.lives_count[rows])[:, None] + moved = self.rate * contrast / (np.sqrt(second / correction) + self.FLOOR) + bound = np.array([route["bound"] for route in self.routes]) + self.lives_gains[rows] = np.clip(self.lives_gains[rows] + moved, 0.0, bound) + self.lives_second[rows] = second + self._place() + + def learn(self, graph: Any, learned: Any, beta: float) -> None: + """A lesson's step for the lives it taught; the phases hold every life's row.""" + if ( + not self.routes + or learned.free is None + or learned.nudged is None + or learned.nudged is learned.free + ): + return + minus_state = learned.free if learned.opposite is None else learned.opposite + moved = self._movement_on_device(graph, learned.nudged, minus_state) + if moved is None: + raise RuntimeError("the lives' phases and memories rest on their device") + rows = np.arange(len(moved)) if learned.taught is None else np.flatnonzero( + np.asarray(learned.taught, dtype=bool) + ) + if not len(rows): + return + span = beta if learned.opposite is None else 2.0 * beta + contrast = self._contrast_on_device(graph, moved / span, None, len(moved)) + self.step_lives(contrast[rows], rows) + + +class _LivesCritic(_CriticOnDevice): + """The critic on the device with one weight row and one bias per life.""" + + def _bias(self) -> Any: + return self.xp.here(np.asarray(self.owner.b_critic, dtype=float)) + + def value(self, state: BrainState) -> np.ndarray: + return self.xp.host(self.value_here(state)) + + def value_here(self, state: BrainState) -> Any: + part = self.xp.part(state, self.source) + return (part * self.weights).sum(axis=1) + self._bias() + + def unread(self, delta: Any) -> tuple[Any, Any, Any]: + trace = self.trace + assert trace is not None + xp = self.xp + d = xp.here(delta) if isinstance(delta, np.ndarray) else xp.floats(delta) + trace = trace / (1.0 + (trace * trace).sum(axis=1, keepdims=True)) + step = self.owner.config.eta_critic * (d[:, None] * trace) # each life its own row + weights = self.weights + step[:, :-1] + return weights, xp.all_finite(weights), step[:, -1] + + def candidate(self, delta: np.ndarray) -> tuple[Any, Any]: + weights, finite, step = self.unread(delta) + read = self.xp.host(self.xp.cat([finite.reshape(1).to(step.dtype), step], 0)) + bias = np.asarray(self.owner.b_critic, dtype=float) + read[1:] + if not read[0] or not np.isfinite(bias).all(): + raise ValueError("critic parameters must remain finite") + return weights, bias + + +class _LivesActorCritic(ActorCritic): + """``ActorCritic`` of lives, one stream per life: the critic rests on the device with a row + per life, and an outcome steps each life on its own stream's eligibility and dopamine.""" + + def reside(self, kernel: Any) -> None: + if kernel is None: + raise RuntimeError("the lives' critic rests on their device") + if self._critic_device is not None: + if not any(name in self.__dict__ for name in _CriticOnDevice.OWNED): + return + self.leave_device() + self._critic_device = _LivesCritic(self, kernel) + + def _hearing_step(self, delta: np.ndarray) -> None: + """Each life's carried gains stepped on its own trace by its own dopamine.""" + if self.trace_hearing is not None and self.hearing is not None: + self.hearing.step_lives( + delta[:, None] * self.trace_hearing, np.arange(len(delta)) + ) + + def _learn_device( + self, kernel: Any, plus: BrainState, minus: BrainState, value: np.ndarray, + reward: np.ndarray, done: np.ndarray, next_drive: np.ndarray, + bootstrap: np.ndarray | None, observed: np.ndarray, warm: BrainState | None = None, + ) -> dict[str, float]: + """``ActorCritic._learn_device`` per life: each stream's step is its own life's, with no + mean over the streams. The outcome's checks and the dopamine come to the host in one + read before anything is installed.""" + torch = kernel.torch + cfg = self.config + span = 2.0 * self.learner.config.beta + decay = cfg.gamma * cfg.lam + batch = len(reward) + if batch != kernel.lives or not observed.all(): + raise ValueError("an outcome of these lives has one observed stream per life") + s_plus, s_minus = plus.device["s"], minus.device["s"] + neurons = s_plus - s_minus + synapses = self.synapses + if self._trace_device is None or tuple(self._trace_device[0].shape) != (batch, synapses): + self._trace_device = ( + torch.zeros((batch, synapses), dtype=neurons.dtype, device=neurons.device), + torch.zeros_like(neurons), + ) + self.trace = self.trace_bias = None + trace, trace_bias = self._trace_device + trace_bias = decay * trace_bias + neurons / span + free = self._free + assert free is not None + self._critic_eligibility(free, batch) + self._memory_eligibility(batch, decay) + self._hearing_eligibility(batch, decay) + next_state = self._next_state(next_drive, done, warm) + resting = self._critic_device + assert isinstance(resting, _LivesCritic) + exact = torch.float32 if trace.device.type == "mps" else torch.float64 + + def here(values: Any) -> Any: + return torch.as_tensor(np.asarray(values, dtype=float), dtype=exact, + device=trace.device) + + following = resting.value_here(next_state).to(exact) + ends = torch.as_tensor(np.asarray(done), device=trace.device) + boot = torch.zeros_like(following) if bootstrap is None else here(bootstrap) + td_here = here(reward) + cfg.gamma * torch.where(ends, boot, following) - here(value) + delta_here = td_here + if cfg.dopamine_cap > 0: + delta_here = torch.clamp(delta_here, -cfg.dopamine_cap, cfg.dopamine_cap) + critic_weights, critic_finite, critic_step = resting.unread(delta_here) + d = delta_here.to(trace.dtype) + self.updates += 1 + # the recall memory's maps step on each life's own trace by its own dopamine + unread = None + traced, memory = self.trace_memory, self.memory + if traced is not None and getattr(memory, "_device", None) is not None: + unread = memory._device.proposal_unread(memory._device.weighted(traced, delta_here)) + one = torch.ones((), dtype=exact, device=trace.device) + mapped = (one, 0.0 * one) if unread is None else (unread[3], unread[4]) + next_brain = self.learner.brain + eta, eta_bias = self._rates() + step_bias = eta_bias * (d[:, None] * trace_bias) + step = decay * trace * d[:, None] + kernel.contrast_edges(s_plus, s_minus, d) / span + update = self.learner._device_candidate(kernel, eta * step, step_bias) + read = torch.cat([ + torch.stack([torch.isfinite(td_here).all().to(exact), critic_finite.to(exact), + mapped[0].to(exact), mapped[1].to(exact)] + + [flag.to(exact) for flag in update["flags"]] + + [number.to(exact) for number in update["steps"]]), + critic_step.to(exact), td_here, delta_here, + ]).cpu().numpy().astype(float) + if not read[0]: + raise ValueError("TD errors must remain finite") + critic_bias = np.asarray(self.b_critic, dtype=float) + read[10 : 10 + batch] + if not read[1] or not np.isfinite(critic_bias).all(): + raise ValueError("critic parameters must remain finite") + if not read[2]: + raise ValueError("the address map's reward step must be finite") + self.learner._device_refusal(read[4:8]) + td_error = read[10 + batch : 10 + 2 * batch] + delta = read[10 + 2 * batch :] + capped = self._capped(td_error, observed) + report = self.learner._install_device(update, read[8:10]) + trace.mul_(decay) + kernel.add_contrast(trace, s_plus, s_minus, span, None) + self._accept_critic((critic_weights, critic_bias)) + if unread is not None: + memory.accept((*unread[:3], float(read[3]), *unread[6:])) + report["address_reward_step"] = float(read[3]) + self._hearing_step(delta) + if done.any(): + ended = torch.as_tensor(np.flatnonzero(done), device=trace.device) + trace[ended] = 0.0 + trace_bias[ended] = 0.0 + self._forget_critic(done) + if self.trace_memory is not None: + self.trace_memory = _forgotten(self.trace_memory, done) + if self.trace_hearing is not None: + self.trace_hearing[done] = 0.0 + self._trace_device = (trace, trace_bias) + self._free = next_state + self._free_brain = next_brain + self._drive = _held_drive(next_drive) + self._pending = None + self._pending_memory = None + self._pending_hearing = None + report["delta"] = float(np.abs(delta).mean()) + report["dopamine"] = float(delta.mean()) + report["td_error"] = float(np.abs(td_error).mean()) + report["capped"] = capped + report["value"] = float(np.asarray(value).mean()) + report["free_steps"] = float(next_state.steps) + report["saturation"] = self._saturation(free) + report["trace"] = float(trace.abs().mean()) + return report + + +class _LivesRandom: + """Each life's own generator, drawn as its lone brain draws it: one row per life.""" + + def __init__(self, generators: Sequence[np.random.Generator]) -> None: + self.generators = list(generators) + + def random(self, shape: Any) -> np.ndarray: + shape = tuple(np.atleast_1d(shape)) + if not shape or shape[0] != len(self.generators): + raise ValueError("a draw of these lives takes one row per life") + return np.stack([g.random(shape[1:]) for g in self.generators]) + + +def _supported(brain: Any) -> None: + """Refuse a composition with a part the lives axis does not cover, naming it.""" + + def refuse(what: str) -> ValueError: + return ValueError(f"Lives holds the parts its lives axis covers: {what}") + + if brain.hippocampus is not None: + raise refuse( + "compose with episodic=False (the store's consolidated records have no lives axis)" + ) + for name, what in ( + ("arousal", "no arousal (Lives steps, acts and teaches; live is a lone brain's)"), + ("learned_code", "no learned code"), + ("_episodic", "no memory_choice"), + ("efference", "no efference copy"), + ("history", "no heard history"), + ): + if getattr(brain, name, None) is not None: + raise refuse(what) + if getattr(brain, "_coded_keys", False): + raise refuse("no code of the observation (a code is drawn from a brain's seed)") + recall = brain.recall + if recall is not None and not recall.resides: + raise refuse( + "a recall memory of one head, without a window, the command, the read of when, " + "tokens, their gain or the cortex at unit norm" + ) + hearing = brain.hearing + if hearing is not None and any(route["slot"] is not None for route in hearing.routes): + raise refuse("the carried gains of the working trace and the read") + learner, graph = brain.learner, brain.brain + cfg = learner.config + if cfg.rehearsal or cfg.qualified: + raise refuse("no rehearsal (learning_rehearsal=0) and no qualified teaching") + if graph.apical is not None: + raise refuse("no apical compartment") + agent = brain.basal_ganglia + if agent.maturity is not None or agent.salience is not None: + raise refuse("no maturity and no salience") + + +class Lives: + """Many continuing lives of one composed anatomy, each a brain of its own, stepped together. + + ``Lives(inputs, actions, lives=128, backend="torch")`` composes ``lives`` brains of + ``Brain.compose(inputs, actions, seed=seeds[i], episodic=False, **options)``, the default + composition without the store, one per seed (``range(lives)`` by default). Every operation + takes one row of observations per life and returns one answer per life: ``act``, ``step``, + ``lesson`` and ``learn`` mean what they mean on a lone brain with one stream, life by life. + ``life(i)`` is life ``i`` as a lone brain on the CPU, with everything it learned. + + On ``backend="cpu"`` each life is its own ``Brain`` and the lives run in turn, to the bit as + lone brains. On ``backend="torch"`` (CUDA where there is a card, else the torch CPU device, or + the ``device`` named) they are the streams of one brain whose parameters carry a lives axis, + and each moment of all of them is one settle and one update; each life agrees with its lone + brain on that device within the solver's tolerance. The lives share their anatomy and nothing + they learn. A moment any life refuses (a settle that does not qualify, a step that is not + finite) is refused for every life. + """ + + def __init__( + self, inputs: int, actions: int, lives: int, *, seeds: Sequence[int] | None = None, + backend: str = "cpu", device: str | None = None, precision: str | None = None, + **options: Any, + ) -> None: + from .generic import Brain + + if lives < 1: + raise ValueError("lives must be at least one") + self.seeds = list(range(lives)) if seeds is None else [int(s) for s in seeds] + if len(self.seeds) != lives: + raise ValueError("one seed per life") + if backend not in ("cpu", "torch"): + raise ValueError("Lives runs on backend='cpu' or backend='torch'") + if not isinstance(inputs, (int, np.integer)): + raise ValueError("Lives takes a vector of inputs, its length") + options = {"episodic": False, **options} + self.inputs, self.actions, self.backend = int(inputs), int(actions), backend + self._options = options + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + lone = [Brain.compose(inputs, actions, seed=s, backend="cpu", **options) + for s in self.seeds] + _supported(lone[0]) + topology = lone[0].connectome + for other in lone[1:]: + if not (np.array_equal(other.connectome.pre, topology.pre) + and np.array_equal(other.connectome.post, topology.post)): + raise ValueError("the lives' seeds compose different anatomies") + if backend == "cpu": + self.brains: list[Any] | None = lone + self.brain: Any = None + return + self.brains = None + place: dict[str, Any] = {"backend": "torch"} + if device is None: # CUDA where there is a card, else the torch CPU device (never MPS) + import torch + + device = "cuda" if torch.cuda.is_available() else "cpu" + place["device"] = device + if precision is not None: + place["precision"] = precision + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + brain = Brain.compose(inputs, actions, seed=self.seeds[0], **place, **options) + graph = brain.brain + kernel = graph._torch + assert kernel is not None + if kernel.device.type == "mps": + raise ValueError( + "Lives checks its settles in float64 on its device: CUDA or the torch CPU device " + "(MPS has no float64)" + ) + efficacy = np.stack([b.brain.efficacy for b in lone]) + bias = np.stack([b.brain.bias for b in lone]) + lives_kernel = _LivesTorchKernel( + graph.connectome, efficacy, kernel.gain_pre, bias, graph.neuron_model, + str(kernel.device), graph.layout, + "float32" if kernel.dtype == kernel.torch.float32 else "float64", + ) + graph._torch = lives_kernel + graph.__class__ = _LivesGraph + for name in ("_efficacy", "_bias", "_weights_host", "_flat", "_csr"): + if hasattr(graph, name): + setattr(graph, name, None) + learner = brain.learner + learner.__class__ = _LivesLearner + synapses, n = graph.connectome.synapses, graph.connectome.n + learner.__dict__["_device_moments"] = None + for name, width in (("velocity", synapses), ("velocity_bias", n), + ("second_moment", synapses), ("second_moment_bias", n)): + learner.__dict__["_" + name] = np.zeros((lives, width)) + agent = brain.basal_ganglia + agent.leave_device() + agent.__class__ = _LivesActorCritic + agent.w_critic = np.stack([b.basal_ganglia.w_critic for b in lone]) + agent.b_critic = np.array( # type: ignore[assignment] # one bias per life + [b.basal_ganglia.b_critic for b in lone], dtype=float + ) + agent.rng = _LivesRandom( # type: ignore[assignment] # each life's own generator + [b.basal_ganglia.rng for b in lone] + ) + # the memories' device paths read each stream through its own life's synapses and maps + lives_kernel.__dict__["_arrays"] = _LivesArrays(lives_kernel) + if brain.recall is not None: + brain.recall._device = _LivesRecall( + brain.recall, lives_kernel, [b.recall for b in lone] + ) + if brain.hearing is not None: + _LivesHearing.take( + brain.hearing, [cast(Hearing, b.hearing) for b in lone], lives_kernel + ) + self.brain = brain + + @property + def lives(self) -> int: + return len(self.seeds) + + def _rows(self, observations: Any) -> np.ndarray: + x = np.asarray(observations, dtype=float) + if x.shape != (self.lives, self.inputs): + raise ValueError(f"one row of {self.inputs} observations per life ({self.lives})") + return x + + def _each(self, name: str, x: np.ndarray, *rest: Any, **named: Any) -> list[Any]: + assert self.brains is not None + out = [] + for i, brain in enumerate(self.brains): + args = [None if r is None else np.asarray(r)[i : i + 1] for r in rest] + kwargs = {k: None if v is None else np.asarray(v)[i : i + 1] + for k, v in named.items()} + out.append(getattr(brain, name)(x[i : i + 1], *args, **kwargs)) + return out + + def act(self, observations: Any, *, greedy: bool = False) -> np.ndarray: + """Each life's action on its own row: ``Brain.act`` life by life.""" + x = self._rows(observations) + if self.brains is not None: + return np.concatenate([b.act(x[i : i + 1], greedy=greedy) + for i, b in enumerate(self.brains)]) + return np.asarray(self.brain.act(x, greedy=greedy)) + + def step( + self, observations: Any, *, reward: Any = None, done: Any = None, teacher: Any = None, + ) -> np.ndarray: + """Each life's ``Brain.step`` on its own row, its own reward, its own ending and, with + ``teacher``, its own label taught before the act.""" + x = self._rows(observations) + if self.brains is not None: + return np.concatenate( + self._each("step", x, reward=reward, done=done, teacher=teacher) + ) + return np.asarray(self.brain.step(x, reward=reward, done=done, teacher=teacher)) + + def learn(self, reward: Any, done: Any, next_observations: Any) -> None: + """Each life's ``Brain.learn`` from its own outcome.""" + x = self._rows(next_observations) + if self.brains is not None: + for i, b in enumerate(self.brains): + b.learn(np.asarray(reward)[i : i + 1], np.asarray(done)[i : i + 1], x[i : i + 1]) + return + self.brain.learn(reward, done, x) + + def lesson(self, observations: Any, labels: Any) -> None: + """Each life's ``Brain.lesson`` of its own label on its own row.""" + x = self._rows(observations) + y = np.asarray(labels) + if self.brains is not None: + for i, b in enumerate(self.brains): + b.lesson(x[i : i + 1], y[i : i + 1]) + return + self.brain.lesson(x, y) + + def reset(self) -> None: + """Every life's stream starts afresh, as ``Brain.reset``; what the lives learned stays.""" + if self.brains is not None: + for brain in self.brains: + brain.reset() + return + self.brain.reset() + + def parameters(self, i: int) -> dict[str, Any]: + """Life ``i``'s learned parameters on the host, as a lone brain holds them: its + ``efficacy`` and ``bias``; the critic's ``w_critic`` and ``b_critic``; the optimizer's + ``velocity``, ``velocity_bias``, ``second_moment`` and ``second_moment_bias`` and its + counts ``updates`` and ``contrast_updates``; with a recall memory ``recall``, its maps + by kind; with carried gains ``carried_gains`` by route, their ``carried_second`` and + ``carried_count``.""" + if self.brains is not None: + b = self.brains[i] + learner = b.learner + out = { + "efficacy": np.array(b.brain.efficacy), "bias": np.array(b.brain.bias), + "w_critic": np.array(b.basal_ganglia.w_critic), + "b_critic": np.array(b.basal_ganglia.b_critic, dtype=float), + **{name: np.array(getattr(learner, name)) for name in _MOMENTS}, + "updates": int(learner.updates), + "contrast_updates": int(learner.contrast_updates), + } + if b.recall is not None: + out["recall"] = { + name: tuple(np.array(a) for a in getattr(b.recall, name)[0]) + for name in _MAP_KINDS if getattr(b.recall, name) + } + if b.hearing is not None: + out["carried_gains"] = b.hearing.gains() + out["carried_second"] = np.array(b.hearing.second, dtype=float) + out["carried_count"] = int(b.hearing.count) + return out + brain = self.brain + kernel = brain.brain._torch + agent = brain.basal_ganglia + learner = brain.learner + moments = learner._moments_on_device(kernel) + resting = agent._critic_device + weights = (resting.xp.host(resting.weights[i]) if resting is not None + else np.asarray(agent.w_critic)[i]) + out = { + "efficacy": kernel.scale[i].detach().cpu().double().numpy(), + "bias": kernel.bias_param[i].detach().cpu().double().numpy(), + "w_critic": weights, "b_critic": np.asarray(agent.b_critic, dtype=float)[i], + **{name: moments[name][i].detach().cpu().double().numpy() for name in _MOMENTS}, + "updates": int(learner._applied(self.lives)[i]), + "contrast_updates": int(learner._counts(self.lives)[i]), + } + if brain.recall is not None: + out["recall"] = brain.recall._device.maps_of(i) + if brain.hearing is not None: + out["carried_gains"] = brain.hearing.gains_of(i) + out["carried_second"] = brain.hearing.lives_second[i].copy() + out["carried_count"] = int(brain.hearing.lives_count[i]) + return out + + def life(self, i: int) -> Any: + """Life ``i`` as a lone brain on the CPU: the composition of its seed with its learned + parameters, optimizer, critic, recall maps, carried gains and random state; its stream + starts afresh. Save it as any brain.""" + from .generic import Brain + + if self.brains is not None: + return self.brains[i] + held = self.parameters(i) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + lone = Brain.compose(self.inputs, self.actions, seed=self.seeds[i], backend="cpu", + **self._options) + lone.learner.brain = lone.learner.brain.with_parameters( + efficacy=held["efficacy"], bias=held["bias"] + ) + for name in _MOMENTS: + setattr(lone.learner, name, held[name]) + lone.learner.updates = held["updates"] + lone.learner.contrast_updates = held["contrast_updates"] + lone.basal_ganglia.updates = self.brain.basal_ganglia.updates + lone.basal_ganglia.w_critic = held["w_critic"] + lone.basal_ganglia.b_critic = float(held["b_critic"]) + lone.basal_ganglia.rng.bit_generator.state = ( + self.brain.basal_ganglia.rng.generators[i].bit_generator.state + ) + if lone.recall is not None: + for name, maps in held["recall"].items(): + setattr(lone.recall, name, [maps]) + if lone.hearing is not None: + for route in lone.hearing.routes: + lone.hearing.set(route, held["carried_gains"][route["name"]]) + lone.hearing.second = held["carried_second"] + lone.hearing.count = held["carried_count"] + return lone diff --git a/src/cadence/memory.py b/src/cadence/memory.py index d01e98d5..22b4817a 100644 --- a/src/cadence/memory.py +++ b/src/cadence/memory.py @@ -42,6 +42,17 @@ class SynapticMemory(FastSynapses): without the gate. Every other stream keeps its residual. Each writing row moves the noise of the records its cue reads toward its squared error, at ``GATE_RATE`` times the cue's squared weight on the record relative to its largest. + + With a fade (``fade`` above 0, ``Brain.compose(memory_fade=...)``), records nobody uses fade + (#253). Each key coordinate ``k`` keeps a depth, the sum of the unit cue's magnitude + ``|c_k|`` over every moment that cues it and every write that uses it, and a trace of its + recent cueing, the moments' sum kept at ``1 - 1 / FADE_IDLE`` a moment. Every moment of life + (``fade_moment``) the records along coordinate ``k`` keep + ``1 - fade * i_k / (1 + (depth_k / FADE_SCALE) ** FADE_ORDER)`` of themselves, + ``i_k = 1 / (1 + (recent_k / FADE_FLOOR) ** 2)`` the coordinate's idleness: a record in use + does not fade, one idle for some fifteen thousand moments starts to, and a deeply used one + fades far more slowly. Each stream keeps its residual around the shared records. No count is + kept: the depth and the trace are sums of activity, as maturity's use is. """ decay: float = 0.9 @@ -50,9 +61,19 @@ class SynapticMemory(FastSynapses): consolidated: np.ndarray = field(init=False) gate_noise: np.ndarray | None = field(default=None, repr=False) maturity: Maturity | None = field(default=None, repr=False, compare=False) + fade: float = 0.0 + fade_depth: np.ndarray | None = field(default=None, repr=False) + fade_recent: np.ndarray | None = field(default=None, repr=False) GATE_WIDTH = 2.0 GATE_RATE = 0.05 + # the fade's constants (``fade``): the depth at which a record fades at half the rate, the + # law's order, the moments the recent trace keeps ``1 - 1 / FADE_IDLE`` over, and the trace + # below which a coordinate counts as idle + FADE_SCALE = 2000.0 + FADE_ORDER = 4.0 + FADE_IDLE = 2000.0 + FADE_FLOOR = 0.01 def __post_init__(self) -> None: super().__post_init__() @@ -66,11 +87,46 @@ def __post_init__(self) -> None: "set separator.center=0 and apply any fixed centering to the input" ) self.consolidated = np.zeros((self.key_width, len(self.post))) + if not np.isfinite(self.fade) or not 0 <= self.fade < 1: + raise ValueError("memory_fade must lie in [0, 1)") + self.fade = float(self.fade) + if self.fade and self.fade_depth is None: + self.fade_depth = np.zeros(self.key_width) + self.fade_recent = np.zeros(self.key_width) + + def fade_moment(self, key: np.ndarray | None) -> None: + """One moment of life of a fading store (``fade``): the coordinates ``key`` cues + (``(batch, pre)``, the moment's place key; None for a moment that cues none) deepen and + are marked recent, then every idle record fades by its depth's law and the recent trace + decays one moment.""" + if not self.fade: + return + assert self.fade_depth is not None and self.fade_recent is not None + if key is not None: + k = np.atleast_2d(np.asarray(key, dtype=float)) + if k.shape[1] == len(self.pre): + k = k[np.any(k != 0, axis=1)] + if len(k): + cue = self.separator.code(k) if self.separator is not None else k + used = np.abs(self._delta_unit(cue)).sum(axis=0) + self.fade_depth = self.fade_depth + used + self.fade_recent = self.fade_recent + used + idle = 1.0 / (1.0 + (self.fade_recent / self.FADE_FLOOR) ** 2) + lost = self.fade * idle / (1.0 + (self.fade_depth / self.FADE_SCALE) ** self.FADE_ORDER) + if self._device is not None: # a fading store rests on the host (``reside``) + self.leave_device() + shed = self.consolidated * lost[:, None] + self.consolidated = self.consolidated - shed + self.strength = self.strength - shed # each stream keeps its residual + self.fade_recent = self.fade_recent * (1.0 - 1.0 / self.FADE_IDLE) def reside(self, kernel: Any) -> None: - """The strengths on the device of a brain's ``kernel``, unless a separator or a surprise - gate keeps them on the host.""" - if kernel is None or self.separator is not None or self.gate_noise is not None: + """The strengths on the device of a brain's ``kernel``, unless a separator, a surprise + gate or a fade keeps them on the host.""" + if ( + kernel is None or self.separator is not None or self.gate_noise is not None + or self.fade + ): self.leave_device() return held = self._device @@ -108,6 +164,9 @@ def arrays(self) -> dict[str, np.ndarray]: """The arrays a checkpoint keeps, the consolidated records among them (and the records' noise scales, with a surprise gate).""" kept = {} if self.gate_noise is None else {"gate_noise": self.gate_noise} + if self.fade: + assert self.fade_depth is not None and self.fade_recent is not None + kept.update(fade_depth=self.fade_depth, fade_recent=self.fade_recent) return {**super().arrays(), "consolidated": self.consolidated, **kept} def clear(self) -> None: @@ -115,6 +174,9 @@ def clear(self) -> None: self.consolidated.fill(0) if self.gate_noise is not None: self.gate_noise.fill(0) + if self.fade_depth is not None and self.fade_recent is not None: + self.fade_depth = np.zeros_like(self.fade_depth) + self.fade_recent = np.zeros_like(self.fade_recent) self.reset(len(self.strength)) def _baseline(self) -> np.ndarray: @@ -215,6 +277,9 @@ def observe( if noise is not None: self.gate_noise = noise self.writes += len(rows) + if self.fade: # a write uses the records its cue reads: the writing rows' unit cues + assert self.fade_depth is not None + self.fade_depth = self.fade_depth + np.abs(cue).sum(axis=0) def _gate( self, cue: np.ndarray, value: np.ndarray, previous: np.ndarray, mask: np.ndarray @@ -243,7 +308,9 @@ def _gate( return gain, noise def to_dict(self) -> dict[str, float | int | str]: - gate = {} if self.gate_noise is None else {"surprise_gate": True} + gate: dict[str, Any] = {} if self.gate_noise is None else {"surprise_gate": True} + if self.fade: + gate["fade"] = self.fade return { **super().to_dict(), "kind": "consolidating", diff --git a/src/cadence/plasticity.py b/src/cadence/plasticity.py index a9ed5df8..3a589d52 100644 --- a/src/cadence/plasticity.py +++ b/src/cadence/plasticity.py @@ -530,7 +530,7 @@ def _hearing_step(self, delta: np.ndarray) -> None: if self.trace_hearing is not None and self.hearing is not None: self.hearing.step((delta[:, None] * self.trace_hearing).mean(axis=0)) - def _memory_proposal(self, delta: np.ndarray) -> Any: + def _memory_proposal(self, delta: Any) -> Any: """The memory's parameters stepped on their trace by the dopamine, as every synapse steps: proposed and checked by the memory, not installed. None without a trace, or when the memory takes no step.""" @@ -539,6 +539,8 @@ def _memory_proposal(self, delta: np.ndarray) -> Any: return None if not isinstance(trace, np.ndarray): # the memory's on the device return self.memory.propose(self.memory._device.weighted(trace, delta)) + if not isinstance(delta, np.ndarray): # a dopamine made on the device, to the host's trace + delta = np.asarray(delta.detach().cpu().numpy(), dtype=float) weighted = delta.reshape(-1, *([1] * (trace.ndim - 1))) * trace return self.memory.propose(weighted.mean(axis=0)) @@ -987,23 +989,61 @@ def _learn_device( self._memory_eligibility(batch, decay) self._hearing_eligibility(batch, decay) next_state = self._next_state(next_drive, done, warm) - next_value_raw = self.value(next_state) - next_value = np.where( - done, 0.0 if bootstrap is None else np.asarray(bootstrap, dtype=float), next_value_raw - ) - raw_target = reward + cfg.gamma * next_value - td_error = raw_target - value - if not np.isfinite(td_error).all(): - raise ValueError("TD errors must remain finite") - delta = td_error - capped = self._capped(delta, observed) - if cfg.dopamine_cap > 0: - delta = np.clip(delta, -cfg.dopamine_cap, cfg.dopamine_cap) - delta = np.where(observed, delta / observed.mean(), 0.0) + resting = self._critic_device + if resting is None: # the critic on the host reads the dopamine there + next_value_raw = self.value(next_state) + next_value = np.where( + done, 0.0 if bootstrap is None else np.asarray(bootstrap, dtype=float), + next_value_raw, + ) + raw_target = reward + cfg.gamma * next_value + td_error = raw_target - value + if not np.isfinite(td_error).all(): + raise ValueError("TD errors must remain finite") + delta = td_error + capped = self._capped(delta, observed) + if cfg.dopamine_cap > 0: + delta = np.clip(delta, -cfg.dopamine_cap, cfg.dopamine_cap) + delta = np.where(observed, delta / observed.mean(), 0.0) + critic = self._critic_candidate(delta) + d = torch.as_tensor( + np.asarray(delta, dtype=float), dtype=trace.dtype, device=trace.device + ) + else: + # the dopamine on the device, from the value the critic made there: the outcome's + # checks, the critic's and the update's, and the dopamine come to the host in one read + # before anything is installed + exact = torch.float32 if trace.device.type == "mps" else torch.float64 + + def here(values: Any) -> Any: + return torch.as_tensor(np.asarray(values, dtype=float), dtype=exact, + device=trace.device) + + following = resting.value_here(next_state).to(exact) + ends = torch.as_tensor(np.asarray(done), device=trace.device) + boot = torch.zeros_like(following) if bootstrap is None else here(bootstrap) + td_here = here(reward) + cfg.gamma * torch.where(ends, boot, following) - here(value) + delta_here = td_here + if cfg.dopamine_cap > 0: + delta_here = torch.clamp(delta_here, -cfg.dopamine_cap, cfg.dopamine_cap) + shown = torch.as_tensor(np.asarray(observed), device=trace.device) + delta_here = torch.where( + shown, delta_here / float(observed.mean()), torch.zeros_like(delta_here) + ) + critic_weights, critic_finite, critic_step = resting.unread(delta_here) + d = delta_here.to(trace.dtype) self.updates += 1 - critic = self._critic_candidate(delta) - d = torch.as_tensor(np.asarray(delta, dtype=float), dtype=trace.dtype, device=trace.device) - proposal = self._memory_proposal(delta) # checked before anything is installed + proposal = unread = None + traced, memory = self.trace_memory, self.memory + # the memory's trace on the device, as ``_memory_proposal`` finds it there + on_device = traced is not None and not isinstance(traced, np.ndarray) + if resting is not None and on_device: + # the memory's step on the device too, its check read with the outcome's + unread = memory._device.proposal_unread(memory._device.weighted(traced, delta_here)) + else: + proposal = self._memory_proposal( # checked before anything is installed + delta if resting is None else delta_here + ) next_brain = self.learner.brain eta, eta_bias = self._rates() step_bias = eta_bias * (d[:, None] * trace_bias).mean(dim=0) @@ -1020,7 +1060,33 @@ def _learn_device( for first, stop, part in kernel.contrast_parts(s_plus, s_minus): part = (part * salience[first:stop][:, kernel._row_index[0]]).div_(span) step += torch.mv(part.T, d[first:stop]) - report = self.learner._apply_device(kernel, eta * (step / batch), step_bias) + if resting is None: + report = self.learner._apply_device(kernel, eta * (step / batch), step_bias) + else: + update = self.learner._device_candidate(kernel, eta * (step / batch), step_bias) + one = torch.ones((), dtype=exact, device=trace.device) + mapped = (one, 0.0 * one) if unread is None else (unread[3], unread[4]) + read = torch.cat([ + torch.stack([torch.isfinite(td_here).all().to(exact), critic_finite.to(exact), + critic_step.to(exact), mapped[0].to(exact), mapped[1].to(exact)] + + [flag.to(exact) for flag in update["flags"]] + + [number.to(exact) for number in update["steps"]]), + td_here, delta_here, + ]).cpu().numpy().astype(float) + if not read[0]: + raise ValueError("TD errors must remain finite") + critic_bias = self.b_critic + float(read[2]) + if not read[1] or not np.isfinite(critic_bias): + raise ValueError("critic parameters must remain finite") + if not read[3]: + raise ValueError("the address map's reward step must be finite") + if unread is not None: + proposal = (*unread[:3], float(read[4]), *unread[6:]) + self.learner._device_refusal(read[5:9]) + td_error, delta = read[11 : 11 + batch], read[11 + batch :] + capped = self._capped(td_error, observed) + critic = (critic_weights, critic_bias) + report = self.learner._install_device(update, read[9:11]) trace.mul_(decay) kernel.add_contrast(trace, s_plus, s_minus, span, salience) self._accept_critic(critic) @@ -1127,8 +1193,35 @@ def _learn_mlx( delta = np.clip(delta, -cfg.dopamine_cap, cfg.dopamine_cap) delta = np.where(observed, delta / observed.mean(), 0.0) self.updates += 1 - critic = self._critic_candidate(delta) - proposal = self._memory_proposal(delta) # checked before anything is installed + resting, traced, memory = self._critic_device, self.trace_memory, self.memory + made: list[Any] = [] # made in the update's launch, read after it + if resting is not None: + unread_critic = resting.unread(delta) + made += [unread_critic[1], unread_critic[2], unread_critic[0]] + else: + critic = self._critic_candidate(delta) + unread = None + on_device = traced is not None and not isinstance(traced, np.ndarray) + if on_device: # the memory's trace on the device, as ``_memory_proposal`` finds it there + unread = memory._device.proposal_unread(memory._device.weighted(traced, delta)) + if unread is not None: + made += [unread[3], unread[4], *unread[5]] + else: + proposal = self._memory_proposal(delta) # checked before anything is installed + found: dict[str, Any] = {} + + def checked() -> None: + """The critic's and the memory's checks, in their order, from the update's launch.""" + if resting is not None: + bias = self.b_critic + float(unread_critic[2].item()) + if not bool(unread_critic[1].item()) or not np.isfinite(bias): + raise ValueError("critic parameters must remain finite") + found["critic"] = (unread_critic[0], bias) + if unread is not None: + if not bool(unread[3].item()): + raise ValueError("the address map's reward step must be finite") + found["proposal"] = (*unread[:3], float(unread[4].item()), *unread[6:]) + next_brain = learner.brain eta, eta_bias = self._rates() scale = kernel.scale if kernel.scale is not None else kernel.to_blocks(next_brain.efficacy) @@ -1136,8 +1229,12 @@ def _learn_mlx( new, bias_trace, size, bias_size = kernel.rewarded( plus.device, minus.device, None if held is None else held[0], None if held is None else held[1], decays, span, delta, eta, eta_bias, scale, tables, - learner.config.scale_cap, + learner.config.scale_cap, also=made, before=checked, ) + if resting is not None: + critic = found["critic"] + if on_device: + proposal = found.get("proposal") # nothing refuses from here: what alone holds the eligibility lets it be written over old = None if held is None else list(held[0]) held = self._trace_mlx = None diff --git a/src/cadence/resident.py b/src/cadence/resident.py index 5009225e..370b3e2f 100644 --- a/src/cadence/resident.py +++ b/src/cadence/resident.py @@ -42,6 +42,8 @@ def _memory_on_device(kernel: Any, streams: int, width: int) -> bool: and on MLX from the size that pays for it. ``CADENCE_TORCH_MEMORY=1`` places them at any size on any torch device but MPS (float32 alone), ``CADENCE_MLX_MEMORY=1`` on MLX at any size, and ``=0`` keeps them on the host.""" + if getattr(kernel, "lives", 0): # lives keep their critic and memories on their device + return True if hasattr(kernel, "mx"): choice = os.environ.get("CADENCE_MLX_MEMORY", "") if choice == "0" or kernel.plain: @@ -76,11 +78,22 @@ def _memory_on_device(kernel: Any, streams: int, width: int) -> bool: _MLX_SWEEP_ROWS = 32 # Reasoned, not read at the rule's size. Float64 named on CUDA: that device does float64 at a small # part of its float32 rate. Memories that stay on the host (``Brain._memories_stay_on_host``): a -# moment crosses. A card the table has not read: a stated, conservative margin. +# moment crosses. A card the table has not read and that has no calibration: a stated margin. _AUTO_CUDA_FLOAT64 = 4 _AUTO_HOST_MEMORY = 4 _AUTO_UNSEEN = 2 _AUTO_READ_CARDS = ("A10G",) +# A card the table has not read is calibrated once (``_card_calibration``) where a brain's work +# reaches this fraction of the table's crossing: the chamber's default brain is timed on the card +# and on one core at these rows, and where the two cross, in ``rows * synapses``, scales the +# table's crossings by its ratio to the A10G's crossing read by the same calibration. +_CALIBRATION_FROM = 0.25 +_CALIBRATION_ROWS = (16, 64, 256) +# The A10G's crossing on the chamber's brain, by the calibration's interpolation of the default +# backend's table (docs/backends.md): 43.7 rows of 13,120 synapses. +_CALIBRATION_A10G = 573_568.0 +_CALIBRATIONS: dict[str, dict[str, Any]] = {} # by card, this process's +_CALIBRATING = False # a calibration in progress: its own brains name their backends class _Candidate(NamedTuple): @@ -134,10 +147,11 @@ def _fastest_backends( needed = (_AUTO_CUDA_APICAL if apical else _AUTO_CUDA_WORK) * margin if floats == "float64": needed *= _AUTO_CUDA_FLOAT64 - if not learned_code and work >= needed: + if not learned_code and work >= needed * _CALIBRATION_FROM: card = _cuda_card() if card is not None and not any(name in card for name in _AUTO_READ_CARDS): - needed *= _AUTO_UNSEEN + calibration = _card_calibration(card) + needed *= _AUTO_UNSEEN if calibration is None else calibration["scale"] if card is not None and work >= needed: out.append(_Candidate( "torch", "cuda", floats, f"{said} are past where {card} passes one CPU core" @@ -147,6 +161,100 @@ def _fastest_backends( return out +def _crossing(rows: Sequence[int], ratios: Sequence[float]) -> float: + """The rows at which the device's time over the core's crosses one, by straight lines in the + logarithms of both between the readings, and past them along the nearest two.""" + x, y = np.log(np.asarray(rows, dtype=float)), np.log(np.asarray(ratios, dtype=float)) + for k in range(len(x) - 1): + if y[k] >= 0.0 >= y[k + 1] and y[k] != y[k + 1]: + return float(np.exp(x[k] + (x[k + 1] - x[k]) * y[k] / (y[k] - y[k + 1]))) + k = 0 if y[0] < 0.0 else len(x) - 2 + slope = (y[k + 1] - y[k]) / (x[k + 1] - x[k]) + if slope >= 0.0: # the device does not gain on the core with rows: it never passes it + return float("inf") if y[0] >= 0.0 else float(rows[0]) + return float(np.exp(x[k] - y[k] / slope)) + + +def calibrate_card(rows: Sequence[int] = _CALIBRATION_ROWS) -> dict[str, Any]: + """Time the chamber's default brain, ``Brain.compose(35, 18)``, on this machine's CUDA card in + float32 and on one CPU core, at each of ``rows``: a greedy ``act``, a ``lesson`` and a ``step`` + that learns from the outcome of the step before it, two of each after one to warm. Returns the + card, the device's time over the core's by rows (the geometric mean of the three), the rows + where they cross, that crossing in ``rows * synapses`` (``work``) and the seconds it took.""" + import time + import warnings + + from .generic import Brain + + card = _cuda_card() + if card is None: + raise RuntimeError("no CUDA card to calibrate") + started = time.perf_counter() + rng = np.random.default_rng(0) + ratios = [] + for count in rows: + x = rng.random((count, 35)) * (rng.random((count, 35)) < 0.3) + y = rng.integers(0, 18, count) + seconds = [] + for on_card in (False, True): + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + brain = Brain.compose( + 35, 18, seed=0, backend="torch", device="cuda", precision="float32" + ) if on_card else Brain.compose(35, 18, seed=0, backend="cpu") + times: list[list[float]] = [[], [], []] + for t in range(3): + for k in range(3): + if k == 2: # a step that learns: the outcome of a sampled action + brain.step(x) + start = time.perf_counter() + if k == 0: + brain.act(x, greedy=True) + elif k == 1: + brain.lesson(x, y) + else: + brain.step(x, reward=np.ones(count)) + if t: + times[k].append(time.perf_counter() - start) + seconds.append(np.exp(np.mean([np.log(min(kind)) for kind in times]))) + ratios.append(float(seconds[1] / seconds[0])) + crossing = _crossing(rows, ratios) + return { + "card": card, "rows": list(rows), "ratios": ratios, "crossing": crossing, + "work": crossing * 13_120, "seconds": time.perf_counter() - started, + } + + +def _card_calibration(card: str) -> dict[str, Any] | None: + """The calibration of ``card``: this process's, the cache's (``CADENCE_PLAN_CACHE``), or made + here once and kept in both, with ``scale``, the factor on the table's crossings. None where + ``CADENCE_CALIBRATE=0`` or the calibration fails, and the stated margin holds.""" + held = _CALIBRATIONS.get(card) + if held is not None: + return held + if os.environ.get("CADENCE_CALIBRATE", "") == "0": + return None + from .brain import _store_plan, _stored_plan + + name = f"card {card}" + held = _stored_plan(name) + if held is None or "work" not in held: + global _CALIBRATING + if _CALIBRATING: + return None + _CALIBRATING = True + try: + held = {"name": name, **calibrate_card()} + except Exception: # a calibration that fails is the stated margin's + return None + finally: + _CALIBRATING = False + _store_plan(held) + held = {**held, "scale": float(held["work"]) / _CALIBRATION_A10G} + _CALIBRATIONS[card] = held + return held + + def _backend_of_the_run() -> _Candidate | None: """``CADENCE_BACKEND`` names a backend for a whole run: ``cpu``, ``mlx``, ``torch`` or ``torch:``, and with ``:float32`` or ``:float64`` after a device its precision.""" @@ -1418,13 +1526,29 @@ def join(block: Any) -> Any: xp.settle(out) return out - def weighted(self, trace: Any, delta: np.ndarray) -> Any: - """The rows' eligibility weighted by the dopamine ``delta`` and averaged.""" - d = self.xp.here(np.asarray(delta, dtype=float)) + def weighted(self, trace: Any, delta: Any) -> Any: + """The rows' eligibility weighted by the dopamine ``delta`` (on the host or on the device) + and averaged.""" + xp = self.xp + d = xp.floats(delta) if isinstance(delta, type(trace)) else xp.here( + np.asarray(delta, dtype=float) + ) return (d[:, None, None] * trace).mean(axis=0) def propose(self, step: Any) -> tuple[Any, ...] | None: """``Recall.propose`` of a step on the event's rows and the blocks that step.""" + unread = self.proposal_unread(step) + if unread is None: + return None + xp = self.xp + read = xp.read(unread[3], unread[4], also=unread[5]) + if not read[0]: + raise ValueError("the address map's reward step must be finite") + return (*unread[:3], float(read[1]), *unread[6:]) + + def proposal_unread(self, step: Any) -> tuple[Any, ...] | None: + """``propose`` with nothing read: the maps, a flag of their and the step's finiteness, the + step's norm, what MLX makes with them, then the position and anchored maps.""" recall, xp = self.recall, self.xp positions = bool(recall.position_reward) and bool(self.position_maps) if not (recall.address_reward or positions): @@ -1454,10 +1578,7 @@ def propose(self, step: Any) -> tuple[Any, ...] | None: changed.append(anchor_maps[0][0]) every = [*changed, *stepped] norm = xp.sqrt(sum((part * part).sum() for part in stepped)) - read = xp.read(xp.all_finite(*every), norm, also=every) - if not read[0]: - raise ValueError("the address map's reward step must be finite") - return (w_x, w_c, b, float(read[1]), position_maps, anchor_maps) + return (w_x, w_c, b, xp.all_finite(*every), norm, every, position_maps, anchor_maps) def accept(self, proposal: tuple[Any, ...]) -> None: """``Recall.accept``.""" @@ -1597,6 +1718,23 @@ def add(self, state: BrainState, batch: int, decay: float) -> None: trace[:, -1] = trace[:, -1] + 1.0 self.trace = trace # made by the outcome's candidate, which reads it + def value_here(self, state: BrainState) -> Any: + """``ActorCritic.value`` left on the device, in the maps' floats.""" + xp = self.xp + return xp.part(state, self.source) @ self.weights + self.owner.b_critic + + def unread(self, delta: Any) -> tuple[Any, Any, Any]: + """``candidate`` with nothing read: the weights, a flag of their finiteness and the bias's + step, on the device, for a caller that reads them with the rest of an outcome.""" + trace = self.trace + assert trace is not None + trace = trace / (1.0 + (trace * trace).sum(axis=1, keepdims=True)) + xp = self.xp + d = xp.here(delta) if isinstance(delta, np.ndarray) else xp.floats(delta) + step = self.owner.config.eta_critic * (d[:, None] * trace).mean(axis=0) + weights = self.weights + step[:-1] + return weights, xp.all_finite(weights), step[-1] + def candidate(self, delta: np.ndarray) -> tuple[Any, float]: """``ActorCritic._critic_candidate``: one read, of the checks and the bias's step.""" owner, xp = self.owner, self.xp diff --git a/src/cadence/stream.py b/src/cadence/stream.py index 7b91d024..5a1f3f9b 100644 --- a/src/cadence/stream.py +++ b/src/cadence/stream.py @@ -31,6 +31,7 @@ import numpy as np +from . import _moment from ._configuration import real from .brain import _MLX_MAP_ENTRIES, BrainState, _MlxMatrix, _resident from .brain import Brain as NeuralGraph @@ -667,6 +668,7 @@ def __init__(self, rate: float, connectome: Connectome, association: np.ndarray, self.routes: list[dict[str, Any]] = [] self.second = np.zeros(0) self.count = 0 + self.compiled_moment = False # the brain's ``learning_compiled_moment`` (``_contrast``) def add(self, name: str, memory: Any, slot: int | None, start: int, stop: int, neurons: np.ndarray, bound: float = BOUND) -> None: @@ -741,6 +743,10 @@ def _contrast( (route["synapses"], route["rows"], route["cols"], route["width"]) for route in self.routes ], movement) + compiled = ( + backs is None and self.__dict__.get("compiled_moment", False) + and graph._efficacy is not None and not graph.layout.parallel_synapses + ) for k, route in enumerate(self.routes): memory = route["memory"] if len(memory.trace) != batch: @@ -748,6 +754,12 @@ def _contrast( trace = memory.trace if rows is None else memory.trace[rows] held = trace[:, route["start"]:route["stop"]] amplitude = float(memory.amplitude) + if compiled: # the compiled moment's: the same equation, other bits + out[:, k] = _moment.route_contrast( + graph, route, movement, held, amplitude * self.gain(route), amplitude, + self.model, + ) + continue v = amplitude * self.gain(route) * held if backs is None: if graph.layout.parallel_synapses: # the last of a pair holds the entry @@ -850,8 +862,9 @@ class Recall: executed), ``address_reward`` and ``position_reward`` (reward trains the maps through the actor's eligibility, ``contrast_rows``, ``propose``, ``accept``), ``tokens`` (which actions are tokens a slot can hold), ``gain`` (a learned gain per port on those tokens), - ``copy`` (plastic synapses from the read ports into the motor region) and ``drive_gain`` - (the read's drive is one more route of ``Hearing``: one learned gain on every port, at most 1). + ``drive_gain`` (the read's drive is one more route of ``Hearing``: one learned gain on every + port, at most 1) and ``cortex_norm`` (every map reads the cortex of the moment before at unit + norm). A retired setting (``RETIRED``) is refused at construction, under ``Brain.retune`` and on load with one sentence naming the release that retired it and what @@ -880,7 +893,6 @@ class Recall: position: int = 32 position_rate: float = 0.1 boundary: bool = False - copy: float = 0.0 tokens: tuple[int, ...] | None = None action: bool = False # several heads read in turn, each from the share of every slot the heads before it left @@ -904,6 +916,16 @@ class Recall: # on every port and ``trace`` keeps what the ports held at the latest read, at amplitude one drive_gain: bool = False hearing: float = 1.0 + # every map reads the cortex of the moment before at unit norm per stream, the event's + # energy, so a lesson's step on its columns weighs no more than the step on the event's rows + cortex_norm: bool = False + # with ``gain``: the gains drive nothing and learn nothing through the memory's first + # ``gain_after`` lessons, so the read forms before the value path answers for it; at that + # lesson (at birth with 0) every next port's gain starts at ``gain_next`` and every recall + # port's at zero. ``gain_lessons`` counts the lessons the gains have seen + gain_after: int = 0 + gain_next: float = 0.0 + gain_lessons: int = 0 keys: np.ndarray = field(init=False, repr=False) values: np.ndarray = field(init=False, repr=False) nexts: np.ndarray = field(init=False, repr=False) @@ -924,15 +946,17 @@ class Recall: "capacity", "next") # options beside the founder memory: a memory describes and saves an option only where it # differs from its founder, and a saved memory without one takes its founder; each is - # runtime except the window, the heads, the position term, the copy pathway and the - # executed command, which shape the memory or the anatomy + # runtime except the window, the heads, the position term, the tokens and the executed + # command, which shape the memory or the anatomy OPTIONS = ( "address_reward", "position_reward", "window", "heads", "position", - "position_rate", "boundary", "copy", "tokens", "action", "anchor", - "relative", "slope", "when", "gain", "grid", "drive_gain", + "position_rate", "boundary", "tokens", "action", "anchor", + "relative", "slope", "when", "gain", "grid", "drive_gain", "cortex_norm", + "gain_after", "gain_next", ) - CONSTRUCTION_OPTIONS = ("window", "heads", "position", "copy", "tokens", "action", "anchor", - "relative", "slope", "when", "gain", "grid", "drive_gain") + CONSTRUCTION_OPTIONS = ("window", "heads", "position", "tokens", "action", "anchor", + "relative", "slope", "when", "gain", "grid", "drive_gain", + "cortex_norm", "gain_after", "gain_next") # what ``Brain.compose(..., recall=True)`` gives a new brain's memory over the founders above, # ordinary settings a caller's value overrides. The founder stays what a saved memory without # the setting means, so a saved brain keeps the values it was saved with @@ -981,6 +1005,9 @@ class Recall: "the share the heads before it left; cadence 0.94.0 takes it"), "null": ("0.95.0", None, "every moment's read takes its whole mass from the slots held; " "cadence 0.94.0 takes it"), + "copy": ("0.96.0", 0.0, "the read reaches the motor region through the association, " + "and recall_gain with recall_tokens is the learned path from a read to its " + "token's action; cadence 0.95.2 takes it"), }) @classmethod @@ -1030,7 +1057,7 @@ def setting(name: str, value: Any) -> Any: raise ValueError(Recall.removed(name)) if name in ( "amplitude", "recency", "address", "address_reward", "position_reward", - "position_rate", "copy", "slope", "gain", + "position_rate", "slope", "gain", "gain_next", ): return real(name, value, low=0) if name == "temperature": @@ -1040,10 +1067,19 @@ def setting(name: str, value: Any) -> Any: return number if name in ( "next", "boundary", "action", "anchor", "relative", "when", "drive_gain", + "cortex_norm", ): if not isinstance(value, bool): raise ValueError(f"{name} must be boolean") return value + if name == "gain_after": + if ( + isinstance(value, (bool, np.bool_)) + or not isinstance(value, (int, np.integer)) + or value < 0 + ): + raise ValueError("gain_after must be a nonnegative integer: lessons") + return int(value) if name in ("capacity", "window", "heads", "position"): low = 0 if name == "position" else 1 # no position term at 0 if ( @@ -1144,49 +1180,24 @@ def _port_neurons(self, name: str, width: int) -> np.ndarray: ) return np.concatenate([neurons, command]) - def copy_synapses(self) -> tuple[np.ndarray, np.ndarray, np.ndarray]: - """The copy pathway's synapses: every synapse from a head's read port neuron into the - motor region, their indices, and for each the value coordinate it reads and the motor - neuron it drives (the motor population's order).""" - connectome = self.connectome - motor = np.asarray(connectome.populations.get("motor", ()), dtype=np.int64) - pre, post = np.asarray(connectome.pre), np.asarray(connectome.post) - value = -np.ones(connectome.n, dtype=np.int64) - for recall, following in self._heads: - value[recall] = np.arange(len(recall)) - value[following] = np.arange(len(following)) - local_motor = -np.ones(connectome.n, dtype=np.int64) - local_motor[motor] = np.arange(len(motor)) - synapses = np.flatnonzero((value[pre] >= 0) & (local_motor[post] >= 0)) - return synapses, value[pre[synapses]], local_motor[post[synapses]] - - def _check_copy(self) -> None: - """With ``copy`` every read port reaches the motor region; ``tokens`` names one - coordinate per motor neuron within the event.""" + def _check_tokens(self) -> None: + """``tokens`` names one coordinate per motor neuron.""" motor = self.connectome.populations.get("motor", ()) if self.tokens is not None and len(motor) and len(self.tokens) != len(motor): raise ValueError( f"tokens name one observation coordinate per motor neuron ({len(motor)})" ) - if not self.copy: - return - if not len(motor): - raise ValueError("a copy pathway needs a population named 'motor'") - synapses, _, _ = self.copy_synapses() - pre = np.asarray(self.connectome.pre)[synapses] - for recall, following in self._heads: - for region in (recall, following): - if len(region) and not np.isin(region[: self.key_width], pre).any(): - raise ValueError( - "copy needs synapses from every read port into the motor region; " - "Brain.compose adds them" - ) def _check_gain(self) -> None: """With ``gain`` every read port reaches the motor region through its gain on the declared - tokens, whose coordinates lie in the value.""" + tokens, whose coordinates lie in the value; ``gain_after`` and ``gain_next`` are + settings of those gains.""" if not self.gain: + if self.gain_after or self.gain_next: + raise ValueError("gain_after and gain_next are settings of gain: they need gain") return + if self.gain_next and not self.next: + raise ValueError("gain_next starts the next ports' gains: it needs next") if self.tokens is None: raise ValueError("gain multiplies the declared tokens' identity: it needs tokens") if not len(self.motor): @@ -1212,6 +1223,8 @@ def _rewarded(position_reward: float, position: int, position_rate: float) -> No def __post_init__(self) -> None: self._device: _RecallOnDevice | None = None # what rests on a device (``reside``) + # the brain's ``learning_compiled_moment``, set by the brain at each read (``_compiles``) + self.compiled_moment = False self._actor: Any = None # the actor that keeps the read's eligibility for name in (*self.RUNTIME, *self.CONSTRUCTION, *self.OPTIONS): setattr(self, name, self.setting(name, getattr(self, name))) @@ -1320,7 +1333,7 @@ def __post_init__(self) -> None: # every head's grid maps ``(G_x, G_c, g)``, one column per residue of each modulus; none # without ``grid`` self.grid_maps = self._zero_maps(self.heads if self.grid else 0, sum(self.grid)) - self._check_copy() + self._check_tokens() # the motor region, which the gains drive (``gain``); empty where the anatomy has none self.motor: np.ndarray = np.asarray(populations.get("motor", ()), dtype=np.int64) self._motor_columns = columns(self.motor) @@ -1330,6 +1343,8 @@ def __post_init__(self) -> None: self.gains: np.ndarray = np.zeros( self.heads * (2 if self.next else 1) if self.gain else 0 ) + if self.gain and not self.gain_after and self.gain_lessons == 0: + self.gains = self._released() pairs = [(token, j) for j, token in enumerate(self.tokens or ()) if token >= 0] offset = self.key_width - len(self.observed) self._token_values = np.array([offset + t for t, _ in pairs], dtype=np.int64) @@ -1371,6 +1386,7 @@ def resides(self) -> bool: return ( self.heads == 1 and self.window == 1 and not self.action and not self.when and not self.gain and self.tokens is None and self.target == "recall" + and not self.cortex_norm ) def reside(self, kernel: Any) -> None: @@ -1979,19 +1995,29 @@ def _scope(self) -> Iterator[None]: bank array itself instead of a bitwise snapshot of it; they leave with the operation. A nested operation shares the outermost one's; a copy taken during one holds its token, which stops binding when the operation ends.""" - if self._scope_token in _OPERATIONS: + token = self._enter() + try: yield - return + finally: + if token is not None: + self._leave(token) + + def _enter(self) -> object | None: + """Begin an operation (``_scope``): its token, to ``_leave`` it with, or None within an + operation in progress, which this one shares.""" + if self._scope_token in _OPERATIONS: + return None token = object() self._scope_token = token _OPERATIONS.add(token) self._contents, self._content_keys = [], None - try: - yield - finally: - _OPERATIONS.discard(token) - self._scope_token = None - self._contents, self._content_keys = [], None + return token + + def _leave(self, token: object) -> None: + """End the operation ``_enter`` began with ``token``: its content products leave.""" + _OPERATIONS.discard(token) + self._scope_token = None + self._contents, self._content_keys = [], None def _content(self, address: np.ndarray, keys: np.ndarray) -> np.ndarray: """Content logits with reusable raw dot products and the current temperature. @@ -2057,6 +2083,8 @@ def stimulate(self, drive: np.ndarray, observations: np.ndarray) -> np.ndarray: into its own port.""" if self._device is not None: return cast(np.ndarray, self._device.stimulate(drive, observations)) + if self._compiles(len(observations)): + return self._stimulate_compiled(drive, self._port(observations)) moment = self._moment(self._port(observations)) self.last = moment out = np.array(_on_host(drive), dtype=float) @@ -2084,6 +2112,44 @@ def stimulate(self, drive: np.ndarray, observations: np.ndarray) -> np.ndarray: ) return out + def _compiles(self, rows: int, begun: bool = True) -> bool: + """Whether this read and its lesson run as the compiled moment + (``learning_compiled_moment``, ``cadence._moment``): a brain's option, on the host, for one + head of one event per moment without a command, the features of when or the gains, at + streams of the bank's batch that hold slots, with every map on the host. Without + ``begun`` the streams may also begin at this moment (no slot yet, or another batch).""" + if not self.__dict__.get("compiled_moment", False): + return False + if ( + self.heads != 1 or self.window != 1 or self.action or self._whens or self.gain + or (begun and (not self.slots or self.streams != rows)) + or not isinstance(self.head_maps[0][0], np.ndarray) + ): + return False + return _moment.available() + + def _stimulate_compiled( + self, drive: np.ndarray, x: np.ndarray, inplace: bool = False + ) -> np.ndarray: + """``stimulate`` as one compiled pass (``_compiles``): the same equations, other bits. + ``inplace`` writes the read into ``drive``, a host array of the caller's own.""" + c = _of(self.previous, len(x)) + key, q, p, read, following = _moment.recall_read(self, x, c) + self.last = _Moment(x, c, q, key, x) + out = drive if inplace else np.array(_on_host(drive), dtype=float) + p.flags.writeable = False + amplitude = 1.0 if self.drive_gain else self.amplitude + recall_columns, next_columns = self._head_columns[0] + out[:, recall_columns] = amplitude * read + if self.next: + out[:, next_columns] = amplitude * following + if self.drive_gain: + self.trace = out[:, self._port_columns].copy() + out[:, self._port_columns] = self.amplitude * self.hearing * self.trace + self._attended = p + self._head_reads = None + return out + def _reads(self, moment: _Moment) -> tuple[list[np.ndarray], list[np.ndarray | None]]: """Every head's address and attention at ``moment``.""" addresses = [moment.address] + [self._address(moment, h) for h in range(1, self.heads)] @@ -2156,6 +2222,14 @@ def _gain_step(self, read: np.ndarray, movement: np.ndarray) -> np.ndarray: "bt,bt->b", read[:, self._token_values], movement[:, self._token_motors] )) + def _released(self) -> np.ndarray: + """The gains as ``gain_after`` releases them: ``gain_next`` on every next port, zero on + every recall port.""" + gains = np.zeros(self.heads * (2 if self.next else 1)) + if self.next: + gains[1::2] = self.gain_next + return gains + def learn_gain(self, learned: LearnedState) -> dict[str, float]: """The gains' lesson after a teacher lesson whose phases are ``learned``, at its taught rows: each port's gain takes the mean over the rows of the row's rate (``_gain_rates``) @@ -2167,6 +2241,12 @@ def learn_gain(self, learned: LearnedState) -> dict[str, float]: if not self.gain: return {} quiet = {"gain_rows": 0.0, "gain_refused": 0.0} + if self.gain_after: + self.gain_lessons += 1 + if self.gain_lessons <= self.gain_after: # held at zero, released at the last + if self.gain_lessons == self.gain_after: + self.gains = self._released() + return quiet moment = self.last if ( moment is None @@ -2328,6 +2408,9 @@ def witness( self.writes += len(x) if self.action: # after the write, which begins streams of another batch afresh self.acted = done.copy() + if self.cortex_norm: # the maps read it at unit norm (a silent cortex stays silent) + norm = np.linalg.norm(cortex, axis=1, keepdims=True) + cortex = np.where(norm > 0, cortex / np.where(norm > 0, norm, 1.0), cortex) self.previous = cortex if self.window > 1: # this moment enters the window, the oldest leaves it past = _of(self.past, len(x)) @@ -2340,6 +2423,9 @@ def _observe(self, key: np.ndarray, value: np.ndarray) -> None: since the others (``reset`` of its rows) holds fewer slots: its latest held slot, wherever it lies, learns this moment as its next. A stream whose latest slot closed an episode (``end``) teaches it nothing, and its new slot is open again.""" + if self.__dict__.get("compiled_moment", False) and self._compiles(len(key)): + if _moment.recall_write(self, key, value): # the compiled moment's write, the same bits + return if len(key) != self.streams: self._empty(len(key)) successor: tuple[Any, ...] | None = None @@ -2539,6 +2625,8 @@ def learn( learned.free.activation if learned.opposite is None else learned.opposite.activation ) movement = (plus - minus)[:, self.association] + if self._compiles(len(moment.address)) and graph._efficacy is not None: + return self._learn_compiled(graph, rows, movement, positions, quiet) reports = [] moved_positions = refused_positions = 0 # the attention every competing head read with, taken before any head's lesson steps @@ -2569,6 +2657,64 @@ def learn( report["position_refused"] = float(refused_positions) return report + def _learn_compiled( + self, graph: NeuralGraph, rows: np.ndarray, movement: np.ndarray, positions: bool, + quiet: dict[str, float], + ) -> dict[str, float]: + """``learn`` of one head as one compiled pass (``_compiles``): the same equations, other + bits.""" + moved, error, w_x, w_c, w, address_ok, position_ok = _moment.address_lesson( + self, graph, rows, movement, positions + ) + return self._learned_compiled( + moved, error, w_x, w_c, w, address_ok, position_ok, positions, quiet + ) + + def _learned_compiled( + self, moved: int, error: float, w_x: np.ndarray, w_c: np.ndarray, w: np.ndarray, + address_ok: bool, position_ok: bool, positions: bool, quiet: dict[str, float], + parts: Sequence[tuple[np.ndarray, np.ndarray, np.ndarray]] | None = None, + ) -> dict[str, float]: + """What the compiled address lesson returns (``_moment._lesson_kernel``), taken: the + packed maps ``(w_x, w_c, w)`` of the ``moved`` rows' step become head 0's maps, the + address map where its step is finite (``address_ok``) and with ``positions`` every + position map where theirs is (``position_ok``); the lesson's report. ``parts`` are the + packed maps' six parts as arrays of their own, where the caller holds them.""" + if not moved: + return quiet + columns = _moment.packed_maps(self)[3] + + def part(k: int) -> tuple[np.ndarray, np.ndarray, np.ndarray]: + if parts is not None: + return parts[k] + start, stop = int(columns[k]), int(columns[k + 1]) + return (np.ascontiguousarray(w_x[:, start:stop]), + np.ascontiguousarray(w_c[:, start:stop]), w[start:stop].copy()) + + report = dict(quiet) + if self.address: + if address_ok: + self._set_map(0, *part(0)) + report["address_rows"] = float(moved) + report["address_error"] = float(error) + else: + report["address_refused"] = float(moved) + if positions: + if position_ok: + for k, name in enumerate(_moment.PACKED, start=1): + if not getattr(self, name) or ( + name == "slope_maps" and self.grid_maps and self.slots > self.position + ): + continue + m_x, m_c, m = part(k) + if name == "slope_maps": # one number each + m_x, m_c = m_x[:, 0].copy(), m_c[:, 0].copy() + setattr(self, name, _with_head(getattr(self, name), 0, (m_x, m_c, m))) + report["position_rows"] = float(moved) + else: + report["position_refused"] = float(moved) + return report + def _position_step( self, rows: np.ndarray, head: int, found: tuple[np.ndarray, ...] ) -> tuple[int, int]: @@ -3033,6 +3179,8 @@ def to_dict(self) -> dict[str, Any]: "neurons": self.neurons, "writes": int(self.writes), } + if self.gain_after: # the lessons the delayed gains have seen + out["gain_lessons"] = int(self.gain_lessons) if self.inputs: # only a memory keyed on a code of the observation records its width out["inputs"] = self.inputs # an option at its founder is left out of the description @@ -3296,6 +3444,7 @@ class FastSynapses: def __post_init__(self) -> None: self._device: _StoreOnDevice | None = None # what rests on a device (``reside``) + self.compiled_moment = False # the brain's ``learning_compiled_moment`` (``recall``) for name in ("pre", "post"): raw = np.asarray(getattr(self, name)) if ( @@ -3414,6 +3563,12 @@ def recall(self, key: np.ndarray) -> np.ndarray: """ cue = self._port(key, len(self.pre), "key") same_batch = len(self.strength) == len(cue) + if ( + self.separator is None and self.rule == "delta" + and self.__dict__.get("compiled_moment", False) + ): + if _moment.available(): # the compiled moment's read: the same equation, other bits + return _moment.store_read(self, cue) strength = self.strength if same_batch else self._baseline()[None, :, :] if self.separator is not None: cue = self.separator.code(cue) diff --git a/tests/test_compiled_moment.py b/tests/test_compiled_moment.py new file mode 100644 index 00000000..d4f3e49f --- /dev/null +++ b/tests/test_compiled_moment.py @@ -0,0 +1,565 @@ +"""``learning_compiled_moment`` (opt-in): the work around the solve kernels of a moment on the CPU in +compiled passes (``cadence._moment``): the recall memory's read and its address lesson and the +place store's read. The same equations, other bits; without the option every bit is the ordinary +path's.""" + +from __future__ import annotations + +import copy +import dataclasses +import tempfile +from pathlib import Path + +import numpy as np +import pytest + +import cadence as cd +from cadence import brain as brain_module + +pytest.importorskip("numba") +pytestmark = pytest.mark.skipif( + not brain_module._FUSED, reason="CADENCE_FUSED=0 reads the NumPy loop alone" +) + + +def brain(**options) -> cd.Brain: + return cd.Brain.compose(8, 4, modules=(32,), seed=3, backend="cpu", **options) + + +def lived(subject: cd.Brain, moments: int = 6, seed: int = 0) -> np.random.Generator: + """A few taught moments on four streams, so the slots and the maps hold something.""" + rng = np.random.default_rng(seed) + for _ in range(moments): + x = rng.random((4, 8)) + subject.lesson(x, rng.integers(0, 4, 4)) + subject.act(x, greedy=True) + return rng + + +def test_the_option_is_off_by_default_and_named_by_retune() -> None: + assert cd.LearnerConfig().compiled_moment is False + assert "compiled_moment" not in cd.LearnerConfig().to_dict() + subject = brain() + assert "learning_compiled_moment" not in subject.describe()["genes"] # off: left out + subject.retune(learning_compiled_moment=True) + assert subject.learner.config.compiled_moment is True + assert subject.describe()["genes"]["learning_compiled_moment"] is True + with pytest.raises(ValueError, match="compiled_moment"): + cd.LearnerConfig(compiled_moment=1) # type: ignore[arg-type] + + +def test_without_the_option_a_life_keeps_its_bits() -> None: + plain, off = brain(), brain(learning_compiled_moment=False) + lived(plain) + lived(off) + np.testing.assert_array_equal(plain.brain.efficacy, off.brain.efficacy) + for a, b in zip(plain.recall.head_maps[0], off.recall.head_maps[0], strict=True): + np.testing.assert_array_equal(a, b) + assert "compiled_moment" not in vars(plain.recall) or not plain.recall.compiled_moment + + +# the default read, and one whose slots pass its position table, where the grid's terms read and +# the slope's lean stays +READS = [{}, {"recall_position": 2}] + + +def _pair(**options) -> tuple[cd.Brain, cd.Brain, np.random.Generator]: + subject = brain(**options) + rng = lived(subject) + other = copy.deepcopy(subject) + other.retune(learning_compiled_moment=True) + return subject, other, rng + + +@pytest.mark.parametrize("options", READS) +def test_the_compiled_read_is_the_reads_equation(options) -> None: + plain, compiled, rng = _pair(**options) + assert plain.recall.slots > (2 if options else 0) + x = rng.random((4, 8)) + a, b = plain.stimulus(x), compiled.stimulus(x) + np.testing.assert_allclose(b, a, rtol=0, atol=1e-12) + np.testing.assert_allclose(compiled.recall._attended, plain.recall._attended, atol=1e-12) + np.testing.assert_allclose(compiled.recall.last.address, plain.recall.last.address, + atol=1e-12) + np.testing.assert_allclose(compiled.recall.trace, plain.recall.trace, atol=1e-12) + + +@pytest.mark.parametrize("options", READS) +def test_the_compiled_lesson_steps_the_maps_as_the_ordinary_one(options) -> None: + plain, compiled, rng = _pair(**options) + x = rng.random((4, 8)) + labels = np.array([0, 1, 2, 3]) + plain.lesson(x, labels) + compiled.lesson(x, labels) + assert plain.last_learning["demonstration_accepted_presentations"] > 0 + # the contrast and the update of every synapse and bias + np.testing.assert_allclose(compiled.brain.efficacy, plain.brain.efficacy, rtol=0, atol=1e-12) + np.testing.assert_allclose(compiled.brain.bias, plain.brain.bias, rtol=0, atol=1e-12) + for name in ("head_maps", "position_maps", "anchor_maps", "relative_maps", "slope_maps", + "grid_maps"): + for ours, theirs in zip(getattr(compiled.recall, name), getattr(plain.recall, name), + strict=True): + for u, v in zip(ours, theirs, strict=True): + np.testing.assert_allclose(u, v, rtol=0, atol=1e-11) + for key, value in plain.last_learning.items(): + if "address" in key or "position" in key: + assert compiled.last_learning[key] == pytest.approx(value, abs=1e-10), key + + +def test_streams_that_began_since_read_and_learn_with_fewer_slots() -> None: + plain, compiled, rng = _pair() + for subject in (plain, compiled): # two streams begin again: they hold fewer slots + subject.recall.reset(4, rows=np.array([1, 3])) + assert not plain.recall.held.all() + x = rng.random((4, 8)) + np.testing.assert_allclose(compiled.stimulus(x), plain.stimulus(x), atol=1e-12) + labels = np.array([3, 2, 1, 0]) + plain.lesson(x, labels) + compiled.lesson(x, labels) + for ours, theirs in zip(compiled.recall.head_maps[0], plain.recall.head_maps[0], strict=True): + np.testing.assert_allclose(ours, theirs, atol=1e-11) + + +def test_a_compiled_life_saves_and_continues() -> None: + subject = brain(learning_compiled_moment=True) + rng = lived(subject) + with tempfile.TemporaryDirectory() as folder: + loaded = cd.Brain.load(subject.save(Path(folder) / "brain")) + assert loaded.learner.config.compiled_moment is True + x = rng.random((4, 8)) + np.testing.assert_array_equal(subject.act(x, greedy=True), loaded.act(x, greedy=True)) + + +def test_a_read_the_kernels_do_not_take_runs_the_ordinary_path() -> None: + plain = brain(recall_heads=2) + rng = lived(plain) + compiled = copy.deepcopy(plain) + compiled.retune(learning_compiled_moment=True) + x = rng.random((4, 8)) + np.testing.assert_array_equal(compiled.stimulus(x), plain.stimulus(x)) + + +def test_the_carried_gains_step_as_the_ordinary_ones() -> None: + plain, compiled, rng = _pair() + assert plain.hearing is not None + x = rng.random((4, 8)) + labels = np.array([1, 0, 3, 2]) + plain.lesson(x, labels) + compiled.lesson(x, labels) + for name, gain in plain.carried_gains.items(): + assert compiled.carried_gains[name] == pytest.approx(gain, abs=1e-12), name + + +@pytest.mark.parametrize("began", [False, True]) +def test_the_compiled_write_keeps_every_bit(began: bool, monkeypatch: pytest.MonkeyPatch) -> None: + from cadence import _moment + + taken = [] + write = _moment.recall_write + + def counted(*args: object) -> bool: + taken.append(write(*args)) # type: ignore[arg-type] + return taken[-1] + + monkeypatch.setattr(_moment, "recall_write", counted) + subject = brain(recall_capacity=6) + rng = lived(subject, moments=9) # past the capacity: the oldest slots are dropped + recall = subject.recall + if began: # two streams begin again, and one closed an episode + recall.reset(4, rows=np.array([1, 3])) + recall.end(4, rows=np.array([0])) + for _ in range(4): + subject.stimulus(rng.random((4, 8))) # a read the write's focus follows + recall.compiled_moment = True + key = rng.random((4, recall.address_width)) + value = rng.random((4, recall.value_width)) + plain = copy.deepcopy(recall) # its slots are no views of rooms: the ordinary write + plain.compiled_moment = False + plain._observe(key, value) + recall._observe(key, value) + for name in ("keys", "values", "nexts", "held", "told", "focus", "ended"): + np.testing.assert_array_equal(getattr(recall, name), getattr(plain, name), name) + assert any(taken) # the compiled write ran where the rooms had space + + +def _chamber_life(subject: cd.Brain, seed: int, moments: int = 66) -> list[np.ndarray]: + """Taught and acted moments on streams of four, as the grammar chamber lives them: a lesson, + then the act of the same observations. Every third lesson shows the brain's own answer, so + it teaches no row and the act after it reads the lesson's read and solve. The streams begin + again after 22 moments, past the first room behind the memory's slots.""" + rng = np.random.default_rng(seed) + actions = [] + for t in range(moments): + if t % 22 == 0: + subject.reset() + x = (rng.random((4, 8)) < 0.3).astype(float) + labels = rng.integers(0, 4, 4) + if t % 3 == 0: + labels = copy.deepcopy(subject).act(x, greedy=True) + if t % 22 >= 2: + subject.lesson(x, labels) + actions.append(subject.act(x, greedy=True)) + return actions + + +# the compiled passes through the general path, with the shortcuts, with an act and a lesson's +# judgment each as one compiled call, and with a taught lesson as one compiled call too +MODES = [(False, False, False), (True, False, False), (True, True, False), (True, True, True)] + + +def _mode(monkeypatch, shortcuts: bool, one_call: bool, one_lesson: bool) -> None: + from cadence import _moment + + monkeypatch.setattr(_moment, "SHORTCUTS", shortcuts) + monkeypatch.setattr(_moment, "ONE_CALL", one_call) + monkeypatch.setattr(_moment, "ONE_LESSON", one_lesson) + + +@pytest.mark.parametrize("options", READS) +def test_the_shortcuts_keep_the_compiled_moments_bits(options, monkeypatch) -> None: + from cadence import _moment + + whole: dict[int, int] = {} + call = _moment.one_call + + def counted(subject, x, state, mode, spans, after): + found = call(subject, x, state, mode, spans, after) + if found is not None and found[3][0] == 0: + whole[mode] = whole.get(mode, 0) + 1 + return found + + monkeypatch.setattr(_moment, "one_call", counted) + phases = _moment.lesson_phases + + def counted_phases(*args): + found = phases(*args) + if found is not None: + whole[2] = whole.get(2, 0) + 1 + return found + + monkeypatch.setattr(_moment, "lesson_phases", counted_phases) + taught = _moment.taught_lesson + + def counted_lessons(*args): + found = taught(*args) + if found is not None: + whole[3] = whole.get(3, 0) + 1 + return found + + monkeypatch.setattr(_moment, "taught_lesson", counted_lessons) + lives = [] + for mode in MODES: + _mode(monkeypatch, *mode) + subject = brain(learning_compiled_moment=True, **options) + actions = _chamber_life(subject, 5) + lives.append((subject, actions)) + for other in lives[1:]: + _same_life(lives[0], other) + # every act but those after a lesson that taught nothing (they read the lesson's solve) and + # every lesson's judgment ran as one call, the acts that begin the streams among them + assert whole[0] >= 2 * 44 and whole[1] >= 2 * 54, whole + # the phases of every lesson that taught a row, and every such lesson whole in the last mode + assert whole[2] >= 30 and whole[3] >= 30, whole + + +def _same_life(first, second) -> None: + (a, acts_a), (b, acts_b) = first, second + for x, y in zip(acts_a, acts_b, strict=True): + np.testing.assert_array_equal(x, y) + np.testing.assert_array_equal(a.brain.efficacy, b.brain.efficacy) + np.testing.assert_array_equal(a.brain.bias, b.brain.bias) + for name in ("head_maps", "position_maps", "anchor_maps", "relative_maps", "slope_maps", + "grid_maps"): + for ours, theirs in zip(getattr(a.recall, name), getattr(b.recall, name), strict=True): + for u, v in zip(ours, theirs, strict=True): + np.testing.assert_array_equal(u, v) + for name in ("keys", "values", "nexts", "held", "told", "focus", "previous", "ended"): + np.testing.assert_array_equal(getattr(a.recall, name), getattr(b.recall, name), name) + assert a.recall.writes == b.recall.writes + np.testing.assert_array_equal(a.working_memory.trace, b.working_memory.trace) + np.testing.assert_array_equal(a.working_memory.last, b.working_memory.last) + np.testing.assert_array_equal(a.basal_ganglia.state.v, b.basal_ganglia.state.v) + assert a.carried_gains == b.carried_gains + assert dict(a.last_settlement) == dict(b.last_settlement) + assert a.last_learning == b.last_learning + assert list(a.last_learning) == list(b.last_learning) + # the optimizer's history, the gains' and the blocks the next solve reads + for name in ("velocity", "velocity_bias", "second_moment", "second_moment_bias"): + np.testing.assert_array_equal(getattr(a.learner, name), getattr(b.learner, name), name) + assert (a.learner.updates, a.learner.contrast_updates) == ( + b.learner.updates, b.learner.contrast_updates + ) + if a.hearing is not None: + np.testing.assert_array_equal(a.hearing.second, b.hearing.second) + assert a.hearing.count == b.hearing.count + np.testing.assert_array_equal(a.brain._blocks, b.brain._blocks) + + +# The options of the other 0.96 lanes, each named beside the compiled moment. +CROSS: dict[str, dict] = { + "cortex_norm": {"recall_cortex_norm": True}, + "fade": {"memory_fade": 1.2e-4}, + "maturity": {"maturity": True}, + "fade+maturity": {"memory_fade": 1.2e-4, "maturity": True}, + "cortex_norm+fade": {"recall_cortex_norm": True, "memory_fade": 1.2e-4}, + "gains": {"recall_cortex_norm": True, "recall_gain": 1.0, "recall_tokens": [0, 1, 2, 3], + "recall_gain_after": 5, "recall_gain_next": 2.0}, + "rate_zero": {}, +} + + +def _crossed(name: str, compiled: bool) -> cd.Brain: + subject = brain(learning_compiled_moment=compiled, **CROSS[name]) + if name == "rate_zero": # every synapse that touches the motor region is held + w = subject.connectome + motor = np.zeros(w.n, dtype=bool) + motor[np.asarray(w.populations["motor"])] = True + rate = np.ones(w.synapses) + rate[motor[w.post] | motor[w.pre]] = 0.0 + subject.learner.synapse_rate = rate + return subject + + +def _mixed_life(subject: cd.Brain, seed: int, rounds: int = 16) -> list[np.ndarray]: + """Every operation a life mixes, on streams of four: a lesson (every third shows the brain's + own answer and teaches no row), the greedy act of the same observations, a sampled moment of + life and its outcome, and an outcome reported on its own.""" + rng = np.random.default_rng(seed) + actions = [] + for t in range(rounds): + if t % 8 == 0: + subject.reset() + x = (rng.random((4, 8)) < 0.3).astype(float) + labels = rng.integers(0, 4, 4) + if t % 3 == 0: + labels = copy.deepcopy(subject).act(x, greedy=True) + if t % 8 >= 1: + subject.lesson(x, labels) + actions.append(subject.act(x, greedy=True)) + y = (rng.random((4, 8)) < 0.3).astype(float) + actions.append(subject.step(y)) + actions.append(subject.step(x, reward=rng.random(4))) + subject.learn(rng.random(4), np.zeros(4, dtype=bool), y) + return actions + + +@pytest.mark.parametrize("name", CROSS) +def test_each_compiled_path_keeps_its_bits_beside_another_lanes_option(name, monkeypatch) -> None: + lives = [] + for mode in MODES: + _mode(monkeypatch, *mode) + subject = _crossed(name, True) + lives.append((subject, _mixed_life(subject, 11))) + for other in lives[1:]: + _same_life(lives[0], other) + for attribute in ("consolidated", "fade_depth", "strength"): + ours = getattr(lives[0][0].hippocampus, attribute, None) + if ours is not None: + np.testing.assert_array_equal(getattr(other[0].hippocampus, attribute), ours) + np.testing.assert_array_equal(other[0].recall.gains, lives[0][0].recall.gains) + + +@pytest.mark.parametrize("name", CROSS) +def test_the_compiled_moment_agrees_with_the_ordinary_path_beside_another_lanes_option( + name, +) -> None: + plain, compiled = _crossed(name, False), _crossed(name, True) + for ours, theirs in zip(_mixed_life(compiled, 11), _mixed_life(plain, 11), strict=True): + np.testing.assert_array_equal(ours, theirs) + np.testing.assert_allclose(compiled.brain.efficacy, plain.brain.efficacy, rtol=0, atol=1e-8) + np.testing.assert_allclose(compiled.brain.bias, plain.brain.bias, rtol=0, atol=1e-8) + np.testing.assert_allclose(compiled.recall.previous, plain.recall.previous, atol=1e-8) + for ours, theirs in zip(compiled.recall.head_map(0), plain.recall.head_map(0), strict=True): + np.testing.assert_allclose(ours, theirs, rtol=0, atol=1e-8) + store = plain.hippocampus + np.testing.assert_allclose(compiled.hippocampus.consolidated, store.consolidated, atol=1e-8) + if getattr(store, "fade", 0.0): + np.testing.assert_allclose(compiled.hippocampus.fade_depth, store.fade_depth, atol=1e-8) + if name == "rate_zero": + held = compiled.learner.synapse_rate == 0 + fresh = _crossed(name, True) + np.testing.assert_array_equal( + compiled.brain.efficacy[held], fresh.brain.efficacy[held] + ) + if name == "cortex_norm": + np.testing.assert_allclose(np.linalg.norm(compiled.recall.previous, axis=1), 1.0) + + +# The lesson in one call at other settings of the learner and the wiring: each part of the update +# it composes (the founder's plain steps, the absolute floor, steps without momentum, the lesson's +# share), a brain with one carried route, one without carried gains, and a short position table. +# A name of ``LearnerConfig`` (the dict under "config") replaces the composed learner's setting. +LESSONS: dict[str, dict] = { + "founder": {"teaching": "founder"}, + "normalized": {"teaching": "normalized"}, + "no_momentum": {"config": {"momentum": 0.0}}, + "plain_steps": {"config": {"momentum": 0.0, "normalize": 0.0, "normalize_relative": 0.0}}, + "share": {"learning_surprise_share": True}, + "one_route": {"recall_drive_gain": False}, + "no_gains": {"carried_gain": 0.0, "recall_drive_gain": False}, + "position": {"recall_position": 2}, +} + + +def _taught_brain(name: str) -> cd.Brain: + options = dict(LESSONS[name]) + config = options.pop("config", {}) + subject = brain(learning_compiled_moment=True, **options) + if config: + subject.learner.config = dataclasses.replace(subject.learner.config, **config) + return subject + + +@pytest.mark.parametrize("name", LESSONS) +def test_a_lesson_in_one_call_keeps_the_bits_of_its_passes(name, monkeypatch) -> None: + from cadence import _moment + + whole = [0, 0] + taught = _moment.taught_lesson + + def counted(*args): + found = taught(*args) + whole[0] += 1 + whole[1] += found is not None + return found + + monkeypatch.setattr(_moment, "taught_lesson", counted) + lives = [] + for one_lesson in (False, True): + _mode(monkeypatch, True, True, one_lesson) + subject = _taught_brain(name) + lives.append((subject, _chamber_life(subject, 7, moments=30))) + _same_life(*lives) + assert whole[0] == whole[1] >= 12, whole # every taught lesson ran whole + + +@pytest.mark.parametrize("streams", [9, 24]) +def test_a_lesson_of_many_rows_in_one_call_keeps_the_bits_of_its_passes( + streams, monkeypatch +) -> None: + """Lessons that teach eight rows and more: the carried gains step on the mean of their + contrasts over the rows, summed row by row as NumPy sums two columns and more.""" + lives = [] + for one_lesson in (False, True): + _mode(monkeypatch, True, True, one_lesson) + subject = cd.Brain.compose(8, 4, modules=(32,), seed=3, backend="cpu", + learning_compiled_moment=True) + rng = np.random.default_rng(1) + actions, taught = [], 0 + for t in range(10): + x = (rng.random((streams, 8)) < 0.3).astype(float) + actions.append(subject.act(x, greedy=True)) + y = (rng.random((streams, 8)) < 0.3).astype(float) + wrong = (copy.deepcopy(subject).act(y, greedy=True) + 1 + t % 3) % 4 + report = subject.lesson(y, wrong) # every row's answer differs from its label + taught += report["accepted_presentations"] == streams + actions.append(subject.act(y, greedy=True)) + assert taught == 10 + lives.append((subject, actions)) + _same_life(*lives) + + +def test_a_lesson_in_one_call_leaves_the_packed_maps_a_read_would_make(monkeypatch) -> None: + """After a lesson in one call that stepped every map, the next read takes the packed maps + the call returned: the arrays ``packed_maps`` makes from the memory's maps, to the bit. + Where a map stays (the slope's, once the slots pass the position table) the next read packs + them again.""" + from cadence import _moment + + _mode(monkeypatch, True, True, True) + for options, kept_whole in zip(READS, (True, False), strict=True): + subject = brain(learning_compiled_moment=True, **options) + rng = lived(subject, moments=8) + x = rng.random((4, 8)) + wrong = (copy.deepcopy(subject).act(x, greedy=True) + 1) % 4 + report = subject.lesson(x, wrong) + assert report["address_rows"] > 0 and report["position_rows"] > 0 + recall = subject.recall + kept = recall.__dict__["_packed"] + current = all( + a is b for a, b in zip(kept[2], _moment.packed_parts(recall), strict=True) + ) + assert current == kept_whole + del recall.__dict__["_packed"] + made = _moment.packed_maps(recall) + if kept_whole: + for ours, theirs in zip(kept[1], made, strict=True): + assert ours.tobytes() == theirs.tobytes() and ours.shape == theirs.shape + for ours, theirs in zip(kept[3], recall.__dict__["_packed"][3], strict=True): + assert ours.tobytes() == theirs.tobytes() and ours.shape == theirs.shape + + +def test_the_compiled_contrast_is_the_block_contrast() -> None: + """The contrast's kernel against ``blocks.block_contrast`` and NumPy's mean: the same + products and sums. The synapses' terms agree to the last place or two (one BLAS makes the + products for NumPy, another may for the kernel); the neurons' terms to the bit.""" + from cadence import _moment + from cadence.blocks import block_contrast + + subject = brain(learning_compiled_moment=True) + layout, n = subject.brain.layout, subject.connectome.n + rng = np.random.default_rng(0) + for rows in (1, 2, 3, 7, 16, 40): + minus = rng.random((rows, n)) + plus = minus + 0.1 * rng.standard_normal((rows, n)) + held = int(layout.starts[1]) # a range that is the same in both phases + plus[:, :held] = minus[:, :held] + span = 0.2 + synapses, neurons = _moment.contrast(layout, plus, minus, span) + np.testing.assert_allclose( + synapses, block_contrast(layout, plus, minus, rows * span), rtol=1e-12, atol=1e-15 + ) + np.testing.assert_array_equal(neurons, (plus - minus).mean(axis=0) / span) + assert _moment.contrast(layout, plus.astype(np.float32), minus, span) is None + + +def test_a_lesson_the_update_refuses_raises_as_its_passes_do(monkeypatch) -> None: + """An update that is not finite is refused whole: the call returns nothing, the passes in + turn raise as they do without it, and the brain keeps its parameters and moments.""" + raised = [] + for one_lesson in (False, True): + _mode(monkeypatch, True, True, one_lesson) + subject = brain(learning_compiled_moment=True) + rng = lived(subject) + subject.learner.velocity = np.full(subject.connectome.synapses, np.inf) + before = subject.brain.efficacy.copy(), copy.deepcopy(subject.recall.head_maps) + x = rng.random((4, 8)) + wrong = (subject.act(x, greedy=True) + 1) % 4 + subject.reset() + with pytest.raises(ValueError, match="finite") as refusal: + subject.lesson(x, wrong) + raised.append(str(refusal.value)) + np.testing.assert_array_equal(subject.brain.efficacy, before[0]) + for ours, theirs in zip(subject.recall.head_maps[0], before[1][0], strict=True): + np.testing.assert_array_equal(ours, theirs) + assert np.isinf(subject.learner.velocity).all() + assert raised[0] == raised[1] + + +def test_a_lesson_whose_route_is_not_the_kernels_kind_runs_its_passes(monkeypatch) -> None: + """A carried route's memory that the kernel cannot read as it is (here a trace that is not + writeable) leaves the lesson to the passes in turn, with their bits.""" + from cadence import _moment + + found: list[object] = [] + taught = _moment.taught_lesson + + def counted(*args): + found.append(taught(*args)) + return found[-1] + + monkeypatch.setattr(_moment, "taught_lesson", counted) + lives = [] + for one_lesson in (False, True): + _mode(monkeypatch, True, True, one_lesson) + subject = brain(learning_compiled_moment=True) + rng = lived(subject) + x = rng.random((4, 8)) + wrong = (copy.deepcopy(subject).act(x, greedy=True) + 1) % 4 + subject.working_memory.trace.flags.writeable = False + del found[:] + subject.lesson(x, wrong) + assert found == ([None] if one_lesson else []) + subject.working_memory.trace.flags.writeable = True + lives.append((subject, [subject.act(x, greedy=True)])) + _same_life(*lives) diff --git a/tests/test_default_backend.py b/tests/test_default_backend.py index 58e28a8b..838cc471 100644 --- a/tests/test_default_backend.py +++ b/tests/test_default_backend.py @@ -111,7 +111,9 @@ def test_a_brain_no_launch_takes_passes_one_core_at_its_rows_on_mlx(monkeypatch) assert choose(1, SPEECH, settling=3_000) == ("cpu",) -def test_a_card_the_table_has_not_read_takes_a_stated_margin(monkeypatch): +def test_a_card_the_table_has_not_read_takes_a_stated_margin_without_a_calibration(monkeypatch): + monkeypatch.setenv("CADENCE_CALIBRATE", "0") + monkeypatch.setattr(resident, "_CALIBRATIONS", {}) work = resident._AUTO_CUDA_WORK on_a_machine(monkeypatch, card="NVIDIA A10G") assert choose(1, work) == ("torch", "cpu") @@ -120,6 +122,95 @@ def test_a_card_the_table_has_not_read_takes_a_stated_margin(monkeypatch): assert choose(1, resident._AUTO_UNSEEN * work) == ("torch", "cpu") +def _calibrated(monkeypatch, tmp_path, crossing_rows: float) -> list[int]: + """A machine with an unseen card whose calibration finds ``crossing_rows`` on the chamber's + brain; returns the calibrations made, one entry each.""" + monkeypatch.delenv("CADENCE_CALIBRATE", raising=False) + monkeypatch.setenv("CADENCE_PLAN_CACHE", str(tmp_path / "plans.json")) + monkeypatch.setattr(resident, "_CALIBRATIONS", {}) + from cadence import brain as graph + + monkeypatch.setattr(graph, "_PLANS", {}) + made: list[int] = [] + + def calibrate(rows=resident._CALIBRATION_ROWS): + made.append(1) + return {"card": "Some Other Card", "rows": list(rows), "ratios": [2.0, 1.0, 0.5], + "crossing": crossing_rows, "work": crossing_rows * CHAMBER, "seconds": 3.0} + + monkeypatch.setattr(resident, "calibrate_card", calibrate) + on_a_machine(monkeypatch, card="Some Other Card") + return made + + +def test_an_unseen_card_takes_the_crossing_its_calibration_reads(monkeypatch, tmp_path): + made = _calibrated(monkeypatch, tmp_path, crossing_rows=2 * 43.717106725290485) + work = resident._AUTO_CUDA_WORK + assert choose(1, work) == ("cpu",) # half the A10G's speed: twice its crossings + assert choose(1, 2 * work + 64) == ("torch", "cpu") + assert choose(4, DEEP, apical=True) == ("cpu",) # 4 rows pass the A10G's crossing there + assert choose(8, DEEP, apical=True) == ("torch", "cpu") + assert made == [1] # once per process + + +def test_a_faster_card_takes_the_device_sooner_than_the_margin(monkeypatch, tmp_path): + _calibrated(monkeypatch, tmp_path, crossing_rows=43.717106725290485 / 2) + work = resident._AUTO_CUDA_WORK + assert choose(1, work // 2 + 64) == ("torch", "cpu") + assert choose(1, work // 2 - 64) == ("cpu",) + + +def test_a_calibration_is_kept_for_later_processes(monkeypatch, tmp_path): + made = _calibrated(monkeypatch, tmp_path, crossing_rows=43.717106725290485) + choose(1, resident._AUTO_CUDA_WORK) + monkeypatch.setattr(resident, "_CALIBRATIONS", {}) + from cadence import brain as graph + + monkeypatch.setattr(graph, "_PLANS", {}) + assert choose(1, resident._AUTO_CUDA_WORK + 64) == ("torch", "cpu") + assert made == [1] # the second process reads the cache + assert "card Some Other Card" in (tmp_path / "plans.json").read_text() + + +def test_a_card_never_calibrated_below_a_quarter_of_the_tables_work(monkeypatch, tmp_path): + made = _calibrated(monkeypatch, tmp_path, crossing_rows=1.0) + assert choose(1, resident._AUTO_CUDA_WORK // 5) == ("cpu",) + assert made == [] + + +def test_a_failed_calibration_takes_the_stated_margin(monkeypatch, tmp_path): + _calibrated(monkeypatch, tmp_path, crossing_rows=1.0) + + def broken(rows=resident._CALIBRATION_ROWS): + raise RuntimeError("the card refused") + + monkeypatch.setattr(resident, "calibrate_card", broken) + work = resident._AUTO_CUDA_WORK + assert choose(1, work) == ("cpu",) + assert choose(1, resident._AUTO_UNSEEN * work) == ("torch", "cpu") + + +@pytest.mark.parametrize( + ("ratios", "crossing"), + [((2.0, 1.0, 0.5), 64.0), ((4.0, 2.0, 1.0), 256.0), ((0.5, 0.25, 0.125), 4.0), + ((1.0, 1.0, 1.0), float("inf")), ((2.0, 3.0, 4.0), float("inf"))], +) +def test_the_calibration_crossing_is_read_between_and_past_its_rows(ratios, crossing): + assert resident._crossing((16, 64, 256), ratios) == pytest.approx(crossing) + + +def test_the_a10g_reference_is_the_tables_crossing_read_by_the_calibration(): + core = {16: (0.93, 6.82, 8.36), 64: (2.10, 23.1, 23.9), 256: (8.32, 77.3, 79.9)} + a10g = {16: (1.99, 9.49, 9.65), 64: (2.72, 17.0, 15.7), 256: (3.49, 16.2, 24.5)} + + def mean(times): + return float(np.exp(np.mean(np.log(times)))) + + ratios = [mean(a10g[rows]) / mean(core[rows]) for rows in (16, 64, 256)] + work = resident._crossing((16, 64, 256), ratios) * CHAMBER + assert work == pytest.approx(resident._CALIBRATION_A10G, rel=1e-6) + + def test_memories_that_stay_on_the_host_need_more_work(monkeypatch): work = resident._AUTO_CUDA_WORK on_a_machine(monkeypatch, card="NVIDIA A10G") diff --git a/tests/test_graft.py b/tests/test_graft.py index 2b529120..7d64a604 100644 --- a/tests/test_graft.py +++ b/tests/test_graft.py @@ -349,9 +349,9 @@ def test_wiring_the_donor_lacks_starts_at_zero_even_with_kept_joints(): def test_a_graft_carries_the_donors_contact_counts_and_keeps_this_brains_signs(tmp_path): - """A recall copy pathway with tokens is laid out again from the brain's construction on - load; with the donor's signs carried, the brain's connectome and its learner's disagreed.""" - tokens = dict(recall=True, recall_founder=True, recall_copy=2.0) + """A graft carries the donor's contact counts and keeps this brain's signs, so the brain's + connectome and its learner's agree after a load.""" + tokens = dict(recall=True, recall_founder=True) host = cd.Brain.compose(6, 4, seed=1, modules=(16,), recall_tokens=[0, 1, 2, 3], **{"recall": False, **tokens}) donor = cd.Brain.compose(6, 4, seed=7, modules=(16,), **{"recall": False, **tokens}) grafted = host.graft(donor, ["recall", "next", "motor", "association"]) diff --git a/tests/test_grow.py b/tests/test_grow.py index b11acaf6..43458df8 100644 --- a/tests/test_grow.py +++ b/tests/test_grow.py @@ -78,7 +78,7 @@ def check_identical(trained, grown, inputs=6, moments=8, atol=1e-12): LAYOUTS = { "founder": dict(), "recall heads": dict(recall=True, recall_founder=True, recall_heads=2, recall_temperature=0.4), - "recall action": dict(recall=True, recall_founder=True, recall_action=True, recall_copy=0.5, recall_capacity=16), + "recall action": dict(recall=True, recall_founder=True, recall_action=True, recall_capacity=16), "episodic": dict(memory_choice=True), "history": dict(history_decays=(0.5, 0.9), history_amplitude=2.0), "efference": dict(efference_amplitude=1.0, efference_decay=0.3), @@ -519,14 +519,14 @@ def test_sampled_choices_follow_the_trained_brain_without_new_slots(): [ ({}, dict(efference_amplitude=0.0), "above zero"), ({}, dict(recall_temperature=0.3), "grow it with recall=True"), - ({}, dict(recall=True, recall_founder=True, recall_copy=1.0, recall_tokens=[0, 1, 2, 3]), "tokens"), + ({}, dict(recall=True, recall_founder=True, recall_tokens=[0, 1, 2, 3]), "tokens"), ( - dict(recall=True, recall_founder=True, recall_copy=1.0, recall_tokens=[0, 1, 2, 3]), + dict(recall=True, recall_founder=True, recall_tokens=[0, 1, 2, 3]), dict(recall_heads=2), "tokens", ), ( - dict(recall=True, recall_founder=True, recall_copy=1.0, recall_tokens=[0, 1, 2, 3]), + dict(recall=True, recall_founder=True, recall_tokens=[0, 1, 2, 3]), dict(slots=(4, 2)), "per action", ), diff --git a/tests/test_lives.py b/tests/test_lives.py new file mode 100644 index 00000000..3e4d13d9 --- /dev/null +++ b/tests/test_lives.py @@ -0,0 +1,349 @@ +"""Independent lives over one topology (cadence#274, #167): each life follows the lone brain of its +seed, on the CPU to the bit and on a torch device within its rounding, and nothing one life learns +reaches another. Two compositions: the default one without the store (the recall memory, the +carried gains, the deep teaching point judged by surprise) and the plain one without the recall +memory.""" + +from __future__ import annotations + +import warnings + +import numpy as np +import pytest + +import cadence as cd + +INPUTS, ACTIONS = 12, 5 +OPTIONS = {"modules": (16,)} +COMPOSITIONS = {"default": {}, "plain": {"recall": False}} +MAPS = ("head_maps", "position_maps", "anchor_maps", "relative_maps", "slope_maps", "grid_maps") + + +def lone(seed: int, composition: dict | None = None, **place) -> cd.Brain: + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + return cd.Brain.compose(INPUTS, ACTIONS, seed=seed, episodic=False, **OPTIONS, + **(composition or {}), **place) + + +def some_lives(count: int, composition: dict | None = None, **place) -> cd.Lives: + return cd.Lives(INPUTS, ACTIONS, count, **OPTIONS, **(composition or {}), **place) + + +def a_life(lives: cd.Lives, brains: list[cd.Brain], moments: int, seed: int = 0, + ends=None, teaches: int = 3) -> int: + """Step the lives and the lone brains alike: rewards, endings and a teacher every third + moment; returns how many actions differed. ``ends(t)`` gives the moment's endings in place + of random ones.""" + rng = np.random.default_rng(seed) + count, differed = lives.lives, 0 + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + for t in range(moments): + x = np.eye(INPUTS)[rng.integers(INPUTS, size=count)] * rng.random((count, INPUTS)) + reward, done = rng.random(count), rng.random(count) < 0.15 + if ends is not None: + done = ends(t) + teacher = rng.integers(ACTIONS, size=count) if t % teaches == 1 else None + outcome = {} if t == 0 else {"reward": reward, "done": done} + got = lives.step(x, teacher=teacher, **outcome) + for i, brain in enumerate(brains): + one = {} if t == 0 else {"reward": reward[i : i + 1], "done": done[i : i + 1]} + taught = None if teacher is None else teacher[i : i + 1] + differed += int(brain.step(x[i : i + 1], teacher=taught, **one)[0] != got[i]) + return differed + + +def gaps(lives: cd.Lives, brains: list[cd.Brain]) -> dict[str, float]: + """The largest difference between each life and its lone brain, by part.""" + out: dict[str, float] = {} + + def widen(name: str, ours, theirs) -> None: + gap = float(np.abs(np.asarray(ours, dtype=float) - np.asarray(theirs, dtype=float)).max()) + out[name] = max(out.get(name, 0.0), gap) + + for i, brain in enumerate(brains): + held = lives.parameters(i) + learner, agent = brain.learner, brain.basal_ganglia + widen("efficacy", held["efficacy"], brain.brain.efficacy) + widen("bias", held["bias"], brain.brain.bias) + widen("w_critic", held["w_critic"], agent.w_critic) + widen("b_critic", held["b_critic"], agent.b_critic) + widen("velocity", held["velocity"], learner.velocity) + widen("second_moment", held["second_moment"], learner.second_moment) + assert held["contrast_updates"] == learner.contrast_updates + assert held["updates"] == learner.updates + if brain.recall is not None: + for name, maps in held["recall"].items(): + for ours, theirs in zip(maps, getattr(brain.recall, name)[0], strict=True): + widen(name, ours, theirs) + if brain.hearing is not None: + for name, gain in held["carried_gains"].items(): + widen("gain " + name, gain, brain.carried_gains[name]) + widen("carried_second", held["carried_second"], brain.hearing.second) + assert held["carried_count"] == brain.hearing.count + return out + + +@pytest.mark.parametrize("composition", COMPOSITIONS.values(), ids=COMPOSITIONS) +def test_lives_on_the_cpu_are_lone_brains_to_the_bit(composition): + lives = cd.Lives(INPUTS, ACTIONS, 3, seeds=[4, 5, 6], **OPTIONS, **composition) + brains = [lone(s, composition, backend="cpu") for s in (4, 5, 6)] + assert a_life(lives, brains, 12) == 0 + assert max(gaps(lives, brains).values()) == 0.0 + + +torch = pytest.importorskip("torch") +DEVICES = ["cpu", *(["cuda"] if torch.cuda.is_available() else [])] + + +@pytest.mark.parametrize("composition", COMPOSITIONS.values(), ids=COMPOSITIONS) +@pytest.mark.parametrize("device", DEVICES) +def test_each_life_on_a_device_follows_its_lone_brain_in_float64(device, composition): + place = {"backend": "torch", "device": device, "precision": "float64"} + lives = some_lives(6, composition, **place) + brains = [lone(s, composition, **place) for s in range(6)] + assert a_life(lives, brains, 30) == 0 + found = gaps(lives, brains) + assert max(found.values()) < 1e-9, found + moved = max( + float(np.abs(lives.parameters(i)["efficacy"] - lone(i, composition, backend="cpu") + .brain.efficacy).max()) + for i in range(6) + ) + assert moved > 0.05 # the lives learned + + +@pytest.mark.parametrize( + ("actions", "options"), + [(5, {"recall_address_reward": 0.01}), (6, {"slots": 2}), (5, {"modules": (16, 8)}), + (5, {"carried_gain": 0.0}), (5, {"recall_position": 0}), (5, {"observers": (8,)}), + (5, {"teaching": "founder"}), (5, {"learning_surprise": False})], + ids=["address reward", "two slots", "two regions", "no carried gain", "no position term", + "an observer", "founder teaching", "every row taught"], +) +def test_variants_of_the_composition_follow_their_lone_brains(actions, options): + place = {"backend": "torch", "device": "cpu", "precision": "float64"} + composed = {**OPTIONS, **options} + lives = cd.Lives(INPUTS, actions, 3, **place, **composed) + rng = np.random.default_rng(2) + slots = options.get("slots", 1) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + brains = [cd.Brain.compose(INPUTS, actions, seed=s, episodic=False, **place, **composed) + for s in range(3)] + for t in range(18): + x = np.eye(INPUTS)[rng.integers(INPUTS, size=3)] * rng.random((3, INPUTS)) + reward, done = rng.random(3), rng.random(3) < 0.15 + teacher = None + if t % 3 == 1: + shape = (3, slots) if slots > 1 else (3,) + teacher = rng.integers(actions // slots, size=shape) + outcome = {} if t == 0 else {"reward": reward, "done": done} + got = lives.step(x, teacher=teacher, **outcome) + for i, brain in enumerate(brains): + one = {} if t == 0 else {"reward": reward[i : i + 1], "done": done[i : i + 1]} + taught = None if teacher is None else teacher[i : i + 1] + np.testing.assert_array_equal(brain.step(x[i : i + 1], teacher=taught, **one)[0], + got[i]) + found = gaps(lives, brains) + assert max(found.values()) < 1e-9, found + + +def test_the_default_lives_learn_every_part_each_on_its_own_lessons(): + """The parts the default composition adds are exercised and differ between lives: each life's + own count of lessons that taught it, its recall maps and its carried gain.""" + place = {"backend": "torch", "device": "cpu", "precision": "float64"} + lives = some_lives(6, **place) + brains = [lone(s, **place) for s in range(6)] + assert a_life(lives, brains, 36) == 0 + held = [lives.parameters(i) for i in range(6)] + assert len({one["contrast_updates"] for one in held}) > 1 # judged life by life + for name in ("head_maps", "position_maps", "anchor_maps", "slope_maps"): + assert max(float(np.abs(one["recall"][name][0]).max()) for one in held) > 1e-4, name + assert len({round(one["carried_gains"]["trace"], 9) for one in held}) > 1 + assert max(gaps(lives, brains).values()) < 1e-9 + + +def test_streams_of_uneven_length_past_the_position_table_follow_their_lone_brains(): + """One life's stream ends while the others' run past the position table: the slope's rule + reads each life's own slots, and the grid maps learn.""" + place = {"backend": "torch", "device": "cpu", "precision": "float64"} + lives = some_lives(4, **place) + brains = [lone(s, **place) for s in range(4)] + + def ends(t: int) -> np.ndarray: + done = np.zeros(4, dtype=bool) + done[1], done[2] = t in (36, 44), t == 20 + return done + + assert a_life(lives, brains, 56, ends=ends) == 0 + found = gaps(lives, brains) + assert max(found.values()) < 1e-9, found + assert brains[0].recall.slots > brains[0].recall.position > brains[1].recall.slots + assert max( + float(np.abs(lives.parameters(i)["recall"]["grid_maps"][0]).max()) for i in range(4) + ) > 1e-4 + + +@pytest.mark.parametrize("composition", COMPOSITIONS.values(), ids=COMPOSITIONS) +@pytest.mark.parametrize("device", DEVICES) +def test_each_life_in_float32_follows_its_lone_brain_within_rounding(device, composition): + """A float32 life follows its lone brain within the spread float32 gives a lone brain's own + two memory paths: every action the same over these moments, the parameters near.""" + place = {"backend": "torch", "device": device, "precision": "float32"} + lives = some_lives(4, composition, **place) + brains = [lone(s, composition, **place) for s in range(4)] + assert a_life(lives, brains, 16) == 0 + found = gaps(lives, brains) + assert max(found.values()) < 5e-3, found + + +@pytest.mark.parametrize("composition", COMPOSITIONS.values(), ids=COMPOSITIONS) +def test_a_solve_that_needs_its_fallback_and_its_rescue_follows_the_lone_brains(composition): + """A budget of eight sweeps: some lives qualify at once, some at the half step, some on the + damping ladder. Each life's solve is its own.""" + place = {"backend": "torch", "device": "cpu", "precision": "float64"} + tight = {**composition, "learning_free_steps": 8} + lives = some_lives(6, tight, **place) + brains = [lone(s, tight, **place) for s in range(6)] + rng = np.random.default_rng(0) + ways = set() + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + for t in range(8): + x = rng.random((6, INPUTS)) * 3 + reward = rng.random(6) + outcome = {} if t == 0 else {"reward": reward} + got = lives.step(x, **outcome) + for i, brain in enumerate(brains): + one = {} if t == 0 else {"reward": reward[i : i + 1]} + assert brain.step(x[i : i + 1], **one)[0] == got[i] + report = brain.last_settlement + ways.add((report["damping_halvings"], "rescue_steps" in report)) + assert {(0, False), (1, False), (0, True)} <= ways or {(0, False), (1, True)} <= ways, ways + assert max(gaps(lives, brains).values()) < 1e-9 + + +@pytest.mark.parametrize("composition", COMPOSITIONS.values(), ids=COMPOSITIONS) +def test_greedy_acts_and_lessons_follow_the_lone_brains(composition): + place = {"backend": "torch", "device": "cpu", "precision": "float64"} + lives = some_lives(4, composition, **place) + brains = [lone(s, composition, **place) for s in range(4)] + rng = np.random.default_rng(3) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + for _ in range(8): + x = np.eye(INPUTS)[rng.integers(INPUTS, size=4)] + y = rng.integers(ACTIONS, size=4) + lives.lesson(x, y) + got = lives.act(x, greedy=True) + for i, brain in enumerate(brains): + brain.lesson(x[i : i + 1], y[i : i + 1]) + assert brain.act(x[i : i + 1], greedy=True)[0] == got[i] + assert max(gaps(lives, brains).values()) < 1e-9 + + +@pytest.mark.parametrize("composition", COMPOSITIONS.values(), ids=COMPOSITIONS) +def test_a_life_learns_nothing_of_another(composition): + """Two lives of one seed, one rewarded and one not: they part, and each follows its own + lone brain.""" + place = {"backend": "torch", "device": "cpu", "precision": "float64"} + lives = cd.Lives(INPUTS, ACTIONS, 2, seeds=[7, 7], **place, **OPTIONS, **composition) + x = np.eye(INPUTS)[[2, 2]] + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + lives.step(x) + for _ in range(8): + lives.step(x, reward=np.array([1.0, 0.0])) + first, second = lives.parameters(0), lives.parameters(1) + assert np.abs(first["efficacy"] - second["efficacy"]).max() > 1e-6 + assert abs(float(first["b_critic"]) - float(second["b_critic"])) > 1e-6 + + +@pytest.mark.parametrize("composition", COMPOSITIONS.values(), ids=COMPOSITIONS) +def test_a_life_is_a_lone_brain_that_carries_what_it_learned_and_saves(tmp_path, composition): + """``life(i)`` is a lone brain on the CPU with everything the life learned: its parameters, + optimizer, critic, recall maps, carried gains and random state are its lone brain's.""" + place = {"backend": "torch", "device": "cpu", "precision": "float64"} + lives = some_lives(3, composition, **place) + brains = [lone(s, composition, **place) for s in range(3)] + a_life(lives, brains, 12) + one, beside = lives.life(1), brains[1] + assert one.placement["backend"] == "cpu" + + def near(ours, theirs, name: str) -> None: + np.testing.assert_allclose(ours, theirs, rtol=0, atol=1e-9, err_msg=name) + + near(one.brain.efficacy, beside.brain.efficacy, "efficacy") + near(one.brain.bias, beside.brain.bias, "bias") + near(one.basal_ganglia.w_critic, beside.basal_ganglia.w_critic, "w_critic") + near(one.basal_ganglia.b_critic, beside.basal_ganglia.b_critic, "b_critic") + for name in ("velocity", "velocity_bias", "second_moment", "second_moment_bias"): + near(getattr(one.learner, name), getattr(beside.learner, name), name) + assert one.learner.updates == beside.learner.updates + assert one.learner.contrast_updates == beside.learner.contrast_updates + assert one.basal_ganglia.rng.bit_generator.state == beside.basal_ganglia.rng.bit_generator.state + if one.recall is not None: + for name in MAPS: + for ours, theirs in zip(getattr(one.recall, name)[0], + getattr(beside.recall, name)[0], strict=True): + near(ours, theirs, name) + assert one.carried_gains == pytest.approx(beside.carried_gains, abs=1e-9) + near(one.hearing.second, beside.hearing.second, "carried second") + assert one.hearing.count == beside.hearing.count + x = np.eye(INPUTS)[[3]] + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + one.step(x) + one.step(x, reward=np.ones(1), teacher=np.array([2])) # it lives on as any brain + loaded = cd.Brain.load(one.save(tmp_path / "life.npz"), backend="cpu") + np.testing.assert_array_equal(loaded.brain.efficacy, one.brain.efficacy) + assert loaded.carried_gains == one.carried_gains + + +def test_a_reset_starts_every_stream_afresh_and_keeps_what_the_lives_learned(): + place = {"backend": "torch", "device": "cpu", "precision": "float64"} + lives = some_lives(3, **place) + brains = [lone(s, **place) for s in range(3)] + a_life(lives, brains, 6) + learned = lives.parameters(2)["efficacy"] + lives.reset() + for brain in brains: + brain.reset() + np.testing.assert_array_equal(lives.parameters(2)["efficacy"], learned) + assert a_life(lives, brains, 9, seed=1) == 0 + assert max(gaps(lives, brains).values()) < 1e-9 + + +def test_the_lives_recall_memory_rests_on_the_device_alone(): + lives = some_lives(2, backend="torch", device="cpu") + with pytest.raises(RuntimeError, match="rests on their device"): + lives.brain.recall.keys # noqa: B018 # a host read of the lives' slots + + +@pytest.mark.parametrize( + ("options", "named"), + [({"episodic": True}, "episodic=False"), ({"arousal": True}, "no arousal"), + ({"apical": 0.25, "modules": (16, 8)}, "apical"), ({"recall_heads": 2}, "one head"), + ({"efference_amplitude": 1.0}, "efference"), ({"learning_rehearsal": 0.5}, "rehearsal"), + ({"recall_cortex_norm": True}, "cortex at unit norm")], +) +def test_lives_refuse_a_part_their_axis_does_not_cover(options, named): + with pytest.raises(ValueError, match=named): + cd.Lives(INPUTS, ACTIONS, 2, backend="torch", device="cpu", **{**OPTIONS, **options}) + + +def test_lives_refuse_a_code_drawn_from_a_seed(): + with pytest.raises(ValueError, match="code of the observation"): + cd.Lives(32, ACTIONS, 2, backend="torch", device="cpu", code=64, **OPTIONS) + + +def test_a_moment_of_lives_takes_one_row_per_life(): + lives = some_lives(3, backend="torch", device="cpu") + with pytest.raises(ValueError, match="one row"): + lives.step(np.eye(INPUTS)[[1, 2]]) + with pytest.raises(ValueError, match="one seed per life"): + cd.Lives(INPUTS, ACTIONS, 3, seeds=[1, 2], **OPTIONS) + with pytest.raises(ValueError, match="vector of inputs"): + cd.Lives((8, 8), ACTIONS, 2) diff --git a/tests/test_memories_on_device.py b/tests/test_memories_on_device.py index a81ec33e..50d2e360 100644 --- a/tests/test_memories_on_device.py +++ b/tests/test_memories_on_device.py @@ -262,6 +262,7 @@ def refused(*_args, **_kwargs): raise ValueError("refused") monkeypatch.setattr(placed.learner, "_apply_device", refused) + monkeypatch.setattr(placed.learner, "_device_candidate", refused) monkeypatch.setattr(placed.learner, "apply", refused) with pytest.raises(ValueError, match="refused"): placed.learn(rng.random(2), np.zeros(2, dtype=bool), observations(rng, 2)) diff --git a/tests/test_memory_fade.py b/tests/test_memory_fade.py new file mode 100644 index 00000000..b72ad66e --- /dev/null +++ b/tests/test_memory_fade.py @@ -0,0 +1,247 @@ +"""``memory_fade``: records nobody uses fade, slower the deeper their use (#253). + +Each key coordinate of the store's consolidated records keeps a depth (the unit cue's magnitude +summed over the moments that cue it and the writes that use it) and a trace of its recent +cueing. Every moment of life an idle record keeps ``1 - fade / (1 + (depth / FADE_SCALE) ** +FADE_ORDER)`` of itself. Off (the founder) the brain is the founder's to the bit. +""" + +from __future__ import annotations + +import copy +import json +import tempfile +from pathlib import Path + +import numpy as np +import pytest + +import cadence as cd +from cadence.memory import SynapticMemory + + +def life(brain, moments=300, seed=1, idle=0): + """Six binary inputs, three actions, paid when the action is the count of active inputs + modulo three; then ``idle`` blank moments with nothing paid.""" + rng = np.random.default_rng(seed) + reward = None + for _ in range(moments): + x = (rng.random((1, 6)) < 0.4).astype(float) + action = brain.live(x, reward=None if reward is None else np.array([reward])) + reward = float(action[0] == int(x[0].sum()) % 3) + for _ in range(idle): + brain.live(np.zeros((1, 6)), reward=None if reward is None else np.array([reward])) + reward = 0.0 + return brain + + +def checkpoint(brain, folder): + path = Path(folder) / "brain.npz" + brain.save(path) + return path + + +def test_off_is_the_founder_to_the_bit(): + founder = life(cd.Brain.compose(6, 3, seed=2, arousal=True, recall=False)) + off = life(cd.Brain.compose(6, 3, seed=2, arousal=True, memory_fade=0.0, recall=False)) + assert off.hippocampus.fade == 0.0 and off.hippocampus.fade_depth is None + with tempfile.TemporaryDirectory() as a, tempfile.TemporaryDirectory() as b: + left, right = np.load(checkpoint(founder, a)), np.load(checkpoint(off, b)) + assert sorted(left.files) == sorted(right.files) + for name in left.files: + assert np.array_equal(left[name], right[name]), name + assert "memory_fade" not in founder.describe()["genes"] + assert founder.parameters() == off.parameters() + + +def test_a_record_in_use_keeps_and_an_idle_one_fades(): + store = SynapticMemory(np.arange(4), np.arange(4, 6), fade=0.01) + key, value = np.eye(4)[[0]], np.array([[1.0, 0.0]]) + store.reset(1) + for _ in range(50): # coordinate 0 written and cued, coordinate 1 written once then idle + store.observe(key, value) + store.fade_moment(key) + store.observe(np.eye(4)[[1]], value) + used = store.consolidated.copy() + for _ in range(200): # coordinate 0 stays in use + store.fade_moment(key) + assert store.consolidated[0, 0] == pytest.approx(used[0, 0], rel=1e-6) + assert store.consolidated[1, 0] < 0.5 * used[1, 0] + assert store.fade_depth[0] > store.fade_depth[1] > 0 + + +def test_only_a_row_that_writes_deepens_its_records(): + store = SynapticMemory(np.arange(3), np.arange(3, 5), fade=0.01) + store.reset(2) + keys, values = np.eye(3)[[0, 1]], np.array([[1.0, 0.0], [0.0, 1.0]]) + mask = np.array([[True, False], [False, False]]) # the second row observes no value: no write + store.observe(keys, values, value_mask=mask) + assert store.fade_depth.tolist() == [1.0, 0.0, 0.0] + store.observe(keys, values, np.array([False, True])) # the first row is gated off + assert store.fade_depth.tolist() == [1.0, 1.0, 0.0] + + +def test_a_deep_record_fades_far_more_slowly(): + shallow = SynapticMemory(np.arange(2), np.arange(2, 4), fade=1e-3) + deep = SynapticMemory(np.arange(2), np.arange(2, 4), fade=1e-3) + key, value = np.eye(2)[[0]], np.array([[1.0, 0.0]]) + for store, moments in ((shallow, 100), (deep, 6000)): + store.reset(1) + for _ in range(moments): + store.observe(key, value) + store.fade_moment(key) + start = shallow.consolidated[0, 0], deep.consolidated[0, 0] + for _ in range(30000): # idle: nothing cues either + shallow.fade_moment(None) + deep.fade_moment(None) + assert shallow.consolidated[0, 0] < 0.05 * start[0] + assert deep.consolidated[0, 0] > 0.9 * start[1] + + +def test_every_moment_of_life_cues_and_fades(): + brain = life(cd.Brain.compose(6, 3, seed=2, arousal=True, memory_fade=0.05, recall=False), + moments=200) + store = brain.hippocampus + depth, recent = store.fade_depth.copy(), store.fade_recent.copy() + assert depth.min() > 0 and recent.min() > 0 + life(brain, moments=0, idle=100) # blank moments cue nothing: the depth stays, the trace decays + assert np.array_equal(store.fade_depth, depth) + assert np.allclose(store.fade_recent, recent * (1 - 1 / store.FADE_IDLE) ** 100) + store.FADE_IDLE = 5.0 # a short memory of use: the idle records fade within the stretch + before = np.abs(store.consolidated).sum() + life(brain, moments=0, idle=400) + assert np.abs(store.consolidated).sum() < 0.5 * before + + +def test_save_and_load_continue_bit_for_bit(): + brain = life(cd.Brain.compose(6, 3, seed=2, arousal=True, memory_fade=0.01, maturity=True, + recall=False), idle=50) + with tempfile.TemporaryDirectory() as folder: + loaded = cd.Brain.load(checkpoint(brain, folder)) + for name in ("fade_depth", "fade_recent", "consolidated"): + assert np.array_equal(getattr(loaded.hippocampus, name), getattr(brain.hippocampus, name)) + assert loaded.hippocampus.fade == brain.hippocampus.fade + assert loaded.describe()["genes"]["memory_fade"] == 0.01 + life(brain, moments=60, seed=4, idle=20) + life(loaded, moments=60, seed=4, idle=20) + for name in ("fade_depth", "fade_recent", "consolidated"): + assert np.array_equal(getattr(loaded.hippocampus, name), getattr(brain.hippocampus, name)) + + +def saved_format(path): + with np.load(path) as archive: + return json.loads(str(archive["brain"]))["format"] + + +def test_a_fading_brain_saves_a_format_of_its_own(monkeypatch): + def brain(**settings): + return life(cd.Brain.compose(6, 3, seed=2, arousal=True, recall=False, **settings), + moments=40, idle=10) + + with tempfile.TemporaryDirectory() as folder: + # without the fade: the founder's format, and maturity's, as before + for settings, name in (({}, "cadence-brain/1"), ({"maturity": True}, "cadence-brain/2")): + path = checkpoint(brain(**settings), folder) + assert saved_format(path) == name + loaded = cd.Brain.load(path) + assert loaded.hippocampus.fade == 0.0 and loaded.hippocampus.fade_depth is None + assert saved_format(checkpoint(loaded, folder)) == name + # with the fade: one name, with a maturity or without + for settings in ({"memory_fade": 0.01}, {"memory_fade": 0.01, "maturity": True}): + fading = brain(**settings) + path = checkpoint(fading, folder) + arrays = dict(np.load(path)) + meta = json.loads(str(arrays["brain"])) + assert meta["format"] == "cadence-brain/3" # which 0.95.2 and earlier refuse + assert ("maturity" in meta) == ("maturity" in settings) + loaded = cd.Brain.load(path) + assert loaded.hippocampus.fade == 0.01 + for name in ("fade_depth", "fade_recent", "consolidated", "strength"): + assert np.array_equal(getattr(loaded.hippocampus, name), getattr(fading.hippocampus, name)) + assert (loaded.basal_ganglia.maturity is None) == ("maturity" not in settings) + assert saved_format(checkpoint(loaded, folder)) == "cadence-brain/3" + # the name and the fade belong together + store = meta["memories"]["hippocampus"] + faded_out = {**meta, "memories": {**meta["memories"], + "hippocampus": {k: v for k, v in store.items() if k != "fade"}}} + for changed in ({**meta, "format": "cadence-brain/1"}, {**meta, "format": "cadence-brain/2"}, + faded_out): + np.savez(path, **{**arrays, "brain": np.array(json.dumps(changed))}) + with pytest.raises(ValueError, match="belongs to checkpoint format cadence-brain/3"): + cd.Brain.load(path) + # a loader that knows only the earlier names refuses the file, as 0.93.1 refuses maturity's + np.savez(path, **arrays) + with monkeypatch.context() as earlier: + earlier.setattr("cadence.generic.FADE_FORMAT", "cadence-brain/unknown") + with pytest.raises(ValueError, match="unsupported Brain checkpoint format"): + cd.Brain.load(path) + assert cd.Brain.load(path).hippocampus.fade == 0.01 + + +def test_parameters_count_the_depths_and_traces(): + off = cd.Brain.compose(6, 3, seed=2, recall=False) + on = cd.Brain.compose(6, 3, seed=2, memory_fade=0.01, recall=False) + assert on.parameters() - off.parameters() == 2 * on.hippocampus.key_width + + +def test_refusals(): + with pytest.raises(ValueError, match="memory_fade must lie in"): + cd.Brain.compose(6, 3, memory_fade=1.0, recall=False) + with pytest.raises(ValueError, match="memory_fade"): + cd.Brain.compose(6, 3, memory_fade=-0.1, recall=False) + with pytest.raises(ValueError, match="memory_fade needs the episodic memory"): + cd.Brain.compose(6, 3, memory_fade=0.01, episodic=False, recall=False) + brain = cd.Brain.compose(6, 3, memory_fade=0.01, recall=False) + with pytest.raises(ValueError, match="construction"): + brain.retune(memory_fade=0.02) + + +def test_step_moments_fade_and_a_refused_moment_changes_nothing(monkeypatch): + brain = cd.Brain.compose(6, 3, seed=2, memory_fade=0.01, recall=False) + brain.step(np.ones((1, 6))) + brain.step(np.ones((1, 6)), reward=np.array([1.0])) + assert brain.hippocampus.fade_depth.sum() > 0 + before = {n: getattr(brain.hippocampus, n).copy() for n in ("fade_depth", "fade_recent", "consolidated")} + with pytest.raises(ValueError): + brain.step(np.ones((1, 5))) # the wrong width is refused before the moment + for name, value in before.items(): + assert np.array_equal(getattr(brain.hippocampus, name), value), name + + +def test_learn_on_its_own_cues_by_the_next_observations(): + brain = cd.Brain.compose(6, 3, seed=2, memory_fade=0.01, recall=False) + x = np.eye(6)[[1]] + brain.act(x) + depth = brain.hippocampus.fade_depth.copy() + recent = brain.hippocampus.fade_recent.copy() + brain.learn(np.array([1.0]), np.array([False]), np.eye(6)[[4]]) # reward, done, next + kept = 1 - 1 / brain.hippocampus.FADE_IDLE # the trace takes the cue, then decays a moment + moved = brain.hippocampus.fade_recent - recent * kept + assert moved[4] == pytest.approx(kept) and np.allclose(moved[[0, 1, 2, 3, 5]], 0.0) + assert brain.hippocampus.fade_depth[4] - depth[4] == pytest.approx(1.0) + one = cd.Brain.compose(1, 2, seed=2, memory_fade=0.01, recall=False) # a key as wide as the reward + one.act(np.zeros((1, 1))) + one.learn(np.array([1.0]), np.array([False]), np.zeros((1, 1))) # a blank moment cues nothing + assert not one.hippocampus.fade_recent.any() + + +def test_grow_and_graft_keep_the_fade(): + brain = life(cd.Brain.compose(6, 3, seed=2, arousal=True, memory_fade=0.01, recall=False), + moments=120, idle=30) + grown = brain.grow(efference_amplitude=1.0) + with tempfile.TemporaryDirectory() as folder: + assert saved_format(checkpoint(grown, folder)) == "cadence-brain/3" + assert grown.hippocampus.fade == brain.hippocampus.fade + for name in ("fade_depth", "fade_recent"): + assert np.array_equal(getattr(grown.hippocampus, name), getattr(brain.hippocampus, name)) + grafted = brain.graft(copy.deepcopy(brain), ["sensory", "association", "motor"]) + assert grafted.hippocampus.fade == brain.hippocampus.fade + for name in ("fade_depth", "fade_recent"): + assert np.array_equal(getattr(grafted.hippocampus, name), getattr(brain.hippocampus, name)) + + +def test_clear_erases_the_depths_with_the_records(): + brain = life(cd.Brain.compose(6, 3, seed=2, arousal=True, memory_fade=0.01, recall=False), + moments=80) + brain.hippocampus.clear() + assert not brain.hippocampus.fade_depth.any() and not brain.hippocampus.fade_recent.any() diff --git a/tests/test_moment_on_device.py b/tests/test_moment_on_device.py index 90ed161b..74a9ba16 100644 --- a/tests/test_moment_on_device.py +++ b/tests/test_moment_on_device.py @@ -17,6 +17,9 @@ DEVICES = ["cpu", *(["cuda"] if torch.cuda.is_available() else [])] +# an outcome's one read where the critic rests on the device: its eleven checks and numbers, then +# the TD error and the dopamine of every row +OUTCOME_READ = 11 def brain_on_torch(seed: int = 3) -> cd.Brain: @@ -377,7 +380,7 @@ def refused(*_args, **_kwargs): raise ValueError("refused") with monkeypatch.context() as patch: - patch.setattr(brain.learner, "_apply_device", refused) + patch.setattr(brain.learner, "_device_candidate", refused) # the update's, refused with pytest.raises(ValueError, match="refused"): brain.learn(np.ones(3), np.zeros(3, dtype=bool), x) assert agent._critic_device is held and "trace_critic" not in agent.__dict__ @@ -540,7 +543,7 @@ def counted(self, *args, **kwargs): brain.lesson(x, np.arange(rows) % motor) brain.step(x) brain.step(observed(), reward=reward) - assert reads and max(reads) <= rows * motor + assert reads and max(reads) <= max(rows * motor, OUTCOME_READ + 2 * rows) @pytest.mark.parametrize("device", DEVICES) @@ -567,7 +570,7 @@ def counted(self, *args, **kwargs): brain.wait(x) else: brain.live(x, reward=np.array([float(t % 3 == 0)])) - assert reads and max(reads) <= motor + assert reads and max(reads) <= max(motor, OUTCOME_READ + 2) @pytest.mark.parametrize("device", ["cpu", "torch"]) @@ -606,3 +609,69 @@ def test_a_brain_grown_while_it_streams_is_the_brain_grown_after_a_reset(device, assert set(a.files) == set(b.files) for name in a.files: np.testing.assert_array_equal(a[name], b[name], err_msg=name) + + +# -- an outcome's reads (cadence#274: the critic and the dopamine on the device) + + +def _reads_of(monkeypatch, call) -> list[str]: + """Every read of a torch tensor's numbers on the host that ``call`` makes, by kind.""" + reads: list[str] = [] + for name in ("cpu", "item", "tolist", "__bool__", "__float__", "__int__"): + original = getattr(torch.Tensor, name) + + def counted(self, *args, _original=original, _name=name, **kwargs): + reads.append(_name) + return _original(self, *args, **kwargs) + + monkeypatch.setattr(torch.Tensor, name, counted) + call() + monkeypatch.undo() + return reads + + +@pytest.mark.parametrize("precision", ["float32", "float64"]) +def test_an_outcome_reads_its_checks_and_dopamine_once(monkeypatch, precision): + """Where the critic rests on the device the dopamine is made there: the outcome's checks (the + dopamine's, the critic's and the update's), the critic's bias step, the report's steps and the + dopamine itself come to the host in one read before anything is installed.""" + monkeypatch.setenv("CADENCE_TORCH_MEMORY", "1") + brain = cd.Brain.compose(12, 5, modules=(16,), seed=3, backend="torch", device="cpu", + precision=precision, recall=False) + _a_life_of_rewards(brain, moments=5) + x = np.eye(12)[[1, 7, 3]] + brain.act(x) + assert brain.basal_ganglia._critic_device is not None + with monkeypatch.context() as patch: + patch.setenv("CADENCE_TORCH_MEMORY", "1") + reads = _reads_of( + patch, lambda: brain.learn(np.ones(3), np.array([False, True, False]), x) + ) + # the next state's settle (its stop, one sweep behind, and its residual check), the store's + # write, the one read of the outcome and the report's trace + settle = reads.count("item") + 1 + assert len(reads) - settle <= 4, reads + + +def test_a_refused_dopamine_leaves_the_outcome_pending(monkeypatch): + monkeypatch.setenv("CADENCE_TORCH_MEMORY", "1") + brain = brain_on_torch() + _a_life_of_rewards(brain, moments=5) + x = np.eye(12)[[1, 7, 3]] + brain.act(x) + agent = brain.basal_ganglia + held = agent._critic_device + weights, scale = held.weights.clone(), brain.brain._torch.scale.clone() + updates = brain.learner.updates + + def unbounded(state): + return torch.full((3,), float("inf"), dtype=held.weights.dtype) + + with monkeypatch.context() as patch: + patch.setattr(held, "value_here", unbounded) + with pytest.raises(ValueError, match="TD errors must remain finite"): + brain.learn(np.ones(3), np.zeros(3, dtype=bool), x) + assert agent._pending is not None and brain.learner.updates == updates + assert torch.equal(held.weights, weights) and torch.equal(brain.brain._torch.scale, scale) + brain.learn(np.ones(3), np.zeros(3, dtype=bool), x) # the same outcome, accepted + assert brain.learner.updates == updates + 1 diff --git a/tests/test_observers_compat.py b/tests/test_observers_compat.py index bb767654..19c323f4 100644 --- a/tests/test_observers_compat.py +++ b/tests/test_observers_compat.py @@ -20,13 +20,13 @@ "efference and history": {"efference_amplitude": 1.0, "history_decays": (0.5, 0.9), "apical": 0.25}, "torch": {"apical": 0.25, "recall": True, "recall_founder": True, "backend": "torch"}, - "copy and own actions": {"recall": True, "recall_founder": True, "recall_copy": 1.0, "recall_tokens": [0, 1, 2], + "tokens and own actions": {"recall": True, "recall_founder": True, "recall_tokens": [0, 1, 2], "recall_action": True, "apical": 0.25}, - "held heads, copy and own actions": { - "learned_code": 8, "attention": True, "recall": True, "recall_heads": 2, "recall_copy": 1.0, + "held heads, tokens and own actions": { + "learned_code": 8, "attention": True, "recall": True, "recall_heads": 2, "recall_tokens": [0, 1, 2], "recall_action": True, "apical": 0.25, }, - "torch copy and own actions": {"apical": 0.25, "recall": True, "recall_founder": True, "recall_copy": 1.0, + "torch tokens and own actions": {"apical": 0.25, "recall": True, "recall_founder": True, "recall_tokens": [0, 1, 2], "recall_action": True, "backend": "torch"}, } diff --git a/tests/test_rate_zero.py b/tests/test_rate_zero.py new file mode 100644 index 00000000..b4fb5c4b --- /dev/null +++ b/tests/test_rate_zero.py @@ -0,0 +1,147 @@ +"""A synapse rate of zero holds the synapse, its optimizer moments and, where every inbound +synapse of a neuron steps at zero, that neuron's bias (``Learner.held``).""" + +import warnings + +import numpy as np +import pytest + +import cadence as cd + + +def held_motor(backend="cpu", **place): + """A composed brain whose motor neurons step at rate zero on every synapse they touch.""" + brain = cd.Brain.compose(6, 3, modules=(12,), seed=1, backend=backend, recall=False, + teaching="deep", **place) + w = brain.connectome + motor = np.zeros(w.n, dtype=bool) + motor[np.asarray(w.populations["motor"])] = True + rate = np.ones(w.synapses) + rate[motor[w.post] | motor[w.pre]] = 0.0 + brain.learner.synapse_rate = rate + return brain, rate, motor + + +def teach(brain, lessons=3): + """Lessons, and sampled acts paid by reward: both reach the held synapses' step.""" + rng = np.random.default_rng(0) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + for _ in range(lessons): + brain.lesson(rng.random((4, 6)), rng.integers(3, size=4)) + brain.step(rng.random((4, 6))) + brain.step(rng.random((4, 6)), reward=rng.random(4) - 0.5) + brain.learn(rng.random(4) - 0.5, np.zeros(4, dtype=bool), rng.random((4, 6))) + + +def check_held(brain, rate, motor, efficacy, bias): + learner = brain.learner + moved_efficacy = np.abs(np.asarray(brain.brain.efficacy, dtype=float) - efficacy) + moved_bias = np.abs(np.asarray(brain.brain.bias, dtype=float) - bias) + assert moved_efficacy[rate == 0].max() == 0.0 + assert moved_bias[motor].max() == 0.0 + assert moved_efficacy[rate > 0].max() > 0 and moved_bias[~motor].max() > 0 + for name in ("velocity", "second_moment"): + assert np.abs(getattr(learner, name)[rate == 0]).max() == 0.0 + assert np.abs(getattr(learner, name)[rate > 0]).max() > 0 + for name in ("velocity_bias", "second_moment_bias"): + assert np.abs(getattr(learner, name)[motor]).max() == 0.0 + + +def test_held_names_the_synapses_and_the_neurons_they_stop(): + brain, rate, motor = held_motor() + synapses, neurons = brain.learner.held() + assert np.array_equal(synapses, rate == 0) and np.array_equal(neurons, motor) + brain.learner.synapse_rate = np.where(rate == 0, 0.5, 1.0) + assert brain.learner.held() == (None, None) + brain.learner.synapse_rate = None + assert brain.learner.held() == (None, None) + + +def test_a_neuron_with_one_moving_inbound_synapse_steps_its_bias(): + brain, rate, motor = held_motor() + w = brain.connectome + first = int(np.flatnonzero(motor)[0]) + inbound = np.flatnonzero(w.post == first) + rate = rate.copy() + rate[inbound[0]] = 1.0 + brain.learner.synapse_rate = rate + neurons = brain.learner.held()[1] + assert not neurons[first] and neurons[motor].sum() == motor.sum() - 1 + + +def test_rate_zero_holds_synapses_biases_and_moments_on_the_cpu(): + brain, rate, motor = held_motor() + efficacy, bias = np.array(brain.brain.efficacy), np.array(brain.brain.bias) + teach(brain) + check_held(brain, rate, motor, efficacy, bias) + + +def test_rate_zero_holds_on_the_numpy_path(monkeypatch): + from cadence import learning + + monkeypatch.setattr(learning, "_compiled_passes", lambda: False) + brain, rate, motor = held_motor() + efficacy, bias = np.array(brain.brain.efficacy), np.array(brain.brain.bias) + teach(brain) + check_held(brain, rate, motor, efficacy, bias) + + +def test_rate_zero_holds_on_torch(): + pytest.importorskip("torch") + brain, rate, motor = held_motor("torch", device="cpu", precision="float64") + efficacy, bias = np.array(brain.brain.efficacy), np.array(brain.brain.bias) + teach(brain) + check_held(brain, rate, motor, efficacy, bias) + + +def test_rate_zero_holds_on_mlx(): + pytest.importorskip("mlx.core") + brain, rate, motor = held_motor("mlx") + efficacy, bias = np.array(brain.brain.efficacy), np.array(brain.brain.bias) + teach(brain) + check_held(brain, rate, motor, efficacy, bias) + + +def test_rate_zero_holds_in_lives_on_torch(): + """``Lives`` on a torch device keep each life's held synapses, biases and moments.""" + pytest.importorskip("torch") + lives = cd.Lives(6, 3, 2, modules=(12,), backend="torch", device="cpu", precision="float64", + recall=False, teaching="deep") + learner, w = lives.brain.learner, lives.brain.connectome + motor = np.zeros(w.n, dtype=bool) + motor[np.asarray(w.populations["motor"])] = True + rate = np.ones(w.synapses) + rate[motor[w.post] | motor[w.pre]] = 0.0 + learner.synapse_rate = rate + before = [lives.parameters(i) for i in range(2)] + rng = np.random.default_rng(0) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + for _ in range(3): + lives.lesson(rng.random((2, 6)), rng.integers(3, size=2)) + lives.step(rng.random((2, 6))) + lives.learn(rng.random(2) - 0.5, np.zeros(2, dtype=bool), rng.random((2, 6))) + for i in range(2): + after = lives.parameters(i) + assert np.abs(after["efficacy"] - before[i]["efficacy"])[rate == 0].max() == 0.0 + assert np.abs(after["efficacy"] - before[i]["efficacy"])[rate > 0].max() > 0 + assert np.abs(after["bias"] - before[i]["bias"])[motor].max() == 0.0 + for name in ("velocity", "second_moment"): + assert np.abs(after[name][rate == 0]).max() == 0.0 + assert np.abs(after[name][rate > 0]).max() > 0 + for name in ("velocity_bias", "second_moment_bias"): + assert np.abs(after[name][motor]).max() == 0.0 + + +def test_rates_above_zero_teach_as_before(): + """Without a rate of zero nothing is held, and lessons teach to the bit as with no rate.""" + plain, _, _ = held_motor() + rated, _, _ = held_motor() + plain.learner.synapse_rate = None + rated.learner.synapse_rate = np.ones(rated.connectome.synapses) + teach(plain) + teach(rated) + assert np.array_equal(plain.brain.efficacy, rated.brain.efficacy) + assert np.array_equal(plain.brain.bias, rated.brain.bias) + assert np.array_equal(plain.learner.second_moment, rated.learner.second_moment) diff --git a/tests/test_recall_composed.py b/tests/test_recall_composed.py index 1d392ba7..1b58145d 100644 --- a/tests/test_recall_composed.py +++ b/tests/test_recall_composed.py @@ -85,7 +85,6 @@ def test_a_new_memory_teaches_at_the_deep_point_with_no_apical(): [{"recall_position": 0}, {"recall_position_rate": 0.0}, {"recall_heads": 2}, {"recall_window": 2}, {"recall_next": False}, {"recall_action": True}, {"recall_when": True}, {"recall_capacity": 8}, - {"recall_copy": 2.0, "recall_tokens": [0, 1, 2, 3]}, {"recall_gain": 1.0, "recall_tokens": [0, 1, 2, 3]}, {"recall_address": 0.0}, {"recall_address_reward": 1.0}, {"recall_position_reward": 1.0}, {"recall_grid": ()}, {"recall_position": 64}, {"carried_gain": 0.0}, {"recall_drive_gain": False}], @@ -465,7 +464,7 @@ def test_the_width_is_fixed_at_birth_so_growth_keeps_its_rates(): @pytest.mark.parametrize( "options", - [{"observers": (8,)}, {"modules": (32, 96)}, {"recall_heads": 2}, {"recall_copy": 2.0, "recall_tokens": [0, 1, 2, 3]}, + [{"observers": (8,)}, {"modules": (32, 96)}, {"recall_heads": 2}, {"recall_gain": 1.0, "recall_tokens": [0, 1, 2, 3]}, {"recall_when": True}, {"history_decays": (0.5,)}, {"efference_amplitude": 1.0}, {"learned_code": 16}, {"apical": 0.25}, {"arousal": True, "maturity": True}, {"memory_choice": True}, {"learning_rehearsal": 1.0}, {"inputs": (6, 6)}, {"inputs": 64, "code": 32}], diff --git a/tests/test_recall_copy.py b/tests/test_recall_copy.py deleted file mode 100644 index d1535d53..00000000 --- a/tests/test_recall_copy.py +++ /dev/null @@ -1,226 +0,0 @@ -"""The copy pathway (``recall_copy``, ``recall_tokens``): plastic synapses from every head's read -port into the motor region, learned by the settle's contrast. With declared tokens it starts as -their identity from the recall ports, at zero from the next ports and as the negative identity -from the present observation (the read's echo). The founder has none; the options are described -and saved where they are on. -""" -from __future__ import annotations - -import warnings - -import numpy as np -import pytest - -import cadence as cd - -TOKENS = 6 -INPUTS = TOKENS + 3 # the tokens, a HEAR, an ASK and a NOISE flag -HEAR, ASK, NOISE = TOKENS, TOKENS + 1, TOKENS + 2 - - -def composed(**options): - options.setdefault("modules", (16,)) - return cd.Brain.compose(INPUTS, TOKENS, seed=3, episodic=False, recall=True, recall_founder=True, **options) - - -def event(*on): - x = np.zeros((1, INPUTS)) - x[0, list(on)] = 1.0 - return x - - -def pathway(brain): - """The copy synapses as ``(port, value coordinate, motor) -> efficacy`` and their groups.""" - synapses, value, motor = brain.recall.copy_synapses() - pre = np.asarray(brain.connectome.pre)[synapses] - ports = {} - for h, (recall, following) in enumerate(brain.recall._heads): # noqa: SLF001 - for name, region in (("recall", recall), ("next", following)): - ports.update({int(i): (name, h) for i in region}) - return [ - (ports[int(p)], int(v), int(m), float(brain.brain.efficacy[s]), s) - for p, v, m, s in zip(pre, value, motor, synapses, strict=True) - ] - - -def test_the_founder_has_no_pathway_and_the_options_are_described_where_on(): - founder = composed() - assert founder.recall.copy == 0.0 and founder.recall.tokens is None - assert not len(founder.recall.copy_synapses()[0]) - genes = founder.describe()["genes"] - assert "recall_copy" not in genes and "recall_tokens" not in genes - tokened = composed(recall_copy=2.0, recall_tokens=range(TOKENS)) - genes = tokened.describe()["genes"] - assert genes["recall_copy"] == 2.0 and genes["recall_tokens"] == list(range(TOKENS)) - - -def test_a_random_pathway_is_the_brain_without_it_plus_its_synapses(): - founder, copied = composed(), composed(recall_copy=0.5) - synapses = copied.recall.copy_synapses()[0] - assert len(synapses) == 2 * INPUTS * TOKENS # every recall and next neuron, every action - keep = np.ones(copied.connectome.synapses, dtype=bool) - keep[synapses] = False - a, b = founder.connectome, copied.connectome - pairs_a = set(zip(a.pre.tolist(), a.post.tolist(), a.sign.tolist(), strict=True)) - pairs_b = set(zip(b.pre[keep].tolist(), b.post[keep].tolist(), b.sign[keep].tolist(), - strict=True)) - assert pairs_a == pairs_b # every other synapse, drawn as before - magnitude = np.abs(copied.connectome.sign[synapses]) - assert magnitude.max() > 0 and magnitude.max() <= 0.5 * np.sqrt(6.0 / (INPUTS + TOKENS)) - - -def test_declared_tokens_start_the_pathway_as_their_identity_with_echo_suppression(): - brain = composed(recall_copy=2.0, recall_tokens=range(TOKENS)) - for (port, _h), value, motor, weight, _ in pathway(brain): - expected = 2.0 if port == "recall" and value == motor else 0.0 - assert weight == expected, (port, value, motor) - pre, post = np.asarray(brain.connectome.pre), np.asarray(brain.connectome.post) - echo = np.isin(pre, brain.sensory_index) & np.isin(post, brain.motor_index) - sensory = {int(n): i for i, n in enumerate(brain.sensory_index)} - motor = {int(n): i for i, n in enumerate(brain.motor_index)} - for s in np.flatnonzero(echo): - same = sensory[int(pre[s])] == motor[int(post[s])] - assert brain.brain.efficacy[s] == (-2.0 if same else 0.0) - assert brain.learner.tie_groups is None # every synapse of the pathway learns on its own - - -def test_a_newborn_produces_a_token_it_was_never_taught_from_its_read(): - brain = composed(recall_copy=2.0, recall_tokens=range(TOKENS)) - for token in range(TOKENS): - brain.reset() - brain.act(event(HEAR, token), greedy=True) - brain.act(event(NOISE), greedy=True) # a moment without the token - assert brain.act(event(HEAR, ASK), greedy=True)[0] == token - - -def test_a_lesson_moves_the_synapses_the_read_used_and_leaves_untaught_tokens(): - brain = composed(recall_copy=2.0, recall_tokens=range(TOKENS)) - with warnings.catch_warnings(): - warnings.simplefilter("error") - brain.act(event(HEAR, 2), greedy=True) - brain.lesson(event(HEAR, ASK), [4]) # the read names 2, the lesson asks for 4 - weights = {(port, value, motor): weight for (port, _h), value, motor, weight, _ in - pathway(brain)} - assert weights[("recall", 2, 2)] < 2.0 # the read named the wrong token: it weakened - assert weights[("recall", 2, 4)] > 0.0 # and the read's token learns the taught action - assert all(weights[("recall", t, t)] == 2.0 for t in (0, 1, 3, 5)) # untouched, untaught - - -def test_the_pathway_and_its_ties_survive_a_checkpoint(tmp_path): - brain = composed(recall_copy=2.0, recall_tokens=range(TOKENS), recall_heads=2) - brain.act(event(HEAR, 1), greedy=True) - brain.lesson(event(HEAR, ASK), [1]) - twin = cd.Brain.load(brain.save(tmp_path / "copy.npz")) - assert twin.recall.copy == 2.0 and twin.recall.tokens == tuple(range(TOKENS)) - np.testing.assert_array_equal(twin.brain.efficacy, brain.brain.efficacy) - np.testing.assert_array_equal(twin.connectome.sign, brain.connectome.sign) - for b in (brain, twin): - b.lesson(event(HEAR, ASK), [3]) - np.testing.assert_array_equal(twin.brain.efficacy, brain.brain.efficacy) - assert twin.describe() == brain.describe() - - -def test_several_heads_each_have_their_ports_on_the_pathway(): - brain = composed(recall_copy=1.0, recall_tokens=range(TOKENS), recall_heads=2) - ports = {port for port, *_ in pathway(brain)} - assert ports == {("recall", 0), ("next", 0), ("recall", 1), ("next", 1)} - for (port, _h), value, motor, weight, _ in pathway(brain): - assert weight == (1.0 if port == "recall" and value == motor else 0.0) - - -def test_a_joined_learned_code_takes_tokens_after_its_units(): - brain = cd.Brain.compose( - 8, 4, modules=(16,), seed=0, learned_code=16, attention=True, - recall_copy=1.0, recall_tokens=[0, 1, 2, 3], - ) - for (port, _h), value, motor, weight, _ in pathway(brain): - assert weight == (1.0 if port == "recall" and value == 16 + motor else 0.0) - brain.act(np.eye(8)[[1]], greedy=True) - with pytest.raises(ValueError, match="observation's own coordinates"): - cd.Brain.compose(40, 4, modules=(16,), learned_code=16, attention=True, - recall_copy=1.0, recall_tokens=[0, 1, 2, 3]) - - -@pytest.mark.parametrize( - ("options", "message"), - [ - (dict(recall_tokens=range(TOKENS - 1)), "one observation coordinate per"), - (dict(recall_copy=-1.0), "copy"), - (dict(recall_copy=1.0, recall_tokens=range(TOKENS - 1)), "one observation coordinate per"), - (dict(recall_copy=1.0, recall_tokens=[INPUTS] * TOKENS), "below"), - (dict(recall_copy=1.0, recall_tokens=[0.5] * TOKENS), "integer"), - (dict(recall_copy=1.0, recall_tokens="abc"), "sequence"), - ], -) -def test_invalid_pathways_are_refused(options, message): - with pytest.raises(ValueError, match=message): - composed(**options) - - -def test_a_fixed_code_and_a_custom_anatomy_without_the_synapses_are_refused(): - with pytest.raises(ValueError, match="observation's own coordinates"): - cd.Brain.compose(40, 4, code=64, recall=True, recall_founder=True, recall_copy=1.0, - recall_tokens=[0, 1, 2, 3]) - founder = composed() - with pytest.raises(ValueError, match="copy needs synapses"): - cd.Brain(founder.connectome, recall=True, recall_founder=True, recall_copy=1.0, episodic=False) - with pytest.raises(ValueError, match="construction setting"): - composed(recall_copy=1.0).retune(recall_copy=2.0) - - -def test_a_lesson_whose_answer_is_no_token_leaves_the_read_as_it_was(): - # actions 0 to 3 are tokens, 4 and 5 are not (a yes and a no): the read's lessons take - # their target from the slots' contents, which hold no yes or no - tokens = [0, 1, 2, 3, -1, -1] - brain = composed(recall_copy=2.0, recall_tokens=tokens) - brain.act(event(HEAR, 1), greedy=True) - brain.act(event(NOISE, 3), greedy=True) - before = [np.array(m, copy=True) for m in brain.recall.head_maps[0]] - report = brain.lesson(event(HEAR, ASK), [5]) - assert report["address_rows"] == 0.0 - for a, b in zip(before, brain.recall.head_maps[0], strict=True): - np.testing.assert_array_equal(a, b) - report = brain.lesson(event(HEAR, ASK), [3]) # a token: the read learns - assert report["address_rows"] == 1.0 - assert any(not np.array_equal(a, b) - for a, b in zip(before, brain.recall.head_maps[0], strict=True)) - # every action declared no token: the pathway starts at zero and no lesson moves the read - none = composed(recall_copy=2.0, recall_tokens=[-1] * TOKENS) - assert not any(weight for *_, weight, _ in pathway(none)) - none.act(event(HEAR, 1), greedy=True) - assert none.lesson(event(HEAR, ASK), [2])["address_rows"] == 0.0 - # without tokens every taught row teaches the read, as before - plain = composed(recall_copy=2.0) - plain.act(event(HEAR, 1), greedy=True) - assert plain.lesson(event(HEAR, ASK), [5])["address_rows"] == 1.0 - - -def test_under_surprise_a_lesson_whose_answer_is_no_token_teaches_every_row(): - x = np.concatenate([event(HEAR, 0), event(HEAR, 1)]) - taught = {} - for name, tokens in (("tokens", list(range(TOKENS))), ("none", [-1] * TOKENS)): - brain = composed(recall_tokens=tokens, learning_surprise=True) - brain.act(x, greedy=True) - answer = np.asarray(brain._greedy( # noqa: SLF001 the act the lesson judges by - brain._qualified(brain.stimulus(x), brain._activity(), operation="act"), # noqa: SLF001 - brain._placed(x), brain._decision_key(brain._coded(x)), # noqa: SLF001 - )) - brain.lesson(x, answer) # the label is the brain's own answer on both rows - taught[name] = brain.last_learning["demonstration_accepted_presentations"] - assert taught["tokens"] == 0.0 # answered as shown: no surprise, no lesson - assert taught["none"] == 2.0 # no token answers: taught at every lesson, right or not - - -def test_tokens_without_the_pathway_declare_which_answers_are_tokens(): - # no copy pathway: the tokens only tell the read's lessons which answers a slot can hold - brain = composed(recall_tokens=[0, 1, 2, 3, -1, -1], learning_surprise=True) - assert not len(brain.recall.copy_synapses()[0]) - assert brain.describe()["genes"]["recall_tokens"] == [0, 1, 2, 3, -1, -1] - plain = composed(learning_surprise=True) - np.testing.assert_array_equal(brain.brain.efficacy, plain.brain.efficacy) - brain.act(event(HEAR, 1), greedy=True) - before = [np.array(m, copy=True) for m in brain.recall.head_maps[0]] - assert brain.lesson(event(HEAR, ASK), [5])["address_rows"] == 0.0 - for a, b in zip(before, brain.recall.head_maps[0], strict=True): - np.testing.assert_array_equal(a, b) - assert brain.last_learning["demonstration_accepted_presentations"] == 1.0 diff --git a/tests/test_recall_cortex_norm.py b/tests/test_recall_cortex_norm.py new file mode 100644 index 00000000..8e2030dd --- /dev/null +++ b/tests/test_recall_cortex_norm.py @@ -0,0 +1,90 @@ +"""``recall_cortex_norm``: every map of the read reads the cortex of the moment before at unit +norm per stream.""" + +from __future__ import annotations + +import numpy as np +import pytest + +import cadence as cd + + +def _brain(**options): + return cd.Brain.compose(12, 6, backend="cpu", recall=True, seed=3, **options) + + +def _life(brain, lessons=4): + rng = np.random.default_rng(0) + for _ in range(lessons): + brain.reset() + for _ in range(3): + x = np.eye(12)[rng.integers(12, size=2)] + brain.lesson(x, rng.integers(6, size=2)) + brain.act(x, greedy=True) + + +def test_the_cortex_the_maps_read_is_at_unit_norm(): + brain = _brain(recall_cortex_norm=True) + brain.reset() + brain.act(np.eye(12)[[1, 2]], greedy=True) + assert np.allclose(np.linalg.norm(brain.recall.previous, axis=1), 1.0) + assert brain.recall.cortex_norm and not brain.recall.resides + + +def test_the_option_is_off_by_default_and_the_life_without_it_is_unchanged(): + plain, named = _brain(), _brain(recall_cortex_norm=False) + assert plain.recall.cortex_norm is False + _life(plain) + _life(named) + for a, b in zip(plain.recall.head_map(0), named.recall.head_map(0), strict=True): + np.testing.assert_array_equal(a, b) + assert np.linalg.norm(plain.recall.previous, axis=1).max() != pytest.approx(1.0) + + +def test_a_life_with_it_differs_and_is_saved_with_it(tmp_path): + plain, normed = _brain(), _brain(recall_cortex_norm=True) + _life(plain) + _life(normed) + assert not np.array_equal(plain.recall.head_map(0)[1], normed.recall.head_map(0)[1]) + loaded = cd.Brain.load(normed.save(tmp_path / "normed")) + assert loaded.recall.cortex_norm is True + assert normed.describe()["genes"]["recall_cortex_norm"] is True + assert "recall_cortex_norm" not in plain.describe()["genes"] + + +def test_it_is_refused_unless_boolean(): + with pytest.raises(ValueError, match="cortex_norm must be boolean"): + _brain(recall_cortex_norm=1) + + +def test_the_compiled_moment_carries_the_cortex_at_unit_norm_too(): + """A greedy act of ``learning_compiled_moment`` writes the moment in its own pass + (``cadence._moment.act_after``): the cortex it carries is the one ``Recall.witness`` carries, + and the life agrees with the ordinary path's to rounding.""" + pytest.importorskip("numba") + from cadence import brain as brain_module + + if not brain_module._FUSED: + pytest.skip("CADENCE_FUSED=0 reads the NumPy loop alone") + + def lived(compiled): + brain = cd.Brain.compose(16, 4, backend="cpu", seed=3, recall_cortex_norm=True, + learning_compiled_moment=compiled) + rng = np.random.default_rng(7) + norms = [] + for t in range(12): + x = (rng.random((2, 16)) < 0.25).astype(float) + brain.act(x, greedy=True) + norms.append(np.linalg.norm(brain.recall.previous, axis=1)) + if t % 4 == 3: + brain.lesson(x, rng.integers(4, size=2)) + return brain, np.array(norms) + + plain, plain_norms = lived(False) + compiled, compiled_norms = lived(True) + assert compiled.recall._compiles(2) # the compiled read and write ran + assert np.allclose(plain_norms, 1.0) and np.allclose(compiled_norms, 1.0) + np.testing.assert_allclose(compiled.recall.previous, plain.recall.previous, atol=1e-9) + np.testing.assert_allclose(compiled.brain.efficacy, plain.brain.efficacy, atol=1e-8) + for a, b in zip(compiled.recall.head_map(0), plain.recall.head_map(0), strict=True): + np.testing.assert_allclose(a, b, atol=1e-8) diff --git a/tests/test_recall_drive_gain.py b/tests/test_recall_drive_gain.py index 0ad7dfc6..44a8c6c8 100644 --- a/tests/test_recall_drive_gain.py +++ b/tests/test_recall_drive_gain.py @@ -396,9 +396,9 @@ def test_the_value_path_reads_the_port_as_it_is_held(): @pytest.mark.parametrize( "options", - [{"learned_code": 32}, {"recall_copy": 0.5, "recall_tokens": (0, 1, 2, 3)}, + [{"learned_code": 32}, {"recall_gain": 1.0, "recall_tokens": (0, 1, 2, 3)}, {"recall_window": 3}, {"recall_next": False}], - ids=["learned_code", "copy", "window", "no_next"], + ids=["learned_code", "gain", "window", "no_next"], ) def test_it_composes_with_the_memorys_other_forms(tmp_path, options): brain = routed(**options) diff --git a/tests/test_recall_gain.py b/tests/test_recall_gain.py index dc16e6a2..e0e55344 100644 --- a/tests/test_recall_gain.py +++ b/tests/test_recall_gain.py @@ -57,7 +57,7 @@ def test_it_is_an_option_that_needs_tokens_and_a_newborn_reads_as_without_it(): np.testing.assert_array_equal(brain.recall.gains, np.zeros(2)) assert brain.describe()["genes"]["recall_gain"] == RATE assert brain.parameters() == founder.parameters() + 2 - assert not len(brain.recall.copy_synapses()[0]) # no synapse is added + assert brain.connectome.synapses == founder.connectome.synapses # no synapse is added np.testing.assert_array_equal(founder.brain.efficacy, brain.brain.efficacy) rng = np.random.default_rng(1) for t in range(6): diff --git a/tests/test_recall_gain_after.py b/tests/test_recall_gain_after.py new file mode 100644 index 00000000..dd7d1b76 --- /dev/null +++ b/tests/test_recall_gain_after.py @@ -0,0 +1,67 @@ +"""``recall_gain_after`` and ``recall_gain_next``: the value path released once the read has +formed, on the next ports.""" + +from __future__ import annotations + +import numpy as np +import pytest + +import cadence as cd + + +def _brain(**options): + options = {"recall_gain": 1.0, "recall_tokens": list(range(6)), **options} + return cd.Brain.compose(12, 6, backend="cpu", recall=True, seed=3, **options) + + +def _lessons(brain, count, seed=0): + rng = np.random.default_rng(seed) + for _ in range(count): + brain.reset() + x = np.eye(12)[rng.integers(6, size=2)] + brain.lesson(x, rng.integers(6, size=2)) + brain.act(x, greedy=True) + + +def test_the_gains_are_held_at_zero_and_released_on_the_next_ports(): + brain = _brain(recall_gain_after=3, recall_gain_next=10.0) + assert not brain.recall.gains.any() + _lessons(brain, 2) + assert brain.recall.gain_lessons == 2 and not brain.recall.gains.any() + _lessons(brain, 1, seed=1) + assert brain.recall.gain_lessons == 3 + np.testing.assert_array_equal(brain.recall.gains, [0.0, 10.0]) + + +def test_without_a_delay_the_next_ports_start_at_their_gain(): + np.testing.assert_array_equal(_brain(recall_gain_next=4.0).recall.gains, [0.0, 4.0]) + + +def test_at_their_founders_the_gains_live_as_without_them(): + plain, named = _brain(), _brain(recall_gain_after=0, recall_gain_next=0.0) + _lessons(plain, 4) + _lessons(named, 4) + np.testing.assert_array_equal(plain.recall.gains, named.recall.gains) + assert "recall_gain_after" not in named.describe()["genes"] + + +def test_the_count_is_saved_and_the_release_continues_after_a_load(tmp_path): + brain = _brain(recall_gain_after=3, recall_gain_next=10.0) + _lessons(brain, 2) + loaded = cd.Brain.load(brain.save(tmp_path / "delayed")) + assert loaded.recall.gain_lessons == 2 and loaded.recall.gain_after == 3 + for b in (brain, loaded): + _lessons(b, 1, seed=1) + np.testing.assert_array_equal(brain.recall.gains, loaded.recall.gains) + np.testing.assert_array_equal(loaded.recall.gains, [0.0, 10.0]) + + +@pytest.mark.parametrize("options, match", [ + ({"recall_gain": 0.0, "recall_gain_after": 3}, "need gain"), + ({"recall_gain_after": -1}, "nonnegative integer"), + ({"recall_gain_after": 2.5}, "nonnegative integer"), + ({"recall_gain_next": -1.0}, "gain_next"), +]) +def test_bad_settings_are_refused(options, match): + with pytest.raises(ValueError, match=match): + _brain(**options) diff --git a/tests/test_recall_memory.py b/tests/test_recall_memory.py index 8711365a..aadb18bc 100644 --- a/tests/test_recall_memory.py +++ b/tests/test_recall_memory.py @@ -1110,9 +1110,9 @@ def test_the_options_at_their_founders_build_the_founder_memory(tmp_path): "address_reward": 0.0, "position_reward": 0.0, "window": 1, "heads": 1, "position": 32, "position_rate": 0.1, - "boundary": False, "copy": 0.0, "tokens": None, "action": False, + "boundary": False, "tokens": None, "action": False, "anchor": False, "relative": False, "slope": 0.0, "when": False, "drive_gain": False, - "gain": 0.0, "grid": (), + "gain": 0.0, "grid": (), "cortex_norm": False, "gain_after": 0, "gain_next": 0.0, } founder = composed(seed=5, arousal=True, learning_surprise=True) explicit = composed( diff --git a/tests/test_recall_tokens.py b/tests/test_recall_tokens.py new file mode 100644 index 00000000..e351685e --- /dev/null +++ b/tests/test_recall_tokens.py @@ -0,0 +1,108 @@ +"""Declared tokens (``recall_tokens``): which answers are tokens a slot can hold, so a lesson whose +answer is no token leaves the read as it was. The copy pathway (``recall_copy``) left in 0.96.0: +its name is refused at construction, under retune and in a saved memory.""" +from __future__ import annotations + +import json + +import numpy as np +import pytest +from checkpoints import flat, metadata, write + +import cadence as cd + +TOKENS = 6 +INPUTS = TOKENS + 3 # the tokens, a HEAR, an ASK and a NOISE flag +HEAR, ASK, NOISE = TOKENS, TOKENS + 1, TOKENS + 2 + + +def composed(**options): + options.setdefault("modules", (16,)) + return cd.Brain.compose(INPUTS, TOKENS, seed=3, episodic=False, recall=True, + recall_founder=True, **options) + + +def event(*on): + x = np.zeros((1, INPUTS)) + x[0, list(on)] = 1.0 + return x + + +def test_the_copy_pathway_is_refused_by_name_with_what_replaces_it(tmp_path): + for value in (0.0, 2.0): + with pytest.raises(ValueError, match="recall_copy was retired in 0.96.0.*recall_gain"): + composed(recall_copy=value, recall_tokens=range(TOKENS)) + with pytest.raises(ValueError, match="recall_copy was retired in 0.96.0"): + composed().retune(recall_copy=value) + with np.load(composed().save(tmp_path / "founder.npz"), allow_pickle=False) as data: + arrays = flat({name: data[name].copy() for name in data.files}) + meta = metadata(arrays) + meta["recall"]["copy"] = 2.0 # as a 0.95 memory with the pathway saved it + arrays["generic"] = np.array(json.dumps(meta)) + write(tmp_path / "copy.npz", arrays) + with pytest.raises(ValueError, match="recall_copy was retired in 0.96.0.*0.95.2"): + cd.Brain.load(tmp_path / "copy.npz") + meta["recall"]["copy"] = 0.0 # the value the memory keeps: a memory without the pathway + arrays["generic"] = np.array(json.dumps(meta)) + write(tmp_path / "plain.npz", arrays) + assert cd.Brain.load(tmp_path / "plain.npz").recall.tokens is None + + +def test_invalid_tokens_are_refused(): + with pytest.raises(ValueError, match="one observation coordinate per"): + composed(recall_tokens=range(TOKENS - 1)) + with pytest.raises(ValueError, match="integer"): + composed(recall_tokens=[0.5] * TOKENS) + with pytest.raises(ValueError, match="sequence"): + composed(recall_tokens="abc") + + +def test_tokens_declare_which_answers_are_tokens_and_add_no_synapse(): + brain = composed(recall_tokens=[0, 1, 2, 3, -1, -1], learning_surprise=True) + assert brain.describe()["genes"]["recall_tokens"] == [0, 1, 2, 3, -1, -1] + plain = composed(learning_surprise=True) + np.testing.assert_array_equal(brain.brain.efficacy, plain.brain.efficacy) + np.testing.assert_array_equal(brain.connectome.pre, plain.connectome.pre) + brain.act(event(HEAR, 1), greedy=True) + before = [np.array(m, copy=True) for m in brain.recall.head_maps[0]] + assert brain.lesson(event(HEAR, ASK), [5])["address_rows"] == 0.0 + for a, b in zip(before, brain.recall.head_maps[0], strict=True): + np.testing.assert_array_equal(a, b) + assert brain.last_learning["demonstration_accepted_presentations"] == 1.0 + + +def test_a_lesson_whose_answer_is_no_token_leaves_the_read_as_it_was(): + # actions 0 to 3 are tokens, 4 and 5 are not (a yes and a no): the read's lessons take + # their target from the slots' contents, which hold no yes or no + brain = composed(recall_tokens=[0, 1, 2, 3, -1, -1]) + brain.act(event(HEAR, 1), greedy=True) + brain.act(event(NOISE, 3), greedy=True) + before = [np.array(m, copy=True) for m in brain.recall.head_maps[0]] + assert brain.lesson(event(HEAR, ASK), [5])["address_rows"] == 0.0 + for a, b in zip(before, brain.recall.head_maps[0], strict=True): + np.testing.assert_array_equal(a, b) + assert brain.lesson(event(HEAR, ASK), [3])["address_rows"] == 1.0 # a token: the read learns + assert any(not np.array_equal(a, b) + for a, b in zip(before, brain.recall.head_maps[0], strict=True)) + none = composed(recall_tokens=[-1] * TOKENS) # no answer is a token: no lesson moves the read + none.act(event(HEAR, 1), greedy=True) + assert none.lesson(event(HEAR, ASK), [2])["address_rows"] == 0.0 + plain = composed() # without tokens every taught row teaches the read + plain.act(event(HEAR, 1), greedy=True) + assert plain.lesson(event(HEAR, ASK), [5])["address_rows"] == 1.0 + + +def test_under_surprise_a_lesson_whose_answer_is_no_token_teaches_every_row(): + x = np.concatenate([event(HEAR, 0), event(HEAR, 1)]) + taught = {} + for name, tokens in (("tokens", list(range(TOKENS))), ("none", [-1] * TOKENS)): + brain = composed(recall_tokens=tokens, learning_surprise=True) + brain.act(x, greedy=True) + answer = np.asarray(brain._greedy( # noqa: SLF001 the act the lesson judges by + brain._qualified(brain.stimulus(x), brain._activity(), operation="act"), # noqa: SLF001 + brain._placed(x), brain._decision_key(brain._coded(x)), # noqa: SLF001 + )) + brain.lesson(x, answer) # the label is the brain's own answer on both rows + taught[name] = brain.last_learning["demonstration_accepted_presentations"] + assert taught["tokens"] == 0.0 # answered as shown: no surprise, no lesson + assert taught["none"] == 2.0 # no token answers: taught at every lesson, right or not diff --git a/tests/test_recall_when.py b/tests/test_recall_when.py index 2ee6e65b..f0f5d876 100644 --- a/tests/test_recall_when.py +++ b/tests/test_recall_when.py @@ -64,8 +64,8 @@ def test_the_read_drives_the_when_region_with_its_attention(): @pytest.mark.parametrize( "options", - [{}, {"recall_copy": 0.5}, {"recall_copy": 0.5, "recall_heads": 2}], - ids=["recall", "copy", "copy_heads"], + [{}, {"recall_heads": 2}], + ids=["recall", "heads"], ) def test_a_newborn_when_channel_settles_as_the_memory_without_it(options): rng = np.random.default_rng(2)