Repository navigation
mina-json-bytes: escape the token prefix instead of refusing the input - #53
Merged
Merged
Conversation
Where the input already holds __MINA_BYTE_, encode writes its first byte as the token __MINA_BYTE_5f__, so decode gives the text back and cannot take it for a token. A zkApp URI is free text, so one account could stop every ledger rewrite before.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
mina-json-bytes encodestops with "the input already contains the text__MINA_BYTE_". Without that rule,decodecould not tell such text from a token. But a zkApp URI is free text that any account owner sets. One account with that text in its URI stops every ledger rewrite that uses the tool, for example each dry-run ledger build in the mina repository (MinaProtocol/mina#19987, "Known limit").Change
Where the input holds the text
__MINA_BYTE_,encodewrites the first_of it as the token__MINA_BYTE_5f__, and continues after it:decodereads from left to right. It gives back the_, and the text after the token is no longer a complete token, so it stays text.decodeis unchanged: it already accepted a token for any byte. Sodecode(encode(x)) == xholds for every input now, and nothing is refused.transcode.rs: the encoder writes the text before each prefix (valid UTF-8 as it is, other bytes as tokens), then the escaped_. A character cut just before a prefix is invalid, because_follows it. A prefix split across two reads is still found, because the last bytes stay pending as before.error.rs:PrefixInInputis removed.token.rs,README.md: describe the escape. The README no longer lists the refusal as a limit.Tests
__MINA_BYTE__MINA_BYTE_,___MINA_BYTE_); a prefix split across reads at every read size; a cut character before a prefix.__MINA_BYTE_and__MINA_BYTE_ff__as parts, and requires every input to encode. It was also run once with 200,000 inputs instead of 3,000: every input came back exactly at every read size, and every output was valid UTF-8.tests/cli.rs: the two refusal tests now check that a URI holding__MINA_BYTE_ff__next to a raw0xffbyte comes back unchanged throughencodeanddecode, and throughjq -c ..cargo test: 20 unit tests and 12 CLI tests pass.cargo clippy --all-targets -- -D warningsis clean.After a release with this change, the mina toolchain image needs the new version.