Repository navigation
Add guaranteed-reproducible PRNGs to rand? #1588
Description
Activity
If you want guaranteed reproducibility can't you just use the named PRNG? Maybe I am misunderstanding the question.
Yes — except that none of those named PRNGs are currently publicly export from
rand.Motivation is partially convenience and partially to make it more obvious how users may set up a reproducible PRNG (currently another crate must be added as a dependency).
Ah true, I remember having to do this.
I would say exporting
Xoshiro256PlusPluswould be a good idea, also under this name.As argued in the linked issue, I don't think we need it and we should recommend use of a concrete PRNG crate (we could reference them in
StdRng/SmallRngdocs).When I'm sufficiently worried about long-term reproducibility that I'd opt for a generator with such a guarantee, I generally wouldn't be satisfied if the guarantee only covered
RngCoremethods or something like that. What I care about is that my program overall remains reproducible, which means e.g. any samplingRngmethods my program uses (and the trait impls backing them) can't have value-breaking changes either. Even ifrandwas willing to guarantee that for a larger subset of its APIs, it's very difficult for me as a user to ensure that I'm only using the guaranteed-stable subset. Depending directly on a specificrand_fooPRNG crate only solves this if you can make do with onlyrand_core::RngCoreand avoid depending onrandentirely, but that's rare in my experience.So I think rand, as a general-purpose crate that has good reasons to make value-breaking changes from time to time, is not in a good position to try and address the need for reproducibility. Offering it only for the simple cases, but not for the other APIs that come along for the ride, will result in just as many people being mistaken about whether their
rand-using program will be reproducible with future releases ofrand. That's not helping anyone.So I think rand, as a general-purpose crate that has good reasons to make value-breaking changes from time to time, is not in a good position to try and address the need for reproducibility.
The same is true of any library offering a wide variety of random algorithms? The solution here is simple enough: use a fixed version of
rand. We should not make value-breaking changes in patch releases (outside of security concerns, though this was never yet an issue).Using a fixed version is not great because it means I'll effectively be on my own with maintaining that code once upstream (quite reasonably) stops doing so. Whether value-breaking changes are made in patch releases or only in minor releases is immaterial -- eventually I'll have to choose between eating a value-breaking change or sticking with an unmaintained version of the library. This won't be an issue if my code stops being actively developed before upstream moves on, but in many cases I don't want to make assumptions about that. And if I end up having to vendor the library, I'd always prefer one that is as small and simple as possible for my specific use case over a library that does basically everything.
The only way around this is if a library is aligned with my priorities w.r.t. reproducibility: making a credible promise to avoid value-breaking changes, by only adding new APIs without changing the old ones (possibly deprecating them but ideally without the implication that they'll be removed eventually). Of course, that's undesirable for everyone who doesn't need long-term reproducibility and wants to get improvements automatically. But it's not inherently impossible for a maintainer to do that, if that's their priority.
eventually I'll have to choose between eating a value-breaking change or sticking with an unmaintained version of the library. [...] And if I end up having to vendor the library, I'd always prefer one that is as small and simple as possible for my specific use case over a library that does basically everything.
If you're talking about
rand(notrand_distr), then unless you care about nightly features, there isn't much to maintain — about the only thing in rand v0.8 which "broke" is thatgenwill soon be a reserved keyword. As for bug fixes, v0.9 includes a couple of portability fixes and one single bug fix toIteratorRandom::choose_multiple_weightedfor extremely small seeds (a value-breaking change, thus this could not be back-ported).(I'm assuming you're not talking about maintenance of security — but even here nothing of note happened in the last four years, and if it did I expect that we would release a patch.)
So I don't buy your argument that
randis not a good choice if you care about long-term reproducibility.I don't know how easy or hard it would be for me to take over bugfix-only maintenance of a specific
randversion. Since I'm not familiar with the code base or its history, determining that for myself would take non-trivial effort. I appreciate you sharing information about this now, but imagine if we weren't having this conversation and I'd just be looking at docs.rs/rand to make my decision. That part is just less daunting with a library that's less than, say, 1K lines of code.In any case, if I'm happy to use a fixed version of a library then it doesn't matter if the library offers reproducibility guarantees across its releases (of course, consistent results across platforms still matter). If I'll be using rand 0.8.5 forever, then I'm not affected by value-breaking changes in later releases. Conversely, if I want to avoid pinning a specific version and instead keep updating rand, then I need reproducibility guarantees for all APIs that I'm using or might use by accident in the future, not just for the
RngCoreimpls. That's what my first comment was about: to enable meaningful long-term reproducibility without version pinning, rand would have to make a much stronger commitment than just keeping some specific PRNG impls intact. I don't think rand can reasonably do that without unduly compromising on competing priorities.Reacted by Diggory Hardy and mstoecklrust-random/book#82 is possibly of interest to people here.
It technically reduces the guarantees, allowing
SmallRngandStdRngalgorithms to be changed in any patch release post-1.0. In theory, this reduces the number of reasons we might have to make a new minor release, thereby hopefully improving the longevity of 1.0 (and any potential successor).It also includes a note about supporting old releases with new patches, including the possibility of back-porting new (compatible) features.
This is about the best compromise I can see between stability guarantees and avoiding stagnation.
Much of this discussion is off-topic.
randdoes put very significant efforts towards stability and is fairly conservative about accepting value-breaking changes. For example, I don't think there will be any value-breaking changes betweenrandv0.9 and v0.10 (#1643).To get back on topic, several PRNGs are available within
randinternally.Motivation of providing stable PRNGs in
rand: reproducibility is easier; especially this will be easier for new users who don't read the docs since there will now be named algorithms inrand::rngs.Xoshiro128PlusPlusandXoshiro256PlusPlusare defined internally with zero internal dependencies. Currently, only the one actually used bySmallRng(dependent on platform pointer width) is actually compiled, but compiling both is not a big cost. There is a significant chance that we change theSmallRngalgorithm(s) in the future, but even if we do there is not a large cost to keeping the code for these PRNGs. Even if we did change the algorithm a few times and publicly export all algorithms which had been used forSmallRngfor some minimum period of time (say, one year), we wouldn't end up with a very large list of stable PRNGs inrandwithin the next few decades. I therefore think it's fine to make these algorithms publicly accessible with a stability guarantee inrand.ChaCha12Rnghas been used for a while (and before thatChaCha20Rng) for our CSPRNG (StdRng). The significant difference to the above argument is that we use an external dependency (currentlyrand_chachawith active discussions to switch tochacha20) for this. If we did switch the implementation away fromChaChain the future we would not want to be forced to keep a dependency onchacha20or equivalent just forrand::rngs::ChaCha*Rng. Therefore I think we should not provide stable ChaCha CSPRNGs inrand.FWIW, the conclusions make sense to me. Seems like directly exporting the concrete
RngCoreimpls backingSmallRnghas little downside and some upside. For reasons discussed above I don't think it'll help with reproducibility across rand versions for most programs because it doesn't address value breaking changes in distributions, but the improved portability is nice (I wasn't aware thatSmallRngis platform-dependent). A ChaCha-based RNG is a much bigger commitment and there's third party crates for that which can implement the traits for multiplerand_coreversions if someone wants that.Reacted by Diggory Hardy- added a commit that references this issue
on Jul 30, 2025 If we did switch the implementation away from ChaCha in the future we would not want to be forced to keep a dependency on chacha20 or equivalent just for rand::rngs::ChaCha*Rng. Therefore I think we should not provide stable ChaCha CSPRNGs in rand.
I'd like to revisit this point with the following arguments:
- We have kept with ChaCha for several years now. It is a widely used well tested block cipher. It seems increasingly unlikely we'd need to switch away.
- We are however in the process of replacing our implementation of ChaCha, in a way which respects reproducibility of results. It may therefore be that users of
rand_chacha::ChaChaXRngwill want to update to point at the new implementation too. - Tests of reproducibility in the
randcrate apply to whichever implementation of ChaCha we may be using. Related: Added test vectors for StdRng, verified using rand_chacha. #1654.
It therefore seems that it might actually be a good idea to export
rand::rngs::ChaCha8Rng(and 12, 20 round variants).Does anyone disagree with this argument?
Should the pub exports be feature-gated (
chachafeature)? (We already havestd_rngfeature forStdRng.)It therefore seems that it might actually be a good idea to export rand::rngs::ChaCha8Rng (and 12, 20 round variants).
Re-exporting those behind
chachacrate feature sounds good to me.
This question came up recently regarding a possible adoption to
libstd(read from here), but I'm not sure we ever really asked the question ofrand.StdRngandSmallRngare deterministic but not reproducible (and in the latter case also not portable). Should we add a PRNG with guaranteed reproducibility as a new item underrand::rngs?We already have five PRNGs available in
randif you count the ChaCha variants:ChaCha8Rng,ChaCha12Rng,ChaCha20RngXoshiro128PlusPlus,Xoshiro256PlusPlusI'm not sure if we should ever add a guaranteed-reproducible ChaCha PRNG in
randsince if we ever wanted to change the generator behindThreadRngit would add dependencies. Given how long we've been using ChaCha in this role this may be less of an issue now.The Xoshiro variants are more acceptable (if only because they require a lot less code; both are directly implemented in
rand), though selecting one of these is likely sufficient, e.g.rang::rngs::Xoshiro256PlusPlus.CC @hanna-kruppe @joshtriplett in case of interest