Fix #1060: make the uplift multiplier bootstrap seedable - #1061
Open
Hari20032005 wants to merge 1 commit into
Open
Hari20032005 wants to merge 1 commit into
Hari20032005 wants to merge 1 commit into
Conversation
calc_uplift drew its multiplier bootstrap from the global numpy stream, so DRTester.evaluate_uplift() and evaluate_all() returned different uniform confidence bands on every call for identical inputs, with no way for a caller to pin them. Thread a random_state through calc_uplift, evaluate_uplift and evaluate_all, normalized with sklearn's check_random_state. evaluate_uplift builds one RandomState and shares it across the per-treatment calls, so each treatment still draws independent multipliers while the whole call is reproducible from a single seed. Defaults to None, so existing callers are unaffected. Add a regression test. Signed-off-by: Hari20032005 <hariharan944212005@gmail.com>
Hari20032005
force-pushed
the
fix/uplift-bootstrap-random-state
branch
from
September 4, 2026 07:18
c804626 to
1452617
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1060.
calc_upliftdrew the multiplier bootstrap behind the uniform confidence bands from the global numpy stream, soDRTester.evaluate_uplift()andevaluate_all()returned different bands on every call for identical inputs — and no caller could pin them.The change
econml/validate/utils.py:random_stateis threaded throughcalc_uplift,evaluate_upliftandevaluate_all, normalized withsklearn.utils.check_random_state— the convention already used in_ortho_learner.py:741andorf/_ortho_forest.py:234.evaluate_upliftbuilds oneRandomStateand passes that instance into each per-treatmentcalc_upliftcall, rather than passing the raw seed down. That keeps each treatment's multipliers independent — as they are today — while making the whole call reproducible from a single seed.evaluate_alldoes the same across its qini and toc calls.The parameter defaults to
None, so existing callers see no behavior change.Effect
Three
calc_upliftcalls on identical inputs, before:The point estimate and standard error were already deterministic — they never touch the bootstrap — which isolates the multiplier draws as the only source of the drift. With a seed passed, all three lines are identical.
Tests
test_uplift_random_stateineconml/tests/test_drtester.pyasserts that:evaluate_allis reproducible end to end.econml/tests/test_drtester.pyis 6 passed locally on Python 3.13 (numpy 2.5.2, scikit-learn 1.9.0), andruff checkis clean on the three changed files.🤖 Generated with Claude Code