[CodexVeritas]

Summary

MonetaryCostManager hard limits did not stop calls made through GeneralLlm or AgentSdkLlm. Neither class checked the limit itself. They relied on LitellmCostTracker.log_pre_api_call raising HardLimitExceededError, but that check never stopped a call:

• litellm's Logging.pre_call wraps each callback's log_pre_api_call in try/except and only logs the exception;
• async_log_pre_api_call is never called.

Once usage passed a limit, calls kept going, and the only signal was a warning once usage passed the limit.

Changes

• GeneralLlm checks the limit before every call. _mockable_direct_call_to_model calls MonetaryCostManager.raise_error_if_limit_would_be_reached() first, so exa/ and asknews/ models are covered too.
• AgentSdkLlm checks the limit in both get_response and stream_response. The streamed path is used by AgentRunner.run_streamed.
• RetryableModel does not retry HardLimitExceededError. Retrying could not succeed and waited 5–60s per attempt before raising the same error.
• LitellmCostTracker's pre-call hooks are removed, since they could not stop a call. The class docstrings now say how limits are enforced. They also note that litellm calls made outside these classes are tracked but not limited.

Semantics are unchanged otherwise. A call is refused once usage is already above the limit, and concurrent calls that start before then still run, as the class docstring describes.

Testing

• New test_hard_limit_enforcement.py (no network):
  • GeneralLlm calls the model while usage is at or under the limit.
  • Once usage is over the limit, GeneralLlm raises HardLimitExceededError without calling litellm or retrying. With the retry exclusion removed, the same test made 3 attempts over 20s.
  • AgentSdkLlm.get_response and stream_response raise without calling LitellmModel.
  • Before this change, all 3 over-limit cases failed with "DID NOT RAISE".
• End-to-end through litellm with mock_response (no network). With usage over the limit, GeneralLlm.invoke raised HardLimitExceededError immediately and usage did not change. Before this change the same call went through.
• code_tests/unit_tests: 969 passed. The only failures are 14 order-dependent tests that also fail on main and pass when run alone.
• Mocked hard-limit tests in code_tests/integration_tests/test_ai_models/test_models_incurring_cost.py (-k error_thrown): 15 passed.
• Compatibility with the open PRs. A trial merge with Count each GeneralLlm call's cost exactly once #358 and Count each CombinedLlmArchetype call's cost once #359 had no conflicts, and the combined cost and limit tests passed (48).

🤖 Generated with Claude Code