-
Notifications
You must be signed in to change notification settings - Fork 615
feat: Propagate exceptions to AfterToolCallEvent for decorated tools (#1565) #1566
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
feat: Propagate exceptions to AfterToolCallEvent for decorated tools (#1565) #1566
Conversation
zastrowm
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good implementation! Left some comments/requested-changes of the tests
| assert after_event.exception is not None | ||
| assert isinstance(after_event.exception, ValueError) | ||
| assert "decorated tool error" in str(after_event.exception) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can this be simplified to
| assert after_event.exception is not None | |
| assert isinstance(after_event.exception, ValueError) | |
| assert "decorated tool error" in str(after_event.exception) | |
| assert after_event.exception == ValueError("decorated tool error") |
Or something similar? We should be able to verify the exact exception rather than "decorated tool error" in str(after_event.exception)", given that we control it.
Alternatively, do:
exception = ValueError("decorated tool error")
early on, and then check that it's:
assert after_event.exception == exception
| assert after_event.exception is not None | ||
| assert isinstance(after_event.exception, RuntimeError) | ||
| assert "runtime error from decorated tool" in str(after_event.exception) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Along the lines of my earlier comment, let's simplify this assertion
| after_event = hook_events[-1] | ||
| assert isinstance(after_event, AfterToolCallEvent) | ||
| assert after_event.exception is None | ||
| assert after_event.result["status"] == "success" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: newline after this I think
|
|
||
|
|
||
| @pytest.mark.asyncio | ||
| async def test_executor_stream_decorated_tool_exception_in_hook( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we add a test case that ensures that exception is None, but the result is failed if the decorator returns an error state without throwing? I'd like to ensure that behavior is solidified in a test
|
Looks like there are also some linting failures; |
Yeah thats my bad, I optimistically tried to resolve conflicts in the UI. Should be good now |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Description
Motivation
Hook providers need the ability to distinguish between expected errors (like validation failures that the model should adjust for and retry) and unexpected errors (like assertion failures or configuration errors that should fail the task immediately). Currently,
AfterToolCallEvent.exceptionis alwaysNonefor@tool-decorated functions because the decorator catches all exceptions and converts them to error results before the executor can see them. This forces all tool errors to be returned to the model, even when propagating the exception would be more appropriate.Resolves: #1565
Public API Changes
ToolResultEventnow accepts an optionalexceptionparameter, andAfterToolCallEvent.exceptionis populated for decorated tools:The error result is still returned to the model by default; hooks must explicitly re-raise if they want to propagate.
Use Cases
AssertionError,ConfigurationError, etc. instead of letting the model retry futilelyValueErrorto return to the model while propagating other exceptionsRelated Issues
#1565
Documentation PR
strands-agents/docs#482
Type of Change
New feature
Testing
How have you tested the change? Verify that the changes do not break functionality or introduce warnings in consuming repositories: agents-docs, agents-tools, agents-cli
hatch run prepareChecklist
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.