Repository navigation
fix(models): keep url_context retrieval status on model responses - #8236
Open
karthik-0306 wants to merge 1 commit into
Open
karthik-0306 wants to merge 1 commit into
karthik-0306 wants to merge 1 commit into
Conversation
Gemini reports which URLs the url_context tool retrieved, and whether each retrieval succeeded, in candidate.url_context_metadata. LlmResponse had no field for it, so LlmResponse.create and the streaming aggregator dropped it, and a failed or paywalled retrieval looked the same as a successful one to callbacks, plugins and stored events. Add the field, set it in both branches of LlmResponse.create, and carry it through the streaming aggregator the way citation_metadata is. Fixes google#8226
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Link to Issue or Description of Change
1. Link to an existing issue (if applicable):
Problem:
Gemini reports which URLs the
url_contexttool retrieved, and the status of each retrieval (SUCCESS,ERROR,PAYWALL,UNSAFE), incandidate.url_context_metadata.LlmResponsehad no field for it, so bothLlmResponse.createand the SSEStreamingResponseAggregatordropped it. To callbacks, plugins and stored events, a failed or paywalled retrieval looked the same as a successful one.Solution:
url_context_metadata: types.UrlContextMetadata | NonetoLlmResponse.LlmResponse.create.StreamingResponseAggregator(progressive and non-progressive), the same waycitation_metadatais.The flow finalizer already copies every
LlmResponsefield onto theEvent, so no flow change is needed. The v1 session schema stores events as JSON, so the field is persisted without a migration. The legacy v0 layout reports it through the existingwarn_event_fields_not_storedmechanism, like other fields added after that layout froze.Testing Plan
Unit Tests:
All four new cases fail on
mainand pass with this change:tests/unittests/models/test_llm_response.py:test_llm_response_create_keeps_url_context_metadata[success]and[error].tests/unittests/utils/test_streaming_utils.py:url_context_defaultandurl_context_progressivecases added to the existing parametrizedtest_close_preserves_metadata. The metadata arrives only on the first chunk, so the test checks that aggregation keeps it.pytest results:
sessions/test_session_util.py,sessions/test_v0_storage_event.py,events/,models/test_llm_response.py,utils/test_streaming_utils.py: 394 passed.sessions,a2a,cli,plugins): identical tomainin my environment.ruff,isort,pyink,codespellandmypyare clean on the changed files.Manual End-to-End (E2E) Tests:
Ran an
InMemoryRunnerwith aGeminimodel (onlyapi_clientmocked) and theurl_contexttool, where the model's response reportsURL_RETRIEVAL_STATUS_ERROR:url_context_metadatawithURL_RETRIEVAL_STATUS_ERRORin bothStreamingMode.NONEandStreamingMode.SSE.mainthe same script raisesAttributeError: 'Event' object has no attribute 'url_context_metadata'.Checklist
Additional context
This adds a public field to
LlmResponse/Event.EXCLUDED_EVENT_FIELDSin the agent test runner is unchanged: the field is deterministic grounding data likegrounding_metadata, and the event dumps there useexclude_none, so existing test fixtures are unaffected. I'm happy to open a follow-up docs PR if you'd like the field documented.