Repository navigation
feat(sessions): store a batch of events in one transaction in DatabaseSessionService - #8257
Open
EdwardGlockner wants to merge 1 commit into
Open
EdwardGlockner wants to merge 1 commit into
EdwardGlockner wants to merge 1 commit into
Conversation
…eSessionService BufferedSessionService batches writes only when the wrapped store overrides append_events, and no built-in store did, so with DatabaseSessionService every buffered event was still its own transaction. DatabaseSessionService.append_events now does one stale check, merges the state deltas in order, adds one row per event and commits once. append_event calls it with a single event.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Link to Issue or Description of Change
Problem:
BufferedSessionServiceonly batches writes when the wrapped store overridesappend_events, and none of the built-in stores do. So on top ofDatabaseSessionService,max_write_delayandmax_batch_eventsdon't do anything and each event is still its own transaction.Solution:
I added
DatabaseSessionService.append_events. It takes the same locks and does the same stale check asappend_event, but once for the whole batch, then merges the state deltas in order, adds a row per event and commits once. Partial events are skipped like before.append_eventnow just calls it with one event, so there's only one write path to maintain. I also updated the sentence in the buffer guide that said no built-in store batches.Testing Plan
Unit Tests:
The new tests in
test_session_service.pycheck that a batch ends up the same as appending the events one by one (on the v0 and v1 schemas), that it's one transaction, and that nothing gets stored if the session is stale or a write fails halfway. There's also one intest_buffered_session_service.pythat checks the buffer actually sends the queued events in one call now.pytest tests/unittests/sessions tests/unittests/test_runners.pypasses (1393 tests).test_workflow_resumed_twice_stores_each_output_once[buffered_database]is flaky for me on Windows, but it fails on main too.Manual End-to-End (E2E) Tests:
Timed against a SQLite file on Windows, Python 3.11, median of 3 runs:
BufferedSessionService, flushing 20 eventsThe stored events and state come out the same either way.
Checklist