Skip to content

feat(sessions): store a batch of events in one transaction in DatabaseSessionService - #8257

Open
EdwardGlockner wants to merge 1 commit into
google:mainfrom
EdwardGlockner:feat/db-append-events
Open

EdwardGlockner wants to merge 1 commit into
google:mainfrom
EdwardGlockner:feat/db-append-events

Conversation

@EdwardGlockner

Copy link
Copy Markdown

Link to Issue or Description of Change

Problem:
BufferedSessionService only batches writes when the wrapped store overrides append_events, and none of the built-in stores do. So on top of DatabaseSessionService, max_write_delay and max_batch_events don't do anything and each event is still its own transaction.

Solution:
I added DatabaseSessionService.append_events. It takes the same locks and does the same stale check as append_event, but once for the whole batch, then merges the state deltas in order, adds a row per event and commits once. Partial events are skipped like before. append_event now just calls it with one event, so there's only one write path to maintain. I also updated the sentence in the buffer guide that said no built-in store batches.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

The new tests in test_session_service.py check that a batch ends up the same as appending the events one by one (on the v0 and v1 schemas), that it's one transaction, and that nothing gets stored if the session is stale or a write fails halfway. There's also one in test_buffered_session_service.py that checks the buffer actually sends the queued events in one call now.

pytest tests/unittests/sessions tests/unittests/test_runners.py passes (1393 tests). test_workflow_resumed_twice_stores_each_output_once[buffered_database] is flaky for me on Windows, but it fails on main too.

Manual End-to-End (E2E) Tests:

Timed against a SQLite file on Windows, Python 3.11, median of 3 runs:

main this PR
10 events 1400 ms 119 ms
50 events 8211 ms 152 ms
BufferedSessionService, flushing 20 events 2608 ms 125 ms

The stored events and state come out the same either way.

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.

…eSessionService

BufferedSessionService batches writes only when the wrapped store
overrides append_events, and no built-in store did, so with
DatabaseSessionService every buffered event was still its own
transaction.

DatabaseSessionService.append_events now does one stale check, merges
the state deltas in order, adds one row per event and commits once.
append_event calls it with a single event.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BufferedSessionService can't batch writes: DatabaseSessionService has no append_events

2 participants