Skip to content

Navigation Menu

Sign in
Appearance settings
Sign up
Appearance settings

Commit b9e53c6

Browse filesBrowse the repository at this point in the historyBrowse files
authored
feat(bigtable): add Session lifecycle (Start, Close, ForceClose, readLoop, heartBeatLoop) (#20215)
## Summary Third and final PR in the Session core stack. Adds the lifecycle orchestration that drives a Session from \`Start\` through teardown, plus the \`readLoop\` that dispatches server frames to the vRPC handlers landed in #20213. **Stacks on** #20213 (Session vRPC dispatch + slot lifecycle). Deletes the minimal \`ForceClose\` stub that PR shipped, replacing it with the full lifecycle-shaped body. ### What lands **New file: \`session_lifecycle.go\`** (~640 LOC) - \`Session.Start(ctx, OpenSessionRequest)\` — transitions New→Starting, Sends the OpenSession frame, fires \`onStart\`, spawns \`readLoop\` + \`heartBeatLoop\`. Wraps a failed \`Send\` as \`codes.Unavailable\` so retry plumbing treats pre-wire OpenSession loss the same as any other transport-side loss. - \`ForceClose\` — full body: \`transitionTo(Closed)\` → \`setCloseReason\` → \`notifyClosing\` (once) → \`cancelActiveRPCs\` → \`signalQuiescent\` → \`notifyClosed\`. - \`Close(ctx, CloseSessionRequest)\` — graceful drain: Ready→Closing, waits on \`quiescent\`, sends CloseSession, transitions to WaitServerClose, arms the pool's stuck-session sweep to eventually ForceClose if the server never confirms. - \`notifyClosing\` / \`notifyClosed\` — once-guarded hook dispatchers with the strict \`onClosing\`-precedes-\`onClose\` ordering enforced. - \`readLoop(ctx)\` — Recv-loop that dispatches every SessionResponse variant via \`handleSessionResponse\`, drives \`handleClose\` on stream termination, records \`msgsRecv\` counters + resets heartbeat deadline on every recognized frame. - \`handleSessionResponse\` — dispatch switch to \`handleOpenSession\` / \`handleVRPCResponse\` / \`handleErrorResponse\` / \`handleSessionParameters\` / \`handleGoAway\` / \`handleSessionRefreshConfig\`. Heartbeat frames reset the deadline; unknown-payload branch records a debug tag but doesn't reset (so a misbehaving server can't keep the watchdog satisfied with junk). - \`handleOpenSession\` — Starting→Ready transition, PeerInfo extract from the bidi header (synchronous, matches Java's onHeaders synchrony), fires \`onActive\`. - \`handleGoAway\` / \`handleClose\` / \`handleErrorResponse\` / \`handleSessionParameters\` / \`handleSessionRefreshConfig\` — protocol-level handlers. - \`heartBeatLoop\` — Timer + \`heartbeatWake\` reactive-wake pair. Only enforces the deadline while a vRPC is in flight (idle sessions legitimately receive no server heartbeats). See SESSION_SPEC.md #7. - \`peerInfoExtracter\` — parses the \`bigtable-peer-info\` header, stamps \`s.peerInfo\` atomically before \`onActive\` fires. - \`closeReasonLabel\` / \`closeReasonToCause\` — CloseSessionRequest.Reason → string / sentinel error mapping for close-reason attribution. **New file: \`session_lifecycle_test.go\`** (~660 LOC) — 30+ tests covering Start, ForceClose, Close, readLoop, handleSessionResponse dispatch, handleOpenSession + peerInfo parsing, handleGoAway, handleClose, handleErrorResponse (rpc_id=0 harmlessly drops via routeVRPCFrame guards), heartBeatLoop (reactive wake, idle-gate, missed-heartbeat ForceClose). **Edits to \`session_vrpc.go\`** - Delete the minimal \`ForceClose\` stub introduced in the prior PR — the full body lives in \`session_lifecycle.go\` now. **Edits to \`session_test.go\`** - \`hookCounts\` helper (start/active/close callback counters). - \`setSlotForTest\` helper (seed the in-flight slot from tests). ### Stack 1. #20211 — Session debug surface 2. #20213 — Session vRPC dispatch + slot lifecycle 3. **This PR** — Session lifecycle ## Test plan - [x] \`go build ./internal/transport/\` passes - [x] \`go vet ./internal/transport/\` clean - [x] \`go test ./internal/transport/ -count=1 -short -timeout 90s\` passes — 30+ new lifecycle tests plus all pre-existing tests
1 parent ef97df3 commit b9e53c6
Copy full SHA for b9e53c6

9 files changed

+3,102-21Lines changed: 3102 additions & 21 deletions

File tree

Expand file treeCollapse file tree
Open diff view settings
Filter options
Expand file treeCollapse file tree
Open diff view settings
Collapse file

‎bigtable/internal/transport/debug_tracer.go‎

Copy file name to clipboardExpand all lines: bigtable/internal/transport/debug_tracer.go
+11Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,13 +93,24 @@ const (
9393
tagSessionHeartbeatMissed = "session_heartbeat_missed"
9494
tagSessionForceCloseNeverStarted = "session_force_close_never_started"
9595
tagSessionCloseNoReason = "session_close_no_reason"
96+
// tagSessionReadLoopPanic fires when readLoop's deferred recover
97+
// catches a panic from handleSessionResponse (or any downstream
98+
// handler). Session is force-closed with REASON_ERROR carrying the
99+
// panic value in the description.
100+
tagSessionReadLoopPanic = "session_read_loop_panic"
96101

97102
// vRPC dispatch observations.
98103
tagSessionVRPCNil = "session_vrpc_nil"
99104
tagSessionVRPCErrorNil = "session_vrpc_error_nil"
100105
tagSessionVRPCIDMismatch = "session_vrpc_id_mismatch"
101106
tagSessionVRPCResponseWrongState = "session_vrpc_response_wrong_state"
102107
tagSessionVRPCDuplicateResult = "session_vrpc_duplicate_result"
108+
// tagSessionVRPCCancelledDrained fires when a server response finally
109+
// arrives for an rpc whose caller already returned via ctx.Done: the
110+
// drain succeeds, currentCancel != nil, and no one is waiting on
111+
// resultChan. Bookkeeping-only — the drain still fires OnSlotDrained
112+
// so the pool re-enqueues the session.
113+
tagSessionVRPCCancelledDrained = "session_vrpc_cancelled_drained"
103114

104115
// Pool-scoped anomalies.
105116
tagSessionPoolStuckSessionSwept = "session_pool_stuck_session_swept"
Collapse file

‎bigtable/internal/transport/session.go‎

Copy file name to clipboardExpand all lines: bigtable/internal/transport/session.go
+65-9Lines changed: 65 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -61,11 +61,19 @@ type Stream interface {
6161

6262
// SessionHooks holds optional lifecycle callbacks. Nil fields are skipped.
6363
// Hooks must not block.
64+
//
65+
// OnSlotDrained fires on every successful drainSlot on the wire (normal
66+
// deliver, cancelled-drain, Send-failure Invoke branch). It is the sole
67+
// "session became free" signal — consumers use it to re-enqueue the
68+
// session in its AFE idle queue and wake one parked Checkout waiter.
69+
// cancelActiveRPCs (session teardown) intentionally does NOT fire it —
70+
// OnClosing/OnClose handle removal from routing structures on that path.
6471
type SessionHooks struct {
65-
OnStart func(ctx context.Context)
66-
OnActive func(s *Session)
67-
OnClosing func(s *Session)
68-
OnClose func(s *Session, err error)
72+
OnStart func(ctx context.Context)
73+
OnActive func(s *Session)
74+
OnSlotDrained func()
75+
OnClosing func(s *Session)
76+
OnClose func(s *Session, err error)
6977
}
7078

7179
func (h SessionHooks) onStart(ctx context.Context) {
@@ -80,6 +88,12 @@ func (h SessionHooks) onActive(s *Session) {
8088
}
8189
}
8290

91+
func (h SessionHooks) onSlotDrained() {
92+
if h.OnSlotDrained != nil {
93+
h.OnSlotDrained()
94+
}
95+
}
96+
8397
func (h SessionHooks) onClosing(s *Session) {
8498
if h.OnClosing != nil {
8599
h.OnClosing(s)
@@ -159,6 +173,12 @@ type Session struct {
159173
heartbeatIntervalNano atomic.Int64
160174
nextHeartbeatDeadlineNano atomic.Int64
161175

176+
// heartbeatWake nudges the heartbeat loop to re-evaluate its Timer
177+
// after the atomic deadline moves. Cap-1 non-blocking channel so a
178+
// burst of resets coalesces into a single wake and hot-path frame
179+
// handling stays allocation-free.
180+
heartbeatWake chan struct{}
181+
162182
// quiescent closes when the in-flight vRPC drains after StateClosing,
163183
// or when ForceClose runs.
164184
quiescent chan struct{}
@@ -184,11 +204,12 @@ type SessionOption func(*Session)
184204
// valid.
185205
func NewSession(logName string, stream Stream, hooks SessionHooks, sessionType SessionType, opts ...SessionOption) *Session {
186206
s := &Session{
187-
logName: logName,
188-
stream: stream,
189-
hooks: hooks,
190-
quiescent: make(chan struct{}),
191-
sessionType: sessionType,
207+
logName: logName,
208+
stream: stream,
209+
hooks: hooks,
210+
quiescent: make(chan struct{}),
211+
sessionType: sessionType,
212+
heartbeatWake: make(chan struct{}, 1),
192213
}
193214
s.state.Store(int32(StateNew))
194215
s.heartbeatIntervalNano.Store(int64(defaultHeartbeatInterval))
@@ -245,3 +266,38 @@ func unavailable(cause error, format string, args ...interface{}) error {
245266
cause: cause,
246267
}
247268
}
269+
270+
// Send writes a SessionRequest under sendMu so concurrent producers don't
271+
// corrupt the underlying stream. grpc.ClientStream.Send is not safe for
272+
// concurrent use; sendMu is the only serialization point.
273+
func (s *Session) Send(req *spb.SessionRequest) error {
274+
s.sendMu.Lock()
275+
err := s.stream.Send(req)
276+
s.sendMu.Unlock()
277+
if err == nil {
278+
s.msgsSent.Add(1)
279+
s.msgsSentByType[classifyReq(req)].Add(1)
280+
}
281+
return err
282+
}
283+
284+
// resetHeartbeatDeadline pushes out the watchdog to (now + heartbeatInterval).
285+
// One atomic load + one atomic store on the hot path, plus a non-blocking
286+
// wake to the heartbeat loop so its Timer picks up the new deadline
287+
// immediately (otherwise the initial bootstrap arm keeps the loop sleeping
288+
// past atomic shortenings — SESSION_SPEC.md #7).
289+
func (s *Session) resetHeartbeatDeadline() {
290+
s.nextHeartbeatDeadlineNano.Store(time.Now().Add(time.Duration(s.heartbeatIntervalNano.Load())).UnixNano())
291+
s.wakeHeartbeatLoop()
292+
}
293+
294+
// wakeHeartbeatLoop signals the heartbeat loop to re-evaluate its Timer.
295+
// Non-blocking send on a cap-1 channel: bursts coalesce into a single
296+
// pending wake, so hot-path frame handlers stay allocation-free and
297+
// contention-free even under high frame arrival rates.
298+
func (s *Session) wakeHeartbeatLoop() {
299+
select {
300+
case s.heartbeatWake <- struct{}{}:
301+
default:
302+
}
303+
}
Collapse file

‎bigtable/internal/transport/session_debug.go‎

Copy file name to clipboardExpand all lines: bigtable/internal/transport/session_debug.go
+18-1Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -150,6 +150,23 @@ const (
150150
// prior attempt's gRPC code + err (if the retry interceptor stashed
151151
// one on ctx).
152152
SessionEventRetry SessionEventKind = "retry"
153+
// SessionEventProtocolError fires when routeVRPCFrame observes a
154+
// state/frame combination that violates the client-server contract:
155+
// a frame arrived in a state readLoop shouldn't be running in
156+
// (New/Starting/Closed) OR the server's rpc_id didn't match our
157+
// active vRPC. Escalated to session teardown via ForceClose so the
158+
// pool's OnClosing/OnClose hooks fire and the session leaves the
159+
// AFE routing set. Kept separate from SessionEventLateFrame so
160+
// operators filtering sessionz for genuine desyncs aren't swamped
161+
// by benign late-after-cancel drops.
162+
SessionEventProtocolError SessionEventKind = "protocol-error"
163+
// SessionEventLateFrame fires when routeVRPCFrame drops a frame
164+
// because there's no active vRPC on this session (activeVRPC()==nil).
165+
// This is a documented race — the caller ctx.Done'd and cancelled
166+
// the slot, but the server's response arrived before the cancel
167+
// landed on the wire. Not a protocol violation; the session stays
168+
// healthy and the frame is dropped.
169+
SessionEventLateFrame SessionEventKind = "late-frame"
153170
)
154171

155172
// SessionEvent is one entry in a session's per-session debug ring buffer.
@@ -164,7 +181,7 @@ type SessionEvent struct {
164181
const maxSessionEvents = 64
165182

166183
// recordEvent appends a SessionEvent to the per-session ring buffer.
167-
// Safe to call from any goroutine (readLoop, heartBeatLoop, etc.). Uses
184+
// Safe to call from any goroutine (readLoop, heartbeatLoop, etc.). Uses
168185
// the same wrap-index scheme as latencySamples so an at-cap append is
169186
// O(1), not an O(N) shift.
170187
func (s *Session) recordEvent(kind SessionEventKind, format string, args ...interface{}) {

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.