Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

feat(observability): add structured logging to JobQueryService and enhance orphan recovery logs #151

Copy link
Copy link

Description

@kvithayathil
Issue body actions

Context

The worker orphan recovery feature (#129) added retry_job() to JobQueryService and _recover_orphaned_jobs() to the worker loop. While worker.py already uses structlog extensively (20+ log calls), job_query_service.py has zero logging — no import, no logger, no log calls.

Gap

JobQueryService (no logging at all)

  • retry_job() — no log when a job is retried (who triggered it, previous state, job ID)
  • cancel_job() — no log when a job is cancelled
  • start_job() — no log when a job is started
  • create_job() — no log when a job is created

Worker orphan recovery (partial logging)

  • _recover_orphaned_jobs_with_session() logs only the count ("Periodic orphan recovery complete", count=recovered) — does not log individual job recoveries (previous status, job ID, stale duration)
  • _terminate_orphans_with_session() has no logging for individual terminations

Proposed scope

  1. Add structlog to job_query_service.py — log all state-changing operations (create, start, cancel, retry) with job ID, previous status, new status
  2. Enhance _recover_orphaned_jobs_with_session() in worker.py — log each recovered job with job ID, previous status, stale duration
  3. Enhance _terminate_orphans_with_session() — log each terminated orphan with job ID, previous status, termination reason

Why separate from #129

  • fix(worker): runtime orphan recovery for stuck OCR jobs #129 is already substantial (7 files, +722 lines, 3 commits)
  • Logging is a cross-cutting observability concern that applies to the entire service, not just retry
  • Deserves its own design discussion (log levels, fields, PII considerations)
  • Original plan was scoped to recovery mechanism only

Related

Reactions are currently unavailable

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:backendBackend (Python/FastAPI) changesBackend (Python/FastAPI) changesenhancementNew feature or requestNew feature or requestobservabilityLogging, monitoring, tracing, alertingLogging, monitoring, tracing, alertingpriority:mediumNormal priorityNormal priority

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    Morty Proxy This is a proxified and sanitized view of the page, visit original site.