tensorrt_llm/_torch/pyexecutor/py_executor.py (1)

3052-3143: ⚠️ Potential issue | 🔴 Critical

Fix the broken logging call and add the missing environment variable gate.

The logger.info(...) call uses multiple positional arguments without %s format placeholders. This causes a TypeError: not all arguments converted during string formatting when the log record is emitted on rank 0. Additionally, the docstring documents that diagnostics require TLLM_LOG_BATCH_WAIT=1, but the code never checks this environment variable—the function will log unconditionally on rank 0.

Suggested fix

     def _maybe_log_batch_wait_decision(
         self,
         context_requests: list[LlmRequest],
         generation_requests: list[LlmRequest],
         num_scheduled_tokens: int,
         wait_threshold: float,
         should_waiting: bool,
     ) -> None:
         """Diagnostics for batch_wait: set TLLM_LOG_BATCH_WAIT=1 (rank 0 only)."""
         if self.dist.rank != 0:
             return
+        if os.getenv("TLLM_LOG_BATCH_WAIT") != "1":
+            return

         num_scheduled_gen_tokens = sum(1 + gen_req.num_draft_tokens
                                        for gen_req in generation_requests)
         num_scheduled_ctx_formula = num_scheduled_tokens - num_scheduled_gen_tokens

         chunk_ctx_sum = 0
-        ctx_summaries: List[str] = []
+        ctx_summaries: list[str] = []
         max_detail = 4
         for i, ctx_req in enumerate(context_requests):
             full_len = len(ctx_req.get_tokens(0))
             begin = ctx_req.context_current_position
             chunk_sz = ctx_req.context_chunk_size
             this_chunk = min(chunk_sz, max(0, full_len - begin))
             chunk_ctx_sum += this_chunk
             reusable = (ctx_req.estimated_reusable_tokens
                         if ctx_req.is_first_context_chunk else 0)
             reusable_in_chunk = max(0, reusable - begin)
             remaining = ctx_req.context_remaining_length
             if (reusable_in_chunk > 0
                     and reusable_in_chunk + chunk_sz < remaining):
                 formula_contrib = chunk_sz
             else:
                 formula_contrib = max(1, chunk_sz - reusable_in_chunk)
             if i < max_detail:
                 ctx_summaries.append(
                     f"rid={ctx_req.py_request_id} full={full_len} pos={begin} "
                     f"chunk_sz={chunk_sz} this_chunk={this_chunk} "
                     f"reusable={reusable} formula_contrib={formula_contrib}")
         n_ctx = len(context_requests)
         if n_ctx > max_detail:
             ctx_summaries.append(f"... +{n_ctx - max_detail} more ctx req(s)")

         logger.info(
-            "batch_wait: formula_total=",
-            num_scheduled_tokens,
-            " formula_ctx=",
-            num_scheduled_ctx_formula,
-            " formula_gen=",
-            num_scheduled_gen_tokens,
-            " chunk_ctx_sum=",
-            chunk_ctx_sum,
-            " threshold=",
-            wait_threshold,
-            " wait_iter=",
-            self.batch_wait_iters_count,
-            "/",
-            self.batch_wait_timeout_iters,
-            " should_defer_ctx=",
-            should_waiting,
-            " num_gen=",
-            len(generation_requests),
-            " ctx_detail=[",
-            "; ".join(ctx_summaries),
-            "]",
+            "batch_wait: formula_total=%s formula_ctx=%s formula_gen=%s "
+            "chunk_ctx_sum=%s threshold=%s wait_iter=%s/%s "
+            "should_defer_ctx=%s num_gen=%s ctx_detail=[%s]",
+            num_scheduled_tokens,
+            num_scheduled_ctx_formula,
+            num_scheduled_gen_tokens,
+            chunk_ctx_sum,
+            wait_threshold,
+            self.batch_wait_iters_count,
+            self.batch_wait_timeout_iters,
+            should_waiting,
+            len(generation_requests),
+            "; ".join(ctx_summaries),
         )

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@tensorrt_llm/_torch/pyexecutor/py_executor.py` around lines 3052 - 3143, The
logging call in _maybe_log_batch_wait_decision currently passes many positional
args to logger.info causing a formatting TypeError and it never obeys the
documented TLLM_LOG_BATCH_WAIT gate; update the function to first check the env
var (e.g. os.environ.get("TLLM_LOG_BATCH_WAIT") == "1") in addition to
self.dist.rank, and replace the multi-argument logger.info(...) call with a
single formatted message (built with an f-string or using .format) that embeds
num_scheduled_tokens, num_scheduled_ctx_formula, num_scheduled_gen_tokens,
chunk_ctx_sum, wait_threshold, self.batch_wait_iters_count,
self.batch_wait_timeout_iters, should_waiting, len(generation_requests), and the
ctx_detail string so the logger receives one string and no extra arguments.