Problem

AI prompt construction for workflow operations currently uses freetext format with markdown sections, which consumes unnecessary tokens due to:

Current Behavior

1. Markdown formatting symbols (**, -, numbered lists) add overhead
2. Redundant section headers and separators
3. JSON data embedded as formatted strings
4. ❌ Higher token consumption (e.g., Refinement: 6,252 tokens)

Expected Behavior

1. Structured format (TOON) for prompt construction
2. Reduced redundancy through array compression and hierarchical indentation
3. Direct object encoding without quotes and brackets
4. ✅ Lower token consumption (e.g., Refinement: 5,793 tokens)

Solution

Implemented TOON (Token-Oriented Object Notation) format for all AI prompts using the official @toon-format/toon library:

Key Changes

• Refinement Prompts: 7.3% token reduction (459 tokens saved per request)
• Workflow Name Generation: Structured prompt with TOON encoding
• Slack Description Generation: Structured prompt with TOON encoding
• Locale Support: Added responseLocale field for multilingual responses
• Simplified Implementation: Removed unnecessary abstractions and configurations

TOON Format Benefits

• Array compression: nodes[3]: id,type,name (CSV-style)
• No quotes needed for strings
• Indentation-based hierarchy (YAML-like)
• Direct library usage (no custom implementation)

Changes

New Files

File: src/extension/services/refinement-prompt-builder.ts (118 lines)

• Builds TOON-formatted prompts for workflow refinement
• Encodes all structured data (currentWorkflow, conversationHistory, guidelines)
• Uses @toon-format/toon official library

File: src/extension/services/workflow-name-prompt-builder.ts (41 lines)

• TOON prompt builder for workflow name generation
• Simple structure with requirements array

File: src/extension/services/slack-description-prompt-builder.ts (40 lines)

• TOON prompt builder for Slack description generation
• Optimized for brief descriptions (100-150 chars)

Modified Files

File: src/extension/services/refinement-service.ts

• Replaced 130+ lines of inline prompt construction with RefinementPromptBuilder
• Added promptFormat: 'toon' to all logs and metrics
• Simplified prompt construction logic

File: src/extension/commands/workflow-name-generation.ts

• Integrated WorkflowNamePromptBuilder
• Removed inline prompt construction (20 lines reduced)
• Added promptFormat logging

File: src/extension/commands/slack-description-generation.ts

• Integrated SlackDescriptionPromptBuilder
• Removed inline prompt construction (20 lines reduced)
• Added promptFormat logging

File: src/extension/services/ai-metrics-service.ts

• Removed getConfiguredPromptFormat() (always use TOON)
• Removed unused PromptFormat import

File: src/shared/types/ai-metrics.ts

• Added PromptFormat type: 'freetext' | 'json' | 'toon'
• Added promptFormat field to AIGenerationMetrics interface

File: package.json

• Added dependency: @toon-format/toon: ^1.4.0
• Removed cc-wf-studio.ai.promptFormat configuration (TOON is now always used)
• Updated cc-wf-studio.ai.collectMetrics description

Impact

Token Reduction

• Refinement Prompt: 7.3% reduction (6,252 → 5,793 tokens, 459 tokens saved)
• Expected for Name/Description: Similar 7-10% reduction (not yet measured)

Cost Savings (Estimated)

• Individual user (50 refinements/month): ~$0.07/month, ~$0.84/year
• Team usage (500 refinements/month): ~$0.70/month, ~$8.40/year

User Experience

• ✅ No breaking changes to user-facing features
• ✅ Multilingual support maintained via responseLocale
• ✅ All existing workflows continue to work
• ✅ No configuration required (TOON always enabled)

Code Quality

• ✅ Removed ~228 lines of unnecessary code
• ✅ Eliminated custom TOON implementation
• ✅ Removed unused abstractions (PromptBuilderBase, toon-encoder.ts)
• ✅ Simplified language instruction logic

Testing

• Manual E2E testing completed (Refinement prompt verified with 7.3% reduction)
• Code quality checks passed (npm run format, npm run lint, npm run check)
• Build successful (npm run build)
• Locale support verified (responseLocale field added)

Notes

Why 7.3% instead of expected 28-35%?

The original plan estimated 28-35% reduction for fixed text portions only. In practice:

• Fixed text (role, task, guidelines): ~20-30% reduction ✅
• Variable data (currentWorkflow, conversationHistory): ~56% of total prompt
• TOON encoding applies to entire structured object, but variable content limits compression

The 7.3% overall reduction is realistic given the large proportion of variable JSON data.

Design Simplifications Made

1. Removed LANGUAGE_INSTRUCTIONS: Using responseLocale only
2. Removed configuration options: TOON is always enabled (simpler UX)
3. Direct library usage: No custom wrapper (toon-encoder.ts)
4. No abstract base class: Each builder is self-contained

Future Optimization Opportunities

• Dynamic schema filtering (only send relevant node types)
• Workflow data compression for large workflows
• Conversation history summarization