AI Pipeline & Chat

LLM Provider Routing

Supported Providers

Role-Based Model Assignment

Each AI task has a "role" with its own model configuration:

Fallback & Extended Thinking

Each role has a primary and fallback model. If the primary fails (rate limit, timeout, error), the fallback is tried automatically.

For complex tasks, "extended thinking" allows the model to reason internally before responding. Configurable budget per role (e.g., BUILDER_THINKING_BUDGET=8000 tokens). Requires Anthropic provider.

Prompt Caching

For Anthropic models, large system prompts (>4000 chars) are automatically marked for ephemeral caching, reducing cost on repeated calls with the same system prompt.

Build Pipeline

Stage 1: Parse

Reads CSV metadata files from the AS/400 extraction: TTPGMOUT.csv (program list), TTPGM2PGM.csv (call relationships), TTPGM2FIL.csv (file access), TTFIL2FLD.csv (field definitions).

Stage 2: Build Program Documents

For each program, the LLM analyzes the source code and generates structured documentation.

• Programs batched by size (max 18 sections, 80K chars per batch)
• Large programs (>2500 lines) get smaller batches
• Up to 6 concurrent LLM calls (MAX_PARALLEL_LLM)

Output per program (output_program/{PROGRAM}.json):

{
  "documentMetadata": { "programName", "generatedAt", "language" },
  "programInfo": { "purpose", "businessContext", "module", "description" },
  "io": { "inputs": [...], "outputs": [...] },
  "fileAccess": [{ "fileName", "accessType", "fields": [...] }],
  "callGraph": { "callsThisPgm": [...], "thisPgmCalls": [...] },
  "errorHandling": [{ "code", "description" }],
  "changeImpactReport": { ... }
}

Stage 3: System Overview

The LLM analyzes the complete call graph, file dependencies, and program documentation to generate a system-level architecture document with narrative description, module identification, key business flows, risk assessment, and statistics.

Stage 4: Embedding & Indexing

• ChromaDB: Stores semantic vector embeddings of program and file descriptions for RAG search
• BM25: Full-text index of source code, field names, and descriptions for exact-match queries

Stage 5: Auto-Tagging

The AI classifies each program by business function and assigns appropriate tags from the collection's tag set.

Progress Tracking

• build_progress.json — Structured progress for UI (stage, done/total, last_activity)
• build_log.txt — Human-readable log (appended in real-time)
• Admin panel shows live progress with floating log window

Specification Document Generation

On-demand generation of formal specification documents at three audience levels:

Documents are cached in the generated_spec_docs table. Stale detection compares generation timestamp vs program JSON modification time. Manual regeneration is available via "Regenerate" button.

Chat System

Afunana's chat interface allows users to ask natural language questions about their codebase using a Retrieval-Augmented Generation (RAG) pipeline.

Query Processing Pipeline

Step 1: Intent Classification

The chat_classifier role (Claude Haiku) determines the query type: bug analysis, feature request, design question, documentation lookup, or code generation.

Step 2: Query Planning

The chat_planner role analyzes the query and determines what context is needed: which programs are relevant, which files to examine, whether source code is needed, and what search queries to run.

Step 3: Context Retrieval (Hybrid Search)

Two search systems run in parallel:

• ChromaDB (Semantic) — Searches embedded descriptions, returns top K results by vector similarity. Good for conceptual queries.
• BM25 (Full-Text) — Searches source code, field names, descriptions by term frequency. Good for exact matches.

Hybrid Search Score Weighting

Step 4: Source Code Selection

Step 5: Response Generation

The chat_answer role (Claude Sonnet) generates the response with extended thinking enabled (10,000 token budget by default). Response includes markdown formatting, code snippets, and line references with citations linking to specific programs and line numbers.

Step 6: Streaming

Responses stream token-by-token to the frontend. Users see the response being generated in real-time. Stop button allows canceling mid-generation.

Chat Session Management

• Sessions stored in dbo.chat_sessions, messages in dbo.chat_messages
• Grouped by user and collection
• Sessions listed in sidebar with title (derived from first message), grouped by date
• Rename and delete individual sessions; clear all sessions option
• Previous messages sent as context with configurable window size

Chat Attachments

Maximum 10 MB per file, 5 files per message. Extracted text added to the query context.

MCP Tools (Model Context Protocol)

Afunana exposes 7 MCP tools for integration with Claude Desktop and other MCP clients:

Chat Configuration

Privacy

Chat content logging to audit trail is configurable (AUDIT_LOG_CHAT_CONTENT). When disabled, only the event type and metadata are logged. Chat sessions are per-user and per-collection — users cannot see each other's conversations.