LLM Prompt Lifecycle & Skill Selection Architecture

Deep Dive Architecture Specification

How LLM Agents Select & Load Skills from Disk

The fundamental challenge in building production agents: The LLM cannot know a skill exists unless it is in the request payload, but loading 100+ skills full-text crashes performance. Discover how agent systems solve this using Two-Tier Progressive Disclosure.

1 The Core Paradox (Context vs Discovery)

Constraint

If an agent has 50 or 100 domain skills stored on disk (guidelines, personas, tool packages, prompt templates), loading all of them full-text into every API request creates severe bottlenecks:

  • ✕
    Context Window Bloat & High Cost:

    50 detailed skills easily consume 40,000+ tokens per request, dramatically multiplying API costs.

  • ✕
    Time-To-First-Token Latency (TTFT):

    Processing massive system prompts slows down pre-fill generation and response times.

  • ✕
    Instruction Noise & Rule Conflicts:

    Loading dozens of irrelevant rules increases model hallucinations and tool misuse.

2 Two-Tier Progressive Disclosure Solution

Architecture

Instead of loading all skill bodies upfront, agent frameworks split skill files into two distinct layers:

Tier 1: Metadata Catalog Index (~15 Tokens / Skill) Always in System Prompt

A lightweight manifest listing only the filename and a 1-sentence description. 

• caveman.md: "Respond like smart caveman constraints"
• market_intel.md: "Adds stock quote & FX conversion tools"
Tier 2: Full Body & Schemas (On Demand) Fetched From Disk Only When Needed

Complete markdown rules, persona constraints, and function schemas fetched from disk ONLY when Tier 1 matches user intent.

3 Pattern A: In-Loop Meta-Tool (`load_skill`)

Model-Driven

The LLM operates as its own dynamic kernel. When the user prompt matches a description in the Tier 1 Catalog Index, the LLM emits a tool call to the skill loader:

// Outbound LLM Generation:

tool_calls: [{ name: "load_skill", args: { skill_name: "caveman.md" } }]

Execution Steps:
  1. Application receives load_skill request.
  2. Application reads skills/caveman.md from local disk.
  3. Application injects full text into Turn 2 system prompt payload.

4 Pattern B: Pre-LLM Semantic Router (RAG)

Framework-Driven

Agent frameworks (LangChain, AutoGen, LlamaIndex) run semantic similarity matching before sending the first prompt to the LLM:

// Pre-Flight Framework Step:

embedding_search(user_prompt) ➔ matches ["caveman.md", "market_intel.md"]

Execution Steps:
  1. User query matched against vector store of skill descriptions.
  2. Relevant skills auto-injected into Turn 1 system prompt.
  3. LLM gets relevant skills instantly without needing a load_skill turn.

API Serialization Reality: Why Tier 1 Catalog Lives Inside `system_prompt` Text

Standard API Spec
The API Constraints:

In standard LLM API specifications (OpenAI, Anthropic, Gemini), there is no native role: "catalog" message type. The API strictly recognizes four message roles: system, user, assistant, and tool.

The Serialization Strategy:

Because no separate catalog role exists in the JSON schema, the Tier 1 Metadata Index is formatted as structured markdown text inside the system message string under a dedicated heading (e.g. AVAILABLE SKILLS CATALOG).

Turn Cognitive Decision Mechanics: Analyzing Phase

Trigger Condition: Evaluating
1. Input Context Gap

Reading payload...

2. Decision Reasoning (Why)

Analyzing model reasoning...

3. Action Dispatched (How)

Evaluating model act...

Inbound Request Payload (Sent TO LLM) 0 Tokens
[REQUEST PAYLOAD] Turn 1 Input Payload
Outbound Response (Emitted BY LLM) 0 Tokens
LLM Decision & Generation [LLM RESPONSE]

Context Window Stack & Active State Registry

0 Tokens
Token Capacity Allocation Max 8,192 Tok
system_prompt core_tools loaded_skills user_prompt tool_response assistant_cot

Active Skill Registry 0 Loaded

Available Tools in Schema 1 Core Tool

Raw API Interchange Payload (Standard JSON Spec)