Building a self-maintaining prompt engineering system
Building a self-maintaining prompt engineering system
Prompt engineering artifacts form a layered system where every layer depends on the ones below it. A context file defines a rule. An instruction file references that rule. An agent enforces it. A prompt orchestrates the agent. This architecture works well when itβs fresh β but it decays through three forces: platform drift (VS Code ships monthly, Copilot evolves weekly), internal drift (subtle inconsistencies accumulate as you add artifacts), and ecosystem evolution (external best practices improve upon what youβve built).
All three forces share the same outcome: the system silently degrades. Manual audits catch it eventually, but βeventuallyβ is the failure mode.
This article describes a system that maintains itself β using the same agents and prompts it manages to detect, assess, and fix its own drift.
Series context: This is a case study applying patterns from How to Design Orchestrator Prompts and How to Design Subagent Orchestrations. The implementation articles cover pe-meta (strategic oversight) and pe-gra (per-artifact operations).
Table of contents
- The problem: silent degradation
- The goal
- Architecture: three tiers
- The self-update cycle
- Key design decisions
- What makes it work in practice
- Current status and limitations
- References
The problem: silent degradation
Silent degradation takes two forms that require different responses.
Structural staleness β dangling references, stale validation timestamps, broken chain alignment, mismatched metadata. These are deterministic to detect. A grep_search for a renamed file finds every broken reference. Fail-closed is the right response.
Capability staleness β the artifacts still parse, still reference-resolve, still pass structural checks, but the logic they encode is obsolete. A new model family is released and existing model-specific guidance is outdated. A feature moves from preview to GA and the βpreview limitationsβ note remains. Capability staleness passes all integrity checks yet produces progressively worse outputs. This is the harder variant β and the primary motivating concern.
The goal
Build a self-update system that maintains PE artifacts at peak reliability (consistent outcomes), effectiveness (achieving stated purpose), and efficiency (minimal resource consumption) β with the maximum degree of autonomy that assessed risk allows.
The autonomy gradient
Autonomy isnβt binary. Itβs a gradient calibrated to assessed risk:
| Level | Scope | Examples |
|---|---|---|
| Autonomous | Low impact, high confidence | Reference updates, formatting, stale timestamps |
| Autonomous + notify | Low-medium impact, validated | Redundancy removal, adding metadata fields |
| Human approval | High impact or medium confidence | Restructuring artifacts, changing rules |
| Human-only | Architectural or strategic | Vision changes, principle modifications |
Each proposed change carries an assessed risk level (impact Γ confidence). Changes below the threshold proceed automatically. Changes above escalate β only the individual step, not the whole change-set.
Architecture: three tiers
The system is organized into three operational tiers, each with a distinct role:
βββββββββββββββββββββββββββββββββββββββββββββββ
β pe-meta (Tier 0.09) β
β Strategic oversight β vision alignment, β
β ecosystem coherence, quality bar β
β 4 agents + 6 prompts β
βββββββββββββββββββββββββββββββββββββββββββββββ€
β pe-gra (Tier 0.02) β
β Per-artifact operations β research, β
β build, validate for each artifact type β
β 24 agents + 20 prompts β
βββββββββββββββββββββββββββββββββββββββββββββββ€
β Context files (Tier 0.00) β
β Rules, patterns, contracts β the β
β knowledge base that agents enforce β
β 32 context files + 12 instruction files β
βββββββββββββββββββββββββββββββββββββββββββββββpe-meta looks at the system as a whole β does the PE infrastructure align with the vision? Are artifacts at the right quality bar? Is the ecosystem coherent?
pe-gra operates on individual artifacts β researching requirements, building files, and validating compliance for each specific artifact type (prompts, agents, context files, instructions, skills, templates, hooks, snippets).
Context files encode the rules, patterns, and contracts that both tiers enforce. Theyβre the knowledge base β updated by pe-gra builders, validated by pe-gra validators, and audited strategically by pe-meta.
The triad pattern
pe-gra uses a consistent researcher β builder β validator triad for each artifact type:
| Role | Mode | Purpose |
|---|---|---|
| Researcher | plan (read-only) |
Analyze gaps, discover patterns, recommend changes |
| Builder | agent (read+write) |
Create or update files following specifications |
| Validator | plan (read-only) |
Check structural compliance, catch regressions |
This separation enforces single-responsibility and provides natural quality gates. The builder canβt skip validation β it hands off to the validator, which hands back fix instructions if needed. A loop cap (max 2 round-trips) prevents infinite fix cycles.
The self-update cycle
The system follows a four-phase cycle: Detect β Assess β Propose β Execute.
Detect
Five trigger sources initiate scoped health checks:
| Signal | Source | Example |
|---|---|---|
| Platform event | VS Code/Copilot release notes | New YAML field, tool behavior change |
| Model event | Provider release notes | New model capabilities |
| Ecosystem event | Trusted external sources | Better approach from Microsoft, Anthropic |
| Internal event | File modification tracker | An artifact was edited |
| Scheduled | Time-based interval | Catch anything event triggers missed |
Assess
Risk classification uses a progressive model that matures with the rollout phases. At Phase 1, two coarse inputs provide the baseline: tier (which layers can depend on this artifact?) and dependent count (how many artifacts reference this?). At Phase 2+, the system upgrades to actual dependency topology with coupling-aware analysis β distinguishing behavioral dependencies (an agent enforcing rules from a context file) from informational mentions (a βsee alsoβ in a references section). This progressive refinement improves blast radius accuracy without requiring Phase 3 infrastructure at Phase 1.
The breaking/non-breaking classification is the critical decision. Metadata-driven classification checks whether a change alters the artifactβs declared goal, scope, or boundaries. N-1 structural separation enables deterministic classification at the content level β if a diff touches a Rule block, itβs a breaking candidate; if it only touches Rationale or Example blocks, itβs non-breaking.
Propose
When assessment reveals drift, the system generates a scoped correction plan. For ecosystem changes, it decides whether to adopt (rely on the external artifact directly) or incorporate (absorb useful ideas into owned artifacts).
Changes are decomposed into the smallest individually-verifiable steps, ordered by risk: non-regressive steps first (autonomous), potentially-regressive steps next (escalated), optimization steps last.
Execute
Before committing any change, at least one independent validation pass runs. Risk-reduction techniques include static validation, redundant LLM processing (two independent passes), and reversibility guarantees (pre-change snapshots).
Key design decisions
Metadata contracts on every artifact
Every agent, prompt, and context file carries YAML metadata declaring its goal:, scope:, boundaries:, and rationales:. This transforms implicit knowledge into machine-readable state. When a change is proposed, the system checks it against the declared metadata β a change that contradicts the artifactβs goal: is automatically flagged as breaking.
Category-based references (Level 1.5)
Cross-artifact references use functional category IDs (e.g., agent-patterns) rather than specific filenames (e.g., 02.04-agent-shared-patterns.md). Categories are stable semantic identifiers maintained in STRUCTURE-README.md. When files are renamed or split, only the category mapping updates β consumer artifacts donβt change.
Handoff data contracts
Every agent-to-agent transition specifies what data is sent, what template to use, and the maximum token budget. This prevents token bloat during multi-agent orchestration and ensures both sides agree on the data format.
Validation at 3 reference types
The system validates all cross-artifact reference types: π file references, markdown links, AND slash-command references (/name). This was added after discovering that 174 broken slash-command references persisted undetected across 29 files β the validators only checked π references and markdown links.
What makes it work in practice
The vision document is the anchor. Every strategic review checks artifacts against the visionβs rationales. Without a clear βwhat weβre trying to achieve and why,β thereβs nothing to validate against.
Decomposition controls context rot. Each agent has a focused scope (one artifact type, one role). Handoffs carry only essential state. Token budgets prevent context window exhaustion.
The quality bar compounds. PE-for-PE artifacts are held to an βexemplaryβ standard β stricter boundaries, category-based references, handoff contracts, embedded test scenarios. Since pe-gra agents are the reference implementation that all domain artifacts copy, their quality propagates through everything they build.
Safety takes precedence over efficiency. The system defaults to conservative autonomy thresholds. Loop caps prevent infinite fix cycles. Pre-change compatibility gates block contradicting changes. Reversibility guarantees enable confident rollback.
Guidance quality bounds autonomy. The systemβs ability to make autonomous decisions is only as good as the rules it follows. Before increasing autonomy levels, the guidance those rules come from must be clear (unambiguous), complete (no gaps), non-contradictory (rules agree across files), and properly prioritized (conflicts are resolved explicitly). This is why pe-meta doesnβt just check whether artifacts are structurally correct β it checks whether the guidance they follow is good enough to trust for autonomous operation.
Current status and limitations
The system is operational at Phase 2 readiness (structured metadata in place, deterministic diffs for YAML changes). The universal blocker to Phase 3 (autonomous prose edits) is N-1 structural separation β rule-bearing sections in agent bodies still use ad-hoc formatting rather than labeled Rule/Rationale/Example blocks.
What works today:
- pe-meta strategic reviews detecting Level 2 reference violations, missing contracts, boundary gaps
- pe-gra validators catching structural compliance issues, tool alignment violations, broken references
- Automated slash-command reference integrity checking
- Metadata-driven breaking/non-breaking classification for YAML changes
Known limitations:
- Capability staleness (Type B) detection relies on manual trigger (scheduled review)
- Engine is embedded in the Learning Hub repository but easily portable to other repositories β not yet portable via MCP/SDK/extension
References
- Vision document:
06.00-idea/self-updating-prompt-engineering/20260501.01-vision.v8.md - pe-meta implementation: Self-maintaining PE: pe-meta implementation
- pe-gra implementation: Self-maintaining PE: pe-gra implementation
- Orchestrator patterns: How to Design Orchestrator Prompts
- Subagent patterns: How to Design Subagent Orchestrations