ICM Workspace: Multi-Draft Hugo Blog Processing Pipeline
A production-ready implementation of an Interpretable Context Methodology (ICM) workspace for processing multiple raw markdown drafts into publication-ready Hugo blog posts. Uses three discrete compiler-style passes, human-in-the-loop review, and a configurable agent CLI — all coordinated through the filesystem.
1. Directory Tree
hugo_blog_workspace/
├── CLAUDE.md # Layer 0: Global identity
├── CONTEXT.md # Layer 1: Task routing
├── processing_drafts.py # Full agent pipeline (all 3 stages via CLI)
├── speedy_processing_drafts.py # Fast pipeline (stage 1 agent, stages 2-3 inline)
├── .env # Environment configuration
├── .python-version
├── pyproject.toml
├── uv.lock
├── .venv/
├── processing_queue/ # Raw drafts to process
├── ready_for_hugo/ # Final Hugo-ready output
├── _config/
│ ├── hugo_archetype.md # Front-matter template
│ ├── existing_tags.md # Tag taxonomy (~177 tags)
│ └── edit_history.log # HITL edit audit trail
└── stages/
├── 01_tax_and_meta/
│ ├── CONTEXT.md # Stage contract
│ ├── active_input.md # Current draft (mounted per run)
│ └── output/
├── 02_hugo_assembly/
│ ├── CONTEXT.md
│ └── output/
└── 03_syntax_audit/
├── CONTEXT.md
└── output/
2. Architecture
This workspace follows the five-layer context hierarchy defined in the ICM paper (Van Clief & McDermott, 2026, arXiv:2603.16021).
| Layer | File | Purpose |
|---|---|---|
| 0 | CLAUDE.md |
Global identity — tells the agent what workspace it’s in |
| 1 | CONTEXT.md |
Task routing — maps the workflow across stages |
| 2 | stages/*/CONTEXT.md |
Stage contracts — what to read, how to process, what to write |
| 3 | _config/ |
Reference material — stable rules and conventions |
| 4 | active_input.md, output/ |
Working artifacts — per-run content |
Three processing stages
| Stage | Reads | Writes |
|---|---|---|
| 1 — Taxonomy Analysis | raw draft + _config/existing_tags.md |
output/meta_proposal.txt (title + tags) |
| 2 — Hugo Assembly | meta_proposal + _config/hugo_archetype.md + raw text |
output/publication_ready_post.md |
| 3 — Syntax Audit | assembled post | output/audited_post.md (validated) |
Stage 1: stages/01_tax_and_meta/CONTEXT.md
Reads the raw draft and the tag taxonomy, then formulates a title and selects 2-5 matching tags. If the post centers on a tool not in the taxonomy, it suggests PROPOSED_NEW_TAGS:.
The output includes comment lines (visible in vim) that explain each field:
TITLE: Automating Terminal Environments with Chezmoi
# Edit tags for this draft — you can enter new tags here (comma-separated, e.g. "tag1", "tag2")
TAGS: ["linux", "chezmoi", "automation"]
# Add new tags to existing_tags.md (comma-separated, e.g. toolname, concept)
PROPOSED_NEW_TAGS:
Stage 2: stages/02_hugo_assembly/CONTEXT.md
Injects the verified title, tags, and system timestamp into the Hugo archetype template, then appends the raw body text below the front-matter delimiter.
Stage 3: stages/03_syntax_audit/CONTEXT.md
Validates code block fencing (language tags, proper closing), header hierarchy, and table formatting.
3. One-Shot Agent vs ICM Pipeline
One-shot: just ask the agent
"Here are 4 raw markdown files. Turn them into Hugo blog posts."
Pros: Simple. One conversation, minimal setup.
Cons: Everything is a black box. If the title is wrong, the tags don’t match your taxonomy, or the code blocks are broken — re-prompt or edit the final output manually. Same mistakes next time.
ICM: staged with contracts
[copy file] → [stage 1: taxonomy] → [you edit] → [stage 2: assembly] → [stage 3: audit]
| Dimension | One-shot | ICM |
|---|---|---|
| See intermediate state | No | Yes — meta_proposal.txt is readable |
| Correct mid-pipeline | No, start over | Yes — edit taxonomy before assembly |
| Fix for all future runs | No | Yes — amend stage contract or reference file |
| Tag taxonomy evolves | No | Yes — PROPOSED_NEW_TAGS: auto-syncs |
| Edit history | No | Yes — _config/edit_history.log |
When ICM wins: A recurring workflow (raw notes → blog posts, weekly or monthly). The pipeline compounds — every HITL edit improves the system.
4. Two Scripts
| Script | Stage 1 | Stage 2 | Stage 3 | When to use |
|---|---|---|---|---|
processing_drafts.py |
agent CLI | agent CLI | agent CLI | Full pipeline, most flexible |
speedy_processing_drafts.py |
agent CLI | inline Python | inline Python | Faster — stages 2 & 3 are ~instant |
5. Why Processing Is Slow
Each agent call: model startup (~5-15s), context processing (~10-30s), reasoning (~15-60s), output generation (~10-30s). Total: 30s to 2 min per call.
| Script | Agent calls per file | Est. time for 4 files |
|---|---|---|
processing_drafts.py |
3 | 12-24 min |
speedy_processing_drafts.py |
1 | 4-8 min |
Stages 2 and 3 are purely mechanical — Python does them instantly.
6. Environment Configuration
ICM_AGENT=claude
EDITOR=vim
ICM_HITL=true
| Variable | Default | Purpose |
|---|---|---|
ICM_AGENT |
claude |
Agent CLI binary |
EDITOR |
vim |
Editor for HITL review gate |
ICM_HITL |
true |
Toggle HITL gate (true/false/1/0/yes/no) |
Prepending VAR=value before a command sets that variable for the duration of that command only. The same variables work from .env.
7. Pipeline Flow
- Mount — copy draft →
stages/01_tax_and_meta/active_input.md - Stage 1 — agent writes
meta_proposal.txt(title + tags) - HITL gate — opens
$EDITORwith the proposal. Edit and save - Log edits — changes recorded in
_config/edit_history.log - Auto-sync tags —
PROPOSED_NEW_TAGS:appended to_config/existing_tags.md - Timestamp — appends
GENERATED_SYSTEM_DATE:to proposal - Stage 2 — assembles Hugo markdown with front-matter
- Stage 3 — validates code blocks and syntax
- Export — moves final file to
ready_for_hugo/{slugified_filename} - Cleanup — removes temporary stage files
8. Tag Auto-Sync
PROPOSED_NEW_TAGS only adds to the taxonomy for future runs — it does not merge into the current post’s front-matter. To tag the current post, edit the TAGS array directly.
9. How to Use This Workflow
First-time setup
- Configure
.env— set your agent, editor, and HITL preference:ICM_AGENT=claude EDITOR=vim ICM_HITL=true - Make sure your tag taxonomy exists at
_config/existing_tags.md(one tag per line). - Place raw drafts in
processing_queue/as.mdfiles.
Running the pipeline
- Open a terminal in
hugo_blog_workspace/. - Run the script:
uv run processing_drafts.py - The script picks up all
.mdfiles fromprocessing_queue/and processes them one by one.
What happens for each file
- Stage 1 runs — claude reads the draft and tag taxonomy, writes a proposal with title + tags.
- Vim opens with the proposal. You see:
TITLE: Automating Terminal Environments with Chezmoi # Edit tags for this draft — you can enter new tags here (comma-separated, e.g. "tag1", "tag2") TAGS: ["linux", "chezmoi", "automation"] # Add new tags to existing_tags.md (comma-separated, e.g. toolname, concept) PROPOSED_NEW_TAGS:- Edit the title if needed
- Edit TAGS — pick 2-5 tags, or add new ones (they go into the Hugo front-matter)
- Add PROPOSED_NEW_TAGS if claude missed a tool central to the post (these get saved to
existing_tags.mdfor future runs) - Save and quit (
:wq)
- Stages 2 and 3 run — Hugo front-matter is assembled, code blocks are audited.
- Final file appears in
ready_for_hugo/with the same name (lowercased). - Script moves to the next file in the queue.
After the run
- Copy files from
ready_for_hugo/into your Hugo site’scontent/directory. - Check
_config/edit_history.logto see what you changed during review. - New tags you proposed are now in
_config/existing_tags.mdfor next time.
Skipping the vim review
For fully automated runs, set ICM_HITL=false in .env or prepend it:
ICM_HITL=false uv run processing_drafts.py
The script accepts whatever claude proposed and moves on.
Using the fast script
uv run speedy_processing_drafts.py
Same workflow, but Stages 2 and 3 are instant (inline Python). Only Stage 1 calls claude.
10. Usage
11. Reference Config Files
_config/hugo_archetype.md
+++
title = "ASSIGN_TITLE"
date = "ASSIGN_DATE"
draft = false
toc = true
tags = []
# ```code block
# ```
+++
12. Complete File Reference
CLAUDE.md (Layer 0)
# Layer 0: Global Workspace Identity
You are an expert technical content engineering agent operating inside a local, multi-pass ICM workspace.
Your sole focus is transforming unformatted engineering notes into publication-ready Hugo markdown files while maintaining
strict file-scoping boundaries.
## Core Operational Rules
- Never read directories, relative paths, or data streams outside of what is explicitly stated in the active Layer 2 stage contract.
- Write all transformed text outputs directly to the local `output/` directory of the active stage folder.
- Never alter code snippets, structural configurations, or inline code comments provided in raw working input files.
- Maintain total structural transparency. Avoid using binary representations or proprietary wrapper markup.
CONTEXT.md (Layer 1)
# Layer 1: Workspace Task Routing
## Macro Workflow Definition
This workspace manages the sequential transformation of unorganized engineering notes and raw markdown files into structured,
front-matter-compliant Hugo blog entries without using code-level multi-agent orchestration frameworks.
## Linear Execution passes
1. **01_tax_and_meta**: Evaluates the raw draft file against our historical taxonomy to determine the title and select correct, standardized categories.
2. **02_hugo_assembly**: Merges the raw text with a structural Hugo TOML archetype and the verified metadata block.
3. **03_syntax_audit**: Performs static analysis on the final file to verify code block syntax formatting, markdown headers, and front-matter boundaries before publication.
stages/01_tax_and_meta/CONTEXT.md (Stage 1 Contract)
# Layer 2: Stage Contract - Taxonomy Analysis & Title Extraction
## Inputs
- Layer 4 (working): active_input.md
- Layer 3 (reference): ../../_config/existing_tags.md
## Process
1. Read the active engineering text inside "active_input.md".
2. Ingest the standardized, lowercased tags file at "../../_config/existing_tags.md".
3. Formulate a highly specific, direct title based on the core technical concept covered in the text.
4. Cross-reference the content and select a maximum of 2 to 5 matching tags strictly from the list of existing tags.
5. If the post centers deeply on a specific application or tool not found in the list, append a single line titled "PROPOSED_NEW_TAGS: toolname". Otherwise, leave that line completely blank.
6. Emit the result precisely matching the layout defined below, including the comment lines. Do not output conversational explanations or introductory fluff.
## Outputs
- meta_proposal.txt -> output/
---
## Target Output Format Example
TITLE: Automating Terminal Environments with Chezmoi
# Edit tags for this draft — you can enter new tags here (comma-separated, e.g. "tag1", "tag2")
TAGS: ["linux", "chezmoi", "automation"]
# Add new tags to existing_tags.md (comma-separated, e.g. toolname, concept)
PROPOSED_NEW_TAGS:
stages/02_hugo_assembly/CONTEXT.md (Stage 2 Contract)
# Layer 2: Stage Contract - Front Matter Assembly & Structural Compilation
## Inputs
- Layer 4 (working): ../../stages/01_tax_and_meta/active_input.md
- Layer 4 (working): ../01_tax_and_meta/output/meta_proposal.txt
- Layer 3 (reference): ../../_config/hugo_archetype.md
## Process
1. Read the parsed configuration block at "../01_tax_and_meta/output/meta_proposal.txt".
2. Read the baseline front-matter template at "../../_config/hugo_archetype.md".
3. Inject the `TITLE` string directly into the template's title parameter field.
4. Inject the `TAGS` array elements directly into the template's tags field.
5. Locate the injected `DATE` configuration parameter passed into the environment by the batch coordinator script and populate the date variable string following TOML formatting boundaries.
6. Copy the raw text file from "../../stages/01_tax_and_meta/active_input.md" and place it directly below the trailing front-matter divider line (`+++`).
## Outputs
- publication_ready_post.md -> output/
stages/03_syntax_audit/CONTEXT.md (Stage 3 Contract)
# Layer 2: Stage Contract - Static Code Block & Content Audit Pass
## Inputs
- Layer 4 (working): ../02_hugo_assembly/output/publication_ready_post.md
## Process
1. Read the assembled Hugo markdown file at "../02_hugo_assembly/output/publication_ready_post.md".
2. Parse the text body to verify code block structural syntax rules:
- Every single code block MUST open with triple backticks followed instantly by a valid lowercase language designator (e.g., ```python, ```bash, ```toml).
- Code blocks must close securely with triple backticks. If any code is left naked or missing a language tag, rewrite the block to enforce absolute compliance.
3. Check the markdown layout architecture:
- Ensure the post structural hierarchy utilizes clean headers (##, ###).
- Verify that no markdown tables contain built-in export parameters or hidden CSV strings.
4. If the structural syntax fully complies with these validation checks, write the finalized content file to output/. If structural flaws were found and fixed, output the corrected version.
## Outputs
- audited_post.md -> output/
_config/hugo_archetype.md
+++
title = "ASSIGN_TITLE"
date = "ASSIGN_DATE"
draft = false
toc = true
tags = []
# ```code block
# ```
+++
.env
ICM_AGENT=claude
EDITOR=vim
ICM_HITL=true
processing_drafts.py
#!/usr/bin/env python3
"""
Local ICM Multi-Draft Coordinator Engine.
Handles automated system date/timezone detection, file management, and
human-in-the-loop review execution across a batch of staging files.
"""
import datetime
import os
import re
import shutil
import subprocess
import sys
def load_env_file(path=".env"):
"""Load a simple KEY=VALUE .env file (no external dependency)."""
if not os.path.exists(path):
return
with open(path) as f:
for line in f:
line = line.strip()
if not line or line.startswith("#"):
continue
match = re.match(r"^([A-Za-z_][A-Za-z0-9_]*)=(.*)$", line)
if match:
key, val = match.group(1), match.group(2).strip()
if len(val) >= 2 and val[0] == val[-1] and val[0] in ('"', "'"):
val = val[1:-1]
os.environ.setdefault(key, val)
QUEUE_DIR = "processing_queue"
STAGE1_DIR = "stages/01_tax_and_meta"
STAGE1_INPUT = os.path.join(STAGE1_DIR, "active_input.md")
STAGE1_OUTPUT = os.path.join(STAGE1_DIR, "output/meta_proposal.txt")
STAGE2_DIR = "stages/02_hugo_assembly"
STAGE2_OUTPUT = os.path.join(STAGE2_DIR, "output/publication_ready_post.md")
STAGE3_DIR = "stages/03_syntax_audit"
STAGE3_OUTPUT = os.path.join(STAGE3_DIR, "output/audited_post.md")
FINAL_EXPORT_DIR = "ready_for_hugo"
EDIT_LOG = "_config/edit_history.log"
ICM_AGENT = os.environ.get("ICM_AGENT", "claude")
EDITOR = os.environ.get("EDITOR", "vim")
ICM_HITL = os.environ.get("ICM_HITL", "true").lower() in ("true", "1", "yes")
def initialize_workspace():
if not os.path.exists(QUEUE_DIR) or not os.listdir(QUEUE_DIR):
print(f"[-] Aborted: Staging directory '{QUEUE_DIR}' is empty or missing.")
sys.exit(1)
os.makedirs(FINAL_EXPORT_DIR, exist_ok=True)
os.makedirs(os.path.dirname(EDIT_LOG), exist_ok=True)
def get_warsaw_timestamp():
now = datetime.datetime.now(datetime.timezone.utc).astimezone()
return now.strftime("%Y-%m-%dT%H:%M:%S%z")
def build_agent_args(prompt):
agent = ICM_AGENT
if "claude" in agent:
return [agent, "-p", prompt, "--print", "--dangerously-skip-permissions"]
return [agent, prompt]
def run_agent_pass(stage_name, contract_path, expected_output=None):
print(f"[*] Executing Pass: {stage_name} (agent: {ICM_AGENT})")
cwd = os.getcwd()
prompt = (
f"You are executing an ICM pipeline stage. "
f"Read the stage contract at {contract_path}/CONTEXT.md and follow its Process section exactly. "
f"All relative paths in the contract are relative to the contract's directory ({contract_path}/). "
f"The working directory is {cwd}. "
f"Write the output to the path specified under Outputs. "
f"Do not add any commentary or explanation."
)
args = build_agent_args(prompt)
result = subprocess.run(args, capture_output=True, text=True, cwd=cwd)
if result.returncode != 0:
print(f"[-] Agent error (exit {result.returncode}): {result.stderr[:500]}")
return False
if expected_output and not os.path.exists(expected_output):
print(f"[-] Agent exited OK but did not create expected output: {expected_output}")
print(f" Last 500 chars of response:\n{result.stdout[-500:]}")
return False
return True
TAGS_FILE = "_config/existing_tags.md"
def sync_proposed_tags(proposal_text):
match = re.search(r"^PROPOSED_NEW_TAGS:\s*(.*)$", proposal_text, re.MULTILINE)
if not match:
return
raw = match.group(1).strip()
if not raw:
return
new_tags = [t.strip() for t in raw.split(",") if t.strip()]
if not new_tags:
return
if not os.path.exists(TAGS_FILE):
print(f" [*] Creating {TAGS_FILE} with proposed tags")
with open(TAGS_FILE, "w") as f:
f.write("\n".join(new_tags) + "\n")
return
with open(TAGS_FILE) as f:
existing = set(line.strip() for line in f if line.strip())
added = [t for t in new_tags if t not in existing]
if added:
with open(TAGS_FILE, "a") as f:
for tag in added:
f.write(tag + "\n")
print(f" [*] Added {len(added)} new tag(s) to {TAGS_FILE}: {', '.join(added)}")
def log_human_modification(filename, content_before, content_after):
if content_before != content_after:
with open(EDIT_LOG, "a") as f:
timestamp = datetime.datetime.now().isoformat()
f.write(f"\n--- EDIT RECORDED AT {timestamp} FOR {filename} ---\n")
f.write(">>> ORIGINAL AGENT PROPOSAL:\n" + content_before + "\n")
f.write(">>> HUMAN MODIFICATION:\n" + content_after + "\n")
def execute_pipeline():
drafts = [f for f in os.listdir(QUEUE_DIR) if f.endswith(".md")]
print(f"[+] Found {len(drafts)} targets within processing queue.")
for idx, draft_name in enumerate(drafts, start=1):
print("\n" + "=" * 70)
print(f" PROCESSING FILE [{idx}/{len(drafts)}]: {draft_name}")
print("=" * 70)
source_file_path = os.path.join(QUEUE_DIR, draft_name)
shutil.copy2(source_file_path, STAGE1_INPUT)
if not run_agent_pass("Taxonomy Analysis", STAGE1_DIR, STAGE1_OUTPUT):
continue
if not os.path.exists(STAGE1_OUTPUT):
print("[-] Error: Stage 1 failed to emit metadata proposal text.")
continue
with open(STAGE1_OUTPUT, "r") as f:
proposal_before = f.read()
if ICM_HITL:
print(f"\n[!] HITL Checkpoint: Launching {EDITOR} to review taxonomy for: {draft_name}")
print(f" Adjust titles or array tags directly inside {EDITOR}.")
input(f" Press [ENTER] to launch {EDITOR}...")
subprocess.run([EDITOR, STAGE1_OUTPUT])
else:
print(f"[*] HITL skipped (ICM_HITL=false)")
with open(STAGE1_OUTPUT, "r") as f:
proposal_after = f.read()
log_human_modification(draft_name, proposal_before, proposal_after)
sync_proposed_tags(proposal_after)
current_time_str = get_warsaw_timestamp()
with open(STAGE1_OUTPUT, "a") as f:
f.write(f"\nGENERATED_SYSTEM_DATE: {current_time_str}\n")
if not run_agent_pass("Hugo Post Compilation", STAGE2_DIR, STAGE2_OUTPUT):
continue
if not run_agent_pass("Static Content Audit", STAGE3_DIR, STAGE3_OUTPUT):
continue
clean_slug = draft_name.lower().replace(" ", "-")
final_destination = os.path.join(FINAL_EXPORT_DIR, clean_slug)
shutil.move(STAGE3_OUTPUT, final_destination)
for temp_file in [STAGE1_INPUT, STAGE1_OUTPUT, STAGE2_OUTPUT]:
if os.path.exists(temp_file):
os.remove(temp_file)
print(f"[+] Output Verification Complete: {final_destination}")
print("\n[+] Batch processing queue finished running.")
def main():
load_env_file()
initialize_workspace()
execute_pipeline()
if __name__ == "__main__":
main()
speedy_processing_drafts.py
#!/usr/bin/env python3
"""
Speed-optimised ICM pipeline.
Only Stage 1 (taxonomy analysis) requires the agent.
Stages 2 & 3 run inline as hardened Python.
"""
import datetime
import os
import re
import shutil
import subprocess
import sys
def load_env_file(path=".env"):
if not os.path.exists(path):
return
with open(path) as f:
for line in f:
line = line.strip()
if not line or line.startswith("#"):
continue
match = re.match(r"^([A-Za-z_][A-Za-z0-9_]*)=(.*)$", line)
if match:
key, val = match.group(1), match.group(2).strip()
if len(val) >= 2 and val[0] == val[-1] and val[0] in ('"', "'"):
val = val[1:-1]
os.environ.setdefault(key, val)
QUEUE_DIR = "processing_queue"
STAGE1_DIR = "stages/01_tax_and_meta"
STAGE1_INPUT = os.path.join(STAGE1_DIR, "active_input.md")
STAGE1_OUTPUT = os.path.join(STAGE1_DIR, "output/meta_proposal.txt")
STAGE2_DIR = "stages/02_hugo_assembly"
STAGE2_OUTPUT = os.path.join(STAGE2_DIR, "output/publication_ready_post.md")
STAGE3_DIR = "stages/03_syntax_audit"
STAGE3_OUTPUT = os.path.join(STAGE3_DIR, "output/audited_post.md")
FINAL_EXPORT_DIR = "ready_for_hugo"
EDIT_LOG = "_config/edit_history.log"
ICM_AGENT = os.environ.get("ICM_AGENT", "claude")
EDITOR = os.environ.get("EDITOR", "vim")
ICM_HITL = os.environ.get("ICM_HITL", "true").lower() in ("true", "1", "yes")
def initialize_workspace():
if not os.path.exists(QUEUE_DIR) or not os.listdir(QUEUE_DIR):
print(f"[-] Aborted: Staging directory '{QUEUE_DIR}' is empty or missing.")
sys.exit(1)
os.makedirs(FINAL_EXPORT_DIR, exist_ok=True)
os.makedirs(os.path.dirname(EDIT_LOG), exist_ok=True)
def get_warsaw_timestamp():
now = datetime.datetime.now(datetime.timezone.utc).astimezone()
return now.strftime("%Y-%m-%dT%H:%M:%S%z")
def build_agent_args(prompt):
agent = ICM_AGENT
if "claude" in agent:
return [agent, "-p", prompt, "--print", "--dangerously-skip-permissions"]
return [agent, prompt]
def run_agent_pass(stage_name, contract_path, expected_output=None):
print(f"[*] Executing Pass: {stage_name} (agent: {ICM_AGENT})")
cwd = os.getcwd()
prompt = (
f"You are executing an ICM pipeline stage. "
f"Read the stage contract at {contract_path}/CONTEXT.md and follow its Process section exactly. "
f"All relative paths in the contract are relative to the contract's directory ({contract_path}/). "
f"The working directory is {cwd}. "
f"Write the output to the path specified under Outputs. "
f"Do not add any commentary or explanation."
)
args = build_agent_args(prompt)
result = subprocess.run(args, capture_output=True, text=True, cwd=cwd)
if result.returncode != 0:
print(f"[-] Agent error (exit {result.returncode}): {result.stderr[:500]}")
return False
if expected_output and not os.path.exists(expected_output):
print(f"[-] Agent exited OK but did not create expected output: {expected_output}")
print(f" Last 500 chars of response:\n{result.stdout[-500:]}")
return False
return True
TAGS_FILE = "_config/existing_tags.md"
def sync_proposed_tags(proposal_text):
match = re.search(r"^PROPOSED_NEW_TAGS:\s*(.*)$", proposal_text, re.MULTILINE)
if not match:
return
raw = match.group(1).strip()
if not raw:
return
new_tags = [t.strip() for t in raw.split(",") if t.strip()]
if not new_tags:
return
if not os.path.exists(TAGS_FILE):
print(f" [*] Creating {TAGS_FILE} with proposed tags")
with open(TAGS_FILE, "w") as f:
f.write("\n".join(new_tags) + "\n")
return
with open(TAGS_FILE) as f:
existing = set(line.strip() for line in f if line.strip())
added = [t for t in new_tags if t not in existing]
if added:
with open(TAGS_FILE, "a") as f:
for tag in added:
f.write(tag + "\n")
print(f" [*] Added {len(added)} new tag(s) to {TAGS_FILE}: {', '.join(added)}")
def log_human_modification(filename, content_before, content_after):
if content_before != content_after:
with open(EDIT_LOG, "a") as f:
timestamp = datetime.datetime.now().isoformat()
f.write(f"\n--- EDIT RECORDED AT {timestamp} FOR {filename} ---\n")
f.write(">>> ORIGINAL AGENT PROPOSAL:\n" + content_before + "\n")
f.write(">>> HUMAN MODIFICATION:\n" + content_after + "\n")
def run_stage2():
"""Hugo assembly: parse meta proposal, inject into archetype, append body."""
print("[*] Executing Pass: Hugo Post Compilation (inline)")
with open(STAGE1_OUTPUT) as f:
meta = f.read()
title_match = re.search(r"^TITLE:\s*(.+?)$", meta, re.MULTILINE)
if not title_match:
print("[-] Error: meta_proposal.txt missing 'TITLE:' line.")
return False
title = title_match.group(1).strip()
if not title:
print("[-] Error: TITLE is empty.")
return False
tags_match = re.search(r"^TAGS:\s*(\[[\s\S]*?\])\s*$", meta, re.MULTILINE)
tags_str = tags_match.group(1) if tags_match else "[]"
tags_str = re.sub(r"\s+", " ", tags_str).strip()
archetype_path = "_config/hugo_archetype.md"
if not os.path.exists(archetype_path):
print(f"[-] Error: archetype file not found at '{archetype_path}'.")
return False
with open(archetype_path) as f:
template = f.read()
for placeholder in ["ASSIGN_TITLE", "ASSIGN_DATE"]:
if placeholder not in template:
print(f"[-] Error: archetype missing placeholder '{placeholder}'.")
return False
if not re.search(r"tags\s*=\s*\[\s*\]", template):
print("[-] Error: archetype missing 'tags = []' line.")
return False
result = template.replace("ASSIGN_TITLE", title)
result = result.replace("ASSIGN_DATE", get_warsaw_timestamp())
result = re.sub(r"tags\s*=\s*\[\s*\]", f"tags = {tags_str}", result)
with open(STAGE1_INPUT) as f:
body = f.read()
result = result.rstrip() + "\n" + body + "\n"
os.makedirs(os.path.join(STAGE2_DIR, "output"), exist_ok=True)
with open(STAGE2_OUTPUT, "w") as f:
f.write(result)
print(f" Title: {title}")
print(f" Tags: {tags_str}")
return True
def run_stage3():
"""Syntax audit: code-block fencing, header hierarchy, table hygiene."""
print("[*] Executing Pass: Static Content Audit (inline)")
with open(STAGE2_OUTPUT) as f:
lines = f.readlines()
in_code_block = False
new_lines = []
changes = []
warnings = []
for i, line in enumerate(lines):
stripped = line.rstrip("\n")
if stripped.startswith("```"):
if not in_code_block:
rest = stripped[3:]
if not rest or rest.strip() == "":
new_lines.append("```text\n")
changes.append(f"line {i+1}: added 'text' language tag")
in_code_block = True
continue
in_code_block = True
else:
in_code_block = False
new_lines.append(line)
if in_code_block:
new_lines.append("```\n")
changes.append("line 1 past end: added closing fence for unclosed block")
seen_levels = set()
for i, line in enumerate(new_lines):
stripped = line.rstrip("\n")
m = re.match(r"^(#{1,6})\s+", stripped)
if m:
level = len(m.group(1))
for intermediate in range(2, level):
if intermediate not in seen_levels and intermediate - 1 in seen_levels:
warnings.append(f"line {i+1}: header skips {'#' * intermediate}")
break
seen_levels.add(level)
for i, line in enumerate(new_lines):
stripped = line.rstrip("\n")
if "|" in stripped:
cells = stripped.split("|")
for cell in cells:
if re.search(r"\{\.?\w+\}", cell.strip()):
warnings.append(f"line {i+1}: cell contains hidden export parameter")
break
os.makedirs(os.path.join(STAGE3_DIR, "output"), exist_ok=True)
with open(STAGE3_OUTPUT, "w") as f:
f.writelines(new_lines)
for c in changes:
print(f" [fix] {c}")
for w in warnings:
print(f" [warn] {w}")
if not changes and not warnings:
print(" No issues found.")
return True
def execute_pipeline():
drafts = [f for f in os.listdir(QUEUE_DIR) if f.endswith(".md")]
print(f"[+] Found {len(drafts)} targets within processing queue.")
for idx, draft_name in enumerate(drafts, start=1):
print("\n" + "=" * 70)
print(f" PROCESSING FILE [{idx}/{len(drafts)}]: {draft_name}")
print("=" * 70)
source_file_path = os.path.join(QUEUE_DIR, draft_name)
shutil.copy2(source_file_path, STAGE1_INPUT)
if not run_agent_pass("Taxonomy Analysis", STAGE1_DIR, STAGE1_OUTPUT):
continue
if not os.path.exists(STAGE1_OUTPUT):
print("[-] Error: Stage 1 failed to emit metadata proposal text.")
continue
with open(STAGE1_OUTPUT, "r") as f:
proposal_before = f.read()
if ICM_HITL:
print(f"\n[!] HITL Checkpoint: Launching {EDITOR} to review taxonomy for: {draft_name}")
print(f" Adjust titles or array tags directly inside {EDITOR}.")
input(f" Press [ENTER] to launch {EDITOR}...")
subprocess.run([EDITOR, STAGE1_OUTPUT])
else:
print("[*] HITL skipped (ICM_HITL=false)")
with open(STAGE1_OUTPUT, "r") as f:
proposal_after = f.read()
log_human_modification(draft_name, proposal_before, proposal_after)
sync_proposed_tags(proposal_after)
current_time_str = get_warsaw_timestamp()
with open(STAGE1_OUTPUT, "a") as f:
f.write(f"\nGENERATED_SYSTEM_DATE: {current_time_str}\n")
if not run_stage2():
continue
if not run_stage3():
continue
clean_slug = draft_name.lower().replace(" ", "-")
final_destination = os.path.join(FINAL_EXPORT_DIR, clean_slug)
shutil.move(STAGE3_OUTPUT, final_destination)
for temp_file in [STAGE1_INPUT, STAGE1_OUTPUT, STAGE2_OUTPUT]:
if os.path.exists(temp_file):
os.remove(temp_file)
print(f"[+] Output Verification Complete: {final_destination}")
print("\n[+] Batch processing queue finished running.")
def main():
load_env_file()
initialize_workspace()
execute_pipeline()
if __name__ == "__main__":
main()
13. Issues and Fixes
Issue 1: Placeholder agent CLI (clio)
Original script called clio run — a binary that doesn’t exist.
Fix: Replaced with claude -p. Made agent configurable via ICM_AGENT env var.
Issue 2: Agent name and editor hardcoded
Hardcoded clio and vim. Anyone else would have to edit Python code.
Fix: Added ICM_AGENT, EDITOR, ICM_HITL env vars with .env file support.
Issue 3: Pipeline took too long
All three stages called claude: ~3-6 min per file.
Fix: Created speedy_processing_drafts.py — only Stage 1 uses claude, Stages 2-3 are inline Python. Cuts to ~1-2 min per file.
Issue 4: Missing output files
Claude sometimes responds with text instead of writing the file. Script only checked exit code (was 0) and silently skipped.
Fix: Added expected_output parameter — verifies the file was actually created after each agent call.
Issue 5: No feedback loop for new tags
PROPOSED_NEW_TAGS: had no effect — the tag list never grew.
Fix: Added sync_proposed_tags() — parses proposal after HITL gate, appends new tags to existing_tags.md. Taxonomy evolves organically.
Issue 6: Duplicate identity files
Both AGENTS.md and CLAUDE.md with identical content.
Fix: Deleted AGENTS.md. CLAUDE.md is the single Layer 0 identity file.
Issue 7: Vim template confusing
Comment above TAGS said “from existing_tags.md” — users thought they couldn’t add new tags there.
Fix: Changed to # Edit tags for this draft — you can enter new tags here (comma-separated, e.g. "tag1", "tag2").
Based on “Interpretable Context Methodology: Folder Structure as Agentic Architecture” (Van Clief & McDermott, 2026). arXiv:2603.16021. MIT license. https://github.com/RinDig/Interpretable-Context-Methodology-ICM-