That sounds cleaner than it was.

The actual path was: rent something small, find out whether the agent earns its keep, hit the limits of small cloud hardware, repurpose a desktop we already owned, then decide which parts really belong locally and which parts still make sense in the cloud. The important lesson is not “local good, cloud bad.” It is that different parts of an agent have very different hardware and trust profiles.

Act 1: the rented trial

The first Hermes box was deliberately modest: an AWS t4g.large in us-east-1, ARM64/Graviton, 2 vCPU, 8 GB RAM, no GPU. It had a stable Elastic IP, a 120 GB /data volume for working state, and a tiny 20 GB root disk that repeatedly reminded me it was tiny. To keep cost under control, an EventBridge schedule stopped and started the instance around waking hours.

That was the right starting point. I did not know yet whether a self-hosted agent was going to be a useful piece of infrastructure or just another weekend experiment. A small rented box made the trial reversible.

The initial setup was simple in spirit: install Hermes, connect it to Discord, give it a persona, configure tools and scheduled jobs, and point it at a model. In practice it used gpt-5.5 through OpenAI Codex as the main model, with DeepSeek as a cheaper metered fallback when quota pressure showed up.

At that stage the cloud box was a good fit. It was cheap enough, always reachable enough, and isolated from my laptop. Most importantly, it let me test the workflow without committing to local hardware or network plumbing.

Act 2: the agent became useful

The trial stopped being a toy when the agent started doing jobs that I would not expect from a normal SaaS chat assistant.

The biggest one was pull request review. Not just “read the diff and give an opinion,” but: check out the branch, install dependencies with the repository’s package manager, run tests, inspect failures, and only then comment. For open-constructs/cdk-terrain and skillrig/cli, pollers watched PRs and reviewed them in isolated git worktrees.

That distinction matters. A read-only assistant can summarize a diff. A self-hosted agent can run the code. It can create scratch environments, execute the same commands a maintainer would run, and produce feedback grounded in real tool output.

The same box also picked up the blog workflow for this site, using GitHub Pages and Jekyll under the sakul-learning identity. It handled source-grounded drafts, local builds, commits, and publishing. On top of that came smaller but useful jobs: an email watcher that forwarded AgentMail events to Discord, and a three-times-per-day AI feed.

That is the point where self-hosting started to make sense. The agent was no longer a novelty; it was infrastructure.

Act 3: the memory wall

Execution was the first win. Memory was the first wall.

Early on, Hermes could search previous conversation transcripts. That helps, but transcript search is not the same as durable memory. A transcript can tell the agent that something was said. It cannot reliably decide whether that statement is still true, whether it was a one-off task, whether it should become a persistent preference, or whether it belongs in a reusable skill instead of ordinary memory.

So I looked at a few paths: owned Postgres with pgvector, mem0, and Hindsight. The lesson I kept coming back to was simple: nearest-neighbour search is retrieval, not memory judgment. A vector table can find similar notes. It cannot, by itself, decide which preferences are global, which environment facts are stale, and which procedures should be promoted into skills.

Hindsight was attractive because it tries to provide that memory layer rather than just a bare vector store. The problem was the EC2 box.

I tried Hindsight’s local embedded install path cautiously, starting with a dry run. That dry run resolved to 159 packages, including the full CUDA toolkit, on a GPU-less ARM instance. Worse, it wanted to downgrade cryptography inside the agent’s own virtualenv. That was an immediate stop.

The lesson is worth spelling out: dry-run any memory backend before you install it, and read what it touches. A memory layer should not mutate the agent’s runtime out from underneath it.

On the EC2 box the pragmatic answer was Hindsight Cloud. It had no local footprint, avoided the fragile shared virtualenv problem, and worked well enough. But it also meant that the most personal part of the agent — long-term memory — lived in another SaaS service. That was acceptable for the trial. It was not the endpoint I wanted.

Act 4: the GPU under the desk

The hardware answer was already in the room: a desktop called podmaster, with an Intel i5-12400F, 32 GB RAM, a 1 TB NVMe, and a GTX 1650 with 4 GB VRAM. It runs Arch Linux and a Hyprland desktop. The CPU has no integrated GPU, so the GTX 1650 is shared between the display and any ML workload, but most of the day it is still sitting there underused.

The design became:

• keep the heavy, hardware-sensitive services on the Arch host;
• run the Hermes gateway itself in an Ubuntu 24.04 KVM VM;
• bridge the VM onto the LAN so I can administer it directly;
• add a private host-services path over libvirt NAT for services that should be reachable from the VM but not from the LAN.

The scary part was networking. Building a bridge over the live SSH interface is exactly the kind of change that can lock you out. The fix was to treat it like a migration with rollback: clone the MAC address onto the bridge so DHCP kept the same lease and IP, and run a dead-man auto-revert timer in case the new network did not come back cleanly.

Once the VM had stable networking, the split became useful. Firecrawl moved to the Arch host, where x86_64 Docker images build normally instead of requiring ARM image substitutions. It is bound to the host side of libvirt NAT at 192.168.122.1:3002, reachable by the VM and not exposed to the LAN.

The same pattern made local Hindsight feasible. The EC2 instance had no GPU and a fragile shared Python environment. The desktop had a GPU and enough room to isolate services properly.

Act 5: the one-day cutover

The actual cutover happened on 2026-06-08.

The new Hermes VM was installed fresh on Ubuntu 24.04, pinned to the same Hermes commit as the EC2 box for configuration parity. I copied the portable profile state — config, secrets, scripts, internal crons, skills, kanban state — but avoided copying architecture-specific runtime. Toolchains were re-provisioned from source, mostly through mise. GitHub CLI auth and the blog SSH key carried over. The old separate /data volume became a plain directory on the VM’s single disk.

Local Hindsight was installed on the Arch host in an isolated uv virtualenv, not inside Hermes’ own environment. That isolation was the fix for the earlier dry-run scare. The package name mattered too: the agent-memory package is hindsight-all / hindsight-client; plain pip install hindsight is an unrelated Chrome-forensics package.

The local stack used bge-small embeddings plus a cross-encoder reranker on the GPU, with embedded pgvector Postgres, bound privately at 192.168.122.1:8888. Because the host’s sudo requires a password, it runs as a user systemd service with linger enabled rather than a root service.

Then came the identity handoff. The EC2 gateway had to be stopped and disabled first, because only one process should own the Discord identity. After copying the live state.db, I started the VM gateway and watched it connect: Discord came up cleanly. Then I migrated 41 memories from Hindsight Cloud to the local backend.

Finally, the EC2 nightly stop/start schedules were disabled and the instance was stopped. Full termination is intentionally delayed for a short soak period, but the live agent moved.

Act 6: what actually runs locally on an old GPU

The GTX 1650 has 4 GB VRAM, but it is also driving the desktop. In practice only about 1 GB was comfortably free once the desktop was running. That budget decides what belongs on the GPU.

Embeddings fit. bge-small is roughly 130 MB, and in this setup it uses around 300 MB on the GPU while idle and closer to 800 MB under load. The reranker also fits. That is the sweet spot: memory indexing, similarity search, and reranking are data-resident workloads where local ownership matters. Keeping those vectors and the index on hardware I control is exactly the win I wanted.

A capable reasoning LLM does not fit. Hindsight’s judgment step — deciding what is durable, what is stale, what should be merged, and what should become a skill — needs model quality. Trying to cram that into the leftover VRAM would be worse than honest cloud use.

So the final design is hybrid. Embeddings, reranking, and the vector store run locally. The narrow reasoning step uses a cheap cloud LLM, currently DeepSeek’s fast model tier. That is not ideological purity, but it is a good engineering trade.

“Local memory” does not have to mean “local everything.” For me it means the data and the index stay mine, and only the part that genuinely benefits from a capable external model leaves the machine.

Gotchas I would repeat to myself next time

First: dry-run dependency-heavy installs. If a memory backend wants to install CUDA on a GPU-less box or downgrade a security-sensitive package in the agent’s own virtualenv, stop.

Second: optional dependencies are still dependencies. A fresh Hermes install can look healthy while missing runtime pieces such as discord.py for the gateway or hindsight-client for the memory tool. A config doctor that says a provider is available is not the same thing as proving the import path works.

Third: package names matter. hindsight-all and hindsight-client are the Vectorize/Hindsight packages for agent memory. hindsight is something else.

Fourth: boring infrastructure constraints dominate the story. ARM versus x86 affects Docker images. Password-gated sudo affects service design. A single disk changes backup and /data assumptions. A small GPU shared with the desktop is not a local-AI fantasy box.

Finally: test the path you think you migrated. An agent can answer correctly from transcript search even when long-term memory did not fire. If you want to prove memory routing, test with an empty context and a question only the new memory backend should know.

The takeaway

A rented cloud box is the right way to discover whether a self-hosted agent deserves to exist. It is cheap, reversible, and avoids turning a curiosity into home-lab plumbing too early.

But once the agent becomes useful, the constraints that made the trial cheap start to cap it. Small ARM CPU. No GPU. Fragile disk layout. Shared virtualenv pressure. SaaS memory for the agent’s most personal data.

Repurposing hardware I already owned changed the shape of the system. The desktop under the desk did not become a frontier-model server, and it did not need to. It became a place to keep the agent’s data-resident workloads close: memory embeddings, reranking, vector search, web-reading infrastructure, and private services reachable only by the VM.

The cloud did not disappear. It got narrower. Instead of renting a whole always-on environment for everything, I now pay external providers for the slice that still needs them: high-quality reasoning and model calls. The rest lives on hardware I control.

That feels like the right lesson from this migration. Self-hosting an agent is not about pretending the cloud has no value. It is about learning which parts of the agent are yours enough, private enough, and repetitive enough that they should stop being rented.

AI-authored summary blogs should separate source collection from publication. Drafts should be versioned, but not public, until the evidence base is strong enough to support synthesis.

Source map

• GitHub Pages provides a low-friction public publishing target.
• Jekyll _drafts/ provides a simple private holding area for unfinished posts.
• Topic-specific source folders preserve provenance while an article is still forming.

Draft

A useful AI blog should not turn every early note into a public article. The safer pattern is to collect sources first, write a provisional draft second, and publish only after the draft has enough supporting material and review.

This site uses three states:

1. sources/<topic>/ for raw source notes and quotes.
2. _drafts/<topic>.md for unpublished synthesis.
3. _posts/YYYY-MM-DD-<topic>.md for final public articles.

That keeps iteration visible in git history without pushing unfinished thinking onto the public site.

The first version of spec-driven development is easy to describe: write down what good looks like before asking people or agents to build it.

That remains true, but it is no longer enough.

As AI-assisted engineering gets more capable, the specification stops being a passive document. It becomes shared infrastructure: a durable place where teams record intent, surface decisions, coordinate human review, route work to agents, detect drift, and learn from implementation.

That is the direction Specledger points toward. It frames SDD as a collaborative platform rather than a folder of documents: requirements, design, implementation, checkpoints, session history, dashboards, deltas, and human decision points all tied back to a shared source of truth.

The evolution is from spec as document to spec as ledger.

The old SDD baseline

In the earlier article, I described spec-driven development as the bridge between research and development.

Research discovers what good looks like. Specifications make that definition inspectable. Development executes against it. Feedback keeps the system honest.

That model is still the foundation. A useful specification turns vague intent into something people can review:

• What problem are we solving?
• What behavior matters?
• What is explicitly out of scope?
• What constraints are non-negotiable?
• What examples prove the work is correct?
• What risks or assumptions still need validation?

This already makes software development better because it moves feedback earlier. It is cheaper to fix a sentence than a production incident.

But AI agents change the pressure on the process.

If a human misreads vague intent, the team may lose a day. If a powerful agent misreads vague intent, it can produce a large, plausible, internally consistent wrong implementation very quickly. Cheap code generation makes the quality of the target more important, not less.

So SDD needs to evolve beyond “write a spec, then implement.” It needs an operating model for collaboration.

The downloaded “Spec Driven Review Process” conversation makes this more concrete. The starting point was a Product Owner describing how much real review work happens before code review: reading a spec for implicit intent, inconsistency, terminology drift, muddled concepts, scope that can be reduced, phase boundaries, roadmap consequences, and decisions that will constrain future work.

That is a useful correction to how people often talk about AI coding. The review surface is not only the pull request. In SDD, the specification itself needs review because it is the thing the implementation will optimize against.

The best short version from that conversation was:

Does this document create a coherent, executable path from intent to delivery while minimizing ambiguity, risk, waste, and future rework?

That question belongs before implementation. It turns spec review into an explicit quality function rather than an informal human habit.

Specledger’s useful reframing

Specledger’s product language is direct: it wants to be the single source of truth for spec-driven development. The important part is not just that it manages specs. The important part is that it treats SDD as a coordination problem.

The platform emphasizes:

• human dashboards for tracking, reviewing, and steering AI collaboration
• spec deltas and checkpoints so changes leave a trail
• session indexing so AI work becomes reusable organizational context
• multi-repo support for features that do not fit neatly inside one repository
• CLI bootstrap and agent compatibility so the workflow can live inside normal engineering tools

That is a stronger framing than “documentation for AI.” Documentation is something you read. A ledger is something teams use to coordinate, audit, and reconcile reality.

In a serious AI-assisted workflow, the core question is not “can the agent write code?” The question is “can the team keep intent, implementation, review, and future memory aligned while the agent writes code?”

Specledger is interesting because it attacks that alignment problem directly.

The command prompts show the workflow shape

The most concrete evidence is in the repositories’ .agents/commands prompts.

The standard Specledger command set looks like a complete SDD lifecycle:

1. /specledger.constitution establishes project principles.
2. /specledger.specify turns a natural language feature description into a structured spec.
3. /specledger.clarify asks targeted questions and writes the answers back into the spec.
4. /specledger.plan turns the spec into architecture, stack choices, phases, and design artifacts.
5. /specledger.tasks creates dependency-ordered implementation work, backed by the sl issue tracker.
6. /specledger.verify checks consistency across spec.md, plan.md, and tasks.md before implementation.
7. /specledger.implement executes the task plan.
8. /specledger.checkpoint performs a critical divergence review between implementation and plan.
9. /specledger.spike gives uncertainty a first-class research workflow.
10. /specledger.checklist creates focused review checklists.
11. /specledger.onboard walks a user through the whole process.

That sequence matters because it is not just a prompt library. It is an attempt to make the implicit engineering loop explicit.

The commands repeatedly encode the same pattern:

• discover the current feature context with sl spec info
• read the generated artifact before editing it
• treat the constitution as authoritative
• preserve handoffs between phases
• make missing context visible instead of hallucinating it
• map requirements to design, tasks, and tests
• verify consistency before implementation
• checkpoint divergence after implementation

This is SDD becoming operational.

Spec review becomes a first-class workflow

The ChatGPT conversation also sketched a review pipeline that fits naturally into Specledger’s world.

The important move is to stop treating “review” as one generic pass. A useful spec review has angles, and each angle looks for different failure modes:

• Product Owner: implicit intent, terminology drift, muddled concepts, downscope opportunities, phase boundaries, stakeholder alignment, and roadmap impact
• QA: testability, acceptance criteria, edge cases, negative paths, undefined success and failure states, and regression risk
• Security: trust boundaries, authentication, authorization, secrets, data exposure, auditability, supply chain, abuse cases, and multi-tenancy
• Architecture: system boundaries, module boundaries, API contracts, data flow, coupling, extensibility, reversibility, migration paths, and technical debt
• Delivery: sequencing, hidden dependencies, team boundaries, critical path, milestones, and risk concentration
• Constitution: whether the proposed work passes, weakly aligns with, or violates the project’s operating principles
• Roadmap: decisions that constrain future roadmap items, scope that should move in or out of the workstream, and sequencing across future work
• Operations, Cost, UX, and Data: production ownership, lifecycle cost, human workflow clarity, and data lifecycle concerns

That list is valuable because it explains why a single reviewer often misses things. A QA reviewer is looking for objective testability. A roadmap reviewer is looking for future constraint. A constitution reviewer is looking for principle violations. A Product Owner is looking for intent clarity and scope control.

The review pipeline from the conversation also had a practical entrypoint: inspect which artifacts actually exist, then ask the user which artifacts and reviewers are in scope. Maybe the filesystem only has spec.md. Maybe it also has plan.md, quickstart.md, a constitution, and a roadmap. Review should still be possible with the available artifacts, but missing artifacts should be treated as missing context rather than automatic failure.

That is exactly the kind of workflow a ledger can preserve. Each finding can be anchored to an artifact location, classified by severity, tied to evidence, assigned a suggested resolution, and turned into a question with a recommended answer when human judgment is required.

The most interesting pattern was borrowed from Matt Pocock’s grill-me skill: ask one question at a time, provide a recommended answer, and inspect the codebase or artifacts instead of asking the user when the answer is already discoverable. For SDD, that becomes a disciplined way to resolve ambiguity without turning review into an endless meeting.

From linear commands to collaborative workflows

The improved prompts in skillrig/cli push the idea further.

The experimental specledger.implement-workflow command intentionally skips the durable issue ledger for a faster path, then launches a deterministic multi-agent implementation workflow. The pipeline is not random fan-out. It is dependency ordered:

• scaffold the public API first
• implement primitives in parallel where files are disjoint
• implement operations once primitives exist
• wire the CLI
• add tests
• verify and repair until checks pass
• synchronize documentation

The prompt is opinionated about how to use agents safely. Every subagent prompt must begin with a SKILLS: line, because the design artifacts say what to build while skills carry how the repository builds things. It also insists on final verification through make check.

That is a useful evolution. A spec alone can tell an agent the goal. A workflow tells the agent system how to divide labor without losing the goal.

The paired specledger.verify-workflow command is even more revealing. It verifies artifacts without tasks.md by sending multiple independent reviewers through the same spec, plan, research, data model, contracts, and quickstart. The prompt explicitly says independent reviewers catch different problems, then merges the findings into one report.

That is a mature SDD pattern:

Do not trust one confident pass. Use independent review to detect drift, stale wording, missing coverage, and contradictions before implementation starts.

The checkpoint-workflow prompt then closes the loop after implementation. It takes an adversarial reviewer stance: assume the implementation has gaps until proven otherwise. It compares actual code and test results against the planned artifacts and classifies divergences.

This is the loop becoming inspectable:

1. specify intent
2. clarify decisions
3. plan implementation
4. run multi-angle spec review
5. resolve high-impact ambiguities one question at a time
6. verify artifacts
7. execute workflow
8. checkpoint divergence
9. update the spec or fix the implementation

A platform for shared SDD workflows

This is where Specledger’s platform angle becomes important.

A local .agents/commands directory can encode a good workflow for one repository. But real SDD is social. Requirements come from users, product, design, engineering, security, QA, operations, and previous implementation history. If the workflow only lives in one agent’s context window, it is fragile.

A shared platform can give teams several things that plain prompt files cannot fully provide:

• a common place to review requirements and decisions
• durable checkpoints that survive chat sessions
• traceability from spec changes to implementation changes
• visibility into which decisions were human-made and which were agent-proposed
• session indexing so prior work becomes searchable context
• multi-repo coordination for features that cross service boundaries
• shared workflow conventions across teams and tools

That is why the phrase “ledger” is useful. A ledger is not just storage. It records changes in a way that can be inspected later.

For AI-assisted development, that is the difference between “the agent did something” and “the team can explain why the system changed.”

The human role moves to decision quality

This also changes the human role.

In a naive agent workflow, the human asks for code, waits, and reviews the result. That is a weak loop because the most important decisions may already be buried inside generated implementation.

In an evolved SDD workflow, humans steer earlier:

• approve or correct requirements
• resolve clarifying questions
• choose which reviewers and artifacts are in scope
• judge findings from Product, QA, Security, Architecture, Delivery, Constitution, Roadmap, SRE, Cost, UX, and Data angles
• review tradeoffs in the plan
• decide when ambiguity is acceptable
• choose which risks need spikes
• inspect verification findings before implementation
• checkpoint divergence after implementation

The agent still executes, but execution is surrounded by decision points.

This matches Specledger’s stated principle: humans steer, AI executes. The value is not that humans micromanage every line of code. The value is that humans keep authority over intent, tradeoffs, and acceptance.

SDD as organizational memory

The next step is memory.

A single spec helps one feature. A ledger of specs, decisions, checkpoints, sessions, and deltas helps the organization learn.

That matters because many engineering failures are not novel. Teams rediscover the same constraints, repeat the same architectural arguments, forget why a tradeoff was chosen, or lose context when a chat session ends.

Specledger’s session indexing and context-compounding language points at this deeper value. If every feature leaves behind a structured trail, future agents and future humans can start from a better place:

• previous decisions are easier to find
• old assumptions can be challenged explicitly
• recurring review failures can become checklist items
• stable implementation patterns can become skills
• cross-repo dependencies can be made visible instead of tribal

The spec becomes more than a pre-code artifact. It becomes part of the team’s long-term memory.

The tension: speed versus durability

The skillrig/cli workflow prompts also expose a healthy tension.

The experimental implementation workflow says it skips the durable sl issue ledger because the quickstart is intentionally smaller. That is a real tradeoff. Sometimes a team wants the full traceable workflow. Sometimes it wants a faster, bounded, deterministic workflow that still reads the design artifacts and gates on checks.

This is probably where SDD will keep evolving.

Not every feature needs the same amount of ceremony. A tiny bug fix does not need the same ledger as a multi-repo payment integration. But every workflow still needs a way to preserve the right amount of intent, verification, and review.

The mature version of SDD is not maximum documentation. It is calibrated traceability.

Use more ledger when the risk, ambiguity, or coordination cost is high. Use lighter workflows when the target is already clear. But do not remove the feedback loop.

Why this matters for AI engineering

AI makes implementation faster, but it does not make intent obvious.

That creates a new bottleneck:

The scarce resource is not generated code. The scarce resource is shared, inspectable, correct intent.

Spec-driven development began as a way to make intent explicit. Platforms like Specledger suggest the next stage: make intent collaborative, traceable, reviewable, executable, and memorable.

The practical shape is emerging:

• specs define behavior
• plans connect behavior to architecture
• tasks or workflows divide execution
• verification checks alignment before code
• checkpointing checks divergence after code
• dashboards and deltas keep humans in the steering loop
• session indexes and skills let context compound

That is how SDD evolves from a writing habit into an engineering system.

Conclusion

The future of spec-driven development is not simply better prompts or longer requirements documents.

It is shared workflow infrastructure.

Specledger is interesting because it treats SDD as a collaborative ledger of intent: a place where humans, AI agents, specs, plans, issues, checkpoints, reviews, and sessions can stay aligned.

That is the right direction for agentic software development. The more capable the agents become, the more important it is to know what they are supposed to optimize for, who approved the tradeoffs, how divergence is detected, and what the team learns from each loop.

The spec is no longer just where clarity lives.

It is where collaboration, control, and memory begin.

The reason it’s worth the effort isn’t just data ownership. It’s that a self-hosted agent can do things a subscription assistant won’t. The agent this is written for — a self-hosted Hermes — doesn’t just read a pull request and opine. It checks out the branch, installs dependencies, runs the tests, and exercises real scenarios before it comments. That’s well beyond what a read-only “sandbox” memory provider is willing to give you.

Security disclaimer. Running untrusted code from PR branches is powerful and dangerous. Doing it safely needs strict mechanisms — allowlists for trusted authors, scrubbing of untrusted inputs, and tight action boundaries — so a hostile PR can’t turn your agent into a foothold. We’ll cover that hardening in a future article. Until then: only point this kind of automation at repositories and authors you control.

But before you migrate anything in, you need to know what you’re migrating into. The first problem in migrating memories out of ChatGPT is not extraction — it’s classification, and that only makes sense once you’ve chosen a memory architecture. So we’ll work in that order: the substrate, how Hermes memory actually works, the pitfalls we hit standing it up, and then — in the appendix — the validated pipeline for bringing your ChatGPT memories across.

First question: what substrate do you want the agent to trust?

A memory export is a mixture of preferences, project summaries, stale corrections, private facts, temporary commitments, and instructions that made sense in one product but are dangerous in another. Paste that blob straight in and you don’t get continuity — you get memory debt. The starting question isn’t “how do I copy memories?” It’s what kind of substrate do I want the agent to trust?

For Hermes-style agents, four substrates are worth comparing before importing anything:

A few notes from evaluating each:

• Obsidian is excellent when the human wants a personal knowledge base — it’s a notebook first, an agent backend second. Its strength is reviewability: drop imported memories into folders, backlink and tag them, decide what deserves promotion. We treat it as a path worth evaluating but didn’t explore here — a great human-review staging area if you want one, orthogonal to the agent’s live recall.
• Honcho is almost the opposite — built for agents that need statefulness: it stores messages and events, reasons over them in the background, builds representations of users and agents, and returns prompt-ready context. Powerful, because the agent doesn’t manually maintain every fact. We researched self-hosting it seriously, and the catch on a small box is real: it’s several long-running services (API, a background “deriver”, Postgres+pgvector, Redis) and the deriver must call out to an LLM/embedding provider, so it’s neither free nor fully local unless you accept a trusted external endpoint. Strong as a reasoning layer; heavy as the only layer.
• Local filesystem memory is the simplest and most auditable substrate. OpenClaw’s docs make it explicit: the agent remembers by writing plain Markdown; the model only “remembers” what’s on disk. Hermes has the same spirit. The advantage is transparency; the danger is entropy.
• Owned Postgres / pgvector is the pragmatic fourth option if you already run the infrastructure — a Firecrawl compose stack with Postgres gives you most of an owned semantic-search layer if the image has (or can add) pgvector. But “nearest-neighbour search” is retrieval, not memory judgment: a table can find similar memories; it can’t decide which preference is global, which note is stale, or which lesson should become a skill.

The lesson is to layer them, not collapse them: source notes/Obsidian for raw imported material and human review; local files for curated, durable, inspectable facts; a reasoning/semantic provider (Honcho, Hindsight) for inferred recall; owned Postgres/pgvector when you need local retrieval without surrendering the data plane. Keep the export path under your control throughout.

How Hermes agent memory actually works

Concretely, Hermes starts with a built-in memory that is just plain Markdown: a MEMORY.md of durable facts and a USER.md profile, always loaded, transparent, version-controllable — the model only “remembers” what’s literally on disk. On top of that it supports a pluggable external provider for semantic recall, in a layered model: the files stay the inspectable source of truth; the provider adds embedding search and synthesis and is swappable.

That’s the design. Standing it up taught us where the sharp edges are.

The pitfalls we hit standing it up

1. The built-in file memory has a silent ceiling

Built-in memory has a character cap (ours was 2,200). One day the agent quietly started refusing to store new memories — Memory at 2,113/2,200 chars; adding this entry would exceed the limit. It didn’t fail loudly; it just stopped learning. A flat growing file also accumulates entropy (duplicates, contradictions) and offers no semantic recall (it finds only what it wrote verbatim and can grep for). Great as a source of truth; you want a semantic layer on top.

2. “Local embedded” backends can try to reshape your runtime

Hindsight has a tempting local-embedded mode: an on-instance daemon with built-in Postgres that auto-stops when idle, local embeddings, your own LLM key. Lovely on paper. But a dry-run of the install resolved to 159 packages including the full CUDA toolkit (useless on a GPU-less ARM box) and wanted to downgrade cryptography inside the agent’s own virtualenv. Lesson: --dry-run any backend install and read what it touches. A memory layer should never get to mutate your agent’s dependencies.

3. “Just run mem0 locally” is not a flag you flip

mem0 is the obvious open-source choice, and Hermes ships a mem0 plugin — but the stock plugin is Mem0-Cloud only: it instantiates the hosted client and needs a Mem0 API key. Self-hosted/OSS mode is an open, unmerged feature request, not shipped. “Local mem0” means writing a custom provider, not selecting one.

4. A cheap LLM key is not an embeddings key

We wanted everything on one inexpensive DeepSeek key. Worth knowing: DeepSeek’s API has no embeddings endpoint — chat completions only. Any vector backend still needs a separate embedding model. The clean answer is a small local embedder (bge-small, 384-dim, ~100 MB, no key) — another resident process on a RAM-tight box.

5. Ladybug and Holographic — local options worth knowing

Two local providers are worth a mention even though we didn’t adopt them. Ladybug is a community plugin that backs memory with an embedded graph database (a fork of Kuzu), giving memories a typed, linked model — preferences, facts, projects, people, events — with importance scores and named edges, and needing no API key. That data model is more expressive than a flat vector store, and it’s a genuinely promising project. But it’s young — only about two months old, a couple of commits, effectively a single maintainer — and it loads its stack (graph DB + ONNX embeddings) in-process and resident, with no idle release. One to watch and experiment with, not yet to depend on for a daily driver. Holographic, by contrast, is Hermes’ zero-dependency escape hatch: pure-Python Holographic Reduced Representations over SQLite — no LLM, no embeddings, no network, no keys. Tiny and fully local; the trade-off is that recall is algebraic, not semantic.

6. Mind the auxiliary-LLM layer — it’s where memory quietly breaks

The pitfall most people miss. Your agent doesn’t make one LLM call per turn; it makes many, most not to your main model: context compression, title generation, triage, profile description, kanban decomposition, and — crucially — memory itself (extracting facts to retain, reasoning over a recall) are model calls separate from the main turn.

Two things bit us. First, these auxiliaries defaulted to providers we had no credit or auth for, so they failed — and compression was set to fail open, meaning when the summarizer errored it proceeded without a summary and silently dropped context the agent needed. Memory loss that looks like the model “forgetting” is often a broken auxiliary. We repointed every auxiliary to one funded, cheap model. Second, in cloud memory the retain/recall calls are metered, so the same levers that control RAM locally now control your bill: retain less often, lean recall budget. Treat the auxiliary layer as first-class: configure it, fund it, pick a cheap model on purpose.

7. For a small box, cloud memory is a reasonable default — with an escape hatch

Stack the constraints — 8 GB, no GPU, disk pressure, one cheap LLM key — and a cloud memory service is the better engineering choice for this box; the agent keeps only a light HTTP client. mem0 Cloud (free Hobby) caps at ~1,000 retrievals/month, which an always-on agent burns in days; Hindsight Cloud is usage-based with no request wall, tunable via retain/recall frequency. We chose Hindsight Cloud, keeping built-in MEMORY.md as the always-on source of truth. The one rule: pick a provider with a real export path so cloud stays low-regret — which is also what makes the appendix’s migration work in reverse.

8. A task board is working memory, too

The most useful thing we learned wasn’t about the store. During parallel work — several Discord threads open at once — the agent kept “forgetting” a task, then lost it after a compression. Two facts: each Discord thread is its own isolated session (they don’t share live context), and compression can summarize an in-flight instruction away. The fix isn’t more memory tokens — it’s a shared task board. Hermes’ SQLite-backed kanban is durable across every session, thread, and scheduled job, and survives compression. We wired our PR-reviewer to record its work there: the script deterministically creates one task per PR (idempotent — re-reviews append rather than duplicate), the agent only comments progress, and the script owns closing/blocking it, so a mid-review compression can’t orphan it. The task board as first-class working memory is a direction worth exploring much further.

Takeaways

• Choose a substrate deliberately and layer them; keep the built-in Markdown files as the inspectable source of truth and watch the size cap — it fails silently.
• --dry-run any backend install. On a small/ARM/no-GPU box, “local embedded” can mean CUDA and a runtime-mutating dependency set.
• A cheap LLM key ≠ embeddings.
• Configure and fund the auxiliary-LLM layer deliberately — compression and memory extraction are where context quietly disappears.
• For constrained boxes, cloud memory is fine if it has an export path.
• Don’t put durable, multi-step work in the chat; a shared task board is the cross-session working memory that holds up.

The store is half the system. The other half is making sure the agent doesn’t lose the thread — and that you can always take your memory with you.

Appendix: Migrating your ChatGPT memories in

Now that there’s a system to migrate into — built-in files for durable facts, a semantic provider on top, source notes/Obsidian for raw material, skills for procedures, a task board for working memory — here’s the validated pipeline for bringing your ChatGPT context across. Remember the framing: the export is raw source material, not memory. The work is classification.

Two exports, two purposes

There are two useful ways to get information out of ChatGPT, and they solve different problems.

1. OpenAI’s official data export (archival). Per OpenAI’s Help Center, go to Settings → Data Controls → Export Data, confirm, and download the emailed ZIP (the link expires; exports can take time). You can also request data through the Privacy Portal. This is good for archival analysis but too large and noisy to import directly — keep it as a private audit trail.

2. A prompt-based memory export (the import). Anthropic’s Claude import page describes the lightweight pattern: copy a prompt into the old provider, ask it to extract the important context, paste the result in. This doesn’t migrate every transcript — it migrates the distilled profile: preferences, instructions, projects, work style, recurring context. For Hermes, the same idea applies, but the output should be classified before it touches memory.

Here is a migration prompt adapted from the Claude-style workflow, tuned for agent memory systems rather than one destination product. The validation in it — verbatim preservation, the uncertain-source label, the strict one-category rule, the per-entry destination + confidence, and the “state whether more remain” loop — is what keeps the import honest:

Export all of my stored memories and any durable context you have learned about me from past conversations. Preserve my words verbatim where possible, especially for instructions, preferences, corrections, and standing rules.

Only include information that appears in stored memory or durable cross-chat context. Do not invent facts from this conversation. If you are unsure whether something is stored memory or inferred from chat history, label it as uncertain.

Classify each item into exactly one category:

1. Instructions: explicit rules I asked you to follow in future conversations, including tone, format, style, approvals, safety boundaries, and corrections to your behavior.
2. Identity: stable personal facts such as name, location, languages, interests, family, education, or public biographical context.
3. Career: current and past roles, organizations, skills, domains, and professional responsibilities.
4. Projects: ongoing or meaningful projects. Use one entry per project with project name, purpose, status, important decisions, and known repository or workspace if available.
5. Preferences: broad working style, tool choices, writing preferences, learning preferences, and taste.
6. Environment: durable machine, account, repository, deployment, or toolchain facts that future agents may need.
7. Procedures: reusable workflows, troubleshooting steps, or lessons learned that should become skills rather than ordinary memory.
8. Temporary or expiring context: reminders, deadlines, one-off tasks, phase status, pending approvals, or anything likely to become stale.
9. Contradictions and uncertainty: entries that conflict, appear outdated, or need human review.

For each entry, output:
- category
- date if known, otherwise [unknown]
- source confidence: high / medium / low
- exact memory text or closest faithful wording
- recommended destination: user profile, agent memory, skill, project note, scheduled task, archive, or discard
- reason for the recommendation

Wrap the entire export in a single Markdown code block. After the code block, state whether more memories remain.

If it says more remain, continue until the export is complete. Then do not paste it straight into the agent — put it in a source folder first.

A classification scheme for the import

Sort each item by how it will be used:

• User profile — stable facts that shape interaction style (name, role, “prefers concise technical answers”). Compact and stable; these affect the agent across every session.
• Agent memory — durable environment/project facts (“this repo’s test command is X”, “the blog uses Jekyll drafts before publishing”). Not temporary progress or stale artifact IDs.
• Skills — procedures don’t belong in ordinary memory. “When debugging this pipeline, run these five commands in order” becomes a skill: triggers, exact steps, pitfalls, verification. Procedural memory is operational — a bad procedure makes the agent repeat the same failure forever.
• Project notes / Obsidian — background too detailed for always-loaded memory but worth searching sometimes. If the agent may need to search it occasionally but shouldn’t read it every session, it belongs in notes, not memory.
• Scheduled tasks / commitments — “follow up after the interview tomorrow” is a scheduled task, not a permanent memory. OpenClaw’s docs draw the same line: commitments differ from durable facts.
• Discard / archive — stale phase status, old PR numbers and commit hashes, “currently working on…”, temporary approvals, contradicted preferences, sourceless facts. A useful import is the smallest set that improves future behavior, not the largest.

The pipeline

1. Export raw account data (official export) — audit trail, stored privately.
2. Run the structured export prompt — continue until no memories remain.
3. Save the export as source material — a source folder or Obsidian note; don’t inject it yet.
4. Normalize each entry — assign scope (global user / project / environment / procedure / temporary / archive), durability (permanent / long-lived / short-lived / expired), confidence (high / medium / low), source (explicit / inferred / unknown), destination.
5. Promote only high-confidence durable entries — compact facts → user profile / agent memory; reusable workflows → skills; detail → notes; stale → discard.
6. Run an initial consolidation/review pass — which entries contradict, which are too temporary, which instructions are unsafe without action boundaries (see the disclaimer up top), which should become skills, which need human review.
7. Schedule recurring consolidation (“dreaming”) — memory cleanup is routine, not one-time. Run it asynchronously, off the critical path; don’t overwrite the source; produce a reviewable diff (additions, removals, merges) with evidence; prefer recency only with evidence; and separate cleanup from promotion (a discovered lesson should still clear a confidence threshold or human review). A dream that can’t cite why it wants to delete or rewrite a memory shouldn’t be allowed to mutate the store.

The target is a memory system that is transparent (you can see what the agent believes), scoped (project memories don’t bleed across work), action-aware (approvals and risky instructions carry boundaries), and self-cleaning. Migrating memories isn’t a one-time copy-paste; it’s the start of a memory operating system you own.

Side note: sharing a single ChatGPT conversation with your agent

The memory-export prompt above is for durable profile context. Sometimes you want a smaller move: take one useful ChatGPT conversation and hand it to your local agent as source material. Public ChatGPT share links are good for this because they can be fetched without giving the agent your ChatGPT account.

A practical setup is to install a public-share exporter such as csctf (chat_shared_conversation_to_file) and give Hermes a small procedural skill for it. Then the handoff becomes simple:

fetch this ChatGPT share into /data: https://chatgpt.com/share/...

The agent should download the share as Markdown and HTML, put it in a data mount or other agreed source folder, spot-check the result, and report the paths. That turns a one-off conversation into inspectable source material that can be summarized, linked from source notes, or promoted into a real Hermes skill later.

Here is the reusable Hermes skill, with machine-specific installation paths removed:

---
name: chatgpt-share-export
description: Use when the user asks to fetch, download, export, or archive a public ChatGPT shared conversation URL into a data directory using csctf.
version: 1.0.0
author: Hermes Agent
license: MIT
metadata:
  hermes:
    tags: [chatgpt, export, shared-conversations, csctf, markdown, archive]
    related_skills: [hermes-agent]
---

# ChatGPT Share Export

## Overview

Use this skill to turn a public ChatGPT share URL such as `https://chatgpt.com/share/...` or `https://chat.openai.com/share/...` into local Markdown and HTML files under an agreed data or source-material directory.

The preferred tool is `csctf`, installed as a local CLI from `chat_shared_conversation_to_file`. Use a system Chromium or browser available on the host if Playwright's bundled Chromium cannot be installed for the platform.

## When to Use

Use when the user says things like:

- "fetch this ChatGPT share into `/data`"
- "download this ChatGPT shared conversation"
- "export this share URL as markdown/html"
- "archive this ChatGPT conversation"
- "run csctf on this ChatGPT share link"

Also applicable to csctf-supported public AI share links from Claude, Gemini, and Grok, but the primary expected trigger is ChatGPT share URLs.

Do not use this for private ChatGPT conversations that are not public share URLs. csctf fetches public share pages; it is not an authenticated ChatGPT account exporter.

## Default Output Location

Use the user's requested data directory by default, commonly:

```bash
/data/chatgpt-shares
```

For a URL with ID `<share-id>`, use this base filename pattern:

```text
<data-dir>/chatgpt-shares/chatgpt-share-<share-id>
```

Expected outputs:

```text
<data-dir>/chatgpt-shares/chatgpt-share-<share-id>.md
<data-dir>/chatgpt-shares/chatgpt-share-<share-id>.html
```

If the user asks for a different directory or filename, respect that.

## Quick Recipe

1. Extract the share ID from the URL.
2. Ensure the output directory exists.
3. Run csctf with an explicit `--outfile` base path.
4. Spot-check the generated Markdown and HTML.
5. Report the absolute paths and basic verification results.

Command template:

```bash
set -euo pipefail
export BUN_INSTALL="$HOME/.bun"
export PATH="$BUN_INSTALL/bin:$PATH"

URL='https://chatgpt.com/share/<share-id>'
ID='<share-id>'
OUTDIR='<data-dir>/chatgpt-shares'
BASE="$OUTDIR/chatgpt-share-$ID"

mkdir -p "$OUTDIR"
csctf "$URL" --timeout-ms 90000 --outfile "$BASE"
```

If `csctf` is not on `PATH`, use the local binary path configured on that machine.

## Verification / Spot Check

After running, verify both files exist and are non-empty:

```bash
MD="$BASE.md"
HTML="$BASE.html"
stat -c '%n %s bytes' "$MD" "$HTML"
wc -l -w -c "$MD" "$HTML"
```

Then inspect the beginning of the Markdown for title, source URL, retrieval timestamp, and role sections:

```bash
python3 - <<'PY'
from pathlib import Path
p = Path('<data-dir>/chatgpt-shares/chatgpt-share-<share-id>.md')
text = p.read_text(errors='replace')
for line in text.splitlines()[:80]:
    if line.strip():
        print(line[:220])
PY
```

Also confirm the HTML title/content references the same conversation:

```bash
python3 - <<'PY'
from pathlib import Path
p = Path('<data-dir>/chatgpt-shares/chatgpt-share-<share-id>.html')
text = p.read_text(errors='replace')
for needle in ['<title>', 'ChatGPT Conversation', 'Source:']:
    print(needle, needle in text)
PY
```

If the user asks for a deeper check, read targeted portions of the Markdown with `read_file`, or search for expected phrases using `search_files`.

If Playwright's browser download is unsupported on the host, configure csctf to use an available system Chrome/Chromium installation.

## Common Pitfalls

1. **Playwright Chromium install can fail on some Linux/ARM combinations.** Use `PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1 bun install` and rely on a system Chrome/Chromium where needed.

2. **Forgetting explicit `--outfile`.** Without it, csctf writes to the current directory with an inferred filename. Use an explicit base path under the data/source directory so the user gets a stable, easy-to-find artifact.

3. **Stopping after the CLI says success.** Always spot-check file sizes, line counts, and the first Markdown content before reporting success.

4. **Assuming this works for private conversations.** It only works on public share URLs; private account history needs a different export path.

5. **Treating the export as memory immediately.** A conversation transcript is source material. Summaries, preferences, and procedures still need classification before promotion into memory or skills.

## Reporting Format

When done, tell the user:

- Markdown path
- HTML path
- file sizes / line counts
- detected title
- brief spot-check result

Example:

```text
Exported and spot-checked:

- <data-dir>/chatgpt-shares/chatgpt-share-<id>.md
- <data-dir>/chatgpt-shares/chatgpt-share-<id>.html

Title detected: <title>
Markdown: <bytes> bytes, <lines> lines
HTML: <bytes> bytes, <lines> lines
```

## Verification Checklist

- [ ] URL is a public share URL.
- [ ] Output directory exists under the requested data/source directory.
- [ ] csctf completed with exit code 0.
- [ ] Markdown and HTML files exist and are non-empty.
- [ ] Markdown contains title, source URL, and role sections.
- [ ] HTML contains the expected title or conversation heading.
- [ ] Final response includes absolute paths.

That last pitfall is the important memory-system point: sharing a ChatGPT conversation with an agent is not the same as making it memory. The export is evidence. The agent still has to decide whether the conversation contains a durable user preference, a project note, a reusable procedure, a temporary task, or nothing worth promoting.

Terraform’s framework is a mature provider-development layer around a versioned protocol. It sits between provider authors and Terraform Core, giving Go interfaces for providers, resources, data sources, functions, schemas, plan modification, state upgrades, and provider servers. The framework’s job is to make the Terraform Plugin Protocol usable without requiring every provider author to hand-roll gRPC protocol implementations.

formae’s SDK, as described in the Platform Engineering article, the Ergo primer copied into the same gist, and the public docs/repository, is newer and more opinionated about orchestration. The plugin is not just a provider binary answering Terraform Core’s RPCs. It is a resource actor in a larger agent system. The formae article opens with a useful division of responsibility: the agent knows how to schedule, queue, order, rate-limit, execute, and retry plugin operations; the resource plugin owns the actual API interaction. The plugin author’s public surface is deliberately small: implement resource CRUD, status, list, rate limits, discovery filters, and label configuration; put resource schemas in Pkl; ship a manifest.

That makes the comparison less like “which SDK is better?” and more like this:

Terraform optimizes for a stable, ecosystem-scale contract between Terraform Core, provider binaries, the Registry, and Terraform configuration. formae optimizes for an agent-managed resource lifecycle where plugin schema, discovery, async progress, and reconciliation are first-class platform concerns.

Both are plugin systems. They solve different parts of the infrastructure problem.

Two plugin boundaries

Terraform’s boundary is protocol-first. The Terraform Plugin Protocol is a versioned interface between Terraform CLI and provider plugins, implemented with Protocol Buffers and gRPC. The framework supports protocol versions 5 and 6; provider servers wrap a provider.Provider and expose a protocol-specific server implementation. In the cloned repository, providerserver.NewProtocol5, NewProtocol6, and Serve adapt a framework provider into tfprotov5 or tfprotov6 servers.

That protocol boundary matters because Terraform has an enormous provider ecosystem. Compatibility is not only a technical concern; it is a distribution model. Terraform Registry discovery, CLI compatibility, provider release versioning, and migration from SDKv2 all depend on the idea that a provider is a separately distributed executable speaking a known protocol.

formae’s boundary is agent-and-actor-first, and the copied article makes clear why that boundary exists. The first implementation loaded Go native plugins in-process with the standard plugin package: compile a shared object, export a Plugin symbol, plugin.Open it, assert it implements ResourcePlugin, then call it directly. That was elegant inside a monorepo, but not viable for a public SDK. Go native plugins require host and plugin to match on Go toolchain, shared package versions, build flags, and transitive dependencies. For an infrastructure plugin ecosystem wrapping many cloud SDKs, that is dependency lockstep disguised as extensibility.

So formae kept the tiny ResourcePlugin interface but moved the execution boundary. The public documentation now says plugins run as separate processes or remote services and communicate with formae through documented public interfaces, not by being linked into formae. The repo’s pkg/plugin/README.md adds the implementation detail: the external-plugin entry point calls sdk.RunWithManifest, which reads formae-plugin.pkl, extracts schemas from schema/pkl/PklProject, wraps the user’s ResourcePlugin, starts an Ergo node, and announces capabilities to the agent. The PluginActor builds an announcement with supported resources, resource schemas, match filters, label config, namespace, version, and max request rate, then sends it to the agent’s PluginCoordinator.

That history sharpens the Terraform comparison. Terraform and formae both end up with separately distributed binaries, but for different reasons. Terraform’s separation is about an ecosystem-scale protocol contract between Core, providers, and the Registry. formae’s separation is about preserving a small Go authoring surface while escaping Go shared-library lockstep, isolating plugin failures, decoupling licenses, and giving the agent freedom to place plugin actors locally or remotely.

The Ergo primer in the second gist explains the last point. Ergo brings Erlang/OTP-style actors to Go: nodes host lightweight processes, actors communicate by sending messages to PIDs, and the caller does not need to care whether the recipient is on the same node, a separate plugin process, or a remote satellite. The formae article calls this network transparency. In tests, the PluginOperator can run locally on the agent’s node. In the OSS agent, it can run as a local persistent plugin process. At larger scale, it can move behind satellite agents without changing the agent’s mental model of “send a message to the operator.”

So Terraform treats the plugin boundary as a versioned RPC surface that Terraform Core controls. formae treats it as a process/actor boundary where the plugin announces capabilities and the agent orchestrates resource work.

That is the first architectural trade-off:

Interfaces: broad Terraform concepts vs. one resource plugin contract

Terraform’s framework decomposes provider development into many concepts. The core provider.Provider interface in the cloned repo requires Metadata, Schema, Configure, DataSources, and Resources. Optional interfaces add functions, ephemeral resources, list resources, actions, state stores, config validators, meta schemas, and more. The resource.Resource interface requires Metadata, Schema, Create, Read, Update, and Delete, while optional interfaces add import, configuration, validation, plan modification, state moves, state upgrades, identity, and identity upgrades.

This is a very Terraform-shaped design. It reflects the fact that Terraform is not only a CRUD engine. It has data sources, functions, plan-time semantics, unknown values, import, moved blocks, state schema upgrades, provider-defined functions, and emerging concepts such as ephemeral resources and actions. The framework exposes all of that surface because provider authors sometimes need to participate in all of it.

formae’s public ResourcePlugin is intentionally narrower:

type ResourcePlugin interface {
    RateLimit() RateLimitConfig
    DiscoveryFilters() []MatchFilter
    LabelConfig() LabelConfig

    Create(ctx, req) (*CreateResult, error)
    Read(ctx, req) (*ReadResult, error)
    Update(ctx, req) (*UpdateResult, error)
    Delete(ctx, req) (*DeleteResult, error)
    Status(ctx, req) (*StatusResult, error)
    List(ctx, req) (*ListResult, error)
}

The repo comments say plugin identity and schema methods are handled automatically by the SDK. The internal FullResourcePlugin adds Name, Version, Namespace, SupportedResources, and SchemaForResourceType, but plugin authors do not implement those directly. The SDK derives identity from the manifest and schemas from the Pkl package.

That is a strong opinion: resource plugins should stay focused on remote-system operations. Schema declaration, metadata, wrapping, startup, and announcement are SDK responsibilities. The formae article also gives a pragmatic reason for keeping the surface this small: AI coding agents write much of the plugin code, so reliability comes from a narrow contract, LLM-oriented documentation, and conformance tests that turn “did it get it right?” into a runnable answer.

Terraform asks provider authors to model Terraform semantics directly. formae asks plugin authors to implement infrastructure operations and lets the agent/SDK attach those operations to the platform model. That makes Terraform’s framework more expressive for Terraform-native behavior, while formae’s SDK is easier to generate, test, and supervise as a platform operation worker.

Schema: Go framework types vs. Pkl as the declarative source of truth

Schema is where the two systems diverge most sharply.

Terraform’s framework expresses schemas in Go. A resource returns a resource/schema.Schema with Attributes, Blocks, descriptions, deprecation messages, and a Version. A string attribute is a Go struct with fields such as Required, Optional, Computed, Sensitive, Validators, PlanModifiers, Default, and WriteOnly. These are not incidental flags. They encode Terraform’s contract with the practitioner and with state: required vs optional input, computed output, sensitive display, plan-time transformations, validation, deprecation, write-only behavior, and compatibility with specific Terraform versions.

The framework also has a rich value model. The cloned types/basetypes/string_value.go shows StringValue carrying an explicit state: known, null, or unknown. That is one of the framework’s headline improvements over SDKv2. Provider code can distinguish “the user set null” from “Terraform does not know the value yet” from “this is a known string.” In Terraform, that distinction is central to planning.

formae’s schema source of truth is Pkl. The docs show a resource class annotated with @formae.ResourceHint and fields annotated with @formae.FieldHint. A resource type such as SFTP::Files::File declares an identifier JSONPath like $.path; fields can be mutable or createOnly, where changing them requires replacement. The repo README says schema/pkl/PklProject describes supported resource types, fields, validation rules, create-only fields, discoverability, extractability, and parent-resource mappings. At startup, pkg/plugin/descriptors extracts resource descriptors and JSON schemas from the Pkl package.

This makes formae’s schema story more language-oriented. Pkl is not just metadata stapled onto Go structs; it is the declarative contract that the agent can evaluate, verify, extract, and use for resource discovery/reconciliation. The code in descriptors/extract_schema.go stages plugin dependencies, rewrites @formae references so plugin schemas resolve against the agent’s formae schema version, generates a wrapper Pkl project, resolves dependencies, and runs an extractor to produce resource descriptors.

That dependency rewriting is a subtle but important schema-evolution mechanism. It avoids every plugin having to update its own PklProject pin in lockstep whenever the agent’s base schema adds a feature. Terraform’s framework handles schema evolution primarily inside provider code and protocol compatibility. formae is using a package/module schema language and an agent-controlled extraction pass to keep plugin schema interpretation aligned with the running platform.

State and evolution

Terraform has one of the hardest schema-evolution problems in infrastructure software: it must preserve user state across provider upgrades, Terraform upgrades, schema changes, moved resources, imports, and drift refreshes. The framework exposes that reality directly.

A Terraform resource schema has a Version. The ResourceWithUpgradeState interface lets a resource return a map from prior state versions to StateUpgrader implementations. The comments are explicit: Terraform does not store previous schema information, so breaking changes to state data types must be handled by providers. There is also ResourceWithMoveState for moved configuration blocks that change resource type, and identity schema/identity upgrade interfaces for newer resource identity support.

That is a powerful but demanding model. Terraform gives provider authors tools to precisely control state compatibility, but provider authors must think in Terraform state terms.

formae’s schema evolution story appears more centralized. The article’s title foregrounds schema evolution, and the repo shows the mechanism behind that framing: schemas are Pkl packages, the SDK extracts descriptors at startup, the manifest declares minFormaeVersion, and the extractor rewrites plugin dependencies against the agent’s formae schema. The docs also emphasize FieldHint semantics like createOnly, identifier extraction, discoverability, extractability, and parent-child resource relationships. These are schema-level concepts the agent can reason about before or around plugin calls, instead of asking each plugin method to rediscover the platform contract.

The trade-off is maturity vs. centralization. Terraform has years of battle-tested state migration machinery exposed through framework interfaces. formae can make schema evolution feel more like updating a declarative package interpreted by the agent, but that also means its evolution guarantees depend on how consistently the agent, Pkl schema library, descriptor extraction, and plugins are versioned together.

Planning and reconciliation

Terraform is famous for planning. Its plugin framework is built around Terraform Core’s plan/apply model. Provider authors can validate configuration, modify plans, mark replacement requirements, preserve unknowns, and upgrade state. The framework’s ResourceWithModifyPlan comments show the constraints: config values must be preserved, known planned values cannot later be changed inconsistently, unknown values may remain unknown or be filled with appropriate values, and errors prevent further plan modification.

In other words, Terraform’s provider framework gives plugins a seat at the planning table, but Terraform Core owns the table.

formae, by contrast, centers the agent’s reconciliation loop. The copied article says the formae agent knows how to schedule, queue, order, rate-limit, execute, and retry plugin operations; the resource plugin owns the actual API interaction. The plugin docs say the agent discovers installed plugins, spawns them, receives capabilities, routes operations, and enforces rate limits. The conformance test suite validates create, read, update, replace, destroy, extract, discover, forced sync, and out-of-band deletion. That is not just CRUD testing; it is lifecycle testing through the agent and CLI.

This is an important distinction for platform teams:

• Terraform’s unit of work is a plan/apply transaction against state.
• formae’s unit of work is closer to an agent-managed operation and inventory reconciliation loop.

Terraform practitioners expect a plan to explain what will happen before apply. formae users may expect the agent to continuously know what exists, discover unmanaged resources, detect drift, queue long-running operations, and reconcile desired state with inventory.

Neither model is universally superior. Terraform’s plan is still the gold standard for human-reviewed infrastructure change. Agent reconciliation is attractive when infrastructure systems have asynchronous operations, rate limits, discovery needs, and drift behavior that do not fit cleanly into a single human-approved transaction.

Async operations and rate limits

Terraform providers can of course wait for cloud operations, retry, and poll. But the framework’s public resource interface still reads as synchronous CRUD from Terraform Core’s perspective: Create, Read, Update, and Delete mutate response objects and diagnostics during an RPC. Provider authors usually implement waiting inside those methods or with helper libraries.

formae bakes async progress into the public plugin interface. CRUD results embed a ProgressResult; long-running cloud operations return InProgress with a NativeID and tracking metadata. The agent then calls Status on a polling schedule until the operation reaches success or failure. The SDK also classifies recoverable errors such as throttling, network failure, service internal error, timeout, not stabilized, not found, and resource conflict.

This is one of formae’s clearest advantages for cloud-provider ergonomics. Many real infrastructure APIs are not synchronous: create starts a job, update triggers stabilization, delete returns before eventual removal, discovery is paginated, and rate limits vary by namespace or service. If the platform’s agent owns polling, rate limits, and retries, plugin code can be thinner and more consistent.

Terraform can model the same behavior, but the provider often owns more of the operational loop. formae pushes more of that loop into the platform: scheduling, ordering, rate limiting, retry classification, and status polling become agent concerns rather than ad hoc code inside every plugin.

Discovery and inventory

Terraform has data sources and import, but it is not primarily an inventory discovery engine. Terraform state tracks resources Terraform manages; unmanaged resource discovery is usually handled by import workflows, external tools, or provider-specific data sources.

formae makes discovery part of the plugin contract. List is a required method. DiscoveryFilters and LabelConfig are part of the public interface. The Pkl schema can mark resource types as discoverable and extractable. The conformance tests create resources out-of-band, register targets, trigger discovery scans, and assert that resources appear on an $unmanaged stack.

That makes formae more natural for platform teams that want to answer, “What already exists? What can we bring under management? What drifted?” Terraform can answer those questions in pieces, but its core mental model begins with declared configuration and state. formae’s model seems to put inventory and discovery closer to the center.

Testing philosophy

Terraform’s framework ecosystem emphasizes unit tests, acceptance tests, migration tests, and provider behavior compatibility. The migration guide recommends test-driven development when moving from SDKv2 to the framework, especially because user Terraform configuration should usually not change during migration.

formae’s conformance tests are more platform-integrated. The plugin scaffold wires in tests that run through the real formae CLI and agent. The suite validates not only CRUD but extract, discover, forced sync idempotency, replacement behavior, destroy, and out-of-band deletion. That is a different testing posture: not “does this provider method behave?” but “does this plugin behave correctly inside the platform’s lifecycle?”

This also connects back to the article’s AI-agent point. If coding agents generate most plugin code, tests need to be more than examples for humans; they need to be executable feedback loops. Terraform’s framework relies on provider authors understanding Terraform semantics and building the right test matrix. formae tries to move more of that burden into a standard conformance harness that generated plugins can run against repeatedly.

For a young ecosystem, that is a smart move. A conformance suite can encode platform expectations before dozens of plugins drift into subtly incompatible interpretations of CRUD, discovery, replacement, or progress.

Licensing and ecosystem implications

Terraform’s Plugin Framework repository is MPL-2.0 and belongs to a large ecosystem where public providers are distributed through a registry. The framework README says it has reached general availability, follows semantic versioning, supports Terraform v0.12 and above, and recommends tagged releases.

formae’s docs explicitly state a plugin policy: plugins are independent works, executed out-of-process or as remote services, communicating only through documented public interfaces; plugin developers may choose open-source or proprietary licenses; plugin licenses should not impose terms on formae components. The repo files are marked FSL-1.1-ALv2.

This is not a side issue. Plugin architecture is often license architecture. Both systems use out-of-process plugins, but formae’s policy is unusually explicit about non-derivative boundaries and plugin licensing independence. The move away from Go native plugins therefore solved two problems at once: dependency lockstep and license coupling. Terraform’s mature ecosystem has already normalized separately distributed providers; formae is documenting that boundary early because its first implementation made the cost of in-process linking concrete.

Developer experience: who is the plugin author?

Terraform’s framework is best for developers who need to expose a system to Terraform practitioners. They must understand Terraform configuration, planning, state, unknown values, import, schema migrations, and acceptance testing. The reward is access to Terraform’s ecosystem and a highly recognizable workflow.

formae’s SDK seems designed for platform engineers who need to add a resource type to an agent-driven infrastructure platform. They implement operations against a target API, describe resources in Pkl, and rely on the SDK/agent for startup, schema extraction, registration, rate limiting, progress polling, discovery, conformance testing, and actor placement. Whether the operator runs locally in a workflow test, as a persistent plugin process, or behind a satellite agent is meant to be a deployment detail rather than a different plugin authoring model.

That makes formae potentially easier for internal platform teams, especially when the target system is not a traditional Terraform provider candidate. But it also means a formae plugin is useful inside formae’s operational model, not as a general-purpose Terraform provider. Terraform has broader ecosystem reach. formae can be more opinionated because it does not need to support every Terraform use case.

Where Terraform is stronger

Terraform’s Plugin Framework is stronger when compatibility, ecosystem reach, and plan/state semantics dominate.

1. Ecosystem maturity. Terraform has a huge provider ecosystem, registry distribution, mature protocol compatibility, and years of provider edge cases.
2. Plan semantics. Terraform’s plan/apply model and unknown/null value handling are deeply developed.
3. State migration machinery. Resource schema versions, state upgraders, moved-state support, identity schema, and protocol compatibility give provider authors explicit tools for long-lived resources.
4. Language integration. Provider-defined functions, data sources, resources, actions, ephemeral resources, and other Terraform concepts integrate with HCL and Terraform Core.
5. Incremental migration from SDKv2. terraform-plugin-mux lets complex providers migrate one resource or data source at a time.

If the question is, “How do I expose an API to Terraform users with minimal surprises over many provider releases?” Terraform’s framework is the obvious answer.

Where formae is stronger

formae’s Plugin SDK is stronger when agent-managed operations, discovery, and schema-as-platform-contract matter more than Terraform ecosystem reach.

1. Async operation model. Status and ProgressResult make long-running cloud workflows first-class.
2. Discovery by design. List, discovery filters, discoverable schemas, unmanaged inventory, and conformance tests treat discovery as core behavior.
3. Agent-owned orchestration. Scheduling, queuing, ordering, rate limits, plugin announcements, actor supervision, and retries sit in the platform rather than in every plugin.
4. Topology flexibility. Ergo network transparency lets the same operator model run in-process for tests, out-of-process locally, or remotely behind satellite agents.
5. Pkl schema packages. Resource schemas can be evaluated, extracted, verified, version-resolved, and used by the agent outside plugin method calls.
6. Conformance as scaffold. New plugins start with a lifecycle test suite that exercises real agent behavior, which is especially useful when AI agents generate much of the plugin code.
7. License and dependency boundary clarity. The docs explicitly frame plugins as independent out-of-process works, and the architecture avoids native Go plugin dependency lockstep.

If the question is, “How do I add a new resource family to an agent that manages inventory, discovery, async operations, and reconciliation?” formae’s SDK is the more targeted design.

The deeper design lesson

The Terraform Plugin Framework and formae Plugin SDK both demonstrate a move away from ad hoc plugins toward strongly mediated extension systems.

Terraform learned that provider authors need safer abstractions over a complex protocol. The framework gives them interfaces, typed values, schemas, validators, plan modifiers, state upgraders, and server adapters. It abstracts repetitive protocol machinery while still exposing Terraform’s semantics.

formae appears to be starting from a different lesson: infrastructure plugins are not only API adapters; they are participants in an agent’s operational system. The SDK hides identity/schema wrapping, extracts Pkl descriptors, announces capabilities through actors, standardizes progress and retry behavior, and tests plugins through real platform lifecycle operations.

One design is protocol-centered. The other is agent-centered.

That distinction will matter more as infrastructure tools become more autonomous. A plugin framework for a human-reviewed plan/apply tool must be conservative about state and compatibility. A plugin SDK for an agentic infrastructure platform must be conservative about scheduling, rate limits, reconciliation, discovery, and restart-safe operation. The same cloud API might need both kinds of adapters.

A practical recommendation

For most teams, this is not an either/or decision.

Use Terraform Plugin Framework when:

• your users already manage the system with Terraform;
• plan output and state compatibility are non-negotiable;
• you need Terraform Registry distribution;
• provider-defined data sources, functions, imports, or state migrations are central;
• you need broad compatibility with Terraform workflows.

Use formae Plugin SDK when:

• you are extending formae itself;
• resource discovery and inventory matter as much as desired-state application;
• operations are long-running, rate-limited, or need platform-level polling;
• Pkl schemas are part of your platform contract;
• you want plugin authors to implement remote-system operations while the agent owns orchestration.

The exciting possibility is not that formae replaces Terraform or that Terraform absorbs formae’s model. It is that infrastructure plugin architecture is splitting into two useful categories:

1. Protocol plugins for ecosystem-compatible declarative tools.
2. Agent plugins for autonomous reconciliation platforms.

Terraform’s Plugin Framework is one of the best examples of the first category. formae’s Plugin SDK is an early, concrete example of the second.

That is why the formae articles matter to this comparison. The Go plugin history explains why the process boundary exists; the Ergo primer explains why that boundary can be actor-shaped rather than just RPC-shaped; the Pkl schema machinery explains how the agent can keep reasoning about resource types outside the plugin’s Go code. Together they point to a different center of gravity: an infrastructure platform where plugins are supervised workers in an agent’s resource graph, not only RPC endpoints for a plan/apply engine.

That difference is exactly why the comparison is useful.

A messy infrastructure repository is not just an aesthetic problem. It is an operational risk.

Gruntwork’s guide, “Your Infrastructure Repo Is a Mess — Here’s How to Fix It”, is useful because it names real failure modes: copy-pasted environments, mega-modules, unclear ownership, state files with too much blast radius, and deployment paths that depend on tribal knowledge.

But the guide also exposes a deeper problem.

Terragrunt is correct that plain Terraform and OpenTofu are under-tooled for large infrastructure systems. The question is whether the answer should be yet another niche tool around a configuration language, or whether Infrastructure as Code should become more serious about the word code.

Terragrunt diagnoses a real gap

The Gruntwork critique lands because many HCL repositories do decay in predictable ways.

A team starts with a simple Terraform file. Then it adds staging. Then production. Then another region. Then another account. Soon the same provider, backend, tag, VPC, IAM, and service settings are copied across many folders. Small differences become hard to classify. Is staging different from production on purpose, or did someone forget to copy the last change?

The opposite strategy is also painful: one giant module that deploys everything. That avoids some repetition, but it creates a module with too many inputs, slow plans, unclear ownership, and a blast radius much larger than the change being reviewed.

Terragrunt’s answer is hierarchy, inheritance, generated configuration, stacks, and live-repo conventions. That can make a Terraform/OpenTofu estate more navigable.

The uncomfortable question is: why does an Infrastructure as Code ecosystem need so much external machinery to organize and reuse code?

The suspicion: this is configuration trying to impersonate software

Terraform modules are useful, but they are not software libraries in the way Go, Python, TypeScript, or Java developers usually mean that phrase.

In mainstream software ecosystems, teams expect a rich set of ordinary capabilities:

• packages with public APIs
• dependency management
• semantic versioning
• local development against unpublished packages
• unit tests, integration tests, mocks, fixtures, and contract tests
• object composition
• dependency injection
• reusable interfaces
• override points
• monorepo tooling
• build graphs
• task runners
• generated clients
• type checking
• documentation generated from the code itself

Go is a good reference point because its toolchain is intentionally boring and widely understood. Go code is organized into packages and modules. The go tool knows how to fetch, build, install, and test them. The standard testing package supports tests, benchmarks, examples, and fuzzing. Modules can be versioned and published from ordinary repositories.

The JavaScript and TypeScript world is messier, but it has the same basic shape. npm, pnpm, yarn, Jest, Vitest, Turborepo, Nx, Lerna, and many other tools exist because organizing and shipping code at scale is a common software problem. Teams may argue over the best tool, but the category is mature.

By comparison, Terraform modules feel constrained. They can be composed, but not extended like classes or refined through normal override mechanisms. There is no first-class dependency injection model comparable to mature application frameworks. Provider configuration has special global behavior. The Terraform docs themselves emphasize that provider configurations are shared across module boundaries and defined only in the root module; reusable child modules must not contain provider blocks. That is understandable for Terraform’s execution model, but it is not the same as a general software package system.

So Terragrunt’s hierarchy is not just a convenience. It is a workaround for missing language and ecosystem capabilities.

Infrastructure is no longer only operations plumbing

A common defense of simpler configuration is that infrastructure lifecycle is different from application lifecycle. The argument goes something like this: infrastructure is too dangerous, too stateful, and too operationally sensitive for the complexity of software engineering patterns.

That objection made more sense when infrastructure mostly meant racks, switches, network appliances, operating systems, and long-lived servers. It is less convincing in cloud-native systems.

Modern cloud infrastructure is often software-defined and architecture-shaped:

• queues and topics encode asynchronous workflow
• serverless functions encode business reactions to events
• API gateways encode product interfaces
• event buses encode domain integration
• managed databases encode storage and access patterns
• object stores encode data pipelines
• identity policies encode service boundaries
• software-defined networks encode reachability and segmentation

AWS’s serverless guidance describes managed services as a way for the provider to handle capacity provisioning, patching, platform management, availability foundations, and other infrastructure management tasks so teams can focus more directly on business logic and application behavior. That is not classic server-rack operations. It is application architecture expressed through cloud resources.

Enterprise Integration Patterns make the same point from a different direction. Patterns such as publish-subscribe channels, point-to-point channels, request-reply, correlation identifiers, message routers, splitters, aggregators, dead-letter channels, competing consumers, and service activators are software architecture patterns. In the cloud, many of those patterns map directly onto managed infrastructure resources such as SNS, SQS, EventBridge, Lambda, API Gateway, Step Functions, and queues with dead-letter policies.

If the infrastructure resource is implementing part of the integration pattern, then the IaC describing it is not merely configuration. It is part of the software architecture.

CDK is interesting because it treats infrastructure as libraries

AWS CDK is not automatically better than Terraform. It has its own risks: generated CloudFormation can be opaque, abstractions can hide details, and teams can write terrible code in any language.

But CDK is useful for this discussion because it has a more honest model of Infrastructure as Code.

The AWS CDK construct model has layers:

• L1 constructs map closely to individual CloudFormation resources.
• L2 constructs provide intent-based APIs with defaults, helper methods, permissions wiring, and resource integration logic.
• L3 constructs, also called patterns, combine multiple resources into reusable architecture-level solutions.

That layering matters. It gives teams escape hatches. If a high-level pattern is too opinionated, drop to L2. If L2 is missing or too constrained, drop to L1. If a team repeats the same architecture across products, it can publish an L3 construct as a normal library package.

AWS Solutions Constructs makes this explicit: it provides multi-service, well-architected patterns for defining repeatable infrastructure in familiar programming languages. It supports TypeScript, JavaScript, Python, and Java. Those constructs can use conditionals, loops, object-oriented techniques, normal tests, normal review, normal package distribution, and normal dependency management.

This is much closer to what software teams already know how to do.

L2 service integrations versus L3 solution patterns

The L2/L3 distinction also gives a better vocabulary for reusable infrastructure than “module.”

An L2 construct is close to a service-level abstraction. It can encode best-practice defaults and expose methods for common interactions. For example, a bucket abstraction can expose permission helpers. A queue abstraction can expose consumer wiring. A function abstraction can expose event-source bindings.

An L3 construct is closer to an architecture pattern. It might represent a static website behind CloudFront, a queue-processing worker with a dead-letter queue, an API backed by a database, or a load-balanced Fargate service. AWS Prescriptive Guidance describes L3 constructs as reusable patterns for resource interactions, resource extensions, and custom resources.

That distinction maps well to software architecture:

• L2: reusable service integration building blocks
• L3: high-level solution patterns

This is what many Terraform module libraries try to approximate, but HCL makes the abstraction boundary awkward. A module can expose variables and outputs, but it does not feel like an API with methods, behavior, dependency injection, override points, or typed composition. The result is often a giant input surface instead of a small, expressive interface.

TerraConstructs is the sharper test case

TerraConstructs complicates the argument in a productive way. It is not simply another wrapper around HCL in the same category as Terragrunt. Its own positioning is closer to this: keep Terraform/OpenTofu as the operational substrate, but move authoring into CDKTF constructs with TypeScript, strong typing, object-oriented composition, tests, and AWS CDK-inspired L2 abstractions.

That is much closer to the direction this post is arguing for. TerraConstructs explicitly tries to bridge AWS CDK developer experience with Terraform/OpenTofu operational ecosystem. It claims to provide deterministic, type-safe L2 constructs, synthesize to pure Terraform/OpenTofu output, support existing Terraform-adjacent tooling such as tflint, infracost, and OPA, and validate constructs with unit and end-to-end tests using Terratest.

This is a better answer to the Gruntwork problem than just saying “use CDK instead.” Many organizations have real Terraform assets: provider knowledge, state workflows, policy checks, CI conventions, cost tooling, drift tooling, and platform teams who understand Terraform plans. TerraConstructs says: do not throw that away, but stop pretending raw modules are enough of an abstraction system.

The CDKTF docs make the contrast direct. Constructs can represent a single resource, a group of resources, a subsystem, or a full architectural pattern. They can create, modify, enrich, and validate resources; expose methods; operate on a construct tree; use Aspects for cross-cutting concerns; and be tested like normal application code. HashiCorp’s own docs describe constructs as a superset of Terraform modules.

That supports the central critique: the weakness is not Terraform’s operations model. The weakness is HCL modules as the primary abstraction mechanism for increasingly software-shaped cloud systems.

The important point is not HashiCorp’s late stewardship decision. The important point is that CDKTF’s corporate steward let the project stagnate badly enough that the community had to decide whether the code-first Terraform path was worth saving. CDK Terrain is the stronger signal: developers looked at the same problem and chose continuation, repair, and community governance rather than giving up on the model.

CDK Terrain, at cdktn.io, is a community-driven fork and continuation of CDKTF hosted by the Open Constructs Foundation. Its docs describe the same core promise this article cares about: define infrastructure in familiar programming languages, synthesize Terraform-compatible configuration, and keep using Terraform or OpenTofu as the execution engine. It supports TypeScript, Python, Java, C#, and Go; can use Terraform and OpenTofu providers and modules; can interoperate with existing HCL projects; and provides testing helpers around synthesized output.

That matters because it changes the story. CDKTF was not killed by lack of need. The need is obvious: teams want the Terraform/OpenTofu ecosystem, but they also want testing, dependency management, abstractions, reusable code patterns, and real language tooling. CDK Terrain is a prime example of a community pulling together around a better future for IaC: execution-engine agnostic, open source, and aimed at preserving the software-engineering path HashiCorp neglected.

So CDK Terrain and TerraConstructs should become central examples in the article, not footnotes. They are the middle path:

• not raw HCL modules
• not Terragrunt-only hierarchy over config
• not CloudFormation-only AWS CDK
• but CDK-style constructs that synthesize to Terraform/OpenTofu

That middle path tests the whole thesis. If it works, the right critique of Terragrunt is not “custom tools are bad.” It is that custom tools should move IaC toward ordinary software engineering capabilities, not deeper into bespoke configuration orchestration.

LLMs make the abstraction problem sharper

There is a tempting counterargument: if LLMs can write Terraform for us, maybe HCL’s rough edges matter less. Ask an agent for a VPC, an SQS dead-letter pattern, a Lambda integration, a Kubernetes deployment, or an IAM policy, and it can produce pages of plausible HCL in seconds.

That is useful, but it is not a substitute for architecture.

The problem with a messy IaC repository was never merely that humans type too slowly. The problem is that a large system needs stable abstractions, tests, reviewable APIs, versioned reuse, ownership boundaries, and a way to express intent without duplicating implementation detail everywhere. An LLM that generates another thousand lines of HCL can make the repository larger faster. It does not automatically make the design better.

Token economics make this more important, not less. Recent AI-market reporting and pricing analysis point in the same direction: the heavily subsidized era of abundant cheap inference is under pressure from rate limits, enterprise metering, feature restrictions, price changes, investor expectations, and margin discipline. Even if model quality keeps rising, teams should not design their infrastructure workflow around repeatedly regenerating and re-reviewing low-level configuration forever.

The better use of LLMs is at a higher level of intent. Let the model help design or call a tested construct: “give this service an event-driven ingestion path with a dead-letter queue, least-privilege permissions, alarms, and cost tags.” Then the durable artifact should be a software library with tests, defaults, override points, and a public API — not a one-off blob of HCL that must be audited from scratch every time.

This is another reason proper software engineering matters in IaC. Good constructs amortize thinking. They reduce repeated token spend. They reduce repeated human review. They give agents something safer to compose. And they keep the important decisions in versioned, tested code instead of scattering them across generated configuration.

The critique of “yet another tool”

Terragrunt may still be the right pragmatic choice for teams already invested in Terraform/OpenTofu. If the organization has years of modules, policies, state, providers, and operational workflows, then Terragrunt’s conventions can impose much-needed structure without replacing the whole stack.

But we should be clear-eyed about what kind of solution it is.

Terragrunt is niche tooling for a niche configuration ecosystem. It teaches teams more Terragrunt-specific concepts: terragrunt.hcl, hierarchical includes, dependency blocks, generated files, stacks, live repo conventions, and wrapper-driven workflows. Those concepts can be effective, but they do not transfer as widely as ordinary software engineering practices.

TerraConstructs is different enough that it should be judged separately. It is still custom IaC tooling, but its abstractions are closer to transferable software concepts: typed classes, construct trees, package-managed libraries, tests, methods, composition, and cross-cutting aspects. That does not automatically make it safe or durable, but it makes it a more serious answer to the “Infrastructure as Code” promise than another layer of HCL orchestration.

A Go package, a Python library, or a TypeScript construct library teaches patterns that apply beyond one IaC tool. Package boundaries, tests, mocks, dependency injection, semantic versioning, monorepo build graphs, and API design are not infrastructure-specific. They are how software teams already manage complexity.

If Infrastructure as Code is supposed to be code, then the default should not be: write config, discover it cannot scale, then add another config wrapper.

The default should be: use mature software engineering tools to model infrastructure as part of the application architecture.

The better framing

The point is not to say “Terragrunt is bad.” That would be too easy and not quite fair.

A better position is:

1. Gruntwork is right about the failure modes of messy IaC repositories.
2. Terragrunt is a pragmatic response to Terraform/OpenTofu’s missing repo, reuse, inheritance, and orchestration capabilities.
3. But Terragrunt also demonstrates the limits of treating IaC as configuration first and code second.
4. Cloud infrastructure increasingly encodes software architecture patterns, especially in serverless and managed-service systems.
5. Therefore, the long-term direction should be boring, transferable software engineering: packages, tests, types, dependency management, composition, dependency injection, and reusable architecture libraries.

The strongest version of the argument is not anti-Terragrunt. It is anti-accidental-specialization.

When the problem is software complexity, the solution should look more like software engineering than another bespoke layer of configuration semantics.

AWS describes an AI-Driven Development Lifecycle (AI-DLC) for financial services: a software delivery model where AI agents do more than autocomplete code, but humans still keep oversight and accountability.

The main idea is to place AI across the whole development lifecycle. AI can help create plans, user stories, application code, tests, infrastructure-as-code, documentation, and operational insights. Humans review the work, provide business and regulatory context, approve important decisions, and make sure the output is safe and aligned with the organization’s standards.

I read this as a middle path between two extremes. Fully autonomous AI development can move quickly, but it is risky for regulated industries. Simple AI-assisted coding is easier to control, but it only improves small parts of the workflow. AI-DLC tries to capture more value by letting AI orchestrate larger chunks of delivery while adding governance, traceability, and human checkpoints.

For financial services, the most important theme is not just speed. It is speed with control: faster software delivery, but with evidence, audit trails, testing, security, resilience, and approval gates built into the process.

A second source from LCMH makes the idea feel broader than one regulated-industry use case. It frames AI-DLC as a general software-development shift where AI can initiate workflows, decompose work, and propose action plans while humans validate direction and quality. That changes the developer’s role: less pure execution, more judgment, review, and risk ownership.

The LCMH article also adds a useful caution: productivity gains should not be measured only by code volume or velocity. If AI-DLC works, it is because teams combine automation with strong engineering practices such as clear specifications, modular systems, review checkpoints, and human accountability. The interesting question is not just “can AI write more code?” but “can teams safely coordinate more of the lifecycle through AI without losing control?”

Future updates

• Compare AWS’s AI-DLC framing with “agentic software engineering” and AI pair-programming workflows.
• Look for examples of teams using AI agents with compliance or audit requirements.
• Explore where human review should be mandatory versus optional.
• Compare financial-services governance needs with broader AI-DLC adoption in less regulated software teams.
• Explore the shift from developer-as-builder to developer-as-reviewer, validator, and decision-maker.
• Check whether reported productivity gains depend on prerequisites such as modular code, strong specs, monorepos, typed languages, and strict review.
• Gather counterexamples: where AI-generated code or infrastructure creates hidden risk.

Source

• AWS for Industries: AI-Driven Development Lifecycle for Financial Services, published May 26, 2026.
• LCMH: AI-DLC: how AI is transforming the software development lifecycle, accessed June 1, 2026.

Software teams often talk about speed as if the main bottleneck is typing code. If code can be generated faster, reviewed faster, merged faster, and deployed faster, then it seems like better software should arrive sooner.

But that misses the deeper constraint.

A lot of software work does not fail because the code was typed too slowly. It fails because the team did not yet know what good meant.

That is the useful distinction in Joe’s essay, “R&D Is Two Jobs, and Research Doesn’t Run on Autopilot”. Research and development are often compressed into one phrase, but they are different types of work. Research discovers the target. Development executes against a target.

Spec-driven development sits exactly at that boundary.

A specification is not just a document. It is the artifact that turns research into development. It captures the current definition of good well enough that humans, tests, tools, and AI agents can all work against it.

Research finds the target

Research is the phase where the team is still asking questions such as:

• What problem are we solving?
• Who is it for?
• What does success look like?
• What tradeoffs matter?
• Which assumptions are risky?
• Is this even the right thing to build?

The output of research is not primarily code. The output is knowledge.

Sometimes that knowledge comes from prototypes. Sometimes it comes from customer conversations. Sometimes it comes from technical spikes. Sometimes it comes from writing down a model and realizing the model does not make sense.

This is why autonomous coding agents struggle when pointed at vague work. They are powerful at producing plausible software, but plausibility is not the same as correctness. If the goal is unclear, an agent can generate something coherent while still missing the actual question.

Joe’s essay describes the software factory as a machine for converging on a target. But it cannot invent the target. Someone still has to define what good means.

That definition is research work.

Development executes against the target

Development begins when the work becomes legible enough to check.

A development-ready task has some combination of:

• a clear interface
• a scenario
• an acceptance criterion
• a test
• an example
• a constraint
• a definition of done

This is the terrain where agents, automation, CI, test suites, and parallel execution become much more effective.

Once the target is known, the work can be decomposed. A team can say: implement this behavior, satisfy this contract, preserve this invariant, make this failing test pass, or refactor without changing these observable outcomes.

In other words, development scales when the specification is good enough.

Specifications are feedback devices

The second article, “The Feedback Principle”, gives another way to understand specifications.

Its central claim is simple: the earlier feedback is detected, the cheaper it is to handle.

A quality function is any activity that provokes feedback. Testing is a quality function. Code review is a quality function. Pair programming, user interviews, architecture review, mockups, BDD scenarios, CI pipelines, and retrospectives are all quality functions because they expose information while there is still time to act on it.

A specification is also a quality function.

A good spec is not valuable because it satisfies process theater. It is valuable because it provokes feedback before code exists.

When a stakeholder reads a scenario and says, “That is not how the workflow actually works,” the spec has done its job. When QA asks, “What happens if this field is missing?” the spec has done its job. When an engineer notices that two requirements contradict each other, the spec has done its job.

The cheapest bug is the one found while it is still a sentence.

Shift feedback left into the specification

The Feedback Principle reframes “shift left” as moving quality functions earlier, where feedback is cheaper.

Spec-driven development is one of the strongest forms of shifting left because it moves verification and validation into the design stage.

Before writing production code, the team can ask:

• Is this the right behavior?
• Is this behavior complete?
• Can we test it?
• Can an implementer understand it?
• Can an agent execute against it?
• Are the edge cases explicit?
• Are the tradeoffs visible?
• Would a user recognize this as solving their problem?

This matters because late feedback is expensive. A missing requirement caught in production may require incident response, rollback, customer communication, and a rewrite. The same missing requirement caught in a specification may require a five-minute conversation and an edited paragraph.

Spec-driven development is not about writing more documentation. It is about making the specification the earliest executable surface for feedback.

Verification and validation

The Feedback Principle also separates verification from validation:

• Verification: Did we build the system right?
• Validation: Did we build the right system?

Specs connect these two.

A weak specification only supports verification. It lets us check whether the system conforms to what was written, but it does not help us know whether what was written is worth building.

A strong specification supports both. It captures enough product intent to validate the direction, and enough technical precision to verify the implementation.

This is especially important in AI-assisted development. Agents are increasingly good at verification-oriented work: making tests pass, implementing interfaces, following examples, and resolving mechanical issues. But validation still requires context, judgment, and taste.

So the human role does not disappear. It moves upstream.

The human becomes responsible for shaping the target, testing the assumptions, and maintaining the quality of the specification.

The agent era makes specs more important, not less

There is a tempting but wrong conclusion to draw from coding agents: if code becomes cheap, specs become less important.

The opposite is true.

As code generation becomes cheaper, the cost of unclear intent goes up. A human developer given a vague task might stop and ask questions. An agent may confidently produce a large amount of coherent but misdirected code.

The bottleneck shifts from implementation capacity to target quality.

In an agentic workflow, a specification is not merely a communication artifact between humans. It becomes the control surface for the whole system.

The spec tells the agent:

• what to build
• what not to build
• how success will be checked
• which examples matter
• which constraints are non-negotiable
• when the task is complete

Without that, the agent is not a factory. It is a high-throughput ambiguity amplifier.

Spec-driven development as a loop

Spec-driven development should not be imagined as a waterfall process where a perfect document is written upfront and then handed to implementation.

That would misunderstand both research and feedback.

A better model is a loop:

1. Explore the problem space.
2. Write the current understanding as a specification.
3. Provoke feedback from users, QA, engineers, stakeholders, tests, and prototypes.
4. Revise the specification.
5. Execute against the parts that are now clear.
6. Learn from implementation and production.
7. Update the specification again.

The spec is not the end of thinking. It is the medium through which thinking becomes inspectable.

What makes a specification useful?

A useful specification should make feedback easier and cheaper.

That means it should include things like:

• concrete scenarios
• examples of expected behavior
• non-goals
• edge cases
• acceptance criteria
• domain language
• constraints
• open questions
• risks and assumptions
• validation notes
• testable outcomes

The most important part may be the open questions. A spec that pretends uncertainty does not exist is dangerous. It makes research look like development. It invites the team to execute before the target is known.

A good spec separates what is known from what is assumed.

That distinction determines how much autonomy is safe.

The leash length depends on spec quality

Joe’s essay makes an important point about agent autonomy: the better you know what good looks like, the longer the leash you can give the agent.

This is a useful principle for spec-driven development generally.

When the spec is vague, keep the loop tight:

• shorter tasks
• more human review
• more prototypes
• more stakeholder feedback
• more validation

When the spec is clear, the loop can widen:

• more automation
• more parallel work
• more autonomous implementation
• more reliance on tests and acceptance criteria

The danger is not using agents. The danger is giving development-level autonomy to research-stage work.

Spec quality determines safe autonomy.

Conclusion

Spec-driven development is not bureaucracy. It is a way of making software work cheaper, faster, and safer by moving the discovery of mistakes earlier.

Joe’s research-versus-development distinction explains why specifications matter: they define the target that development needs in order to execute well.

The Feedback Principle explains why specifications are economically powerful: they provoke feedback while change is still cheap.

Together, they suggest a simple theory:

Research discovers what good looks like.
Specifications make that definition inspectable.
Development executes against it.
Feedback keeps the whole system honest.

In the age of AI coding agents, this becomes even more important. The future of software development may involve more autonomous implementation, but autonomy only works when the target is clear.

The spec is where that clarity lives.