Agentic Harnesses Need a Proper Workspace

Welcome to AutomatED: the newsletter on how to teach better with tech.

In each edition, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

Today, I share some updates on where I’ve been, I explain agentic harnesses like Claude Cowork and OpenAI Codex, and I discuss how I’ve accelerated my productivity with them.

Hi—

Before I tell you where I have been, a request: please hit reply, (re)introduce yourself, and tell me how you are doing and what you are working on.

I mean it! It has been quiet here for a while, and I know that a group as large and diverse as this one (roughly 5,500 professors, learning experience designers, and edtech product people) contains an equal number of rather different stories: courses redesigned, jobs changed, research stalled or finished, AI adopted enthusiastically, AI avoided with equal enthusiasm, …

I should also be clear about what this email does and does not announce. This is the first installment of a deliberately finite three-part series. The second will arrive Monday, August 31, and the third on Tuesday, September 8. I may write other things later, but I am making no promise that the old weekly cadence is returning. In fact, I can promise it won’t. It was tough to sustain!

What I can promise is that these three pieces concern the most important practical change in how I think about AI since I last wrote to you regularly. And, to boot, I’ll give you a tool to leverage it!

👋 “Where Have You Been?”

For readers who joined long ago, or who understandably no longer keep my biography near the top of their working memory, I am Dr. Graham Clay. I trained as a philosopher, spent close to a decade teaching and researching at universities, and eventually moved most of my work toward AI and education. Through AutomatED, I wrote about AI pedagogy, helped professors and institutions think through its use, built AI workflows and tools, and tried to clearly and fairly explain the technology in its many manifestations.

Then, after consulting for a while, I stepped away from the regular AutomatED rhythm to join Kognity, an interactive digital curriculum and learning platform focused on the IB DP curriculum trusted by 2,000+ schools across 130+ countries, as its full-time AI Strategist.

The title I had is somehow both too vague and quite accurate. I led company-wide AI content-generation strategy across evaluation, workflow architecture, agentic integrations, people, and tooling. I helped build systems for writing curriculum, assessments, and teacher materials. At peak I managed quite a few people, directed several content production teams, and I helped a content organization of roughly 20 people move from predominantly manual production toward designing and operating AI workflows.

That last sentence is easy to read too quickly.

A content organization that adopts AI does not simply give everyone a better autocomplete box. Roles change (a lot). The people who once wrote or edited each individual item begin designing the process by which thousands of items are created and checked. Learning experts who once applied their judgment one artifact at a time begin writing rubrics, constructing evaluation datasets (norming scorers around the rubrics, identifying outliers, and ensuring statistical benchmarks are met), calibrating AI judges, and deciding which disagreements matter. Subject-matter experts and learning experience designers remain responsible for “final” review, while their expertise also moves earlier into the architecture of the system.

The AI systems I helped build at Kognity produced 35–90% lower costs and 50–90% faster delivery than manual baselines. One effort shipped 480 teacher slide decks (20-50 slides each, with diagrams, teacher supports, and more), which reached about 10,000+ downloads. But the numbers belie the messy process that led to the results. Many execs seem to think that we can sprinkle AI over a process and deduct 90% from the cost, but it’s far from the truth.

The deeper lesson I’d share concerned where human expertise belongs.

We often imagine “human in the loop” as a person stationed at the end of a production line, inspecting whatever the AI has made. Sometimes that is necessary. Yet it is only one place for human judgment, and often not the most important one.

Experts also need to design the workflow, determine what good work consists in, encode that standard in rubrics and stated process, assemble the datasets against which judgments are tested, validate the AI systems that apply those judgments, define review gates, and decide how organizational roles should change. A learning designer’s judgment becomes more powerful when it is made explicit, tested, versioned, and applied across many outputs. It also becomes more demanding, because the organization must be able to say what standard it is using, who established it, and where the system should stop.

That work changed the question I ask about AI. For years, the natural question was: What can this model do? (For those of you who are long-time readers, you saw it in my many contests to see if AI could complete a given academic task.)

Now, the fact is that AI can complete most work when positioned properly in our working environments. So, the more pertinent question is: What kind of working environment should we build around AI to position it most effectively?

🤖 Agents? Harnesses?

Nowadays, the environment around the most powerful deployments of AI models is often called an agentic harness, a term apparently designed to make an important idea sound like outdoor equipment.

The model is the part that takes in a prompt and generates a response, while reasoning along the way (read: generates intermediate responses that guide its production of subsequent ones by becoming parts of their prompts). The harness is the software environment around it. Depending on the product and the permissions a user grants, a harness may let the model inspect project context, search files, use tools, edit documents or code, browse the web, connect to other services, complete a sequence of steps, recover after an interruption, and coordinate several agents working on different parts of a task.

A blank chatbot answers a prompt. A harness can participate in situated work.

Longtime readers will recognize the argument underneath this. I have argued in this newsletter before that increasingly capable AI is better understood as something like a new colleague than as ordinary software (see AI Integration is Like Hiring), and that what usually stands between a capable model and a useful result is what I once called integration effort: the work of giving it the right context, connecting it to the right tools, and checking whether the output is actually good.

A harness is, by design, intended to reduce integration effort, and thus if you are a user of AI, you should dedicate your own integration effort to understanding and improving how your chosen harness works instead of supplying AI the relevant inputs manually or semi-manually every time.

Suppose a teaching center is revising its AI policy after a difficult semester. The work involves the existing policy, faculty feedback, institutional constraints, examples of assignments, minutes from several meetings, a new decision about student data, and an unresolved question about assessment. In a blank chat, somebody has to assemble that context, explain which version controls, request a draft, and manually compare the answer with the documents.

Inside a capable harness, the system may be authorized to inspect the project, find the relevant committee record, apply the project’s instructions, draft the revision in the correct file, run checks, show the changes, and retain enough task state to continue later. The quality of the model still matters enormously. The useful work also depends on what surrounds it.

I think this interaction form will persist even though the present products, interfaces, and leading models certainly will not remain fixed.

One reason is that a harness connects model reasoning to the actual circumstances of work. Organizations do not operate through isolated questions and answers. They operate through documents, databases, permissions, recurring processes, decisions, and people with different authority.

A second is that a harness makes iterative verification possible. An AI can inspect, act, test, compare the result with the requirement, and revise. For a professor, that might mean checking an assessment against learning objectives and a rubric. For a teaching center, it might mean comparing a proposed guide with institutional policy and the cited sources. Work becomes a loop rather than a single impressive-looking response.

And the harness gives an organization somewhere to encode how work should happen: which sources govern, which tools may be used, which files may be changed, what requires approval, and how a result should be reviewed. Those controls will sometimes be weak or badly designed. Still, the possibility is substantially more important than asking every employee to become a virtuoso prompt writer in a blank text box or, even less productively, writing manually.

💡 Four Harnesses, Four Ideas of Work

The differences among current harnesses help explain why the surrounding environment deserves more attention.

The first one to the market to make a big splash was Claude Code. It is centered on the terminal and the software repository. It can follow project instructions stored with the work, load rules when particular paths are involved, use local repository-scoped memory, call tools, and place parallel work in isolated Git worktrees. Its design assumes that the repository is a durable working environment and that continuity should serve developers operating inside it.

Cowork presents another model, also from Claude but not for software developers. It is a desktop knowledge-work harness rather than a developer-first terminal tool. Projects can bundle files, links, instructions, chats, connectors, and project-scoped memory. That makes the interaction more coherent for the many intelligent knowledge workers who have no desire to learn Git or operate inside a terminal. Continuity is correspondingly tied more closely to the product’s own project and memory system.

OpenAI’s Codex is also highly capable at local repository work, but it can sit inside the broader ChatGPT Work surface (OpenAI’s analog to Cowork). That broader environment can combine projects, chats, files, connected sources, tools, and longer-running tasks. It is important, however, to distinguish checked-in project instructions and files from native personal or project memories. Those memories may be helpful recall, but they are separate stores with their own behavior. For consequential work, I would no more make a product’s synthesized memory the canonical record than I would make my recollection of a committee meeting the official minutes.

Then there is Muse Code, Meta’s new terminal coding harness. It has a local event log to which model calls, tool runs, approvals, and edits are appended before action; crash-and-restart recovery; persistent background agents; isolated worktrees; and a model co-trained with the harness itself. Muse’s event log is intended to answer: What did the agent do, and where should it resume?

A project ledger, like the tooling I’ve been developing (more below and on Monday) answers another question: What is currently true, what has been decided, what supports it, and what remains unresolved?

Perfectly replaying an agent’s actions would not necessarily tell a new collaborator that a decision was later superseded, a source turned out to be weak, or an apparently promising proposal was rejected. Conversely, a concise project record will not reconstruct the precise seventeenth tool call in a crashed 20-hour coding session. Execution recovery and project continuity live at different layers.

The comparison points to something larger. Models increasingly arrive inside opinionated runtimes, developed by companies that want you to stay in their product space (let’s be honest). Their memory stores, permissions, context-loading rules, recovery behavior, and relationships to files all differ. Some are optimized for code, some for desktop knowledge work, and some for a product’s own cloud project space. Model–harness co-training may widen the differences further.

🧠 My (WIP) Answer: the Work Ledger

These products make it inaccurate to say that agents simply forget everything. They can carry instructions, memories, files, summaries, task state, and repository context across sessions. In many ordinary cases, native continuity may be entirely sufficient.

A system can remember far more than a blank chat, however, and still leave a project with state that is fragmented across products, stale relative to the controlling source, opaque to the user, or difficult to move elsewhere.

Consider a course redesign that lasts six months. One chat remembers an early assessment plan. A project contains the revised rubric. A local coding agent has learned a formatting preference. A committee later rejects the original assessment plan, but that decision lives in meeting notes none of the other systems automatically treats as authoritative.

Nothing here requires an AI to have forgotten. The danger is that it remembers the wrong thing, or that it cannot distinguish recollection from a governing decision.

The same problem appears in research projects, accreditation work, curriculum revisions, grant development, faculty-development programs, and departmental policy. Useful continuity requires selection: the current objective, accepted decisions and their reasons, relevant sources and constraints, open questions, and the authority boundaries around the next task. More context does not automatically produce better context.

Over the past several months, I have been developing a system for this problem in my own cross-project work.

The Work Ledger is a local, Git-backed, human-readable repository designed to work across AI harnesses. (Yes, read that again: it is harness neutral, in that it enables you to move your work from one harness to another, so you are neither locked in nor stuck when you run out of usage with one.) It is for projects with continuing workstreams, sources, decisions, constraints, preferences, and unresolved questions. One way to put its core idea succinctly is: governed work continuity with an evidence trail.

In practice, a few things happen. Before an AI touches your project, it reads a short, current summary of where things actually stand, not the entire back-catalog of everything anyone has ever said about it. It also keeps your original sources (the actual policy, the actual feedback, the actual data) separate from whatever gets pulled out of them later, so a rough paraphrase never quietly stands in for the real thing. And before it starts a real task, it writes down exactly what it's about to rely on and why, before doing the work rather than after, so you can catch it in advance if it's about to lean on the wrong thing.

For the handful of conclusions a project will actually keep building on, the bar goes up further. The AI has to say what it's claiming, what's actually backing it up, and the strongest reason it might be wrong, before that gets to count as settled (I call that check TRACER; the full mechanics are explained in Monday's post). And nothing becomes official until a second, independent pass checks it and logs it in a permanent, dated record you can always look back through. Put simply: the AI has to earn the right to call something true, and either way, you get the receipts.

That is a great many mechanisms. Their practical effect is simpler.

When I open a fresh session, including in a different harness (I often run out of usage with one harness and need to keep working; 1-2B tokens a day burns through even the heaviest Pro plan with Codex 🫣), I want the AI to find the current project state without treating every old conversation as equally authoritative. I want it to know which source supports a claim, which decision replaced an older one, what remains uncertain, and what it is allowed to change. When a task ends, I want a reviewable account of what changed before the result becomes part of the project’s durable memory.

The system has established its value for my own work. It has helped me resume consequential projects, recover source relationships, and move between AI surfaces without reconstructing the whole argument from memory. One beta user I’ve shared it with says it is significantly better than native Cowork, even after a day or two of use. And this is coming from someone who used Cowork 4+ hours a day for more than 6 months!

But it won’t be a fit for everyone. The people for whom this may matter most are those with continuing workstreams and real consequences if an AI resumes from the wrong premise. If you occasionally ask ChatGPT to improve an email or suggest discussion questions for tomorrow, you do not need this level of structure. Please enjoy your lighter life.

But if you have ever returned to a project after three weeks and thought, “I know we settled this, but I no longer trust either my memory or the chat history,” then you know the shape of the problem.

On Monday, I will examine the Ledger itself and compare it with Obsidian, native AI memory, Memcrate, LLM Wiki-style systems, and related alternatives. Some of them solve portions of the problem with much less friction, which is precisely why the comparison is worth making.

In essence, different types of harnesses are different types of teams of people that have a thin management layer built-in, with the memory substrate you choose, like the Ledger, adding a more opinionated operational process on top of that base layer. As I argued back in April 2024 (“Will Students Need Management Skills as AI Develops?”), AI is increasingly something you manage and direct, so your theory of how to do so should reflect your broader views on management, operations, and process.

On Tuesday, September 8, I intend to share a version of the ledger for you to use.

For now, hit reply. Tell me who you are these days, what you are working on, and whether continuity across AI-assisted work is a real problem for you or merely an elaborate solution looking for one.