Give your AI agent a video tool (MCP)

If your agent can call MCP tools, it can ship finished video. This is the practical guide: setup for every major client, the full tool reference, a worked autonomous pipeline, the error-handling patterns that matter, and (the part most write-ups skip) the schema design decisions that determine whether your agent produces valid videos 100% of the time or "usually."

The one-line setups#

The Clipkit MCP server runs two ways, and they expose the same fifteen tools:

Hosted: nothing to install, no auth, no API key. Add https://www.clipkit.dev/mcp as a Streamable HTTP server in any MCP client that supports remote servers.

Local (stdio): npx -y @clipkit/mcp-server. Client-specific one-liners:

# Claude Code
claude mcp add clipkit -- npx -y @clipkit/mcp-server

// Claude Desktop / Cursor / Cline / Windsurf: mcpServers config
{ "clipkit": { "command": "npx", "args": ["-y", "@clipkit/mcp-server"] } }

claude.ai (paid plans): Settings → Connectors → Add custom connector
→ https://www.clipkit.dev/mcp

Building a custom loop? Any MCP client library works: the server is standard MCP over stdio or Streamable HTTP. Step-by-step pages for thirteen clients live at clipkit.dev/docs/mcp.

The fifteen tools#

All fifteen work with zero credentials. Every tool ships MCP behavior annotations (readOnlyHint / openWorldHint / destructiveHint, explicitly set, which some directories now require), so clients can auto-permit reads and confirm destructive calls.

The interactive loop#

For an assistant with a human in the conversation, the canonical loop is: create_project → add_element × N → validate_project → preview_still → targeted edit_element revisions → open_in_editor. The human fine-tunes on the timeline and exports the MP4 in-browser (client-side WebGPU: free, watermark-free, no account). Iteration costs nothing on this path, which matters more than it sounds: agents revise a lot, and a metered render API turns every revision into a line item.

The autonomous pipeline#

No human in the loop? Swap the editor handoff for the render API. The skeleton, in pseudocode:

def make_video(brief, data):
    pid = mcp.create_project(title=brief.title)
    for element in compose(brief, data):        class=class="tok-str">"tok-cmt"># LLM authors elements
        mcp.add_element(project_id=pid, element=element)

    for attempt in range(3):                    class=class="tok-str">"tok-cmt"># validate-fix loop
        result = mcp.validate_project(project_id=pid)
        if result.valid: break
        apply_fixes(pid, result.errors)         class=class="tok-str">"tok-cmt"># LLM patches named errors
    else:
        raise NeedsHuman(pid)                   class=class="tok-str">"tok-cmt"># 3 strikes → escalate

    frame = mcp.preview_still(project_id=pid, time=1)
    if not approve(frame):                      class=class="tok-str">"tok-cmt"># optional cheap gate:
        raise NeedsHuman(pid)                   class=class="tok-str">"tok-cmt">#   human or VLM checks 1 frame

    return render_api.render(pid)               class=class="tok-str">"tok-cmt"># paid: MP4 URL back

The two structural decisions worth copying regardless of tool: bound the fix loop and escalate with context (an agent that can't validate in three attempts has a problem retries won't fix, and open_in_editor makes escalation a link a human can act on in seconds), and gate on a preview frame, not the full render (deterministic rendering means one approved frame vouches for the whole export; a vision model can even do the approving for pennies).

Error handling: let the validator drive#

The design premise is that your retry logic should never parse stack traces. Three patterns cover production:

• Validation errors are instructions. elements.0.layer: Required names the element and the field. Feed the error text straight back to the model with "fix this". That's the whole repair prompt. In our benchmark transcripts, one round of validator feedback fixed virtually every authoring mistake.
• Input errors are contracts. Tools reject malformed calls with the exact parameter path (id_or_url: Required). Same treatment: the error is the prompt.
• Escalation is a link. When automation taps out, open_in_editor turns the failure into something a human resolves on a timeline in under a minute, infinitely better than a JSON blob in a dead-letter queue.

The schema decisions that make agents reliable#

We measured agents authoring video across formats: 60 cells, 5 briefs, 3 model tiers, every artifact validated and rendered. The schema-validated JSON path: 15/15 valid. Agents writing render code for identical briefs: 13/15. Same models. The delta is substrate design, and the five decisions below are the transferable lessons (they apply to any agent tool, not just video):

1. Fail early and specifically. A validator that says elements.0.layer: Required turns a failure into a next action. A renderer crash after typecheck turns it into a debugging session. If your tool has one investment dollar, spend it on error messages.
2. Make edits granular. edit_element patches one field of one element. Small operations mean small diffs, cheap retries, and no regenerate-everything failure mode when one word changes.
3. Serve the docs as tools. read_docs and get_schema mean the model grounds itself in the current format on demand, instead of confidently authoring against a version remembered from training data. Cheap to build, outsized effect.
4. Borrow conventions models already know. In the Clipkit Protocol, layers stack ascending like CSS z-index and x/y is the top-left corner like the CSS box model. Conventions with a million training examples are conventions agents don't get wrong; novel clever ones get violated at exactly the volume where you can't afford it.
5. Render deterministically. Same document, same video, every time. This converts "validated + previewed" into "done": no flaky-output retry logic, no approved-but-different exports, and caching actually works.

What to expect it to do well#

Structured, data-shaped video: launch announcements, changelog videos from release notes, animated captions, charts and metric recaps in motion, title cards, personalized and templated content at volume. Media enters as public URLs or via ingest_asset; text, shapes, gradients, captions, charts, transitions, and particles the engine draws itself.

What it won't do, by design: invent footage. There is no generative model in the render path, which is exactly why output is deterministic and reviewable. If the job needs generated clips, produce them with a generation model and let the agent compose them: place, trim, caption, brand. Generation and composition are different layers that stack well; see "Agentic video editing, explained" for the architecture argument.

A starter prompt library#

System-prompt fragments and user asks that reliably produce good compositions, whichever agent framework you run:

• "Before composing, call read_docs for the authoring guide." One line in your system prompt; eliminates most format hallucination at the cost of one tool call.
• "Give every element an id." Makes later edit_element calls targeted instead of guesswork.
• "Validate after every 3–4 elements, not only at the end." Smaller diffs between validations mean errors localize to what just changed.
• "16:9 is 1920×1080, vertical is 1080×1920; set dimensions at create_project time." Resizing later is possible but re-layout costs more calls than starting right.
• Concrete user asks that exercise the full loop: "Turn these release notes into a 20-second changelog video with animated captions." / "Animate these five numbers as a bar chart, one per beat." / "Make a 12-second launch teaser: dark bg, bold amber headline, end on our URL."

Testing your integration#

Before trusting a pipeline, run the five-minute conformance pass, the same sequence a directory reviewer would: create_project → add_element (a text element with an id and a layer) → edit_element (patch its text) → validate_project (expect "Valid") → preview_still at t=1 (expect image content) → open_in_editor (expect a public-editor URL that opens with no login). Then break it on purpose: add an element with no layer and confirm your retry loop repairs from the validator message alone. If those seven calls behave, everything else is composition.

Operational notes for production#

Three lifecycle details that matter once real traffic flows. Anonymous projects are ephemeral: compositions created without an account are instances with a limited lifetime, perfect for the compose-approve-render window, wrong for archival; persist the JSON document itself (it's small, and set_project can resurrect it anywhere). Ownership is claim-on-open: the first signed-in user to open a composition in the editor becomes its owner, which is exactly the handoff you want ("agent drafts, teammate claims") but worth knowing before you post editor links in public channels. Idempotency lives in your hands: the document is the state, so retried pipelines should re-set_project from their stored source rather than blindly re-adding elements; a duplicate add_element call is two headlines, not an error.

Costs, plainly#

Composing, validating, frame previews, and browser rendering: free, no account, no API key. An agent can iterate to a finished, validated, human-approved composition at zero cost. Cloud rendering and professional formats (ProRes, AV1): paid per second of output on a public rate card, and only when a server actually renders. A free account makes projects durable (first signed-in opener of a composition becomes its owner). For the assistant-with-a-human case, the entire loop through exported MP4 routinely costs nothing.

Start here#

Connect the server (one line, above), run one conversational video to feel the loop, then wire the pseudocode pipeline against your real data source: release notes, metrics, listings, whatever your product turns into video. If you're choosing between this and having agents write renderer code, the measured comparison is in "Clipkit vs Remotion"; if you want the ten-minute human-first version, "How to make videos with Claude" walks the same loop conversationally.