Every Stage My AI Runs to Edit a Video

The reel attached to this article started as one take, 209 seconds long. Three minutes and twenty nine seconds of me talking, restarting, pausing to think, and saying the same line three times until one of them came out right.

What went out was 35 seconds. Twelve kept pieces of that take, stitched in order, captioned word by word, graded and finished. No human editor opened it. No CapCut. I did not touch a timeline.

I want to walk you through exactly what happened in between, because "my AI edits my videos" is the kind of sentence that sounds like marketing until someone shows you the receipts. So the numbers below are all from this one video's own edit record, not from a good week I picked out.

The finished reel is right below, and the rest of this is how it got made.

Why I built this before anything else

Years ago I worked as a video editor at a media company. There were nights I was still cutting at 3am, on videos exactly like this one, and the work was not hard so much as endless. Watch the take. Find where the sentence actually starts. Trim the pause. Find the good version of the line. Type the caption. Check the caption. Move to the next one.

I knew precisely how many hours that was, which is why the editor was one of the first things I ever built for myself. Not because editing is the most valuable thing I do, but because it was the most repetitive, and repetitive is what these systems are actually good at.

If you want the version of this argument that applies to your business rather than mine, I wrote it up as the difference between a normal AI and an agent. This piece is the specific machine.

Stage one: the footage files itself

I record on a camera, pull the card, and the footage gets filed automatically into a dated folder for that reel, alongside a small record of what was shot and where it came from. Nothing clever, but it's the stage everyone skips, and skipping it is why most people's footage lives in six places and none of them are findable in a month.

Everything downstream reads from that record, so the machine always knows which take belongs to which video.

Stage two: it listens to the whole take

The take gets transcribed with word-level timestamps, so the system doesn't just know what I said, it knows exactly when each word starts and ends. That timing is what makes everything after this possible. Captions that land on the syllable, cuts that don't clip a word in half, and the ability to find a specific sentence in three minutes of audio.

The tool doing this is whisperX, which is free and open source. It's one of five I lean on constantly, and I broke down what each one gives an AI in the five tools I plug into Claude Code.

Stage three: it decides what to cut

This is the stage that does the actual editing, and it runs three different passes.

Silence. It finds every stretch of dead air and removes it. On this video that came to roughly 173 seconds, which is more than the finished reel by a factor of five. The single longest gap was 46.9 seconds where I stopped, thought, and started again.

Retakes. When I say a line twice, the later version wins. That's a rule I set, not something it worked out, and it exists because the second attempt is almost always the better one. On this reel it dropped two.

Judgment. This is the interesting one. It looks for lines I started and abandoned, or said three ways, and picks the keeper on the meaning rather than the audio. On this video it found the taste line, which I attempted three times. The first attempt was complete and fine. The second trailed off mid sentence. It kept the third, because that was the only one that carried the full contrast I had written, the bit about the machine being fast but having no taste.

It also caught something I would probably have missed. Near the end I said "and if you want a full breakdown of this machine, comment the word" and then trailed off without ever saying which word. If that had shipped, the whole call to action would have been broken. It kept the later attempt that finishes the sentence properly.

Eighteen cuts in total, twelve segments kept, 209 seconds down to 34.9.

Stage four: it writes the captions and checks its own audio

Captions get built from those same word-level timestamps, so each one appears as I say it rather than as a block. Then an audio QC pass runs over the result looking for the failures that are easy to miss on a first watch: words stretched oddly by a cut, gaps that landed in the wrong place, sentences that got clipped. On this video that pass came back clean across the five windows it checked.

Stage five: it builds the visuals and renders

A scene plan decides what's on screen for each beat. Then the whole thing gets rendered in code, not dragged around a timeline. Captions, graphics, transitions, music and sound effects all come out of one render, which is why every reel on my account looks like it came from the same place. It did.

What it got wrong on this exact video

Here's the part most build-in-public posts leave out.

Three captions were wrong and had to be corrected. All three are worth understanding, because they show you the actual shape of the failure rather than a vague "AI isn't perfect".

1. It heard "my job strings to just one thing". I said "shrinks".
2. It heard "watch every single card before it gets published". I said "cut". A mishearing that is almost reasonable, and completely wrong in a video about video editing.
3. On two lines, including the call to action, the opening words got stamped onto the abandoned attempt rather than the one that was kept. So the cut was right, but the caption arrived missing its first few words.

That third one is the failure mode worth knowing about, because it isn't a mishearing at all. The transcription assigned words to a segment that later got deleted, so the caption lost text that the audio clearly contains. No amount of a better model fixes that on its own. It needs something checking the captions against the segments that actually survived.

So the honest version of the claim is this. The machine did the editing. A check ran after it, and the check found three things.

The job I kept

I watch every single cut before it goes out.

That's the whole of my remaining job on the editing side, and I do not intend to give it up. The machine is fast and it has no taste. It cannot tell that a line is technically fine and lands flat, or that a joke needed the pause it just removed, or that I look like I'm about to say something more.

There are three points where it stops and waits for me: after the rough cut, after the captions, and before anything goes out. Those exist because I put them there. An editing system with no stop in it is just a faster way to publish something you have not watched.

What this actually replaced, and what it did not

It replaced the hours. Finding the sentence, trimming the pause, typing the caption, checking the caption. All of that is gone and I do not miss any of it.

It did not replace the decisions. What to make, what the hook is, whether the take is any good, whether the thing is worth publishing at all. Every one of those is still mine, and I think that split is going to hold for a while.

The result is that raw footage goes in one end and a finished reel comes out the other, every single video on this account, and my involvement shrinks to filming it and watching it. On a good day that's the difference between a reel taking twenty minutes of my time and taking three hours.

If you run a business and you're wondering what the equivalent is for you, the honest test is not "what could AI do". It's narrower than that. Find the job you do every week where you already know every step, where you could write the instructions down without thinking, and where you would recognise a bad result instantly. Editing was mine. Yours is probably something duller, and duller is better, because the boring jobs are the ones with clear finish lines.

The stage-by-stage machine is not the impressive part. The impressive part is that it runs whether or not I remember it exists.