When I first looked into removing people and equipment from video, temporal consistency seemed like the hard part. I expected to spend most of the time on optical flow, selecting source frames, and keeping generated textures from flickering over time.

The first real bottleneck appeared one stage earlier. It was surprisingly difficult to produce a single reference frame that I could trust.

In Video Inpainting, a background pixel exposed in at least one other frame can be propagated as a known pixel. A region hidden throughout the entire shot has no such evidence; a generative prior has to invent it. If the reconstruction in the reference frame is wrong, the later stages can propagate that mistake with impressive consistency. A fabricated brick pattern may remain perfectly stable across time and still be the wrong background.

Image Inpainting was therefore not a standalone beautification step in this pipeline. It produced the background hypothesis that the video stage would inherit. Plausibility mattered, but so did a stricter condition: did every pixel that was not supposed to change remain where it was?

A Mask Does Not Necessarily Define a Pixel Boundary

I started with general image editors such as Nano Banana and GPT Image. Give them an image, a rough region, and a request to remove an object, and the first result can look excellent. They recognize large objects and fill the opening with a convincing background. In a single-image comparison, they can even appear stronger than a purpose-built Inpainting model.

The problem surfaced when I tried to composite the result back onto the source. A wall changed tone outside the mask. An object moved by a few pixels. A straight edge became slightly longer or acquired a different perspective. To a person, it was nearly the same picture. To the pipeline, it was a newly rendered picture that no longer aligned with the original.

That led to a simple question: if gpt-image-2 accepts a mask for an edit, does it leave every pixel outside that mask completely untouched?

The proprietary implementation is not public, so there is little value in speculating about its internal graph. What matters in production is the API contract. The official OpenAI documentation describes GPT Image masking as prompt-based. The mask guides the edit; it is not documented as an exact compositing boundary, and the model may not follow its shape with complete precision.

For a production caller, the answer has to be no. A mask input in gpt-image-2 is not a documented guarantee of bit-identical preservation outside the mask. A particular output may happen to preserve those pixels, but the API does not require it to do so.

Nano Banana is described in even more semantic terms. The Gemini API documentation calls the feature Inpainting (semantic masking) and shows a user defining the region conversationally. The example asks the model to leave the rest untouched, but that sentence is an editing instruction, not a contract stating that the server copies every pixel outside a binary mask from the source.

Two different meanings had been hiding behind the same word, “mask”:

• semantic guidance that tells a model where and what to edit;
• a spatial contract stating that pixels outside a boundary must come from the source.

General editing APIs usually provide something closer to the first. Video compositing needed the second.

Visually Similar Can Still Be Unusable for Compositing

The same failure shows up in actual reports. In one gpt-image-2 mask-edit test posted to the OpenAI Developer Community, the author reported changes outside the mask, changes to objects that were not part of the request, and more regeneration than the edit required. The behavior remained even after the prompt explicitly asked the model to preserve the walls, floor, lighting, perspective, composition, and shadows.

The attachments in that thread label the uploaded mask as 1920×1306 and the result as 1521×1034. Those two labels alone do not prove that the API always changes resolution; an upload path may also transform an attachment. They are enough to show why a production pipeline cannot silently assume that the response shares the source coordinate system. Output dimensions have to be requested and verified. Even matching dimensions do not guarantee that a building edge or a piece of furniture remains at the same coordinates.

I also downloaded the original mask-api.png and composition-image-2.png attachments from the same thread. This gives us a pair in which the black selection is present on both sides. I resized the 1521×1034 result to match the 1920×1306 masked input with Lanczos, then computed the RGB absolute difference without registration. The delta alone is amplified 4× so that small changes remain visible.

Even with those caveats, the difference is not confined to the selected region. It appears across the brick wall, stairs, lighting, and furniture edges. At minimum, this response cannot be treated as a local patch that drops straight into the source coordinate system. The figure visualizes an observed output; it does not prove how the proprietary API works internally.

Misalignment has also been reported around Nano Banana. In an Adobe Community post, a user working with Gemini 2.5 through Photoshop Generative Fill described a generated layer that arrived at a different scale or shifted by a few pixels, making it unusable for precise work.

A few pixels are easy to dismiss in a standalone image. Across video frames, they create a moving boundary, doubled edges, and propagation inputs whose geometry is already wrong. A plausible new image and a patch that can be pasted onto the source are different deliverables.

Why Not Composite the Original Back Outside the Mask?

The simplest defense is to discard the generated pixels outside the mask.

$$I_{out}=M\odot I_{edit}+(1-M)\odot I_{src}$$

This construction makes the pixels in $1-M$ mathematically identical to the source. It is a useful baseline and a sensible safety measure.

Object Removal makes the boundary less convenient. A silhouette mask may leave the person's cast shadow outside the edit. A reflection can remain in glass or metal. Expanding the mask to include those effects may also include a foreground table, a caption, or another person that must be preserved. If the generated image is geometrically offset, the hard composite introduces a seam along the boundary.

The task was no longer simple hole filling. A model had to identify the object, find the shadow and reflection associated with it, and preserve foreground content in overlapping regions. The input was a 2D image and a mask, but the decision required a form of depth awareness.

Unpredictable Completion Time Hurts More Than a Slow Average

Latency also matters when a general generation API becomes a batch stage. The OpenAI documentation notes that complex prompts may take up to two minutes. The same section warns that precise placement can remain difficult in structured or layout-sensitive compositions.

Gemini users have also reported large service-dependent variance. One Google AI Developers Forum post describes a period when image-generation time increased from 30 seconds to three minutes while repeated 429 failures occurred. That is a time-specific operational report, not the normal latency of the model. Its relevance is that production capacity cannot be planned from an average alone.

A slow but perfectly aligned response can still be processed in a queue. The expensive case is waiting more than a minute, running a pixel and geometry check, then sending the job again because the first output cannot be composited. Inspection and retry time eventually dominate the raw API call.

Treating Photoshop as an Internal API

There was a purpose-built option available to us. The broadcaster already had an Adobe enterprise account, and Firefly-backed Generative Fill in Photoshop produced results that were much closer to practical object removal. Under the contract we were using, however, access to the desktop feature and access to a separate development API were different products. API use required an additional enterprise discussion.

For a while, I considered leaving Photoshop open on a dedicated workstation and exposing it to other machines on the internal network. A controller could open an image, run Generative Fill on a selection, save the result, and make the whole setup look like an ordinary API from the outside.

It was possible, but not a comfortable service boundary. Fully headless automation remained tied to Photoshop's UI and application state. A dialog or update could stop the worker. Replicating the workstation still left licensing, state, file I/O, and load balancing to solve. Most importantly, the workstation would become a single point of failure. Having access to a good model was not the same as having an operable backend.

Narrowing the Search to Models with Code and Weights

I then limited the candidates to object-removal models whose code and weights were actually available at the time.

This is not a paper leaderboard. It is a record of which outputs I could pass to the next video stage on the same set of broadcast-production samples. The models use different backbones, recommended resolutions, and sampling schedules, so calling it a controlled quantitative benchmark would be misleading.

Under those conditions, OSOR was the best fit. It provides both SDXL Inpainting and FLUX Fill variants, and its one-step design made it fast. The paper and repository report sub-second processing for a 1024×1024 image on a single A100. More important for this workflow, it often preserved the surrounding image while removing a cast shadow that was not perfectly covered by the object mask.

Other candidates sometimes removed only part of the target or generated a similar object in the opening. OSOR was not universally correct, but it was fast enough to inspect and retry, and it was the most practical default for the recurring people-and-equipment cases in our material.

The Scenes OSOR Could Not Remove Had Something in Common

Once OSOR became the baseline, its failure cases became easier to describe.

The first group involved defocused backgrounds. A staff member or camera outside the focal plane already has weak boundaries and collapsed texture. A person reads it as “a camera in the back,” but the pixels resemble a smeared block of color. When the blur itself becomes part of the background prior, the model may leave part of the object or replace it with a different, unidentifiable blur.

The second group involved complex artwork and set decoration. Repeating wallpaper or floorboards provide a local rule that a model can extend. An object covering the middle of an avant-garde painting hides a region with no repeatable rule and no unique answer. A powerful generator can produce a plausible different painting. That is not the same thing as reconstructing the artwork that was actually behind the object.

The third group exposed what a mask cannot say. A broad mask drawn around a person in the background may also cover a foreground prop or a caption. A conventional Inpainting input identifies which pixels may be redrawn, but not which object inside that region must disappear and which one must remain. The ambiguity occurs in an image, yet it depends on depth and object identity.

The training data problem points in the same direction. Pasting an object onto a background creates perfectly aligned before-and-after pairs at scale. It struggles to reproduce real defocus, contact shadows, reflections, partial occlusion, and foreground overlap. Photographing a scene before and after physically moving an object preserves those effects, but it is expensive to scale and difficult to keep lighting and dynamic backgrounds perfectly fixed.

It would be too easy to blame every failed sample on OSOR. The training prior, the input mask, and the scene's depth ambiguity may all be involved. The image is also only an intermediate result. Some errors grow through video propagation, while other uncertain regions may be corrected by observations from adjacent frames. The meaningful evaluation is ultimately the end-to-end Video Inpainting output, not a handful of isolated stills.

The Choice I Made for the First Stage

In the current pipeline, I do not treat a general image-editing API as a drop-in Inpainting API for compositing. That is not a claim that gpt-image-2 or Nano Banana is a weak model. They are often better at understanding a natural-language change and reorganizing the whole scene around it. They simply require an additional verification and post-processing layer when pixel identity and source coordinates are part of the requirement.

OSOR became the default Object Removal path because it offered the best balance of speed, outside-region preservation, and effect removal among the candidates I tested. Before a result enters the video stage, I normalize it to the source dimensions and check unmasked pixel differences and edge alignment. Hard compositing remains a safety measure, but I do not assume that the object mask is the complete effect mask.

Complex paintings, defocus, and occlusion still revealed the ceiling of mask-only Inpainting. That led me to MaskFlow, which is built on Qwen Image Edit. On several cases that defeated the removal-specific models, the general editing model produced a markedly better reconstruction.

The question then changed.

Which should we trust: Inpainting that constrains the edit region, or Editing that can name the object to remove?

The next part will look at MaskFlow and at why pixel locality and semantic control do not trade places cleanly.