Consider a fairly ordinary request to a broadcast archive.

“Find a scene aired after 2020 where presenter A is eating and talking with someone outdoors. Ideally it was filmed in Seoul. I need a clean version that we can reuse online.”

Nothing about this sounds particularly exotic to a producer. To a system, though, it is several problems packed into one sentence. Who should it look for? Over what interval do the people actually eat together? How does it distinguish somewhere that looks like Seoul from a location confirmed to be in Seoul? Where is the corresponding clean version, and who can establish that it is available for online reuse?

Video embeddings can retrieve similar scenes. A high similarity score does not establish the location, identity, or rights. Exact filters for dates and names do not necessarily capture the relaxed conversation the producer had in mind, either.

Ontology often enters the discussion here. Connect programs, episodes, people, scenes, files, and rights, and an agent should be able to navigate the systems to find what it needs. I agree with the direction. But when I look at those diagrams, another question tends to come first.

Who drew that arrow, and based on what? How would we know if it were wrong?

This article starts with the basics of ontology, using broadcast archives as the running example. It separates what a system can represent from what it can actually establish, then asks why even evaluating the benefits of adoption is difficult. The programs, people, and time intervals below are fictional examples. The numerical illustrations are not measurements from a broadcast deployment.

What an ontology defines

An ontology starts by making explicit what kinds of things exist in a particular domain, what they mean, and how they relate. Even the word “video” needs some agreement. Are we talking about a program, an episode, or a file sitting on a disk?

Consider a person holding a cup at a table. A viewer sees a scene. An archivist may focus on the performer and filming location; an editor needs an interval and its source file. They are looking at the same picture, but different entities matter to their work. Building an ontology means agreeing on which distinctions to record and how to connect them.

An ontology is therefore not a copy of reality. It is a model of selected aspects of reality for a purpose. A props inventory might identify each individual cup. A person-search system may have no reason to do so. Greater detail does not automatically make the model better.

Noy and McGuinness’s Ontology Development 101 describes defining concepts and properties, then populating them with individual instances. It also recommends beginning with competency questions: questions the knowledge base should be able to answer. The starting point is not a diagram of every conceivable relationship.

The terms are easier to distinguish with a broadcast example.

These are not mutually exclusive boxes. A good database schema already contains domain meaning, and an ontology can include individual instances. Separating definitions from populated data is still useful: it tells us where a problem belongs.

Distinguishing kinds from individual things

Person, Segment, and Place in the illustration are classes. Person A, segment 42, and Seoul are instances: particular entities belonging to those classes. Defining what “person” means and recording a particular person are different layers of the work.

A class is more than a folder for records. It carries a meaning for membership. In this simplified model, Presenter can describe people who have worked as presenters, making it a subclass of Person. Recording who hosted a particular episode would still require a separate, episode-specific role. How precisely we represent occupations, roles, and periods is itself a modeling choice.

Three relationships are especially worth keeping apart.

Everyday language can make all three sound like “belonging to” something. They are not interchangeable. A segment being part of a broadcast version does not make it a subclass of that version. Person A is not a subclass of Presenter, either. Confusing these relationships changes what can be concluded from the data. RDF Schema's subclass and type represent the first two distinctions; a part relationship needs its own domain definition.

Distinguishing relationships from values

Segment_42 depictsPerson Person_A connects two entities. Segment_42 startMs 751000 attaches a numerical value to an entity. OWL distinguishes object properties, which connect individuals, from data properties, which connect individuals to data values. OWL 2 Structural Specification

The difference is more than formatting. Storing a location as the string "Seoul" gives us a display name. Connecting it to the entity Place_Seoul lets us reuse that entity's other names and administrative relationships. Of course, recognizing the word “Seoul” in a caption does not justify asserting that filming occurred there. This example concerns representation, not verification.

Names and identifiers should also remain distinct. A stage name, a legal name, and an English spelling can label the same person ID. Identical names should not automatically merge different people. Shared IDs matter because systems need to refer to the same entity, not merely group strings with matching spellings.

What an axiom lets us infer

Beyond listing concepts and relationships, an ontology can state axioms: assertions that are taken to hold within the model. “A presenter is a kind of person” is one example. Combined with a record that A is a presenter, it lets the system also treat A as a person.

General definition
  Presenter —subclass-of→ Person

Statement about an individual
  Person_A —instance-of→ Presenter

Consequence of the two statements
  Person_A —instance-of→ Person

The system has not recognized A's face again. It has made a consequence of the supplied facts and definitions explicit. If this inference is reflected in queries, a request for people can include A even when A was recorded only as a Presenter. A reasoner can handle this, or queries can expand the relevant relationships, depending on the implementation.

Relationship direction can carry meaning too. Declare depictsPerson and appearsInSegment as inverses, and “segment 42 depicts A” entails “A appears in segment 42.” Relationships do not acquire this behavior merely by having suggestive names. Their meaning must be declared and used by the system. OWL 2: Inverse Properties

General definitions are often discussed as the TBox, and statements about particular individuals as the ABox. Assertions about individuals can also be axioms; the word is not restricted to general rules. The useful distinction is between the definition layer and the data layer that uses it. OWL 2 Primer

Relationship definition
  Segment —depictsPerson→ Person

Individual statements
  Segment_42 —depictsPerson→ Person_A
  Segment_42 —startMs→ 751000
  Segment_42 —endMs→ 766000

RDF represents statements as subject–predicate–object triples. Segment_42 / depictsPerson / Person_A is one triple. Statements sharing entities form a graph. Crucially, these are claims about the world. Encoding a claim as RDF does not verify it. RDF Concepts

What changes when we search?

Return to “find the clean version of a scene featuring presenter A.” The system must find A's appearances, identify the relevant editorial version and episode, then follow the correspondence to a clean version and its media file. Shared meanings for person, segment, episode, and version let the identifiers returned by different systems fit together.

Following stored relationships is not necessarily logical inference. Joining records to find a clean version is a query. Deriving that A belongs to Person from a subclass definition is inference. Using an ontology does not require elaborate reasoning for every search. Consistent terminology, linked identifiers, and stable query rules can already be useful.

Nor does an ontology automatically fill missing arrows. Without interval mappings, the system cannot export the corresponding clean-version interval precisely. Without location evidence, it cannot establish Seoul as the filming location. An ontology can make the question more precise; it does not supply the missing data that would answer it.

Even “the same video” needs a definition

The first difficult question is often not sophisticated inference. It is deciding what counts as the same thing.

Suppose episode 12 has a broadcast master, a clean version without captions, a low-resolution proxy, and an online recut. They all relate to the same episode. They are not the same file, nor necessarily the same editorial result. A proxy may be a different encoding of the same edit. An online recut may have different scene ordering, music, and duration. Collapsing everything into “the same video” can make search look simpler while breaking the eventual export.

I would separate at least the following. This is a minimal model for the discussion, not a transcription of any particular standard.

Program → Episode → EditorialVersion → MediaFile
                         │
                         └→ Segment [start, end)

Broadcast version → master MXF / proxy with the same timeline
Clean version → clean MOV
Online recut → web MP4

A segment needs to identify its editorial version and timeline. The broadcast master’s 12:31 is not guaranteed to be the same scene at 12:31 in the clean version. A slate, advertisement, or removed shot can shift the position. Mapping intervals between versions is another piece of data. If the mapping is uncertain, that uncertainty needs to survive as well.

People require similar care. A performer, a fictional character, a detected face, an audio speaker, and a person mentioned in a caption cannot all be represented by one undifferentiated relationship. Actor A plays character B, while another actor mentions B. What should a search for “videos with B” return? The answer depends on whether the user means the real performer or the fictional character.

The original request also needs more than two names and an “eating” tag. A’s meal and B’s interview could appear in different scenes of the same episode. That does not mean they ate together. One option is to model the meal or conversation as an event, with participants, intervals, and supporting evidence. Temporal overlap is a useful clue, but cross-cutting means it is not proof of interaction on its own.

Broadcasters do not have to invent every concept from scratch. EBUCorePlus provides an ontology for media enterprises, following EBUCore and CCDM, with a public repository for inspecting its concepts and relationships. Adopting a standard is still different from correctly mapping internal episode numbers, file IDs, production notes, and rights records. A shared vocabulary does not fill gaps in an organization’s data.

Logically valid is not the same as factually correct

Constraints and inference create an understandable expectation that an ontology will catch bad data. It can catch some problems. We need to be precise about which ones.

Suppose the domain of depictsPerson is Segment and its range is Person. RDFS domain and range declarations do not behave like a form that rejects an invalid input type. They support inferring that the relationship’s subject and object belong to those classes. RDF Schema: Domain and Range

Conditions such as “a segment must identify its reference version” or “the start must precede the end” can be checked through validation. SHACL serves this role for RDF data. But a perfectly well-formed statement that segment 42 depicts person A can still be wrong: the person in the footage might be B.

Identity links deserve particular caution. owl:sameAs does not mean “looks similar” or “probably the same person.” It asserts that two identifiers refer to the same individual. Under that semantics, information about either applies to the other. OWL 2: Individual Equality

A face cluster is not itself a person. It is a data object grouping observations. Even after identification, it is better to connect the cluster to the person it depicts than to declare the two objects identical. Before verification, store an identity candidate separately. Asserting sameAs and attaching a confidence of 0.82 does not make an ordinary OWL reasoner apply identity at 82% strength.

Then there is absence versus uncertainty. Under OWL’s open-world assumption, a missing statement is not necessarily false. No record of A appearing is different from evidence that A does not appear. OWL 2 Primer

An application can deliberately treat a particular table as complete and query it under a closed-world assumption. That scope must be explicit. Treating an unanalyzed scene as “no person present,” or a file with missing contracts as “no restrictions,” creates very different kinds of mistakes.

Agents need shared meaning, not necessarily a graph database

An agent that searches an archive, locates a file, and calls an editing tool needs the systems’ meanings to line up. How does a search engine’s clip ID map to the MAM’s asset ID? Does “clean” mean no captions, or no music either? Which timeline does the editing tool expect?

Without an explicit agreement, the agent must repeatedly infer these meanings from tool descriptions and documents. An ontology moves some of that interpretation into shared definitions and relationships. The model no longer has to reconstruct every API’s business meaning on its own.

That does not make RDF, OWL, or a graph database mandatory for every agent. A bounded workflow may be well served by a relational schema, common IDs, narrowly scoped tools, and policy checks. Explicit ontology becomes more valuable when departments use different names for the same entities and workflows repeatedly traverse complex relationships. The graph database is a storage and query mechanism. The ontology addresses what the stored relationships mean.

It is worth reading GraphRAG with that distinction in mind. The paper extracts entity relationships and builds community summaries to answer questions about an entire document collection. Its results do not establish accurate person identification in broadcast footage or safe reuse decisions. Summarizing a collection’s themes and clearing a particular 15-second clip are different evaluation tasks.

“Reusable” is especially difficult to treat as a permanent property of a video. The relevant conditions depend on the version and interval, channel, territory, time, and intended use. Music or inserted footage may introduce additional conditions. The ODRL Information Model can represent permissions, prohibitions, duties, and constraints. It does not discover missing contracts or replace the process that establishes applicable rights.

I would therefore separate candidate retrieval from action authorization. An agent can surface an uncertain match for review. Before export or distribution, the system should check authoritative records against the current request. Unknown conditions should trigger clarification or review; a prohibition should block execution.

Populating the relationships is the hard part

Defining Segment depictsPerson Person is much easier than populating it accurately across a large historical archive. This is where we must decide how far an automated observation can travel before being treated as fact.

Take OCR. An entertainment broadcast may contain dialogue subtitles, name labels, reaction captions, signs, and a station logo in the same frame. Correctly reading the characters is not enough. A name could identify someone on screen, or announce the next guest. Useful output therefore includes location, display interval, caption type, and the entity it is supposed to describe—not just a string.

For faces, detection, tracking or clustering, and named identification are distinct steps. A face exists; these observations probably belong together; this person is A. Those are different claims. ASR has a similar boundary: obtaining words is not the same as attributing them to a named speaker. The visible person need not be the one speaking.

Object detection and scene descriptions can extend meaning even more quietly.

This table is not a benchmark of a model’s failure rate. The point is that an observation may be correct while the next claim still needs additional evidence. Contracts and production intent are not necessarily recoverable by watching more frames.

A VLM can hide these boundaries in a fluent sentence: “Two people are drinking coffee and talking at a café in Seoul.” Person count, venue type, drink identity, geographic location, and interaction have all arrived together. Splitting that sentence into several confirmed relationships can make a single interpretation look like carefully established data.

At extraction time, I care less about how elaborate the description is than whether we can return to what was observed and what was inferred. If someone corrects the location, the system should be able to find the original description and the relationships derived from it.

Accuracy is not the only constraint

Finding brief captions and short actions requires adequate temporal sampling. There is a limitation here before model quality enters the picture.

Consider a simplified example. The pipeline samples one frame every $\Delta$ seconds. A subtitle remains visible for $d$ seconds, with its starting phase uniformly distributed within the sampling period. Ignoring video-boundary effects, the probability of sampling it at least once is:

$$P(\text{sample hits subtitle}) = \min\left(1, \frac{d}{\Delta}\right)$$

If the interval is two seconds and the caption lasts half a second, that probability is 25% in this toy model. Even perfect OCR has no opportunity to read a frame it never receives. This is not a measured accuracy figure; it separates sampling misses from recognition errors.

Denser sampling increases decoding and inference work. Tracking, multimodal cross-checking, and human review add further costs. In my earlier article on the Video Retrieval pipeline, decoding and failure handling outside the model also affected end-to-end throughput.

An arithmetic example helps establish scale. Suppose an imaginary archive contains 100,000 hours, each worker processes the entire analysis pipeline at 20 times real time, and four workers run with perfect parallel efficiency:

$$T_{\text{wall}} = \frac{100{,}000\ \text{h}}{20 \times 4} = 1{,}250\ \text{h} \approx 52\ \text{days}$$

That excludes I/O contention, retries, and human review. It is not an estimate for a particular model. It shows why the cost of a few demonstrations is a poor guide to the cost of processing an archive.

This does not establish an immutable accuracy–speed trade-off. Shared preprocessing, adaptive sampling, and better models can improve both. But under a resource budget, we still need to know what information was omitted and which workflows depend on it. Cost per video hour should be accompanied by review time and cost per verified, useful relationship.

Can one confidence score represent trust?

A face model’s 0.9, an OCR score of 0.9, and an LLM’s written “90%” are not interchangeable. Without calibration against evaluation data, matching numerical scales do not justify comparing or averaging them. Two models reading the same caption do not necessarily supply two independent pieces of evidence, either.

Calibration asks, for example, whether predictions assigned a confidence of 0.9 are actually correct about 90% of the time. Guo and colleagues’ On Calibration of Modern Neural Networks examines the distinction between classification accuracy and calibrated confidence. Its methods do not automatically calibrate every relationship type across different broadcast genres and historical periods.

I do not think attaching a confidence score to an edge is enough. A relationship should be managed as an assertion with a source and a state.

These dimensions should not be flattened into one status list. A statement can come from an official document and be human-verified. A human-verified statement can later be withdrawn. An official document does not automatically make every OCR value extracted from it authoritative.

An identity candidate for a face track might begin with a record like this. The IDs and score are fictional, and this is an illustrative application record, not a standard serialization.

{
  "claimId": "identity-019",
  "subject": "face-track-81",
  "predicate": "candidateIdentity",
  "candidateObject": "person-A",
  "reviewStatus": "unreviewed",
  "evidence": {
    "mediaFileId": "episode-12-broadcast-proxy",
    "editorialVersionId": "episode-12-broadcast-v1",
    "timebase": "milliseconds-from-file-start",
    "startMs": 751000,
    "endMs": 766000
  },
  "generatedBy": "identity-pipeline-v3",
  "rawModelScore": 0.82,
  "calibrationId": null
}

This candidate record must not silently become a confirmed identity link. In practice, the lineage should also identify model weights, prompts, preprocessing versions, reference faces or documents, and review history. W3C’s PROV-O provides a foundation for expressing provenance among data, processing activities, and responsible agents. It does not calibrate confidence or supply the organization’s approval criteria.

Evidence is needed for more than reviewing an individual result. When an incorrect identity link is withdrawn, we need to find the indexes, summaries, and caches that used it. Removing a relationship and undoing its effects can be harder than adding one.

Metadata needs maintenance after ingestion

Models change, schemas change, and business conditions change. Calling all of this “stale metadata” makes it difficult to decide what should be reprocessed.

A new face model does not automatically invalidate a historical appearance already checked by a person. An expired usage condition may require a rights update without reanalyzing any faces. Splitting appearsIn into visual and audio appearances changes the meaning of existing data and the queries that consume it.

Standards are not exempt. EBU’s TECH 3397 v2 notice explicitly identifies a backward-compatibility break. Choosing a standard gives us a reference for managing change; it does not eliminate change.

Separating raw observations from derived relationships helps. Original media and documents, model observations, normalized candidates, and currently adopted statements should remain distinguishable. Some corrections can then be made by updating a mapping or review decision instead of rerunning the entire analysis pipeline.

There are also at least two relevant clocks. When was a condition valid in the business world, and when did the system learn about it? If a change effective September 1 is entered on September 3, auditing a September 2 decision requires both dates. A model execution timestamp cannot answer that question on its own.

This is not an argument for retaining every intermediate artifact forever. Face references, speaker information, and contracts need access controls and retention boundaries. Corrections or deletion requests may need to propagate to derived relationships and search indexes. The evidence collected to make a system trustworthy creates its own maintenance obligations.

Without an inventory, the graph may be premature

For an archive with little structured metadata, a file inventory may matter more than an ontology initially. What exists, where is it stored, which files are readable, and how are the versions related?

Clues may be scattered across filenames, media headers, EPG records, subtitle sidecars, cue sheets, editing-project data, and production spreadsheets. Finding a cast list or location record may be more useful than asking a vision model to infer the same information again. Matching checksums can establish exact file duplication; visual resemblance should not collapse distinct editorial versions.

Nor does every observation belong in a graph database. Media can remain in a MAM or object store, dense tracks in relational or columnar storage, and transcripts in full-text search. A vector index retrieves semantically similar candidates; relational queries narrow identities, versions, and periods. Assertion management does not have to be a separate product. Distinguishing logical responsibilities is not the same as operating six independent services.

Agreement between departments may take longer than the technical implementation. Who confirms a person ID? Which source takes priority when a production document contradicts an archive entry? Who reports rights changes? Which tools need updating when the definition of “clean version” changes?

Sharing business meaning also means agreeing on ownership and change authority. Distributing a schema file does not create that agreement.

Why the benefits are difficult to evaluate

After the work of building it, evaluation might seem straightforward: compare performance before and after. But the same questions reappear. What is the correct answer, and what exactly improved?

Ground truth requires archive knowledge too

Usually, nobody knows how many scenes across the entire archive satisfy the opening request. We can inspect the top 20 results and judge how many fit. Measuring corpus-wide recall requires knowing how many relevant scenes remain hidden in thousands of unreviewed hours.

Pooling candidates from several retrieval methods and having people judge them is a practical starting point. Material outside the pool is still unjudged. Automatically counting an unjudged result as wrong can penalize a new method for discovering a useful scene. The evaluation needs to document the scope of its ground truth and its treatment of unknowns.

This predates agents. Buckley and Voorhees’s Retrieval Evaluation with Incomplete Information examines how incomplete relevance judgments affect retrieval metrics. For broadcast archives, that incompleteness must remain part of how we interpret the scores.

Assessors may also disagree for legitimate reasons. One wants any outdoor conversation; another needs usable reactions and enough frames around the cut. Semantic relevance, editability, and reuse conditions should be assessed separately. Otherwise, we cannot explain the disagreement. An absent contract is also different from an irrelevant scene.

Even the ontology does not have one uniquely correct shape. A conversation could be represented as one event or a sequence of turns. Node and edge counts will differ. Neither representation is automatically more accurate; the question is whether it supports the required tasks.

Improvement is not necessarily an ontology effect

Suppose the old system indexed filenames, while the new one adds ASR, face recognition, VLM descriptions, and a graph. Better search would not establish that the ontology caused the improvement. Richer metadata, normalized identities, and relationship reasoning have changed together.

Where feasible, I would separate the comparisons as follows. This is an evaluation proposal, not a results table.

Apart from the factor being tested, match the models, candidate counts, reviewed evidence, UI, and response budgets as closely as possible. Testing the graph store itself also requires a comparison implementing the same semantics and rules in SQL. If the experiment instead measures the whole adoption package, report that combined benefit without assigning it to a single component.

Data splits matter as well. Adjacent scenes from one episode, or the same identities on both sides of a tuning/evaluation split, can reuse already-resolved identities and nearly identical shots. Retrieval for known people and generalization to new people, programs, or low-quality historical footage should be evaluated separately.

A small relationship error can have a large operational effect

Imagine a face cluster incorrectly assigned to person A and used across hundreds of scenes. The identity error is one event, but it appears repeatedly in search results, statistics, recommendations, and summaries. A low error rate across all triples does not describe that impact well.

Conservative extraction creates a different problem. Precision can rise when the system only commits to easy relationships. Returning “unknown” for every difficult request also reduces wrong answers. Neither helps much if the user must repeat the entire search manually. We need coverage—the proportion of requests handled automatically—alongside error rates, plus the human effort required to resolve abstentions.

Tasks with asymmetric failure costs need separate accounting. An incorrect reuse approval, an unnecessary exclusion of usable footage, and an unresolved case are different outcomes. Observing no incident in a small test does not establish the absence of rare, high-cost errors.

Does an LLM judge solve the problem?

LLM-based scoring can help review scene descriptions and answers. But asking a model whether another model’s description sounds valid is not the same as checking the original evidence.

The LLM-as-a-Judge study discusses scalability alongside position, verbosity, and self-preference biases. That does not make model judges useless. For broadcast evaluation, I would give them the source intervals and necessary records, blind the system identities, vary presentation order, and analyze disagreements with human assessments.

Plausibility, evidential correctness, and clip usability should remain separate scores. A citation link is not sufficient, either. We need to check which interval supports which claim.

What should count as a benefit?

Time to the first search result matters less than time to obtaining a usable clip. Faster retrieval may have shifted work into review and correction. Conversely, similar retrieval accuracy can still have operational value if incorrect identities are much easier to trace and fix.

Collapsing these into one score can hide expensive failures behind a favorable average. It is better to establish acceptable error and abstention levels for each workflow. User studies also need to account for learning: someone who already found a scene with one system will have an advantage when trying the other. Comparable but different tasks, or counterbalanced ordering, can reduce that effect.

Even this evaluation is incomplete. Rare incidents, new program formats, and changing user requests will not all appear in a short proof of concept. “Difficult to evaluate” does not mean “impossible to evaluate.” It means being specific about the scope of the benefits we have actually demonstrated.

Where I would start

I would not begin by assigning every scene an identity, location, action, product, and emotion. I would first collect recurring production requests and identify the decisions that must not be automated without further checks.

Establish files, versions, episodes, and timelines. Connect existing subtitles and production records. Build a text-and-video retrieval baseline. If person search fails repeatedly, add face tracks and identity review. If locating the clean version takes too long, improve version relationships and interval mappings first. Reducing one recurring workflow failure is more valuable than enriching a relationship nobody uses.

Expensive analysis can be applied selectively to frequently used or newly requested material. But enriching only the top search candidates creates a feedback loop: already-discoverable content becomes even easier to discover. Random and stratified inspection outside the candidate set should remain part of the process.

Historical archives and new productions also deserve different approaches. Historical work reconstructs scattered records. New productions can assign identifiers and connect cast, locations, editorial versions, cue sheets, and rights records while the work happens. Some information that would otherwise need AI inference can simply be recorded at its source.

Review and maintenance do not disappear. Better models may lower the cost of proposing relationships, but departments still need to agree on definitions, incorporate rights changes, and withdraw bad assertions. If nobody owns that work after the proof of concept, we have not finished calculating the adoption cost.

More relationships should mean clearer accountability

I do not regard ontology as unnecessary work. As agents combine data from several systems and begin taking actions, organizations have more reason to make their implicit meanings explicit: distinguish files from content, observations from interpretations, and evidence from permission.

Making relationships explicit does not remove uncertainty. It can also make an incorrect link easier to reuse and propagate. A dense graph gives the impression that the system knows a great deal. Sometimes it is repeating the same guess through several paths.

That is why I would not end an evaluation with a count of nodes and edges. Did the producer find a suitable scene? Could they actually use it? When something was wrong, could the system trace the cause and repair the consequences? The human time required to answer those questions belongs in the accounting too.

What I want is not a system that claims to know every relationship in the archive. It is a system that can explain what supports its claims, what remains unknown, and which actions are justified in that state. An ontology can provide a foundation for that explanation. It cannot supply the evidence or take responsibility on its own.