Introduction

Before introducing the paper, it helps to understand a limitation of existing vision–language models such as CLIP and ALIGN. MERU challenges their use of a Euclidean space, where the same geometric notion of distance applies everywhere. Without some familiarity with CLIP, the motivation can be difficult to follow.

Euclidean and non-Euclidean geometry use different distance relationships. In a flat Euclidean plane, a triangle's interior angles sum to $180^\circ$; on a sphere, they do not. Likewise, relationships among embeddings change according to how strongly their space bends—represented in hyperbolic geometry by curvature.

Fortunately, the paper does not bury the idea beneath pages of derivation. Its thesis is simple: image–text relationships are represented more appropriately by non-Euclidean, specifically hyperbolic, geometry than by an ordinary Euclidean embedding. Why?

Hierarchical Relationships Between Text and Images

Consider this image.

Ask $n$ people on the street to summarize it in one sentence or describe their reaction. Assuming they do not mistake you for a street recruiter and walk away, they will give different answers:

• “It is a picture of a dog and a cat.”
• “A cat is pouncing on a dog.”
• “What an adorable picture.”

People naturally understand an implicit hierarchy in how text describes an image. The first sentence identifies only semantic objects: dog and cat. It says nothing about their relationship. Even so, it describes the image correctly and should lie close to it in an image–text metric space.

The second sentence also defines a relationship between the objects. We cannot be absolutely sure from one frame that the cat is pouncing, but the interpretation is reasonable and should likewise remain close to the image. The green part of the figure represents this added relational information.

Finally, “What an adorable picture” contains little object-specific detail. Ignoring people who do not find cats and dogs cute, we can arrange the information in these expressions hierarchically:

As a description becomes more specific to an image, it occupies a more detailed semantic region beneath broader concepts. A Euclidean model compares all descriptions primarily through cosine similarity—the angle between vectors—and does not explicitly capture this hierarchy. The contrastive objective in conventional VLMs depends only on embedding similarity.

A Short Recap of Hyperbolic Space

Hyperbolic geometry has long been studied for non-Euclidean modalities. One influential paper learned hierarchical embeddings in the Poincaré disk or ball. A Poincaré model projects every point of a hyperbolic space onto a bounded surface. Once the projection relationship is defined, embeddings can be analyzed on that surface.

The advantage becomes clear for graphs, a representative non-Euclidean modality whose embeddings suffer geometric distortion (helpful explanation). Curvature Regularization to Prevent Distortion in Graph Embedding provides a good overview.

Take the graph in A. Beyond classifying blue and red nodes, an embedding must represent which nodes are connected. Ordinary graph objectives often preserve proximity: adjacent nodes should have similar embeddings. They do not explicitly preserve relationships among distant nodes or the graph's global branching structure. Through diffusion-like aggregation and repeated random-walk effects, representations can drift toward a rounded Gaussian cloud like B1.

The local mapping is not necessarily wrong, but it distorts long-range relations. Geodesic distance and graph shortest-path distance remain positively correlated in both B1 and the oracle-like B2 mapping (C), whereas Euclidean distance loses that trend (D). A perfect oracle embedding is unavailable to a learned model. Hyperbolic Riemannian learning changes the problem by replacing an impossible objective-level guarantee with a geometric constraint whose space naturally supports tree-like growth.

Riemannian Manifolds

A smooth manifold is easiest to imagine as a flexible sheet. Lying flat, it is a Euclidean manifold; bent, it lives in a more general ambient space. Although the full sheet may curve, every sufficiently small neighborhood is locally Euclidean, much as a differential approximates a smooth function around one point.

A Riemannian manifold needs two things: a smooth manifold $\mathcal{M}$ and a metric $g$ that measures lengths on it. If $g$ is the ordinary Euclidean inner product, the geometry is Euclidean. Conventional CLIP training can be viewed as learning an implicit manifold while fixing metric $g$ between image and text embeddings to the Euclidean inner product—implemented through cosine similarity.

MERU instead uses a hyperbolic space, a manifold of constant negative curvature. It adopts the Lorentz, or hyperboloid, model, defined as a surface in one additional ambient dimension.

Hyperbolic Space in the Lorentz Model

To visualize the Lorentz model, rotate a hyperbola around its symmetry axis and keep the upper sheet. That extra axis is conventionally called time, while the remaining $n$ axes are spatial. A vector is written

$$[\mathbf{x}_{\text{space}},x_{\text{time}}],$$

where $\mathbf{x}_{\text{space}}\in\mathbb{R}^n$ and $x_{\text{time}}\in\mathbb{R}$. Bold symbols denote vectors; italic symbols denote scalars.

The Lorentzian inner product differs from the Euclidean one by the sign of the time component:

$$\left\langle \mathbf{x}, \mathbf{y} \right\rangle_\mathcal{L} = \left\langle \mathbf{x}_\text{space},\mathbf{y}_\text{space} \right\rangle - x_\text{time} y_\text{time}.$$

The Lorentz norm follows as

$$\lVert\mathbf{x}\rVert_\mathcal{L} =\sqrt{\left|\langle\mathbf{x},\mathbf{x}\rangle_\mathcal{L}\right|}.$$

For negative curvature $-c$, with $c>0$, points on the upper hyperboloid satisfy

$$\begin{aligned} \mathcal{L}_c^n &= \left\{ \mathbf{x} \in \mathbb{R}^{n+1}: \left\langle \mathbf{x}, \mathbf{x} \right\rangle_\mathcal{L} = -1/c,\ c > 0 \right\} \newline x_\text{time} &= \sqrt{1/c + \lVert \mathbf{x}_\text{space} \rVert^2}. \end{aligned}$$

This supplies the smooth manifold; we still need its distance and maps.

Distance and Maps in the Lorentz Model

Geodesics

A geodesic is the shortest path between two points while remaining on a manifold. In general relativity, light appears to bend under gravity because it follows a shortest path through curved spacetime. If the universe is imagined as a curved sheet, the light is not choosing a curved detour; that trajectory is the sheet's geodesic.

In the Lorentz model, the geodesic between two points is the intersection of the hyperboloid with the plane spanned by those points and the origin.

Its distance is

$$d_\mathcal{L}(\mathbf{x}, \mathbf{y}) = \sqrt{1/c} \cdot \cosh^{-1}\left(-c \left\langle \mathbf{x}, \mathbf{y} \right\rangle_\mathcal{L}\right).$$

Tangent Space

At point $\mathbf{z}$, the tangent space contains every vector with zero Lorentzian inner product against $\mathbf{z}$. This local space is Euclidean:

$$\mathcal{T}_{\mathbf{z}}\mathcal{L}^n = \left\{ \mathbf{v} \in \mathbb{R}^{n+1}: \left\langle \mathbf{z}, \mathbf{v}\right\rangle_\mathcal{L} = 0 \right\}.$$

Any vector $\mathbf{u}$ in the ambient space can be orthogonally projected onto it:

$$\mathbf{v} = \operatorname{proj}_{\mathbf{z}}(\mathbf{u}) = \mathbf{u} + c\mathbf{z} \left\langle\mathbf{z},\mathbf{u} \right\rangle_\mathcal{L}.$$

Exponential and Logarithm Maps

The exponential map lifts a tangent vector onto the manifold:

$$\begin{aligned} &\operatorname{exp}_{\mathbf{z}} : \mathcal{T}_{\mathbf{z}}\mathcal{L}^n \rightarrow \mathcal{L}^n \newline \mathbf{x}=\operatorname{exp}_{\mathbf{z}}(\mathbf{v}) &= \cosh(\sqrt{c} \lVert \mathbf{v} \rVert_\mathcal{L})\mathbf{z} + \frac{\sinh (\sqrt{c} \lVert \mathbf{v} \rVert_\mathcal{L})} {\sqrt{c}\lVert \mathbf{v} \rVert_\mathcal{L}}\mathbf{v}. \end{aligned}$$

MERU later simplifies this at the origin and needs only the spatial component, eliminating one hyperbolic-cosine term. The logarithm map moves in the opposite direction, from the manifold into a tangent space:

$$\begin{aligned} &\operatorname{log}_{\mathbf{z}} : \mathcal{L}^n \rightarrow \mathcal{T}_{\mathbf{z}}\mathcal{L}^n \newline \mathbf{v}=\operatorname{log}_{\mathbf{z}}(\mathbf{x}) &= \frac{\cosh^{-1}(-c\left\langle\mathbf{z},\mathbf{x} \right\rangle_\mathcal{L})} {\sqrt{(c\left\langle \mathbf{z}, \mathbf{x} \right\rangle_\mathcal{L})^2-1}} \operatorname{proj}_{\mathbf{z}}(\mathbf{x}). \end{aligned}$$

Method

Like CLIP, MERU has separate image and text encoders. It adds two things:

1. A way to lift ordinary encoder embeddings from Euclidean into Lorentz space
2. An objective that learns image and text semantics while exploiting the hierarchy of that space

Lifting Embeddings

Suppose the image and text encoders each produce an $n$-dimensional vector, as in CLIP. The Lorentz model needs one additional time axis. MERU first places the encoding in the tangent space at the origin, then applies the exponential map to lift it onto the hyperboloid.

Expanding the Dimension

$$\mathbf{v} = (\mathbf{v}_\text{enc}, 0) \in \mathbb{R}^{n+1}.$$

Onto the Hyperboloid

At the origin, the spatial component simplifies to

$$\mathbf{x}_\text{space} = \frac{\sinh (\sqrt{c} \lVert \mathbf{v} \rVert_\mathcal{L})} {\sqrt{c}\lVert \mathbf{v} \rVert_\mathcal{L}} \mathbf{v}_\text{space}.$$

The time scalar follows from the hyperboloid constraint:

$$x_\text{time} = \sqrt{1/c + \lVert \mathbf{x}_\text{space} \rVert^2}.$$

That completes the conversion from a CLIP embedding to a hyperbolic one.

Numerical Issue

The lift introduces exponential growth. With CLIP-like initialization, an $n$-dimensional Euclidean encoding has norm roughly $\sqrt{n}$. Exponentiating that makes $e^{\sqrt{n}}$ an enormous initial scale and destabilizes training. MERU therefore multiplies image and text encoder outputs by separate learned scale parameters $\alpha$ before lifting them.

Entailment Loss

CLIP contrastive loss learns pair relationships implicitly through similarity. MERU adds an entailment loss that uses the topology of hyperbolic space to impose a hierarchical image–text relationship.

Viewed down the Lorentz time axis, each text embedding defines an angular entailment cone, or aperture, containing image embeddings compatible with that text. An image already inside need not be pulled closer; excessively collapsing its angle could destroy representation diversity. An image outside is penalized until it enters the cone. Contrastive learning still controls distances across pairs, while entailment organizes their hierarchy under negative curvature. The simplicity of the final model was surprising:

$$\mathcal{L}_\text{entail}(\mathbf{x}, \mathbf{y}) = \max \left(0,\operatorname{ext}(\mathbf{x},\mathbf{y})-\operatorname{aper}(\mathbf{x})\right).$$

Experimental Results

Image–Text Retrieval

MERU improves retrieval broadly over CLIP. This is perhaps the experiment in which expanding from Euclidean to non-Euclidean space should show its benefits most clearly.

Zero-Shot Image Classification

Zero-shot classification tests whether the geometry also improves general representation learning. Retrieval demonstrates the main contribution; classification is a supporting result, so it need not establish a new state of the art. It shows that the representation remains useful and can improve.

Ablations

The paper includes many quantitative and qualitative ablations.

The left plot varies embedding dimensionality; performance remains strong across sizes. The right removes individual method components. Fixing curvature, as in the original Poincaré approach, badly hurts the large model. Omitting the Lorentz norm from contrastive learning makes the large model fail to converge and diverge, perhaps because of a mismatch between the convergence rates of the Transformer backbone and the geometric projection.

Discussion and Conclusion

Many studies are trying to make CLIP representations more robust across modalities for retrieval, linear probing, and fine-tuning. Designing objectives and constraints that express richer image–text semantics may be one of the hardest parts of future multimodal learning.

MERU's contribution is not the Lorentz model itself, but a direction for extending an existing CLIP representation. It attaches a hyperbolic Riemannian module to the end of an established framework and adds an objective made possible by the new manifold. Because it leaves the encoders unchanged and modifies only their projection and training geometry, the same idea could be tested on many encoder-based networks. Further experiments on hierarchical embeddings would help establish how generally valid the method is.