OmniUE is the first Omni-Interactive Universal Embedder that learns a unified embedding space across text, video, and audio and supports omni-interactive querying

we propose the first Omni-Interactive Universal Embedder (OmniUE), which not only learns a unified embedding space across text, video, and audio by leveraging intermediate-layer representations from dedicated learnable tokens, but also supports omni-interactive querying, enabling users to provide inputs in the form of text, visual regions of interest, and audio spans.

http://arxiv.org/abs/2608.27044v1