Claims Browser

EditaLive delivers state-of-the-art editing performance with faithful preservation of facial expressions and low-latency real-time streaming inference

"Extensive experiments demonstrate that EditaLive delivers state-of-the-art editing performance with faithful preservation of facial expressions and low-latency real-time streaming inference"

A pretrained image animation model that naturally decouples appearance from motion can be repurposed as a base model for instruction-based human-centric video editing

"we start from a pretrained image animation model (Wan-Animate), which naturally decouples appearance from motion, and repurpose it as the base model for instruction-based human-centric video editing by reference frame editing and video reconstruction via the collected CharEdit-50K dataset"

An aligned self-rollout distillation strategy can compress a video editing model into a two-step sampler while reducing training-inference discrepancies and mitigating appearance drift

"design an aligned self-rollout distillation strategy that compresses the model into a two-step sampler, where fixed RoPE and align forcing reduce training--inference discrepancies, and first-frame preserved sparse attention filters redundant historical information to mitigate appearance drift"

Cross-cultural meme transcreation has three core challenges: culture-specific knowledge understanding, intent and tone preservation, and multimodal consistency

"we first provide an explicit task analysis of cross-cultural meme transcreation and identify three core challenges: culture-specific knowledge understanding, intent and tone preservation, and multimodal consistency"

A multi-agent framework with specialized agents can effectively address cross-cultural meme transcreation challenges through cultural adaptation, target text rewriting, revision, and conditional visual adjustment

"we propose a multi-agent framework with specialized agents that are coordinated to address these challenges through cultural adaptation, target text rewriting, revision, and conditional visual adjustment"

The proposed meme transcreation method achieves 33.1% average improvement over the strongest baseline in human evaluation and 60% Top-1 ranking rate versus 26% for the second-best baseline in LLM-as-a-Judge evaluation

"In human evaluation, it achieves the best performance on all four dimensions and delivers a 33.1% average improvement over the strongest baseline, while in LLM-as-a-Judge, it attains the highest Top-1 ranking rate (60% versus 26% for the second-best baseline)"

The remaining bottlenecks in cross-cultural meme transcreation lie in humor reconstruction and image-text alignment rather than simple cultural knowledge gaps

"Our error analysis suggests that the remaining bottlenecks lie in humor reconstruction and image-text alignment rather than simple cultural knowledge gaps, pointing to future work on humor transfer"

Multimodal foundation models show substantial inconsistencies when handling semantically equivalent queries across different modalities (text vs. speech) and languages (English vs. Arabic)

"it remains unclear whether semantically equivalent queries yield consistent judgments across modality (text vs. speech) and language (English vs. Arabic)"

Modality and language shifts introduce substantial triplet-level inconsistencies in multimodal models that are not fully captured by aggregate accuracy metrics

"modality and language shifts introduce substantial triplet-level inconsistencies that are not fully captured by aggregate accuracy"

Speech input amplifies partial failures in multimodal models compared to text input

"speech amplifying partial failures"

Contrastive instability (the conditional rate at which a model fails to resolve all statements within a triplet) can isolate fragmented reasoning from complete failure in multimodal models

"We define contrastive instability as the conditional rate at which a model fails to resolve all statements within a triplet, isolating fragmented reasoning from complete failure"

Explorable image-to-scene generation is essential for applications in gaming, robotics, and virtual reality

"Explorable image-to-scene generation is essential for applications in gaming, robotics, and virtual reality."

Object-Conditioned Social Diffusion (OCSD) achieves state-of-the-art results on pedestrian trajectory forecasting benchmarks, reducing two-second path error by 31.3% on Humans in Kitchens and 33.2% on HOI-M3 compared to prior work

"OCSD achieves state-of-the-art results on the Humans in Kitchens (HiK) and HOI-M3 benchmarks. It reduces the two-second path error by 121.5 mm (31.3%) on HiK and 130.5 mm (33.2%) on HOI-M3 compared to prior work"

Integrating motion history, multi-person interactions, and object cues into a unified conditional diffusion framework enables more accurate forecasting of human movement in complex scenes

"OCSD uses an object-conditioning mechanism that modulates denoising at every timestep, enabling fine-grained human-object reasoning, and a social encoder that models the interactions between all humans in the scene"

OCSD produces more realistic long-term trajectory forecasts by naturally handling varying group sizes and complex social interactions

"our model naturally handles varying group sizes, complex social interactions, and supports sampling multiple plausible futures"

Multimodal representation learning is shifting from traditional two-tower architectures to LLM-based embedders due to their strong instruction-following capabilities

"Multimodal representation learning has been shifting from traditional two-tower architectures to large language model (LLM)-based embedders due to their strong instruction-following capabilities."

OmniUE is the first Omni-Interactive Universal Embedder that learns a unified embedding space across text, video, and audio and supports omni-interactive querying

"we propose the first Omni-Interactive Universal Embedder (OmniUE), which not only learns a unified embedding space across text, video, and audio by leveraging intermediate-layer representations from dedicated learnable tokens, but also supports omni-interactive querying, enabling users to provide inputs in the form of text, visual regions of interest, and audio spans."

OmniUE achieves 10.5% improvement on textual-interactive video benchmarks over state-of-the-art baselines

"OmniUE consistently surpasses state-of-the-art baselines across diverse modalities, with average improvements of 10.5% on textual-interactive video benchmarks (MMEB-v2-video)"

OmniUE achieves 83.7% improvement on visual-interactive benchmarks over state-of-the-art baselines

"OmniUE consistently surpasses state-of-the-art baselines across diverse modalities, with average improvements of 10.5% on textual-interactive video benchmarks (MMEB-v2-video), 1.1% on audio tasks (MAEB), 83.7% on visual-interactive benchmarks (SCaR), and 24.1% on our omni-interactive OmniCHOIR benchmark."