Claims Browser

Large language model judges are increasingly being used across various evaluation scenarios, making their judgment capabilities valuable intellectual property

"Large language model (LLM) judges are increasingly used across various evaluation scenarios, making their judgment capabilities valuable intellectual property."

Black-box access to LLM judges exposes their capabilities to model extraction attacks

"However, black-box access exposes these capabilities to model extraction attacks."

JUDGESTEALER can achieve up to 73.3%, 87.0%, and 71.6% accuracy for pointwise, pairwise, and listwise evaluation respectively when extracting LLM judge capabilities

"Extensive experiments on state-of-the-art LLM-as-a-judge and reward models show that JUDGESTEALER consistently outperforms existing extraction baselines, achieving up to 73.3%, 87.0%, and 71.6% accuracy for pointwise, pairwise, and listwise evaluation, respectively."

JUDGESTEALER demonstrates robustness against representative extraction defenses

"Moreover, JUDGESTEALER demonstrates robustness against representative extraction defenses."

Strong cross-protocol agreement in LLM judges allows pointwise scores to be transformed into pairwise and listwise supervisions without additional victim queries

"JUDGESTEALER exploits the strong cross-protocol agreement to acquire pointwise scores and transform them into pairwise and listwise supervisions without additional victim queries."

Decentralized multitask learning can estimate task relationship graphs directly from distributed data without assuming known structure

"This paper investigates decentralized multitask learning over networks when the underlying task relationships are unknown. While existing graph-regularized multitask frameworks typically assume a known structure, practical settings often require learning inter-task dependencies directly from distributed data."

A two-phase decentralized strategy using Gaussian Markov random field priors can estimate graph Laplacians from noisy stochastic gradient iterates and enable cooperative multitask learning

"We propose a decentralized two-phase strategy that first estimates a generalized graph Laplacian from noisy non-cooperative stochastic gradient iterates, and subsequently exploits the learned graph to enable cooperative multitask diffusion learning. This framework is motivated by a Gaussian Markov random field prior, which gives rise to a decentralized maximum likelihood estimator for the graph Laplacian."

Cooperation enabled by learned task graphs significantly improves performance over non-cooperative learning and approaches true-graph baseline performance with sufficiently small estimation stepsizes

"Simulation results corroborate the theoretical findings and demonstrate that cooperation enabled by the learned task graph significantly improves performance over non-cooperative learning, while approaching the true-graph baseline when the estimation stepsize is sufficiently small."

Little is known about how representation learning methods deal with out-of-domain speech and how they could be adapted in a few shot to new domains

"little is known about how such methods deal with out-of-domain speech and how could they be adapted in a few shot to new domains"

Adaptive domain normalization fine-tuning of pretrained Contrastive Predictive Coding models yields a 23.6% relative improvement on across-speaker ABX scores for accented speech

"the proposed method yields a relative improvement of 23.6% on across-speaker ABX scores on average compared to non adapted models"

ABX-Accent benchmark provides a standardized way to evaluate representation learning methods on accented speech with 10 different English accents

"We introduce ABX- Accent, a benchmark based on the AESRC dataset that features 10 different accents of English. It includes a small (< 10 hours) unlabelled training set in each of the accents and adaptations of the Zero Resources Challenge ABX evaluation metrics to each of the accents"

Explicit visual intermediates can help multimodal large language models externalize spatial evidence and updated visual states, but their utility depends on the image editor's ability to faithfully realize transformations

"Explicit visual intermediates can help multimodal large language models (MLLMs) externalize spatial evidence and updated visual states, but their utility depends on whether an image editor can faithfully realize the required transformation."

Utility of visual intermediates for MLLMs is strongly task-conditioned, with gains concentrated in visual cue injection, grounding, and counterfactual state realization

"we find that utility is strongly task-conditioned. Gains concentrate in visual cue injection, grounding, and counterfactual state realization"

Visual intermediates requiring symbol-sensitive construction or structural extrapolation are substantially less reliable for MLLMs

"intermediates requiring symbol-sensitive construction or structural extrapolation are substantially less reliable"

The consolidated Qwen pipeline with visual intermediates improves mean task score from 0.343 to 0.445, representing a 29.7% relative improvement

"our consolidated Qwen pipeline improves the mean task score from 0.343 to 0.445 ($+10.2$ points; $+29.7\%$ relative)"

Existing VLM and Medical-VLM efforts have mainly focused on improving accuracy while leaving calibration in the medical domain underexplored

"existing efforts mainly focused on improving accuracy, leaving calibration in the medical domain underexplored"

Multi-Class Margin (MCM) regularization achieves lowest Expected Calibration Error on 10 out of 12 in-domain settings and remains competitive under domain shifts

"Multi-Class Margin (MCM) regularization, which achieves lowest ECE on 10 out of 12 settings in in-domain and remains competitive under domain shifts"

MVC-Bench provides a structured evaluation framework and actionable guidance for improving calibration in safety-critical medical workflows

"MVC-Bench provides a structured evaluation framework and actionable guidance for improving calibration in safety-critical medical workflows"