Alberto Purpura.

Existing benchmarks often conflate instruction compliance with overall task success. This paper introduces MOSAIC, a modular framework that dynamically generates prompts with up to 20 application-oriented constraints, enabling granular, per-constraint analysis. Across five LLM families, it shows that compliance is not monolithic — it varies with constraint type, quantity, and position, exposing model-specific weaknesses and primacy/recency biases.

LLMs often produce output that is conceptually correct but violates formal constraints like word limits or formatting rules. This paper proposes a multi-agentic workflow that separates optimization of the core task from its specific output constraints, using quantitative compliance scores as iterative feedback signals. The method yields significantly higher instruction-following scores on Llama 3.1 8B and Mixtral-8x 7B without any model fine-tuning.

This work proposes an automated multi-stage pipeline for checking marketing content against compliance requirements, without relying on external knowledge bases. It benchmarks fine-tuning strategies — SFT vs. GRPO — and evaluates how reasoning tokens improve smaller models' ability to detect violations. The study also systematically tests how different reward function combinations shape model behavior under GRPO training.

GRAID tackles data scarcity in harmful text classification by generating training examples that are geometrically spread across the embedding space, then diversifying them stylistically through a multi-agent reflection loop. The pipeline is model-agnostic and domain-agnostic, designed to improve guardrail coverage without manual annotation. On two benchmark datasets it achieves an average F1 gain of 12% over baselines.