Paper-Conference

A framework for benchmarking AI scientist capabilities using adversarial, fast-moving real-world domains, where expert practitioners independently produce observable ground truth after an information cutoff. Instantiated in Formula 1 car design for the 2026 regulations and Magic: The Gathering deck building, with frontier models ideating from pre-cutoff information only and their outputs scored against real innovations and Pro Tour decklists.

Framing oncology clinical development as an offline decision-making problem, where an agent predicts the next six-month trial portfolio of a drug programme from information available at the decision date. Built on a temporal dataset of 31.7k public records turned into 881 decision episodes across 45 historical programmes, used to train offline policies and compare them against frontier LLM agents on held-out drug, sponsor, drug-class and temporal splits.

A framework for evaluating the reliability of LLMs as assistants in the biomedical domain

A innovative pipeline that uses clinical ontologies to generate comorbidity and patient representations for downstream AI tasks.