Researchers released Taste-Bench to measure an LLM agent's taste at real decision forks. It tests whether a model picks the better next step in long-horizon tasks. The dataset contains 502 questions mined from software engineering and machine-learning trajectories. No expert annotation was used to build it. The best frontier model answers 59.7% of the questions correctly. Questions are filtered to remove trivial or undecidable cases. Models must answer correctly in both option orders to score a point. The dataset is gated to limit training contamination. A full evaluation run requires 1,004 requests. The code and dataset are open-source under MIT and CC BY 4.0 licenses.

• 502 questions
• 59.7% best model accuracy
• 4,657 mined forks
• 1,004 requests per full run