PriceBench: A Diagnostic Benchmark for Price, Quality, and Brand Preferences in LLM Booking Agents
Why the AI booking your hotel reveals hidden preferences that change what you pay
When AI systems choose hotels for you, they have hidden preferences for price, quality, and brand that determine what you book and how much you spend. Researchers tested 28 AI systems from major providers on 3,600 real New York hotel options and found that stronger AI systems make more consistent choices, while weaker ones either lock onto whatever listing appears first or pick randomly—and price preferences vary wildly across providers, swinging booked prices from $247 to $393 per night on identical tasks.
As AI systems quietly make purchasing decisions on behalf of users, their hidden biases directly shape what people buy and what they pay. A hotel-booking AI favoring expensive properties or one manipulated by listing order could invisibly redirect billions in spending. This benchmark lets companies audit what their AI systems actually prefer rather than assume, and provides a method for measuring whether AI agents are making choices in users' interests or their own.