Creativity or brute force? Using brainteasers as a window into the problem-solving abilities of large language models

Abstract

Accuracy remains a standard metric for evaluating AI systems, but it offers limited insight into how models arrive at their solutions. In this work, we introduce a benchmark based on brainteasers written in long narrative form to probe more deeply into the types of reasoning strategies that models use. Brainteasers are well-suited for this goal because they can be solved with multiple approaches, such as a few-step solution that uses a creative insight or a longer solution that uses more brute f

Relevance Assessment

Research Gap

Notes are automatically saved as you type

Search Queries