Needle-in-a-Haystack

Needle-in-a-haystack is a standard test for long-context models: a small piece of information (the “needle”) is buried inside a very long document (the “haystack”), and the model must find it.

It checks whether a big context window actually works, not just how large it is advertised to be. In our benchmark every tested model retrieved the needle in 100% of cases.