I'm not entirely sure how this kind of study jives well with other study, such as "Reasoning models don't always say what they think" [0], discussion [1].
To quote the article:
We can’t be certain of either the “legibility” of the Chain-of-Thought (why, after all, should we expect that words in the English language are able to convey every single nuance of why a specific decision was made in a neural network?) or its “faithfulness”—the accuracy of its description. There’s no specific reason why the reported Chain-of-Thought must accurately reflect the true reasoning process; there might even be circumstances where a model actively hides aspects of its thought process from the user.
So if we can't trust the reasoning, then what's the point of checking whether they are "effective" or not?
> When controlling for the number of tokens, NoThinking outperforms Thinking across a diverse set of seven challenging reasoning datasets
Interesting. I thought the "thinking" was useful because it pulls in a lot of concepts into the context, but I guess not then?
It has also been said before that the text a model outputs during its Thinking step isn't actually a view into its inner thoughts. There are times when the model will think X but eventually answer Y.
But even so: the models _are_ better, right? So is the Thinking step then mostly useful during training?
I'm not entirely sure how this kind of study jives well with other study, such as "Reasoning models don't always say what they think" [0], discussion [1].
To quote the article:
So if we can't trust the reasoning, then what's the point of checking whether they are "effective" or not?[0]: https://www.anthropic.com/research/reasoning-models-dont-say...
[1]: https://news.ycombinator.com/item?id=43572374
> When controlling for the number of tokens, NoThinking outperforms Thinking across a diverse set of seven challenging reasoning datasets
Interesting. I thought the "thinking" was useful because it pulls in a lot of concepts into the context, but I guess not then?
It has also been said before that the text a model outputs during its Thinking step isn't actually a view into its inner thoughts. There are times when the model will think X but eventually answer Y.
But even so: the models _are_ better, right? So is the Thinking step then mostly useful during training?