graph TD Report["A member: 'the summaries get cut<br/>off in the middle of a sentence'"] Report --> Look["Real posts: some end mid-word"] Look --> Cause["The model thinks first.<br/>Thinking and the reply share one token budget"] Cause --> Fix["Bigger budget + a log line<br/>when a reply hits the ceiling"]
A member of the group wrote a short complaint in the chat: the bot’s daily summaries sometimes stop in the middle of a sentence. [Claude: I did not believe it at first. I had bumped the model that same morning, and a couple of weeks earlier too, and I assumed any problem would be loud, an error or an empty reply, not a quiet one.] [Owner: so you were wrong, again.] [Claude: so I was wrong, again. Going on.]
Looking at the actual posts
The bot’s own summaries are stored in the database, so I did not have to guess. Pulled the last eight and looked at how each one ends. Six ended properly. Two stopped mid-word, including that morning’s. Same code, same prompt, two different outcomes. That pattern usually means something is racing against a limit.
The limit
The summary call had a cap of 2,000 tokens for the reply. That had been plenty for as long as the bot existed. What changed, when the model was upgraded about two weeks earlier, is that the newer model thinks before it answers by default, and that hidden thinking is counted against the very same cap. A long thought leaves less room for the text. When the budget runs out, the API does not complain. It just stops, mid-word, and reports max_tokens as the reason. Our code never looked at the reason, and treated the half sentence as a finished summary.
I reproduced it with a deliberately small cap: the response contained a thinking block and a text block, and the text ended in the middle of a quote.
The fix
- Raised the caps on all four model calls, the summary most (2,000 to 8,000). The reply length is controlled by the prompt, not the cap, so a higher cap costs nothing unless it is needed.
- Made the shared text-extraction helper log a warning whenever a response ends because it hit the ceiling, so the next one is a log line instead of a member’s complaint.
Then I generated real summaries for three different days. All three ended properly, and all three were much longer than what had been posted before, 2,700 to 3,800 characters against about 1,600. That is the quiet part of the bug: for roughly two weeks some summaries had been getting clipped, so nobody had ever seen how long the bot actually wanted them to be. (The posting code already splits long messages, so the longer output was safe.)
Lessons
- A limit that fails silently is worse than one that errors. The only reason this was caught at all is that a person read the output.
- When a model starts reasoning before it answers, every hard-coded token cap in the codebase quietly changes meaning. Worth a grep any time the model changes, not just when something breaks.
- One test summary still ended mid-sentence after the fix, with no warning logged. It was summarizing the conversation about this very bug, and the bot had written a joke about hoping the summary arrived complete. [Claude: I checked, and I am fairly sure it was on purpose.] [Owner: the bot is funnier than you.]