It's challenging to keep running experiments. In a perfect world, we would run our experiments again and update the results. But some experiments are expensive, and some experiments are inherently difficult to reproduce. I have a paper nearing completion that involves interaction between trained (human) professionals and a model. It was a logistical challenge to bring those people together (also, we had to pay them), and the roleplaying that we asked LLMs to do can't be done anymore--they refuse for safety reasons. Stuff like this is pretty common.

I am hopeful that we can eventually move to open weights models, but at the time we did the experiments, the open weights models did not have the capabilities we needed. The ever-shifting sands of frontier models makes reproducibility difficult, and not just for scientists.

Unfortunately it is this long. I know people in academia studying human-AI interfaces and up until recently they were still using gpt-3.5.

The Anthropic and OAI APIs report token usage (input, output, and in some cases reasoning/thinking). The input tokens consist of the system prompt, the user input/messages, and any other context data.

Given the same model the only variable is the token count. The more tokens, the more passes through the LLM (to generate the next output token), the more energy used.

It's well known that non-English text consumes more tokens [1], [2] and thus more energy. This is due to two factors:

1. Latin script languages having different character pairings to English and modified characters, including language features like modifying verbs for person/tense/etc. (e.g. Spanish);

2. Non-Latin scripts that take more bytes to represent in UTF-8 and (esp. for ideographic and syllabic languages like Chinese, Japanese, and Korean) have lower common character pair sequences, meaning shorter tokens for those scripts and thus more tokens for a given text sequence.

[1] https://ikriv.com/blog/?p=5322 "LLM tokens and foreign languages" (5 Aug 2025)

[2] https://medium.com/@programmerraja/how-your-llm-costs-5x-mor...

Look at the token usage counts.

It's explained too..