Study: non-English languages can consume far more AI tokens, with the most expensive using 1.5x-5x more than the cheapest, resulting in a “tokenizer tax”
restofworld.org via Techmeme
Some languages require far more tokens for AI models to process, raising costs and reducing context for the people who use them.
Kort telegram från Techmemes öppna flöde, på källans språk. Hela originalartikeln har inte lästs.
Mer från Techmeme
DeepSeek open sources Super-Agent as a research prototype, a proactive coding agent it says can monitor group chats for coding issues and suggest solutions
Techmeme för 1 tim sedan
Insurance software maker Guidewire agrees to acquire Hyperexponential, which makes AI-powered insurance underwriting and pricing software, for $1.7B in cash
Techmeme för 1 tim sedan
Tests: chatbots rely on three obscure AI-built voter sites for US midterm answers; Copilot cited the error-prone Ballot Brief in 75% of tests and ChatGPT in 22%
Techmeme för 6 tim sedan
Researchers are paid $5K for reporting a bug to OpenAI that would let attackers run code in logged-in sessions of all who opened the hacker's ChatGPT share link
Techmeme för 11 tim sedan