AI · Startups · Japan Tech

Sakana AI: The Tokyo Lab Co-Founded by a Transformer Paper Author Who Says Making Models Bigger Is the Wrong Bet

In 2017, Llion Jones was one of eight co-authors of “Attention Is All You Need” — the paper that introduced Transformer architecture and became the foundation of GPT, Claude, Gemini, and every other major AI model today. In 2023, he left Google and moved to Tokyo to build Sakana AI, a company whose central argument is that making those models bigger and bigger indefinitely is not the right path.

The Paper That Started It All

In June 2017, eight researchers at Google published a paper titled “Attention Is All You Need”. Llion Jones was one of those eight authors. The paper explicitly declared all eight as equal contributors and randomised the listed author order — Jones’s credited contribution was the multi-GPU implementation code. The architecture itself emerged from the collective work of the team. The paper introduced the Transformer — a way of processing sequences of data using attention mechanisms that replaced older recurrent approaches and turned out to be dramatically more scalable. Within a few years, virtually every major AI model in the world was built on it. GPT-4, Claude, Gemini, Llama, Mistral — all Transformers. Wikipedia describes it as the most cited computer science paper on Google Scholar.

Jones spent the years after the paper at Google, watching what the industry did with the architecture he helped create. What it did, overwhelmingly, was scale — build bigger and bigger Transformer models, train them on more and more data, using more and more compute. GPT-3 had 175 billion parameters. GPT-4 is estimated at over a trillion. The trend line pointed toward ever-larger models requiring ever-larger datacenters consuming ever-larger amounts of electricity. In 2023, Jones left Google.

David Ha and Llion Jones present Sakana AI’s founding thesis — nature-inspired intelligence and a new paradigm for language models — at the NTT R&D Forum, December 2023. Both co-founders on stage together. Not a Wangdoo production.

What Sakana Is and What It Is Arguing

Sakana AI was founded in July 2023 in Tokyo by Jones alongside David Ha — formerly of Google Brain and Stability AI — and Ren Ito — formally Chairman, operationally COO — a former Japanese diplomat and early Mercari executive. The name comes from the Japanese word for fish. The school of fish is the company’s central metaphor: a single fish is relatively simple, but a school moves as a coherent, intelligent entity through local rules and collective behaviour. No single fish has to be enormous or all-knowing.

The argument Sakana is making is not that Transformer models are wrong — Jones’s own research is built on them. The argument is that the dominant strategy of making one model as large as possible, and expecting that model to do everything, is approaching a ceiling. Compute costs are growing faster than capability gains. Energy consumption is becoming genuinely problematic. And the models that emerge from this approach are expensive to run, expensive to maintain, and opaque.

Sakana’s alternative: instead of one enormous generalist model, use many smaller specialised models that collaborate. Combine them through evolutionary algorithms — the same principles of variation, selection, and inheritance that drive biological evolution. Let the system adapt rather than pre-programming everything. Sakana’s researchers put it directly in their own blog: “The future of AI will not consist of a single, gigantic, all-knowing AI system that requires enormous energy to train, run, and maintain, but rather a vast collection of small AI systems — each with their own niche and specialty, interacting with each other, with newer AI systems developed to fill a particular niche.”

What Sakana Has Actually Built

The company has moved from research to products faster than most academic-style AI labs. Its published work covers three distinct areas.

Evolutionary Model Merge — announced in March 2024, approximately eight months after founding. A method for building new AI models by combining existing open-source ones through evolutionary algorithms, without large compute requirements. The practical implication: you can create capable specialised models without training from scratch, which is something only well-resourced labs can afford to do. Sakana released two models built this way at launch — EvoLLM-JP (a Japanese-language math reasoning model) and EvoVLM-JP (a Japanese vision-language model) — both of which outperformed their parent models on several benchmarks. The research was subsequently published in Nature Machine Intelligence.

The AI Scientist — an agentic system that generates research ideas, designs experiments, runs them, and writes the resulting paper. AI Scientist v2 submitted three fully AI-generated papers to the ICLR 2025 workshop “I Can’t Believe It’s Not Better.” One received peer-review scores of 6, 7, and 6 — above the human acceptance threshold. Sakana describes this as, “to our knowledge, the first fully AI-generated paper that has passed the same peer-review process that human scientists go through.” The paper was subsequently withdrawn — the workshop organisers were aware of the experiment and raised questions about publishing AI-generated work — but the competitive score is the milestone Sakana points to. The research describing the full AI Scientist system was then published in Nature in March 2026, in collaboration with researchers at the University of British Columbia, the Vector Institute, and the University of Oxford.

The Darwin Gödel Machine — an AI system that writes, tests, and improves its own code iteratively. The name references Gödel’s incompleteness theorems and Darwin’s theory of evolution simultaneously. The system represents Sakana’s most ambitious research direction: AI that genuinely improves itself rather than being improved by humans tuning it.

On the product side, Sakana has shipped Namazu — Japanese-language LLMs tailored specifically for Japan’s linguistic and cultural context — and most recently Fugu, a multi-agent orchestration API that routes tasks across a pool of frontier models — Claude Opus 4.8, GPT-5.5, and Gemini 3.1 Pro — picking whichever is best suited for each part of a request and synthesising the outputs behind a single API call. Fugu is the direct commercial implementation of Sakana’s collective intelligence thesis: you call one endpoint, a trained orchestrator assembles the right combination of models behind it.

Why Tokyo and Why Japan

The choice of Japan is not incidental. David Ha has spoken publicly about it: Japan’s severe labour shortage — driven by an ageing population and decades of demographic decline — creates genuine, urgent demand for AI that can substitute for unavailable human workers. The major Japanese banks Sakana works with are not adopting AI because it is fashionable. They are doing it because they cannot hire enough people to handle lending and M&A activity at the scale Japanese economic conditions now require.

Sakana has deployed an AI underwriting agent with MUFG to evaluate loan applications and generate approval explanations. With Sumitomo Mitsui Financial Group, it built an AI agent that helps companies seeking to sell their businesses find potential buyers and automatically generate pitch materials — confirmed from a primary interview with David Ha in CommonWealth Magazine, April 2026. With Daiwa Securities Group, it is building a wealth management platform for retail investors. In March 2026, it won its first contract with Japan’s Acquisition, Technology and Logistics Agency (ATLA) — the procurement arm of the Ministry of Defense. The project, confirmed by David Ha in the same CommonWealth Magazine interview, involves developing small vision-language models deployable on edge devices such as drones, enabling real-time analysis of visual and textual data transmitted back to command systems. The In-Q-Tel investment in Sakana’s Series B is also notable: In-Q-Tel is the investment arm of the US intelligence community, and its participation signals that Sakana’s research is relevant to Western governments as well as Japanese ones.

The Japan angle also connects to the broader sovereign AI story that has shaped 2026. Governments from the UAE to Switzerland have been investing in AI infrastructure that does not depend on US providers. Sakana fits squarely into this trend: a frontier AI lab not in California, not building on American cloud infrastructure, backed by Japan’s largest financial institutions and explicitly positioned as providing “sovereign AI solutions that reflect national cultures and values,” in David Ha’s own words.

The Honest Questions About Fugu

Sakana’s Fugu model, launched in June 2026, describes Fugu Ultra as standing “shoulder-to-shoulder” with Anthropic’s Fable 5 and Mythos on benchmarks including GPQA-Diamond, CharXiv Reasoning, and TerminalBench — while being clear it does not outperform Fable 5 on every test. On SWE-Bench Pro, for example, Fugu Ultra scores 73.7% against Fable 5’s 86.0%, a 13-point gap on one of the most relevant coding benchmarks. It is also, at present, impossible to verify independently.

The more interesting question about Fugu is not the benchmark number but the architecture claim. If an orchestration layer — a coordinating system that routes tasks to whichever underlying model is best suited — genuinely achieves frontier-level performance without requiring a single frontier-scale model, that is the proof of concept Sakana’s entire thesis depends on. The benchmark claim is the commercial pitch. The architecture question is the scientific one.

My Take — Mr Wangdoo

The part of the Sakana story that is genuinely worth understanding — separate from the funding rounds and the benchmark claims — is the argument about scaling. The dominant AI industry position is that capability comes primarily from scale: more parameters, more data, more compute. Sakana’s position is that this has limits, and that collective intelligence — many smaller models working together — is a more sustainable path. That is not a fringe view. It is a serious research position that connects to decades of work in evolutionary computation, complex systems, and swarm intelligence.

The specific question worth asking about Sakana is not whether the research is serious — a Nature publication and commercial deployments at Japan’s largest banks settle that — but whether the collective intelligence architecture can win on general tasks rather than just the specific niches it’s been optimised for. Evolutionary Model Merge is genuinely clever and the AI Scientist is genuinely impressive. But both produce results in relatively constrained domains: Japanese-language models, structured scientific paper generation. Whether many small models routing through Fugu can match a single large frontier model on the messy, open-ended tasks that enterprises actually need done is the proof of concept the company still needs to demonstrate publicly. The Fugu benchmark claims would be that proof — which is exactly why it matters that those specific comparisons cannot currently be verified.

What I find hardest to ignore is the Llion Jones dimension. He is not arguing against the thing he helped create. He is arguing that the specific way the industry has used it — scaling one model as large as possible, indefinitely — is not the only or necessarily the best path. That is a more modest and more defensible claim than “Transformers are wrong.” It is also a claim with serious research behind it from a team with serious credentials. Whether it pays off commercially in the next three years is genuinely unknown. Whether it is worth understanding now — if you care about how AI systems are actually built and where they are headed — seems clear.

Frequently Asked Questions