Citadel AI: Tokyo Startup Watching AI for Errors – Explained
Citadel AI Explained: The Tokyo Startup Built to Watch AI for Mistakes Before They Become Disasters
A small University of Tokyo-incubated company has spent five years building tools that test and monitor AI systems for the kind of silent failures, drift, bias, hidden errors, that don’t show up until something has already gone wrong. Here’s what it actually does, who relies on it, and why almost nobody outside Japan has heard of it.
Editorial Transparency: This article is independently written and not sponsored by Citadel AI. Facts are verified directly against Citadel AI’s own website, blog, and GitHub repositories, the University of Tokyo’s own feature profile of the company, and funding records on Crunchbase and CB Insights — all accessed directly. Wangdoo has no commercial relationship with Citadel AI.
I have not used Citadel AI’s products myself. This explains what the company’s own materials and existing reporting describe, not a hands-on account of running them.
Citadel AI was founded in Tokyo in December 2020 by Hironori “Rick” Kobayashi, a former trading-house executive, and Kenny Song, a former product manager for machine learning infrastructure at Google Brain. The two met online, discovered they both wanted to start an AI company in Japan, and built the business around a problem Song had seen firsthand: AI models that work fine in testing quietly degrade once they’re actually running in production, and almost nobody has good tools to catch it happening. According to the University of Tokyo’s own profile of the company, they spent a year in UTokyo’s FoundX startup support program before raising their first funding.
What It Actually Does
Citadel AI sells two main products that cover an AI system’s life before and after it goes live. Citadel Lens runs before deployment and during development: it stress-tests a model against industry standards and generates reports on explainability, robustness, fairness, bias, and data quality, the kind of documentation increasingly required by regulations like the EU AI Act and standards like ISO 42001. Citadel Radar takes over once a model is live, watching it continuously for drift, anomalies, and the kind of fraudulent or unusual input data that can quietly poison a model’s outputs over time. Both tools are described as model-agnostic, meaning they work by testing inputs and outputs from the outside rather than requiring deep access to a model’s internals, which the company positions as important because that kind of internal access tends to break every time a model gets updated.
Citadel Lens
Pre-deployment testing and reporting: robustness, explainability, fairness, and data quality reports, plus automated Model Cards and Data Cards for compliance documentation.
Citadel Radar
Continuous production monitoring for drift, bias, anomalies, and fraudulent or unusual input data, designed to catch silent performance decay before it causes real damage.
Lens for LLMs
A version of Lens specifically for language models: automated and human-combined evaluation, jailbreak and safety testing in English and Japanese, and side-by-side comparison between different LLMs.
LangCheck
Citadel AI’s free, open-source Python library for evaluating LLM outputs, supporting multiple languages including English and Japanese, with thousands of monthly downloads on GitHub.
Watch — Lens for LLMs Demo
Citadel AI’s own product walkthrough of Lens for LLMs, linked directly from the company’s blog. Source: Vimeo.
Why a Company Like This Comes Out of Japan Specifically
Japan’s regulatory environment gives a company like Citadel AI a clearer reason to exist than it might have elsewhere. The country has its own AI Guidelines for Business sitting alongside the EU AI Act and frameworks like NIST’s AI Risk Management Framework, and Citadel AI builds its reporting explicitly around all of them at once. For a multinational manufacturer or healthcare company trying to satisfy regulators in multiple markets simultaneously, a single tool that documents compliance against several overlapping standards is a more concrete selling point than “trustworthy AI” as an abstract idea.
Who’s Actually Using It
The clearest evidence that this isn’t just an idea on a pitch deck is who’s adopted it. Suntory Holdings, the Japanese beverage and spirits group, uses Citadel AI’s tools to monitor machine learning systems including a model that predicts pallet recovery in its logistics operations, and has taken an equity stake in the company. BSI, the British Standards Institution, selected Citadel AI from 54 companies worldwide after a year-long technical evaluation process to help assess high-risk AI systems in medical devices, automobiles, and credit screening. DeepEyeVision, a healthcare AI company, uses Citadel Lens for the kind of strict reliability assessment medical AI requires, and NTT Data has partnered with Citadel AI to verify Citadel Radar’s performance using open financial data.
The Open-Source Side
Unlike a lot of enterprise AI governance vendors, Citadel AI gives away a real piece of its technology for free. LangCheck is a Python library, released under an open license on GitHub, that provides building blocks for evaluating LLM-generated text: checking fluency, toxicity, sentiment, factual consistency, and more, with support for turning those checks directly into unit tests for an LLM application. It’s positioned as the foundation underneath Citadel AI’s paid Lens product, and the company has since built LangCheck Studio, a free browser-based playground where anyone can try LLM-as-a-judge evaluation techniques without installing anything.
What It’s Not
It’s not a household name, and it’s not chasing one. Total funding sits around $4.6 million, modest next to Japan’s bigger AI exports like Sakana AI’s roughly $2.65 billion valuation. The team is small, in the range of 11 to 50 people according to LinkedIn, and pricing isn’t public; the company quotes each customer individually based on use case. Coverage so far is limited to the University of Tokyo’s own feature profile, Tokyo startup job boards, funding databases, and niche newsletters, nothing from Verge, TechCrunch, Engadget, Wired, or Yahoo.
Who It’s Actually Built For
✅ Fits This Profile
- Organisations deploying AI in regulated, high-stakes settings, healthcare, finance, automotive, manufacturing, where Citadel AI’s named customers already sit
- Teams that need to document compliance against multiple overlapping standards (EU AI Act, ISO 42001, Japan’s AI Guidelines) rather than just one
- Developers wanting a free, open-source starting point for LLM evaluation before committing to a paid platform, via LangCheck
❌ Doesn’t Really Fit
- Small teams or individual developers running low-stakes AI applications, where the compliance-and-monitoring focus is more overhead than benefit
- Anyone expecting public, self-serve pricing — Citadel AI’s commercial products are quote-based, not listed
- Anyone looking for a household-name vendor — this is a deliberately under-the-radar specialist, not a company optimising for broad visibility
My Take — Mr Wangdoo
What’s genuinely interesting about Citadel AI, reading through its blog history and customer list rather than just its homepage, is how unglamorous the actual work is. There’s no flashy “AI agent” branding here, just years of incremental releases: Model Cards, Data Cards, object detection support, jailbreak testing in Japanese specifically, each one solving a narrow documentation or testing gap rather than chasing a headline. That’s not the kind of story that gets picked up by outlets chasing the next big AI announcement, but it’s exactly the kind of infrastructure that becomes load-bearing once enough companies are running AI systems they can’t afford to get wrong.
The detail I keep coming back to is the BSI selection, one of 54 companies evaluated worldwide, for a process that took a full year. That’s a slow, unglamorous kind of validation, the opposite of a splashy funding headline, but it’s arguably a more meaningful signal about whether the underlying technology actually works than any press release would be.
What is Citadel AI?
A Tokyo-based startup, founded in 2020, that builds tools to test AI models before deployment (Citadel Lens) and monitor them continuously once they’re live (Citadel Radar), catching issues like data drift, bias, and silent performance decay.
Who founded it?
Hironori “Rick” Kobayashi, a former trading-house executive, and Kenny Song, a former Google Brain product manager, who co-founded the company in December 2020 after meeting online.
Does it have a free or open-source option?
Yes. LangCheck is a free, open-source Python library for evaluating LLM outputs, available on GitHub, and LangCheck Studio is a free browser-based playground built on the same technology.
Who actually uses it?
Named customers include Suntory Holdings, BSI (the British Standards Institution), DeepEyeVision, and a verification partnership with NTT Data, spanning manufacturing, certification, healthcare, and finance.
How much funding has it raised?
Around $4.6 million in total, including a ¥520 million ($3.7 million) Series A in 2023 led by UTokyo Innovation Platform, with Suntory also taking an equity stake.
Has this been covered by mainstream tech media?
No coverage was found from The Verge, TechCrunch, Engadget, Wired, or Yahoo. Existing coverage is limited to the University of Tokyo’s own profile, Tokyo startup directories, and funding databases.
Sources & References
- Citadel AI — Official website — products, customers, certifications
- The University of Tokyo — Building a citadel of reliable AI for the real world — founder profile, founding story
- GitHub — citadel-ai/langcheck — open-source LLM evaluation library
- Citadel AI — Series A funding announcement — funding amount, investors, BSI evaluation detail
- Citadel AI — LangCheck Studio blog post — source of the Lens for LLMs demo video
- Citadel AI — Crunchbase profile — funding history, company data