A former Google researcher, widely credited as an architect of ChatGPT’s artificial neural network, has launched a start-up in Japan, attracting significant VC interest. The future is vague but boundless

OpenAI’s groundbreaking product development in generative artificial intelligence (AI) was based on a now-storied research paper by eight computer scientists at Google. The 2017 treatise, titled “Attention Is All You Need,” is sometimes referred to as “the transformer paper” after Google’s deep-learning architecture. Indeed, the T in ChatGPT stands for transformer.

Over the past few years, all eight authors have left Google. One is at OpenAI, while the other seven have launched their own start-ups. They have collectively raised more than USD 1bn from the likes of Andreessen Horowitz, Index Ventures, and Nvidia’s NVentures, with two of their companies achieving unicorn status. It’s been an entirely North American story until now.

Llion Jones was the last of the authors to leave Google, having set up Japan-based Sakana.AI last year with a view to pursuing new avenues of generative AI without the bureaucracy of a corporate giant. Last month, Sakana raised a USD 30m seed round featuring Lux Capital, Khosla Ventures, 500 Global, and a host of Japanese VC firms, including Jafco, Geodesic Capital, Miyako Capital, and Global Brain.

Data gatherers

This was not an easy round to access, and for investors used to the relatively less competitive world of Japanese start-ups, it stretched the definition of a seed-stage valuation. Global Brain got a place at the table by mobilising its corporate VC fund alongside telecoms operator KDDI. Sakana had prioritised getting strategic backing in Japan; Sony and NTT Group also participated.

“I still remember the day we heard about this in the investment committee. All the other authors of that paper had launched AI start-ups earlier than him [Jones], and he was still working at Google at that time, the last person not making his own business,” said Ken Kajii, a general partner at Global Brain. “That part was the most attractive to us. He knows the technology transforming the market.”

Jones set up Sakana with fellow ex-Google researcher David Ha, who will serve as CEO. Several other ex-Google employees are part of the founding team. The plan is to explore new architectures for large language models (LLMs), the paradigm for processing unstructured data that makes generative AI possible.

In the simplest terms, Sakana seeks to increase compute power efficiency by connecting small LLMs together rather than building massive systems. The company claims to take inspiration from natural processes related to evolution and collective intelligence such as the behaviour of schools of fish.

The key value proposition is in a perceived pending cliff in computing capacity as generative AI applications go mainstream. The idea is that demand for computing capacity is growing so fast, it will soon outstrip the supply of sufficiently powerful chipsets. The latest version of ChatGPT, for example, is reportedly as much as 500x larger than its predecessor.

“We have to make sure we are on the same page [with investees] about long-term outcomes. What they are pursuing, the pain points in the AI ecosystem, the long-term issues of rising computational complexity, that’s a big barrier. We do not believe this trend is sustainable,” said Shigeru Mitarai, a director at Global Brain who led technical due diligence on the deal.

Global ambitions

Sakana is notable as part of a fledgling generation of Japanese start-ups that are attempting to be global from day one. These companies have English-speaking talent with experience at global companies, early international backing, and command significant valuations.

Global Brain’s recent experience in this theme also includes IT outsourcing services provider Josys, which has around 100 employees across Japan, India, Singapore, Vietnam, and the US. The start-up has raised USD 125m in its first two funding rounds, both of which featured Global Brain. Jafco joined the second round, which will be primarily used to expand in the US and Asia.

Going global in deep tech is a decidedly different challenge from IT services, however. This puts Sakana’s hefty seed round at odds with its lack of commercial product in the context of a VC winter, where investors are prioritising unit economics and the path to profitability above all else.

“As a venture capital fund, we need to make a bet. Is this going to be a home run? We don’t know, but we think this one has a higher chance than the others. Think of the early days of the internet, when nobody knew how to monetize it, then Google and Facebook emerged,” Kajii added.

“The VC winter – I’m calling it an ice age right now, but lots of great companies emerge in this kind of climate. Yes, the valuation is high. Can we justify that with a short-term profit? No. But is there potential to have huge value? Yes.”