Adept aims to build AI that can automate any software process

In 2016 at TechCrunch Disrupt New York, several of the original developers behind what became Siri unveiled Viv, an AI platform that promised to connect various third-party applications to perform just about any task. The pitch was tantalizing — but never fully realized. Samsung later acquired Viv, folding a pared-down version of the tech into its Bixby voice assistant.

Six years later, a new team claims to have cracked the code to a universal AI assistant — or at least to have gotten a little bit closer. At a product lab called Adept that emerged from stealth today with $65 million in funding, they’re  — in the founders’ words — “build[ing] general intelligence that enables humans and computers to work together creatively to solve problems.”

It’s lofty stuff. But Adept’s cofounders, CEO David Luan, CTO Niki Parmar and chief scientist Ashish Vaswani, boil their ambition down to perfecting an “overlay” within computers that works using the same tools people do. This overlay will be able to respond to commands like “generate a monthly compliance report” or “draw stairs between these two points in this blueprint,” Adept asserts, all using existing software like Airtable, Photoshop, Tableau and Twilio to get the job done.

“[W]e’re training a neural network to use every software tool in the world, building on the vast amount of existing capabilities that people have already created.” Luan told TechCrunch in an interview via email. “[W]ith Adept, you’ll be able to focus on the work you most enjoy and ask our [system] to take on other tasks … We expect the collaborator to be a good student and highly coachable, becoming more helpful and aligned with every human interaction.”

From Luan’s description, what Adept is creating sounds a little like robotic process automation (RPA), or software robots that leverage a combination of automation, computer vision, and machine learning to automate repetitive tasks like filing forms and responding to emails. But the team insists that their technology is far more sophisticated than what RPA vendors like Automation Anywhere and UiPath offer today.

“We’re building a general system that helps people get things done in front of their computer: a universal AI collaborator for every knowledge worker … We’re training a neural network to use every software tool in the world, building on the vast amount of existing capabilities that people have already created,” Luan said. “We think that AI’s ability to read and write text will continue to be valuable, but that being able to do things on a computer will be significantly more valuable for enterprise … [M]odels trained on text can write great prose, but they can’t take actions in the digital world. You can’t ask [them] to book you a flight, cut a check to a vendor, or conduct a scientific experiment. True general intelligence requires models that can not only read and write, but act when people ask it to do something.”

Adept isn’t the only one exploring this idea. In a February paper, scientists at Alphabet-backed DeepMind describe what they call a “data-driven” approach for teaching AI to control computers. By having an AI observe keyboard and mouse commands from people completing “instruction-following” computer tasks, like booking a flight, the scientists were able to show the system how to perform over a hundred tasks with “human-level” accuracy.

Not-so-coincidentally, DeepMind cofounder Mustafa Suleyman recently teamed up with LinkedIn cofounder Reid Hoffman to launch Inflection AI, which — like Adept — aims to use AI to help humans work more efficiently with computers.

Adept’s ostensible differentiator is a brain trust of AI researchers hailing from DeepMind, Google and OpenAI. Vaswani and Parmar helped to pioneer the Transformer, an AI architecture that has gained considerable attention within the last several years. Dating back to 2017, Transformer has become the architecture of choice for natural language tasks, demonstrating an aptitude for summarizing documents, translating between languages, and even classifying images and analyzing biological sequences.

Among other products, OpenAI’s language-generating GPT-3 was developing using Transformer technology.

“Over the next few years, everyone just piled onto the Transformer, using it to solve many decades-old problems in rapid succession. When I led engineering at OpenAI, we scaled up the Transformer into GPT-2 (GPT-3’s predecessor) and GPT-3,” Luan said. “Google’s efforts scaling Transformer models yielded [the AI architecture] BERT, powering Google search. And several teams, including our founding team members, trained Transformers that can write code. DeepMind even showed that the Transformer works for protein folding (AlphaFold) and Starcraft (AlphaStar). Transformers made general intelligence tangible for our field.”

At Google, Luan was the overall tech lead for what he describes as the “large models effort” at Google Brain, one of tech giant’s preeminent research divisions. There, he trained bigger and bigger Transformers with the goal of eventually building one general model to power all machine learning use cases, but his team ran into a clear limitation. The best results were limited to models engineered to excel in specific domains, like analyzing medical records or responding to questions about particular topics.

“Since the beginning of the field, we’ve wanted to build models with similar flexibility as human intelligence-ones that can work for a diverse variety of tasks … [M]achine learning has seen more progress in the last five years than in the prior 60,” Luan said. “Historically, long-term AI work has been the purview of large tech companies, and their concentration of talent and compute has been unimpeachable. Looking ahead, we believe that the next era of AI breakthroughs will require solving problems at the heart of human-computer collaboration.”

Whatever form its product — and business model — ultimately takes, can Adept succeed where others failed? If it can, the windfall could be substantial. According to Markets and Markets, the market for business process automation technologies — technologies that streamline enterprise customer-facing and back-office workloads — will grow from $9.8 billion in 2020 to $19.6 billion by 2026. One 2020 survey by process automation vendor Camunda (a biased source, granted) found that 84% of organizations are anticipating increased investment in process automation as a result of industry pressures, including the rise of remote work.

“Adept’s technology sounds plausible in theory, [but] talking about Transformers needing to be ‘able to act’ feels a bit like misdirection to me,” Mike Cook, an AI researcher at the Knives & Paintbrushes research collective, which is unaffiliated with Adept, told TechCrunch via email. “Transformers are designed to predict the next items in a sequence of things, that’s all. To a Transformer, it doesn’t make any difference whether that prediction is a letter in some text, a pixel in an image, or an API call in a bit of code. So this innovation doesn’t feel any more likely to lead to artificial general intelligence than anything else, but it might produce an AI that is better suited to assisting in simple tasks.”

It’s true that the cost of training cutting-edge AI systems is lower than it once was. With a fraction of OpenAI’s funding, recent startups including AI21 Labs and Cohere have managed to build models comparable to GPT-3 in terms of their capabilities.

Continued innovations in multimodal AI, meanwhile — AI that can understand the relationships between images, text and more — put a system that can translate requests into a wide range of computer commands within the realm of possibility. So does work like OpenAI’s InstructGPT, a technique that improves the ability of language models like GPT-3 to follow instructions.

Cook’s main concern is how Adept trained its AI systems. He notes that one of the reasons other Transformer models have had such success with text is that there’s an abundance of examples of text to learn from. A product like Adept’s would presumably need a lot of examples of successfully-completed tasks in applications (e.g., Photoshop) paired with text descriptions, but this data that doesn’t occur that naturally in the world.

In the February DeepMind study, the scientists wrote that, in order to collect training data for their system, they had to pay 77 people to complete over 2.4 million demonstrations of computer tasks.

“[T]he training data is probably created artificially, which raises a lot of questions both about who was paid to create it, how scalable this is to other areas in the future, and whether the trained system will have the kind of depth that other Transformer models have,” Cook said. “It’s [also] not a ‘path to general intelligence’ by any means … It might make it more capable in some areas, but it’s probably going to be less capable than a system trained explicitly on a particular task and application.”

Even the best-laid roadmaps can run into unforeseen technical challenges, especially where it concerns AI. But Luan is placing his faith in Adept’s founding senior talent, which includes the former lead for Google’s model production infrastructure (Kelsey Schroeder) and one of the original engineers on Google’s production speech recognition model (Anmol Gulati).

“[W]hile general intelligence is often described in the context of human replacement, that’s not our north star. Instead, we believe that AI systems should be built with people at the center,” Luan said. “We want to give everyone access to increasingly sophisticated AI tools that help empower them to achieve their goals collaboratively with the tool; our models are designed to work hand-in-hand with people. Our vision is one where people remain in the driver’s seat: discovering new solutions, enabling more informed decisions, and giving us more time for the work that we actually want to do.”

Greylock and Addition co-led Adept’s funding round. The round also saw participation from Root Ventures and angels including Behance founder Scott Belsky (founder of Behance), Airtable founder Howie Liu, Chris Re, Tesla Autopilot lead Andrej Karpathy, and Sarah Meyohas.