Each week we find a new topic for our readers to learn about in our AI Education column.
This week on AI education, we delve into another frequently recurring but rarely defined phrase in the artificial intelligence discourse: compound AI. For anyone working in the financial industry in a role that encounters investing, the concept of compound or composite AI should be pretty familiar—composite AI is just different artificial intelligence models applied in a coordinated manner to achieve a difficult-to-reach outcome. So just like some investment offerings are model portfolios comprised of other models—models of models—composite or compound AI is a model comprised of artificial intelligence models.
Compound artificial intelligence, as it turns out, represents a transition in thinking around AI from individual models built for specific tasks to more generalized systems. The different ways that many different types of AI components can be combined and rearranged is a stepping stone to general artificial intelligence. Composite AI, for our purposes, can be used interchangeably with compound AI, the difference being that composite AI is a blending of different models to broaden the knowledge and capabilities of the system, while compound AI is a combination of different interacting components to complete a complex task or achieve a more complex goal.
How We Got Here
In recent weeks we have discussed some of the more complex, intricate and inter-related applications of generative artificial intelligence, from AI agents to neural networks to large language models. Many of these applications are able function because they are designed as multiple layers of software or hardware, the web of overlapping, interconnected nodes in these layers allow a program or machine to not only answer yes or no questions on a grand scale, but to also understand the nuances of relationships, assign weights or priorities and make decisions using high numbers of complex variables. This is how artificial intelligence can understand speech, see images, read language, write text, diagnose ailments and create treatment plans, among many other high-level tasks.
Composite AI takes this layered, interconnected, interrelated design concept a step further. It combines the best parts of multiple AI models to solve problems so complex that a single AI could not tackle them efficiently.
How Compound AI Works
In our imaginations, a compound AI is probably best thought of as a wheel with spokes. At the hub of the wheel are foundational models designed to not only handle big tasks, but also to offer broad insights and to decompose instructions into specialized segments. Models responsible for decomposing and designating segments of tasks are called orchestrators. At the end of the spokes are specialized models designed to handle specific tasks.
So a compound AI might include different large language models, each with their own specialized domain or language and external tools like data retrieval tools or search engines to deliver new information into the system. The ability to incorporate new information, including data in real time, and to adapt to that information, makes compound and composite systems stand out from static AI models.
Consider a retrieval augmented generation system, or RAG, quickly becoming a ubiquitous application of generative AI within businesses. An RAG combines a large language model with information retrieval, two different types of AI. Typically, a user queries these systems to deliver output or make insights based on data they have recorded and stored, or based on a combination of their own data and data found from an external source. After being queried, an RAG’s retriever system goes into the user’s database to find the relevant information, perhaps then also searching the internet or another external information source to find additional information needed to fulfil the user’s request. Then, another component, a prompt, creates a prompt for a large language model. The large language model, the third component, then delivers output for the user in natural language—a personalized client email, for example, or a timely social media post.
Why Compound or Composite AI Is the Way of the Future
Think of computer hardware—the first modern computers, like all of their ancestors since the abacus, were relatively large, inefficient, and very powerful (for their time), but were also designed to be general computation devices. Over time, as more applications for computers were found, more powerful, specialized and miniaturized hardware was developed. Later, as a retail market for general computing emerged, specialized, miniaturized components were combined into what we recognize as computers today, capable of completing many different tasks.
Within a compound AI system, different models and tolls can be optimized for specific tasks or different parts of tasks, which in turn improves the accuracy and efficiency of the system as a whole. While an individual model can only be improved by scaling up with more data and more training, compound and composite AI offer another path to superior results. While scaling up an individual model necessarily implies greater resource use, including electricity, optimizing with compound or composite AI offers more flexibility in balancing performance and cost.
Compound AI systems are capable of being generalists and specialists. Not only are they capable of being fine-tuned, but they are also better oriented to evolve over time—when one component of the system becomes obsolete, it can be swapped out for a newer version or a different model. If one component of a system is creating a problem in a specific task, it can be excluded. Compound and composite systems designed around creating a consensus among different AI models are not just more accurate than monolithic systems, they also may introduce more security and resilience into artificial intelligence as they are less likely to break down due to a single point of failure: compound systems tend to have some built-in redundancy, and a failure in one component does not necessarily jeopardize the whole.
