Each week we find a new topic for our readers to learn about in our AI Education column.
I’m not a great visual artist. Having a nearly 4 year old in the house has really brought me to that conclusion—he asks me to draw all kinds of weird stuff, like a duck, or a truck, or a cat, or a bat, and what I end up drawing usually doesn’t really look like much of anything.
Luckily, his mom is a wonderful artist.
But what if instead of asking Dad to draw a duck or a truck or a cat or a bat, he could just yell at a computer to create one he could interact with in three dimensions?
Welcome to another edition of AI Education, where this week we’re going to take yet another approach at artificial intelligence topics. To date, we’ve usually focused on either foundational topics for AI or noteworthy companies, models, or developers related to AI. Today we’re going to discuss 3-D generative artificial intelligence, or 3D GenAI.
We’ve discussed generative artificial intelligence, or GenAI, several times before—it is the technology underlying popular AI chatbots and image generators. Well, 3D GenAI uses the same technology to develop—or draw—three-dimensional digital models or real-world objects. It is not 3D-printing, as 3D printing is only concerned with physically creating a digital 3D image. 3D GenAI uses automation to create the 3D image, which can then be made into a physical object using processes including 3D printing.
As for how we got here this week: We’ve been reading a little bit about different types of generative AI outputs, and the possibilities for 3D generative AI kind of jumped out at us as interesting to discuss. Our (brief and tangential) personal history with 3D modeling and design—mostly related to a family member who worked on a couple of gaming projects—helped us to grasp its significance: Software can automatically create a 3D model for us based on natural language prompts. The rough equivalent of Amazon’s Alexa can now create almost instantly what once required many hours of professional labor to produce.
What Is 3D Modeling
3D modeling has its roots in sculpture and architectural drafting, but my first intimate encounter with it was in video game design. Around 30 years ago, video games moved formally into three dimensions where the environments and objects players interacted with were truly rendered in three-dimensional space rather than as a two-dimensional, scrolling screen. However, this innovation was preceded by three-dimensional drafting and computer-aided design (CAD) software and three-dimensional videos.
Creating something in 3D is much different from two dimensional drawing. Think about the leap made in the middle ages and renaissance when artists began to understand how to use light and space and depth in their paintings to create the illusion of three dimensions. We’re talking about more than that— 3D modeling creates an object that we can spin or rotate around in real or virtual space. It involves computers having the same understanding of light and shadow and depth and space that generations of artists slowly developed, and that most individual artists only develop with training and practice.
3D CAD allowed architects and engineers to make precise models of the objects they were building or working with. Designers also used to similar software to create everything from advertising to clothes. The software took the manual skill of drafting out of the hands of humans—no longer did someone need to have a basic level of artistic skill to draft the design of a building or machine or an outfit for others to follow—but they still had to understand the principles of design and how to articulate a vision or an idea into a practical, real-world object.
Over the years, innovators have found new, far-reaching applications for 3D modeling, including in the molecular formulation phase of drug and materials development, in studying archeological sites and objects, in mapping geographical locations, in dentistry, surgery and internal medicine, and in the creation of virtual actors and characters.
What 3D Generative AI Means
That short history lesson is to illustrate that, even with software, 3D modeling could be a painstaking process, especially in animation, where creators might have to spend hours posing and re-skinning their models repeatedly to create the desired effects. One problem being that there was no one agreed upon method for building a three-dimensional object and representing it digitally.
Until recently, 3D generative AI was still pretty slow, inaccessible and rudimentary, and in some ways, it still is. While 3D designers are keeping their eyes on the technology, they’re not yet feeling like it is sophisticated enough to displace them. Our emphasis is on the yet, of course. 3D GenAI models are not only becoming faster, they’re also generating output of high enough quality and precision to be used in today’s video games—and they’re now interacting in natural language.
Now, someone who wants to develop a video game can just ask an artificial intelligence generator to create a character or a setting for them in natural language, using text or voice prompts to tell the AI model the specifics of what they’re looking for. Anyone could describe the dimensions and uses of a living or working space they desired, and 3D generative AI could generate a usable schematic. An entrepreneur or thought leader with an idea for a product—like a household appliance—could speak their idea aloud to a 3D generative AI, and create plans for that device that they could then feed into a 3D printer.
3D generative AI greatly narrows the gap between inspiration and reality.
3D Generative AI Types and Providers
3D GenAI can be sorted by input type. There are text-to-3D models, that create models from text descriptions; image-to-3d models which convert a two-dimensional image to a 3D model; and video-to-3D models that can create a 3D object from an uploaded video.
Believe it or not, this is a maturing space, with quite a few 3D AI generators available for a relatively low cost and easily accessible. Much of the big, professional software remains impractical for the general public. Providers, large and small, include:
- Blender
- Ponzu
- Luma AI
- RDFY AI
- Autodesk
- Meshy
- Masterpiece X
- Spline
- Sloyd
- NeROIC
- Houdini
- Rokoko Vision
- NVIDIA Omniverse
