Lenovo, Autodesk®, and Intel® partner with an artificial intelligence visionary to help produce Hollywood blockbusters. In what sounds like the plot of a science fiction film, a comic book artist turned computer scientist has created a computer that can think. Harnessing the power of a room full of Lenovo ThinkStation workstations powered by Intel® Xeon® processors, he’s using his creation to help produce stunning special effects for some of the latest Hollywood blockbusters.
Athena puts the hardware and software to the test. Bolden estimates that Athena can produce CG renderings and animations up to three times faster than the multicomputer, labor-intensive practices currently used by competing animation services. But Athena isn’t just a faster animation system; it actually creates finished animations based on natural language input. Bolden and his team at BTI provide text-based input, and Athena produces rendered, animated output based on its understanding of the scene being described.
Athena is an artificial intelligence (AI) that automatically knows, based on the story line, which character to put on camera, how to coordinate dialogue and interaction with the stage, how the stage props or environment will interact with characters, and how to best render high-quality computer graphics. And Athena learns from each experience and uses that accumulated knowledge to solve future problems.
In many ways, Athena is similar to Watson, the AI system developed by IBM that appeared on the television game show Jeopardy! in 2011, during which it beat two of the show’s past champions. Watson received game clues as text files, which it then parsed and compared against its 15-terabyte database of human knowledge. “You could think of Watson as kind of the left brain solution,” said Bolden, “whereas Athena is more creative— more of a right brain solution. Athena can understand what you say through natural language, but instead of answering a question, it shows you the results visually.”
With Athena, Bolden provides textbased input, which the program converts to visual output. The goal for Athena is to enable artists to spend more time being artists and less time as what Bolden refers to as “pipeline junkies,” operators who have to figure out a lot of math and code and settings to produce images and animations.
BTI used Athena to produce sequences for Transformers: Dark of the Moon using Autodesk Maya® software. Artists first removed actors, effects, and vehicles in each shot, filled in what would have been behind them, and then reassembled the scene in 3D.
When processing the text, if Athena doesn’t understand something, it presents a series of question marks around unknown words or phrases. Bolden can then either provide a more detailed description or simply specify a link to a visual object. Once Athena generates its output, artists can build upon those results using Autodesk Maya or Autodesk 3ds Max® software. Athena then watches what the artists subsequently do and adds that knowledge to its ever-growing database so that it can leverage that knowledge in the future.
Bolden certainly hadn’t set out to create an artificial intelligence. He began his career as an intern at Marvel Comics, but found he also had a knack for programming. He left Marvel and went up the coast to attend the University of California at Berkeley, where he studied artificial intelligence and network infrastructure. Bolden left Marvel but returned as his friends there were getting into the movie business. According to Bolden, they said “Hey, Allen. You do something with computers, right?’ So I started utilizing some of the stuff from my AI work.” Bolden says that to a large extent, Athena was just “blind luck.”
Bolden was called in by a studio to fix a problem they had run into on a project. He had only five days to complete the task. “I thought it would at least make some of the calculations faster,” Bolden explained, “but it ended up doing much more than I thought. That was the start of Athena.”
When Bolden first created Athena, it ran on a cluster of nearly 70 computers, using whatever systems he could get his hands on. Today, Athena runs on a cluster of approximately 30 Lenovo ThinkStation D20 workstations—mostly dual quad-core machines powered by Intel. Two years ago, BTI started out with 45 computer graphic artists working on three different projects simultaneously. To date, the firm has coordinated as many as 150 CG artists in the U.S., with additional artists in Taiwan, Seoul, and Hong Kong contributing to BTI’s efforts as it works on portions of three major motion pictures at a time.
In addition to the Lenovo workstations comprising Athena, BTI uses several Lenovo ThinkStation D20, C20, and S20 workstations.
A team of nearly 50 BTI artists worked on select scenes for the summer 2011 blockbuster Transformers: Dark of the Moon. The team’s work shows up in approximately 10 minutes of the film and mostly involves compositing— combining visual elements from different sources into a single image to make them appear to be part of the same scene. Since the film would be shown in 3D, this work was quite intensive, with scenes originally shot in 2D having to be recreated in 3D using a process called projection mapping.
All of the projection mapping was done using Autodesk Maya software. Artists first had to remove all of the actors and vehicles in each shot and fill in what would have been behind them, a process known as clean plating. Depending on the angles, some of these shots required recreating the entire background in CG using Autodesk Maya. Then the entire scene was reassembled.
“It can be very render-intensive,” Bolden explained, “because now you’re dealing with both the 2D compositing and the CG. We found a good methodology for doing this, which we were able to teach Athena.” Bolden estimates that approximately 60 percent of the projection mapping was done using Autodesk Maya software, with the remainder done in the Nuke compositing software from The Foundry.
In another Transformers sequence in which Shia LaBeouf gazes out at an explosion, BTI’s artists had to create the entire sky because the camera rotated around the actor’s head. The entire scene was generated in Autodesk Maya, including the sun and sky. “Autodesk Maya was the clear choice for us,” Bolden said.
BTI also produced nearly 40 minutes of Smurfs, using similar techniques. On average, Bolden estimates that BTI renders nearly 10,000 frames per week. At 24 frames per second, that’s the equivalent of around seven minutes of a film, representing anywhere from 10 to 25 separate shots. All of BTI’s work is rendered in-house, requiring approximately three terabytes (TB) of storage. BTI currently maintains a storage capacity of 17TB of operational storage on a RAID1 configuration.
Bolden has compared the results he’s obtained using the firm’s Lenovo workstations to those of other computers in BTI’s arsenal. For Transformers: Dark of the Moon, it took nearly 96 minutes to render 196 frames of a typical CG shot on a system equipped with a six-core processor and 8GB of RAM. The same sequence required only 25 minutes on a Lenovo ThinkStation D20 equipped with dual quad-core Intel Xeon processors and 12GB of RAM. In another test, a stylized character scene containing 21 hair systems with 140,000 separate hair strings modeled in Autodesk Maya and rendered in mental ray®, Bolden’s Lenovo ThinkStation S20 workstation was able to complete each frame in just over five minutes while his personal system was unable to render the scene at all. Bolden’s coworkers were even more enthusiastic in their praise. “As the lead Nuke compositor at Bit Theory, I have been using the Lenovo ThinkStation D20 workstation powered by Intel Xeon processors extensively for the past four months for stereo conversion on such films as Transformers: Dark of the Moon, Conan the Barbarian, and Smurfs. The Foundry’s Nuke software has crashed no more than five times during the entire project, while many of our other artists have reported that Nuke would crash more than twice a day on their six-core systems,” reported Abo Biglarpour. “Artists also found that their computers slowed down significantly when viewing 4K plates at full resolution, but with the Lenovo ThinkStation D20, no issues were ever reported when working with 4K plates at full resolution.”
BTI continues to push technology to its limit to develop Athena. BTI’s development goals for the technology are to create workflow efficiencies that make CG production faster and cheaper without sacrificing quality. BTI is designing Athena to be easy to use by creative professionals and also to scale for both large and small projects.
Athena is still in its early stages of development; the company has embarked on an 18-month R&D project. It hopes to increase Athena’s power by running it on up to 40 Lenovo workstations and to develop it to the point that it could have commercial potential beyond BTI’s own projects.
The personal relationship with Athena appears to extend to its hardware as well. Bolden and his team at BTI “love the Lenovo machines,” and look forward to expanding their use of Lenovo workstations.