The AI List

“Actors are cat­tle,” Alfred Hitch­cock once said. An inflam­ma­to­ry metaphor per­haps, but at the height of the stu­dio sys­tem – when actors were treat­ed like com­modi­ties to be bought, trad­ed and sold – these three words cut the film indus­try deep. In his own face­tious way, Hitch had a point. Today’s movie stars sit com­fort­ably atop the celebri­ty food chain, with the biggest names in the biz fre­quent­ly pop­u­lat­ing rich lists and pow­er rank­ings. Since the 1980s we’ve wit­nessed the rise of the brand actor; dom­i­nat­ed by a select few who are able to gen­er­ate vast per­son­al for­tunes through an exten­sive net­work of mul­ti­me­dia con­tracts. At a time when the glob­al econ­o­my is in a frag­ile state of recov­ery, how­ev­er, could this breed be about to see its suprema­cy wane?

When Dis­ney announced in 2005 that it was plug­ging back into the TRON main­frame for a big-bud­get sequel to its cult ​’80s sci-fi, the first ques­tion asked was whether Jeff Bridges would be repris­ing his role as ENCOM CEO Kevin Fly­nn and his dig­i­tal alter ego, Clu. To the col­lec­tive delight of TRON fans every­where, the answer was yes, but this raised anoth­er issue: Clu, being a syn­thet­ic pro­jec­tion in a vir­tu­al world, would not have aged phys­i­cal­ly. How, then, do you take The Dude back to 1982? Make-up or some degree of dig­i­tal restora­tion would do the trick to an extent. But the mak­ers of TRON: Lega­cy opt­ed for some­thing a lit­tle more radical.

In the film Bridges appears as a dig­i­tal­ly ren­dered ver­sion of his younger self; cap­tured by VFX com­pa­ny Dig­i­tal Domain and lead­ing spe­cial effects super­vi­sor Eric Bar­ba, who pre­vi­ous­ly took Brad Pitt through the sea­sons in The Curi­ous Case of Ben­jamin But­ton. ​“In cre­at­ing a dig­i­tal Jeff, we first had to take a life cast,” explains Bar­ba. ​“We got Jeff into our Nova con­tour rig, much like we did on Ben­jamin But­ton, and we were able to do a library of face shapes from him.

“It’s actu­al­ly pret­ty com­pli­cat­ed,” he con­tin­ues. ​“There’s a rig made, and that rig is dri­ven by an ani­ma­tor, with every­thing ref­er­enc­ing back to the library of shapes that came off Jeff’s face. We put a four-cam­era head-rig [on Jeff] dur­ing prin­ci­ple pro­duc­tion so he could step in and do the lines, walk out of scene, and then we would get an ani­ma­tor in, who was watch­ing off- screen, who we would shoot the actu­al scene with. Once we had the four-cam­era selects, we would pick up the data and those points would be tracked by a computer.”

To all intents and pur­pos­es, the Jeff Bridges in TRON: Lega­cy is not Jeff Bridges. But as Bar­ba reveals, Bridges’ pres­ence in the ear­ly pro­duc­tion stages was essen­tial for the ani­ma­tion team. ​“Because the track­ing points live in 3D space, we had to apply them to an algo­rithm we wrote that we could then com­pare to the archive of facial shapes. But the algo­rithm, as sophis­ti­cat­ed as it is, doesn’t know the dif­fer­ence between, say, a hap­py smile or a sad smile or a surly smile. So we had to look at Jeff and just make those adjust­ments. The nuances in human expres­sions are so fine, the lit­tle things that you see, and the com­put­er may say, ​‘Well this is what the shapes did’, but it may not come across quite the same. You have to man­u­al­ly tweak it. And you keep tweak­ing it until it feels just right.”

Although the intri­ca­cies of this com­plex motion cap­ture process might not seem entire­ly nov­el in con­tem­po­rary VFX terms, Dig­i­tal Domain is way ahead in chart­ing the vir­tu­al actor fron­tier. And, as Bar­ba sug­gests, TRON: Lega­cy will be looked back on as the first ray of a new dawn. ​“After work­ing on Ben­jamin But­ton, peo­ple asked me if what we did with Brad Pitt would open up new avenues. I think TRON: Lega­cy is the first real step for­ward in that sense. Much like the first dinosaur we saw walk in Juras­sic Park, we’re break­ing new ground now.” He con­tin­ues: ​“It’s still an incred­i­bly dif­fi­cult process, par­tic­u­lar­ly in terms of get­ting the per­sona of the char­ac­ter right. It’s still in the arti­san stage. It’s not in the push-but­ton stage yet.”

No one can yet pre­dict how far this inno­v­a­tive dig­i­tal art form can be tak­en. How close are we to being able to repli­cate real life actors using CGI? Could we ever get to the stage where vir­tu­al actors could replace humans? Could the two even become indis­tin­guish­able? Per­haps not entire­ly, sug­gests Paul Franklin, visu­al effects super­vi­sor on Inception.

“You can use dig­i­tal effects to make crea­tures or whole char­ac­ters, like in Avatar, but they are dri­ven by a real per­for­mance under­neath,” he says. ​“There will always be a role for actors inside that some­where – whether they are in a motion cap­ture rig or what­ev­er – and audi­ences will always respond to real human emo­tion. So whether you’re film­ing an actor direct­ly or inter­pret­ing them through com­put­er graph­ics, there’s always got to be that emo­tion­al core. Com­put­ers will nev­er, or at least not for a very long time, be able to spon­ta­neous­ly gen­er­ate that.”

In the case of the TRON fran­chise, how­ev­er, Dis­ney now has an entire cat­a­logue of dig­i­tal shapes indi­vid­u­al­ly assigned to Jeff Bridges’ facial expres­sions. If addi­tion­al instal­ments are green­lit, could the real Bridges poten­tial­ly be ren­dered super­flu­ous? ​“I think that’s the actor’s fear,” says Bar­ba, ​“but I’ll be the first one to say, ​‘I need Jeff.’ Jeff makes Jeff ​‘Jeff’. Jeff does things no one else does; you can’t have an ani­ma­tor doing that. It won’t come across as Jeff. With­out the guid­ance of Jeff’s per­for­mance, it wouldn’t look like Jeff as soon as he moves. Jeff moves his face in a par­tic­u­lar way. If some­one else moves his face, it instant­ly stops being Jeff.”

At present the tech­nol­o­gy relies on a phys­i­cal tem­plate, but as ani­ma­tors become more adept at repli­cat­ing actors’ expres­sions and ges­tures, there may be noth­ing stop­ping future gen­er­a­tions of film­mak­ers adding entire chap­ters to famil­iar sto­ries with­out an actor inputting infor­ma­tion before­hand. ​“There’s a whole gen­er­a­tion of film­mak­ers com­ing through who are com­plete­ly at ease with this kind of work; guys who have grown up in a world of com­put­ers and videogames,” explains Franklin. ​“They don’t see it as a threat; they just see it as anoth­er part of the film­mak­ing process.”

As Hol­ly­wood con­tin­ues to eat itself, remakes and reboots are now par for the course. But what if a stu­dio decides it wants to regen­er­ate pub­lic inter­est in a fran­chise whose star attrac­tion is now unavail­able – be it through death or ill­ness – or sim­ply unsuit­able for the part because of age? While MGM looks increas­ing­ly unlike­ly to be able to meet Daniel Craig’s ask­ing price, thus begin­ning the search for the next Bond, why not turn to tech­nol­o­gy and restore a younger, fuller-haired Sean Con­nery to the role? Cine-purists might shud­der at the thought, but if a digi-Bond was offered at a frac­tion of the cost of the real thing, you can bet stu­dio exec­u­tives would at the very least enter­tain the idea.

With an esti­mat­ed bud­get of $150 mil­lion, the ani­ma­tors on TRON: Lega­cy had the finan­cial means to realise this new blue­print for the vir­tu­al actor. But what hap­pens when the for­mu­la becomes more afford­able? ​“Com­put­ers are get­ting cheap­er and more pow­er­ful all the time, so you’ve got a straight­for­ward increase in horse­pow­er behind what you can do for the same price,” says Franklin. ​“This time next year I could do cer­tain things on Incep­tion for a reduced price, which then increas­es the scope for what the film­mak­er can do.”

As the cre­ative process gets cheap­er, the pos­si­bil­i­ties become infi­nite. But how might this impact the future of con­ven­tion­al act­ing? In this post-Bran­do age, where few work­ing screen actors can claim to be true pio­neers of their craft, does the dig­i­tal medi­um sim­ply rep­re­sent the next step? Bar­ba reck­ons so. ​“I love humans. I love actors. But I think there’s a place for this tech­nique when you have to bring some­one back or de-age them or tell a sto­ry you couldn’t ordi­nar­i­ly tell. It might only be an inter­pre­ta­tion with today’s tech­nol­o­gy as it is, but there’s no rea­son why you couldn’t bring back some of the great names some day.”

Regard­less of when the tech­nol­o­gy arrives, how­ev­er, are audi­ences ready to embrace dig­i­tal rein­car­na­tions of screen icons? The suc­cess of Avatar and Ben­jamin But­ton sug­gests so, but if com­put­er-gen­er­at­ed actors are ever to com­pete with the real thing, ani­ma­tors like Bar­ba must first over­come a num­ber of obsta­cles, name­ly an anthro­po­mor­phic phe­nom­e­non known as the ​‘uncan­ny valley’.

Coined by Japan­ese roboti­cist Masahi­ro Mori, this term denotes the point at which the lev­el of verisimil­i­tude achieved by robots and oth­er fac­sim­i­les of humans caus­es a neg­a­tive emo­tion­al response from the human observ­er. In Mori’s pro­posed graph, the pos­i­tiv­i­ty of human reac­tion plot­ted against the anthro­po­mor­phism of robots hits a dip – or ​‘val­ley’ – at the point of ​‘almost human’ real­ism. Put sim­ply, the more real­is­tic a robot appears, the more like­ly we are to find it unset­tling. Every vir­tu­al per­for­mance is rel­a­tive to the skill and artistry of the indi­vid­ual ani­ma­tor, but the more fre­quent­ly well-recog­nised actors are copied or dig­i­tal­ly res­ur­rect­ed, the high­er the like­li­hood of imper­fec­tions, no mat­ter how dis­creet, being ampli­fied becomes.

As Bar­ba admits, the great­est chal­lenge fac­ing his trade is find­ing a way to trick the human mind. ​“From the day we come out of the womb, we see oth­er human faces. Whether it’s a baby in Africa or Indone­sia or the Unit­ed States, the expres­sions a human face makes, regard­less of lan­guage or cul­ture, are inher­ent to our species. A smile is a smile. We learn to read these nuances instant­ly – we don’t even think about it. The same thing applies with a CG head – you read it instant­ly. And when it’s not right, you know it’s not right. It comes down to an infi­nite amount of sub­tleties, whether you hit enough of them so that the audi­ence instant­ly buys the per­for­mance. When you nail it, it’s mag­ic. When you miss it, you’re back in the valley.”

For now, this phe­nom­e­non pos­es the sin­gle great­est threat to the devel­op­ment of vir­tu­al act­ing. But the cur­rent rate of tech­no­log­i­cal progress may well dic­tate a shift in atti­tudes. As a con­cept, vir­tu­al act­ing is still very much in its infan­cy, but this is a sci­ence, not a gim­mick. It may be hard to imag­ine a time when flesh-and-blood stars have become com­plete­ly out­mod­ed, but at $80 mil­lion a pop, it’s no won­der Hol­ly­wood is so keen to nur­ture the growth of the vir­tu­al actor. One thing is for cer­tain: the game is chang­ing. The rev­o­lu­tion sequence has been initiated.