motion capture（动作捕捉）和performance capture（表演捕捉）的“前世今生”之路
NO.1 最近，越来越多的国内外影视项目采用motion capture（简称mo-cap，动作捕捉)技术。最近抽空翻阅了一堆相关的资料加深学习与了解。这里，给大家安排我觉得不错的学习顺利，翻译剪辑整理出来，方便大家学习使用。今天会分享三个相关视频内容，分别从：1.什么是
NO.1 最近，越来越多的国内外影视项目采用motion capture（简称mo-cap，动作捕捉)技术。最近抽空翻阅了一堆相关的资料加深学习与了解。这里，给大家安排我觉得不错的学习顺利，翻译剪辑整理出来，方便大家学习使用。今天会分享三个相关视频内容，分别从：1.什么是
an industry standard method for capturing different aspects of an actor’s
performance. Like general movements or facial expressions and then converting
them into digital data. This data can be used in many different ways, like
applying it to an animated character for more realistic movements or taking it
up a notch and creating full-on digital human in movies.
way back to the 1910s when an animator named Max Fleischer came up with a
technique called rotoscoping. He used rotoscoping in his animated series called
“Out of the Inkwell” and the way it worked was Max’s brother. David, would
dress up as a clown for the role of Coco the Clown and then act normally for
the roll in front of the camera. Later on, Max would use the footage as a
reference to draw over frame by frame to create the animation by tracing the
human movements and translating it to the animated character.
but with different techniques, the most common way of doing motion capture today
is to cover the actor’s body mostly around the joints with reflective markers. The
actor will then play his or her role normally in front of many cameras that are
designed to track the reflective trackers and save their movement data in a 3D
environment. These trackers come in many different varieties with different
application but they all have the same purpose, tracking body movements or
facial expressions in detail. That VFX artists can use later for the film
project.
grown, especially in VFX heavy movies from “The Planet of the Apes” and “The Lord
of the Rings” to Marvel Universe and Star Wars films. The most common use of
motion capture in films is to animate fantastical characters. Examples of using
motion capture for character creation are rampant but the biggest name in this
is Andy Serkis.
capture. He has a list of characters that he has brought to life, like Caesar
from “The Planet of the Apes” series. Glum from “The Lord of the Rings”, Supreme
Leader Snoke from the recent “Star Wars” series and Captain Haddock from “The
Adventure of Tintin” Serkis even played two roles in the 2005 Peter Jackson’s
adaptation of “King Kong”. One as the Kong himself in a motion capture style,
and one as Lumpy the Cook in a normal acting style.
using these special suits and cameras specifically made for motion capture. Every
movement of the actor is translated to the body movement of the CG character. The
other way of using motion capture in the film is specifically designed for
capturing the facial expressions which is called “performance capture” . In
performance capture the actor’s face is covered with trackers and then a camera
is mounted on their head , facing their face. This way every movement of facial
muscles is tracked and recorded. One of the most notable performances for this
method of motion capture is done by Benedict Cumberbatch for the role of Smaug
in the Hobbit series. This example shows that motion capture has come to the
level that movements and expressions can be translate not only to human-like figures
but on to anything even a dragon.
performance capture that has become more abundant in the past decade is the
process of designing and creating digital humans. Star Wars franchise is one of
the pioneers of using this technology in their films. In “Rouge One: A Star
Wars Story”, the VFX team ILM used their award-winning technology to recreate Grand
Moff Tarkin which was played by the late Peter Cushin in 1977. For recreating Grand
Moff Tarkin, another actor, Guy Henry was tasked to play in place of Peter
Cushing. The technology requires different scans that need to be done before
even the actors sets foot on the film stage. First , there would be a light
scan of the face which would capture extremely high detailed scans of the face
of the actor capturing. Every light reaction in different facial expressions. Then
there will be another scan done in a Medusa Rig owned by Disney research. After
all is done , the performance capture part begins where the actor in this case,
Guy Henry, will mount the rig on his head and start performing his parts. Then all
these data combined with the scans and digital recreations of Peter Cushing’s
face. The Grand Moff Tarkin of the film “Rouge One: A Star Wars Story” was
created of course the process was a bit more detailed and more complicated. So for
a deeper explanation of this process you can check out the great video done by “WIRED”
that talks about the history and complete process of creating digital humans.
Henry将头戴式相机安装在他的头上，并开始表演他的部分。然后，所有这些数据与Peter Cushin的面部扫描和数字再现相结合。电影《星球大战外传：侠盗一号》中的Tarkin是由他创造的，当然这个过程更详细也更复杂。因此，要想更深入地解释这个过程，你可以看看《连线》(WIRED)做的一个很棒的视频，它讲述了创造数字人类的历史和完整过程。
NO.2
Andy Serkis 作为专业且资深的动作捕捉演员，参与了《指环王》《人猿星球》《丁丁历险记》《金刚》等好莱坞大片，且这些视效量巨大的影片中，大量地使用了motion capture这个技术。所以，作为“资深前辈”的他为大家“代入式”讲解动作捕捉技术的发展与进步，我觉得是个很不错的学习案例。
Wolfe the Planet of Apes is probably the
most evolved performance capture that is seen today. But when I think back to
the Rings and where it all started there’s been such a huge journey in terms of
the evolution of the technology. The hardware, the cameras, the software,
everything that goes into transforming and actors performance into an avatar
character.
《人猿星球》中的沃尔夫可能是当今最具表现力的演员。但当我回想起《指环王》以及它的起源时，就技术的发展而言，这是一段漫长的旅程。包括硬件，摄像头，软件，所有用于转换的东西以及演员的表演。
Motion capture came out the medical
industry but then it was used in early video games. So that would be by using a
suit with retro reflective markers placed at joints in the body. The
information which is then piped into the computer that provides an electronic
skeleton which is then added to an animated rig.
运动捕捉是从医疗产业产生的，然后又被应用于视频游戏里。它通过在肢体关节处放置反光标记的衣服，然后把些信息传输到计算机，由计算机生成一个骨骼，然后再被添加到一个动画装置中。
The
Lord of Ring 2001 《指环王》2001
So the process of shooting on all the Rings
was to go on set and work with the other actors on location. Then we’d have to
do a second pass with the camera. I would stand off the side and then they
would react to the ball on a stick and they would then capture their
performances. Then I would go back to a motion capture studio where I’d climb
into the motion capture suit. They would play back the scenes that I’ve been
working and I would be able to see on the screen in real time. A representation
of Gollum, so if I lifted up my right hand Gollum all right and if I twisted my
body it would totally move in sync with my movements. It was in effect being
like a puppeteer and marionette at the same time.
《指环王》的拍摄过程我是与其他演员一起在片场完成的。我们需要在摄影机前完成两遍。我会先站在一边，然后他们对球棒做出反应，然后他们会捕捉他们的表演。然后我再回到动作捕捉工作室，穿上动作捕捉服。他们会回放我一直在工作的场景，我可以在屏幕上实时看到。是咕噜的表演，所以如果我举起我的右手，咕噜也一样，如果我扭动我的身体，它会和我的动作完全同步。它和我实际上就像一个木偶师和提线木偶。
King
Kong 2005 《金刚》2005
The next film that I worked on was King
Kong. The difference this time around was that it made the transition from
motion capture to performance capture. Because I was wearing a hundred and
thirty two retro reflective markers on my face, which were assigned to muscle
groups that would then directly drive the digital mask of King Kong. All of
which were picked up by a bank of cameras, only a hundred and eighty degree
filed, so I couldn’t turn my back. The basis of the character design the actor
sees from a very early point. So you see, I know that his facial design was
going to accentuate some of my own facial structure. So this is where we get
the transition to what we call performance capture. It’s capture audio facial
and body all at the same time. The task in post-production is always going to
be that ability to take the actors performance and actually be interpolated
performance whilst retaining it but also putting on the physiognomy of a 1/8.
So it’s almost like looking at a costume that you’re going to put on and choose
as an actor and you find a relationship between yourself in the Avatar.
我参与的下一部电影是《金刚》。这次的不同之处在于，它从动作捕捉过渡到了表演捕捉。因为我脸上带着132个反光标记，它们被分配到肌肉组，然后这些数据直接驱动金刚的数字面部。所有的镜头都被一排摄像头拍到了，但是它们只有180度的空间，所以我不能转身。演员很早就看到了角色的基础设计。所以你看，我知道他的面部设计是为了突出我自己的面部结构。这就是我们过渡到我们所谓的表演捕捉的地方。它能同时捕捉声音、面部和身体。后期制作的任务就是将演员的表演和现实的东西融合，同时保留实际大概1/8的内容吧。这就像你在看你要穿上的动捕服，然后选择一个角色，然后你在阿凡达与自己中找到之间的关系。
The
Adventure of TinTin-2011 《丁丁历险记》 2011
Then next significant change was TinTin.
This was when started to introduce head mounted cameras. So this this allowed
the actor to be shot in 360 degrees. It meant that you are untethered and
therefore can move about and play part very naturally. The next significant
change was taking everything out of a motion capture studio and being able to
shoot on location and this happened with the Rise of the Planet of the Apes.
下一个重要的变化就是《丁丁历险记》。这就是我们开始引进头戴式相机的时候。这让演员可以360度被拍摄。这意味着你是不受束缚的，因此可以很自然地移动和扮演角色。下一个重大的变化是把所有的东西都从动作捕捉工作室里移出来放进外景中，这在《猩球崛起》中发生了。
The
Rise of the Planet of the Apes-2011 《猩球崛起》 2011
So this was the first time that we had this
real synergy and ability to create the scene once in camera both using
traditional film cameras but also motion capture cameras and boom mounted
cameras for facial expressions. Now this is still interiors. This is the first
for instance in Rise of the Planet of the Apes. It was the house that Caesar brought
up in and they also built a section of San Francisco bridge for the un-active
end sequence where there were upwards of I think 200 motion capture cameras
placed all along a stretch of purpose-built set.
所以这是我们第一次有这种真正的协同作用和能力来在摄像机中创造场景，使用传统的胶片摄像机和动作捕捉摄像机以及用于面部表情的吊杆式摄像机。这仍然是内部。这在《猩球崛起》中是首次出现。那是凯撒带进来的房子，他们还为非活动的结束镜头建造了旧金山桥的一部分，我想有超过200个动作捕捉摄像机沿着一段专门建造的场景放置。
The next jump forward was then taking and
making all these suits much more robust and by going outside. We need to have a
slightly different approach to the markers. They were no longer
retro-reflective but actually sent out an infrared pulse so that meant that we
could shoot outdoors in bridge light. Dawn to it a stage further by taking it
into much more. Shall we say challenging environments such as the Vancouver
rain forests and when we’re shooting in New Orleans, we were 100% humidity and
the technology had to stand up to all of that and that again was taken to a
further extreme in War for the Planet of the Apes.
下一步就是让这些动捕服更加牢固，并且可以在外拍摄。我们需要用稍微不同的方法来处理标记。它们不再是反向反射的，而是发出红外脉冲，这意味着我们可以在桥的光线下拍摄户外。到下一阶段的时候我们会遇到更多的挑战。我们可以说，在温哥华热带雨林这样具有挑战性的环境中，当我们在新奥尔良拍摄时，我们的湿度是100%，而技术必须承受所有这些，这在《猩球崛起3》中又被发挥到了进一步的极端。
War
for the Planet of the Apes- 2017《猩球崛起3》2017
When we shooting in conditions and I shoot
up to our wasting snow. All of this technology had been built to the point
where it was in credibly robust. Now as well as rendering and texturing, the fur,
the eyes, the hair, water the ability to
have snow or rain on. For this technology had been evolving so that the
simulations were all becoming much more photo real. So many people think studying
the role for Caesar, for Instance, is purely about copying eight movements and
to put everybody straight on that. That is really sort of room 101. That’s where
you begin after a while you realize that it’s not just about mimicking behavior.
This is about creating a character. So there are all these psychological and emotional
journeys that one has to import into the character. So it’s not just about the
physical build of the character but the whole internalization and that’s why
the misconception that performance capture acting is a sort of genre or type of
acting is completely wrong. It’s no different to any process that you go
through to create a role with your onstage in front of a screen in the more
conventional sense. The actors performance is the actors performance.
当我们在恶劣的条件下拍摄时，我把镜头对准了我们荒芜的雪地。所有这些技术都是建立在一个可靠的坚实的点上。现在，除了渲染和纹理，皮毛，眼睛，头发，水，雪或雨的能力。因为这项技术一直在发展，所以所有的模拟都变得比照片真实得多。所以很多人认为研究凯撒的角色，就是纯粹地模仿八个动作，然后让每个人都明白这一点。这是101号房间。这就是你开始的地方，一段时间后你会意识到这不仅仅是模仿行为。这是关于创造一个角色。所以我们必须把所有这些心理和情感的旅程导入角色中。所以这不仅仅是关于角色的身体构造，而是整个内在化，这就是为什么认为表演捕捉表演是一种类型或表演类型的误解是完全错误的。这和你在舞台上在屏幕前创造一个角色的过程没有什么不同。演员的表演就是演员的表演。
NO.3
对于motion capture的了解已经有了一定的基础，现在可以再加深下对于该技术的认识。且经过这些年的发展，mo-cap的服装、跟踪等相关设备及技术也在不断的提升和变化着。
we’ve all seen some photo like this one ,
with some poor actor in a weird outfit, covered in dots whether it’s on “ The
Hulk”Mark Ruffalo’s Instagram or Star Wars’C3PO’s Anthony Daniels on his
account. But where does that motion go?
By focusing less on the actors an more on
how it works. You can understand the real artistry behind motion capture.
我们会看到一些这样的画面，这些演员被一个覆盖着点状物的怪异的装束包裹着。就比如绿巨人Mark Ruffalo Instagram上的自拍这样，又或者是星球大战里的扮演者Anthony
Daniels社交账号上的这张。不过这样的运动是要走向哪里？让我们把关注点多放在它本身上的话，我们就可以理解动作捕捉背后的真正的艺术了。
There are a few main ways to capture
motion. This is Rokoko’s Smartsuit Pro and it uses inertial sensors. These broadcast
the location of devices embedded inside the suit, kind like how your phone
knows which way it’s turned. More common in high end video games and movies,
you’ll see optical tracking of markers in which a camera is learning where
parts of a person’s body are by looking for high contrast areas.
有几种主流的捕捉运动。这是Rokoko的动捕设备和它所使用的内部传感器。这些设备会被嵌入服装内部，就像你所使用的手机敏感地知道你如何转动屏幕一样。这样的东西更多的常见于高端游戏和电影中，你可以看到光学跟踪标记，它的原理是摄像机通过寻找高对比度区域来学习区分人身体的部分。
Jimmy Corvan directed business development
for motion capture studio, House of Moves for about 10 years. It worked on
everything from the Injustice series to Mortal Kombat to Marvel movies to Barbie’s
blog.
Jimmy
Corvan曾在动作捕捉工作室House of Moves做BD工作超过10年。从《复仇者联盟》系列到《格斗之王》，从漫威电影到芭比的博客等项目，他都参与其中。
Phil Edwaids –VOX 提问：
Is it more common in high end stuff like
what you all are doing? Is that pretty much always going to be optical thing,
rather than a sensor?
这在高端领域是很常见的技术吗？这里更多涉及到的是光学的东西，而不是传感器是吗？
Jimmy Corvan
For the foreseeable future and I’m sure
there’s somebody at one of the inertial companies that will correct me on this ,but
in all of our testing and everything that we’ve found optical motion capture is
sub-millimeter accurate.
在可见的未来我相信有人肯定会纠正我，但是就我们所有的测试和我们所发现的光学动捕来说，它是精确到亚毫米的。
SUB-MILLIMETER ACCURATE 亚毫米的精确
These suits don’t have any sensors at all. They’re
basically fashion, made to be seen really clearly.
这些衣服没有任何的传感器。它们很显而易见的时尚。
Jimmy Corvan
I will always get the question how hard are
they? How do you get the technology in the balls? And I’m like, well , you can
squeeze them. They’re just retro reflective tape on the outside of little
squishy balls.
我经常会被人问及：它们穿起来有多难？那个球球里有什么技术？你其实可以挤一下它们。它们就只是带着复古反光的外表有点粘手的小球而已。
The footage shot with the suits on its fed
through software that interprets what the camera sees, before artists review
it. If you look at Mark Ruffalo’s tweet of him, Tom Holland. And Don Cheadle on
Avenger’s Endgame, you can see how additional symbols allow software to know
more about the location of the suit on the performer. Even if one symbol is
dark or out of focus, the software can figure out what’s going on thanks to the
pattern.
在艺术家进行视觉查阅之前，拍摄这套装束的镜头会通过软件来进行解读。如果你看到Mark Ruffalo和Don Cheadle 的tweet，你就可以了解到软件是如何通过一些符号获取到更多的表演者止信息的。即使是一个黑黑的符号或者失焦的符号，这个软件也可以根据模式找出发生了什么。
Jimmy Corvan
The first step in the cleanup process is
somebody goes through and make sure that you can see every one of those dots in
every frame of the shot. It sounds tedious because it is. So all the markers
have a different name. and somebody’s job is going through making sure you can
see each maker in every single frame and that each marker is properly named. That
process is called “tracking” and “labeling”.
整个过程的第一步是需要确保镜头的每一帧都可以看到每一个点。这听起来有点乏味，但是事实却是如此。所有的标记都有一个不同的名字。有些人的工作就是确保每个标记的正确命名，以及每个标记都可以在每一帧上被看见。这个过程叫做“跟踪”和“标记”。
This can be augmented with facial capture
like this : a range of motion capture gloves and even ability to label and
motion track objects. For hobbyists, there’s stuff like Moby Fitzsimmons. His form
comes from a free to download mocap-ready website. I captured his movements
using AI. Called “Plask”that just figured out the motion without any dots or
sensors. These methods can vary wildly in cost and quality but the output is
actually similar.
这可以通过面部捕捉来增强，比如，一个动捕手套，它可以用来标记和跟踪物体。对于业务爱好者来说，有像Moby Fitzsimmons这样的人物。你可以从网站免费下载它的动作捕捉。我使用一个叫做“Plask”的AI捕捉了它的动作。这个方法可以不用标记点和传感器就能计算出物体的运动。这些方法在成本和质量上差异很大，但是产出的实际效果类似。
BONES/SKELETON
It’s bones. This structure is the basic
output of most motion captures. No cloth, no muscle, no hair, just that the
software calculates as the skeleton. Each dot here is a joint, while the longer
shapes are bones. In this example, they’re my bones. Soon to be Moby’s. These
bones are given a hierarchy. So if the top of your arm rotates, the lower part
usually will too. These subtle variations can convey a lot of movement. These number
represent the rotation of a single elbow joint. Watch how they change as the
elbow moves during the animation.
这是骨头。这个结构是大多数运动捕获的基本输出。没有布料，没有肌肉，没有毛发，只有软件计算出的骨架。这里的每个点是一个关节，而较长的形状是骨头。在这个例子中，它们是我的骨头。但它很快就会是莫比的了。这些骨骼是有等级的。所以如果你的上臂旋转，下臂通常也会旋转。这些微妙的变化可以传达很多动作。这些数字代表单个肘关节的旋转。在动画中，随着肘部的移动，它们会发生同步的变化。
Jimmy Corvan
We’re trying to figure out exactly where
the skeleton exists because that’s what we’re really capturing, is your
skeleton moving. We aren’t capturing the flesh on top. We’ve got very , very
close, but it is not perfect. If the bone lengths are just slightly shorter or
just slightly longer than even something as simple as clapping, like if I were to
do this, it might end up that they go through each other like this. Because the
bones lengths are different. That’s where animators come in. there’s a lot of
that fixing and you end up seeing it in postures.
我们需要努力找出骨骼存在的准确的地方，因为那才是我们真正需要捕捉的，就是骨骼的移动。我们不是要捕捉表面的东西。我们已经非常、非常接近了，但还不够完美。如果骨头的长度比简单的手掌稍短或稍长，就像我这样做，它们可能会像这样穿过彼此。因为骨骼长度是不同的。这就是动画师的用武之地。在最终的姿势里你们可以看到它有很多被修复的地方。
MONKEY BUTT
There’s a very popular phrase in the mo-cap
space called “Monkey Butt”. And that is the hips kind of just back a little bit
and it kind of looks like they have a monkey butt. And so animator will need to
go through and push the hips back forward, kind of undo that work.
在动捕中有一个非常流行的词叫做“猴子屁股”。这是臀部，稍微向后一点，看起来有点像猴子的屁股。所以动画师需要将臀部向前推，就像撤销那个工作一样。
This outfit provides a clue to why motion
capture is harder than it looks after the data is cleaned. The Hulk is
significantly larger than Mark Ruffalo. VFX artists take the time to make Taika
Waititi’s capture fit the body of Korg, but you can see what goes wrong without
extra work. See how Moby’s hands run into his face here. That’s not a problem
when I do it because I don’t have a head like Moby’s. The same goes for
position in space. See how his feet wobble around? This it part of why you don’t
see tracking images on the actor modeling for Groot, a tall talking tree or
actor Sean Gunn, who stands in for Rocket Raccoon. These figures , and even
motion captured figures , are too different to graft on human motion. So the
people just used for reference.
通过这个你就可以理解为什么动捕本身会比它看起来更难。绿巨人比Mark Ruffalo.大得多。视效艺术家需要花时间让Taika Waititi的捕捉适合Korg的身体，但你可以看到有错误的地方。看莫比的手是怎么摸脸的。主持人做的时候就不一样了，因为他们的脸不一样。空间中的位置也是如此。看到他的脚怎么晃了吗?这就是为什么你看不到演员为格鲁特建模的跟踪图像，或演员Sean Gunn（火箭浣熊的替身)的部分原因。这些人物，甚至是捕捉到的动作人物，都是非常的不一样，以至于无法移植到人体动作上。所以人们只是作为参考。
Jimmy Corvan
Even if you’re not using all of the data,
specifically having the timing created by the actor or performer, is really ,
really helpful. A lot of people, like myself included. I talk with my hands,
and if an animator tried to animate me talking with my hands. I’d get the tiny
wrong. Look at this side by side of Hulk and Mark Ruffalo. Ruffalo gives a
great performance that you can really see. But look at Hulk’s shoulder size,
and the cloth and the space his hands take up. You can even see, in this clip,
how his right hand has been tweaked completely after capture.
That way that motion capture tends to get
covered is … look Andy Serkis and look at Gollum. They do a wonderful job of
performing, but there is a huge gap in between. That is months and months and
months and months of work. It’s portrayed . it’s just like there’s an animate
button. eans there’s a ton of animation after the capture.
即使你没有使用所有的数据，特别是由演员或专门的动捕演员进行的工作，是非常非常有用的。很多人，包括我自己。如果一个动画师试图让我用我的手说话。就会有一些小错误发生。看看Hulk和Mark Ruffalo的对比画面。Ruffalo的表演非常棒，你真的可以看到。但看看Hulk的肩膀尺寸，还有他的布料和他的手所占的空间。你甚至可以在这段视频中看到，他的右手在被捕捉后是如何被完全扭曲的。这样动作捕捉就会被掩盖，看看Andy Serkis，看看咕噜姆。他们在表演上做得很好，但两者之间有一个巨大的差距。那是月复一月月复一月的工作。它是描述。就像有一个动画按钮。
Look at all the changes within this short
clip. That level of detail would take a long time for people to animate. But the
limited structure of final format m
看看这短短的片段中的所有变化。这种程度的细节需要人们花很长时间来制作动画。但最终格式的有限结构也意味着在进行捕捉之后有大量的动画工作需要进行。
PS: 如果你有不错的学习资料也欢迎发给我，让我也学习学习！
以上学习资料来自：
https://www.youtube.com/watch?v=O0mLfzbmqcg&t=138s
https://www.youtube.com/watch?v=DpRLTfVEhMk
THE
END
如果你有希望分享的影视制作观点或者内容，欢迎后台留言～
重要提示：如果我翻译了您的原创内容让您不满意，可以联系我随时删除。If I translate your original content and you are not satisfied, please contact me to delete it at any time.我尽力都可以联系到每个我觉得优秀学习内容的原创者。但是因为部分在海外，联系起来不便。所以，这里的翻译内容用于大家学习，请各位不要用于商业行为。任何Hi VFX原创发布或翻译整理的文章内容，如若需要转载或者使用，务必联系征求本号同意并标明出处。
合作提示：Hi VFX同步提供影视项目视效管理及咨询服务。如需发布相关影音资讯及广告推广合作可后台留言沟通。
刷新 