Monocular Real-Time Volumetric Performance Capture

鎯宠薄涓€涓嬶紝鍙渶涓€棰楁櫘閫氭憚鍍忓ご灏辫兘瀹炴椂鎹曟崏浣犵殑浜轰綋杩愬姩锛屽苟瀹炴椂浼犻€佸埌缃戠粶鐨勫彟涓€绔紝鏄笉鏄緢閰凤紝杩欎竴澶╁彲鑳藉緢蹇氨鍒版潵浜嗐€?/p>

鏈枃浣滆€呮潵鑷崡鍔犲窞澶у鍜?Pinscreen锛屼綔鑰呮彁鍑轰簡涓€涓疄鏃剁殑鍗曠洰浜轰綋鍔ㄤ綔鎹曟崏鏂规硶锛?strong>鍙渶涓€棰楁櫘閫氭憚鍍忓ご锛屽鍔犱袱鍧?NVIDIA GV100s  鏄惧崱锛屼究鑳戒互 15 FPS 鐨勫抚鐜囧疄鏃舵崟鎹変汉浣撶殑鍔ㄤ綔銆?/p>

璇ヨ鏂囧凡琚?ECCV 2020 鎺ユ敹锛屽悓鏃?SIGGRAPH 2020 Real Time Live (RTL) 椤圭洰 Volumetric Human Teleportation 鑳屽悗鐨勬妧鏈究婧愯嚜璇ヨ鏂囷紝RTL 鏄绠楁満鍥惧舰瀛﹂《浼?SIGGRAPH 鐨勪竴涓疄鏃舵紨绀虹幆鑺傦紝浼氱幇鍦哄疄鏃跺睍绀轰竴浜涘叿鏈夊ū涔愭€с€佸疄鏃舵€с€佷氦浜掓€у拰鍒涙柊鎬х殑鎶€鏈痙emo锛屽ぇ瀹跺埌鏃跺€?(灏卞湪鍖椾含鏃堕棿8鏈?6鏃?鐐瑰埌10鐐?/em>) 鍙互鐪嬩竴涓嬭璁烘枃鐨勫疄鏃舵晥鏋溿€備粖骞?RTL 鏈変節涓」鐩紝鍏朵粬鍏釜鎴戞斁鍦?鏂囩珷鏈熬锛屾劅鍏磋叮鐨勫彲浠ュ叧娉ㄤ笅銆?/p>

铏氭嫙/澧炲己鐜板疄鍜?D鏄惧紡鎶€鏈湪杩囧幓鍑犲勾鍙栧緱浜嗛暱瓒崇殑杩涙锛岀粰浜轰滑甯︽潵浜嗘矇娴稿紡鐨勪綋楠屽拰鍏ㄦ柊鐨勪氦浜掓柟寮忥紝浣嗗熀浜庡僵鑹茬浉鏈虹殑瑙嗛浼氳浠嶇劧鏄汉浠繙璺濈闈㈠闈氦娴佺殑鏈€甯歌鏂瑰紡锛屽師鍥犲緢绠€鍗曪紝Oculus銆丠TC Vive銆丠ololens锛屽ぇ閮ㄥ垎浜哄彲鑳介兘娌℃湁锛屽€掓槸褰╄壊鐩告満鏂逛究渚挎嵎锛岀瑪璁版湰鐢佃剳銆佹櫤鑳芥墜鏈洪兘鏈夈€傚洜姝わ紝濡傛灉鑳藉閫氳繃鍗曚釜娑堣垂绾х浉鏈哄氨鑳芥崟鎹夊埌涓€涓汉鐨勫瑙傚拰杩愬姩锛屼究鑳界缉灏忔櫘閫氱敤鎴峰湪铏氭嫙鐜涓繘琛屾矇娴稿紡閫氫俊鐨勫樊璺濄€?/p>

鐒惰€岋紝鐢变簬 娣卞害姝т箟 (depth ambiguity)銆?strong>鑷韩閬尅 (self-occlusions)銆?strong>鎷撴墤鍙樺寲 (topology changing) 绛夊師鍥狅紝浠呬粠鍗曚釜瑙嗚鐨勫僵鑹插浘鐗囧簭鍒楅噸寤轰汉浣撶殑鍑犱綍鍜岀汗鐞嗘槸涓€涓浉褰撳洶闅剧殑闂銆備负浜嗗厠鏈嶈繖浜涘洶闅撅紝澶ч儴鍒嗘柟娉曢噰鐢ㄦ暟鎹┍鍔ㄧ殑鏂瑰紡锛岄€氳繃娣卞害绁炵粡缃戠粶鏉ヨ繘琛岄珮绮惧害銆侀珮椴佹鎬х殑浜轰綋寤烘ā銆侾ixel-Aligned Implicit Function (PIFu) 渚挎槸鍏朵腑鐨勪竴绉嶄紭绉€鏂规硶锛屽叾鑳藉浠ヨ緝楂樼簿搴﹂噸寤哄嚭绌胯。鏈嶄汉浣撶殑鍑犱綍鍜岀汗鐞嗕俊鎭€備絾 PIFu 鍦ㄩ噸寤烘椂闇€瑕佸绌洪棿杩涜瀵嗛泦鍒掑垎骞惰绠楋紝鐒跺悗閫氳繃 Marching Cubes 绠楁硶鎻愬彇绛夊€奸潰锛岃繕瑕佷互鍚屾牱鐨勬柟寮忛娴嬬汗鐞嗕俊鎭紝瀵逛簬  鐨勭┖闂村垝鍒嗘暣涓繃绋嬮渶瑕?鏁板崄绉?/strong>锛岄€熷害杈冩參鏃犳硶鐢ㄤ簬瀹炴椂搴旂敤銆備负姝や綔鑰呮彁鍑轰簡涓€绉嶆柊棰栫殑琛ㄩ潰閲嶅缓鍜屾覆鏌撴柟娉曪細

• 琛ㄩ潰閲嶅缓 鏂规硶浠ヤ竴绉嶇敱绮楀埌绮剧殑鏂瑰紡鏉ラ€愭瀹氫綅浜轰綋琛ㄩ潰3D鐐癸紝浣垮緱鍙渶鍦ㄥ皯閲忕偣涓婅绠椾究鍙互鏋勫缓鍑?3D 鍗犳嵁鍦?(3D occupancy fields)銆傚叾鑳藉鍦ㄤ繚鎸侀噸寤虹簿搴︾殑鍩虹涓婏紝姣斿熀绾挎柟娉曞揩2涓暟閲忕骇銆?/section>
• 琛ㄩ潰娓叉煋 鏂规硶鍩轰簬闅愬紡绾圭悊琛ㄨ揪锛岃兘澶熺洿鎺ヤ粠鍏朵粬瑙嗚杩涜娓叉煋锛屼笉闇€瑕佹樉寮忓湴鎻愬彇琛ㄩ潰缃戞牸锛屼篃涓嶉渶瑕佸嚑浣曠粏鍒嗘垨绾圭悊鏄犲皠锛屼粠鑰屽皢娓叉煋鎵€闇€鐨勬椂闂村噺灏戜簡涓€鍗娿€?/section>

鏈€缁堬紝鍦?nbsp; 鐨勭┖闂村垎杈ㄧ巼涓婏紝鏂囩珷鏂规硶鑳藉浠?15 FPS 鐨勫抚鐜囪繘琛屼汉浣撹繍鍔ㄦ崟鎹夈€傛枃绔犱唬鐮佸凡缁忓紑婧愶紝鏅€氭憚鍍忓ご娌￠棶棰橈紝鍙棶棰樻槸鎴戞病鏈変袱鍧楁樉鍗″晩锛侌煒?/p>

椤圭洰涓婚〉锛?em>https://project-splinter.github.io

椤圭洰浠ｇ爜锛?em>https://github.com/Project-Splinter/MonoPort