公开(公告)号：US20060204060A1

公开(公告)日：2006-09-14

申请号：US11435122

申请日：2006-05-16

申请人： Ying Huang ,  Stephen Lin ,  Baining Guo ,  Heung-Yeung Shum

发明人： Ying Huang ,  Stephen Lin ,  Baining Guo ,  Heung-Yeung Shum

IPC分类号： G06K9/00

CPC分类号： G06K9/00335 ,  G10L2021/105

摘要： A novel method for synchronizing the lips of a sketched face to an input voice. The lip synchronization system and method approach is to use training video as much as possible when the input voice is similar to the training voice sequences. Initially, face sequences are clustered from video segments, then by making use of sub-sequence Hidden Markov Models, a correlation between speech signals and face shape sequences is built. From this re-use of video, the discontinuity between two consecutive output faces is decreased and accurate and realistic synthesized animations are obtained. The lip synchronization system and method can synthesize faces from input audio in real-time without noticeable delay. Since acoustic feature data calculated from audio is directly used to drive the system without considering its phonemic representation, the method can adapt to any kind of voice, language or sound.

公开(公告)号：US07433490B2

公开(公告)日：2008-10-07

申请号：US11435122

申请日：2006-05-16

申请人： Ying Huang ,  Stephen Ssu-te Lin ,  Baining Guo ,  Heung-Yeung Shum

发明人： Ying Huang ,  Stephen Ssu-te Lin ,  Baining Guo ,  Heung-Yeung Shum

IPC分类号： G06K9/00 ,  G10L15/00

CPC分类号： G06K9/00335 ,  G10L2021/105

摘要： A novel method for synchronizing the lips of a sketched face to an input voice. The lip synchronization system and method approach is to use training video as much as possible when the input voice is similar to the training voice sequences. Initially, face sequences are clustered from video segments, then by making use of sub-sequence Hidden Markov Models, a correlation between speech signals and face shape sequences is built. From this re-use of video, the discontinuity between two consecutive output faces is decreased and accurate and realistic synthesized animations are obtained. The lip synchronization system and method can synthesize faces from input audio in real-time without noticeable delay. Since acoustic feature data calculated from audio is directly used to drive the system without considering its phonemic representation, the method can adapt to any kind of voice, language or sound.

摘要翻译： 一种用于将草绘脸部的嘴唇同步到输入声音的新颖方法。 唇部同步系统和方法方法是当输入的声音类似于训练声音序列时尽可能多地使用训练视频。 最初，面部序列从视频片段聚类，然后通过利用子序列隐马尔可夫模型，构建了语音信号和面部形状序列之间的相关性。 从这种视频的再次使用，两个连续的输出面之间的不连续性被降低，并且获得准确而逼真的合成动画。 唇同步系统和方法可以实时地从输入音频合成面部，而没有明显的延迟。 由于从音频计算的声学特征数据直接用于驱动系统而不考虑其音位表示，该方法可以适应任何种类的语音，语言或声音。

公开(公告)号：US07133535B2

公开(公告)日：2006-11-07

申请号：US10328215

申请日：2002-12-21

申请人： Ying Huang ,  Stephen Ssu-te Lin ,  Baining Guo ,  Heung-Yeung Shum

发明人： Ying Huang ,  Stephen Ssu-te Lin ,  Baining Guo ,  Heung-Yeung Shum

IPC分类号： G06K9/00 ,  G10L215/00

CPC分类号： G06K9/00335 ,  G10L2021/105

摘要： A novel method for synchronizing the lips of a sketched face to an input voice. The lip synchronization system and method approach is to use training video as much as possible when the input voice is similar to the training voice sequences. Initially, face sequences are clustered from video segments, then by making use of sub-sequence Hidden Markov Models, a correlation between speech signals and face shape sequences is built. From this re-use of video, the discontinuity between two consecutive output faces is decreased and accurate and realistic synthesized animations are obtained. The lip synchronization system and method can synthesize faces from input audio in real-time without noticeable delay. Since acoustic feature data calculated from audio is directly used to drive the system without considering its phonemic representation, the method can adapt to any kind of voice, language or sound.

摘要翻译： 一种用于将草绘脸部的嘴唇同步到输入声音的新颖方法。 唇部同步系统和方法方法是当输入的声音类似于训练声音序列时尽可能多地使用训练视频。 最初，面部序列从视频片段聚类，然后通过利用子序列隐马尔可夫模型，构建了语音信号和面部形状序列之间的相关性。 从这种视频的再次使用，两个连续的输出面之间的不连续性被降低，并且获得准确而逼真的合成动画。 唇同步系统和方法可以实时地从输入音频合成面部，而没有明显的延迟。 由于从音频计算的声学特征数据直接用于驱动系统而不考虑其音素表示，该方法可以适应任何种类的语音，语言或声音。

公开(公告)号：US07933442B2

公开(公告)日：2011-04-26

申请号：US11694840

申请日：2007-03-30

申请人： Jiaping Wang ,  Xin Tong ,  Stephen Lin ,  Baining Guo ,  Heung-Yeung Shum

发明人： Jiaping Wang ,  Xin Tong ,  Stephen Lin ,  Baining Guo ,  Heung-Yeung Shum

IPC分类号： G06K9/00

CPC分类号： G06T19/00 ,  G06T13/00 ,  G06T2210/64

摘要： A method for modeling a time-variant appearance of a material is described. A sample analysis of a material sample is performed, wherein the sample analysis orders surface points of the material sample with respect to weathering from data captured at a single instant in time. An appearance synthesis using the sample analysis is performed, wherein the appearance synthesis generates a time-variant sequence of frames for weathering an object.

摘要翻译： 描述了用于对材料的时变外观进行建模的方法。 进行材料样品的样品分析，其中样品分析根据在单个时刻捕获的数据对材料样品的表面点进行排序。 执行使用样本分析的外观合成，其中外观合成生成用于风化对象的帧的时变序列。

公开(公告)号：US20060028468A1

公开(公告)日：2006-02-09

申请号：US10898435

申请日：2004-07-23

申请人： Yanyun Chen ,  Xin Tong ,  Stephen Lin ,  Baining Guo ,  Heung-Yeung Shum

发明人： Yanyun Chen ,  Xin Tong ,  Stephen Lin ,  Baining Guo ,  Heung-Yeung Shum

IPC分类号： G06T17/00

CPC分类号： G06T15/04

摘要： Techniques are provided for at least modeling any one of mesostructure shadowing, masking, interreflection and silhouettes on a surface, as well as subsurface scattering within a non-homogeneous volume. Such techniques include, at least, acquiring material parameters for a material sample, determining irradiance distribution values for the material sample, synthesizing the material sample onto a mesh of an object. The synthesized object may then be rendered by one of plural rendering techniques.

摘要翻译： 提供技术用于至少建模表面上的任何一种介观结构阴影，掩模，反射和剪影，以及非均匀体积内的地下散射。 这些技术至少包括获取材料样品的材料参数，确定材料样品的辐照度分布值，将材料样品合成到物体的网格上。 然后可以通过多个渲染技术之一来呈现合成对象。

公开(公告)号：US20050280648A1

公开(公告)日：2005-12-22

申请号：US10990244

申请日：2004-11-15

申请人： Xi Wang ,  Xin Tong ,  Stephen Lin ,  Baining Guo ,  Heung-Yeung Shum

发明人： Xi Wang ,  Xin Tong ,  Stephen Lin ,  Baining Guo ,  Heung-Yeung Shum

IPC分类号： G06T15/20 ,  G06T15/50 ,  G06T15/60 ,  G06T17/00 ,  G09G5/00

CPC分类号： G06T15/04

摘要： A “mesostructure renderer” uses pre-computed multi-dimensional “generalized displacement maps” (GDM) to provide real-time rendering of general non-height-field mesostructures on both open and closed surfaces of arbitrary geometry. In general, the GDM represents the distance to solid mesostructure along any ray cast from any point within a volumetric sample. Given the pre-computed GDM, the mesostructure renderer then computes mesostructure visibility jointly in object space and texture space, thereby enabling both control of texture distortion and efficient computation of texture coordinates and shadowing. Further, in one embodiment, the mesostructure renderer uses the GDM to render mesostructures with either local or global illumination as a per-pixel process using conventional computer graphics hardware to accelerate the real-time rendering of the mesostructures. Further acceleration of mesostructure rendering is achieved in another embodiment by automatically reducing the number of triangles in the rendering pipeline according to a user-specified threshold for acceptable texture distortion.

摘要翻译： “mesostructure渲染器”使用预先计算的多维“广义位移图”（GDM），以便在任意几何的开放和闭合表面上提供一般非高度场介观结构的实时渲染。 一般来说，GDM表示沿着体积样品内的任何点的任何射线投射到固体介观结构的距离。 给定预先计算的GDM，然后，介观结构渲染器在对象空间和纹理空间中联合计算介观结构可见度，从而实现纹理失真的控制和纹理坐标和阴影的有效计算。 此外，在一个实施例中，使用传统计算机图形硬件的介面结构渲染器使用GDM来渲染具有局部或全局照明的介观结构作为每像素处理，以加速介观结构的实时渲染。 在另一个实施例中，通过根据用户指定的可接受纹理失真的阈值自动减少渲染流水线中的三角形数量来实现进一步加速的介观结构渲染。

公开(公告)号：US07310101B2

公开(公告)日：2007-12-18

申请号：US10990142

申请日：2004-11-15

申请人： Xi Wang ,  Xin Tong ,  Stephen Lin ,  Baining Guo ,  Heung-Yeung Shum

发明人： Xi Wang ,  Xin Tong ,  Stephen Lin ,  Baining Guo ,  Heung-Yeung Shum

IPC分类号： G09G5/00 ,  G06T15/20 ,  G06T15/60 ,  G06T17/00

CPC分类号： G06T15/04

摘要： A “mesostructure renderer” uses pre-computed multi-dimensional “generalized displacement maps” (GDM) to provide real-time rendering of general non-height-field mesostructures on both open and closed surfaces of arbitrary geometry. In general, the GDM represents the distance to solid mesostructure along any ray cast from any point within a volumetric sample. Given the pre-computed GDM, the mesostructure renderer then computes mesostructure visibility jointly in object space and texture space, thereby enabling both control of texture distortion and efficient computation of texture coordinates and shadowing. Further, in one embodiment, the mesostructure renderer uses the GDM to render mesostructures with either local or global illumination as a per-pixel process using conventional computer graphics hardware to accelerate the real-time rendering of the mesostructures. Further acceleration of mesostructure rendering is achieved in another embodiment by automatically reducing the number of triangles in the rendering pipeline according to a user-specified threshold for acceptable texture distortion.

摘要翻译： “mesostructure渲染器”使用预先计算的多维“广义位移图”（GDM），以便在任意几何的开放和闭合表面上提供一般非高度场介观结构的实时渲染。 一般来说，GDM表示沿着体积样品内的任何点的任何射线投射到固体介观结构的距离。 给定预先计算的GDM，然后，介观结构渲染器在对象空间和纹理空间中联合计算介观结构可见度，从而实现纹理失真的控制和纹理坐标和阴影的有效计算。 此外，在一个实施例中，使用传统计算机图形硬件的介面结构渲染器使用GDM来渲染具有局部或全局照明的介观结构作为每像素处理，以加速介观结构的实时渲染。 在另一个实施例中，通过根据用户指定的可接受纹理失真的阈值自动减少渲染流水线中的三角形数量来实现进一步加速的介观结构渲染。

公开(公告)号：US07286135B2

公开(公告)日：2007-10-23

申请号：US10990244

申请日：2004-11-15

申请人： Xi Wang ,  Xin Tong ,  Stephen Lin ,  Baining Guo ,  Heung-Yeung Shum

发明人： Xi Wang ,  Xin Tong ,  Stephen Lin ,  Baining Guo ,  Heung-Yeung Shum

IPC分类号： G09G5/00 ,  G06T17/00 ,  G06T15/60 ,  G06T15/50

CPC分类号： G06T15/04

摘要： A “mesostructure renderer” uses pre-computed multi-dimensional “generalized displacement maps” (GDM) to provide real-time rendering of general non-height-field mesostructures on both open and closed surfaces of arbitrary geometry. In general, the GDM represents the distance to solid mesostructure along any ray cast from any point within a volumetric sample. Given the pre-computed GDM, the mesostructure renderer then computes mesostructure visibility jointly in object space and texture space, thereby enabling both control of texture distortion and efficient computation of texture coordinates and shadowing. Further, in one embodiment, the mesostructure renderer uses the GDM to render mesostructures with either local or global illumination as a per-pixel process using conventional computer graphics hardware to accelerate the real-time rendering of the mesostructures. Further acceleration of mesostructure rendering is achieved in another embodiment by automatically reducing the number of triangles in the rendering pipeline according to a user-specified threshold for acceptable texture distortion.

摘要翻译： “mesostructure渲染器”使用预先计算的多维“广义位移图”（GDM），以便在任意几何的开放和闭合表面上提供一般非高度场介观结构的实时渲染。 一般来说，GDM表示沿着体积样品内的任何点的任何射线投射到固体介观结构的距离。 给定预先计算的GDM，然后，介观结构渲染器在对象空间和纹理空间中联合计算介观结构可见度，从而实现纹理失真的控制和纹理坐标和阴影的有效计算。 此外，在一个实施例中，使用传统计算机图形硬件的介面结构渲染器使用GDM来渲染具有局部或全局照明的介观结构作为每像素处理，以加速介观结构的实时渲染。 在另一个实施例中，通过根据用户指定的可接受纹理失真的阈值自动减少渲染流水线中的三角形数量来实现进一步加速的介观结构渲染。

公开(公告)号：US20060290719A1

公开(公告)日：2006-12-28

申请号：US11166559

申请日：2005-06-24

申请人： Xin Tong ,  Jiaping Wang ,  Stephen Lin ,  Baining Guo ,  Heung-Yeung Shum

发明人： Xin Tong ,  Jiaping Wang ,  Stephen Lin ,  Baining Guo ,  Heung-Yeung Shum

IPC分类号： G09G5/00 ,  G09G5/02

CPC分类号： G06T11/001 ,  G01N21/4795 ,  G06T15/506

摘要： Representing quasi-homogenous materials is described. In one aspect, quasi-homogenous materials are modeled to generate a material model of a physical sample. The material model identifies how light is scattered by the quasi-homogenous materials. The material model, independent of an object model of the physical sample, provides information that is useful to texture surfaces of arbitrary types and sizes of mesh models (e.g., representing the physical sample or other objects) with the quasi-homogenous materials.

摘要翻译： 描述了准均匀材料。 在一个方面，准均质材料被建模以产生物理样品的材料模型。 材料模型通过准均质材料识别光如何散射。 材料模型独立于物理样本的对象模型，提供了有用于使用准均质材料对任意类型和大小的网格模型（例如，表示物理样本或其他对象）的表面进行纹理的信息。

公开(公告)号：US20050280647A1

公开(公告)日：2005-12-22

申请号：US10990142

申请日：2004-11-15

申请人： Xi Wang ,  Xin Tong ,  Stephen Lin ,  Baining Guo ,  Heung-Yeung Shum

发明人： Xi Wang ,  Xin Tong ,  Stephen Lin ,  Baining Guo ,  Heung-Yeung Shum

IPC分类号： G06T15/20 ,  G06T15/50 ,  G06T15/60 ,  G06T17/00 ,  G09G5/00

CPC分类号： G06T15/04

摘要： A “mesostructure renderer” uses pre-computed multi-dimensional “generalized displacement maps” (GDM) to provide real-time rendering of general non-height-field mesostructures on both open and closed surfaces of arbitrary geometry. In general, the GDM represents the distance to solid mesostructure along any ray cast from any point within a volumetric sample. Given the pre-computed GDM, the mesostructure renderer then computes mesostructure visibility jointly in object space and texture space, thereby enabling both control of texture distortion and efficient computation of texture coordinates and shadowing. Further, in one embodiment, the mesostructure renderer uses the GDM to render mesostructures with either local or global illumination as a per-pixel process using conventional computer graphics hardware to accelerate the real-time rendering of the mesostructures. Further acceleration of mesostructure rendering is achieved in another embodiment by automatically reducing the number of triangles in the rendering pipeline according to a user-specified threshold for acceptable texture distortion.

摘要翻译： “mesostructure渲染器”使用预先计算的多维“广义位移图”（GDM），以便在任意几何的开放和闭合表面上提供一般非高度场介观结构的实时渲染。 一般来说，GDM表示沿着体积样品内的任何点的任何射线投射到固体介观结构的距离。 给定预先计算的GDM，然后，介观结构渲染器在对象空间和纹理空间中联合计算介观结构可见度，从而实现纹理失真的控制和纹理坐标和阴影的有效计算。 此外，在一个实施例中，使用传统计算机图形硬件的介面结构渲染器使用GDM来渲染具有局部或全局照明的介观结构作为每像素处理，以加速介观结构的实时渲染。 在另一个实施例中，通过根据用户指定的可接受纹理失真的阈值自动减少渲染流水线中的三角形数量来实现进一步加速的介观结构渲染。