公开(公告)号：US07016505B1

公开(公告)日：2006-03-21

申请号：US10130295

申请日：2000-11-01

申请人： Kazuhiro Nakadai ,  Hiroshi Okuno ,  Hiroaki Kitano

发明人： Kazuhiro Nakadai ,  Hiroshi Okuno ,  Hiroaki Kitano

IPC分类号： A61F11/06 ,  G10K11/16 ,  H03B29/00

CPC分类号： B25J19/026 ,  G10L15/20 ,  G10L21/0216 ,  G10L2021/02165

摘要： The invention is directed to an auditory robot for a human or animal like robot, e.g., a human like robot (10) having a noise generating source such as a driving system in its interior. The apparatus includes a sound insulating cover (14) with which at least a head part (13) of the robot is covered; a pair of outer microphones (16; 16a and 16b) installed outside of the cover and located at a pair of positions where a pair of ears may be provided spaced apart for the robot, respectively, for collecting an external sound primarily; at least one inner microphone (17; 17a and 17b) installed inside of the cover for primarily collecting a noise from the noise generating source in the robot interior; and a processing module (18) on the basis of signals from the outer and inner microphones for removing from sound signals from the outer microphones (16a and 16b), a noise signal from the internal noise generating source. Thus, the robot auditory apparatus of the invention is made capable of effecting active perception by permitting an external sound from a target to be collected unaffected by a noise in the inside of the robot such as from the driving system.

摘要翻译： 本发明涉及用于人或动物如机器人的听觉机器人，例如具有诸如其内部的驱动系统的噪声产生源的人类似的机器人（10）。 所述装置包括隔音盖（14），所述机器人的至少头部（13）与所述绝缘罩（14）相覆盖; 一对外部麦克风（16; 16a和16b），其安装在所述盖子的外部并且分别位于一对位置处，所述一对耳朵可分别设置用于所述机器人，用于主要收集外部声音; 安装在所述盖内部的至少一个内部麦克风（17; 17a和17b），用于主要收集来自所述机器人内部的噪声发生源的噪声; 以及基于来自外麦克风和内麦克风的信号的处理模块（18），用于从外麦克风（16a和16b）的声音信号中去除来自内部噪声产生源的噪声信号。 因此，本发明的机器人听觉装置能够通过允许来自目标的外部声音被收集而不受来自诸如驱动系统的机器人内部的噪声的影响而影响主动感知。

公开(公告)号：US06967455B2

公开(公告)日：2005-11-22

申请号：US10468396

申请日：2002-03-08

申请人： Kazuhiro Nakadai ,  Ken-ichi Hidai ,  Hiroshi Okuno ,  Hiroaki Kitano

发明人： Kazuhiro Nakadai ,  Ken-ichi Hidai ,  Hiroshi Okuno ,  Hiroaki Kitano

IPC分类号： B25J13/00 ,  B25J19/02 ,  G06T1/00 ,  G10L21/0208 ,  B25J13/08

CPC分类号： G06T1/0014 ,  B25J13/00 ,  B25J13/003 ,  B25J19/023 ,  G10L2021/02087

摘要： A robot visuoauditory system that makes it possible to process data in real time to track vision and audition for an object, that can integrate visual and auditory information on an object to permit the object to be kept tracked without fail and that makes it possible to process the information in real time to keep tracking the object both visually and auditorily and visualize the real-time processing is disclosed. In the system, the audition module (20) in response to sound signals from microphones extracts pitches therefrom, separate their sound sources from each other and locate sound sources such as to identify a sound source as at least one speaker, thereby extracting an auditory event (28) for each object speaker. The vision module (30) on the basis of an image taken by a camera identifies by face, and locate, each such speaker, thereby extracting a visual event (39) therefor. The motor control module (40) for turning the robot horizontally. extracts a motor event (49) from a rotary position of the motor. The association module (60) for controlling these modules forms from the auditory, visual and motor control events an auditory stream (65) and a visual stream (66) and then associates these streams with each other to form an association stream (67). The attention control module (6) effects attention control designed to make a plan of the course in which to control the drive motor, e.g., upon locating the sound source for the auditory event and locating the face for the visual event, thereby determining the direction in which each speaker lies. The system also includes a display (27, 37, 48, 68) for displaying at least a portion of auditory, visual and motor information. The attention control module (64) servo-controls the robot on the basis of the association stream or streams.

摘要翻译： 机器人视觉系统，使得可以实时处理数据以跟踪对象的视觉和试镜，从而可以将物体上的视觉和听觉信息整合在一起，以允许对象被保持跟踪而不会失败，这使得可以处理 公开了实时的信息，以视觉和听觉方式跟踪对象，并且可视化实时处理。 在系统中，响应于来自麦克风的声音信号的试听模块（20）从其中提取音高，从而将它们的声源彼此分离，并且定位声源，例如将声源识别为至少一个扬声器，从而提取听觉事件 （28）。 基于由照相机拍摄的图像的视觉模块（30）通过面部识别并定位每个这样的扬声器，从而提取其视觉事件（39）。 用于水平地转动机器的马达控制模块（40）。 从马达的旋转位置提取马达事件（49）。 用于控制这些模块的关联模块（60）从听觉，视觉和运动控制事件形成听觉流（65）和视觉流（66），然后将这些流彼此关联以形成关联流（67）。 注意力控制模块（6）实现设计的注意力控制，以制定控制驱动电动机的过程的计划，例如，在定位用于听觉事件的声源并定位视觉事件的面部，从而确定方向 每个演讲者都在其中。 该系统还包括用于显示听觉，视觉和运动信息的至少一部分的显示器（27,37,48,68）。 注意力控制模块（64）基于关联流或流来对机器人进行伺服控制。

公开(公告)号：US07526361B2

公开(公告)日：2009-04-28

申请号：US10506167

申请日：2002-08-30

申请人： Kazuhiro Nakadai ,  Hiroshi Okuno ,  Hiroaki Kitano

发明人： Kazuhiro Nakadai ,  Hiroshi Okuno ,  Hiroaki Kitano

IPC分类号： G06F19/00

CPC分类号： G06K9/0057 ,  B25J13/003

摘要： Robotics visual and auditory system is provided which is made capable of accurately conducting the sound source localization of a target by associating a visual and an auditory information with respect to a target. It is provided with an audition module (20), a face module (30), a stereo module (37), a motor control module (40), an association module (50) for generating streams by associating events from said each module (20, 30, 37, and 40), and an attention control module (57) for conducting attention control based on the streams generated by the association module (50), and said association module (50) generates an auditory stream (55) and a visual stream (56) from a auditory event (28) from the auditory module (20), a face event (39) from the face module (30), a stereo event (39a) from the stereo module (37), and a motor event (48) from the motor control module (40), and an association stream (57) which associates said streams, as well as said audition module (20) collects sub-bands having the interaural phase difference (IPD) or the interaural intensity difference (IID) within the preset range by an active direction pass filter (23a) having a pass range which, according to auditory characteristics, becomes minimum in the frontal direction, and larger as the angle becomes wider to the left and right, based on an accurate sound source directional information from the association module (50), and conducts sound source separation by restructuring the wave shape of the sound source.

摘要翻译： 提供了机器人视觉和听觉系统，其能够通过将视觉和听觉信息相对于目标相关联来准确地进行目标的声源定位。 它设置有试听模块（20），面部模块（30），立体声模块（37），电机控制模块（40），通过将来自所述每个模块的事件相关联来生成流的关联模块（50） 20，30，37和40），以及用于基于由关联模块（50）生成的流进行注意控制的注意力控制模块（57），并且所述关联模块（50）生成听觉流（55）和 来自听觉模块（20）的听觉事件（28）的视觉流（56），来自面部模块（30）的面部事件（39），来自立体声模块（37）的立体声事件（39a）以及 来自马达控制模块（40）的马达事件（48）以及关联流（57），所述连接流（57）以及所述试奏模块（20）收集具有所述相位差（IPD）或 通过具有通过范围的有源方向通过滤波器（23a）在预设范围内的昼间强度差（IID），其根据听觉字符 基于来自关联模块（50）的准确的声源方向信息，在正面方向上变得最小，并且随着角度变宽到更大，并且通过重构波形的波形来进行声源分离 声源。

公开(公告)号：US07215786B2

公开(公告)日：2007-05-08

申请号：US10296244

申请日：2001-06-08

申请人： Kazuhiro Nakadai ,  Hiroshi Okuno ,  Hiroaki Kitano

发明人： Kazuhiro Nakadai ,  Hiroshi Okuno ,  Hiroaki Kitano

IPC分类号： H04B15/00 ,  H04R1/02 ,  B25J5/00

CPC分类号： G10L21/0208 ,  G10L2021/02165

摘要： A robot auditory apparatus and system are disclosed which are made capable of attaining active perception upon collecting a sound from an external target with no influence received from noises generated interior of the robot such as those emitted from the robot driving elements. The apparatus and system are for a robot having a noise generating source in its interior, and include: a sound insulating cladding (14) with which at least a portion of the robot is covered; at least two outer microphones (16 and 16) disposed outside of the cladding (14) for collecting an external sound primarily; at least one inner microphone (17) disposed inside of the cladding (14) for primarily collecting noises from the noise generating source in the robot interior; a processing section (23, 24) responsive to signals from the outer and inner microphones (16 and 16; and 17) for canceling from respective sound signals from the outer microphones (16 and 16), noises signal from the interior noise generating source and then issuing a left and a right sound signal; and a directional information extracting section (27) responsive to the left and right sound signals from the processing section (23, 24) for determining the direction from which the external sound is emitted. The processing section (23, 24) is adapted to detect burst noises owing to the noise generating source from a signal from the at least one inner microphone (17) for removing signal portions from the sound signals for bands containing the burst noises.

摘要翻译： 公开了一种机器人听觉装置和系统，其能够在从机器人内部产生的噪声（例如从机器人驱动元件发射的噪声）中收到来自外部目标的声音而获得主动感知。 该装置和系统用于在其内部具有噪声发生源的机器人，并且包括：绝缘包层（14），至少一部分机器人被覆盖; 至少两个外部麦克风（16和16），其布置在所述包层（14）的外部，用于主要收集外部声音; 设置在所述包层（14）的内部的至少一个内部麦克风（17），用于主要从所述机器人内部的所述噪声发生源收集噪声; 响应于来自外部麦克风（16和16）和17的信号的来自外部麦克风（16和16）的相应声音信号的噪声信号的处理部分（23,24），来自内部噪声发生源的噪声信号;以及 然后发出左右声音信号; 以及响应于来自处理部分（23,24）的左和右声音信号的方向信息提取部分（27），用于确定从其发出外部声音的方向。 处理部分（23,24）适于从来自至少一个内部麦克风（17）的信号中检测由于噪声发生源引起的突发噪声，用于从包含脉冲串噪声的频带的声音信号中去除信号部分。

公开(公告)号：US20060241808A1

公开(公告)日：2006-10-26

申请号：US10506167

申请日：2002-08-30

申请人： Kazuhiro Nakadai ,  Hiroshi Okuno ,  Hiroaki Kitano

发明人： Kazuhiro Nakadai ,  Hiroshi Okuno ,  Hiroaki Kitano

IPC分类号： G06F19/00

CPC分类号： G06K9/0057 ,  B25J13/003

摘要： Robotics visual and auditory system is provided which is made capable of accurately conducting the sound source localization of a target by associating a visual and an auditory information with respect to a target. It is provided with an audition module (20), a face module (30), a stereo module (37), a motor control module (40), an association module (50) for generating streams by associating events from said each module (20, 30, 37, and 40), and an attention control module (57) for conducting attention control based on the streams generated by the association module (50), and said association module (50) generates an auditory stream (55) and a visual stream (56) from a auditory event (28) from the auditory module (20), a face event (39) from the face module (30), a stereo event (39a) from the stereo module (37), and a motor event (48) from the motor control module (40), and an association stream (57) which associates said streams, as well as said audition module (20) collects sub-bands having the interaural phase difference (IPD) or the interaural intensity difference (IID) within the preset range by an active direction pass filter (23a) having a pass range which, according to auditory characteristics, becomes minimum in the frontal direction, and larger as the angle becomes wider to the left and right, based on an accurate sound source directional information from the association module (50), and conducts sound source separation by restructuring the wave shape of the sound source.

摘要翻译： 提供了机器人视觉和听觉系统，其能够通过将视觉和听觉信息相对于目标相关联来准确地进行目标的声源定位。 它设置有试听模块（20），面部模块（30），立体声模块（37），电动机控制模块（40），通过将来自所述每个模块的事件相关联来生成流的关联模块（50） 20，30，37和40），以及用于基于由关联模块（50）生成的流进行注意控制的注意力控制模块（57），并且所述关联模块（50）生成听觉流（55）和 来自听觉模块（20）的听觉事件（28）的可视流（56），来自面部模块（30）的面部事件（39），来自立体声模块（37）的立体声事件（39a） 和来自马达控制模块（40）的马达事件（48）以及关联流（57），所述关联流（57）以及所述试奏模块（20）收集具有相位差（IPD）的子带或 通过具有通过的主动方向通过滤波器（23a）在预设范围内的昼间强度差（IID） 根据听觉特性，根据来自关联模块（50）的准确声源方向信息，根据听觉特性在前方方向上变得最小，并且随着角度变宽到更大，并且通过以下方式进行声源分离 重组声源的波形。

公开(公告)号：US20090030552A1

公开(公告)日：2009-01-29

申请号：US10539047

申请日：2003-02-12

申请人： Kazuhiro Nakadai ,  Hiroshi Okuno ,  Hiroaki Kitano

发明人： Kazuhiro Nakadai ,  Hiroshi Okuno ,  Hiroaki Kitano

IPC分类号： G10L21/02 ,  G10L15/20 ,  G06F19/00 ,  G05B19/00

CPC分类号： G06N3/008 ,  G10L15/28 ,  G10L21/028 ,  G10L2015/228 ,  G10L2021/02166

摘要： It is a robotics visual and auditory system provided with an auditory module (20), a face module (30), a stereo module (37), a motor control module (40), and an association module (50) to control these respective modules. The auditory module (20) collects sub-bands having interaural phase difference (IPD) or interaural intensity difference (IID) within a predetermined range by an active direction pass filter (23a) having a pass range which, according to auditory characteristics, becomes minimum in the frontal direction, and larger as the angle becomes wider to the left and right, based on an accurate sound source directional information from the association module (50), and conducts sound source separation by restructuring a wave shape of a sound source, conducts speech recognition of separated sound signals from respective sound sources using a plurality of acoustic models (27d), integrates speech recognition results from each acoustic model by a selector, and judges the most reliable speech recognition result among the speech recognition results.

摘要翻译： 它是具有听觉模块（20），面部模块（30），立体声模块（37），电机控制模块（40）和关联模块（50）的机器人视觉和听觉系统，用于控制这些相应的 模块。 听觉模块（20）通过具有根据听觉特性变为最小的通过范围的有源方向通过滤波器（23a）来收集在预定范围内的具有耳间相位差（IPD）或urala内强度差（IID）的子带 基于来自关联模块（50）的准确的声源方向信息，通过重新构成声源的波形来进行声源分离，进行左侧和右侧的角度的变宽， 使用多个声学模型（27d）对来自相应声源的分离的声音信号进行语音识别，通过选择器对来自每个声学模型的语音识别结果进行积分，并且判断语音识别结果中最可靠的语音识别结果。

公开(公告)号：US08073690B2

公开(公告)日：2011-12-06

申请号：US11792052

申请日：2005-12-02

申请人： Kazuhiro Nakadai ,  Hiroshi Tsujino ,  Hiroshi Okuno ,  Shunichi Yamamoto

发明人： Kazuhiro Nakadai ,  Hiroshi Tsujino ,  Hiroshi Okuno ,  Shunichi Yamamoto

IPC分类号： G10L15/20

CPC分类号： G10L15/20 ,  G10L21/028 ,  G10L2021/02166

摘要： A voice recognition system (10) for improving the toughness of voice recognition for a voice input for which a deteriorated feature amount cannot be completely identified. The system comprises at least two sound detecting means (16a, 16b) for detecting a sound signal, a sound source localizing unit (21) for determining the direction of a sound source based on the sound signal, a sound source separating unit (23) for separating a sound by the sound source from the sound signal based on the sound source direction, a mask producing unit (25) for producing a mask value according to the reliability of the separation results, a feature extracting unit (27) for extracting the feature amount of the sound signal, and a voice recognizing unit (29) for applying the mask to the feature amount to recognize a voice from the sound signal.

摘要翻译： 一种语音识别系统（10），用于提高不能完全识别恶化的特征量的语音输入的语音识别的韧性。 该系统包括用于检测声音信号的至少两个声音检测装置（16a，16b），用于基于声音信号确定声源的方向的声源定位单元（21），声源分离单元（23） 用于根据声源方向将来自声源的声音与声音信号分离，用于根据分离结果的可靠性产生掩码值的掩模产生单元（25），用于提取分离结果的特征提取单元（27） 声音信号的特征量，以及用于将该掩模应用于特征量以从声音信号识别语音的语音识别单元（29）。

公开(公告)号：US07987090B2

公开(公告)日：2011-07-26

申请号：US12187684

申请日：2008-08-07

申请人： Ryu Takeda ,  Kazuhiro Nakadai ,  Hiroshi Tsujino ,  Hiroshi Okuno

发明人： Ryu Takeda ,  Kazuhiro Nakadai ,  Hiroshi Tsujino ,  Hiroshi Okuno

IPC分类号： G10L19/14

CPC分类号： G10L21/0272

摘要： A system capable of reducing the influence of sound reverberation or reflection to improve sound-source separation accuracy. An original signal X(ω,f) is separated from an observed signal Y(ω,f) according to a first model and a second model to extract an unknown signal E(ω,f). According to the first model, the original signal X(ω,f) of the current frame f is represented as a combined signal of known signals S(ω,f−m+1) (m=1 to M) that span a certain number M of current and previous frames. This enables extraction of the unknown signal E(ω,f) without changing the window length while reducing the influence of reverberation or reflection of the known signal S(ω,f) on the observed signal Y(ω,f).

摘要翻译： 一种能够减少声音混响或反射影响以提高声源分离精度的系统。 根据第一模型和第二模型将原始信号X（ω，f）与观测信号Y（ω，f）分离，以提取未知信号E（ω，f）。 根据第一模型，当前帧f的原始信号X（ω，f）被表示为已知信号S（ω，f-m + 1）（m = 1至M）的组合信号，该信号跨越一定 当前帧和前一帧的M个。 这可以在不改变窗口长度的情况下提取未知信号E（ω，f），同时减少已知信号S（ω，f）的混响或反射对观测信号Y（ω，f）的影响。

公开(公告)号：US08889976B2

公开(公告)日：2014-11-18

申请号：US12851994

申请日：2010-08-06

申请人： Kazuhiro Nakadai ,  Takuma Otsuka ,  Hiroshi Okuno

发明人： Kazuhiro Nakadai ,  Takuma Otsuka ,  Hiroshi Okuno

IPC分类号： G04B13/00 ,  G10H1/36 ,  G10L25/90

CPC分类号： G10H1/361 ,  G10H2210/066 ,  G10H2210/076 ,  G10H2250/235 ,  G10L25/90

摘要： A musical score position estimating device includes an audio signal acquiring unit, a musical score information acquiring unit acquiring musical score information corresponding to an audio signal acquired by the audio signal acquiring unit, an audio signal feature extracting unit extracting a feature amount of the audio signal, a musical score feature extracting unit extracting a feature amount of the musical score information, a beat position estimating unit estimating a beat position of the audio signal, and a matching unit matching the feature amount of the audio signal with the feature amount of the musical score information using the estimated beat position to estimate a position of a portion in the musical score information corresponding to the audio signal.

摘要翻译： 音乐分数位置估计装置包括：音频信号获取单元，乐谱信息获取单元，获取与由音频信号获取单元获取的音频信号相对应的乐谱信息，提取音频信号的特征量的音频信号特征提取单元 提取乐谱特征量的乐谱特征提取单元，估计音频信号的拍子位置的拍子位置估计单元以及与音频信号的特征量相匹配的匹配单元与音乐的特征量 使用估计的拍子位置来评估信息，以估计与音频信号相对应的乐谱信息中的一部分的位置。

公开(公告)号：US08416957B2

公开(公告)日：2013-04-09

申请号：US12631434

申请日：2009-12-04

申请人： Hiroshi Tsujino ,  Kazuhiro Nakadai ,  Hiroshi Okuno ,  Takeshi Mizumoto ,  Ikkyu Aihara

发明人： Hiroshi Tsujino ,  Kazuhiro Nakadai ,  Hiroshi Okuno ,  Takeshi Mizumoto ,  Ikkyu Aihara

IPC分类号： H04R29/00

CPC分类号： G01S5/16

摘要： In a sound source localization system using a light emitting device for visualizing sound information, including: a light emitting device (40) including a microphone for receiving sound from a sound source (1, 2) and a light emitting means for emitting light based on the sound from the microphone; a generating section for generating light emitting information for the light emitting device (40); and a sound source localization section (60) for determining a position of the sound source based on the light emitting information from the generating section.

摘要翻译： 在使用发光装置进行可视化声音信息的声源定位系统中，包括：包括用于从声源（1,2）接收声音的麦克风的发光装置（40）和基于 来自麦克风的声音; 用于产生发光装置（40）的发光信息的发生部分; 以及声源定位部分（60），用于基于来自发生部分的发光信息确定声源的位置。