-
公开(公告)号:US11996114B2
公开(公告)日:2024-05-28
申请号:US17321411
申请日:2021-05-15
Applicant: Apple Inc.
Inventor: Ramin Pishehvar , Ante Jukic , Mehrez Souden , Jason Wung , Feipeng Li , Joshua D. Atkins
IPC: G10L15/16 , G06N20/00 , G10L21/0216
CPC classification number: G10L21/0216 , G06N20/00 , G10L15/16 , G10L2021/02166
Abstract: Disclosed is a multi-task machine learning model such as a time-domain deep neural network (DNN) that jointly generate an enhanced target speech signal and target audio parameters from a mixed signal of target speech and interference signal. The DNN may encode the mixed signal, determine masks used to jointly estimate the target signal and the target audio parameters based on the encoded mixed signal, apply the mask to separate the target speech from the interference signal to jointly estimate the target signal and the target audio parameters, and decode the masked features to enhance the target speech signal and to estimate the target audio parameters. The target audio parameters may include a voice activity detection (VAD) flag of the target speech. The DNN may leverage multi-channel audio signal and multi-modal signals such as video signals of the target speaker to improve the robustness of the enhanced target speech signal.
-
公开(公告)号:US20230410828A1
公开(公告)日:2023-12-21
申请号:US17845655
申请日:2022-06-21
Applicant: Apple Inc.
Inventor: Ramin Pishehvar , Mehrez Souden , Sean A. Ramprashad , Jason Wung , Ante Jukic , Joshua D. Atkins
IPC: G10L21/0232 , G06V40/16 , G10L25/84 , G10L21/034 , G10L21/0364 , G10L15/25 , G10L15/06 , G10L15/22
CPC classification number: G10L21/0232 , G06V40/161 , G10L25/84 , G10L21/034 , G10L21/0364 , G10L15/25 , G10L15/063 , G10L15/22
Abstract: Disclosed is a reference-less echo mitigation or cancellation technique. The technique enables suppression of echoes from an interference signal when a reference version of the interference signal conventionally used for echo mitigation may not be available. A first stage of the technique may use a machine learning model to model a target audio area surrounding a device so that a target audio signal estimated as originating from within the target audio area may be accepted. In contrast, audio signals such as playback of media content on a TV or other interfering signals estimated as originating from outside the target audio area may be suppressed. A second stage of the technique may be a level-based suppressor that further attenuates the residual echo from the output of the first stage based on an audio level threshold. Side information may be provided to adjust the target audio area or the audio level threshold.
-
公开(公告)号:US11514928B2
公开(公告)日:2022-11-29
申请号:US16708296
申请日:2019-12-09
Applicant: Apple Inc.
Inventor: Mehrez Souden , Ante Jukic , Jason Wung , Ashrith Deshpande , Joshua D. Atkins
IPC: G10L25/78 , G10L25/81 , G10L25/18 , G10L21/0232 , G10L15/22 , G06N7/00 , G06N20/00 , G10L15/25 , G06V40/16 , G10L21/0208 , G10L21/0216 , G10L17/00
Abstract: A device implementing a system for processing speech in an audio signal includes at least one processor configured to receive an audio signal corresponding to at least one microphone of a device, and to determine, using a first model, a first probability that a speech source is present in the audio signal. The at least one processor is further configured to determine, using a second model, a second probability that an estimated location of a source of the audio signal corresponds to an expected position of a user of the device, and to determine a likelihood that the audio signal corresponds to the user of the device based on the first and second probabilities.
-
公开(公告)号:US11546692B1
公开(公告)日:2023-01-03
申请号:US17370679
申请日:2021-07-08
Applicant: Apple Inc.
Inventor: Symeon Delikaris Manias , Mehrez Souden , Ante Jukic , Matthew S. Connolly , Sabine Webel , Ronald J. Guglielmone, Jr.
Abstract: An audio renderer can have a machine learning model that jointly processes audio and visual information of an audiovisual recording. The audio renderer can generate output audio channels. Sounds captured in the audiovisual recording and present in the output audio channels are spatially mapped based on the joint processing of the audio and visual information by the machine learning model. Other aspects are described.
-
公开(公告)号:US20220369030A1
公开(公告)日:2022-11-17
申请号:US17322539
申请日:2021-05-17
Applicant: Apple Inc.
Inventor: Mehrez Souden , Jason Wung , Ante Jukic , Ramin Pishehvar , Joshua D. Atkins
IPC: H04R3/04 , H04R3/00 , H04R5/04 , G10L21/0216 , G10L25/78
Abstract: A plurality of microphone signals can be captured with a plurality of microphones of the device. One or more echo dominant audio signals can be determined based on a pick-up beam directed towards one or more speakers of a playback device. Sound that is emitted from the one or more speakers and sensed by the plurality of microphones can be removed from plurality of microphone signals, by using the one or more echo dominant audio signals as a reference, resulting in clean audio.
-
公开(公告)号:US10978086B2
公开(公告)日:2021-04-13
申请号:US16517400
申请日:2019-07-19
Applicant: Apple Inc.
Inventor: Jason Wung , Sarmad Aziz Malik , Ashrith Deshpande , Ante Jukic , Joshua D. Atkins
IPC: G10L21/0208 , G10K11/178 , G10L21/0216
Abstract: An echo canceller is disclosed in which audio signals of the playback content received by one or more of the microphones from a loudspeaker of the device may be used as the playback reference signals to estimate the echo signals of the playback content received by a target microphone for echo cancellation. The echo canceller may estimate the transfer function between a reference microphone and the target microphone based on the playback reference signal of the reference microphone and the signal of the target microphone. To mitigate near-end speech cancellation at the target microphone, the echo canceller may compute a mask to distinguish between target microphone audio signals that are echo-signal dominant and near-end speech dominant. The echo canceller may use the mask to adaptively update the transfer function or to modify the playback reference signal used by the transfer function to estimate the echo signals of the playback content.
-
公开(公告)号:US20210020188A1
公开(公告)日:2021-01-21
申请号:US16517400
申请日:2019-07-19
Applicant: Apple Inc.
Inventor: Jason Wung , Sarmad Aziz Malik , Ashrith Deshpande , Ante Jukic , Joshua D. Atkins
IPC: G10L21/0208 , G10K11/178
Abstract: An echo canceller is disclosed in which audio signals of the playback content received by one or more of the microphones from a loudspeaker of the device may be used as the playback reference signals to estimate the echo signals of the playback content received by a target microphone for echo cancellation. The echo canceller may estimate the transfer function between a reference microphone and the target microphone based on the playback reference signal of the reference microphone and the signal of the target microphone. To mitigate near-end speech cancellation at the target microphone, the echo canceller may compute a mask to distinguish between target microphone audio signals that are echo-signal dominant and near-end speech dominant. The echo canceller may use the mask to adaptively update the transfer function or to modify the playback reference signal used by the transfer function to estimate the echo signals of the playback content.
-
公开(公告)号:US11508388B1
公开(公告)日:2022-11-22
申请号:US17100802
申请日:2020-11-20
Applicant: Apple Inc.
Inventor: Mehrez Souden , Symeon Delikaris Manias , Joshua D. Atkins , Ante Jukic , Ramin Pishehvar
IPC: G10L21/0232 , H04R1/40 , G10L25/30 , G06N3/08 , H04R3/00 , G10L21/0216
Abstract: A device for processing audio signals in a time-domain includes a processor configured to receive multiple audio signals corresponding to respective microphones of at least two or more microphones of the device, at least one of the multiple audio signals comprising speech of a user of the device. The processor is configured to provide the multiple audio signals to a machine learning model, the machine learning model having been trained based at least in part on an expected position of the user of the device and expected positions of the respective microphones on the device. The processor is configured to provide an audio signal that is enhanced with respect to the speech of the user relative to the multiple audio signals, wherein the audio signal is a waveform output from the machine learning model.
-
公开(公告)号:US20220366927A1
公开(公告)日:2022-11-17
申请号:US17321411
申请日:2021-05-15
Applicant: Apple Inc.
Inventor: Ramin Pishehvar , Ante Jukic , Mehrez Souden , Jason Wung , Feipeng Li , Joshua D. Atkins
IPC: G10L21/0216 , G10L15/16 , G06N20/00
Abstract: Disclosed is a multi-task machine learning model such as a time-domain deep neural network (DNN) that jointly generate an enhanced target speech signal and target audio parameters from a mixed signal of target speech and interference signal. The DNN may encode the mixed signal, determine masks used to jointly estimate the target signal and the target audio parameters based on the encoded mixed signal, apply the mask to separate the target speech from the interference signal to jointly estimate the target signal and the target audio parameters, and decode the masked features to enhance the target speech signal and to estimate the target audio parameters. The target audio parameters may include a voice activity detection (VAD) flag of the target speech. The DNN may leverage multi-channel audio signal and multi-modal signals such as video signals of the target speaker to improve the robustness of the enhanced target speech signal.
-
公开(公告)号:US11222652B2
公开(公告)日:2022-01-11
申请号:US16516780
申请日:2019-07-19
Applicant: Apple Inc.
Inventor: Ante Jukic , Mehrez Souden , Joshua D. Atkins
Abstract: A learning based system such as a deep neural network (DNN) is disclosed to estimate a distance from a device to a speech source. The deep learning system may estimate the distance of the speech source at each time frame based on speech signals received by a compact microphone array. Supervised deep learning may be used to learn the effect of the acoustic environment on the non-linear mapping between the speech signals and the distance using multi-channel training data. The deep learning system may estimate the direct speech component that contains information about the direct signal propagation from the speech source to the microphone array and the reverberant speech signal that contains the reverberation effect and noise. The deep learning system may extract signal characteristics of the direct signal component and the reverberant signal component and estimate the distance based on the extracted signal characteristics using the learned mapping.
-
-
-
-
-
-
-
-
-