公开(公告)号：US11996114B2

公开(公告)日：2024-05-28

申请号：US17321411

申请日：2021-05-15

Applicant: Apple Inc.

Inventor： Ramin Pishehvar ,  Ante Jukic ,  Mehrez Souden ,  Jason Wung ,  Feipeng Li ,  Joshua D. Atkins

IPC: G10L15/16 ,  G06N20/00 ,  G10L21/0216

CPC classification number: G10L21/0216 ,  G06N20/00 ,  G10L15/16 ,  G10L2021/02166

Abstract: Disclosed is a multi-task machine learning model such as a time-domain deep neural network (DNN) that jointly generate an enhanced target speech signal and target audio parameters from a mixed signal of target speech and interference signal. The DNN may encode the mixed signal, determine masks used to jointly estimate the target signal and the target audio parameters based on the encoded mixed signal, apply the mask to separate the target speech from the interference signal to jointly estimate the target signal and the target audio parameters, and decode the masked features to enhance the target speech signal and to estimate the target audio parameters. The target audio parameters may include a voice activity detection (VAD) flag of the target speech. The DNN may leverage multi-channel audio signal and multi-modal signals such as video signals of the target speaker to improve the robustness of the enhanced target speech signal.

公开(公告)号：US20230410828A1

公开(公告)日：2023-12-21

申请号：US17845655

申请日：2022-06-21

Applicant: Apple Inc.

Inventor： Ramin Pishehvar ,  Mehrez Souden ,  Sean A. Ramprashad ,  Jason Wung ,  Ante Jukic ,  Joshua D. Atkins

IPC: G10L21/0232 ,  G06V40/16 ,  G10L25/84 ,  G10L21/034 ,  G10L21/0364 ,  G10L15/25 ,  G10L15/06 ,  G10L15/22

CPC classification number: G10L21/0232 ,  G06V40/161 ,  G10L25/84 ,  G10L21/034 ,  G10L21/0364 ,  G10L15/25 ,  G10L15/063 ,  G10L15/22

Abstract: Disclosed is a reference-less echo mitigation or cancellation technique. The technique enables suppression of echoes from an interference signal when a reference version of the interference signal conventionally used for echo mitigation may not be available. A first stage of the technique may use a machine learning model to model a target audio area surrounding a device so that a target audio signal estimated as originating from within the target audio area may be accepted. In contrast, audio signals such as playback of media content on a TV or other interfering signals estimated as originating from outside the target audio area may be suppressed. A second stage of the technique may be a level-based suppressor that further attenuates the residual echo from the output of the first stage based on an audio level threshold. Side information may be provided to adjust the target audio area or the audio level threshold.

公开(公告)号：US11810588B2

公开(公告)日：2023-11-07

申请号：US17589889

申请日：2022-01-31

Applicant: Apple Inc.

Inventor： Carlos M. Avendano ,  John Woodruff ,  Jonathan Huang ,  Mehrez Souden ,  Andreas Koutrouvelis

IPC: H04R29/00 ,  G10L21/028 ,  H04R1/10 ,  G06N20/00 ,  G10L21/0232

CPC classification number: G10L21/028 ,  G06N20/00 ,  G10L21/0232 ,  H04R1/1016 ,  H04R1/1041 ,  H04R1/1083 ,  H04R2420/07

Abstract: Implementations of the subject technology provide systems and methods for providing audio source separation for audio input, such as for audio devices having limited power and/or computing resources. The subject technology may allow an audio device to leverage processing and/or power resources of a companion device that is communicatively coupled to the audio device. The companion device may identify a noise condition of the audio device, select a source separation model based on the noise condition, and provide the source separation model to the audio device. In this way, the audio device can provide audio source separation functionality using a relatively small footprint source separation model that is specific to the noise condition in which the audio device is operated.

公开(公告)号：US11514928B2

公开(公告)日：2022-11-29

申请号：US16708296

申请日：2019-12-09

Applicant: Apple Inc.

Inventor： Mehrez Souden ,  Ante Jukic ,  Jason Wung ,  Ashrith Deshpande ,  Joshua D. Atkins

IPC: G10L25/78 ,  G10L25/81 ,  G10L25/18 ,  G10L21/0232 ,  G10L15/22 ,  G06N7/00 ,  G06N20/00 ,  G10L15/25 ,  G06V40/16 ,  G10L21/0208 ,  G10L21/0216 ,  G10L17/00

Abstract: A device implementing a system for processing speech in an audio signal includes at least one processor configured to receive an audio signal corresponding to at least one microphone of a device, and to determine, using a first model, a first probability that a speech source is present in the audio signal. The at least one processor is further configured to determine, using a second model, a second probability that an estimated location of a source of the audio signal corresponds to an expected position of a user of the device, and to determine a likelihood that the audio signal corresponds to the user of the device based on the first and second probabilities.

公开(公告)号：US10546593B2

公开(公告)日：2020-01-28

申请号：US15830955

申请日：2017-12-04

Applicant: Apple Inc.

Inventor： Jason Wung ,  Mehrez Souden ,  Ramin Pishehvar ,  Joshua D. Atkins

IPC: G10L21/00 ,  G10L19/00 ,  G10L21/02 ,  G10L15/02 ,  G10L21/0232 ,  G10L25/30 ,  H04R1/40 ,  G10L25/03 ,  G10L21/0208

Abstract: A number of features are extracted from a current frame of a multi-channel speech pickup and from side information that is a linear echo estimate, a diffuse signal component, or a noise estimate of the multi-channel speech pickup. A DNN-based speech presence probability is produced for the current frame, where the SPP value is produced in response to the extracted features being input to the DNN. The DNN-based SPP value is applied to configure a multi-channel filter whose input is the multi-channel speech pickup and whose output is a single audio signal. In one aspect, the system is designed to run online, at low enough latency for real time applications such voice trigger detection. Other aspects are also described and claimed.

公开(公告)号：US10403299B2

公开(公告)日：2019-09-03

申请号：US15613127

申请日：2017-06-02

Applicant: Apple Inc.

Inventor： Jason Wung ,  Joshua D. Atkins ,  Ramin Pishehvar ,  Mehrez Souden

IPC: H04M9/08 ,  G10L21/02 ,  G10L21/0208 ,  G10L21/0216 ,  G10L21/0232 ,  G10L21/0272 ,  G10L21/038

Abstract: A digital speech enhancement system that performs a specific chain of digital signal processing operations upon multi-channel sound pick up, to result in a single, enhanced speech signal. The operations are designed to be computationally less complex yet as a whole yield an enhanced speech signal that produces accurate voice trigger detection and low word error rates by an automatic speech recognizer. The constituent operations or components of the system have been chosen so that the overall system is robust to changing acoustic conditions, and can deliver the enhanced speech signal with low enough latency so that the system can be used online (enabling real-time, voice trigger detection and streaming ASR.) Other embodiments are also described and claimed.

公开(公告)号：US20190104357A1

公开(公告)日：2019-04-04

申请号：US15721644

申请日：2017-09-29

Applicant: Apple Inc.

Inventor： Joshua D. Atkins ,  Mehrez Souden ,  Symeon Delikaris-Manias ,  Peter Raffensperger

IPC: H04R1/40 ,  G06N3/08 ,  G06F17/30

CPC classification number: H04R1/406 ,  G06F16/61 ,  G06N3/08 ,  H04R2201/405 ,  H04R2430/23 ,  H04R2499/11 ,  H04S2400/01 ,  H04S2400/15

Abstract: Impulse responses of a device are measured. A database of sound files is generated by convolving source signals with the impulse responses of the device. The sound files from the database are transformed into time-frequency domain. One or more sub-band directional features is estimated at each sub-band of the time-frequency domain. A deep neural network (DNN) is trained for each sub-band based on the estimated one or more sub-band directional features and a target directional feature.