公开(公告)号：US11455328B2

公开(公告)日：2022-09-27

申请号：US16869214

申请日：2020-05-07

申请人： Roku, Inc.

发明人： Prashant Ramanathan ,  Jose Pio Pereira ,  Shashank Merchant ,  Mihailo M. Stojancic

IPC分类号： G06F16/00 ,  G06F16/48 ,  G06F16/41 ,  G06F16/28 ,  G06F16/783 ,  G06F16/951 ,  G06F16/22 ,  G06F16/901 ,  G06V10/46 ,  G06V20/40

摘要： The overall architecture and details of a scalable video fingerprinting and identification system that is robust with respect to many classes of video distortions is described. In this system, a fingerprint for a piece of multimedia content is composed of a number of compact signatures, along with traversal hash signatures and associated metadata. Numerical descriptors are generated for features found in a multimedia clip, signatures are generated from these descriptors, and a reference signature database is constructed from these signatures. Query signatures are also generated for a query multimedia clip. These query signatures are searched against the reference database using a fast similarity search procedure, to produce a candidate list of matching signatures. This candidate list is further analyzed to find the most likely reference matches. Signature correlation is performed between the likely reference matches and the query clip to improve detection accuracy.

公开(公告)号：US11445242B2

公开(公告)日：2022-09-13

申请号：US16441996

申请日：2019-06-14

申请人： Roku, Inc.

发明人： Mihailo M. Stojancic ,  Daniel H. Eakins ,  Shashank Merchant ,  Prashant Ramanathan ,  Jose Pio Pereira

IPC分类号： H04H20/14 ,  H04H60/29 ,  H04N21/439 ,  H04N21/8352 ,  G10H1/00 ,  H04N21/414 ,  H04N21/422 ,  H04N21/4223 ,  H04N21/433 ,  H04N21/84 ,  H04N21/845 ,  H04N1/32 ,  H04N21/43

摘要： A mobile device responds in real time to media content presented on a media device, such as a television. The mobile device captures temporal fragments of audio-video content on its microphone, camera, or both and generates corresponding audio-video query fingerprints. The query fingerprints are transmitted to a search server located remotely or used with a search function on the mobile device for content search and identification. Audio features are extracted and audio signal global onset detection is used for input audio frame alignment. Additional audio feature signatures are generated from local audio frame onsets, audio frame frequency domain entropy, and maximum change in the spectral coefficients. Video frames are analyzed to find a television screen in the frames, and a detected active television quadrilateral is used to generate video fingerprints to be combined with audio fingerprints for more reliable content identification.

公开(公告)号：US11336952B2

公开(公告)日：2022-05-17

申请号：US15053064

申请日：2016-02-25

申请人： Roku, Inc.

发明人： Mihailo M. Stojancic ,  Jose Pio Pereira ,  Peter Wendt ,  Shashank Merchant ,  Sunil Suresh Kulkarni

IPC分类号： H04N21/43 ,  H04N1/32 ,  H04N21/439 ,  H04N21/8352 ,  G10H1/00 ,  H04N21/414 ,  H04N21/422 ,  H04N21/4223 ,  H04N21/433 ,  H04N21/84 ,  H04N21/845

摘要： A mobile device responds in real time to media content presented on a media device, such as a television. The mobile device captures temporal fragments of audio-video content on its microphone, camera, or both and generates corresponding audio-video query fingerprints. The query fingerprints are transmitted to a search server located remotely or used with a search function on the mobile device for content search and identification. Audio features are extracted and audio signal global onset detection is used for input audio frame alignment. Additional audio feature signatures are generated from local audio frame onsets, audio frame frequency domain entropy, and maximum change in the spectral coefficients. Video frames are analyzed to find a television screen in the frames, and a detected active television quadrilateral is used to generate video fingerprints to be combined with audio fingerprints for more reliable content identification.

公开(公告)号：US11163818B2

公开(公告)日：2021-11-02

申请号：US16387443

申请日：2019-04-17

申请人： Roku, Inc.

发明人： Jose Pio Pereira ,  Sunil Suresh Kulkarni ,  Shashank Merchant ,  Prashant Ramanathan ,  Pradipkumar Dineshbhai Gajjar

IPC分类号： G06F16/00 ,  G06F16/48 ,  G06F16/41 ,  G06F16/28 ,  G06F16/783 ,  G06F16/951 ,  G06F16/22 ,  G06F16/901 ,  G06K9/00 ,  G06K9/46

摘要： The overall architecture and details of a scalable video fingerprinting and identification system that is robust with respect to many classes of video distortions is described. In this system, a fingerprint for a piece of multimedia content is composed of a number of compact signatures, along with traversal hash signatures and associated metadata. Numerical descriptors are generated for features found in a multimedia clip, signatures are generated from these descriptors, and a reference signature database is constructed from these signatures. Query signatures are also generated for a query multimedia clip. These query signatures are searched against the reference database using a fast similarity search procedure, to produce a candidate list of matching signatures. This candidate list is further analyzed to find the most likely reference matches. Signature correlation is performed between the likely reference matches and the query clip to improve detection accuracy.

公开(公告)号：US11631404B2

公开(公告)日：2023-04-18

申请号：US17444946

申请日：2021-08-12

申请人： Roku, Inc.

发明人： Jose Pio Pereira ,  Sunil Suresh Kulkarni ,  Mihailo M. Stojancic ,  Shashank Merchant ,  Peter Wendt

IPC分类号： G10L15/16 ,  G10L15/20 ,  G06T7/246 ,  G06T7/215 ,  G06K9/62 ,  G06F16/00 ,  G06T7/254 ,  G06F16/45 ,  G06F16/48 ,  G06V10/50 ,  G06V20/40 ,  G10L15/02 ,  G10L15/06 ,  G10L15/10 ,  G10L15/14 ,  G10L21/0232 ,  G10L25/81 ,  G06F16/906 ,  G06F16/44 ,  G10L21/0216

摘要： Audio distortion compensation methods to improve accuracy and efficiency of audio content identification are described. The method is also applicable to speech recognition. Methods to detect the interference from speakers and sources, and distortion to audio from environment and devices, are discussed. Additional methods to detect distortion to the content after performing search and correlation are illustrated. The causes of actual distortion at each client are measured and registered and learnt to generate rules for determining likely distortion and interference sources. The learnt rules are applied at the client, and likely distortions that are detected are compensated or heavily distorted sections are ignored at audio level or signature and feature level based on compute resources available. Further methods to subtract the likely distortions in the query at both audio level and after processing at signature and feature level are described.

公开(公告)号：US20220239977A1

公开(公告)日：2022-07-28

申请号：US17722523

申请日：2022-04-18

申请人： Roku, Inc.

发明人： Mihailo M. Stojancic ,  Jose Pio Pereira ,  Peter Wendt ,  Shashank Merchant ,  Sunil Suresh Kulkarni

IPC分类号： H04N21/439 ,  H04N1/32 ,  H04N21/8352 ,  G10H1/00 ,  H04N21/414 ,  H04N21/422 ,  H04N21/4223 ,  H04N21/433 ,  H04N21/84 ,  H04N21/845

摘要： A mobile device responds in real time to media content presented on a media device, such as a television. The mobile device captures temporal fragments of audio-video content on its microphone, camera, or both and generates corresponding audio-video query fingerprints. The query fingerprints are transmitted to a search server located remotely or used with a search function on the mobile device for content search and identification. Audio features are extracted and audio signal global onset detection is used for input audio frame alignment. Additional audio feature signatures are generated from local audio frame onsets, audio frame frequency domain entropy, and maximum change in the spectral coefficients. Video frames are analyzed to find a television screen in the frames, and a detected active television quadrilateral is used to generate video fingerprints to be combined with audio fingerprints for more reliable content identification.

公开(公告)号：US11361017B1

公开(公告)日：2022-06-14

申请号：US16929858

申请日：2020-07-15

申请人： Roku, Inc.

发明人： Sunil Suresh Kulkarni ,  Pradipkumar Dineshbhai Gajjar ,  Jose Pio Pereira ,  Preshant Ramanathan ,  Mihailo M. Stojancic ,  Shashank Merchant

IPC分类号： G06F16/45 ,  G06F16/48 ,  G06F16/906 ,  G06F16/44

摘要： Techniques are described that exclude use of “stop-fingerprints” from media database formation and search query to an automatic content recognition (ACR) systems based on media content fingerprints updated by stop-fingerprint analysis. A classification process is presented which takes in fingerprints from reference media files as an input and produces a modified set of fingerprints as an output by applying a novel stop-fingerprint classification algorithm. Architecture for the distributed stop-fingerprint generation is presented. Various cases, as stop-fingerprints generation for the entire reference database, stop-fingerprints generation for the individual reference fingerprint files, and temporal fingerprint classification obtained through intermediate steps of the temporal fingerprint classification algorithm are presented. A hash-based signature classification algorithm is also described.

公开(公告)号：US11188587B2

公开(公告)日：2021-11-30

申请号：US16387456

申请日：2019-04-17

申请人： Roku, Inc.

发明人： Jose Pio Pereira ,  Sunil Suresh Kulkarni ,  Shashank Merchant ,  Prashant Ramanathan ,  Pradipkumar Dineshbhai Gajjar

IPC分类号： G06F16/00 ,  G06F16/48 ,  G06F16/41 ,  G06F16/28 ,  G06F16/783 ,  G06F16/951 ,  G06F16/22 ,  G06F16/901 ,  G06K9/00 ,  G06K9/46

摘要： The overall architecture and details of a scalable video fingerprinting and identification system that is robust with respect to many classes of video distortions is described. In this system, a fingerprint for a piece of multimedia content is composed of a number of compact signatures, along with traversal hash signatures and associated metadata. Numerical descriptors are generated for features found in a multimedia clip, signatures are generated from these descriptors, and a reference signature database is constructed from these signatures. Query signatures are also generated for a query multimedia clip. These query signatures are searched against the reference database using a fast similarity search procedure, to produce a candidate list of matching signatures. This candidate list is further analyzed to find the most likely reference matches. Signature correlation is performed between the likely reference matches and the query clip to improve detection accuracy.

公开(公告)号：US11132997B1

公开(公告)日：2021-09-28

申请号：US16140538

申请日：2018-09-25

申请人： Roku, Inc.

发明人： Jose Pio Pereira ,  Sunil Suresh Kulkarni ,  Mihailo M. Stojancic ,  Shashank Merchant ,  Peter Wendt

IPC分类号： G10L15/20 ,  G10L25/81 ,  G10L21/0232 ,  G10L15/06 ,  G10L15/10 ,  G10L15/14 ,  G10L15/02 ,  G10L21/0216

摘要： Audio distortion compensation methods to improve accuracy and efficiency of audio content identification are described. The method is also applicable to speech recognition. Methods to detect the interference from speakers and sources, and distortion to audio from environment and devices are discussed. Additional methods to detect distortion to the content after performing search and correlation are illustrated. The causes of actual distortion at each client are measured and registered and learnt to generate rules for determining likely distortion and interference sources. The learnt rules are applied at the client, and likely distortions that are detected are compensated or heavily distorted sections are ignored at audio level or signature and feature level based on compute resources available. Further methods to subtract the likely distortions in the query at both audio level and after processing at signature and feature level are described.

公开(公告)号：US11630858B2

公开(公告)日：2023-04-18

申请号：US17408248

申请日：2021-08-20

申请人： Roku, Inc.

发明人： Mihailo M. Stojancic ,  Prashant Ramanathan ,  Peter Wendt ,  Jose Pio Pereira ,  Shashank Merchant

IPC分类号： G06F16/00 ,  G06F16/48 ,  G06F16/41 ,  G06F16/28 ,  G06F16/783 ,  G06F16/951 ,  G06F16/22 ,  G06F16/901 ,  G06V10/46 ,  G06V20/40

摘要： The overall architecture and details of a scalable video fingerprinting and identification system that is robust with respect to many classes of video distortions is described. In this system, a fingerprint for a piece of multimedia content is composed of a number of compact signatures, along with traversal hash signatures and associated metadata. Numerical descriptors are generated for features found in a multimedia clip, signatures are generated from these descriptors, and a reference signature database is constructed from these signatures. Query signatures are also generated for a query multimedia clip. These query signatures are searched against the reference database using a fast similarity search procedure, to produce a candidate list of matching signatures. This candidate list is further analyzed to find the most likely reference matches. Signature correlation is performed between the likely reference matches and the query clip to improve detection accuracy.