Multi media computing or entertainment system for responding to user presence and activity

    公开(公告)号:US10444854B2

    公开(公告)日:2019-10-15

    申请号:US16055994

    申请日:2018-08-06

    申请人: Apple Inc.

    摘要: Intelligent systems are disclosed that respond to user intent and desires based upon activity that may or may not be expressly directed at the intelligent system. In some embodiments, the intelligent system acquires a depth image of a scene surrounding the system. A scene geometry may be extracted from the depth image and elements of the scene may be monitored. In certain embodiments, user activity in the scene is monitored and analyzed to infer user desires or intent with respect to the system. The interpretation of the user's intent as well as the system's response may be affected by the scene geometry surrounding the user and/or the system. In some embodiments, techniques and systems are disclosed for interpreting express user communication, e.g., expressed through hand gesture movements. In some embodiments, such gesture movements may be interpreted based on real-time depth information obtained from, e.g., optical or non-optical type depth sensors.

    Multi media computing or entertainment system for responding to user presence and activity

    公开(公告)号:US10048765B2

    公开(公告)日:2018-08-14

    申请号:US14865850

    申请日:2015-09-25

    申请人: Apple Inc.

    摘要: Varying embodiments of intelligent systems are disclosed that respond to user intent and desires based upon activity that may or may not be expressly directed at the intelligent system. In some embodiments, the intelligent system acquires a depth image of a scene surrounding the system. A scene geometry may be extracted from the depth image and elements of the scene, such as walls, furniture, and humans may be evaluated and monitored. In certain embodiments, user activity in the scene is monitored and analyzed to infer user desires or intent with respect to the system. The interpretation of the user's intent or desire as well as the system's response may be affected by the scene geometry surrounding the user and/or the system. In some embodiments, techniques and systems are disclosed for interpreting express user communication, for example, expressed through fine hand gesture movements. In some embodiments, such gesture movements may be interpreted based on real-time depth information obtained from, for example, optical or non-optical type depth sensors. The depth information may be interpreted in “slices” (three-dimensional regions of space having a relatively small depth) until one or more candidate hand structures are detected. Once detected, each candidate hand structure may be confirmed or rejected based on its own unique physical properties (e.g., shape, size and continuity to an arm structure). Each confirmed hand structure may be submitted to a depth-aware filtering process before its own unique three-dimensional features are quantified into a high-dimensional feature vector. A two-step classification scheme may be applied to the feature vectors to identify a candidate gesture (step 1), and to reject candidate gestures that do not meet a gesture-specific identification operation (step-2). The identified gesture may be used to initiate some action controlled by a computer system.

    Compositing Pairs Of Image Frames From Different Cameras Of A Mobile Device To Generate A Video Stream
    6.
    发明申请
    Compositing Pairs Of Image Frames From Different Cameras Of A Mobile Device To Generate A Video Stream 审中-公开
    从移动设备的不同相机合成一对图像帧以生成视频流

    公开(公告)号:US20150103135A1

    公开(公告)日:2015-04-16

    申请号:US14579613

    申请日:2014-12-22

    申请人: Apple Inc.

    摘要: Some embodiments provide a novel method for in-conference adjustment of encoded video pictures captured by a mobile device having at least first and second cameras. The method may involve real-time modifications of composite video displays that are generated by the mobile devices involved in such a conference. Specifically, in some embodiments, the mobile devices generate composite displays that simultaneously display multiple videos captured by multiple cameras of one or more devices. In some cases, the composite displays place the videos in adjacent display areas (e.g., in adjacent windows). In other cases, the composite display is a picture-in-picture (PIP) display that includes at least two display areas that show two different videos where one of the display areas is a background main display area and the other is a foreground inset display area that overlaps the background main display area.

    摘要翻译: 一些实施例提供了一种用于会议调整由具有至少第一和第二相机的移动设备捕获的编码视频图像的新颖方法。 该方法可以涉及由这样的会议中涉及的移动设备生成的复合视频显示的实时修改。 具体地,在一些实施例中,移动设备产生同时显示由一个或多个设备的多个摄像机捕获的多个视频的复合显示器。 在某些情况下,复合显示将视频放置在相邻的显示区域中(例如,在相邻窗口中)。 在其他情况下,复合显示器是画中画(PIP)显示器,其包括显示两个不同视频的至少两个显示区域,其中一个显示区域是背景主显示区域,另一个是前景插入显示 与背景主显示区域重叠的区域。

    Real time denoising of video
    7.
    发明授权
    Real time denoising of video 有权
    视频实时去噪

    公开(公告)号:US08675102B2

    公开(公告)日:2014-03-18

    申请号:US13631796

    申请日:2012-09-28

    申请人: Apple Inc.

    IPC分类号: H04N5/217

    摘要: A video enhancement processing system improves perceptual quality of video data with limited processing complexity. The system may perform spatial denoising using filter weights that may vary based on estimated noise of an input image. Specifically, estimated noise of the input image may alter a search neighborhood over which the denoising filter operates, may alter a profile of weights to be applied based on pixel distances and may alter a profile of weights to be applied based on similarity of pixels for denoising processes. As such, the system finds application in consumer devices that perform such enhancement techniques in real time using general purpose processors such as CPUs or GPUs.

    摘要翻译: 视频增强处理系统以有限的处理复杂度提高视频数据的感知质量。 该系统可以使用可基于输入图像的估计噪声而变化的滤波器权重来执行空间去噪。 具体地,输入图像的估计噪声可以改变去噪滤波器操作的搜索邻域,可以基于像素距离改变要应用的权重的轮廓,并且可以基于用于去噪的像素的相似度来改变要应用的权重的轮廓 过程。 因此,该系统在使用诸如CPU或GPU的通用处理器实时地实现这种增强技术的消费者设备中发现应用。

    SYSTEMS AND METHODS FOR REDUCING FIXED PATTERN NOISE IN IMAGE DATA

    公开(公告)号:US20230336888A1

    公开(公告)日:2023-10-19

    申请号:US18330973

    申请日:2023-06-07

    申请人: Apple Inc.

    IPC分类号: H04N25/67

    CPC分类号: H04N25/67

    摘要: The present disclosure generally relates to systems and methods for image data processing. In certain embodiments, an image processing pipeline may be configured to receive a frame of the image data having a plurality of pixels acquired using a digital image sensor. The image processing pipeline may then be configured to determine a first plurality of correction factors that may correct each pixel in the plurality of pixels for fixed pattern noise. The first plurality of correction factors may be determined based at least in part on fixed pattern noise statistics that correspond to the frame of the image data. After determining the first plurality of correction factors, the image processing pipeline may be configured to configured to apply the first plurality of correction factors to the plurality of pixels, thereby reducing the fixed pattern noise present in the plurality of pixels.

    Multi media computing or entertainment system for responding to user presence and activity

    公开(公告)号:US11561621B2

    公开(公告)日:2023-01-24

    申请号:US16600830

    申请日:2019-10-14

    申请人: Apple Inc.

    摘要: Intelligent systems are disclosed that respond to user intent and desires based upon activity that may or may not be expressly directed at the intelligent system. In some embodiments, the intelligent system acquires a depth image of a scene surrounding the system. A scene geometry may be extracted from the depth image and elements of the scene may be monitored. In certain embodiments, user activity in the scene is monitored and analyzed to infer user desires or intent with respect to the system. The interpretation of the user's intent as well as the system's response may be affected by the scene geometry surrounding the user and/or the system. In some embodiments, techniques and systems are disclosed for interpreting express user communication, e.g., expressed through hand gesture movements. In some embodiments, such gesture movements may be interpreted based on real-time depth information obtained from, e.g., optical or non-optical type depth sensors.

    SYSTEMS AND METHOD FOR REDUCING FIXED PATTERN NOISE IN IMAGE DATA

    公开(公告)号:US20220014697A1

    公开(公告)日:2022-01-13

    申请号:US17377308

    申请日:2021-07-15

    申请人: Apple Inc.

    IPC分类号: H04N5/365

    摘要: The present disclosure generally relates to systems and methods for image data processing. In certain embodiments, an image processing pipeline may be configured to receive a frame of the image data having a plurality of pixels acquired using a digital image sensor. The image processing pipeline may then be configured to determine a first plurality of correction factors that may correct each pixel in the plurality of pixels for fixed pattern noise. The first plurality of correction factors may be determined based at least in part on fixed pattern noise statistics that correspond to the frame of the image data. After determining the first plurality of correction factors, the image processing pipeline may be configured to configured to apply the first plurality of correction factors to the plurality of pixels, thereby reducing the fixed pattern noise present in the plurality of pixels.