SYSTEMS AND METHODS TO INTERACT WITH DISCOVERABLE OBJECTS BY POINTING A LIGHT BEAM USING A HANDHELD DEVICE

    公开(公告)号:US20240038084A1

    公开(公告)日:2024-02-01

    申请号:US18201094

    申请日:2023-05-23

    申请人: KINOO, Inc.

    摘要: Systems and methods are described in which one or more attributes or cues related to a discoverable object are conveyed to a user of handheld device via audible, haptic and/or visual means. Using a light beam generated by the handheld device and pointed in the same direction as a device camera, the user may point toward one or more objects in the user's environment believed to be associated with the one or more attributes or cues. Based on the region being pointed at within camera images, the handheld device may classify objects within images to determine if an object matches one or more templates or classifications of the discoverable object. A match, or mismatch, of objects being pointed at may determine subsequent actions. Systems and methods may provide simple and intuitive methods for human-machine interactions that may be particularly well-suited for learners.

    Systems and methods for time-sharing interactions using a shared artificial intelligence personality

    公开(公告)号:US10915814B2

    公开(公告)日:2021-02-09

    申请号:US16902168

    申请日:2020-06-15

    申请人: Kinoo, Inc.

    IPC分类号: G06N3/08 G06K9/00 G06N99/00

    摘要: Systems and methods are described for time-sharing interactions using a shared artificial intelligence personality (AIP) incorporated within multiple human interaction entities (HIEs). An AIP is an understanding construct that may control a variety of communication experiences to support a sense of ongoing social connectedness. An AIP may be instantiated within two or more HIEs that interact with humans in a human, cartoon or pet-like manner. HIEs may include robots, robotic pets, toys, simple-to-use devices, and graphical user interfaces. The AIP may be periodically updated based on human interactions sensed by the HIEs as well as knowledge of historical and ongoing events. The systems may provide two or more users with intuitive machine companions that exhibit an expert knowledge base and a familiar, cumulative personality. HIEs may continue to operate without interruption in the presence of interruptions, and/or the absence of one or more human participants; allowing participants to “time-share” their sense of connectedness.

    Systems and methods to specify interactive page locations by pointing a light beam using a handheld device

    公开(公告)号:US11989357B1

    公开(公告)日:2024-05-21

    申请号:US18220738

    申请日:2023-07-11

    申请人: KINOO, Inc.

    摘要: Systems and methods are described in which a light beam generated by a handheld device is used to point toward a selected object or location within a printed page containing visual content, for example displayed in a book or magazine. The object(s) and/or location being pointed at may be determined using a device camera pointed in the same direction as the beam coupled with computer vision methods to determine locations of camera-acquired images within a database of page-content templates. The database may include interactive sequences and/or scenarios associated with page objects and locations. Such interactive sequences may augment the visual content of a printed book to include audible elements, additional visual components, and/or haptic stimulation enacted by the handheld device. Systems and methods may provide simple and intuitive methods for human-machine interactions that may be particularly well-suited for children and other learners.

    Systems and methods to interactively control delivery of serial content using a handheld device

    公开(公告)号:US11941185B1

    公开(公告)日:2024-03-26

    申请号:US18091274

    申请日:2022-12-29

    申请人: KINOO, Inc.

    摘要: Systems and methods are described in which user reactions to one or more interactions initiated by a handheld device are a basis for controlling the broadcasting of serial content on a content delivery device such as a television or tablet. Interactions initiated on the handheld device may include visual, acoustic and/or haptic components based on the one or more contexts of concurrent (i.e., current or recently broadcast) or ensuing (i.e., not yet broadcast) content. User reactions may be determined based on device movements or orientations (sensed by a camera or inertial measurement unit), verbal replies (sensed by a microphone), and/or pushbutton indications. Based on user reactions and/or predetermined preferences, serial content may be replayed, paused, bypassed, played in slow-motion, or played in a so-called “fast forward” mode. Systems and methods may provide simple and intuitive methods to control serial content delivery that may be particularly well-suited for young children.

    Systems and methods for collective control of virtual objects

    公开(公告)号:US11409359B1

    公开(公告)日:2022-08-09

    申请号:US17531571

    申请日:2021-11-19

    申请人: KINOO, Inc.

    摘要: Systems and methods are described to enact machine-based, collective control by two individuals of one or more displayed virtual objects. Collective interactions may be implemented by combining an ability to specify one or more locations on a touch-sensitive display using one or more digits of a first user with an ability to monitor a portable, handheld controller manipulated by the other user. Alternatively or in addition, pointing by the first hand to the one or more locations on a display may be enhanced by a stylus or other pointing device. The handheld controller may be tracked within camera-acquired images by following camera-trackable controller components and/or by acquiring measurements from one or more embedded internal measurement units (IMUs). Optionally, one or more switches or sensors may be included within the handheld controller, operable by one or more digits of the second hand to enable alternative virtual object display and/or menu selections during collective interactions.

    Determining tap locations on a handheld electronic device based on inertial measurements

    公开(公告)号:US11947399B2

    公开(公告)日:2024-04-02

    申请号:US18117420

    申请日:2023-03-04

    申请人: KINOO, Inc.

    摘要: Systems and methods are described in which the location of a tap on the body of a handheld device is determined in real time using data streams from an embedded inertial measurement unit (IMU). Taps may be generated by striking the handheld device with an object (e.g., a finger), or by moving the handheld device in a manner that causes it to strike another object. IMU accelerometer, gyroscopic and/or orientation (relative to the magnetic and/or gravitational pull of the earth) measurements are examined for signatures that distinguish a tap at a location on the body of the device compared with signal characteristics produced by taps at other locations. Neural network and/or numerical methods may be used to perform such classifications. Tap locations, tap timing and tap attributes such as the magnitude of applied forces, device orientation, and the amplitude and directions of motions during and following a tap, may be used to control or modulate responses within the handheld device and/or actions within connected devices.

    DETERMINING TAP LOCATIONS ON A HANDHELD ELECTRONIC DEVICE BASED ON INERTIAL MEASUREMENTS

    公开(公告)号:US20240036620A1

    公开(公告)日:2024-02-01

    申请号:US18117420

    申请日:2023-03-04

    申请人: KINOO, Inc.

    摘要: Systems and methods are described in which the location of a tap on the body of a handheld device is determined in real time using data streams from an embedded inertial measurement unit (IMU). Taps may be generated by striking the handheld device with an object (e.g., a finger), or by moving the handheld device in a manner that causes it to strike another object. IMU accelerometer, gyroscopic and/or orientation (relative to the magnetic and/or gravitational pull of the earth) measurements are examined for signatures that distinguish a tap at a location on the body of the device compared with signal characteristics produced by taps at other locations. Neural network and/or numerical methods may be used to perform such classifications. Tap locations, tap timing and tap attributes such as the magnitude of applied forces, device orientation, and the amplitude and directions of motions during and following a tap, may be used to control or modulate responses within the handheld device and/or actions within connected devices.

    Systems and methods to enhance interactive engagement with shared content by a contextual virtual agent

    公开(公告)号:US11366997B2

    公开(公告)日:2022-06-21

    申请号:US17233473

    申请日:2021-04-17

    申请人: KINOO, Inc.

    摘要: Systems and methods are described to enhance interactive engagement during simultaneous delivery of serial or digital content (e.g., audio, video) to a plurality of users. A machine-based awareness of the context of the content and/or one or more user reactions to the presentation of the content may be used as a basis to interrupt content delivery in order to intersperse a snippet that includes a virtual agent with an awareness of the context(s) of the content and/or the one or more user reactions. This “contextual virtual agent” (CVA) enacts actions and/or dialog based on the one or more machine-classified contexts coupled with identified interests and/or aspirations of individuals within the group of users. The CVA may also base its activities on a machine-based awareness of “future” content that has not yet been delivered to the group, but classified by natural language and/or computer vision processing. Interrupting the delivery of content substantially simultaneously to a group of users and initiating dialog regarding content by a CVA enhances opportunities for users to engage with each other about their shared interactive experience.

    SYSTEMS AND METHODS TO MEASURE AND ENHANCE HUMAN ENGAGEMENT AND COGNITION

    公开(公告)号:US20210390876A1

    公开(公告)日:2021-12-16

    申请号:US17200722

    申请日:2021-03-12

    申请人: KINOO, Inc.

    摘要: Systems and methods are described to enact machine-based, simultaneous classification of emotional and cognitive states of an individual in substantially real time. A bootstrap approach is used to identify indicators of emotional state, including boredom, engagement and feeling overwhelmed, by finding additional interactions temporally adjacent to known indicators. A similar bootstrap approach is used to identify new indicators of knowledge acquisition (or confusion), by locating frequent temporally adjacent interactions. Systems and methods may enhance teaching by identifying knowledge presentations that result in successful knowledge acquisition, and learning by allowing instructors associated with an individual to track materials that have been understood (or misunderstood). Machine-based tracking of learning may circumvent boredom by avoiding needless repetition, and facilitate teaching of synergistic topics related to materials recently understood. Systems and methods may provide for an enhanced, individualized environment for both emotional and cognitive development.