Abstract:
A wearable apparatus may automatically monitor consumption by a user of the wearable apparatus by analyzing images captured from an environment of the user. The wearable apparatus may include at least one image capture device configured to capture a plurality of images from an environment of the user of the wearable apparatus. The wearable apparatus may also include at least one processing device configured to: analyze the plurality of images to detect a consumable product represented in at least one of the plurality of images; based on the detection of the consumable product represented in at least one of the plurality of images, analyze one or more of the plurality of images to determine a type indicator associated with the detected consumable product; analyze the one or more of the plurality of images to estimate an amount of the consumable product consumed by the user; determine a feedback based on the type indicator of the detected consumable product and the estimated amount of the consumable product consumed by the user; and cause the feedback to be outputted to the user.
Abstract:
Apparatuses and a method are provided for providing feedback to a user, who may be visually-impaired. In one implementation, a method is provided for providing feedback to a visually impaired user. The method comprises receiving from a mobile image sensor real time image data that includes a representation of an object in an environment of the visually impaired user. The mobile image sensor is configured to be connected to glasses worn by the visually impaired user. Further, the method comprises receiving a signal indicating a desire of the visually impaired user to obtain information about the object. The method also includes accessing a database holding information about a plurality of objects, and comparing information derived from the received real time image data with information in the database. The method comprises providing the visually impaired user with nonvisual feedback that the object is not locatable in the database.
Abstract:
An apparatus and method are provided for performing one or more actions based on triggers detecting within captured image data. In one implementation, a method is provided for audibly reading text retrieved from a captured image. According to the method, real-time image data is captured from an environment of a user, and an existence of a trigger is determined within the captured image data. In one aspect, the trigger may be associated with a desire of the user to hear text read aloud, and the trigger identifies an intermediate portion of the text a distance from a level break in the text. The method includes performing a layout analysis on the text to identify the level break associated with the trigger, and reading aloud text beginning from the level break associated with the trigger.
Abstract:
Systems and methods are disclosed for using a wearable apparatus in social events. In one implementation, a system may comprise an image sensor, an audio sensor, and a processor. The processor may be configured to receive images captured by the image sensor and receive an audio signal representative of sound captured by the audio sensor. The processor may determine, based on the images or the audio signal, whether an individual is a recognized individual of the user. When the individual is not recognized, the processor may identify the individual based on an external resource. The processor may further identify a content source associated with the individual, identify a content item associated with the individual, and provide the content item to a computing device associated with the user.
Abstract:
A device and method are provided for providing feedback based on the state of an object. In one implementation, an apparatus for processing images is provided. The apparatus may include an image sensor configured to capture real time images from an environment of a user and at least one processor device configured to initially process at least one image to determine whether an object is likely to change its state. If a determination is made that the object is unlikely to change its state, the at least one processor device may additionally process the at least one image and provide a first feedback. If a determination is made that the object is likely to change its state, the at least one processor device may continue to capture images of the object and alert the user with a second feedback after a change in the state of the object occurs.
Abstract:
An apparatus and method are provided for identifying and audibly presenting textual information within captured image data. In one implementation, a method is provided for audibly presenting text retrieved from a captured image. According to the method, at least one image of text is received from an image sensor, and the text may include a first portion and a second portion. The method includes identifying contextual information associated with the text, and accessing at least one rule associating the contextual information with at least one portion of text to be excluded from an audible presentation associated with the text. The method further includes performing an analysis on the at least one image to identify the first portion and the second portion, and causing the audible presentation of the first portion.
Abstract:
An apparatus and method are provided for identifying and executing system commands based on captured image data. In one implementation, a method is provided for executing at least one command retrieved from a captured image. According to the method, image data is received from an image sensor, and the image data may include printed information associated with a specific system commands. The method further includes accessing a database including a plurality of predefined system commands associated with printed information, and identifying in the image data an existence of the printed information associated with the specific system command stored in the database. The specific system command is executed after the printed information associated with the specific system command is identified.
Abstract:
Systems and methods are disclosed for selectively reading text. A system may comprise an image capture device, an audio capture device, and a processor. The processor may be configured to receive images captured by the image capture device and audio signals captured by the audio capture device. The processor may analyze the image to identify text represented in the image; identify, based on the image, a structural element of the text; identify a request to read a first portion of the text associated with the structural element, the request being identified by at least one of analyzing the audio signals to detect a spoken request or detecting a gesture in the plurality of images; and present the first portion of text to the user of the wearable device.
Abstract:
Systems and methods are disclosed for locating an object for a user. A system may comprise an image capture device, an audio capture device, and a processor. The processor may be configured to receive images captured by the image capture device and audio signals received by the audio capture device. The processor may analyze the audio signals to identify a descriptor word describing the object and retrieve a visual characteristic of the object based on the descriptor word. The processor may then determine a location of the object in the images based on the visual characteristic, determine a location of a hand of the user in the images, and determine a direction between the hand and the object. The processor may then determine feedback indicative of the direction and provide the feedback to the user.
Abstract:
A hearing aid and related systems and methods are disclosed. In one implementation, a hearing aid system (2300) may include a wearable camera (2301); a microphone (2302); and a processor (2303). The processor (2303) may be programmed to receive images captured by the camera (2301); receive audio signals representative of sounds received by the at least one microphone (2302); determine a look direction (2030) of the user based on analysis of the images; determine an amplitude of a first audio signal associated with an individual or object in a region associated with the look direction of the user; determine an amplitude of a second audio signal from a region other than the look direction of the user; adjust the second amplitude in accordance with the first amplitude; and cause transmission of the second audio signal at the adjusted amplitude to a hearing interface device configured to provide sound to an ear of the user (100).