公开(公告)号：US11745353B2

公开(公告)日：2023-09-05

申请号：US17106889

申请日：2020-11-30

Applicant: X Development LLC

Inventor： Guy Satat

IPC: G06V10/60 ,  B25J9/16 ,  B25J19/02 ,  G01N21/359 ,  G06F18/21 ,  G06F18/24 ,  H04N23/10 ,  H04N23/56 ,  H04N23/90 ,  H04N23/695 ,  G06V10/58 ,  B25J5/00 ,  G01N21/55

CPC classification number: B25J9/1697 ,  B25J9/163 ,  B25J9/1664 ,  B25J19/023 ,  G01N21/359 ,  G06F18/21 ,  G06F18/24 ,  G06V10/60 ,  H04N23/10 ,  H04N23/56 ,  H04N23/695 ,  H04N23/90 ,  B25J5/007 ,  G01N2021/555 ,  G06V10/58

Abstract: A method includes identifying a target surface in an environment of a robotic device. The method further includes controlling a moveable component of the robotic device to move along a motion path relative to the target surface, wherein the moveable component comprises a light source and a camera. The method additionally includes receiving a plurality of images from the camera when the moveable component is at a plurality of poses along the motion path and when the light source is illuminating the target surface. The method also includes determining bidirectional reflectance distribution function (BRDF) image data, wherein the BRDF image data comprises the plurality of images converted to angular space with respect to the target surface. The method further includes determining, based on the BRDF image data and by applying at least one pre-trained machine learning model, a material property of the target surface.

公开(公告)号：US20230247015A1

公开(公告)日：2023-08-03

申请号：US18191599

申请日：2023-03-28

Applicant: X Development LLC

Inventor： Guy Satat ,  Michael Quinlan

IPC: H04L9/40 ,  H04L9/32 ,  H04L9/00 ,  H04L9/30

CPC classification number: H04L63/0807 ,  H04L9/3236 ,  H04L9/006 ,  H04L63/083 ,  H04L63/0876 ,  H04L9/3073 ,  H04L9/3247 ,  H04L9/3263 ,  H04L63/0861 ,  H04L9/321 ,  H04L63/0823 ,  H04L63/0884 ,  H04L63/102 ,  H04L9/3268 ,  H04L9/50

Abstract: A method includes receiving sensor data from a plurality of robot sensors on a robot. The method includes generating a depth map that includes a plurality of pixel depths. The method includes determining, for each respective pixel depth, based on the at least one robot sensor associated with the respective pixel depth, a pixelwise confidence level indicative of a likelihood that the respective pixel depth accurately represents a distance between the robot and a feature of the environment. The method includes generating a pixelwise filterable depth map for a control system of the robot. The pixelwise filterable depth map is filterable to produce a robot operation specific depth map. The robot operation specific depth map is determined based on a comparison of each respective pixelwise confidence level with a confidence threshold corresponding to at least one operation of the robot controlled by the control system of the robot.

公开(公告)号：US20230150151A1

公开(公告)日：2023-05-18

申请号：US18053739

申请日：2022-11-08

Applicant: X Development LLC

Inventor： Eden Rephaeli ,  Marc Strauss ,  Guy Satat

IPC: B25J19/02 ,  B25J5/00 ,  B25J18/00

CPC classification number: B25J19/021 ,  B25J5/007 ,  B25J18/00

Abstract: A sensing device is described for mounting on a movable component of a robotic device. The sensing device includes a plurality of illumination sources comprising at least one ultraviolet (UV) illumination source. The sensing device further includes at least two cameras arranged in a stereo pair. The sensing device additionally includes a camera with a UV filter, wherein the UV filter is configured to allow wavelengths corresponding to UV light and to block wavelengths corresponding to visible and near infrared light, wherein the UV filter allows transmission of light within an angular range such that the UV filter allows for the transmission of light at one end of the angular range to be equivalent to the transmission of light at an opposite end of the angular range.

公开(公告)号：US20230015589A1

公开(公告)日：2023-01-19

申请号：US17934620

申请日：2022-09-23

Applicant: X Development LLC

Inventor： Guy Satat

IPC: H04N13/246 ,  G06T7/70 ,  G06T7/80 ,  G06K9/62 ,  G06V20/10 ,  G06N3/08 ,  H04N13/271

Abstract: A method includes receiving, from a multiscopic image capture system, a plurality of images depicting a scene. The method includes determining, by application of a neural network based on the plurality of images, a disparity map of the scene. The neural network includes a plurality of layers, and the layers include a rectification layer. The method include determining a matching error of the disparity map based on differences between corresponding pixels of two or more images associated with the disparity map. The method includes back-propagating the matching error to the rectification layer of the neural network. Back-propagating the matching error includes updating one or more weights applied to the rectification layer.

公开(公告)号：US11496722B1

公开(公告)日：2022-11-08

申请号：US17132097

申请日：2020-12-23

Applicant: X Development LLC

Inventor： Guy Satat

IPC: H04N13/246 ,  G06T7/70 ,  H04N13/271 ,  G06V20/10 ,  G06T7/80 ,  G06N3/08 ,  G06K9/62

Abstract: A method includes receiving, from a multiscopic image capture system, a plurality of images depicting a scene. The method includes determining, by application of a neural network based on the plurality of images, a disparity map of the scene. The neural network includes a plurality of layers, and the layers include a rectification layer. The method include determining a matching error of the disparity map based on differences between corresponding pixels of two or more images associated with the disparity map. The method includes back-propagating the matching error to the rectification layer of the neural network. Back-propagating the matching error includes updating one or more weights applied to the rectification layer.

公开(公告)号：US11440196B1

公开(公告)日：2022-09-13

申请号：US16717896

申请日：2019-12-17

Applicant: X Development LLC

Inventor： Sean Kirmani ,  Guy Satat ,  Michael Quinlan

IPC: B25J9/16 ,  G06N3/04 ,  G06T7/70 ,  G06N20/20 ,  G06T7/277

Abstract: A method includes receiving sensor data representing a first object in an environment and generating, based on the sensor data, a first state vector that represents physical properties of the first object. The method also includes generating, by a first machine learning model and based on the first state vector and a second state vector that represents physical properties of a second object previously observed in the environment, a metric indicating a likelihood that the first object is the same as the second object. The method further includes determining, based on the metric, to update the second state vector and updating, by a second machine learning model configured to maintain the second state vector over time and based on the first state vector, the second state vector to incorporate into the second state vector information concerning physical properties of the second object as represented in the first state vector.

公开(公告)号：US20220193888A1

公开(公告)日：2022-06-23

申请号：US17382561

申请日：2021-07-22

Applicant: X Development LLC

Inventor： Eden Rephaeli ,  Guy Satat ,  Daniel Lam ,  Benjamin Holson ,  Jiajun Xu

IPC: B25J9/04 ,  B25J9/16

Abstract: A system includes a robotic device, a sensor disposed on the robotic device, and circuitry configured to perform operations. The operations include determining a map that represents stationary features of an environment and receiving, from the sensor, sensor data representing the environment. The operations also include determining, based on the sensor data, a representation of an actor within the environment, where the representation includes keypoints representing corresponding body locations of the actor. The operations also include determining that a portion of a particular stationary feature is positioned within a threshold distance of a particular keypoint and, based on thereon, updating the map to indicate that the portion is to be cleaned. The operations further include, based on the map as updated, causing the robotic device to clean the portion of the particular stationary feature.

公开(公告)号：US20220168909A1

公开(公告)日：2022-06-02

申请号：US17106906

申请日：2020-11-30

Applicant: X Development LLC

Inventor： Guy Satat ,  Eden Rephaeli

IPC: B25J19/02 ,  B25J9/16 ,  B25J5/00 ,  G06T1/00 ,  G06T7/73

Abstract: A method includes receiving, from at least two fixed cameras in a static sensor arrangement on a mobile robotic device, one or more images representative of an environment of the mobile robotic device, wherein the at least two fixed cameras have a combined 360 degree horizontal field of view around the mobile robotic device. The method further includes determining, from the one or more images, a presence of an object in the environment of the mobile robotic device. The method additionally includes controlling a moveable sensor arrangement of the mobile robotic device to move towards the object, wherein the movable sensor arrangement comprises at least one movable camera on the mobile robotic device. The method also includes receiving, from the at least one movable camera of the moveable sensor arrangement, one or more additional images representative of the object.