公开(公告)号：US11586852B2

公开(公告)日：2023-02-21

申请号：US16560612

申请日：2019-09-04

Applicant: Rockwell Automation Technologies, Inc.

Inventor： Thong T. Nguyen ,  Paul D. Schmirler ,  Timothy T. Duffy ,  Hannah M. Schermerhorn ,  Jingbo Liu

IPC: G06K9/62 ,  G06N5/022 ,  G06V30/32

Abstract: A system includes a training system configured to display first image data and a remote expert system configured to display second image data that corresponds to the first image data, receive feedback data associated with the second image data, and transmit a command to the training system based on the feedback data. The command is configured to modify the first image data presented via the training system.

公开(公告)号：US20220253129A1

公开(公告)日：2022-08-11

申请号：US17660928

申请日：2022-04-27

Applicant: Rockwell Automation Technologies, Inc.

Inventor： Thong T. Nguyen ,  John M. Van Hecke ,  Alex L. Nicoll ,  Paul D. Schmirler ,  Mark Bjerke

IPC: G06F3/01 ,  G06T11/60 ,  G09B9/00 ,  G09B19/00 ,  G06F3/03 ,  G06T11/00

Abstract: Techniques for employing augmented reality or virtual reality information are presented. An information management component (IMC) of an augmented reality device (ARD) can monitor and detect user activities and conditions in area in proximity to ARD. Based on user activities and conditions, IMC can determine augmented reality information that can enhance user experience, performance of user activities, or security and safety of user. IMC can present, via an interface component of ARD, the augmented reality information to the user. The augmented reality information can relate to user location; navigation by the user; tasks to be performed by the user; product assembly; maintenance work; system or product design or configuration; remote control of assembly, maintenance, design, or configuration; environmental and/or hazardous conditions; security, identification, and authentication of users; or training the user to perform tasks. IMC can translate information from a language to a different language of the user.

公开(公告)号：US11265513B2

公开(公告)日：2022-03-01

申请号：US16915152

申请日：2020-06-29

Applicant: Rockwell Automation Technologies, Inc.

Inventor： Paul D. Schmirler ,  Thong T. Nguyen ,  Alex L. Nicoll ,  David Vasko

IPC: H04N7/18 ,  G05B23/02 ,  G06T19/00 ,  H04N5/232 ,  H04N13/332 ,  H04N13/111 ,  H04N13/366 ,  H04N13/398 ,  G06F3/04817 ,  G06F3/0482 ,  G06K9/00 ,  H04N5/247

Abstract: An industrial visualization system generates and delivers virtual reality (VR) and augmented reality (AR) presentations of industrial facilities to wearable appliances to facilitate remote or enhanced interaction with automation systems within the facility. VR presentations can comprise three-dimensional (3D) holographic views of a plant facility or a location within a plant facility. The system can selectively render a scaled down view that renders the facility as a 3D scale model, or as a first-person view that renders the facility as a full-scale rendition that simulates the user's presence on the plant floor. Camera icons rendered in the VR presentation can be selected to switch to a live video stream generated by 360-degree cameras within the plant. The system can also render workflow presentations that guide users through the process of correcting detected maintenance issues.

公开(公告)号：US20210342388A1

公开(公告)日：2021-11-04

申请号：US16863768

申请日：2020-04-30

Applicant: Rockwell Automation Technologies, Inc.

Inventor： Thong T. Nguyen ,  Paul D. Schmirler ,  Hannah M. Schermerhorn ,  Kyle Crum ,  Abhishek Mehrotra ,  Kristopher J. Holley

IPC: G06F16/538 ,  G06F16/55

Abstract: A method executable by at least one processor includes receiving image data representative of an industrial equipment assembly, identifying properties associated with the industrial equipment assembly based on the image data, identifying a set of industrial equipment assemblies associated with the industrial equipment assembly based on the properties associated with the industrial equipment assembly and data stored in a database, and categorizing the set of industrial equipment assemblies based on the data associated with the industrial equipment assemblies. The method also includes generating an inquiry based on the categorization of the set of industrial equipment assemblies, presenting the inquiry via an electronic display, receiving information responsive to the inquiry and associated with the industrial equipment assembly, identifying a subset of industrial equipment assemblies based on the information, and presenting a visualization associated with the subset of industrial equipment assemblies via the electronic display.

公开(公告)号：US20210181856A1

公开(公告)日：2021-06-17

申请号：US17184254

申请日：2021-02-24

Applicant: Rockwell Automation Technologies, Inc.

Inventor： Thong T. Nguyen ,  Paul D. Schmirler ,  Timothy T. Duffy

IPC: G06F3/01 ,  G06F3/0481 ,  G06F3/0484 ,  G06T13/20 ,  G06T19/00 ,  G06F1/16 ,  G06F3/0482

Abstract: A method may include receiving, via a processor, image data associated with a user's surrounding and generating, via the processor, a visualization that may include a virtual industrial automation device. The virtual industrial automation device may depict a virtual object within image data, and the virtual object may correspond to a physical industrial automation device. The method may include displaying, via the processor, the visualization via an electronic display and detecting, via the processor, a gesture in image data that may include the user's surrounding and the visualization. The gesture may be indicative of a request to move the virtual industrial automation device. The method may include tracking, via the processor, a user's movement, generating, via the processor, a visualization that may include an animation of the virtual industrial automation device moving based on the user's movement, and displaying, via the processor, the visualization via the electronic display.

公开(公告)号：US11003868B2

公开(公告)日：2021-05-11

申请号：US16738212

申请日：2020-01-09

Applicant: Rockwell Automation Technologies, Inc.

Inventor： Thong T. Nguyen ,  Alex L. Nicoll ,  Calvin C. Steinweg ,  Paul D. Schmirler

IPC: G06K7/08 ,  G06K7/10 ,  G06Q10/08 ,  G06K7/14

Abstract: For filtering display data, a method receives reference data for an equipment instance. The method converts the reference data to display data. The method determines a user action. The method filters the display data based on the user action. The method presents the display data.

公开(公告)号：US20210072819A1

公开(公告)日：2021-03-11

申请号：US16953016

申请日：2020-11-19

Applicant: Rockwell Automation Technologies, Inc.

Inventor： Thong T. Nguyen ,  John M. Van Hecke ,  Alex L. Nicoll ,  Paul D. Schmirler ,  Mark Bjerke

IPC: G06F3/01 ,  G06T11/60 ,  G09B9/00 ,  G09B19/00 ,  G06F3/03 ,  G06T11/00

Abstract: Techniques for employing augmented reality or virtual reality information are presented. An information management component (IMC) of an augmented reality device (ARD) can monitor and detect user activities and conditions in area in proximity to ARD. Based on user activities and conditions, IMC can determine augmented reality information that can enhance user experience, performance of user activities, or security and safety of user. IMC can present, via an interface component of ARD, the augmented reality information to the user. The augmented reality information can relate to user location; navigation by the user; tasks to be performed by the user; product assembly; maintenance work; system or product design or configuration; remote control of assembly, maintenance, design, or configuration; environmental and/or hazardous conditions; security, identification, and authentication of users; or training the user to perform tasks. IMC can translate information from a language to a different language of the user.

公开(公告)号：US10832548B2

公开(公告)日：2020-11-10

申请号：US16398495

申请日：2019-04-30

Applicant: Rockwell Automation Technologies, Inc.

Inventor： Thong T. Nguyen ,  Paul D. Schmirler ,  Kyle Crum ,  David A. Vasko

IPC: G08B21/00 ,  G08B21/02

Abstract: An industrial visualization system defines and enforces a virtual safety shield comprising a three-dimensional space surrounding a wearer of a client device. The dimensions of the virtual safety shield are defined by a specified safe distance surrounding the user that allows sufficient reaction time in response to notification that the wearer is at risk of interacting with a safety zone, hazardous machinery, or vehicles within the plant. If a boundary of a safety zone or hazardous equipment falls within the three-dimensional space defined by the virtual safety shield, the system sends a notification to the user's client device, or places the hazardous equipment in a safe operating mode.

公开(公告)号：US20200336706A1

公开(公告)日：2020-10-22

申请号：US16915152

申请日：2020-06-29

Applicant: Rockwell Automation Technologies, Inc.

Inventor： Paul D. Schmirler ,  Thong T. Nguyen ,  Alex L. Nicoll ,  David Vasko

IPC: H04N7/18 ,  G05B23/02 ,  G06T19/00 ,  H04N5/232

Abstract: An industrial visualization system generates and delivers virtual reality (VR) and augmented reality (AR) presentations of industrial facilities to wearable appliances to facilitate remote or enhanced interaction with automation systems within the facility. VR presentations can comprise three-dimensional (3D) holographic views of a plant facility or a location within a plant facility. The system can selectively render a scaled down view that renders the facility as a 3D scale model, or as a first-person view that renders the facility as a full-scale rendition that simulates the user's presence on the plant floor. Camera icons rendered in the VR presentation can be selected to switch to a live video stream generated by 360-degree cameras within the plant. The system can also render workflow presentations that guide users through the process of correcting detected maintenance issues.

公开(公告)号：US20200273243A1

公开(公告)日：2020-08-27

申请号：US16287828

申请日：2019-02-27

Applicant: Rockwell Automation Technologies, Inc.

Inventor： Timothy T. Duffy ,  Paul D. Schmirler ,  Thong T. Nguyen

IPC: G06T17/10 ,  G06K9/00 ,  G06T7/80 ,  G06F17/50

Abstract: A system for monitoring an industrial system may have a computing device that generates a first three-dimensional (3D) visualization based on 3D video associated with an industrial system. The first 3D visualization may have one or more 3D device visualizations of industrial automation devices, and each 3D device visualization of the industrial automation devices that may depict a 3D object that corresponds to an industrial automation device in the industrial system. The computing device may display the first 3D visualization via an electronic display and detect a gesture in a first set of image data comprising the surrounding of the user and the first 3D visualization. The gesture is indicative of a modification associated with the first 3D visualization. The computing device may receive additional 3D video associated with the industrial system, generate a second 3D visualization based on the modification and the additional 3D video, and display the second 3D visualization via the electronic display.