公开(公告)号：US20230154218A1

公开(公告)日：2023-05-18

申请号：US17454729

申请日：2021-11-12

Applicant: Microsoft Technology Licensing, LLC

Inventor： Jenna HONG ,  Apurva Sandeep GANDHI ,  Gilbert ANTONIUS ,  Tra My NGUYEN ,  Ryan SERRAO ,  Biyi FANG ,  Sheng YI

IPC: G06K9/00 ,  G06F40/284 ,  G06K9/20

CPC classification number: G06K9/00422 ,  G06F40/284 ,  G06K9/00449 ,  G06K9/2054 ,  G06K9/00409

Abstract: A computer system is provided that includes one or more processors configured to receive user input for inked content to a digital canvas, and process the inked content to determine one or more writing regions. Each writing region includes recognized text and one or more document layout features associated with that writing region. The one or more processors are further configured to tokenize a target writing region of the one or more writing regions into a sequence of tokens, process the sequence of tokens of the target writing region using task extraction subsystem that operates on tokens representing both the recognized text and the one or more document layout features of the target writing region, segment the target writing region into one or more sentence segments, and classify each of the one or more sentence segments as a task sentence or a non-task sentence.

公开(公告)号：US20220156486A1

公开(公告)日：2022-05-19

申请号：US17098285

申请日：2020-11-13

Applicant: Microsoft Technology Licensing, LLC

Inventor： Tianyi CHEN ,  Gowtham GANESAN ,  Biyi FANG ,  Sheng YI

IPC: G06K9/00 ,  G06K9/72 ,  G06T7/10 ,  G06K9/62 ,  G06K9/46 ,  G06N3/04 ,  G06N3/08

Abstract: A data processing system for performing a semantic analysis of digital ink stroke data implements obtaining the digital ink stroke data representing handwritten text, drawings, or both; analyzing the digital ink stroke data to extract path signature feature information from the digital ink stroke data; analyzing the path signature feature information using a convolutional neural network (CNN) trained to perform a pixel-level sematic analysis of the digital ink stroke data and to output a pixel segmentation map with semantic prediction information for each pixel of digital ink stroke data; analyzing the pixel segmentation map to generate stroke-level semantic information using a pixel-to-stroke conversion model; and processing the digital ink stroke data based on the stroke-level semantic information.

公开(公告)号：US20200293770A1

公开(公告)日：2020-09-17

申请号：US16355702

申请日：2019-03-15

Applicant: Microsoft Technology Licensing, LLC

Inventor： Tianyi CHEN ,  Yixin SHI ,  Sheng YI

IPC: G06K9/00 ,  G06N3/04 ,  G06F17/15

Abstract: A spatially sparse convolutional neural network (CNN) framework is introduced to that leverages high sparsity of input data to significantly reduce the computational cost of applications that employ CNNs (e.g., inking applications and others) by avoiding unnecessary floating point mathematical operations. The framework, which is compatible with parallelized operations, includes (1) a data structure for sparse tensors that both (a) reduces storage burden and (b) speeds computations; (2) a set of sparse tensor operations that accelerate convolution computations; and (3) the merging of pooling and convolutional layers. Practical applications involving handwriting recognition and/or stroke analysis demonstrate a notable reduction in storage and computational burdens.

公开(公告)号：US20240378915A1

公开(公告)日：2024-11-14

申请号：US18784076

申请日：2024-07-25

Applicant: Microsoft Technology Licensing, LLC

Inventor： Jenna HONG ,  Apurva Sandeep GANDHI ,  Gilbert ANTONIUS ,  Tra My NGUYEN ,  Ryan SERRAO ,  Biyi FANG ,  Sheng YI

IPC: G06V30/32 ,  G06F40/284 ,  G06V10/22 ,  G06V30/412

Abstract: A computer system is provided that includes one or more processors configured to receive user input for inked content to a digital canvas, and process the inked content to determine one or more writing regions. Each writing region includes recognized text and one or more document layout features associated with that writing region. The one or more processors are further configured to tokenize a target writing region of the one or more writing regions into a sequence of tokens, process the sequence of tokens of the target writing region using a task extraction subsystem that operates on tokens representing both the recognized text and the one or more document layout features of the target writing region, segment the target writing region into one or more sentence segments, and classify each of the one or more sentence segments as a task sentence or a non-task sentence.

公开(公告)号：US20230419111A1

公开(公告)日：2023-12-28

申请号：US18458709

申请日：2023-08-30

Applicant: Microsoft Technology Licensing, LLC

Inventor： Tianyi CHEN ,  Sheng YI ,  Yixin SHI ,  Xiao TU

IPC: G06N3/08 ,  G06N3/04

CPC classification number: G06N3/08 ,  G06N3/04

Abstract: The performance of a neural network (NN) and/or deep neural network (DNN) can limited by the number of operations being performed as well as management of data among the various memory components of the NN/DNN. A sparsity-inducing regularization optimization process is performed on a machine learning model to generate a compressed machine learning model. A machine learning model is trained using a first set of training data. A sparsity-inducing regularization optimization process is executed on the machine learning model. Based on the sparsity-inducing regularization optimization process, a compressed machine learning model is received. The compressed machine learning model is executed to generate one or more outputs.

公开(公告)号：US20220392265A1

公开(公告)日：2022-12-08

申请号：US17820036

申请日：2022-08-16

Applicant: Microsoft Technology Licensing, LLC

Inventor： Tianyi CHEN ,  Sheng YI

IPC: G06V40/30 ,  G06K9/62 ,  G06N3/04 ,  H04L9/08

Abstract: Methods, systems, and computer program products are provided for signature verification. Signature verification may be provided for target signatures using genuine signatures. A signature verification model pipeline may extract features from a target signature and a genuine signature, encode and submit both to a neural network to generate a similarity score, which may be repeated for each genuine signature. A target signature may be classified as genuine, for example, when one or more similarity scores exceed a genuine threshold. A signature verification model may be updated or calibrated at any time with new genuine signatures. A signature verification model may be implemented with multiple trainable neural networks (e.g., for feature extraction, transformation, encoding, and/or classification).

公开(公告)号：US20210390384A1

公开(公告)日：2021-12-16

申请号：US16889775

申请日：2020-06-01

Applicant: MICROSOFT TECHNOLOGY LICENSING, LLC

Inventor： Tianyi CHEN ,  Sheng YI ,  Yixin SHI ,  Xiao TU

IPC: G06N3/08 ,  G06N3/04

Abstract: The performance of a neural network (NN) and/or deep neural network (DNN) can limited by the number of operations being performed as well as management of data among the various memory components of the NN/DNN. A sparsity-inducing regularization optimization process is performed on a machine learning model to generate a compressed machine learning model. A machine learning model is trained using a first set of training data. A sparsity-inducing regularization optimization process is executed on the machine learning model. Based on the sparsity-inducing regularization optimization process, a compressed machine learning model is received. The compressed machine learning model is executed to generate one or more outputs.

公开(公告)号：US20230143969A1

公开(公告)日：2023-05-11

申请号：US17522608

申请日：2021-11-09

Applicant: Microsoft Technology Licensing, LLC

Inventor： Biyi FANG ,  Sheng YI ,  Tianyi CHEN

IPC: G06K9/00 ,  G06K9/46

CPC classification number: G06K9/00416 ,  G06K9/00422 ,  G06K9/00442 ,  G06K9/4638

Abstract: Systems and methods for performing spatial line grouping on digital ink stokes. The system includes an electronic processor configured to access a set of hypothetical lines in an electronic document and determine a set of hypothetical line pairings. The electronic processor is also configured to determine, via a gradient boosting tree model, a merge confidence score for each hypothetical line pairing and compare a first merge confidence score with a merge threshold. The first merge confidence score is associated with a first hypothetical line and a first neighboring hypothetical line. The electronic processor is also configured to, in response to the first merge confidence score satisfying the merge threshold, merge the first hypothetical line and the first neighboring hypothetical line to form a first line grouping. The electronic processor is also configured to perform a digital ink stroke analysis on the electronic document based on the first line grouping.

公开(公告)号：US20200302163A1

公开(公告)日：2020-09-24

申请号：US16360006

申请日：2019-03-20

Applicant: Microsoft Technology Licensing, LLC

Inventor： Yixin SHI ,  Aman ORAZAEV ,  Tianyi CHEN ,  Sheng YI

IPC: G06K9/00 ,  G06T7/11 ,  G06N3/04

Abstract: An ink parsing system receives ink strokes at an inking device input and render the received ink strokes into an image in a pixel space. Writing strokes are detected in the image and labeled. Pixels corresponding to the labeled writing strokes are removed from the image. Drawing strokes in the image having the removed pixels are detected using and labeled. Writing objects and drawing objects corresponding, respectively, to the labeled writing strokes and the labeled drawing strokes are output. A digital ink parsing pipeline is thereby provided having accurate ink stroke detection and segmentation.

公开(公告)号：US20190340227A1

公开(公告)日：2019-11-07

申请号：US15970186

申请日：2018-05-03

Applicant: Microsoft Technology Licensing, LLC

Inventor： Yibo SUN ,  Sheng YI ,  Xiao TU

IPC: G06F17/24 ,  G06F17/27 ,  G06F17/21 ,  G06F3/0488 ,  G06F3/0484

Abstract: Described herein is a system and method for visually enhancing digital ink of an electronic document. A trigger to visually enhance digital ink of portion(s) of the electronic document is received. In response to the received trigger, the digital ink of portion(s) of the electronic document to determine a semantic structure of the digital ink in response to the received trigger. The digital ink of the portion(s) of the electronic document are visually enhanced in accordance with the determined semantic structure. Visual enhancement can include horizontal line adjustment, aligning line(s), aligning word in a particular line using a baseline, adjusting vertical spacing of lines, paragraphs, and/or lists, adjusting spacing between words and/or list items in a particular line, modifying ink styling (e.g., ink size, ink thickness, ink color), adjusting sizing of characters in a same group, unifying ink color, and/or unifying ink thickness.