SCHEDULING OF INFERENCE MODELS BASED ON PREEMPTABLE BOUNDARIES

    公开(公告)号:US20230252328A1

    公开(公告)日:2023-08-10

    申请号:US18153764

    申请日:2023-01-12

    CPC classification number: G06N5/048 G06F9/4818

    Abstract: Disclosed herein are systems and methods for inference model scheduling of a multi priority inference model system. A processor determines an interrupt flag has been set indicative of a request to interrupt execution of a first inference model in favor of a second inference model. In response to determining that the interrupt flag has been set, the processor determines a state of the execution of the first inference model based on one or more factors. In response to determining the state of the execution is at a preemptable boundary, the processor deactivates the first inference model and activates the second inference model.

    METHODS OF BATCH-BASED DNN PROCESSING FOR EFFICIENT ANALYTICS

    公开(公告)号:US20240046413A1

    公开(公告)日:2024-02-08

    申请号:US18175185

    申请日:2023-02-27

    CPC classification number: G06T3/4046

    Abstract: Technology is disclosed herein to execute an inference model by a processor which includes a reshape layer. In an implementation, the reshape layer of the inference model receives an output produced by a previous layer of the inference model and inserts padding into the output, then supplies the padded output as an input to a next layer of the inference model. In an implementation, the inference model includes a stitching layer at the beginning of the inference model and an un-stitch layer at the end of the model. The stitching layer of the inference model stitches together multiple input images into an image batch and supplies the image batch as an input to a subsequent layer. The un-stitch layer receives output from a penultimate layer of the inference model and unstitches the output to produce multiple output images corresponding to the multiple input images.

Patent Agency Ranking