Systems, methods, and computer-executable instructions for scheduling neural network workloads on an edge device. A performance model for each neural network model is received. Parameters for each neural network workload is determined based on an associated performance model. Processing core assignments are determined from a plurality of processing cores for each neural network workload based on the corresponding performance model and processing core utilization. Image streams are received and associated with a neural network workload. Each neural network workload is scheduled to run on the processing cores based on the processing core assignments.