Techniques are disclosed for implementing a convolutional neural network that determines an offset field for deforming a kernel to be used in a convolution. The offset field is temporally-based, at least in part, on data generated at an earlier time. Furthermore, techniques are disclosed for using sensor data to train a neural network to learn shapes or configurations of such deformed kernels. The temporal-based deformable convolutions may be used for object identification, object matching, object classification, segmentation, and/or object tracking, in various examples.