Abstract:
A computer-implemented method for training a deep neural network to recognize traffic scenes (TSs) from multi-modal sensors and knowledge data is presented. The computer-implemented method includes receiving data from the multi-modal sensors and the knowledge data and extracting feature maps from the multi-modal sensors and the knowledge data by using a traffic participant (TS) extractor to generate a first set of data, using a static objects extractor to generate a second set of data, and using an additional information extractor. The computer-implemented method further includes training the deep neural network, with training data, to recognize the TSs from a viewpoint of a vehicle.
Abstract:
A video device for predicting driving situations while a person drives a car is presented. The video device includes multi-modal sensors and knowledge data for extracting feature maps, a deep neural network trained with training data to recognize real-time traffic scenes (TSs) from a viewpoint of the car, and a user interface (UI) for displaying the real-time TSs. The real-time TSs are compared to predetermined TSs to predict the driving situations. The video device can be a video camera. The video camera can be mounted to a windshield of the car. Alternatively, the video camera can be incorporated into the dashboard or console area of the car. The video camera can calculate speed, velocity, type, and/or position information related to other cars within the real-time TS. The video camera can also include warning indicators, such as light emitting diodes (LEDs) that emit different colors for the different driving situations.
Abstract:
A system and method for a motorized land vehicle that detects objects obstructing a driver's view of an active road, includes an inertial measurement unit-enabled global position system (GPS/IMU) subsystem for obtaining global position system (GPS) position and heading data of a land vehicle operated by the driver as the vehicle travels along a road, a street map subsystem for obtaining street map data of the GPS position of the vehicle using the GPS position and heading data as the vehicle travels along the road, and a three-dimensional (3D) object detector subsystem for detecting objects ahead of the vehicle and determining a 3D position and 3D size data of each of the detected objects ahead of the vehicle. The street map subsystem merges the street map data, the GPS position and heading data of the vehicle and the 3D position data and 3D size data of the detected objects, to create real-time two-dimensional (2D) top-view map representation of a traffic scene ahead of the vehicle. The street map subsystems finds active roads ahead of the vehicle in the traffic scene, and finds each active road segment of the active roads ahead of the vehicle that is obstructed by one of the detected objects. A driver alert subsystem notifies a driver of the vehicle of each of the active road segments that is obstructed by one of the detected objects.
Abstract:
A computer-implemented method for learning disentangled representations for T-cell receptors to improve immunotherapy is provided. The method includes optionally introducing a minimal number of mutations to a T-cell receptor (TCR) sequence to enable the TCR sequence to bind to a peptide, using a disentangled Wasserstein autoencoder to separate an embedding space of the TCR sequence into functional embeddings and structural embeddings, feeding the functional embeddings and the structural embeddings to a long short-term memory (LSTM) or transformer decoder, using an auxiliary classifier to predict a probability of a positive binding label from the functional embeddings and the peptide, and generating new TCR sequences with enhanced binding affinity for immunotherapy to target a particular virus or tumor.
Abstract:
A system for binding peptide search for immunotherapy is presented. The system includes employing a deep neural network to predict a peptide presentation given Major Histocompatibility Complex allele sequences and peptide sequences, training a Variational Autoencoder (VAE) to reconstruct peptides by converting the peptide sequences into continuous embedding vectors, running a Monte Carlo Tree Search to generate a first set of positive peptide vaccine candidates, running a Bayesian Optimization search with the trained VAE and a Backpropagation search with the trained VAE to generate a second set of positive peptide vaccine candidates, using a sampling from a Position Weight Matrix (sPWM) to generate a third set of positive peptide vaccine candidates, screening and merging the first, second, and third sets of positive peptide vaccine candidates, and outputting qualified peptides for immunotherapy from the screened and merged sets of positive peptide vaccine candidates.
Abstract:
A method for implementing deep reinforcement learning with T-cell receptor (TCR) mutation policies to generate binding TCRs for immunotherapy includes extracting peptides to identify a virus or tumor cells, collecting a library of TCRs from patients, predicting interaction scores between the extracted peptides and the TCRs from the patients, developing a deep reinforcement learning framework with TCR mutation policies to generate TCRs with maximum binding scores, defining reward functions, outputting mutated TCRs, ranking the outputted TCRs to utilize top-ranked TCR candidates to target the virus or the tumor cells, and for each top-ranked TCR candidate, repeatedly identifying a set of self-peptides that the top-ranked TCR candidate binds to and further optimizing it greedily by maximizing a sum of its interaction scores with a given set of peptide antigens while minimizing a sum of its interaction scores with the set of self-peptides until stopping criteria of efficacy and safety are met.
Abstract:
A method for using a multi-hop reasoning framework to perform multi-step compositional long-term reasoning is presented. The method includes extracting feature maps and frame-level representations from a video stream by using a convolutional neural network (CNN), performing object representation learning and detection, linking objects through time via tracking to generate object tracks and image feature tracks, feeding the object tracks and the image feature tracks to a multi-hop transformer that hops over frames in the video stream while concurrently attending to one or more of the objects in the video stream until the multi-hop transformer arrives at a correct answer, and employing video representation learning and recognition from the objects and image context to locate a target object within the video stream.
Abstract:
A method for implementing deep reinforcement learning with T-cell receptor (TCR) mutation policies to generate binding TCRs recognizing target peptides for immunotherapy is presented. The method includes extracting peptides to identify a virus or tumor cells, collecting a library of TCRs from target patients, predicting, by a deep neural network, interaction scores between the extracted peptides and the TCRs from the target patients, developing a deep reinforcement learning (DRL) framework with TCR mutation policies to generate TCRs with maximum binding scores, defining reward functions based on a reconstruction-based score and a density estimation-based score, randomly sampling batches of TCRs and following a policy network to mutate the TCRs, outputting mutated TCRs, and ranking the outputted TCRs to utilize top-ranked TCR candidates to target the virus or the tumor cells for immunotherapy.
Abstract:
A computer-implemented method is provided for action localization. The method includes converting one or more video frames into person keypoints and object keypoints. The method further includes embedding position, timestamp, instance, and type information with the person keypoints and object keypoints to obtain keypoint embeddings. The method also includes predicting, by a hierarchical transformer encoder using the keypoint embeddings, human actions and bounding box information of when and where the human actions occur in the one or more video frames.
Abstract:
A method is provided for peptide-based vaccine generation. The method receives a dataset of positive and negative binding peptide sequences. The method pre-trains a set of peptide binding property predictors on the dataset to generate training data. The method trains a Wasserstein Generative Adversarial Network (WGAN) only on the positive binding peptide sequences, in which a discriminator of the WGAN is updated to distinguish generated peptide sequences from sampled positive peptide sequences from the training data, and a generator of the WGAN is updated to fool the discriminator. The method trains the WGAN only on the positive binding peptide sequences while simultaneously updating the generator to minimize a kernel Maximum Mean Discrepancy (MMD) loss between the generated peptide sequences and the sampled peptide sequences and maximize prediction accuracies of a set of pre-trained peptide binding property predictors with parameters of the set of pre-trained peptide binding property predictors being fixed.