A video management system (VMS) may search for one or more objects and/or events in one or more video streams, and may receive time-stamped metadata that may identify one or more objects and/or events occurring in the corresponding video stream as well as an identifier that uniquely identifies the corresponding video stream. A user may enter a query into a video query engine, wherein the video query engine includes one or more cognitive models. The VMS may apply the search query to the time-stamped metadata via the video query engine to search for one or more objects and/or events in the one or more video streams that match the search query, and returning a search result to the user.