A vehicular vision system includes a side-viewing camera mounted within an exterior rearview mirror assembly attached at a side of a vehicle equipped with the vehicular vision system. The side-viewing camera has a field of view at least sideward of the side of the equipped vehicle at which the exterior rearview mirror assembly is attached. The side-viewing camera captures an image of a scene occurring exterior of the equipped vehicle. The captured image includes an image data set representative of the exterior scene. A control includes an image processor, and the image data set is provided to the control. The control processes a reduced image data set of the image data set provided to the control to detect edges present exterior of the equipped vehicle within an area of interest of the scene occurring exterior of the equipped vehicle that is within the field of view of the side-viewing camera.