The invention discloses a signal processing method based on a multi-view image sensor. The method includes A) acquiring a plurality of images using a plurality of groups of stereo image sensors, wherein each group has at least two cameras and each image has at least two view points; B) performing the parallel processing on the images acquired by the stereo image sensors to fuse the signals, and calculating the point parallax and the depth information of the captured space based on the fused pixels; and C) establishing the stereo image point parallax matrix by the point parallax information and converting the 3D depth coordinate information of the photographed space by the point parallax matrix. The plurality of images, each having at least two view points, are acquired by the plurality of groups of stereo image sensors, each group has at least two cameras, the acquired images are subjected to fusion and conversion, the 3D depth coordinate information of the photographed space is accurately restored, and the method can be applied to the automobile automatic road driving condition recognition, multi-target obstacle detection and tracking, autonomous navigation robots, unmanned aerial vehicles, geo-mapping, 3D space environment detection and reconstruction.