A lens unit including a lens barrel that contains a lens inside and has a rectangular image pickup window through which an optical axis passes, and an image pickup element unit including an image pickup element and an element attachment plate having the image pickup element mounted on one surface. The image pickup element unit is attached to the lens barrel by a plurality of screws such that the image pickup window and the image pickup element are opposite to each other. A shim unit is disposed between the lens barrel and the element attachment plate. The shim unit is made of a shim base attached to a circumferential edge of the image pickup window and having a shape along at least two sides of the image pickup window, and one or a plurality of shims mounted at one or a plurality of portions of the shim base, respectively.