The present invention provides a lens assembly structure of swimming goggles. The lens assembly structure comprises two lenses, two retaining walls, two skirt portions, and two frame portions. Each of the lenses includes an eye side and a water side opposing to the eye side. The two retaining walls are provided on the eye sides of the lenses, respectively, and each of the retaining walls defines a visible area and a connection area. Each of the retaining walls has a configuration, and the corresponding skirt portion is disposed along the configuration. The connection area of the lens is affixed in the corresponding frame portion. The configuration of each of the retaining walls is a curve having a contour which is less complicated than a peripheral configuration of the lens.