In a method of manufacturing a building material, three-dimensional data on a surface pattern selected are obtained and stored, an image based on two-dimensional data of the three-dimensional data is displayed on a screen, and shaped, and an image which is obtained by adding height data of the three-dimensional data to the image thus shaped is displayed on the screen, and evaluated, pieces extracted from the image thus evaluated are shaped and arranged at predetermined positions, the height data of the pieces thus arranged is cut to a predetermined value thereby to process a miniature size model image, the miniature size model image is displayed on the screen, and evaluated, the miniature size model selected through evaluation is arranged at predetermined positions to process a full size model image, a building material model is formed according to the three-dimensional data of the full size model, and evaluated, a metal mold is formed according to three-dimensional data corresponding to the building material model thus evaluated, and a building material is formed by use of the metal mold.