A method of fabricating a refrigerator cabinet or door includes forming a wrapper and an inner liner. The method further includes forming a vacuum insulated core comprising a permeable core material that is disposed inside an impermeable envelope. A sheet of prefabricated compressible foam material is positioned between the vacuum insulated core between the inner door liner and/or the door wrapper. The prefabricated compressible foam material may be cut from a sheet of foam having substantially uniform thickness prior to fabrication of the refrigerator door. The foam compresses to accommodate differences in spacing between the vacuum insulated core and the door wrapper and/or the door liner.