A method of remapping memory, which is suitable for a server without a video graphics array (VGA), is provided. In the invention, a video memory buffer block is set in a system memory. First, a power on self test (POST) is executed for initializing the system memory. Next, a remap function is enabled. Then, a base address and a size of the video memory buffer block are set into a remap register of a chipset. Finally, the video memory buffer block is remapped into a memory address space originally mapped with a system management mode block according to the remap register.