There is provided an operation method of a memory system which includes a host and a storage device, and the operation method includes allocating a portion of a host memory included in the host for a host memory buffer to be used by a controller of the storage device, setting a set feature command such that the host memory buffer is enabled, setting a retention command including information about a response speed of the host memory buffer, selecting an operation mode of the host memory buffer, based on the retention command, and selecting one of a plurality of power states, which the controller supports, based on a performance objective of the operation mode of the host memory buffer.