A cushion robot with omnidirectional wheels is developed to support in-room movement of the elderly. Because the cushion robot works in a narrow, complex surrounding, it must accurately follow the target direction to avoid repetitively switching drives and to guarantee user safety. However, the user seated on the cushion robot causes a load change and a shift of the center of gravity (COG) of the whole system, which decreases the path tracking precision. To address this issue, an adaptive controller is designed to enable the cushion robot to accurately follow the target direction. The adaptive law can estimate the load and the position of the COG shift on-line, and can control the system so that it accurately follows a reference path generated to reach the target. Simulations and experiments are executed under the no-load condition and the load condition. The simulation and experimental results demonstrate that this control scheme effectively deals with the load change and the COG shift. Therefore, the cushion robot is reliable and convenient for assisting in-room movement of the elderly.