A linear eddy viscosity model and a non-linear eddy viscosity model were used to simulate a centrifugal pump at an off-design point. Compared with the inner flow of experimental results, turbulence model was accurate in the calculation of the rotating stall phenomenon in the runner of low flow conditions. The turbulence model and three dimensional (3-D), unsteady flow in a pump-turbine was used to study the instability of a high-head pump-turbine at no-load opening. Fluid coupling and dynamic mesh were used to simulate the change of runner's rotational speed. Stall phenomena in the runner caused by a large incidence angle of flow at the region between stay vanes and guide vanes were analyzed. Calculations based on different moments of inertia were accomplished in order to investigate the influence of moment of inertia to the stall phenomena in the runner. The Rayleigh criterion was introduced to analyze the instability of the stall phenomena in the runner. The explicit characteristics such as the flow-rate, rotational speed, torque of the runner etc. were analyzed. Results show that the moment of inertia has great influence on the stall phenomena in the runner. The Rayleigh criterion can be used to evaluate the instability of the pump-turbine at no-load opening. The flow in the pump-turbine at runaway speed is more stable when the moment of inertia increases. The study of a pump-turbine at no-load opening can provide a basic foundation for the improvement of S characteristics.