This study explores the potential application of hyperspectral imaging (HSI; 430–1,010 nm) coupled with different linear and nonlinear models for rapid nondestructive evaluation of thiobarbituric acid‐reactive substances (TBARS) value in rainbow trout ( Oncorhynchus mykiss) fillets during 12 days of cold storage (4 ± 2°C). HSI data and TBARS value of fillets were obtained in the laboratory. The primary prediction models were established based on linear partial least squares regression (PLSR) and least squares support vector machine (LS‐SVM). In full spectral range, the prediction capability of LS‐SVM ( = 0.829; RMSEP = 0.128 mg malondialdehyde [MDA]/kg) was better than PLSR ( = 0.748; RMSEP = 0.155 mg MDA/kg) model and LS‐SVM model exhibited satisfactory prediction performance ( > 0.82). To simplify the calibration models, a combination of uninformative variable elimination and backward regression (UB) was used as variable selection. Nine wavelengths were selected. Various chemometric analysis methods including linear PLSR and multiple linear regression and nonlinear LS‐SVM and back‐propagation artificial neural network (BP‐ANN) were compared. The simplified models showed better capability than those were built based on the whole dataset in prediction of TBARS values. Moreover, the nonlinear models were preferred over linear models. Among the four chemometric algorithms, the best and weakest models were LS‐SVM and PLSR model, respectively. UB‐LS‐SVM model was the optimal models for predicting TBARS value in rainbow trout fillets ( = 0.831; RMSEP = 0.125 mg MDA/kg). The establishing of lipid‐oxidation prediction model in rainbow trout fish was complicated, due to the fluctuations of TBARS values during storage. Therefore, further researches are needed to improve the prediction results and applicability of HIS technique for prediction of TBARS value in rainbow trout fish.