北京生物医学工程

计算机辅助诊断模型内部验证方法的定量评价

Quantitative evaluation of internal validation methods for computer-aided diagnosis scheme

作者：陈婕卿杨秋英陈卉

单位：首都医科大学生物医学工程学院(北京100069)

分类号：R318;R576;TP311

出版年·卷·期（页码）：0·0·0（0-0）

摘要：

目的定量比较4种常用的内部验证方法，为评价计算机辅助诊断模型性能时选择验证方法提供参考依据。方法利用Logistic回归模型完成大样本集（n=415）和小样本集（n=76）下的胰腺癌诊断任务，分别采用保持法、k折交叉验证法、留一法和0.632 Bootstrap法共4种内部验证方法，并用诊断的正确率、敏感度、特异度和ROC曲线下面积评价诊断的稳定性、偏倚和运算效率。结果对大、小样本集，0.632 Bootstrap验证方法得到的正确率、敏感度、特异度和ROC曲线下面积的标准误分别为0.012、0.014、0.010、0.010以及0.013、0.014、0.010、0.011，均小于其他验证方法，其他方法均不同程度地高估或低估模型性能。结论考虑验证的简洁有效性，k折交叉验证法在大样本量的情况下即可达到内部验证的最佳效果，在小样本量情况下推荐使用0.632 Bootstrap进行验证。

Objective To quantitatively compare four commonly used methods in order to provide reference on the selection of internal validation methods for evaluating a computer-aided diagnosis model. Methods Logistic regression model was used for a diagnostic task on pancreatic cancer datasets with small and large sample sizes (76 and 415, respectively). Four internal validation methods, hold-out, leave-one-out, k-fold cross validation and 0.632 Bootstrap, were used and compared. Diagnosis model stability, bias and efficiency were measured by accuracy, sensitivity, specificity and area under the ROC curve. Results 0.632 Bootstrap validation method was with the minimum standard errors of accuracy, sensitivity, specificity and area under the ROC curve on both large-and small-size datasets, i.e. 0.012, 0.014, 0.010, 0.010, and 0.013, 0.014, 0.010, 0.011, respectively. Other methods underestimated or overestimated the model performance to certain degree. Conclusions Considering the simplicity and effectiveness of these validation methods, it is recommended that k-fold cross validation is preferable on the relative large-size dataset and 0.632 Bootstrap method on the small one.

参考文献：

［1］Gallego C, Martel AL. Improving the accuracy of computer-aided diagnosis for breast MR imaging by differentiating between mass and monmas lesions［J］. Radiology, 2016, 278(3):1266-1271.

［2］杨柳，王钰．泛化误差的各种交叉验证估计方法综述［J］．计算机应用研究，2015，32(5): 1287-1297.

Yang Liu, Wang Yu. Survey for various cross-validation estimators of generalization error［J］. Application Research of Computers, 2015, 32(5):1287-1297.

［3］覃礼堂，柳树深，肖乾芬，等．QSAR模型内部和外部验证方法综述［J］．环境化学，2013，32(7): 1205-1211.

Qin Litang, Liu Shushen, Xiao Qianfen, et al. Internal and external validations of QSAR model: review［J］. Application Research of Computers, 2013, 32(7):1205-1211.

［4］Chirico N, Gramatica P. Real external prodictivity of QSAR Models. Part2. New intercomparable thresholds for different validation criteria and the need for scatter plot inspection［J］. Journal of Chemical Information and Modeling, 2012, 52(8):2044-2058.

［5］Li Jiewang, Bai Li, Su Dan, et al. Retrospective analysis of 102 cases of solid pseudo papillary neoplasm of the pancreas in China ［J］. Journal of International Medical Research, 2013, 41(4):1266-1271.

［6］Uesawa Y, Motege E, Dai Y, et al. Prediction models for feverishness developed during interferon therapy of chronic hepatitis C patients ［J］. Pharmazie, 2010, 65(2):114-126.

［7］Sahiner B, Chan HP, Hadjiiski L. Classifier performance prediction for computer-aided diagnosis using a limited dataset［J］. Medical Physics, 2008, 35(4):1559-1569.

［8］Hussain A, George E, Robert H, et al. Improved accuracy of anticoagulant dose prediction using a pharmacogenetics and artificial neural network-based method［J］. European Journal of Clinical Pharmacology, 2014, 70(3):265-273.

［9］Simon G, Cherry E, Ellie M, et al. Gene expression profiling allows distinction between primary and metastatic squamous cell carcinomas in the lung［J］. American Association for Cancer Research, 2005, 65(8):3063-3071.

［10］Wang TR, Mousseau V, Pedroni N, et al. Assessing the performance of classification-based vulnerability analysis model ［J］. Risk Analysis, 2015, 35(9):1674-1689.

［11］Nagesh A, Bret M, Antoine L, et al. Penalized likelihood phenotyping: unifying voxel wise analyses and multi-voxel pattern analyses in neuroimaging ［J］. Neuroinformatics, 2013, 11(2):227-247.

服务与反馈：

【文章下载】【加入收藏】

提示：您还未登录，请登录！点此登录