北京生物医学工程

结合卷积神经网络与图卷积网络的乳腺癌病理图像分类研究

Research on breast cancer pathological images classification combined with convolution neural network and graph convolution network

作者：汪琳琳施俊韩振奇刘立庄

单位：上海大学通信与信息工程学院（上海 200444）中国科学院上海高等研究院（上海 201210）

关键词：乳腺癌；病理图像分类；图卷积网络；卷积神经网络；空间相关性

分类号：R318

出版年·卷·期（页码）：2021·40·2（130-138）

摘要：

目的乳腺癌的精确诊断对于后续治疗具有重要临床意义，组织病理学分析是肿瘤诊断的金标准。卷积神经网络(convolution neural network，CNN)具有良好的局部特征提取能力，但无法有效捕捉细胞组织间的空间关系。为了有效利用这种空间关系，本文提出一种新的结合CNN与图卷积网络（graph convolution network，GCN）的病理图像分类框架，应用于乳腺癌病理图像的辅助诊断。方法首先对病理图像进行卷积及下采样得到一组特征图，然后将特征图上每个像素位置的特征向量表示为1个节点，构建具有空间结构的图，并通过GCN学习图中蕴含的空间结构特征。最后，将基于GCN的空间结构特征与基于CNN的全局特征融合，并同时对整个网络进行优化，实现基于融合特征的病理图像分类。结果本文在提出框架下进行了3种GCN的比较，其中CNN-sGCN-fusion算法在2015生物成像挑战赛乳腺组织学数据集上获得93.53%±1.80%的准确率，在Databiox乳腺数据集上获得78.47%±5.33%的准确率。结论与传统基于CNN的病理图像分类算法相比，本文提出的结合CNN与GCN的算法有效融合了病理图像的全局特征与空间结构特征，从而提升了分类性能，具有潜在的应用可行性。

Objective The accurate diagnosis of breast cancer is of great clinical significance for subsequent treatment, and histopathological analysis is the gold standard for tumor diagnosis. Convolution neural network (CNN) has good local feature extraction capabilities, but it cannot effectively capture the spatial relationship between cell tissues. In order to effectively utilize this spatial relationship, this paper proposes a new pathological image classification framework combining CNN and graph convolution network (GCN) for the auxiliary diagnosis of breast cancer pathological images. Methods Firstly, the pathological image is convoluted and subsampled to get a group of feature maps. Then, the feature vector of each pixel position on the feature maps is represented as a node to construct the graph with spatial structure, and the spatial structure features contained in the graph are learned by GCN. Finally, the spatial structure features based on GCN are fused with the global features based on CNN, and the whole network is optimized at the same time to achieve pathological image classification based on fusion features. Results This paper compares three types of GCN under the proposed framework. Among them, the CNN-sGCN-fusion algorithm achieved 93.53%±1.80% accuracy on the bioimaging challenge 2015 breast histology dataset, and 78.47%±5.33% accuracy on the Databiox breast dataset. Conclusions Compared with the traditional pathological image classification algorithms based on CNN, the algorithm proposed in this paper combines the global and spatial structure features of pathological images effectively, which improves the classification performance and has potential application feasibility.

参考文献：

[1]?Wild CP. International agency for research on cancer?[J]. Encyclopedia of Toxicology, 2014, 133(9):?1067-1069.

[2]?Gurcan MN, Boucheron LE, Can A, et al.?Histopathological image analysis: a review?[J]. IEEE Reviews in Biomedical Engineering,?2009, 2: 147-171.

[3]?Yener B. Cell-graphs: image-driven modeling of structure-function relationship [J]. Communications of the ACM, 2016, 60(1):?74-84.

[4]?Baba AI, Catoi C. Tumor cell morphology [M]//Comparative Oncology. Bucharest, RO: The Publishing House of the Romanian Academy, 2007.

[5]?Sharma H, Zerbe N, Lohmann S, et al.?A review of graph-based methods for image analysis in digital histopathology [J].?Diagnostic Pathology, 2015,1(1):1-51.?

[6]?He L, Long LR, Antani S,?et al. Histology image analysis for carcinoma detection and grading [J].?Computer Methods and Programs in Biomedicine,?2012, 107(3):?538-556.

[7]?郑光远,刘峡壁,韩光辉.医学影像计算机辅助检测与诊断系统综述 [J].软件学报,2018, 29(5):1471-1514.

Zheng GY, Liu XB, Han GH. Survey on medical image computer aided detection and diagnosis systems [J]. Journal of Software, 2018, 29(5):1471-1514.

[8]?Lecun Y, Bengio Y, Hinton G. Deep learning [J]. Nature, 2015, 521(7553):436-444.

[9]?Shen DG, Wu GR, Suk HI. Deep learning in medical image analysis [J]. Annual Review of Biomedical Engineering, 2017, 19(1):221-248.

[10]?Litjens G,?Kooi T,?Bejnordi BE,?et al. A survey on deep?learning in medical image analysis [J]. Medical Image?Analysis，2017,?42(9):?60-88.

[11]?Spanhol FA, Oliveira LS, Petitjean C, et al.?Breast cancer histopathological image classification using convolutional neural networks [C] // 2016 International Joint Conference on Neural Networks (IJCNN).?Vancouver, BC：IEEE Press, 2016:?2560-2567.

[12]?Araújo T, Aresta G, Castro E, et al. Classification of breast cancer histology images using convolutional neural networks [J]. PLoS One, 2017, 12(6): e0177544.

[13]?Vesal S, Ravikumar N, Davari AA, et al. Classification of breast cancer histology images using transfer learning [C] // International Conference Image Analysis and Recognition (ICIAR)?2018.?Switzerland:?Springer, Cham, 2018:812-819.

[14]?Yan?R, Ren F, Wang ZH, et al. A hybrid convolutional and recurrent deep neural network for breast cancer pathological image classification [C] //?2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). Madrid, Spain: IEEE?Press, 2018: ?957-962.

[15]?Wang CF, Shi J, Zhang Q, et al. Histopathological image classification with bilinear convolutional neural networks [C] //?39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).?Seogwipo: IEEE Press, 2017: 4050-4053.

[16]?Kipf TN, Welling M. Semi-Supervised classification with graph convolutional networks [C] //?International Conference on Learning Representations (ICLR)?2017.??San Juan, Puerto Rico: ICLR, 2017.?

[17]?Wu ZH, Pan SR, Chen FW, et al.?A comprehensive survey on graph neural networks [J]. IEEE Transactions on Neural Networks and Learning System, 2019:1-21.?

[18]?徐冰冰,岑科廷,黄俊杰,等.图卷积神经网络综述[J].计算机学报, 2020,43(5):755-780.

Xu BB, Cen KT, Huang JJ, et al.?A survey of graph convolution neural networks?[J].?Chinese Journal of Computers, 2020,43(5):755-780.

[19]?Hamilton WL, Ying R, Leskovec J. Inductive representation learning on large graphs [C] // 31st Conference on Neural Information Processing Systems (NIPS). Long Beach, CA, USA,?2017:?1024-1034.

[20]?Wang Y, Sun YB, Liu ZW, et al. Dynamic graph cnn for learning on point clouds [J]. ACM Transactions on Graphics, 2019, 38(5):?146-158.

[21]?Bruna J, Zaremba W, Szlam A, et al. Spectral networks and locally connected networks on graphs [C] //?2014?International Conference on Learning Representations (ICLR). Banff, Canada: ICLR, 2014.?

[22]?Zhou YN, Graham S, Koohbanani NA, et al. CGC-Net: cell graph convolutional network for grading of colorectal cancer histology images [C]?//?2019 IEEE International Conference on Computer Vision Workshop (ICCVW).?Seoul, South Korea: IEEE Press, 2019: 388-398.

[23]?Wang JW, Chen RJ, Lu MY, et al.?Weakly supervised prostate TMA classification via graph convolutional network [EB/OL].?[2019-10-29].?https://arxiv.org/pdf/1910.13328v2.pdf.?

[24]?Li RY, Yao JW, Zhu XL,?et al. Graph CNN for survival analysis on whole slide pathological images [M]?//?Medical Image Computing and Computer Assisted Intervention (MICCAI)?2018. Switzerland: Springer,?Cham,?2018, 11071: 174-182.?

[25]?Adnan M, Kalra S, Tizhoosh HR, et al. Representation learning of histopathology images using graph neural networks[C] //?2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).?Seattle, WA, USA: IEEE?Press, 2020: 4254-4261.

[26]?Zhang ML, Zhou ZH. ML-KNN: a lazy learning approach to multi-label learning [J]. Pattern Recognition, 2007, 40(7):?2038-2048.

[27]?Kieffer B, Babaie M, Kalra S, et al. Convolutional neural networks for histopathology image classification: training vs. using pre-trained networks [C]?// 2017 Seventh International Conference on Image Processing Theory, Tools and Applications (IPTA). Montreal, QC: IPTA, 2017:1-6.

[28]?He KM, Zhang XY, Ren SQ, et al. Deep residual learning for image recognition [C] // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV: IEEE Press, 2016: 770-778.

[29]?Li GH, Müller M, Thabet A, et al. DeepGCNs: Can GCNs go as deep as CNNs? [C] //?IEEE International Conference on Computer Vision (ICCV). Seoul,South Korea: IEEE Press, 2019: 9266-9275.

[30]?Bolhasani H, Amjadi E, Tabatabaeian M, et al. A histopathological image dataset for grading breast invasive ductal carcinomas [J]. Informatics in Medicine Unlocked, 2020, 19:100341.??

[31]?Macenko M, Niethammer M, Marron JS, et al. A method for normalizing histology slides for quantitative analysis [C] //?2009 IEEE International Symposium on Biomedical Imaging (ISBI). Boston, MA, USA?: IEEE Press, 2009:?1107-1110.

服务与反馈：

【文章下载】【加入收藏】

提示：您还未登录，请登录！点此登录