基于深度学习的眼底图像出血点检测方法

doi:10.6040/j.issn.1671-9352.0.2019.475

摘要/Abstract

摘要：

提出了一种基于卷积神经网络(convolutional neural networks，CNN)加条件随机场(conditional random fields，CRF)的眼底图像出血点检测方法。首先，为了避免图像背景区域对后续检测的影响，参考眼底图像中的灰度信息并根据眼底中心位置到其边缘的长度，将图像调整到合适的尺寸，再对图像进行线性加权增强其亮度和对比度；然后，用裁剪到的图像块在仿照VGG网络构建的CNN架构上去训练检测出血点的CNN模型；最后，为了克服CNN模型在出血点检测中误检、漏检等问题，采用CRF对CNN模型输出的概率图进行后处理，以实现眼底图像出血点的精确检测。提出的检测方法在公开的Kaggle与Messidor数据库上进行训练和验证，获得了98.8%的准确率、99.4%的召回率和99.1%的F-score。另外，在DIARETDB1数据库上测试的灵敏度达到98.5%，F-score为96.1%。实验结果表明，从图像视觉和定量检测2个方面均说明了提出方法的有效性和优越性。

关键词: 糖尿病视网膜病变, 出血点, 眼底图像, 卷积神经网络, 条件随机场

Abstract:

This paper proposes a method for detecting the bleeding point of fundus images based on convolutional neural networks (CNN) plus conditional random fields (CRF). First, in order to avoid the influence of the background area of the image on subsequent detection, refer to the gray level information in the fundus image and adjust the image to the appropriate size according to the length of the fundus center to its edge, and then linearly weight the image to enhance its brightness and contrast; then, using the cropped image block to train the CNN model for detecting the bleeding point on the CNN architecture built on the VGG network; finally, to overcome the problems of false detection and missed detection in the bleeding point detection of the CNN model, CRF is used for CNN. The probability map of the model output is post-processed to achieve accurate detection of the bleeding point of the fundus image. The detection method proposed in this paper was trained and verified on the open Kaggle and Messidor databases, and achieved 98.8% accuracy, 99.4% recall rate and 99.1% F-score. In addition, the sensitivity tested on the DIARETDB1 database reached 98.5% and the F-score was 96.1%. The experimental results show that the effectiveness and superiority of the proposed method are illustrated from both image visual and quantitative detection.

Key words: diabetic retinopathy, hemorrhages, fundus image, convolutional neural network, conditional random field

中图分类号:

R318

银温社,贺建峰. 基于深度学习的眼底图像出血点检测方法[J]. 《山东大学学报(理学版)》, 2020, 55(9): 62-71.

Wen-she YIN,Jian-feng HE. Detection method of hemorrhages of fundus image based on deep learning[J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2020, 55(9): 62-71.

图/表 11

图1

表1

图2

表2

图3

表3

图4

图5

表4

图6

表5

参考文献 27

1	GIRARD F. Simultaneous macula detection and optic disc boundary segmentation in retinal fundus images[C]//SPIE Medical Imaging. San Diego: SPIE, 2016.
2	SAFI H , SAFI S , HAFEZIMOGHADAM A , et al. Early detection of diabetic retinopathy[J]. Survey of Ophthalmology, 2018, 63 (5): 601- 608. doi: 10.1016/j.survophthal.2018.04.003
3	MA Xiaolong, XIE Xudong, LAM K, et al. A new bottom-up method for saliency detection[C]//IEEE International Symposium on Consumer Electronics. Hsinchu: IEEE, 2013.
4	肖志涛, 赵北方, 张芳, 等. 基于k均值聚类和自适应模板匹配的眼底出血点检测方法[J]. 中国生物医学工程学报, 2015, 34 (3): 264- 271.
	XIAO Zhitao , ZHAO Beifang , ZHANG Fang , et al. Method for detecting fundus hemorrhage point based on k-means clustering and adaptive template matching[J]. Chinese Journal of Biomedical Engineering, 2015, 34 (3): 264- 271.
5	HALOI M , DANDAPAT S , SINHA R . A Gaussian scale space approach for exudates detection, classification and severity prediction[J]. Computer Science, 2015, 56 (1): 3- 6.
6	SRIVASTAVA R , WONG D W , DUAN L , et al. Red lesion detection in retinal fundus images using Frangi-based filters[J]. IEEE Engineering in Medicine and Biology Society, 2015, 2015 (1): 5663- 5666.
7	YANG Chuan , ZHANG Lihe , LU Huchuan , et al. Saliency detection via graph-based manifold ranking[J]. Computer Vision & Pattern Recognition, 2013, 9 (4): 3166- 3173.
8	JIANG Huaizu , WANG Jingdong , YUAN Zejian , et al. Salient object detection: a discriminative regional feature integration approach[J]. International Journal of Computer Vision, 2017, 123 (2): 251- 268.
9	BORJI A , CHENG M M , JIANG H , et al. Salient object detection: a benchmark[J]. IEEE Transactions on Image Processing, 2015, 24 (12): 5706- 5722. doi: 10.1109/TIP.2015.2487833
10	PRATT H , COENEN F , BROADBENT D M , et al. Convolutional neural networks for diabetic retinopathy[J]. Procedia Computer Science, 2016, 90 (7): 200- 205.
11	VAN G M , VAN G B , HOYNG C , et al. Fast convolutional neural network training using selective data sampling:application to hemorrhage detection in color fundus images[J]. IEEE Transactions on Medical Imaging, 2016, 35 (5): 1273- 1284. doi: 10.1109/TMI.2016.2526689
12	YANG Yehui , LI Tao , LI Wensi , et al. Lesion detection and grading of diabetic retinopathy via two-stages deep convolutional neural networks[J]. Medical Image Computing and Computer-Assisted Intervention, 2017, 10435 (3): 533- 540.
13	LAM C , YU C , HUANG L , et al. Retinal lesion detection with deep learning using image patches[J]. Investigative Ophthalmology & Visual Science, 2018, 59 (1): 590- 596.
14	RAMON P , SANDRA A , JACQUES W , et al. A data-driven approach to referable diabetic retinopathy detection[J]. Artificial Intelligence in Medicine, 2019, 96 (3): 93- 106.
15	ORLANDO J I , PROKOFYEVA E , DEL F M , et al. An ensemble deep learning based approach for red lesion detection in fundus images[J]. Computer Methods Programs Biomed, 2018, 153 (10): 115- 127.
16	马文婷.面向眼科医学图像的病变检测研究[D].北京:北京交通大学, 2018.
	MA Wenting. Research on lesion detection for ophthalmic medical images[D]. Beijing: Beijing Jiaotong University, 2018.
17	张诗浩.基于深度学习的眼底图像出血点分割方法研究[D].天津:天津工业大学, 2019.
	ZHANG Shihao. Research on segmentation method of hemorrhages of fundus image based on deep learning[D]. Tianjin: Tianjin Polytechnic University, 2019.
18	GU J , WANG Z , KUEN J , et al. Recent advances in convolutional neural networks[J]. Pattern Recognition, 2018, 77 (1): 354- 377.
19	UCHIDA K, TANAKA M, OKUTOMI M. Coupled convolution layer for convolutional neural network[C]//International Conference on Pattern Recognition. Cancun: ICPR, 2016.
20	DECENCI RE E , ZHANG X , CAZUGUEL G , et al. Feedback on a publicly distributed image database: the Messidor database[J]. Image Analysis & Stereology, 2014, 33 (3): 231- 234.
21	KAUPPI T, KALESNYKIENE V, KAMARAINEN J K, et al. DIARETDB1 diabetic retinopathy database and evaluation protocol[C]//Proceeding of the British Machine Vision Conference. Coventry: DPLP, 2007: 1-10.
22	DJEKOUNE A O , MESSAOUDI K , AMARA K . Incremental circle hough transform: an improved method for circle detection[J]. Optik - International Journal for Light and Electron Optics, 2017, 133 (1): 17- 31.
23	杨俊俐, 姜志国, 周全, 等. 基于条件随机场的遥感图像语义标注[J]. 航空学报, 2015, 36 (9): 3069- 3081.
	YANG Junli , JIANG Zhiguo , ZHOU Quan , et al. Semantic annotation of remote sensing image based on conditional random field[J]. Journal of Aviation, 2015, 36 (9): 3069- 3081.
24	KARIMAGHALOO Z , ARNOLD D L , ARBEL T . Adaptive multi-level conditional random fields for detection and segmentation of small enhanced pathology in medical images[J]. Medical Image Analysis, 2015, 27 (2): 17- 30.
25	常亮, 邓小明, 周明全, 等. 图像理解中的卷积神经网络[J]. 自动化学报, 2016, 42 (9): 1300- 1312.
	CHANG Liang , DENG Xiaoming , ZHOU Mingquan , et al. Convolutional neural network in image understanding[J]. Journal of Automation, 2016, 42 (9): 1300- 1312.
26	YAN Hua , HU Tian . Depth estimation with convolutional conditional random field network[J]. Neurocomputing, 2016, 214 (19): 546- 554.
27	李宗民, 徐希云, 刘玉杰, 等. 条件随机场像素建模与深度特征融合的目标区域分割算法[J]. 计算机辅助设计与图形学学报, 2018, 30 (6): 29- 36.
	LI Zongmin , XU Xiyun , LIU Yujie , et al. Target region segmentation algorithm based on conditional random field pixel modeling and depth feature fusion[J]. Journal of Computer Aided Design and Graphics, 2018, 30 (6): 29- 36.

多维度评价

Viewed

Full text

Abstract

Cited

Shared

Discussed

项目	Kaggle	Messidor	DIARETDB1
正常图像	822	547	5
患病图像	1 562	653	84
合计	2 384	1 200	89

训练阶段		测试阶段DIARETDB1
训练集	验证集	测试阶段DIARETDB1
Kaggle：2 480(-)	620(-)	200(-)
2 416(+)	605(+)	200(-)
Messidor：1 699(-)	425(-)	200(+)
1 760(+)	440(+)	200(+)

Layer	Operation	Input size/pixels	Details
Layer1	Convolution	41×41	4 pixels×4 pixels，k=25，SAME，BN
Layer2	Max pooling	41×41	2 pixels×2 pixels
Layer3	Convolution	20×20	5 pixels×5 pixels，k=50，VALID，BN
Layer4	Max pooling	16×16	2 pixels×2 pixels
Layer5	Fully connected	3 200×1	1 024节点
Layer6	Fully connected	1 024×1	1 024节点
Layer7	Fully connected	1 024×1	2节点
Layer8	Softmax	2×1	2分类

项目	训练时期		测试时期
项目	Loss	Acc/%	P/%	R/%	F-score/%
训练	0.006 7	99.80
验证	0.103 4	97.47	98.50	93.80	96.10

方法	DIARETDB1			MESSIDOR			Kaggle		Year
方法	SE/%	IOU/%	PPV/%	SE/%	SP/%	AUC/%	SE/%	SP/%	Year
k-means clustering^[4]	89.00	—	87.30	—	—	—	—	—	2015
SeS-CNN^[11]	—	—	—	93.10	91.50	—	84.80	90.40	2016
ensemble deep learning^[15]	—	—	—	91.10	—	89.30	—	—	2018
multi-scale area block^[16]	94.91	—	46.84	—	—	—	—	—	2018
multi-level fusion^[16]	66.50	—	48.24	—	—	—	—	—	2018
PixelNet^[17]	—	88.90	—	—	—	—	—	—	2019
CNN+CRF	98.50	92.49	93.80	98.80	99.40	98.00	98.40	98.30	2019