JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE) ›› 2020, Vol. 55 ›› Issue (9): 62-71.doi: 10.6040/j.issn.1671-9352.0.2019.475

•   • Previous Articles     Next Articles

Detection method of hemorrhages of fundus image based on deep learning

Wen-she YIN,Jian-feng HE*()   

  1. Faculty of Information Engineering and Automation, Kunming University of Science and Technology
  • Received:2019-07-11 Online:2020-09-20 Published:2020-09-17
  • Contact: Jian-feng HE E-mail:jfenghe@foxmail.com

RichHTML

101

PDF (PC)

589

Abstract:

This paper proposes a method for detecting the bleeding point of fundus images based on convolutional neural networks (CNN) plus conditional random fields (CRF). First, in order to avoid the influence of the background area of the image on subsequent detection, refer to the gray level information in the fundus image and adjust the image to the appropriate size according to the length of the fundus center to its edge, and then linearly weight the image to enhance its brightness and contrast; then, using the cropped image block to train the CNN model for detecting the bleeding point on the CNN architecture built on the VGG network; finally, to overcome the problems of false detection and missed detection in the bleeding point detection of the CNN model, CRF is used for CNN. The probability map of the model output is post-processed to achieve accurate detection of the bleeding point of the fundus image. The detection method proposed in this paper was trained and verified on the open Kaggle and Messidor databases, and achieved 98.8% accuracy, 99.4% recall rate and 99.1% F-score. In addition, the sensitivity tested on the DIARETDB1 database reached 98.5% and the F-score was 96.1%. The experimental results show that the effectiveness and superiority of the proposed method are illustrated from both image visual and quantitative detection.

Key words: diabetic retinopathy, hemorrhages, fundus image, convolutional neural network, conditional random field

CLC Number: 

  • R318

Fig.1

Detection model flow chart"

Table 1

Statistics on the number of data sets"

项目 Kaggle Messidor DIARETDB1
正常图像 822 547 5
患病图像 1 562 653 84
合计 2 384 1 200 89

Fig.2

Fundus image preprocessing results"

Table 2

Data statistics"

训练阶段测试阶段DIARETDB1
训练集 验证集
                Kaggle：2 480(-) 620(-) 200(-)
2 416(+) 605(+)
           Messidor：1 699(-) 425(-) 200(+)
1 760(+) 440(+)

Fig.3

Training set and test set image block"

Table 3

CNN structure"

Layer Operation Input size/pixels Details
Layer1 Convolution 41×41 4 pixels×4 pixels，k=25，SAME，BN
Layer2 Max pooling 41×41 2 pixels×2 pixels
Layer3 Convolution 20×20 5 pixels×5 pixels，k=50，VALID，BN
Layer4 Max pooling 16×16 2 pixels×2 pixels
Layer5 Fully connected 3 200×1 1 024节点
Layer6 Fully connected 1 024×1 1 024节点
Layer7 Fully connected 1 024×1 2节点
Layer8 Softmax 2×1 2分类

Fig.4

Convolutional neural network structure"

Fig.5

Conditional random field structure(A rectangular node represents a pixel or a block of pixels on the observed image, and a circular node represents a category tag corresponding to a pixel or a block of pixels)"

Table 4

Model performance statistics"

项目训练时期测试时期
Loss Acc/% P/% R/% F-score/%
训练 0.006 7 99.80
验证 0.103 4 97.47 98.50 93.80 96.10

Fig.6

Example of a bleeding point detection(The first behavior is a test image of the Kaggle database, and the second behavior is a test image of the DIARETDB1 database)"

Table 5

5 Bleeding point test results for different models"

方法DIARETDB1MESSIDORKaggleYear
SE/% IOU/% PPV/% SE/% SP/% AUC/% SE/% SP/%
k-means clustering[4] 89.00 — 87.30 — — — — — 2015
SeS-CNN[11] — — — 93.10 91.50 — 84.80 90.40 2016
ensemble deep learning[15] — — — 91.10 — 89.30 — — 2018
multi-scale area block[16] 94.91 — 46.84 — — — — — 2018
multi-level fusion[16] 66.50 — 48.24 — — — — — 2018
PixelNet[17] — 88.90 — — — — — — 2019
CNN+CRF 98.50 92.49 93.80 98.80 99.40 98.00 98.40 98.30 2019
1 GIRARD F. Simultaneous macula detection and optic disc boundary segmentation in retinal fundus images[C]//SPIE Medical Imaging. San Diego: SPIE, 2016.
2 SAFI H , SAFI S , HAFEZIMOGHADAM A , et al. Early detection of diabetic retinopathy[J]. Survey of Ophthalmology, 2018, 63 (5): 601- 608.
doi: 10.1016/j.survophthal.2018.04.003
3 MA Xiaolong, XIE Xudong, LAM K, et al. A new bottom-up method for saliency detection[C]//IEEE International Symposium on Consumer Electronics. Hsinchu: IEEE, 2013.
4 肖志涛, 赵北方, 张芳, 等. 基于k均值聚类和自适应模板匹配的眼底出血点检测方法[J]. 中国生物医学工程学报, 2015, 34 (3): 264- 271.
XIAO Zhitao , ZHAO Beifang , ZHANG Fang , et al. Method for detecting fundus hemorrhage point based on k-means clustering and adaptive template matching[J]. Chinese Journal of Biomedical Engineering, 2015, 34 (3): 264- 271.
5 HALOI M , DANDAPAT S , SINHA R . A Gaussian scale space approach for exudates detection, classification and severity prediction[J]. Computer Science, 2015, 56 (1): 3- 6.
6 SRIVASTAVA R , WONG D W , DUAN L , et al. Red lesion detection in retinal fundus images using Frangi-based filters[J]. IEEE Engineering in Medicine and Biology Society, 2015, 2015 (1): 5663- 5666.
7 YANG Chuan , ZHANG Lihe , LU Huchuan , et al. Saliency detection via graph-based manifold ranking[J]. Computer Vision & Pattern Recognition, 2013, 9 (4): 3166- 3173.
8 JIANG Huaizu , WANG Jingdong , YUAN Zejian , et al. Salient object detection: a discriminative regional feature integration approach[J]. International Journal of Computer Vision, 2017, 123 (2): 251- 268.
9 BORJI A , CHENG M M , JIANG H , et al. Salient object detection: a benchmark[J]. IEEE Transactions on Image Processing, 2015, 24 (12): 5706- 5722.
doi: 10.1109/TIP.2015.2487833
10 PRATT H , COENEN F , BROADBENT D M , et al. Convolutional neural networks for diabetic retinopathy[J]. Procedia Computer Science, 2016, 90 (7): 200- 205.
11 VAN G M , VAN G B , HOYNG C , et al. Fast convolutional neural network training using selective data sampling:application to hemorrhage detection in color fundus images[J]. IEEE Transactions on Medical Imaging, 2016, 35 (5): 1273- 1284.
doi: 10.1109/TMI.2016.2526689
12 YANG Yehui , LI Tao , LI Wensi , et al. Lesion detection and grading of diabetic retinopathy via two-stages deep convolutional neural networks[J]. Medical Image Computing and Computer-Assisted Intervention, 2017, 10435 (3): 533- 540.
13 LAM C , YU C , HUANG L , et al. Retinal lesion detection with deep learning using image patches[J]. Investigative Ophthalmology & Visual Science, 2018, 59 (1): 590- 596.
14 RAMON P , SANDRA A , JACQUES W , et al. A data-driven approach to referable diabetic retinopathy detection[J]. Artificial Intelligence in Medicine, 2019, 96 (3): 93- 106.
15 ORLANDO J I , PROKOFYEVA E , DEL F M , et al. An ensemble deep learning based approach for red lesion detection in fundus images[J]. Computer Methods Programs Biomed, 2018, 153 (10): 115- 127.
16 马文婷.面向眼科医学图像的病变检测研究[D].北京:北京交通大学, 2018.
MA Wenting. Research on lesion detection for ophthalmic medical images[D]. Beijing: Beijing Jiaotong University, 2018.
17 张诗浩.基于深度学习的眼底图像出血点分割方法研究[D].天津:天津工业大学, 2019.
ZHANG Shihao. Research on segmentation method of hemorrhages of fundus image based on deep learning[D]. Tianjin: Tianjin Polytechnic University, 2019.
18 GU J , WANG Z , KUEN J , et al. Recent advances in convolutional neural networks[J]. Pattern Recognition, 2018, 77 (1): 354- 377.
19 UCHIDA K, TANAKA M, OKUTOMI M. Coupled convolution layer for convolutional neural network[C]//International Conference on Pattern Recognition. Cancun: ICPR, 2016.
20 DECENCI RE E , ZHANG X , CAZUGUEL G , et al. Feedback on a publicly distributed image database: the Messidor database[J]. Image Analysis & Stereology, 2014, 33 (3): 231- 234.
21 KAUPPI T, KALESNYKIENE V, KAMARAINEN J K, et al. DIARETDB1 diabetic retinopathy database and evaluation protocol[C]//Proceeding of the British Machine Vision Conference. Coventry: DPLP, 2007: 1-10.
22 DJEKOUNE A O , MESSAOUDI K , AMARA K . Incremental circle hough transform: an improved method for circle detection[J]. Optik - International Journal for Light and Electron Optics, 2017, 133 (1): 17- 31.
23 杨俊俐, 姜志国, 周全, 等. 基于条件随机场的遥感图像语义标注[J]. 航空学报, 2015, 36 (9): 3069- 3081.
YANG Junli , JIANG Zhiguo , ZHOU Quan , et al. Semantic annotation of remote sensing image based on conditional random field[J]. Journal of Aviation, 2015, 36 (9): 3069- 3081.
24 KARIMAGHALOO Z , ARNOLD D L , ARBEL T . Adaptive multi-level conditional random fields for detection and segmentation of small enhanced pathology in medical images[J]. Medical Image Analysis, 2015, 27 (2): 17- 30.
25 常亮, 邓小明, 周明全, 等. 图像理解中的卷积神经网络[J]. 自动化学报, 2016, 42 (9): 1300- 1312.
CHANG Liang , DENG Xiaoming , ZHOU Mingquan , et al. Convolutional neural network in image understanding[J]. Journal of Automation, 2016, 42 (9): 1300- 1312.
26 YAN Hua , HU Tian . Depth estimation with convolutional conditional random field network[J]. Neurocomputing, 2016, 214 (19): 546- 554.
27 李宗民, 徐希云, 刘玉杰, 等. 条件随机场像素建模与深度特征融合的目标区域分割算法[J]. 计算机辅助设计与图形学学报, 2018, 30 (6): 29- 36.
LI Zongmin , XU Xiyun , LIU Yujie , et al. Target region segmentation algorithm based on conditional random field pixel modeling and depth feature fusion[J]. Journal of Computer Aided Design and Graphics, 2018, 30 (6): 29- 36.
[1] Wen-qing WANG,Ao-yang HAN,Li-tao YU,Zhi-sheng ZHANG. Short-term load forecasting model based on autoencoder and PSOA-CNN [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2019, 54(7): 50-56.
[2] QIN Jing, LIN Hong-fei, XU Bo. Music retrieval model based on semantic descriptions [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2017, 52(6): 40-48.
[3] PAN Qing-qing, ZHOU Feng, YU Zheng-tao, GUO Jian-yi, XIAN Yan-tuan. Recognition method of Vietnamese named entity based on#br# conditional random fields [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2014, 49(1): 76-79.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] YANG Jun. Characterization and structural control of metalbased nanomaterials[J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2013, 48(1): 1 -22 .
[2] LUO Si-te, LU Li-qian, CUI Ruo-fei, ZHOU Wei-wei, LI Zeng-yong*. Monte-Carlo simulation of photons transmission at alcohol wavelength in  skin tissue and design of fiber optic probe[J]. J4, 2013, 48(1): 46 -50 .
[3] DONG Wei-wei. A new method of DEA efficiency ranking for decision making units with independent subsystems[J]. J4, 2013, 48(1): 89 -92 .
[4] Ming-Chit Liu. THE TWO GOLDBACH CONJECTURES[J]. J4, 2013, 48(2): 1 -14 .
[5] WANG Kai-rong, GAO Pei-ting. Two mixed conjugate gradient methods based on DY[J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2016, 51(6): 16 -23 .
[6] LI Ya-nan1, LIU Lei-po2, WANG Yu-guang3. Passive sliding mode control for uncertain time-delay systems subjected to input nonlinearity[J]. J4, 2010, 45(6): 99 -104 .
[7] ZHANG Dong-qing, YIN Xiao-bin, GAO Han-peng. Quasi-linearly Armendariz modules[J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2016, 51(12): 1 -6 .
[8] WU Dai-yong. Global attractivity of a nonlinear discrete Logistic model  with feedback control[J]. J4, 2013, 48(4): 114 -110 .
[9] ZHANG Fang-guo. Elliptic curves in cryptography: past, present and future…[J]. J4, 2013, 48(05): 1 -13 .
[10] YUAN Xiu-Hua. Signed edge total domination numbers of graphs[J]. J4, 2009, 44(8): 21 -24 .
Copyright 2018 © Editorial office of JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE)
Supported by Beijing Magtech Co.Ltd