您的位置：山东大学 -> 科技期刊社 -> 《山东大学学报（理学版）》

《山东大学学报(理学版)》 ›› 2019, Vol. 54 ›› Issue (3): 28-37.doi: 10.6040/j.issn.1671-9352.0.2018.601

•   • 上一篇    下一篇

视觉和物体显著性检测方法

许佳1(),蒋鹏2,*()   

  1. 1. 新疆额尔齐斯河流域开发工程建设管理局，新疆 乌鲁木齐 830002
    2. 山东大学齐鲁交通学院，山东 济南 250002
  • 收稿日期:2018-10-18 出版日期:2019-03-20 发布日期:2019-03-19
  • 通讯作者: 蒋鹏 E-mail:xj8476@qq.com;sdujump@gmail.com
  • 作者简介:许佳(1984—)，男，高级工程师，研究方向为水利信息化. E-mail: xj8476@qq.com

A survey of visual saliency and salient object detection methods

Jia XU1(),Peng JIANG2,*()   

  1. 1. Xinjiang Irtysh River Basin Development & Construction Administrative Bureau, Urumqi 830002, Xinjiang, China
    2. School of Qilu Transportation, Shandong University, Jinan 250002, Shandong, China
  • Received:2018-10-18 Online:2019-03-20 Published:2019-03-19
  • Contact: Peng JIANG E-mail:xj8476@qq.com;sdujump@gmail.com

RichHTML

24

PDF (PC)

422

摘要：

显著性检测的目标是快速找出图像视频等视觉数据中最吸引人注意的区域，作为计算机视觉领域的基本任务之一，近年来备受关注，众多的方法被提出。这些显著性检测工作可分为2个分支：视觉显著性检测方法和显著性物体检测方法。尽管这2个分支的方法有很多相同点甚至共享相同的计算模型，但是在不同分支的评价数据集上有巨大的性能差异，很少有工作对这2个分支的方法进行比较和分析。通过详细分析和阐述2个分支主流方法的计算模型、采用的评价机制以及使用的数据集，总结了多种改进视觉显著性检测方法用来检测显著性物体的方式，通过这些方式视觉显著性检测方法可应用于显著性物体检测数据集，其性能达到了领先水平甚至超过了一些主流显著性物体检测方法，从而缓解了2个分支显著性检测方法在不同分支数据集上表现的不一致的问题。

关键词: 显著性检测, 视觉显著性检测, 显著性物体检测

Abstract:

Saliency detection aims to locate the most attractive areas in image or video data, as the basic task of computer vision field, has receive intensive attentions. Many methods have been proposed recently, these methods usually can be classified into two branches: visual saliency detection and salient object detection. Tough the methods of two branches usually share the similar features and even the frameworks, their performances on datasets of different branch have large gap, seldom works have compared and analyzed them. In this work, we will provide a detailed review and analysis of main works in two branches, including their mechanism, metrics and datasets. Besides, in this work, we summarized approaches to boost visual saliency detection methods for the task of salient object detection. With these approaches, visual saliency detection methods can be applied to detect the salient object and show superior performance that even outperform some specialized state-of-the-art salient object detection methods, thus reduce the performance inconsistence in different specialized datasets.

Key words: saliency detection, visual saliency detection, salient object detection

中图分类号: 

  • TP309

图1

显著性检测方法不同分支的检测结果"

图2

Itti显著性检测方法[7]主要结构 给定输入图像，首先将图像分解为一系列特征图(feature maps)，然后对每个特征图中的每个位置计算中心区域C和周边区域S的对比得到显著性线索(Saliency cues)，最后结合所有显著性线索加权计算显著性图(Saliency map)[49]。"

表1

常见特征对比度量方法D和特征对比范围R"

参数 可选项
D KL 散度  L1 范数  L2 范数  χ2(卡方距离)  (内积)
R 局部 部分 全局

图3

显著性计算方向[49]"

表2

显著性特征与其分类学，以及采用这些特征的方法"

组 类 形式 特征 方法(见文献)
节点 像素、超像素、区域 位置、大小、形状 [15, 19, 23, 24, 38, 45]
颜色 RGB、LAB、HSV等 原值、均值、方差、直方图、分布 [3-5, 7-9, 12-24, 26, 27, 29-46]
边缘 方向、梯度、HOG、聚焦度 [4, 7-10, 12, 22, 35, 45]
低层特征 角点 LBP, SIFT, Convex Hull [11, 21, 35, 39]
滤波器 Gabor, LM, Local steering 原值、直方图 [5, 8, 10, 21, 33]
先验 中心、背景 位置、连接度Connectivity [3, 8, 20, 21, 24, 25, 27, 29, 31, 36-38, 45-48]
其他 ICA, PCA, SVD 系数 [1, 2, 12, 23, 27, 29]
高层特征 先验 位置、颜色、语义 中心、颜色冷暖度、人脸 [8, 15, 19, 22, 24, 25, 31, 33]
其他 深度神经网络 特征响应 [39]等

表3

数据集及其属性和难易程度"

数据集 年份 任务 图像数量 物体个数 标注形式 分辨率 标注人数 难易程度
Toronto[1] 2005 V 120 多个 眼动数据 [681, 581] 20 1
MIT[8] 2009 V 1 003 多个 眼动数据 [1 024, 768] 15 2
MSRA1000[14] 2007 O 1 000 单个 像素级标注 [400, 300] 1 1
MSRA10K[16] 2013 O 10 000 单个 像素级标注 [400, 300] 1 3
ECSSD[36] 2013 O 1 000 多个 像素级标注 [X, 400] 5 2
DUT-OMRON[37] 2013 V/O 5 168 多个 眼动数据、包围盒、像素级标注 [X, 400] 5 4

图4

视觉显著性检测结果和显著性物体检测结果的相互转化"

图5

信息扩散方法 (1)给定输入图像，首先通过视觉显著性检测方法预测出人眼注视位置，作为显著性种子s；(2)根据节点间的相似性构建扩散矩阵A-1,把包涵初始显著信息的s扩散到全图，标注出整个显著性物体V。"

图6

MSRA10K数据集上不同评价方法的结果 其中每个图表的每列对应－种视觉显著性检测方法: 是原方法结果，是用区域平均方式改进后的结果，是用信息扩散方式改进后的结果。"

图7

DUT-OMRON数据集上不同评价方法的结果 其中每个图表的每列对应－种视觉显著性检测方法: 是原方法结果，是用区域平均方式改进后的结果，是用信息扩散方式改进后的结果。"

图8

MSRA10K和DUT-OMRON数据集上6种显著性物体检测方法和2组改进后的视觉显著性检测方法的PR曲线"

图9

3种主流显著性物体检测方法和2组改进的视觉显著性检测方法在MSRA10K数据集上的结果"

1 BRUCE N, TSOTSOS J. Saliency based on information maximization[C]// Proceedings of 18th Advances in Neural Information Processing Systems (NIPS 2005). Vancouver: MIT Press, 2005: 155-162.
2 BRUCE N , TSOTSOS J . Saliency, attention, and visual search: an information theoretic approach[J]. Journal of Vision, 2009, 3 (9): 5.1- 24.
3 FRINTROP S, WERNER T, GARCIA M. Traditional saliency reloaded: a good old model in new shape[C]// Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015). Boston: MA, 2015: 82-90.
4 GAO D , VASCONCELOS N . Decision-theoretic saliency: computational principles, biological plausibility, and implications for neurophysiology and psychophysics[J]. Neural Computation, 2009, 21 (1): 239- 271.
doi: 10.1162/neco.2009.11-06-391
5 HAREL J, KOCH C, PERONA P. Graph-based visual saliency[C]// Proceedings of 19th Advances in Neural Information Processing Systems (NIPS 2006). Canada: MIT Press, 2006: 545-552.
6 HOU X, ZHANG L. Saliency detection: a spectral residual approach[C]// Proceedings of 2007 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2007). Minneapolis: MN, 2007: 1-8.
7 ITTI L , KOCH C , NIEBUR E . A model of saliency-based visual attention for rapid scene analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, 20 (11): 1254- 1259.
doi: 10.1109/34.730558
8 JUDD T, EHINGER K, DURAND F, et al. Learning to predict where humans look[C]// Proceedings of 2009 International Conference on Computer Vision (ICCV 2009). Kyoto: IEEE, 2009: 2106-2113.
9 KLEIN D, FRINTROP S. Center-surround divergence of feature statistics for salient object detection[C]// Proceedings of 2011 International Conference on Computer Vision (ICCV 2011). Barcelona: IEEE, 2011: 2214-2219.
10 SEO H , MILANFAR P . Static and space-time visual saliency detection by self-resemblance[J]. Journal of Vision, 2009, 9 (12): 15.1- 27.
11 YANG J, YANG M. Top-down visual saliency via joint crf and dictionary learning[C]// Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2012). Rhode Island: IEEE, 2012: 2296-2303.
12 ZHANG L , TONG M H , MARKS T K , et al. Sun: a Bayesian framework for saliency using natural statistics[J]. Journal of Vision, 2008, 8 (7): 32.1- 20.
13 ACHANTA R, ESTRADA F, WILS P, et al. Salient region detection and segmentation[C]// Proceedings of 2008 International Conference on Computer Vision Systems (ICVS 2008). Berlin: Springer, 2008: 66-75.
14 ACHANTA R, HEMAMI S, ESTRADA F, et al. Frequency-tuned salient region detection[C]// Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2009). Miami: IEEE, 2009: 1597-1604.
15 CHANG K, LIU T, CHEN H, et al. Fusing generic objectness and visual saliency for salient object detection[C]// Proceedings of 2011 International Conference on Computer Vision (ICCV 2011). Barcelona: IEEE, 2011: 914-921.
16 CHENG M , MITRA N J , HUANG X , et al. Global contrast based salient region detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37 (3): 569- 582.
doi: 10.1109/TPAMI.2014.2345401
17 CHENG M, WARRELL J, LIN W, et al. Efficient salient region detection with soft image abstraction[C]// Proceedings of 2013 International Conference on Computer Vision (ICCV 2013). Sydney: IEEE, 2013: 1529-1536.
18 FELZENSZWALB P F , HUTTENLOCHER D P . Efficient graph-based image segmentation[J]. International Journal of Computer Vision, 2004, 59 (2): 167- 181.
doi: 10.1023/B:VISI.0000022288.19776.77
19 GOFERMAN S , ZELNIK-MANOR L , TAL A . Context-aware saliency detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34 (10): 1915- 1926.
doi: 10.1109/TPAMI.2011.272
20 JIANG B, ZHANG L, LU H, et al. Saliency detection via absorbing markov chain[C]// Proceedings of 2013 International Conference on Computer Vision (ICCV 2013). Sydney: IEEE, 2013: 1665-1672.
21 JIANG H, WANG J, YUAN Z, et al. Salient object detection: a discriminative regional feature integration approach[C]// Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2013). Portland: IEEE, 2013: 2083-2090.
22 JIANG P, LING H, YU J, et al. Salient region detection by ufo: uniqueness, focusness and objectness[C]// Proceedings of 2013 International Conference on Computer Vision (ICCV 2013). Syndney: IEEE, 2013: 1976-1983.
23 KIM J, HAN D, TAI Y W, et al. Salient region detection via high-dimensional color transform[C]// Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2014). Columbus: IEEE, 2014: 883-890.
24 LI X, LU H, ZHANG L, et al. Saliency detection via dense and sparse reconstruction[C]// Proceedings of 2013 International Conference on Computer Vision (ICCV 2013). Sydney: IEEE, 2013: 2976-2983.
25 LIU R, CAO J, LIN Z, et al. Adaptive partial differential equation learning for visual saliency detection[C]// Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2014). Columbus: IEEE, 2014: 3866-3873.
26 LIU T , YUAN Z , SUN J , et al. Learning to detect a salient object[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33 (2): 353- 367.
doi: 10.1109/TPAMI.2010.70
27 LU S, MAHADEVAN V, VASCONCELOS N. Learning optimal seeds for diffusion-based salient object detection[C]// Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2014). Columbus: IEEE, 2014: 2790-2797.
28 MAI L, NIU Y, LIU F. Saliency aggregation: a data-driven approach[C]// Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2013). Portland: IEEE, 2013: 1131-1138.
29 MARGOLIN R, TAL A, ZELNIK-MANOR L. What makes a patch distinct?[C]// Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2013). Portland: IEEE, 2013: 1139-1146.
30 PERAZZI F, KRAHENBUHL P, PRITCH Y, et al. Saliency filters: contrast based filtering for salient region detection[C]// Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2012). Providence: IEEE, 2012: 733-740.
31 QIN Y, LU H, XU Y, et al. Saliency detection via cellular automata[C]// Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015). Boston: IEEE, 2015: 110-119.
32 REN Z, HU Y, CHIA L T, et al. Improved saliency detection based on superpixel clustering and saliency propagation[C]// Proceedings of 18th ACM international conference on Multimedia (MM2010). Firenze: ACM, 2010: 1099-1102.
33 SHEN X, WU Y. A unified approach to salient object detection via low rank matrix recovery[C]// Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2012). Providence: IEEE, 2012: 853-860.
34 SIVA P, RUSSELL C, XIANG T, et al. Looking beyond the image: unsupervised learning for object saliency and detection[C]// Procceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2013). Portland: IEEE, 2013: 3238-3245.
35 XIE Y , LU H , YANG M . Bayesian saliency via low and mid level cues[J]. IEEE Transactions on Image Processing, 2013, 22 (5): 1689- 1698.
doi: 10.1109/TIP.2012.2216276
36 YAN Q, XU L, SHI J, et al. Hierarchical saliency detection[C]// Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2013). Portland: IEEE, 2013: 1155-1162.
37 YANG C, ZHANG L, LU H, et al. Saliency detection via graph-based manifold ranking[C]// Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2013). Portland: IEEE, 2013: 3166-3173.
38 ZHU W, LIANG S, WEI Y, et al. Saliency optimization from robust background detection[C]// Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2014). Columbus: IEEE, 2014: 2814-2821.
39 ZOU W, KOMODAKIS N. Harf: hierarchy-associated rich features for salient object detection[C]// Proceedings of 2015 International Conference on Computer Vision (ICCV 2015). Santiago: IEEE, 2015: 406-414.
40 王军, 吴泽民, 杨巍, 等. 基于稀疏恢复与优化的显著性目标检测算法[J]. 计算机科学, 2018, 45 (8): 258- 263.
WANG Jun , WU Zemin , YANG Wei , et al. Salient object detection algorithm based on sparse recovery and optimization[J]. Computer Science, 2018, 45 (8): 258- 263.
41 孙赫赫, 尚晓清, 王冲. 基于区域对比的图像显著性检测方法[J]. 计算机工程与应用, 2018, 54 (10): 192- 195.
doi: 10.3778/j.issn.1002-8331.1612-0439
SUN Hehe , SHANG Xiaoqing , WANG Chong . Method of saliency detection based on region contrast[J]. Computer Eninerring and Applications, 2018, 54 (10): 192- 195.
doi: 10.3778/j.issn.1002-8331.1612-0439
42 刘亚宁, 吴清, 魏雪. 基于流行排序的前景背景显著性检测算法[J]. 科学技术与工程, 2018, 18 (18): 74- 81.
doi: 10.3969/j.issn.1671-1815.2018.18.011
LIU Yaning , WU Qing , WEI Xue . Saliency detection combined foreground with background based on manifold ranking[J]. Science Technology and Engineering, 2018, 18 (18): 74- 81.
doi: 10.3969/j.issn.1671-1815.2018.18.011
43 梁大川, 李静, 刘赛. 基于图和稀疏主成分分析的多目标显著性检测[J]. 计算机研究与发展, 2018, 55 (5): 1078- 1089.
LIANG Dachuan , LI Jing , LIU Sai . Multiple object saliency detection based on graph and sparse principal component analysis[J]. Journal of Computer Research and Development, 2018, 55 (5): 1078- 1089.
44 侯向丹, 刘晓曼, 刘洪普, 等. 基于多尺度多特征的显著性区域检测[J]. 河北工业大学学报, 2017, 1 (6): 26- 31.
HOU Xiangdan , LIU Xiaoman , LIU Hongpu , et al. Salient region detection via multi-scale and multi-feature[J]. Journal of Hebei University of Technology, 2017, 1 (6): 26- 31.
45 ERDEM E , ERDEM A . Visual saliency estimation by non-linearly integrating features using region covariance[J]. Journal of Vision, 2013, 13 (4): 11.
doi: 10.1167/13.4.11
46 JIANG P, VASCONCELOS N, PENG J. Generic promotion of diffusion-based salient object detection[C]// Proceedings of 2015 International Conference on Computer Vision (ICCV 2015). Santiago: IEEE, 2015: 217-225.
47 ZHANG J, SCLAROFF S. Saliency detection: a boolean map approach[C]// Proceedings of 2013 International Conference on Computer Vision (ICCV 2013). Sydney: IEEE, 2013: 153-160.
48 ZHANG J, SCLAROFF S, LIN Z, et al. Minimum barrier salient object detection at 80 fps[C]// Proceedings of 2015 International Conference on Computer Vision (ICCV 2015). Santiago: IEEE, 2015: 1404-1412.
49 蒋鹏.图像内容显著性检测的理论和方法研究[D].济南:山东大学, 2016.
JIANG Peng. Research on theories and methodologies for saliency detection from images[D]. Jinan: Shandong Unversity, 2016.
[1] 吴福生,张焕国,倪明涛,王俊. 基于密码协议实现的行为安全分析模型[J]. 《山东大学学报(理学版)》, 2019, 54(3): 18-27.
[2] 谢小杰,梁英,董祥祥. 社交网络用户敏感属性迭代识别方法[J]. 《山东大学学报(理学版)》, 2019, 54(3): 10-17, 27.
[3] 常天天,陈兴蜀,罗永刚,兰晓. 面向Hive的基于安全域的数据隔离保护框架[J]. 《山东大学学报(理学版)》, 2019, 54(3): 1-9.
[4] 毋泽南,田立勤,王志刚. 一种结合滑动窗口和推荐信任的用户行为信任评估[J]. 《山东大学学报(理学版)》, 2019, 54(1): 53-59.
[5] 杜瑶瑶,潘平,令狐金花. 基于信息距离的信息系统等级保护评价方法[J]. 《山东大学学报(理学版)》, 2019, 54(1): 47-52.
[6] 巫朝霞,王佳琪. 一种无线单频谱安全拍卖算法[J]. 《山东大学学报(理学版)》, 2018, 53(11): 51-55.
[7] 焦鸿儒,秦静. 可实现全部超星量子存取结构的量子秘密共享方案[J]. 山东大学学报（理学版）, 2018, 53(9): 62-68.
[8] 晏燕,郝晓弘. 差分隐私密度自适应网格划分发布方法[J]. 山东大学学报（理学版）, 2018, 53(9): 12-22.
[9] 张建标,李志刚,刘国杰,王超,王玮. 面向Windows环境进程主动动态度量方法[J]. 山东大学学报（理学版）, 2018, 53(7): 46-50.
[10] 刘政,牛芳琳,钱大兴,蔡希彪,郭颖. 基于喷泉码的防窃听编码设计[J]. 山东大学学报（理学版）, 2018, 53(7): 60-64.
[11] 许力冬,王明强. 对10轮AES-128的中间相遇攻击[J]. 山东大学学报（理学版）, 2018, 53(7): 39-45.
[12] 崔朝阳,孙甲琦,徐松艳,蒋鑫. 适用于集群无人机的自组网安全分簇算法[J]. 山东大学学报（理学版）, 2018, 53(7): 51-59.
[13] 刘明明,张敏情,刘佳,高培贤. 一种基于浅层卷积神经网络的隐写分析方法[J]. 山东大学学报（理学版）, 2018, 53(3): 63-70.
[14] 阮树骅,瓮俊昊,毛麾,陈雪莲. 云安全风险评估度量模型[J]. 山东大学学报（理学版）, 2018, 53(3): 71-76.
[15] 康海燕,黄渝轩,陈楚翘. 基于视频分析的地理信息隐私保护方法[J]. 山东大学学报（理学版）, 2018, 53(1): 19-29.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] 杨永伟1,2,贺鹏飞2,李毅君2,3. BL-代数的严格滤子[J]. 山东大学学报（理学版）, 2014, 49(03): 63 -67 .
[2] 赵同欣1,刘林德1*,张莉1,潘成臣2,贾兴军1. 紫藤传粉昆虫与花粉多型性研究[J]. 山东大学学报（理学版）, 2014, 49(03): 1 -5 .
[3] 郭兰兰1,2,耿介1,石硕1,3,苑飞1,雷丽1,杜广生1*. 基于UDF方法的阀门变速关闭过程中的#br# 水击压强计算研究[J]. 山东大学学报（理学版）, 2014, 49(03): 27 -30 .
[4] 李敏1,2,李歧强1. 不确定奇异时滞系统的观测器型滑模控制器[J]. 山东大学学报（理学版）, 2014, 49(03): 37 -42 .
[5] 韩亚飞,伊文慧,王文波,王延平,王华田*. 基于高通量测序技术的连作杨树人工林土壤细菌多样性研究[J]. 山东大学学报（理学版）, 2014, 49(05): 1 -6 .
[6] 王开荣,高佩婷. 建立在DY法上的两类混合共轭梯度法[J]. 山东大学学报（理学版）, 2016, 51(6): 16 -23 .
[7] 马媛媛, 孟慧丽, 徐久成, 朱玛. 基于粒计算的正态粒集下的格贴近度[J]. 山东大学学报（理学版）, 2014, 49(08): 107 -110 .
[8] 徐俊峰. 关于复代数微分方程亚纯解的增长级[J]. J4, 2010, 45(6): 91 -93 .
[9] 裴胜玉，周永权*. 一种基于混沌变异的多目标粒子群优化算法[J]. J4, 2010, 45(7): 18 -23 .
[10] 杜吉祥1,2,余庆1,翟传敏1. 基于稀疏性约束非负矩阵分解的人脸年龄估计方法[J]. J4, 2010, 45(7): 65 -69 .
版权所有 © 2018 《山东大学学报(理学版)》编辑部　
地址: 山东省济南市山大南路27号山东大学中心校区（250100）　电话: +86-0531-88366917　E-mail: xblxb@sdu.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发