JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE) ›› 2026, Vol. 61 ›› Issue (7): 33-44.doi: 10.6040/j.issn.1671-9352.0.2025.070

• Mathematical Biology • Previous Articles    

Pattern dynamics analysis for a predator-prey system considering prey refuge and schooling behavior

JIANG Yuan, LIU Hui, OUYANG Miao, SHEN Pei   

  1. School of Information Engineering, Nanchang Hangkong University, Nanchang 330063, Jiangxi, China
  • Published:2026-07-01

Abstract: This paper considers a cross-diffusive predator-prey model incorporating refuge effects and schooling behavior, revealing the regulatory mechanism of refuge effects on both predator and prey populations. It is shown that as the refuge parameter increases, the schooling behavior of predator and prey populations becomes more pronounced. By employing a suitable refuge threshold parameter, the spatial dynamics of the system are investigated, and a series of spatiotemporal patterns are observed, including spot-like or hexagonal, stripe and mixed spot-stripe patterns. Specifically, linear stability analysis is applied to derive the conditions for Turing instability. Subsequently, multiple-scale analysis is utilized to derive amplitude equations near the Turing bifurcation critical point, and the mechanism and stability of pattern formation are studied. Theoretical results are verified through numerical simulations. Provides a theoretical basis for regulating population size and spatial patterns in species with collective behaviors.

Key words: refuge effects, predator-prey model, Turing stability, pattern dynamics

CLC Number: 

  • O175
[1] Couzin I D, Krause J. Self-organization and collective behavior in vertebrates[J]. Advances in the Study of Behavior, 2003:1-75.
[2] Holling C S. The components of predation as revealed by a study of small-mammal predation of the European pine sawfly[J]. The Canadian Entomologist, 1959, 91(5):293-320.
[3] 尚玉昌. 捕食者-猎物关系的理论和应用研究[J]. 应用生态学报,1990,1(2):177-185. Shang Yuchang. Theoretical and applied studies on predator-prey interaction[J]. Chinese Journal of Applied Ecology, 1990, 1(2):177-185.
[4] She W, Holyoak M, Gu J Y, et al. Abundant top predators increase species interaction network complexity in northeastern Chinese forests[J]. Journal of Animal Ecology, 2025, 94(4):745-759.
[5] Wang X Y, Zanette L, Zou X F. Modelling the fear effect in predator-prey interactions[J]. Journal of Mathematical Biology, 2016, 73(5):1179-1204.
[6] 王慧敏,张睿. 具有恐惧因素的Lotka-Volterra型捕食-食饵系统[J]. 数学的实践与认识,2021,51(11):300-305. Wang Huimin, Zhang Rui. Lotka-Volterra predator-prey system with fear factors[J]. Mathematics in Practice and Theory, 2021, 51(11):300-305.
[7] Hawlena D, Strickland M S, Bradford M A, et al. Fear of predation slows plant-litter decomposition[J]. Science, 2012, 336(6087):1434-1438.
[8] Huang Q H, Wang H, Lewis M A. The impact of environmental toxins on predator-prey dynamics[J]. Journal of Theoretical Biology, 2015, 378:12-30.
[9] Wang W, Liu S T, Tian D D, et al. Pattern dynamics in a toxin-producing phytoplankton-zooplankton model with additional food[J]. Nonlinear Dynamics, 2018, 94(1):211-228.
[10] Zhu H L, Zhang X B, Wang G L, et al. Effect of toxicant on the dynamics of a delayed diffusive predator-prey model[J]. Journal of Applied Mathematics and Computing, 2023, 69(1):355-379.
[11] 史明静,李泽妤,赵艺,等. 一类具有毒素影响和Allee效应的捕食-被捕食模型的定性分析[J]. 北京建筑大学学报,2021,10(11):3370-3376. Shi Mingjing, Li Zeyu, Zhao Yi, et al. Qualitative analysis of a prey-predator system with state feedback bilateral impulsive control and Allee effect in toxic environment[J]. Journal of Beijing University of Civil Engineering and Architecture, 2021, 10(11):3770-3376.
[12] Lotka A J. Elements of physical biology[M]. [S.l.] : Williams & Wilkins, 1925:76-88.
[13] Peterson A N, Soto A P, Mchenry M J. Pursuit and evasion strategies in the predator-prey interactions of FishesFree[J]. Integrative and Comparative Biology, 2021, 61(2):668-680.
[14] Skelhorn J, Halpin C G, Rowe C. Learning about aposematic prey[J]. Behavioral Ecology, 2016, 27(4):955-964.
[15] 庞茹一,陈巧玲. 具有食饵避难所的离散捕食者-食饵模型的余维二分支分析[J]. 数学季刊,2024,39(2):128-143. Pang Ruyi, Chen Qiaoling. Codimension-two bifurcations analysis of a discrete predator-prey model incorporating a prey refuge[J]. Chinese Quarterly Journal of Mathematics, 2024, 39(2):128-143.
[16] 曹倩,李艳玲,单炜华. 含有猎物避难所和恐惧效应的反应扩散捕食者-食饵模型的动力学[J]. 山东大学学报(理学版),2023,58(10):43-53. Cao Qian, Li Yanling, Shan Weihua. Dynamics of a reaction-diffusion predator-prey model incorporating prey refuge and fear effect[J]. Journal of Shandong University(Natural Science), 2023, 58(10):43-53.
[17] Parwaliya A, Singh A, Kumar A. Hopf bifurcation in a delayed prey-predator model with prey refuge involving fear effect[J]. International Journal of Biomathematics, 2024, 17(5):2350042.
[18] Mangal S, Misra O P, Dhar J. SIRS epidemic modeling using fractional-ordered differential equations: role of fear effect[J]. International Journal of Biomathematics, 2024, 17(5):2350044.
[19] 王新宇,薛小平. 无限图上Cucker-Smale模型的集群行为[J]. 中国科学:数学,2023,53(12):1799-1826. Wang Xinyu, Xue Xiaoping. The collective behavior of the Cucker-Smale model on the infinite graphs[J]. Scientia Sinica(Mathematica), 2023, 53(12):1799-1826.
[20] Wrona F J. Group size and predation risk: a field analysis of encounter and dilution effects[J]. The American Naturalist, 1991, 137(2):186-201.
[21] White J W. Can inverse density dependence at small spatial scales produce dynamic instability in animal populations?[J]. Theoretical Ecology, 2011, 4(3):357-370.
[22] Lima S L. Back to the basics of anti-predatory vigilance: the group-size effect[J]. Animal Behaviour, 1995, 49(1):11-20.
[23] Lima S L. The influence of models on the interpretation of vigilance[M] //Bekoff M, Jamieson D. Interpretation and Explanation in the Study of Animal Behavior. New York: Routledge, 2021:246-267.
[24] Vanag V K, Epstein I R. Cross-diffusion and pattern formation in reaction-diffusion systems[J]. Physical Chemistry Chemical Physics, 2009, 11(6):897-912.
[25] Gambino G, Lombardo M C, Sammartino M. Pattern formation driven by cross-diffusion in a 2D domain[J]. Nonlinear Analysis: Real World Applications, 2013, 14(3):1755-1779.
[26] Guin L N. Existence of spatial patterns in a predator-prey model with self-and cross-diffusion[J]. Applied Mathematics and Computation, 2014, 226:320-335.
[27] Wang F T, Yang R Z. Spatial pattern formation driven by the cross-diffusion in a predator-prey model with Holling type functional response[J]. Chaos, Solitons & Fractals, 2023, 174:113890.
[28] 夏蒙棼. 斑图动力学:非线性科学专题之九[J]. 物理通报,1999(4):3-6. Xia Mengfen. Pattern dynamics-the ninth special topic of nonlinear science[J]. Physics Bulletin, 1999(4):3-6.
[29] 张双红,刘思宏,徐袁媛,等. 具有食饵避难所和恐惧效应的比率依赖型捕食者-食饵模型的稳定性[J]. 吉林师范大学学报(自然科学版),2025,46(1):43-54. Zhang Shuanghong, Liu Sihong, Xu Yuanyuan, et al. Stability of ratio dependent predator-prey models with prey refuge and fear effects[J]. Journal of Jilin Normal University(Natural Science Edition), 2025, 46(1):43-54.
[30] 林思佳,陈凤德,陈尚铭,等. 具有避难所的离散捕食者-食饵系统的动力学行为分析[J]. 福州大学学报(自然科学版),2023,51(6):735-741. Lin Sijia, Chen Fengde, Chen Shangming, et al. Dynamical behavior analysis of a discrete predator-prey model with prey refuge[J]. Journal of Fuzhou University(Natural Science Edition), 2023, 51(6):735-741.
[31] Molla H, Rahman M S, Sarwardi S. Dynamical study of a prey-predator model incorporating nonlinear prey refuge and additive Allee effect acting on prey species[J]. Modeling Earth Systems and Environment, 2021, 7(2):749-765.
[32] Chakraborty B, Baek H, Bairagi N. Diffusion-induced regular and chaotic patterns in a ratio-dependent predator-prey model with fear factor and prey refuge[J]. Chaos: An Interdisciplinary Journal of Nonlinear Science, 2021, 31(3):033128.
[33] Han R J, Guin L N, Acharya S. Complex dynamics in a reaction-cross-diffusion model with refuge depending on predator-prey encounters[J]. The European Physical Journal Plus, 2022, 137(1):134.
[34] Zhang H S, Qi H K. Hopf bifurcation analysis of a predator-prey model with prey refuge and fear effect under non-diffusion and diffusion[J]. Qualitative Theory of Dynamical Systems, 2023, 22(4):135.
[35] Chatterjee A, Abbasi M A, Venturino E, et al. A predator-prey model with prey refuge: under a stochastic and deterministic environment[J]. Nonlinear Dynamics, 2024, 112(15):13667-13693.
[36] Zhou Y, Yan X P, Zhang C H. Turing patterns induced by self-diffusion in a predator-prey model with schooling behavior in predator and prey[J]. Nonlinear Dynamics, 2021, 105(4):3731-3747.
[37] 周艳,张存华. 具有集群行为的捕食者-食饵反应扩散系统的稳定性和Turing不稳定性[J]. 山东大学学报(理学版),2021,56(7):73-81. Zhou Yan, Zhang Cunhua. Stability and Turing instability of a predator-prey reaction-diffusion system with schooling behavior[J]. Journal of Shandong University(Natural Science), 2021, 56(7):73-81.
[38] Meng X Y, Wang J G. Dynamical analysis of a delayed diffusive predator-prey model with schooling behaviour and Allee effect[J]. Journal of Biological Dynamics, 2020, 14(1):826-848.
[39] Wang W, Liu S T, Liu Z B, et al. Pattern dynamics in a predator-prey model with schooling behavior and cross-diffusion[J]. International Journal of Bifurcation and Chaos, 2019, 29(11):1950146.
[40] Manna D, Maiti A, Samanta G P. Analysis of a predator-prey model for exploited fish populations with schooling behavior[J]. Applied Mathematics and Computation, 2018, 317:35-48.
[41] 欧阳颀. 非线性科学与斑图动力学导论[M]. 北京:北京大学出版社,2010:130-180. Ouyang Qi. Nonlinear science and the pattern dynamics introduction[M]. Beijing: Peking University Press, 2010:130-180.
[1] MA Tiantian, LI Shanbing. Coexistence solutions of a predator-prey model with Allee effect and density-dependent diffusion in the predator [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2025, 60(4): 84-92.
[2] XU Yingting, ZHAO Jiantao, WEI Xin. Dynamical analysis in a diffusive predator-prey model with cooperative hunting and group defense [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2025, 60(4): 104-117.
[3] Yufeng ZHAO,Guirong LIU. Stationary distribution and probability density function of a stochastic predation system [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2024, 59(10): 74-88.
[4] Qian CAO,Yanling LI,Weihua SHAN. Dynamics of a reaction-diffusion predator-prey model incorporating prey refuge and fear effect [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2023, 58(10): 43-53.
[5] Hang ZHANG,Yujuan JIAO,Jinmiao YANG. Existence of traveling wave solutions for a diffusive predator-prey model [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2023, 58(10): 97-105.
[6] SUN Chun-jie, ZHANG Cun-hua. Stability and Turing instability in the diffusive Beddington-DeAngelis-Tanner predator-prey model [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2022, 57(9): 83-90.
[7] HAN Zhuo-ru, LI Shan-bing. Positive solutions of predator-prey model with spatial heterogeneity and hunting cooperation [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2022, 57(7): 35-42.
[8] WANG Jing, FU Sheng-mao. Effect of fear factor on a predator-prey model with defense mechanish [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2020, 55(3): 121-126.
[9] LI Hai-xia. Qualitative analysis of a diffusive predator-prey model with density dependence [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2019, 54(9): 54-61.
[10] FU Juan, ZHANG Rui, WANG Cai-jun, ZHANG Jing. The stability of a predator-prey diffusion model with Beddington-DeAngelis functional response [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2016, 51(11): 115-122.
[11] ZHANG Li-na, WU Shou-yan. Global behavior of solutions for a modified LeslieGower #br# predator-prey system with diffusion [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2014, 49(1): 86-91.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!