J4 ›› 2011, Vol. 46 ›› Issue (5): 34-38.
• Articles •
Based on the Web surf behaviors of human beings, crawling directions are restricted by extracted crawling subpages, and the associated relationships of crosspage compound object are realized through the properties′ inheritance between crawl datum. Then, the generalized crawl process with deep directional collection support is designed and implemented. Experimental results about the hot posts of the Tianya site show that this method can achieve data collection of complicated objects without changing the main procedure, and has high collection efficiency.
deep collection; directional web crawler; public web opinion
XIA Tian1,2. Deep directional collection of Web data[J].J4, 2011, 46(5): 34-38.
Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks