Fetch the repository succeeded.
Fork this repository
Loading
Sync repository?
This action will force synchronization from rock117/web-crawler, which will overwrite any changes that you have made since you forked the repository, and can not be recovered!!!
Synchronous operation will process in the background and will refresh the page when finishing processing. Please be patient.
Download ZIP
Login prompt
This operation requires login to the code cloud account. Please log in before operating.
Loading...
README.md
#web-crawler
一个简单的web爬虫，采用scala + akka实现，该爬虫可以指定过滤规则，存储处理（文件存储，数据库存储。。。），例子如下
new Crawler().source(ArrayBuffer("http://money.163.com/stock/"))
.processor(FileStore("d:/crawl-sites").process)
.filter((url: String) => {
url.contains("163.com")
}).start
Sign in for post a comment
Report
Please fill in the reason for the report carefully. Provide as detailed a description as possible.
Please select a report type
Comments ( 0 )