/ 详情

关于插件扩展的思考

Done
owner
Opened this issue  
2016-01-06 13:30

放弃Spiderman1的Plugin思路,根据这几年下来的实际经验,采取如下:

  • 配置:AbstractConfBuilder, XMLConfBuilder, YAMLConfBuilder, DbConfBuilder, AnnoConfBuilder
  • 下载:HttpClientDownloader, WebBrowserDownloader
  • 解析:TextParser, HtmlCleanerParser, XMLParser, JsoupParser, WebBrowserParser
  • 队列:MemQueue, DiskQueue, ZBusQueue, KafkaQueue
  • 存储:MemDb, MapDb, ZBusRpcDb, RedisDb

Comments (3)

  1. 增加CheckableQueue,可以支持队列元素入队时执行检查逻辑
    下一步:
    WebBrowserDownloader, JsoupExtractor

2.已经增加并实现了WebDriverDownloader(即WebBrowserDownloader)

Status changed to closed

Sign in to comment

状态
Assignees
Milestones
Pull Requests
关联的 Pull Requests 被合并后可能会关闭此 issue
Branches
Planed to start   -   Planed to end
-
Top level
Priority
参与者(1)
117 l weiwei 1578913730