Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

SethWen/DistributedCrawler

Open more actions menu

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
8 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

设计思路

本项目由 Node.js 开发。该框架主要适用高并发,但每次爬取数据量不大的分布式爬虫

用到的技术包括

  • PM2 负责开启和监控应用;
  • RabbitMQ 负责任务调度;
  • Server 由 Express 编写

项目说明

详细的说明请转到个人博客查看。欢迎您拍砖和 star。

About

高并发但每次爬取数据量不大的分布式爬虫。

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Morty Proxy This is a proxified and sanitized view of the page, visit original site.