Webster is A Powerful and Extensible Web Crawling Framework for Node.js application. You can use Webster to crawl websites and extract structured data from their pages.
pull the example docker image:
docker pull zhuyingda/webster-demo docker run -it zhuyingda/webster-demo
node demo_producer.js env MOD=debug node demo_consumer.js
Or you can deploy on Docker.
npm install webster
sudo apt install chromium-browser chromium-codecs-ffmpeg env MOD=debug EXE_PATH=EXE_PATH=/usr/bin/chromium-browser node demo_consumer.js
You can see more details from here.
This project exists thanks to all the people who contribute. [Contribute].
Become a financial contributor and help us sustain our community. [Contribute]
Support this project with your organization. Your logo will show up here with a link to your website. [Contribute]
Copyright (c) 2017-present, Yingda (Sugar) Zhu