Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Automation Web Crawler Open Source Projects

Open source projects categorized as Automation Web Crawler
Categories > Data Processing > Web Crawler
Huginn ⭐ 41,465
Create agents that monitor and act on your behalf. Your agents are standing by!
dependent packages 52total releases 8most recent commit 16 days agorubygems huginn_agent} Downloads
Crawlee ⭐ 12,106
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
dependent packages 42total releases 747most recent commit 2 days agonpm crawlee} Downloads
Autoscraper ⭐ 5,159
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
dependent packages 1total releases 16most recent commit a year agopypi autoscraper} Downloads
Rod ⭐ 4,505
A Devtools driver for web automation and scraping
dependent packages 140total releases 406most recent commit 3 months ago
Browser Fingerprinting ⭐ 3,353
Analysis of Bot Protection systems with available countermeasures 🚿. How to defeat anti-bot system 👻 and get around browser fingerprinting scripts 🕵️‍♂️ when scraping the web?
most recent commit a year ago
30 Days Of Python ⭐ 1,926
Learn Python for the next 30 (or so) Days.
most recent commit a year ago
Rpa ⭐ 858
UI.Vision: Open-Source RPA Software (formerly Kantu) - Modern Robotic Process Automation with Selenium IDE++
most recent commit a year ago
Easyapplyjobsbot ⭐ 247
A python bot to automatically apply all Linkedin,Glassdoor, etc Easy Apply jobs based on your preferences. Auto login, auto fill additional questions, apply automatically!
most recent commit 4 months ago
Twitter Scraper Selenium ⭐ 207
Python's package to scrap Twitter's front-end easily
total releases 27most recent commit 7 months ago
Ayakashi ⭐ 177
⚡ Ayakashi.io - The next generation web scraping framework
dependent packages 2total releases 40most recent commit 10 months agonpm ayakashi} Downloads
Helena ⭐ 165
A Chrome extension for writing custom web scraping programs and web automation programs. Just demonstrate how to collect the first row of data, then let the extension write the program for collecting all rows.
most recent commit 3 years ago
Clicknium Docs ⭐ 127
A next-generation GUI automation framework for Web and Desktop Application Testing and Automation.
most recent commit 10 months ago
Educative.io_scraper ⭐ 117
Educative.io Course Downloader developed using Python and Selenium. Refer Readme.md for setup instructions.
most recent commit 3 months ago
Cowin Vaccine Notifier ⭐ 97
Automated Python Script to retrieve vaccine slots availability and get notified when a slot is available.
most recent commit 3 years ago
Senpwai ⭐ 97
A desktop app for tracking and batch downloading anime
most recent commit 3 months ago
Anmeldung Berlin ⭐ 91
This app will find and book any service.berlin.de appointment that can be booked online.
most recent commit 3 months ago
Udemy_bot ⭐ 88
An automation bot for free Udemy courses
most recent commit 3 years ago
Instatools ⭐ 83
🧰 A collection of tools built for automating tasks on Instagram.
most recent commit 3 months ago
Browser Pool ⭐ 77
A Node.js library to easily manage and rotate a pool of web browsers, using any of the popular browser automation libraries like Puppeteer, Playwright, or SecretAgent.
dependent packages 7total releases 82most recent commit a year agonpm browser-pool} Downloads
Clauneck ⭐ 57
A tool for scraping emails, social media accounts, and much more information from websites using Google Search Results.
total releases 2most recent commit 9 months ago
Animex V2 ⭐ 52
animeX is a CLI tool for downloading anime directly to your PC
most recent commit 2 years ago
Instagram Bot ⭐ 48
🤖 Python bot to view stories, like and comment on Instagram
most recent commit 3 years ago
Android Web Scraper ⭐ 45
Android Web Scraper is a simple library for android web automation. You can perform web task in background to fetch website data programmatically.
most recent commit 3 years ago
The 6 Most Depended On Automation Web Crawler Open Source Projects
Rod ⭐ 4,505
A Devtools driver for web automation and scraping
dependent packages 140total releases 406latest release November 06, 2023most recent commit 3 months ago
Huginn ⭐ 41,465
Create agents that monitor and act on your behalf. Your agents are standing by!
dependent packages 52total releases 8latest release September 22, 2017most recent commit 16 days agorubygems huginn_agent} Downloads
Ayakashi ⭐ 177
:zap: Ayakashi.io - The next generation web scraping framework
dependent packages 2total releases 40latest release June 29, 2023most recent commit 10 months agonpm ayakashi} Downloads
Autoscraper ⭐ 5,159
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
dependent packages 1total releases 16latest release July 17, 2022most recent commit a year agopypi autoscraper} Downloads
The 10 Latest Releases In Automation Web Crawler Open Source Projects
Botcity Framework Web Python ⭐ 28
BotCity Framework Web - Python
total releases 22latest release November 21, 2023most recent commit 5 months ago
Rod ⭐ 4,505
A Devtools driver for web automation and scraping
dependent packages 140total releases 406latest release November 06, 2023most recent commit 3 months ago
Twitter Scraper Selenium ⭐ 207
Python's package to scrap Twitter's front-end easily
total releases 27latest release October 02, 2023most recent commit 7 months ago
Ayakashi ⭐ 177
:zap: Ayakashi.io - The next generation web scraping framework
dependent packages 2total releases 40latest release June 29, 2023most recent commit 10 months agonpm ayakashi} Downloads
Autoscraper ⭐ 5,159
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
dependent packages 1total releases 16latest release July 17, 2022most recent commit a year agopypi autoscraper} Downloads
Animex Pack ⭐ 11
A lightweight Python library (and command-line utility) for downloading anime.
total releases 6latest release May 21, 2020most recent commit 4 years ago
Booking_scraper ⭐ 12
A booking.com Web Scraper for Data Mining/Harvesting and Automation
total releases 1latest release May 01, 2020most recent commit 3 years ago
Popular Artificial Intelligence Categories
Artificial Intelligence
Robot
Ros
Discord Bot
Home Assistant
Telegram Bot
Chatbot
Robotics
Object Detection
Home Automation
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.