Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Bot Crawler Open Source Projects

Open source projects categorized as Bot Crawler
Browser Fingerprinting ⭐ 3,353
Analysis of Bot Protection systems with available countermeasures 🚿. How to defeat anti-bot system 👻 and get around browser fingerprinting scripts 🕵️‍♂️ when scraping the web?
most recent commit a year ago
Crawler Detect ⭐ 1,842
🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent
dependent packages 110total releases 156most recent commit 5 months agopackagist jaybizzle/crawler-detect} Downloads
Vulnx ⭐ 1,763
vulnx 🕷️ an intelligent Bot, Shell can achieve automatic injection, and help researchers detect security vulnerabilities CMS system. It can perform a quick CMS security detection, information collection (including sub-domain name, ip address, country information, organizational information and time zone, etc.) and vulnerability scanning.
total releases 2most recent commit 10 months ago
Lightnovel Crawler ⭐ 1,185
Generate and download e-books from online sources.
dependent packages 1total releases 30most recent commit 3 months agopypi ebook-crawler} Downloads
Crawler User Agents ⭐ 1,045
Syntactic patterns of HTTP user-agents used by bots / robots / crawlers / scrapers / spiders. pull-request welcome ⭐
dependent packages 8total releases 118most recent commit 3 months agonpm crawler-user-agents} Downloads
Seccrawler ⭐ 777
一个方便安全研究人员获取每日安全日报的爬虫和推送程序，目前爬取范围包括先知社区、安全客、Seebug Paper、跳跳糖、奇安信攻防社区、棱角社区以及绿盟、腾讯玄武、天融信、360等实验室博客，持续更新
most recent commit 4 months ago
Nginx Badbot Blocker ⭐ 759
Block bad, possibly even malicious web crawlers (automated bots) using Nginx
most recent commit 4 years ago
Device_detector ⭐ 711
DeviceDetector is a precise and fast user agent parser and device detector written in Ruby
dependent packages 5total releases 26most recent commit 4 months agorubygems device_detector} Downloads
Scavenger ⭐ 384
Crawler (Bot) searching for credential leaks on paste sites.
most recent commit 3 years ago
Ngcbot ⭐ 341
一个基于✨HOOK机制的微信机器人，支持🌱安全新闻定时推送【FreeBuf，先知，安全客，奇安信攻 🐛美女视频，⚡美女图片，👯帮助菜单。📫 支持积分功能，⚡支持自动拉人，⚡检测广告，🌱自动群发，👯Ai回复，😄自定义程度丰富，小白也可轻
most recent commit 3 months ago
Archivebot ⭐ 328
ArchiveBot, an IRC bot for archiving websites
most recent commit 4 months ago
Bad Bot Blocker ⭐ 327
Block bad, possibly even malicious web crawlers (automated bots) using htaccess.
most recent commit 9 years ago
Fakebrowser ⭐ 290
🤖 Fake fingerprints to bypass anti-bot systems. Simulate mouse and keyboard operations to make behavior like a real person.
dependent packages 2total releases 54most recent commit 2 years agonpm fakebrowser} Downloads
Line Bot Tutorial ⭐ 280
line-bot-tutorial use python flask
most recent commit 2 years ago
Laravel Crawler Detect ⭐ 262
A Laravel wrapper for CrawlerDetect - the web crawler detection library
dependent packages 16total releases 6most recent commit 2 years agopackagist jaybizzle/laravel-crawler-detect} Downloads
Instagram Bot ⭐ 215
An Instagram bot developed using the Selenium Framework
most recent commit 3 years ago
Voight Kampff ⭐ 171
Voight-Kampff is a Ruby gem that detects bots, spiders, crawlers and replicants
dependent packages 3total releases 17most recent commit a year agorubygems voight_kampff} Downloads
Onegram ⭐ 136
This repository is no longer maintained.
total releases 21most recent commit 6 years ago
Nginx Blacklist ⭐ 118
Block web crawlers (automated bots) from accessing your website served with nginx.
most recent commit 10 years ago
Crawler_detect ⭐ 106
Ruby gem to detect bots and crawlers via the user agent
total releases 23most recent commit 4 months ago
Human_power ⭐ 97
Easy generation of robots.txt. Force the robots into submission!
total releases 6most recent commit 9 years ago
Bots Zoo ⭐ 90
most recent commit 5 months ago
Es6 Crawler Detect ⭐ 88
🕷️ This is an ES6 adaptation of the original PHP library CrawlerDetect, this library will help you detect bots/crawlers/spiders vie the useragent.
dependent packages 4total releases 15most recent commit 5 months agonpm es6-crawler-detect} Downloads
The 10 Most Depended On Bot Crawler Open Source Projects
Crawler Detect ⭐ 1,842
🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent
dependent packages 110total releases 156latest release July 21, 2023most recent commit 5 months agopackagist jaybizzle/crawler-detect} Downloads
Laravel Crawler Detect ⭐ 262
A Laravel wrapper for CrawlerDetect - the web crawler detection library
dependent packages 16total releases 6latest release July 06, 2022most recent commit 2 years agopackagist jaybizzle/laravel-crawler-detect} Downloads
Crawler User Agents ⭐ 1,045
Syntactic patterns of HTTP user-agents used by bots / robots / crawlers / scrapers / spiders. pull-request welcome :star:
dependent packages 8total releases 118latest release November 20, 2023most recent commit 3 months agonpm crawler-user-agents} Downloads
Device_detector ⭐ 711
DeviceDetector is a precise and fast user agent parser and device detector written in Ruby
dependent packages 5total releases 26latest release December 01, 2023most recent commit 4 months agorubygems device_detector} Downloads
Express Nobots ⭐ 8
Keep Bots And Crawlers Away From Your Express App
dependent packages 4total releases 6latest release December 11, 2019most recent commit 4 years agonpm express-nobots} Downloads
Voight Kampff ⭐ 171
Voight-Kampff is a Ruby gem that detects bots, spiders, crawlers and replicants
dependent packages 3total releases 17latest release September 30, 2018most recent commit a year agorubygems voight_kampff} Downloads
Local Api Client Typescript ⭐ 54
Official JavaScript/TypeScript library for interacting with Kameleo Client
dependent packages 2total releases 11latest release July 20, 2023most recent commit 9 months agonpm @kameleo/local-api-client} Downloads
Is Google ⭐ 81
Verify that a request is from Google crawlers using Google's DNS verification steps
dependent packages 2total releases 3latest release April 27, 2018most recent commit 5 years agonpm is-google} Downloads
Rpg Telegram Bot ⭐ 10
Text based RPG Dungeon crawler for telegram
dependent packages 2total releases 10latest release July 29, 2019most recent commit 3 years agonpm @bjorky/tiny} Downloads
The 10 Latest Releases In Bot Crawler Open Source Projects
Device_detector ⭐ 711
DeviceDetector is a precise and fast user agent parser and device detector written in Ruby
dependent packages 5total releases 26latest release December 01, 2023most recent commit 4 months agorubygems device_detector} Downloads
Crawler User Agents ⭐ 1,045
Syntactic patterns of HTTP user-agents used by bots / robots / crawlers / scrapers / spiders. pull-request welcome :star:
dependent packages 8total releases 118latest release November 20, 2023most recent commit 3 months agonpm crawler-user-agents} Downloads
Legitbot ⭐ 19
🤔 Is this Web request from a real search engine🕷 or from an impersonating agent 🕵️‍♀️?
total releases 44latest release November 17, 2023most recent commit 4 months ago
Laravel Block Bots ⭐ 34
Block crawlers and high traffic users on your site by IP using Redis
total releases 28latest release September 21, 2023most recent commit 7 months ago
Crawlerdetect ⭐ 38
🕷CrawlerDetect is a Python class for detecting bots/crawlers/spiders via the user agent
dependent packages 1total releases 8latest release August 17, 2023most recent commit 3 months agopypi crawlerdetect} Downloads
Crawler Detect ⭐ 1,842
🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent
dependent packages 110total releases 156latest release July 21, 2023most recent commit 5 months agopackagist jaybizzle/crawler-detect} Downloads
Local Api Client Python ⭐ 53
Official Python library for interacting with Kameleo Client
total releases 9latest release July 21, 2023most recent commit 9 months ago
Local Api Client Typescript ⭐ 54
Official JavaScript/TypeScript library for interacting with Kameleo Client
dependent packages 2total releases 11latest release July 20, 2023most recent commit 9 months agonpm @kameleo/local-api-client} Downloads
Crawlerdetect ⭐ 43
Golang module to detect bots and crawlers via the user agent
dependent packages 2total releases 20latest release March 23, 2023most recent commit 4 months ago
Popular Artificial Intelligence Categories
Artificial Intelligence
Robot
Ros
Discord Bot
Home Assistant
Telegram Bot
Chatbot
Robotics
Object Detection
Home Automation
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.