Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

Crawlkit

A crawler based on Phantom. Allows discovery of dynamic content and supports custom scrapers.
Stars
23
License
mit
Open Issues
1
Most Recent Commit
7 years ago
Programming Language
JavaScript
Monthly Downloads
npm crawlkit Downloads
Dependent Repos
6
Dependent Packages
5
Total Releases
34
Latest Release
May 23, 2016
Categories
Repo
Alternatives To Crawlkit
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Trafilatura2,447pypi trafilatura} Downloads663 months ago39November 29, 202366gpl-3.0Python
Python & command-line tool to gather text on the Web: web crawling/scraping, extraction of text, metadata, comments
Bing Ip2hosts91
3 years agogpl-3.0Shell
bingip2hosts is a Bing.com web scraper that discovers websites by IP address
Tiktok Trending Data60
3 months ago1
Scraping the TikTok discovery web API every 15 minutes using Github Actions to view changes
Crawlkit23npm crawlkit} Downloads657 years ago34May 23, 20161mitJavaScript
A crawler based on Phantom. Allows discovery of dynamic content and supports custom scrapers.
Pukpuk9
2 years ago18August 05, 2022mitPython
HTTP discovery and change monitoring tool
Scraper6
6 years agomitGo
Example of using prometheus discovery and scraping library
Torrentd5
a year agomitGo
Torrent discovery and tracking server
Locust5npm @achannarasappa/locust} Downloads24 years ago4June 09, 2020mitJavaScript
Distributed web data discovery and collection framework built for serverless
Alternatives To Crawlkit
Select To Compare
Pukpuk ⭐ 9
HTTP discovery and change monitoring tool
total releases 18most recent commit 2 years ago
Locust ⭐ 5
Distributed web data discovery and collection framework built for serverless
dependent packages 2total releases 4most recent commit 4 years agonpm @achannarasappa/locust} Downloads


Suggest An Alternative To crawlkit
Alternative Project Comparisons
Crawlkit vs Trafilatura
Crawlkit vs Bing Ip2hosts
Crawlkit vs Tiktok Trending Data
Crawlkit vs Pukpuk
Crawlkit vs Scraper
Crawlkit vs Torrentd
Crawlkit vs Locust
Popular Scraper Projects
Scrapy ⭐ 49,918
Scrapy, a fast high-level web crawling & scraping framework for Python.
dependent packages 445total releases 96latest release September 18, 2023most recent commit 3 months agopypi Scrapy} Downloads
Huginn ⭐ 41,465
Create agents that monitor and act on your behalf. Your agents are standing by!
dependent packages 52total releases 8latest release September 22, 2017most recent commit 15 days agorubygems huginn_agent} Downloads
Devdocs ⭐ 33,315
API Documentation Browser
most recent commit 3 months ago
Cheerio ⭐ 27,702
The fast, flexible, and elegant library for parsing and manipulating HTML and XML.
dependent packages 20,519total releases 70latest release June 26, 2022most recent commit 14 days agonpm cheerio} Downloads
Lux ⭐ 24,752
👾 Fast and simple video download library and CLI tool written in Go
dependent packages 8total releases 40latest release November 06, 2023most recent commit 22 days ago
Popular Discovery Projects
Theharvester ⭐ 10,249
E-mails, subdomains and names Harvester - OSINT
total releases 2latest release February 04, 2019most recent commit 15 days ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Related Searches
Javascript Scraper
Javascript Phantomjs
Javascript Crawler
Javascript Accessibility
Javascript Discovery
Scraper Crawler
Javascript Redirect
Crawler Phantomjs
Scraper Phantomjs
Accessibility Axe

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Javascript
Scraper
Crawler
Accessibility
Discovery
Phantomjs
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.