Project Name	Stars	Most Recent Commit	Total Releases	Latest Release	Open Issues	License	Language
Web Scraper Gcp	15	4 years ago			1	mit	JavaScript
Scrape all the pages and links of a given domain and write the results to Google Cloud BigQuery.
Crawl	12	4 years ago	5	November 17, 2019	9	mit	Jupyter Notebook
A concurrent crawler that minimizes memory use. Output suitable for use with BigQuery.

Alternatives To Crawl

Select To Compare

Web Scraper Gcp ⭐ 15

Scrape all the pages and links of a given domain and write the results to Google Cloud BigQuery.

most recent commit 4 years ago

Crawl ⭐ 12

A concurrent crawler that minimizes memory use. Output suitable for use with BigQuery.

total releases 5most recent commit 4 years ago

Suggest An Alternative To crawl

Alternative Project Comparisons

Crawl vs Web Scraper Gcp

Popular Bigquery Projects

Graphql Engine ⭐ 30,631

Blazing fast, instant realtime GraphQL APIs on your DB with fine grained access control, also trigger webhooks on database events.

dependent packages 3total releases 17latest release June 22, 2022most recent commit 3 months ago

npm @install-binary/hasura-darwin-x64} Downloads

Redash ⭐ 24,479

Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.

dependent packages 3total releases 2latest release May 05, 2020most recent commit 3 months ago

Cube ⭐ 16,806

📊 Cube — The Semantic Layer for Building Data Applications

dependent packages 97total releases 33latest release November 24, 2023most recent commit 3 months ago

Airbyte ⭐ 12,918

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

dependent packages 11total releases 311latest release December 08, 2023most recent commit 3 months ago

Doris ⭐ 11,243

Apache Doris is an easy-to-use, high performance and unified analytics database.

total releases 8latest release September 27, 2023most recent commit 18 days ago

Popular Crawler Projects

Scrapy ⭐ 49,918

Scrapy, a fast high-level web crawling & scraping framework for Python.

dependent packages 445total releases 96latest release September 18, 2023most recent commit 3 months ago

Lux ⭐ 24,752

👾 Fast and simple video download library and CLI tool written in Go

dependent packages 8total releases 40latest release November 06, 2023most recent commit 25 days ago

Colly ⭐ 21,902

Elegant Scraper and Crawler Framework for Golang

dependent packages 328total releases 22latest release March 08, 2022most recent commit a month ago

Easyspider ⭐ 20,149

A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/爬虫软件，可以无代码图形化

most recent commit 24 days ago

Proxy_pool ⭐ 19,442

Python ProxyPool for web spider

most recent commit 4 months ago

Popular Data Processing Categories