Awesome Open Source

Programming Languages

The Top 23 Golang Crawler Spider Open Source Projects

Open source projects categorized as Golang Crawler Spider

Categories > Data Processing > Crawler

Categories > Programming Languages > Golang

Categories > Data Processing > Spider

Colly ⭐ 21,902

Elegant Scraper and Crawler Framework for Golang

dependent packages 328total releases 22most recent commit a month ago

Crawlab ⭐ 10,521

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

total releases 1most recent commit 4 months ago

BitTorrent DHT Protocol && DHT Spider.

dependent packages 2total releases 1most recent commit 3 years ago

Torbot ⭐ 2,338

Dark Web OSINT Tool

most recent commit 4 months ago

Gospider ⭐ 2,190

Gospider - Fast web spider written in Go

dependent packages 1total releases 17most recent commit 6 months ago

Geziyor ⭐ 1,892

Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.

dependent packages 11total releases 6most recent commit a year ago

Go_spider ⭐ 1,629

[爬虫框架 (golang)] An awesome Go concurrent Crawler(spider) framework. The crawler is flexible and modular. It can be expanded to an Individualized crawler easily or you can use the default crawl components only.

dependent packages 2total releases 1most recent commit 6 years ago

Creeper ⭐ 762

🐾 Creeper - The Next Generation Crawler Framework (Go)

total releases 1most recent commit 7 years ago

Fictiondown ⭐ 601

小说下载|小说爬取|起点|笔趣阁|导出Markdown|导出txt|转换epub|广告过滤|自动校对

total releases 5most recent commit 4 months ago

OSINT Swiss Army Knife

most recent commit 3 years ago

Go_jobs ⭐ 536

带你了解一下Golang的市场行情

total releases 1most recent commit 4 years ago

Ants Go ⭐ 368

open source, distributed, restful crawler engine in golang

total releases 1most recent commit 8 years ago

Gospider ⭐ 354

golang实现的爬虫框架，使用者只需关心页面规则，提供web管理界面。基于colly开发。

total releases 3most recent commit a year ago

Webpalm ⭐ 295

WebPalm is a powerful command-line tool for website mapping and web scraping. With its recursive approach, it can generate a complete tree of all webpages and their links on a website. It can also extract data from the body of each page using regular expressions, making it an ideal tool for web scraping and data extraction.

dependent packages 2total releases 8most recent commit 6 months ago

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

total releases 6most recent commit 3 years ago

A web crawler for Go

total releases 8most recent commit a year ago

Go Movies ⭐ 232

golang spider Crawler 爬虫电影

most recent commit 5 months ago

golang light-weight image crawler

dependent packages 1total releases 1most recent commit 6 years ago

A fast tool to fetch URLs from HTML attributes by crawl-in.

total releases 2most recent commit a year ago

Digger is a powerful and flexible web crawler implemented by pure golang

most recent commit 2 years ago

Antch, a fast, powerful and extensible web crawling & scraping framework for Go

dependent packages 1total releases 1most recent commit 4 years ago

Goribot ⭐ 162

[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。

total releases 11most recent commit 4 years ago

Domain names collector - Crawl websites and collect domain names along with their availability status.

total releases 3most recent commit 9 months ago

The 10 Most Depended On Golang Crawler Spider Open Source Projects

Colly ⭐ 21,902

Elegant Scraper and Crawler Framework for Golang

dependent packages 328total releases 22latest release March 08, 2022most recent commit a month ago

Geziyor ⭐ 1,892

Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.

dependent packages 11total releases 6latest release April 19, 2021most recent commit a year ago

💐Marmot A Golang HTTP Download

dependent packages 2total releases 13latest release November 25, 2022most recent commit a year ago

Crawlerdetect ⭐ 43

Golang module to detect bots and crawlers via the user agent

dependent packages 2total releases 20latest release March 23, 2023most recent commit 4 months ago

Webpalm ⭐ 295

WebPalm is a powerful command-line tool for website mapping and web scraping. With its recursive approach, it can generate a complete tree of all webpages and their links on a website. It can also extract data from the body of each page using regular expressions, making it an ideal tool for web scraping and data extraction.

dependent packages 2total releases 8latest release June 19, 2023most recent commit 6 months ago

Go_spider ⭐ 1,629

[爬虫框架 (golang)] An awesome Go concurrent Crawler(spider) framework. The crawler is flexible and modular. It can be expanded to an Individualized crawler easily or you can use the default crawl components only.

dependent packages 2total releases 1latest release August 09, 2015most recent commit 6 years ago

BitTorrent DHT Protocol && DHT Spider.

dependent packages 2total releases 1latest release December 19, 2020most recent commit 3 years ago

Gospider ⭐ 70

⚡ Light weight Golang spider framework | 轻量的 Golang 爬虫框架

dependent packages 1total releases 5latest release March 16, 2021most recent commit 3 years ago

golang light-weight image crawler

dependent packages 1total releases 1latest release September 09, 2018most recent commit 6 years ago

Antch, a fast, powerful and extensible web crawling & scraping framework for Go

dependent packages 1total releases 1latest release December 25, 2017most recent commit 4 years ago

The 10 Latest Releases In Golang Crawler Spider Open Source Projects

Go Crawler ⭐ 33

A web crawling framework implemented in Golang, it is simple to write and delivers powerful performance. It comes with a wide range of practical middleware and supports various parsing and storage methods. Additionally, it supports distributed deployment. 基于golang实现的爬虫框架，编写简单，性能强劲。内置了丰富的实用中间件，支持多种解析、保存方式，

total releases 26latest release November 23, 2023most recent commit 5 months ago

Gathertool ⭐ 46

gathertool是golang脚本化开发库，目的是提高对应场景程序开发的效率；轻量级爬虫库，接口

total releases 58latest release November 01, 2023most recent commit 5 months ago

Webpalm ⭐ 295

WebPalm is a powerful command-line tool for website mapping and web scraping. With its recursive approach, it can generate a complete tree of all webpages and their links on a website. It can also extract data from the body of each page using regular expressions, making it an ideal tool for web scraping and data extraction.

dependent packages 2total releases 8latest release June 19, 2023most recent commit 6 months ago

Tegenaria ⭐ 8

Tegenaria is a crawler framework based on golang

total releases 27latest release March 23, 2023most recent commit a year ago

Crawlerdetect ⭐ 43

Golang module to detect bots and crawlers via the user agent

dependent packages 2total releases 20latest release March 23, 2023most recent commit 4 months ago

💐Marmot A Golang HTTP Download

dependent packages 2total releases 13latest release November 25, 2022most recent commit a year ago

Domain names collector - Crawl websites and collect domain names along with their availability status.

total releases 3latest release August 15, 2022most recent commit 9 months ago

Predator ⭐ 14

High-performance crawler framework based on fasthttp.

total releases 199latest release March 29, 2022most recent commit a year ago

Colly ⭐ 21,902

Elegant Scraper and Crawler Framework for Golang

dependent packages 328total releases 22latest release March 08, 2022most recent commit a month ago

Gospider ⭐ 2,190

Gospider - Fast web spider written in Go

dependent packages 1total releases 17latest release December 25, 2021most recent commit 6 months ago

Popular Data Processing Categories

Jupyter Notebook

Categories

Application Programming Interfaces

Artificial Intelligence

Cloud Computing

Command Line Interface

Computer Science

Configuration Management

Content Management

Data Processing

Integrated Development Environments

Learning Resources

Lists Of Projects

Machine Learning

Operating Systems

Package Managers

Programming Languages

Runtime Environments

Software Architecture

Software Development

Software Performance

Software Quality

Text Processing

User Interface Components

Version Control

Web User Interface

Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source. All rights reserved.