Awesome Open Source

Programming Languages

The Top 23 Golang Spider Open Source Projects

Open source projects categorized as Golang Spider

Categories > Programming Languages > Golang

Categories > Data Processing > Spider

Colly ⭐ 21,902

Elegant Scraper and Crawler Framework for Golang

dependent packages 328total releases 22most recent commit a month ago

Crawlab ⭐ 10,521

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

total releases 1most recent commit 3 months ago

Toplist ⭐ 4,661

今日热榜，一个获取各大热门网站热门头条的聚合网站，使用Go语言编写，多协程异步快速抓取信息，预览:h

most recent commit a year ago

BitTorrent DHT Protocol && DHT Spider.

dependent packages 2total releases 1most recent commit 3 years ago

Torbot ⭐ 2,338

Dark Web OSINT Tool

most recent commit 4 months ago

Gospider ⭐ 2,190

Gospider - Fast web spider written in Go

dependent packages 1total releases 17most recent commit 5 months ago

Go Demo ⭐ 2,183

Go语言实例教程从入门到进阶，包括基础库使用、设计模式、面试易错点、工具类、对接第三方等

most recent commit 9 months ago

Geziyor ⭐ 1,892

Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.

dependent packages 11total releases 6most recent commit a year ago

Go_spider ⭐ 1,629

[爬虫框架 (golang)] An awesome Go concurrent Crawler(spider) framework. The crawler is flexible and modular. It can be expanded to an Individualized crawler easily or you can use the default crawl components only.

dependent packages 2total releases 1most recent commit 6 years ago

Creeper ⭐ 762

🐾 Creeper - The Next Generation Crawler Framework (Go)

total releases 1most recent commit 7 years ago

Bookhunter ⭐ 645

A download tools for clawing the ebooks from internets.

total releases 25most recent commit 3 months ago

Replication Manager ⭐ 611

Signal 18 repman - Replication Manager for MySQL / MariaDB / Percona Server

total releases 24most recent commit 3 months ago

Fictiondown ⭐ 601

小说下载|小说爬取|起点|笔趣阁|导出Markdown|导出txt|转换epub|广告过滤|自动校对

total releases 5most recent commit 4 months ago

OSINT Swiss Army Knife

most recent commit 3 years ago

Go_jobs ⭐ 536

带你了解一下Golang的市场行情

total releases 1most recent commit 4 years ago

🔭 Lightweight URL fuzzer and spider: Discover a web server's undisclosed files, directories and VHOSTs

total releases 31most recent commit a year ago

Ants Go ⭐ 368

open source, distributed, restful crawler engine in golang

total releases 1most recent commit 8 years ago

Gospider ⭐ 354

golang实现的爬虫框架，使用者只需关心页面规则，提供web管理界面。基于colly开发。

total releases 3most recent commit a year ago

Torrent Paradise ⭐ 313

Decentralized DHT search site for IPFS

most recent commit 2 years ago

Webpalm ⭐ 295

WebPalm is a powerful command-line tool for website mapping and web scraping. With its recursive approach, it can generate a complete tree of all webpages and their links on a website. It can also extract data from the body of each page using regular expressions, making it an ideal tool for web scraping and data extraction.

dependent packages 2total releases 8most recent commit 6 months ago

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

total releases 6most recent commit 3 years ago

A web crawler for Go

total releases 8most recent commit a year ago

Urlgrab ⭐ 243

A golang utility to spider through a website searching for additional links.

most recent commit 4 years ago

The 10 Most Depended On Golang Spider Open Source Projects

Colly ⭐ 21,902

Elegant Scraper and Crawler Framework for Golang

dependent packages 328total releases 22latest release March 08, 2022most recent commit a month ago

Geziyor ⭐ 1,892

Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.

dependent packages 11total releases 6latest release April 19, 2021most recent commit a year ago

Cb Spider ⭐ 29

CB-Spider offers a unified view and interface for multi-cloud management.

dependent packages 4total releases 154latest release November 29, 2023most recent commit 3 months ago

Cb Tumblebug ⭐ 39

Cloud-Barista Multi-Cloud Infra Management Framework

dependent packages 3total releases 171latest release November 29, 2023most recent commit 3 months ago

Webpalm ⭐ 295

WebPalm is a powerful command-line tool for website mapping and web scraping. With its recursive approach, it can generate a complete tree of all webpages and their links on a website. It can also extract data from the body of each page using regular expressions, making it an ideal tool for web scraping and data extraction.

dependent packages 2total releases 8latest release June 19, 2023most recent commit 6 months ago

Go_spider ⭐ 1,629

[爬虫框架 (golang)] An awesome Go concurrent Crawler(spider) framework. The crawler is flexible and modular. It can be expanded to an Individualized crawler easily or you can use the default crawl components only.

dependent packages 2total releases 1latest release August 09, 2015most recent commit 6 years ago

💐Marmot A Golang HTTP Download

dependent packages 2total releases 13latest release November 25, 2022most recent commit a year ago

Crawlerdetect ⭐ 43

Golang module to detect bots and crawlers via the user agent

dependent packages 2total releases 20latest release March 23, 2023most recent commit 4 months ago

BitTorrent DHT Protocol && DHT Spider.

dependent packages 2total releases 1latest release December 19, 2020most recent commit 3 years ago

Gospider ⭐ 70

⚡ Light weight Golang spider framework | 轻量的 Golang 爬虫框架

dependent packages 1total releases 5latest release March 16, 2021most recent commit 3 years ago

The 10 Latest Releases In Golang Spider Open Source Projects

Go Pkg Spider ⭐ 190

一个 Golang 实现的相对智能、无需规则维护的通用新闻网站数据提取工具库。含域名探测、网页编码语种识别、网页链接分类

total releases 38latest release December 13, 2023most recent commit 4 months ago

Cb Tumblebug ⭐ 39

Cloud-Barista Multi-Cloud Infra Management Framework

dependent packages 3total releases 171latest release November 29, 2023most recent commit 3 months ago

Cb Spider ⭐ 29

CB-Spider offers a unified view and interface for multi-cloud management.

dependent packages 4total releases 154latest release November 29, 2023most recent commit 3 months ago

Go Crawler ⭐ 33

A web crawling framework implemented in Golang, it is simple to write and delivers powerful performance. It comes with a wide range of practical middleware and supports various parsing and storage methods. Additionally, it supports distributed deployment. 基于golang实现的爬虫框架，编写简单，性能强劲。内置了丰富的实用中间件，支持多种解析、保存方式，

total releases 26latest release November 23, 2023most recent commit 5 months ago

Infor You Mation ⭐ 5

latest release November 21, 2023most recent commit 6 years ago

A simple but full search engine written in golang

latest release November 20, 2023most recent commit 10 years ago

Article Spider ⭐ 140

文章采集工具 Article collection tool

total releases 104latest release November 08, 2023most recent commit 5 months ago

Gathertool ⭐ 46

gathertool是golang脚本化开发库，目的是提高对应场景程序开发的效率；轻量级爬虫库，接口

total releases 58latest release November 01, 2023most recent commit 5 months ago

Bookhunter ⭐ 645

A download tools for clawing the ebooks from internets.

total releases 25latest release October 23, 2023most recent commit 3 months ago

Goscrapy ⭐ 34

GoScrapy: Harnessing Go's power for efficient web scraping, inspired by Python's Scrapy framework.

total releases 26latest release September 20, 2023most recent commit 3 months ago

Popular Programming Languages Categories

Categories

Application Programming Interfaces

Artificial Intelligence

Cloud Computing

Command Line Interface

Computer Science

Configuration Management

Content Management

Data Processing

Integrated Development Environments

Learning Resources

Lists Of Projects

Machine Learning

Operating Systems

Package Managers

Programming Languages

Runtime Environments

Software Architecture

Software Development

Software Performance

Software Quality

Text Processing

User Interface Components

Version Control

Web User Interface

Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source. All rights reserved.