Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About
Search results for golang spider
golang
x
spider
x
66 search results found
Colly
⭐
21,902
Elegant Scraper and Crawler Framework for Golang
Crawlab
⭐
10,521
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台,支持任何语言和框架
Toplist
⭐
4,661
今日热榜,一个获取各大热门网站热门头条的聚合网站,使用Go语言编写,多协程异步快速抓取信息,预览:h
Dht
⭐
2,424
BitTorrent DHT Protocol && DHT Spider.
Torbot
⭐
2,338
Dark Web OSINT Tool
Gospider
⭐
2,190
Gospider - Fast web spider written in Go
Go Demo
⭐
2,183
Go语言实例教程从入门到进阶,包括基础库使用、设计模式、面试易错点、工具类、对接第三方等
Geziyor
⭐
1,892
Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.
Go_spider
⭐
1,629
[爬虫框架 (golang)] An awesome Go concurrent Crawler(spider) framework. The crawler is flexible and modular. It can be expanded to an Individualized crawler easily or you can use the default crawl components only.
Creeper
⭐
762
🐾 Creeper - The Next Generation Crawler Framework (Go)
Bookhunter
⭐
645
A download tools for clawing the ebooks from internets.
Replication Manager
⭐
611
Signal 18 repman - Replication Manager for MySQL / MariaDB / Percona Server
Fictiondown
⭐
601
小说下载|小说爬取|起点|笔趣阁|导出Markdown|导出txt|转换epub|广告过滤|自动校对
Gosint
⭐
567
OSINT Swiss Army Knife
Go_jobs
⭐
536
带你了解一下Golang的市场行情
Scout
⭐
424
🔭 Lightweight URL fuzzer and spider: Discover a web server's undisclosed files, directories and VHOSTs
Ants Go
⭐
368
open source, distributed, restful crawler engine in golang
Gospider
⭐
354
golang实现的爬虫框架,使用者只需关心页面规则,提供web管理界面。基于colly开发。
Torrent Paradise
⭐
313
Decentralized DHT search site for IPFS
Webpalm
⭐
295
WebPalm is a powerful command-line tool for website mapping and web scraping. With its recursive approach, it can generate a complete tree of all webpages and their links on a website. It can also extract data from the body of each page using regular expressions, making it an ideal tool for web scraping and data extraction.
Gopa
⭐
281
[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn
Ant
⭐
271
A web crawler for Go
Urlgrab
⭐
243
A golang utility to spider through a website searching for additional links.
Go Movies
⭐
232
golang spider Crawler 爬虫 电影
Laosj
⭐
199
golang light-weight image crawler
Go Pkg Spider
⭐
190
一个 Golang 实现的相对智能、无需规则维护的通用新闻网站数据提取工具库。含域名探测、网页编码语种识别、网页链接分类
Galer
⭐
189
A fast tool to fetch URLs from HTML attributes by crawl-in.
Digger
⭐
180
Digger is a powerful and flexible web crawler implemented by pure golang
Antch
⭐
177
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
Spider
⭐
173
Scheduler of spiders for scraping and parsing HTML and JSON pages
Goribot
⭐
162
[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。
Apiproject
⭐
159
[https://www.sofineday.com], golang项目开发脚手架,集成最佳实践(gin+gorm+go-redis+mongo+cors+ modules依赖管理+headless爬虫chromedp+makefile+二进制压缩+live
Article Spider
⭐
140
文章采集工具 Article collection tool
Spidy
⭐
138
Domain names collector - Crawl websites and collect domain names along with their availability status.
Yispider
⭐
127
一款分布式爬虫平台,帮助你更好的管理和开发爬虫。 内置一套爬虫定义规则(模版),可使用模版快速定义爬虫,也可当作框架手动开发爬虫。(兴趣使然的项目,用
Go Spider
⭐
99
Go spider
Gopa Abandoned
⭐
97
GOPA, a spider written in Go.(NOTE: this project moved to https://github.com/infinitbyte/gopa )
Blinkist M4a Downloader
⭐
97
Grabs all of the audio files from all of the Blinkist books
Douban Movie
⭐
91
Golang爬虫 爬取豆瓣电影Top250
Gospider
⭐
82
🚀Gospider is a powerful Golang web crawler that includes all the necessary libraries for transitioning from Python to Golang. It provides a fast and seamless transition for Python web crawlers to Golang.
Movie Getter
⭐
79
Go编写的影视资源采集器
Yue Spider
⭐
78
手机淘宝App 闲鱼App 相关爬虫 学习测试
Foot
⭐
71
foot是一个集足球数据采集器,简单分析的项目.AI足球球探为程序全自动处理,全程无人为参与干预足球
Proxy_pool
⭐
70
A simple proxy pool
Gospider
⭐
70
⚡ Light weight Golang spider framework | 轻量的 Golang 爬虫框架
Parse Video
⭐
60
Golang短视频去水印:抖音,皮皮虾,火山,微视,最右,快手,全民小视频,皮皮搞笑,西瓜视频,虎牙
Cassler
⭐
57
🕷️ 🕷️ 🕷️ Validate SSL Certificates around web
Godataaccess
⭐
54
🪲Data access framework in native Golang(Golang实现的类Scrapy框架)
Tool Gin
⭐
54
基于go-gin框架建立减少冗余动作项目,如:下载一些工具
Car Prices
⭐
50
Golang爬虫 爬取汽车之家 二手车产品库
Gotools
⭐
48
create some tools use go lang.
Ispider
⭐
46
简单爬虫实现
Gathertool
⭐
46
gathertool是golang脚本化开发库,目的是提高对应场景程序开发的效率;轻量级爬虫库,接口
Crawlerdetect
⭐
43
Golang module to detect bots and crawlers via the user agent
Gose
⭐
42
A simple but full search engine written in golang
Cb Tumblebug
⭐
39
Cloud-Barista Multi-Cloud Infra Management Framework
Scrapemate
⭐
39
Golang Crawling and scraping framework
Goscrapy
⭐
34
GoScrapy: Harnessing Go's power for efficient web scraping, inspired by Python's Scrapy framework.
Go Crawler
⭐
33
A web crawling framework implemented in Golang, it is simple to write and delivers powerful performance. It comes with a wide range of practical middleware and supports various parsing and storage methods. Additionally, it supports distributed deployment. 基于golang实现的爬虫框架,编写简单,性能强劲。内置了丰富的实用中间件,支持多种解析、保存方式,
Grawler
⭐
31
A web crawler / scraper engine written in Golang
Crawler
⭐
31
十年磨一剑:Crawler4U, a general purpose focused crawler
Cb Spider
⭐
29
CB-Spider offers a unified view and interface for multi-cloud management.
Bangumi_yearly_report
⭐
27
Weibospidergo
⭐
25
weibo spider written by golang colly(使用golang的colly库编写的微博爬虫)
News
⭐
25
喷嚏图挂内容爬虫及托管
Go_spider
⭐
25
A golang spider
Actions Starcharts
⭐
22
自动生成星图(stars 趋势)的 action
Asmr Downloader
⭐
19
A tool for download asmr media from asmr.one(Thanks for the asmr.one)
Marmot
⭐
18
💐Marmot A Golang HTTP Download
Jav
⭐
17
Crawl javbus Magnet
Predator
⭐
14
High-performance crawler framework based on fasthttp.
Musicspider
⭐
14
:trollface: Music Spider. Go 👾 Music Spider 是使用Golang写的音乐聚合爬虫,目前支持的站点包括 网易、QQ、虾米、酷狗、百度。
Go Scrapy
⭐
13
spider,go,scrapy
Shubc
⭐
13
Go bindings for Scrapinghub HTTP API and a sweet command line tool for Scrapy Cloud
Crawl
⭐
12
A concurrent crawler that minimizes memory use. Output suitable for use with BigQuery.
Proksi
⭐
11
An intelligent proxy server. Provide durable, real-time, high-quality proxies as a middleman or datasource server.
Input Field Finder
⭐
11
Spiders given URLs for input fields.
Smzdmforgo
⭐
10
什么值得买文章 自动签到 自定义规则爬取
Visual_spider_go
⭐
10
a visual spider by go
Cb Webtool
⭐
9
Cloud-Barista Web Tool
Spider Engine
⭐
8
一个基于Go的小型搜索引擎
Tegenaria
⭐
8
Tegenaria is a crawler framework based on golang
Scrapio
⭐
7
Simple and easy-to-use scraper and crawler in Go.
Hugh
⭐
7
Gcurl
⭐
7
parse curl bash to code of golang
Wallhaven Spider
⭐
7
wallhaven.cc 采集工具
Neo Go Sdk
⭐
7
Go SDK for the NEO blockchain
Proxy Pool
⭐
7
go ip proxy pool
Goapps
⭐
7
Apps implemented by golang including Spiders, LRU cache, a simple BlockChain network, etc.
Fake Useragent
⭐
7
Provide random user agent
Spider Man
⭐
7
一个基于Go的爬虫框架
Homebrew Juejin Spider
⭐
6
爬取掘金专栏并转为Markdown保存到本地
Ungoliant
⭐
6
A web reconnaissance tool that proxies its results through Burp or ZAP.
Quick
⭐
6
HTTP Request library for Go (简洁高效的HTTP请求库)
Turbo Tor Crawl
⭐
6
Recursive hostnames crawler
Spidergo
⭐
6
🪲 A spider(crawler) written in go.
Comp4321
⭐
6
Search engine course project
Spider
⭐
6
URL Spider - web crawler and wordlist / ngram generator
Botdetector
⭐
5
BotDetector is a golang library that detects Bot/Spider/Crawler from user agent
Map
⭐
5
Simple site map generator, supports couple reporters, depth levels and etc
Related Searches
Golang Command Line (8,308)
Golang Docker (7,769)
Golang Http (4,290)
Golang Server (4,285)
Javascript Golang (3,372)
Golang Database (2,927)
Golang Json (2,652)
Golang Proxy (2,577)
Golang Grpc (2,432)
Python Golang (2,332)
1-66 of 66 search results
Privacy
|
About
|
Terms
|
Follow Us On Twitter
Copyright 2018-2024 Awesome Open Source. All rights reserved.