Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About
Search results for search crawler
crawler
x
search
x
148 search results found
Wooyun_public
⭐
3,701
This repo is archived. Thanks for wooyun! 乌云公开漏洞、知识库爬虫和搜索 crawl and search for wooyun.org public bug(vulnerability) and drops
Ambar
⭐
1,797
🔍 Ambar: Document Search Engine
Open Source Search Engine
⭐
1,504
Nov 20 2017 -- A distributed open source search engine and spider/crawler written in C/C++ for Linux on Intel/AMD. From gigablast dot com, which has binaries for download. See the README.md file at the very bottom of this page for instructions.
Fess
⭐
943
Fess is very powerful and easily deployable Enterprise Search Server.
Spider
⭐
919
Python website crawler.
Baiduspider
⭐
872
BaiduSpider,一个爬取百度搜索结果的爬虫,目前支持百度网页搜索,百度图片搜索,百度知道搜索
Angrysearch
⭐
866
Linux file search, instant results as you type
Ipfs Search
⭐
779
Search engine for the Interplanetary Filesystem.
Tweetscraper
⭐
698
TweetScraper is a simple crawler/spider for Twitter Search without using API
Go Dork
⭐
677
The fastest dork scanner written in Go.
Filemasta
⭐
645
A search application to explore, discover and share online files
Opensearchserver
⭐
419
Open-source Enterprise Grade Search Engine Software
Sparkler
⭐
401
Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.
Search Engines Scraper
⭐
377
Search google, bing, yahoo, and other search engines with python
Clipper.js
⭐
311
HTML to Markdown converter and crawler.
Crawlertutorial
⭐
310
爬蟲極簡教學(fetch, parse, search, multiprocessing, API)- PTT 為例
Stweet
⭐
308
Advanced python library to scrap Twitter (tweets, users) from unofficial API
Googlesearchcrawler
⭐
305
a tool for crawl Google search results
Magic_google
⭐
287
Google search results crawler, get google search results that you need
Harvestman
⭐
278
Quick and dirty web crawling.
Gcse
⭐
277
Project for Go Search, a search engine for finding popular and relevant packages.
Algoliasearch Netlify
⭐
258
Official Algolia Plugin for Netlify. Index your website to Algolia when deploying your project to Netlify with the Algolia Crawler
Laravel Site Search
⭐
248
Create a full-text search index by crawling your site
Nudecrawler
⭐
231
Crawl telegra.ph searching for nudes!
Python Email Crawler
⭐
213
Search on Google, and crawls for emails related to the result
Fxdesktopsearch
⭐
168
A JavaFX based desktop search application.
Findpapers
⭐
164
Findpapers: A tool for helping researchers who are looking for related works
Festin
⭐
150
FestIn - S3 Bucket Weakness Discovery
Weibosearch
⭐
144
A distributed Sina Weibo Search spider base on Scrapy and Redis.
Leetcode Ranking Search
⭐
133
Leetcode Contest Ranking Searcher
Spotlight.js
⭐
131
An object crawler/property search library.
Search
⭐
126
An Open Source Search Engine
Npm Search
⭐
122
🗿 npm ↔️ Algolia replication tool ⛷️ 🐌 🛰️
Php Crawler
⭐
121
A php crawler that finds emails on the internets
Weibo_wordcloud
⭐
118
根据关键词抓取微博数据,再生成词云
Saushengine.v1
⭐
118
Simple Ruby-based search engine
Fastclass
⭐
107
Little tools to download and then weed through images, delete and classify them into groups for building deep learning image datasets (based on crawler and tkinter)
Crawl Anywhere
⭐
98
Crawl-Anywhere - Web Crawler and document processing pipeline with Solr integration.
Enterprise Search
⭐
97
An open source search engine for corporate data and websites.
Ahmia Crawler
⭐
95
Collection of crawlers used by the ahmia search engine
Node Search Engine
⭐
79
Sample search engine with web crawler, built on Node.js + CouchDB + Limestone
Wp Json Scraper
⭐
78
Scrapes WordPress data using the WP-JSON API activated by default since WordPress 4.7
Itbooks
⭐
75
Get itbooks from ebooks's website for free,such as allitebooks,digilibraries,etc
Scraping Ebay
⭐
73
Scraping Ebay's products using Scrapy Web Crawling Framework
Secrawler
⭐
69
A scrapy project can crawl search result of Google/Bing/Baidu
Qr Pirate
⭐
66
crawl QR-codes from search engines and look for bitcoin private keys
Chemrtron
⭐
58
A document viewer; fuzzy match incremental search.
Fishfishjump
⭐
57
Fish Fish Jump is a solution in the python that simply and basic for search engines. 🐟 🐟 🐟
Phinde
⭐
52
Self-hosted search engine for your static blog
Bt Share
⭐
51
a spider which can grab some interesting things =)
Findopendata
⭐
49
A search engine for Open Data
Imagecrawl
⭐
49
Web Image Crawler by scrapy
Pysearch
⭐
48
Web crawler and Search engine in Python.
Gdht
⭐
48
A distributed self-host DHT torrent search suite
Lyrics Crawler
⭐
46
Get the lyrics for the song currently playing on Spotify
Devsearch
⭐
45
A web search engine built with Python which uses TF-IDF and PageRank to sort search results.
Opensearchserver Php Client
⭐
44
OpenSearchServer PHP Client
Images Web Crawler
⭐
42
This package is a complete tool for creating a large dataset of images (specially designed -but not only- for machine learning enthusiasts). It can crawl the web, download images, rename / resize / covert the images and merge folders..
A Simple Search Engine In Python
⭐
42
Python Web Crawler
⭐
38
Nasty
⭐
37
NASTY Advanced Search Tweet Yielder
Simple Twitter Shadowban Checker
⭐
37
Comcrawl
⭐
37
A python utility for downloading Common Crawl data
Alida
⭐
36
Crawling, scraping and indexing application written in Clojure.
Azuresearchcrawler
⭐
35
A simple web crawler, using Abot, that indexes page contents into Azure Search.
Kearch
⭐
35
Distributed search engine
Auctus
⭐
34
Dataset search engine, discovering data from a variety of sources, profiling it, and allowing advanced queries on the index
Podcastcrawler
⭐
33
PHP library to find podcasts
Leboncoin Crawler
⭐
32
Crawler for leboncoin.fr
Python Searchengine
⭐
31
A simple search engine which utilizes whoosh, mongodb, a custom html scraper and simple crawler.
Searchx
⭐
28
基于规则的跨平台一站式聚合搜索工具
Thingiversecrawler
⭐
28
A simple script to batch download things from thingiverse.com
Dhtsearch
⭐
27
[mirror] Standalone DHT search
Search_ads_web_service
⭐
27
Online search advertisement platform & Realtime Campaign Monitoring [Maybe Deprecated]
Video Crawler
⭐
26
Crawl websites for videos from Youtube, Vimeo, Soundcloud, etc
Goodsearcher
⭐
26
A pyLucene-based search module for searching books from goodreads.com
Pimcore Lucene Search
⭐
25
Pimcore Website Indexer (powered by Zend Search Lucene)
Nowcrawling
⭐
23
A web crawler that will help you find files and lots of interesting information.
Seleniumsemanticscraper
⭐
23
Automatically crawl meta data from papers from Semantic Scholar website, based on a given key phrase using Selenium WebDriver and saves it in a .xls (Excel) file. This program uses TKinter, wrapped with appJar as GUI.
Taws
⭐
23
Taws - A personal and private web search engine
Bluebird
⭐
23
Unofficial Python client for Twitter
Python Allrecipes
⭐
22
Python API to search & get recipes from the 'allrecipes.com' website (web crawler, unofficial)
Indieweb Search
⭐
22
Source code for the IndieWeb search engine.
Doogle
⭐
22
Doogle is a search engine and web crawler which can search indexed websites and images
Sqloogle
⭐
21
Crawl, Index, and Search Your SQL.
Gitbucket Fess Plugin
⭐
21
GitBucket plugin for Fess
Algoliasearch Zendesk
⭐
21
Integrate Algolia within your Zendesk Help Center in minutes.
Actor Youtube Scraper
⭐
20
Apify actor to scrape Youtube search results. You can set the maximum videos to scrape per page as well as the date from which to start scraping.
Legitbot
⭐
19
🤔 Is this Web request from a real search engine🕷 or from an impersonating agent 🕵️♀️?
Slackwebhooksgithubcrawler
⭐
19
Search for Slack Webhooks token publicly exposed on Github
Scrapy Azuresearch Crawler Samples
⭐
19
Scrapy as a Web Crawler for Azure Search Samples
Frisbee
⭐
19
Collect email addresses by crawling search engine results.
Searchgar
⭐
18
SearchGar - An actual Search Engine made using Python
Scrapy Searchengines
⭐
18
bing、google、baidu搜索引擎爬虫。python3.6 and scrapy
Php Sitemap
⭐
18
PHP Simple Sitemap Generator
Rutracker_parser
⭐
17
This scripts creates local copy of rutracker and allow to view/search throught this copy
Tensorflowsearch
⭐
17
Visual Search + Crawler + Clarifai based Content Identifier (forked and modified from Visual Search Server)
Gosearch
⭐
17
Web crawler and Search engine in Golang.
Php Google
⭐
17
Google search results crawler, get google search results that you need - php
Cc Lambda
⭐
16
Search the common crawl using lambda functions
Related Searches
Javascript Search (7,741)
Python Search (5,943)
Python Crawler (4,545)
Java Search (2,349)
Ruby Search (1,767)
Google Search (1,561)
Html Search (1,308)
Reactjs Search (1,202)
Javascript Crawler (1,142)
Plugin Search (1,087)
1-100 of 148 search results
Next >
Privacy
|
About
|
Terms
|
Follow Us On Twitter
Copyright 2018-2024 Awesome Open Source. All rights reserved.