Awesome Open Source

Programming Languages

Search results for crawler search engine

search-engine x

65 search results found

Googlescraper ⭐ 2,540

A Python module to scrape several search engines (like Google, Yandex, Bing, Duckduckgo, ...). Including asynchronous networking support.

Ambar ⭐ 1,797

🔍 Ambar: Document Search Engine

Python Seo Analyzer ⭐ 956

An SEO tool that analyzes the structure of a site, crawls the site, count words in the body of the site and warns of any technical SEO issues.

Fess is very powerful and easily deployable Enterprise Search Server.

Ipfs Search ⭐ 779

Search engine for the Interplanetary Filesystem.

Filemasta ⭐ 646

A search application to explore, discover and share online files

Domains ⭐ 508

World’s single largest Internet domains dataset

Opensearchserver ⭐ 419

Open-source Enterprise Grade Search Engine Software

Sparkler ⭐ 401

Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.

Search Engines Scraper ⭐ 377

Search google, bing, yahoo, and other search engines with python

Project for Go Search, a search engine for finding popular and relevant packages.

Collector Http ⭐ 162

Norconex Web Crawler (or spider) is a flexible web crawler for collecting, parsing, and manipulating data from the Internet (or Intranet) to various data repositories such as search engines.

An Open Source Search Engine

Saushengine.v1 ⭐ 118

Simple Ruby-based search engine

GitHub Search: Platform used to crawl, store and present projects from GitHub, as well as any statistics related to them.

Crawl Anywhere ⭐ 98

Crawl-Anywhere - Web Crawler and document processing pipeline with Solr integration.

Ahmia Crawler ⭐ 95

Collection of crawlers used by the ahmia search engine

Node Search Engine ⭐ 79

Sample search engine with web crawler, built on Node.js + CouchDB + Limestone

Couch Crawler ⭐ 77

A search engine built on top of couchdb-lucene

Davedavefind ⭐ 71

A simple search engine based on the web crawler developed in Udacity's CS101 course.

Robots.txt ⭐ 69

Simple robots.txt template. Keep unwanted robots out (disallow). White lists (allow) legitimate user-agents. Useful for all websites.

Search through all your personal data efficiently like web search.

Dig Etl Engine ⭐ 65

Download DIG to run on your laptop or server.

Fishfishjump ⭐ 57

Fish Fish Jump is a solution in the python that simply and basic for search engines. 🐟 🐟 🐟

Librengine ⭐ 55

Privacy Web Search Engine (not meta, own crawler)

Self-hosted search engine for your static blog

Pysearch ⭐ 48

Web crawler and Search engine in Python.

Devsearch ⭐ 45

A web search engine built with Python which uses TF-IDF and PageRank to sort search results.

A Simple Search Engine In Python ⭐ 42

Dbworld Search ⭐ 38

🔍 简单的搜索引擎, django 框架

Crawling, scraping and indexing application written in Clojure.

Distributed search engine

Dataset search engine, discovering data from a variety of sources, profiling it, and allowing advanced queries on the index

Python Searchengine ⭐ 31

A simple search engine which utilizes whoosh, mongodb, a custom html scraper and simple crawler.

Search Engine in Erlang

Search_ads_web_service ⭐ 27

Online search advertisement platform & Realtime Campaign Monitoring [Maybe Deprecated]

Video Crawler ⭐ 26

Crawl websites for videos from Youtube, Vimeo, Soundcloud, etc

Indieweb Search ⭐ 22

Source code for the IndieWeb search engine.

Doogle is a search engine and web crawler which can search indexed websites and images

Legitbot ⭐ 19

🤔 Is this Web request from a real search engine🕷 or from an impersonating agent 🕵️‍♀️?

Searchgar ⭐ 18

SearchGar - An actual Search Engine made using Python

Instagram Crawler ⭐ 18

Scraping instagram profile information

Botanalyse ⭐ 17

botsonar analyse open api

Gosearch ⭐ 17

Web crawler and Search engine in Golang.

Selenium Open Source Search Engine & crawler

Seo Core ⭐ 16

Seo-core library was extracted from Seo-Tracker platform. This is a common way to deal with websites, search engines, crawlers and scrappers.

Information Retrieval ⭐ 15

Elasticsearch, MongoDB, Tornado Server, RESTful API, Python, Information Retrieval, Machine Learning, Web Crawler

Saushengine ⭐ 14

Studybyte ⭐ 14

Studybyte is a search engine designed to help students find educational content effortlessly.

Openapi Web Search ⭐ 13

OpenAPI Web Search: Revolutionizing the Way Developers find API Definitions 🚀

Websearchengine ⭐ 13

https://www.text-mining.ro/ demonstrates how to design & implement a Web Search Engine

YGGo! Distributed Web Search Engine

A complete multi-threaded web-crawler in Python3

Googlesearch Cli ⭐ 11

Search anything on Google without captcha

Implementing Search Engine And Page Rank Algorithm ⭐ 11

Implementing Search Engine and Page Rank Algorithm Using Python

Wgit allows you to crawl and extract the data you want from the web

Web Crawler ⭐ 9

Node.js web crawler, který prochází webové stránky a získává o nich různá data.

Podcast Search Engine

Solr Nutch Vagrant ⭐ 8

Ubuntu Server 16.04 with Solr 4.10.3 and Nutch 1.9

Ahcrawler ⭐ 8

crawler for a searchengine on your website and website analytics

Hitsearch ⭐ 8

final project for cs324 data mining. made a search engine and crawler using HITS algorithm.

Game Search Engine ⭐ 7

A Search Engine for Games

Cloud Computing Search Engine ⭐ 7

A cloud-based web search engine computing Hadoop MapReduce on Amazon EC2 consisting of crawler, indexer, PageRank.

Simplified Search Engine ⭐ 7

Multithreaded Web Crawler, Scraper, Indexer

A simple Crawler-based search engine that demonstrates the main features of a search engine (web crawling, indexing and ranking) and the interaction between them using Java and a Web Interface.

Yggdrasil Network Explorer

Ding Search ⭐ 6

Vyhledávač napsaný v Node.js, který dokáže vyhledávat webové stránky.

a little php search engine with sqlite

Real time search engine for Tor and IoT.

Web Search Engine Uic ⭐ 6

CS 582 Information Retrieval at University of Illinois at Chicago. Multithreaded crawling of UIC domain, inverted index, page rank, SEO with Context Pseudo-Relevance Feedback

Codexdrake ⭐ 6

An open source, privacy-first, self-hosting capable and blazing fast search engine written in JavaScript. Browse anonymously and safely without the need to pay third-party APIs.

Uca Google ⭐ 6

SE2 Autumn of Code 2022

Search Engine ⭐ 5

A search query is processed using NLP after which the results are displayed by crawling the web for relevant web pages & displaying them after normalizing the scores

🌊 7old is a vintage search engine

Search Engine ⭐ 5

A Search Engine that uses hash tables and linked lists to crawl, index, query written in C

House Finder ⭐ 5

Web crawler, flat search engine and notification tool, that I use to find my new flat!

Atproto Pds Search ⭐ 5

This project automatically crawls and visualizes the atproto PDS endpoints indexed in the PLC directory.

Webscraper ⭐ 5

webcrawler made from Beautiful soup

Search Engine And Crawler ⭐ 5

Python based search engine and crawler. Search engine uses Whoosh and the crawler uses Beautiful-soup

Diffbot Sp Search ⭐ 5

Diffbot powered SitePoint search engine

Webcrawler And Search Engine ⭐ 5

Chefmate is a from-scratch search engine built using Python, NodeJS, and React for Purdue 2020 Senior Capstone

Related Searches

Python Crawler (4,545)

Javascript Crawler (1,142)

Crawler Scrapy (988)

Scraper Crawler (896)

Crawler Spider (709)

Python Search Engine (614)

Java Crawler (593)

1-65 of 65 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.