Awesome Open Source

Programming Languages

Search results for spark crawler

9 search results found

Spring Boot Quick ⭐ 2,282

🌿 基于springboot的快速学习示例,整合自己遇到的开源框架,如：rabbitmq(延迟队列)、K

Sparkler ⭐ 401

Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.

Cc Pyspark ⭐ 280

Process Common Crawl data with Python and Spark

《数据采集从入门到放弃》源码。内容简介：爬虫介绍、就业情况、爬虫工程师面试题；HTTP协议介绍； Requests使用；解析器Xpath介绍； MongoDB与MySQL；多线程爬虫； Scrapy介绍；Scrapy-redis介绍；使用docker部署；使用nomad管理docker集群；使用EFK查询docker日志

Cc Index Table ⭐ 78

Index Common Crawl archives in tabular format

Keywordanalysis ⭐ 33

Word analysis, by domain, on the Common Crawl data set for the purpose of finding industry trends

Engineeringteam ⭐ 32

와이빅타 엔지니어링팀의 자료를 정리해두는 곳입니다.

Search_ads_web_service ⭐ 27

Online search advertisement platform & Realtime Campaign Monitoring [Maybe Deprecated]

Steam_recommendation_system ⭐ 25

Recommendation System, Collaborative Filtering, Spark, Hive, Flask, Web Crawler, AWS EC2, AWS RDS

Sparkwarc ⭐ 13

Load WARC files into Apache Spark with sparklyr

Cosr Ops ⭐ 12

Tools for managing deployment & operations of Common Search.

Rsparkler ⭐ 10

RsparkleR provides an R interface for launching virtual machines and deploying Sparkler

Vpm Filter Spark ⭐ 10

Virtual patent marking crawler at iproduct.epfl.ch

Common_crawl_insight ⭐ 7

Sparkler Crawl Environment - a packaged, dockerized version of http://github.com/USCDataScience/sparkler.git

Related Searches

Python Crawler (4,545)

Scala Spark (3,279)

Python Spark (2,053)

Java Spark (1,587)

Apache Spark (1,207)

Spark Hadoop (1,188)

Jupyter Notebook Spark (1,151)

Javascript Crawler (1,142)

Crawler Scrapy (988)

Spark Kafka (985)

1-9 of 9 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.