Awesome Open Source

Programming Languages

The Top 23 Html Crawler Open Source Projects

Open source projects categorized as Html Crawler

Categories > Data Processing > Crawler

Categories > Web User Interface > Html

Easyspider ⭐ 20,149

A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/爬虫软件，可以无代码图形化

most recent commit 22 days ago

Toapi ⭐ 3,417

Every web site provides APIs.

most recent commit 2 years ago

Toapi ⭐ 3,153

Every web site provides APIs.

total releases 40most recent commit 3 years ago

Crawler_illegal_cases_in_china ⭐ 2,467

Collection of China illegal cases about web crawler 本项目用来整理所有中国大陆爬虫开发者涉诉与违规相关的新闻、资料与法律法规。致力于帮助在中国大陆工作的 [AD]中文知识图谱门户

most recent commit 2 years ago

Scrapely ⭐ 1,668

A pure-python HTML screen-scraping library

dependent packages 2total releases 13most recent commit 4 years ago

pypi scrapely} Downloads

Python Crawler ⭐ 1,576

从头开始系统化的学习如何写Python爬虫。 Python版本 3.6

most recent commit 2 years ago

Mlscraper ⭐ 935

🤖 Scrape data from HTML websites automatically by just providing examples

total releases 4most recent commit a year ago

Storm Crawler ⭐ 834

A scalable, mature and versatile web crawler based on Apache Storm

dependent packages 10total releases 36most recent commit 3 months ago

Computerstudent ⭐ 764

计算机专业系统性学习资料（python,c,c++,计算机组成，计算机网络，编译原理，电路，谷歌插件

most recent commit 8 months ago

Skrape.it ⭐ 714

A Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.

dependent packages 3total releases 14most recent commit 4 months ago

Tailwindui Crawler ⭐ 642

tailwindui-crawler downloads the component HTML files locally

most recent commit a year ago

Domains ⭐ 508

World’s single largest Internet domains dataset

most recent commit 6 months ago

Convert HTML to Markdown.

dependent packages 9total releases 3most recent commit 4 years ago

pypi tomd} Downloads

Pywebcopy ⭐ 455

Locally saves webpages to your hard disk with images, css, js & links as is.

dependent packages 2total releases 21most recent commit 4 months ago

pypi pywebcopy} Downloads

Html2article ⭐ 425

Html网页正文提取

total releases 5most recent commit 7 years ago

Prerender_rails ⭐ 358

Rails middleware gem for prerendering javascript-rendered pages on the fly for SEO

dependent packages 1total releases 44most recent commit 5 months ago

rubygems prerender_rails} Downloads

Hquery.php ⭐ 345

An extremely fast web scraper that parses megabytes of invalid HTML in a blink of an eye. PHP5.3+, no dependencies.

dependent packages 4total releases 23most recent commit 3 months ago

npm hquery.php} Downloads

Crawlertutorial ⭐ 310

爬蟲極簡教學（fetch, parse, search, multiprocessing, API）- PTT 為例

most recent commit a year ago

Phpfetcher ⭐ 281

A PHP web crawler framework

most recent commit 7 years ago

Youtube Projects ⭐ 272

This repository contains all the code I use in my YouTube tutorials.

most recent commit 3 years ago

Football Data Collection ⭐ 246

Web Scraper used to create Kaggle European Soccer database

most recent commit 5 years ago

JavaScript + BeautifulSoup = JSSoup

dependent packages 41total releases 15most recent commit 3 years ago

npm jssoup} Downloads

Leetcode Country Ranking ⭐ 219

Find Leetcode ranking by country, username.

most recent commit a year ago

The 10 Most Depended On Html Crawler Open Source Projects

JavaScript + BeautifulSoup = JSSoup

dependent packages 41total releases 15latest release October 03, 2021most recent commit 3 years ago

npm jssoup} Downloads

Xml Parser ⭐ 13

A Node.js XML DOM, Parser & Stringifier.

dependent packages 21total releases 6latest release November 06, 2019most recent commit 2 years ago

npm xml-parse} Downloads

Storm Crawler ⭐ 834

A scalable, mature and versatile web crawler based on Apache Storm

dependent packages 10total releases 36latest release October 25, 2023most recent commit 3 months ago

Convert HTML to Markdown.

dependent packages 9total releases 3latest release July 04, 2017most recent commit 4 years ago

pypi tomd} Downloads

Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler

dependent packages 5total releases 3latest release October 15, 2023most recent commit 6 months ago

Html Scrapper ⭐ 17

A general purpose web scrapper written in server side JavaScript. It is written for ease of use.

dependent packages 4total releases 3latest release July 03, 2015most recent commit 9 years ago

npm html-scrapper} Downloads

Hquery.php ⭐ 345

An extremely fast web scraper that parses megabytes of invalid HTML in a blink of an eye. PHP5.3+, no dependencies.

dependent packages 4total releases 23latest release July 19, 2019most recent commit 3 months ago

npm hquery.php} Downloads

Graphquery ⭐ 104

GraphQuery is a query language and execution engine tied to any backend service.

dependent packages 4total releases 1latest release March 17, 2019most recent commit 5 years ago

Skrape.it ⭐ 714

A Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.

dependent packages 3total releases 14latest release July 19, 2022most recent commit 4 months ago

Metatag Crawler ⭐ 6

This is a simple node.js module for scraping meta information from web pages.

dependent packages 3total releases 6latest release June 06, 2017most recent commit 5 years ago

npm metatag-crawler} Downloads

The 10 Latest Releases In Html Crawler Open Source Projects

🕷Configuration based html scraper

total releases 18latest release November 06, 2023most recent commit 6 months ago

Storm Crawler ⭐ 834

A scalable, mature and versatile web crawler based on Apache Storm

dependent packages 10total releases 36latest release October 25, 2023most recent commit 3 months ago

Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler

dependent packages 5total releases 3latest release October 15, 2023most recent commit 6 months ago

Ronin Web ⭐ 40

ronin-web is a collection of useful web helper methods and commands.

dependent packages 3total releases 16latest release April 04, 2023most recent commit 4 months ago

rubygems ronin-web} Downloads

Slap the pages out of a SPA

dependent packages 2total releases 41latest release February 07, 2023most recent commit 7 months ago

npm spank} Downloads

A fairly intuitive & powerful framework that enables you to collect & save articles and news from all over the web.

dependent packages 2total releases 102latest release February 07, 2023most recent commit 4 months ago

npm @poiw/saffron} Downloads

Node Fetch Dom ⭐ 13

Magic utility that extract javascript global variables from a remote html page.

dependent packages 2total releases 6latest release September 08, 2022most recent commit a year ago

npm @stefcud/fetch-dom} Downloads

Auto Lighthouse ⭐ 105

A utility package for automating lighthouse reporting

dependent packages 2total releases 40latest release August 18, 2022most recent commit a year ago

npm auto-lighthouse} Downloads

Skrape.it ⭐ 714

A Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.

dependent packages 3total releases 14latest release July 19, 2022most recent commit 4 months ago

Penjabarberita ⭐ 16

Extract the article list from its raw news HTML

dependent packages 2total releases 7latest release July 15, 2022most recent commit a year ago

npm @ariya/penjabarberita} Downloads

Popular Data Processing Categories

Jupyter Notebook

Categories

Application Programming Interfaces

Artificial Intelligence

Cloud Computing

Command Line Interface

Computer Science

Configuration Management

Content Management

Data Processing

Integrated Development Environments

Learning Resources

Lists Of Projects

Machine Learning

Operating Systems

Package Managers

Programming Languages

Runtime Environments

Software Architecture

Software Development

Software Performance

Software Quality

Text Processing

User Interface Components

Version Control

Web User Interface

Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source. All rights reserved.