Awesome Open Source

Programming Languages

Search results for typescript web crawler

8 search results found

Crawlee ⭐ 12,106

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Scrapoxy ⭐ 1,669

Scrapoxy is a super proxy aggregator, allowing you to manage all proxies in one place 🎯, rather than spreading it across multiple scrapers 🕸️. It also smartly handles traffic routing 🔀 to minimize bans and increase success rates 🚀.

User Agents ⭐ 871

A JavaScript library for generating random user agents with data that's updated daily.

Pricewise ⭐ 284

Dive into web scraping and build a Next.js 13 eCommerce price tracker within a single video that teaches you data scraping, cron jobs, sending emails, deployment, and more.

A simple browser/client-side web scraper.

Ayakashi ⭐ 177

⚡ Ayakashi.io - The next generation web scraping framework

Animos - Clean and minimal Anime-streaming desktop application without any ads.

Aws Pdf Textract Pipeline ⭐ 148

🔍 Data pipeline for crawling PDFs from the Web and transforming their contents into structured data using AWS textract. Built with AWS CDK + TypeScript

estela, an elastic web scraping cluster 🕸

Retrieve information from BoxRec and return it in JSON format

Tatooine ⭐ 78

A powerful scraper for JavaScript Developers.

Browser Pool ⭐ 77

A Node.js library to easily manage and rotate a pool of web browsers, using any of the popular browser automation libraries like Puppeteer, Playwright, or SecretAgent.

N8n Nodes Browserless ⭐ 29

n8n node to interact with browserless instance

Rhodesapi ⭐ 27

API for Arknights

Lemonderssreader ⭐ 25

📰 Read RSS feed from LeMonde.fr and display news inside the App

Otakudesu Scraper ⭐ 23

unofficial otakudesu.cam rest api

Kindlefy ⭐ 20

📑 A way to automatically sync data with your kindle, such as RSS feeds, manga, and too much more.

Trackpurchase ⭐ 19

단 몇줄의 코드로 다양한 쇼핑 플랫폼에서 결제 내역을 긁어오자!

Scrapeium ⭐ 18

Scrapeium is a powerful query language for scraping the web

Openai Scraper ⭐ 18

This is a template repository for building a web scraper with OpenAI support. The repository provides a basic project structure with TypeScript and Puppeteer pre-configured, as well as OpenAI's GPT-3 API integration. With this template, you can easily build a scraper that uses machine learning to analyze and extract insights from the scraped data.

Mapping sites to AniList and back.

Papercut ⭐ 15

Papercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.

All In One API to easily scrape data from any website, without worrying about captchas and bot detection mecanisms.

⚒ Web crawler that analyzes and dissects subtitles into database entries

WEB TAB makes it easy for you to stay up-to-date with the latest developer news, tools, jobs and events.

Web crawler based on Puppeteer

Books Com Tw Crawler ⭐ 12

books.com.tw crawler 「博客來」資料爬蟲

Spiderwebai ⭐ 12

The fastest way to scrape data to feed your AI models and LLMs

Kirinuki Core ⭐ 12

Kirinuki is a library that convert any html to JSON using CSS selectors.

Moodle Scrape ⭐ 11

Easily scrape data from Moodle LMS sites

Daily Image Discord Bot ⭐ 10

A fun 𝗱𝗶𝘀𝗰𝗼𝗿𝗱 𝗯𝗼𝘁 🤖 that displays a 𝒓𝒂𝒏𝒅𝒐𝒎 𝒊𝒎𝒂𝒈𝒆 from your Google Photos album every 12 𝒉𝒐𝒖𝒓𝒔 🖼️ 📸 Surprise your server with a random image twice a day 🤗

Price_tracker ⭐ 9

Dive into web scraping and build a Next.js 14 eCommerce price tracker within one project that teaches you data scraping, cron jobs, sending emails, deployment, and more.

Extract metadata about web pages

Garimpeiro Olx ⭐ 9

Procurando por algo na OLX? Seja avisado do melhor preço!

The unofficial Node JS API for looking up Minecraft users on Name MC

Scrapyteer ⭐ 9

Web crawling & scraping framework for Node.js on top of headless Chrome browser

Discord Scraping ⭐ 8

A little project that scrapes the discord web client and stores each version's main assets.

Botmation ⭐ 8

A simple TypeScript framework for declaratively composing bots with Puppeteer

Bargain Web Scraper ⭐ 8

Find highly discounted clothing items and be notified via a Telegram Bot and view via a React Web App.

Pobieranie Anime Z Polskich Stron ⭐ 8

Siema, jeślisz szukasz gotowego kodu do pobierania linków do cda, mega, google drive, itd z Polskich stron to idealnie trafiłeś!

Katatachi ⭐ 7

A Python framework to build web scraping applications.

GrocerEZ is a mobile app that uses a combination of web-scraping and crowd-sourcing to determine which store could fulfill your grocery list for the cheapest total

Van Tg Bot ⭐ 7

A K-Pop Telegram Bot

Monster Hunter Now Events ⭐ 6

A tool that auto-generates calendar events for Monster Hunter Now by scraping web news articles, processing them with AI, and creating a convenient calendar subscription.

Zenrows Node Sdk ⭐ 6

SDK to access ZenRows API directly from Node.js. We handle proxies rotation, headless browsers and CAPTCHAs for you.

Cyberpecker ⭐ 6

Get the Latest Scoop on Cyberspace with CyberPecker.

Uvic Course Scraper ⭐ 5

A Node.js scraper library for UVic Course and Schedule Information

Railsconf 2022 Schedule ⭐ 5

Web scrapes RailsConf's 2022 schedule to create an ICS link

Comix Notifier ⭐ 5

Reads comix.com.br's mangas pages configured, it use regex to find the last volume from each title and if it is a new volume, notify users.

Followers Scraper Serverless ⭐ 5

Now you can keep track of your followers from YouTube, Instagram and Twitter accounts - Followers scraper API on AWS serverless

Robust web spider for NodeJS

Spiderman ⭐ 5

Minimalistic web crawler for Node.js

Spiderboi ⭐ 5

A web crawling library written in TypeScript.

Zalando Scraper ⭐ 5

Apify actor extracting data from Zalando

Ah Negao Discord Bot ⭐ 5

🤖 A WIP discord bot that retrieves posts from "Ah Negão! - https://www.ahnegao.com.br" to a discord channel.

Related Searches

Typescript Angular (35,567)

Typescript Reactjs (31,235)

Typescript Command Line (21,993)

Typescript Testing (21,336)

Typescript Vue (5,971)

Typescript Nextjs (5,496)

Typescript Html (5,128)

Typescript Redux (5,035)

Typescript Docker (4,994)

Typescript Nest (4,761)

1-8 of 8 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.