Awesome Open Source
Awesome Open Source

SecretAgent

SecretAgent is a web browser that's built for scraping.

  • [x] Built for scraping - it's the first modern headless browsers designed specifically for scraping instead of just automated testing.
  • [x] Designed for web developers - We've recreated a fully compliant DOM directly in NodeJS allowing you bypass the headaches of previous scraper tools.
  • [x] Powered by Chrome - The powerful Chrome engine sits under the hood, allowing for lightning fast rendering.
  • [x] Emulates any modern browser - BrowserEmulators make it easy to disguise your script as practically any browser.
  • [x] Avoids detection along the entire stack - Don't be blocked because of TLS fingerprints in your networking stack.

Check out our website for more details.

Installation

npm i --save secret-agent

or

yarn add secret-agent

Usage

SecretAgent provides access to the W3C DOM specification without the need for Puppeteer's complicated evaluate callbacks and multi-context switching:

const agent = require('secret-agent');

(async () => {
  await agent.goto('https://example.org');
  const title = await agent.document.title;
  const intro = await agent.document.querySelector('p').textContent;
  await agent.close();
})();

Browse the full API docs.

Contributing

We'd love your help in making SecretAgent a better tool. Please don't hesitate to send a pull request.

License

MIT


Get A Weekly Email With Trending Projects For These Topics
No Spam. Unsubscribe easily at any time.
Typescript (247,336
Proxy (3,379
Browser (2,634
Puppeteer (1,369
Scraping (1,299
Devtools (751
Chromium (524
Automated (308
Mitm (193
Playwright (184
Related Projects