Awesome Open Source
Awesome Open Source

Scavenger - OSINT Bot

bot in action

Anurag's GitHub stats


Just the code of my OSINT bot searching for sensitive data leaks on different paste sites.

Search terms:

  • credentials
  • private RSA keys
  • Wordpress configuration files
  • MySQL connect strings
  • onion links
  • links to files hosted inside the onion network (PDF, DOC, DOCX, XLS, XLSX)
  • SQL dumps
  • API keys
  • complete emails

Keep in mind:

  1. This bot is not beautiful. I wrote it quick and dirty and do not care about code conventions or other shit... I will never care about those things.

  2. The code is not complete so far. Some parts like integrating the credentials in a database are missing in this online repository.

  3. If you want to use this code, feel free to do so. Keep in mind you have to customize things to make it run on your system.

  4. I know that I have some false positives and I know that I miss some credentials. So if you think this is crap...ok. leave now. If you have ideas for a better detection, just let me know!

  5. And again: QUICK AND DIRTY! Do not expect nice code.

Articles About Scavenger


For the bot can be run in two major modes:

  • API mode
  • Scraping mode (using TOR)

I highly recommend to use the API mode. It is the intended method of scraping pastes from and it is just fair to do so. The only thing you need is a PRO account and whitelist your public IP on their site.

To start the bot in API mode just run the program in the following way:

python3 -0

However, it is not always possible to use this intended method, as you might be in NAT mode and therefore you do not have an IP exclusively (whitelisting your IP is not reasonable here). That is the reason I also implemented a scraping mode where fast TOR cycles in combination with reasonable user agents are used to avoid IP blocking and Cloudflare captchas.

To start the bot in scraping mode run it in the following way:


python3 -1

Important note: you need the TOR service installed on your system listening on port 9050. Additionally you need to add the following line to your /etc/tor/torrc file.

MaxCircuitDirtiness 30

This sets the maximum cycle time of TOR to 30 seconds.

To start the module which scrapes random pastes of just type in the following command:

python3 -2


To learn how to use the software you just need to call the script with the -h/--help argument.

python3 -h


 /   _____/ ____ _____ ___  __ ____   ____    ____   ___________
 \_____  \_/ ___\\__  \\  \/ // __ \ /    \  / ___\_/ __ \_  __ \
 /        \  \___ / __ \\   /\  ___/|   |  \/ /_/  >  ___/|  | \/
/_______  /\___  >____  /\_/  \___  >___|  /\___  / \___  >__|
        \/     \/     \/          \/     \//_____/      \/

usage: [-h] [-0] [-1] [-2]

Control software for the different modules of this paste crawler.

optional arguments:
  -h, --help            show this help message and exit
  -0, --pastebinCOMapi  Activate module (using API)
  -1, --pastebinCOMtor  Activate module (standard scraping using
                        TOR to avoid IP blocking)
  -2, --pasteORG        Activate module

So far I implemented modules for the following paste sites:


If you want to observe specific mail addresses just add them to the file notification_targets.txt line by line. If the bot identifies one of these addresses on Pastebin it will write the name of the paste and the corresponding password to the file notification_results.txt.

Just start the module separately (first module I implemented)...


Pastes are stored in data/raw_pastes until they are more then 48000. When they are more then 48000 they get filtered, ziped and moved to the archive folder. All pastes which contain credentials are stored in data/files_with_passwords

Keep in mind that at the moment only combinations like USERNAME:PASSWORD and other simple combinations are detected. However, there is a tool to search for proxy logs containing credentials.

You can search for proxy logs (URLs with username and password combinations) by using file

python3 data/raw_pastes/

If you want to search the raw data for specific strings you can do it using (really slow).


To see statistics of the bot just call


The file searches a folder (with pastes) for sensitive data like credit cards, RSA keys or mysqli_connect strings. Keep in mind that this script uses grep and therefore is really slow on a big amount of paste files. If you want to analyze a big amount of pastes I recommend an ELK-Stack.

python3 data/raw_pastes/ 

There are two scripts which can be used to monitor a specific twitter user. This means every tweet he posts gets saved and every containing URL gets downloaded. To start the stalker just execute the wrapper.


To Do

I discovered other sites like Pastebin which allow to read the latest paste and crawl them. I need to integreate them into my bot. If you know additional sites which are worth a look, just let me know.


Get A Weekly Email With Trending Projects For These Topics
No Spam. Unsubscribe easily at any time.
python (51,899
bot (725
crawler (358
osint (208
pastebin (30
credentials (25
paste (16

Find Open Source By Browsing 7,000 Topics Across 59 Categories