Emailscanner

EmailScanner

EmailScanner is an integration application in python that uses exchangelib to process mail items in Microsoft exchange.

The purpose of this application is to aid and orchestrate security event analysis of reported phishing emails as well as related email security events.

Features

• ActiveDirectory inegration, processed messages are enriched using user or computer AD information.
• Elasticsearch backend is used to store enriched information
• Elasticsearch enrichment - historical phishing email submissions, FireEye detections and Falcon detections are looked up for each phishing email submission
• Cuckoosandbox integration: file attachments can be auto-submitted to a cuckoo instance
• MISP integration: reported phishing emails are scraped for URLs and other artifacts which are in turn queried against a configured MISP instance for matches
• TheHive integration: Processed phishing email submissions can generate an alert in Thehive
• FireEye json email alerts integration - Crowdstrike custom intel api integration ties into this as well (Emails blocked as a result of a malicious URL detection cause the email's domain to be uploaded as a custom IOC in Falcon)
• GeoIP lookup for the sender domain
• Specific folders can be defined for phishing email submissions and cuckoo submissions
• Results of the processing is prepended to the phishing email report.

EmailScanner assumes a typical Phishme/Cofense phishing email report as the "phishing email submission", where the reported phishing email is a .msg attachment or some other mail item attachment. The email body would contain meta-data regarding the report -- This body is modified (prepended to) with Emailscanner processing results.

Most of these features can be turned on/off as desired.

Configuration

The emailscanner.json example configuration file should serve as a reference configuration.

Some description of the configuration items:

• email_notify: Recipient of FireEye EX email alert notifications
• cuckooweb: Cuckoo sandbox web interface URL
• mailboxes: Exchange mailbox accounts to be scanned.
• certs: A domain and certificate path pair, the certificate specified will be used to verify the TLS connection to the Exchange server.
• email_server: Outbound SMTP server
• esenrichment: Boolean value to enable or disable Elasticsearch data enrichment
• mispkey: MISP api key
• misp_enabled: Boolean value to enable or disable MISP integration
• fireeyeaddress: Sender email address for FireEye alerts
• scannedfolders: Folders for which Cuckoosandbox attachment analysis is allowed
• falcon_customioc: Boolean value to enable or disable Crowdstrike Falcon custom IOC api
• phishing_report_address: Recipient address for reported phishing emails, this would typically be the same address as the inbox account.
• activedirectory-enrichment-configuration: A dictionary containing values that allow Emailscanner to connect to AD
• falconapi_url: Falcon custom IOC api URL
• activedirectory-enrichment: Boolean value to enable or disable AD enrichment
• mispui: MISP URL that is accessible by users
• email_alerts: Boolean value to enable or disable email alerting
• falconapi_key: Falcon custom IOC api key
• folders_indexed: Folders that are indexed in Elasticsearch, if you have different folders based on email types (e.g.: a folder for confirmed phishing emails), you should probably add them on this list. even if these folders are not in phishingemailfolders, indexing them will allow you to run reports to see what user,department,job title,etc... is reporting emails sorted to specific folders.
• elasticsearch_config: Elasticsearch instance configuration, this will let you use separate ES instances for FireEye and Phishing email submissions if desired.
• phishingemailfolders: Folders under which phishing email submissions are sorted (this is typically the root inbox folder but can include other folders where messages are sorted to)
• esenrichment_server: Elasticsearch server used for Elasticsearch enrichment
• thehive-url: URL to TheHive api
• cuckoowhitelist: Attachments with extensions in this list are not submitted to cuckoo
• falconapi_user: Falcon custom IOC api user
• mispurl: MISP URL (this could be different than mispui this does not have to be accessible by users,just by this app)
• elasticsearch: Boolean value to enable or disable Elasticsearch integration
• cuckooapi: CuckooSandbox api URL
• email_from: Sender address for email alerts generated by this script (Only FireEye alerts at this time)

Some items maybe missing.

Systemd service

A unit file is included as a reference, if you'd like to run his application as a systemd service.

Bugs and quality

This app is production ready and we use it extensively for everyday operations. With that in mind, there are plenty of bugs to be fixed. The code quality has plenty of room for improvement and it is constantly being improved and developed. For anyone attempting to use this application, please consider it a beta-grade application (alpha-grade if you consider the fireeyeformat and phishingformat scripts). Test it thoroughly before running it in production.

Please submit pull requests if you see any room for improvement or want to add features. Please excuse any typos, lack of commenting and poor variable naming as well.

TODO

• The **format modules are terrible, they need an overhaul (jinjna2 templating ,nicer string processing, etc....)
• Python3 support
• Fix how strings are processed
• Cleanup logging
• Fix pylint errors
• Test with optional features on/off