In the age of the breach, it’s a safe assumption that almost every public account’s credentials have been exposed at some point. “Have I Been Pwned” (HIBP), is a database that contains usernames and other information about any compromise they come across. While available for individuals to search against, certain protections have been put in place to prevent DDoS attacks, making mass scanning using their public API difficult.
As offensive security, this information is very valuable during the passive reconnaissance phase of an engagement, and querying a single email at a time doesn’t scale well against an organization of 10,000 users. While many applications and scripts have been written and shared using the API’s, there wasn’t one available that successfully scans through an entire list of emails.
HIBP leverages CloudFlare as a web application firewall (WAF) that enforces brute-force protection through the use of 2 user-agent-based cookies and rate-limiting. To circumvent these controls, the script first reaches out to CloudFlare leveraging a pre-set user agent and obtains the authentication cookies using an opensource project known as cloudflare-scrape (cfscrape). The script then utilizes the obtained cookies and a built-in 2-second time delay between queries to conform to the rate-limit.
The script can identify whether a specific email address has been breached according to HIPB, obtain any paste information if present, search or obtain a list of breaches, and download a copy of all breaches contained within HIBP.
Example of searching emails for potential breaches and obtaining pastes if they exist within HIBP database
by Quentin Rhoads-Herrera