Darkweb Scraper: Guide and 3 Core Use Cases

Darkweb Scraper is an Apify actor that connects to Tor hidden services (.onion sites) without requiring external proxy configuration or API keys. It extracts structured data such as email addresses, phone numbers, cryptocurrency wallet addresses, social media handles, and exposed API keys across configurable discovery modes.

Define the operational target first

Before launching a run on the Tor network, establish your primary target data and discovery strategy. Dark web infrastructure differs significantly from clearnet websites: hidden services experience frequent downtime, latency is inherently higher, and page structures vary widely.

Choosing your operational mode defines how the crawler discovers content:

• Crawl mode (mode=crawl): The most reliable approach. You supply direct .onion addresses as seed URLs, and the actor crawls only those pages and their linked content according to your depth settings.
• Search mode (mode=search): A best-effort discovery method where you provide a keyword. The actor queries public dark web search engines (Ahmia, Torch, Haystack) to discover relevant .onion sites.
• Combined mode (mode=searchAndCrawl): Merges keyword search results with direct seed URLs, deduplicating discovered links to expand reach.

Practical use cases

Use case 1: Threat intelligence credential discovery

• Outcome: Identify leaked infrastructure credentials, API keys, and internal organizational references across hidden services.
• Question to answer: Are active cloud credentials or secret strings exposed on public hidden services?
• Configure: Set mode to crawl, provide seed .onion forum or paste URLs in startUrls, set maxDepth to 2, set maxPages to 25, set extractApiKeys to true, and set other extractors to false if not needed.
• Working method: Run the crawler against curated paste and discussion hidden services. Evaluate output records specifically targeting the misc.apiKeys field and verify if matching entries contain potential AWS keys, Google API keys, or generic secret strings.
• Deliverable: A structured dataset containing source .onion URLs, timestamps, and extracted API key strings for credential revocation.
• Stop condition: Crawl terminates when maxPages is reached or when no additional unvisited links remain at the specified crawl depth.

Use case 2: Brand protection and unauthorized domain monitoring

• Outcome: Detect counterfeit storefronts, illicit brand mentions, and unauthorized product distribution across hidden services.
• Question to answer: Is our brand name appearing on newly indexed dark web marketplaces or discussion boards, and what external contact channels are advertised?
• Configure: Set mode to search, enter your brand or product name into search, set maxDepth to 1, set maxPages to 20, set extractEmails to true, and set extractSocialHandles to true.
• Working method: Query dark web search engines with exact brand keywords. Filter the resulting dataset by checking whether searchKeywordFound is true, then extract associated communication channels found under emails or misc (Twitter/X, Instagram, Telegram).
• Deliverable: A monitoring register documenting offending .onion addresses, page titles, search term confirmation, and linked social or email identifiers for investigation.
• Stop condition: Completes when search engine discovery reaches its page limit or when the scraper exhausts the top-level search seed results.

Use case 3: Security research and cryptocurrency transaction attribution

• Outcome: Map financial touchpoints and operator attribution networks associated with malicious hidden services.
• Question to answer: Which cryptocurrency addresses and related contact points are published across a specific network of linked hidden services?
• Configure: Set mode to crawl, supply seed marketplace or service URLs in startUrls, set maxDepth to 3, set maxPages to 50, set extractCryptoAddresses to true, and set extractPhones to true.
• Working method: Seed the crawler with an initial service URL and let it traverse internal onion links. The scraper extracts cryptocurrency wallet addresses across supported chains (Bitcoin, Ethereum, Monero, Litecoin, Bitcoin Cash, Ripple) into the structured cryptoAddresses object while tracking discovered onion links via onionLinks.
• Deliverable: An entity-link spreadsheet containing mapped .onion domains, identified cryptocurrency wallets grouped by asset type, phone numbers, and cross-referenced host links.
• Stop condition: Ceases execution upon reaching the maxPages cap or when all discovered paths exceed maxDepth.

Build one workflow end to end

Follow this step-by-step workflow to configure and execute a reliable crawl run:

1. Identify seed targets: Collect one or more validated .onion addresses relevant to your investigation.
2. Select execution mode: Open Darkweb Scraper and set mode to crawl for predictable results, or search if discovering new domains via keywords.
3. Set scope boundaries: Configure maxDepth (for example, 1 for immediate links) and maxPages (for example, 5 for an initial test) to keep execution times manageable.
4. Configure extraction flags: Toggle specific extractors (extractEmails, extractPhones, extractCryptoAddresses, extractSocialHandles, extractApiKeys) based on your analysis objective.
5. Launch the actor: Start the run. The bundled Tor daemon will automatically bootstrap before establishing connections.
6. Export and process data: Consume the output dataset. Records will include fields such as url, title, links, onionLinks, scrapedAt, and specific extracted items.

Configure the documented inputs

The actor supports the following input properties:

• mode (Select, default: crawl): Discovery mode (crawl, search, or searchAndCrawl).
• search (String): Search keyword used when mode is set to search or searchAndCrawl.
• startUrls (Array of objects): Valid .onion URLs to crawl when using crawl or searchAndCrawl mode.
• maxDepth (Integer, default: 1, range: 0 to 10): Maximum link depth to traverse from seed pages. Depth 0 crawls only provided seed URLs.
• maxPages (Integer, default: 5, range: 1 to 5000): Maximum page fetch cap.
• maxItems (Integer, default: 5, range: 1 to 5000): Maximum number of output dataset items.
• extractEmails (Boolean, default: true): Finds embedded email addresses.
• extractPhones (Boolean, default: true): Identifies international format phone numbers.
• extractCryptoAddresses (Boolean, default: true): Extracts Bitcoin, Ethereum, Monero, Litecoin, Bitcoin Cash, and Ripple addresses.
• extractSocialHandles (Boolean, default: true): Extracts Twitter/X, Instagram, and Telegram usernames.
• extractApiKeys (Boolean, default: true): Identifies exposed AWS keys, Google API keys, and generic credential strings.

Account for operating limits and reliability

When operating on Tor hidden services, account for these technical constraints:

• Tor latency: Tor circuit establishment takes 10 to 30 seconds at startup, and individual page loads typically take 5 to 30 seconds.
• Bootstrap timeout: The actor waits up to 120 seconds for the Tor network to bootstrap. If bootstrapping fails, the run terminates with a failure status.
• No JavaScript rendering: The actor does not render JavaScript. Sites that depend on client-side JavaScript rendering may return incomplete data.
• Search engine volatility: Dark web search engines (Ahmia, Torch, Haystack) rotate addresses and go offline frequently. If search engines are offline, search mode may yield zero results.
• Target availability and retries: Hidden services frequently go offline. The scraper automatically retries page fetches up to 2 times with exponential backoff on timeout or network errors. HTTP 4xx and 5xx responses are logged and skipped.
• Omit-empty output behavior: The output dataset omits empty strings, empty arrays, and empty objects entirely rather than publishing empty keys.

Quality control checkpoints

• Check recordType and scrapedAt: Ensure every entry has recordType set to page and carries a valid UTC ISO-8601 timestamp in scrapedAt.
• Filter false positives: The actor automatically filters known false-positive domains from emails and cleans short or letter-containing patterns from phone numbers, but manual verification of extracted secrets is recommended.
• Examine onionLinks: Use the dedicated onionLinks list to differentiate Tor hidden services from clearnet links extracted on the same page.

Frequently asked questions

How should I validate the first run?

Set mode to crawl, enter a single reliable .onion address into startUrls, set maxDepth to 0, and set maxPages to 5. This confirms network bootstrap and verifies extractor behavior before expanding depth.

How should missing fields be handled?

The actor follows an omit-empty contract where empty arrays, objects, or strings are never emitted. If an extractor finds no matches on a page (for example, no exposed emails or cryptocurrency wallets), that field is omitted entirely from the dataset record.

When should I increase the run size?

Increase maxPages and maxDepth once you have validated that the seed site contains multi-page link trees and responsive page loading. Because Tor latency ranges from 5 to 30 seconds per page, keep run sizes proportional to your target review batch.

What should trigger a workflow review?

If a run returns zero records in search mode, review your search queries or switch to crawl mode with direct seed URLs, as dark web search indexes frequently suffer outages or address rotation.

Are search results consistent across runs?

No. Dark web search engines regularly rotate addresses and update their indexed hidden services. Search results can vary significantly across different runs depending on search engine uptime and network status.

Resources

• Darkweb Scraper on Apify