# SimilarWeb Fast Scraper — Website Traffic, Rankings & Analytics (`pro100chok/similarweb-scraper`) Actor

Extract SimilarWeb data for any domain in seconds. Get traffic stats, global rankings, top keywords, traffic sources,
AI traffic insights, and similar sites — all in one fast parallel run.

- **URL**: https://apify.com/pro100chok/similarweb-scraper.md
- **Developed by:** [Raven](https://apify.com/pro100chok) (community)
- **Categories:** SEO tools, Automation, Developer tools
- **Stats:** 1,067 total users, 355 monthly users, 100.0% runs succeeded, 13 bookmarks
- **User rating**: 5.00 out of 5 stars

## Pricing

from $1.50 / 1,000 domain processeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## SimilarWeb Scraper — Website Traffic, Rankings, WHOIS, and Keyword Density

| 🦅 More from Raven ► | [Ahrefs SEO Tools](https://apify.com/pro100chok/ahrefs-seo-tools) | [Semrush Scraper](https://apify.com/pro100chok/semrush-scraper) | [Similarweb Scraper](https://apify.com/pro100chok/similarweb-scraper) |
|---|---|---|---|

Extract website analytics from three data sources: SimilarWeb traffic data, similar sites discovery, and AITDK domain intelligence. Choose one source per run via the **Search Type** dropdown. Analyze up to 50 domains in parallel.

### 📺 Video Tutorial

[![SimilarWeb Scraper Video Tutorial](https://img.youtube.com/vi/KaJKMsp_cFI/maxresdefault.jpg)](https://www.youtube.com/watch?v=KaJKMsp_cFI)

### Pricing

| Plan | Price | Best For | Link |
|------|-------|----------|------|
| **Pay per result** | $2 / 1,000 results | One-time analysis, small batches | [SimilarWeb Scraper](https://apify.com/pro100chok/similarweb-scraper) |
| **Monthly subscription** | $20 / month | Regular monitoring, unlimited usage | [SimilarWeb Website Analytics](https://apify.com/pro100chok/similarweb-website-analytics) |

Both plans have identical functionality — choose the one that fits your workflow.

### Search Types

Select one of three modes in the **Search Type** dropdown:

| Search Type | Description | Output Fields |
|-------------|-------------|---------------|
| **SimilarWeb** | Traffic data, rankings, engagement metrics, traffic sources | SiteName, Title, Category, GlobalRank, CountryRank, Engagments, EstimatedMonthlyVisits, TrafficSources, TopCountryShares |
| **Similar Sites** | Competitor and alternative websites | SiteName, SimilarSites (Site, TotalVisits, CategoryRank, TopCountryRank, Description) |
| **AITDK** | WHOIS domain data and keyword density analysis | SiteName, Whois (registrationDate, expirationDate, registrar, nameservers, status), KeywordDensity (1-5 word phrases) |

### Key Features

- **Bulk domain analysis** — submit up to 50 domains at once, all processed simultaneously
- **3 data sources** — choose SimilarWeb traffic data, similar sites, or AITDK domain intelligence per run
- **Traffic estimates** — monthly visits, bounce rate, pages per visit, average session duration, and 3-month visit history (SimilarWeb mode)
- **Global and country rankings** — worldwide rank, country-specific rank, and industry category rank (SimilarWeb mode)
- **Traffic source breakdown** — percentage split across direct, organic search, paid search, social, referral, email, and display ads (SimilarWeb mode)
- **Geographic distribution** — top 5 countries driving traffic with share percentages (SimilarWeb mode)
- **Similar sites discovery** — related websites with their own traffic data, category rank, and descriptions (Similar Sites mode)
- **WHOIS data** — domain registration date, expiration date, registrar, nameservers, registrant country, and domain status (AITDK mode)
- **Keyword density analysis** — 1-word through 5-word phrase frequency analysis with stop word filtering, 50 languages (AITDK mode)
- **Error resilience** — failed items are logged with error details, other domains still process normally

### What Data You Get

#### SimilarWeb mode

| Field | Description |
|-------|-------------|
| `SiteName` | Domain name |
| `Title` | Website title |
| `Category` | Industry category (e.g. `computers_electronics_and_technology/search_engines`) |
| `GlobalRank` | Worldwide traffic rank |
| `CountryRank` | Rank within the primary traffic country |
| `Engagments` | Total visits, bounce rate, pages per visit, session duration |
| `EstimatedMonthlyVisits` | Last 3 months of visit data (raw numbers and formatted) |
| `TrafficSources` | Percentage split by channel: Direct, Search, Social, Referrals, Mail, Paid |
| `TopCountryShares` | Top 5 countries with traffic share percentages |

#### Similar Sites mode

| Field | Description |
|-------|-------------|
| `SiteName` | Domain name |
| `SimilarSites` | Array of related websites with `Site`, `TotalVisits`, `CategoryRank`, `TopCountryRank`, `Description` |

#### AITDK mode

| Field | Description |
|-------|-------------|
| `SiteName` | Domain name |
| `Whois.registrationDate` | Domain registration date |
| `Whois.expirationDate` | Domain expiration date |
| `Whois.lastUpdated` | Last WHOIS record update |
| `Whois.registrar` | Domain registrar name |
| `Whois.registrantCountry` | Registrant country code |
| `Whois.nameservers` | List of DNS nameservers |
| `Whois.status` | Domain status flags (e.g. `client transfer prohibited`) |
| `KeywordDensity.1word` | Single-word frequency (stop words filtered) |
| `KeywordDensity.2words` | 2-word phrase frequency |
| `KeywordDensity.3words` | 3-word phrase frequency |
| `KeywordDensity.4words` | 4-word phrase frequency |
| `KeywordDensity.5words` | 5-word phrase frequency |

Each keyword density entry contains: `keyword`, `count`, `total` (total words on page), `density` (percentage). Sorted by density descending.

### Use Cases

- **Competitive analysis** — track competitors' traffic trends and traffic sources to spot gaps and opportunities (SimilarWeb mode)
- **On-page SEO audit** — use keyword density analysis to check keyword stuffing, content relevance, and phrase optimization across 1-5 word n-grams (AITDK mode)
- **Domain research** — check domain age, expiration dates, registrar history, and DNS configuration via WHOIS data (AITDK mode)
- **Marketing budget allocation** — understand where competitors get traffic (search, social, direct, paid) and optimize your channel mix (SimilarWeb mode)
- **Lead generation and sales intelligence** — enrich your CRM with website traffic data to qualify leads and prioritize outreach (SimilarWeb mode)
- **Investment due diligence** — evaluate company digital performance using traffic trends, monthly visit history, domain age, and market positioning
- **Partnership discovery** — find similar websites and potential partners in your niche (Similar Sites mode)

### How It Works

1. **Select search type** — choose SimilarWeb, Similar Sites, or AITDK from the dropdown
2. **Enter domains** — provide up to 50 domains (e.g. `google.com`, `shopify.com`)
3. **Actor fetches data** — all domains processed in parallel using the selected data source
4. **Download results** — structured JSON output in the Apify dataset, ready to export as JSON, CSV, or Excel

### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `searchType` | String | Yes | Data source: `similarweb` (traffic, rankings), `similar_sites` (competitors), or `aitdk` (WHOIS, keyword density) |
| `domains` | String\[] | Yes | List of domains to analyze, max 50 per run (e.g. `google.com`, `facebook.com`) |

#### Input examples

**SimilarWeb mode (traffic data):**

```json
{
    "searchType": "similarweb",
    "domains": ["google.com", "facebook.com", "shopify.com"]
}
```

**Similar Sites mode:**

```json
{
    "searchType": "similar_sites",
    "domains": ["google.com"]
}
```

**AITDK mode (WHOIS + keyword density):**

```json
{
    "searchType": "aitdk",
    "domains": ["google.com"]
}
```

### Output Examples

#### SimilarWeb mode

```json
{
    "SiteName": "google.com",
    "Title": "Google",
    "Category": "computers_electronics_and_technology/search_engines",
    "GlobalRank": { "Rank": 1 },
    "CountryRank": { "Country": 840, "CountryCode": "US", "Rank": 1 },
    "Engagments": {
        "Visits": 85756574615,
        "VisitsFormatted": "85.76B",
        "BounceRate": 28.36,
        "PagePerVisit": 8.52,
        "TimeOnSite": 610
    },
    "EstimatedMonthlyVisits": {
        "2025-11-01": 82284033110,
        "2025-12-01": 84172772881,
        "2026-01-01": 85756574615
    },
    "TrafficSources": {
        "Direct": 85.83,
        "Search": 8.35,
        "Referrals": 4.46,
        "Social": 0.76,
        "Paid Referrals": 0.33,
        "Mail": 0.28
    },
    "TopCountryShares": [
        { "CountryCode": "US", "Value": 24.59 },
        { "CountryCode": "IN", "Value": 5.55 },
        { "CountryCode": "JP", "Value": 5.47 }
    ]
}
```

#### Similar Sites mode

```json
{
    "SiteName": "google.com",
    "SimilarSites": [
        { "Site": "bing.com", "TotalVisits": 1234567890, "CategoryRank": 2, "TopCountryRank": 3, "Description": "..." },
        { "Site": "yahoo.com", "TotalVisits": 987654321, "CategoryRank": 5, "TopCountryRank": 8, "Description": "..." }
    ]
}
```

#### AITDK mode

```json
{
    "SiteName": "google.com",
    "Whois": {
        "registrationDate": "1997-09-15T04:00:00Z",
        "expirationDate": "2028-09-14T04:00:00Z",
        "lastUpdated": "2019-09-09T15:39:04Z",
        "registrar": "MarkMonitor Inc.",
        "registrantCountry": "US",
        "nameservers": ["NS1.GOOGLE.COM", "NS2.GOOGLE.COM", "NS3.GOOGLE.COM", "NS4.GOOGLE.COM"],
        "status": ["client delete prohibited", "client transfer prohibited", "client update prohibited"]
    },
    "KeywordDensity": {
        "1word": [
            { "keyword": "google", "count": 5, "total": 366, "density": 1.37 },
            { "keyword": "english", "count": 5, "total": 366, "density": 1.37 }
        ],
        "2words": [
            { "keyword": "cookies und", "count": 3, "total": 366, "density": 0.82 }
        ],
        "3words": [
            { "keyword": "cookies und daten", "count": 3, "total": 366, "density": 0.82 }
        ],
        "4words": [
            { "keyword": "verwenden wir cookies und", "count": 3, "total": 366, "density": 0.82 }
        ],
        "5words": [
            { "keyword": "verwenden wir cookies und daten", "count": 2, "total": 366, "density": 0.55 }
        ]
    }
}
```

### Troubleshooting

| Symptom | Likely Cause | Fix |
|---------|-------------|-----|
| Single-domain failure | Domain has no SimilarWeb data | Check the domain name; this is expected for domains without data |
| WHOIS data missing | Rate limit on the WHOIS lookup | Normal for high-volume runs — other data is still returned |

### FAQ

**What are the search types?**
There are 3 modes: **SimilarWeb** returns traffic and ranking data, **Similar Sites** returns competitor websites, **AITDK** returns WHOIS domain info and keyword density analysis. Select one per run.

**How many domains can I analyze per run?**
Up to 50 domains in a single run, all processed in parallel. Typical response time is 1–10 seconds per domain.

**Does the scraper work with subdomains?**
Yes. Pass subdomains like `translate.google.com` or `maps.google.com` and get separate analytics for each.

**What happens if a domain fails?**
The Actor continues processing all remaining domains. Failed items include an error field with details.

**How does keyword density work?**
In AITDK mode, the Actor fetches the page HTML, strips non-visible elements (scripts, styles, iframes, etc.), tokenizes the visible text, and calculates n-gram frequencies for 1-5 word phrases. Stop words from 50 languages are filtered from single-word results.

**What's the difference between the monthly and pay-per-use plans?**
Identical functionality. Monthly ($20/mo) is better for ongoing monitoring; pay-per-use ($2/1k results) is better for occasional analysis.

### Having issues? Help me fix them faster

If you experience any problems, please share your run data with me so I can debug and improve the Actor:

1. Go to [Apify Security Settings](https://console.apify.com/settings/security)
2. Find **"Share run data with developers"**
3. In the **"Manage list of Actors"** section, check this Actor (or **All Actors**)
4. Save

This data is used **only for debugging** and helps me resolve issues much faster. Thank you!

### Support

Questions, bug reports, or feature requests: **afrcanec@gmail.com**

### 🏷️ Tags

SimilarWeb Scraper, Website Traffic Analyzer, SimilarWeb API, Website Analytics Extractor, Global Ranking Checker, Traffic Source Breakdown, Organic Keywords Extractor, AI Traffic Tracker, Domain WHOIS Lookup, Keyword Density Analyzer, Competitor Traffic Analysis, Website Ranking Data, SEO Analytics Tool, SimilarWeb Data Extractor, Digital Marketing Intelligence, Monthly Visits Tracker, Bounce Rate Analyzer, Similar Sites Finder, How to Scrape SimilarWeb, Extract Website Traffic Data, Domain Research Tool, Market Share Analysis, Website Comparison Tool, Referral Traffic Analytics, Country Traffic Distribution, Web Analytics Scraper, Investment Due Diligence, Lead Qualification Data, Website Performance Metrics, AI Referral Traffic Monitor

# Actor input Schema

## `searchType` (type: `string`):

Select the type of data to retrieve.

## `domains` (type: `array`):

List of domains to analyze (e.g. google.com, facebook.com). Maximum 50 per run.

## Actor input object example

```json
{
  "searchType": "similarweb",
  "domains": [
    "google.com",
    "facebook.com",
    "youtube.com",
    "twitter.com",
    "instagram.com",
    "linkedin.com",
    "amazon.com",
    "wikipedia.org",
    "reddit.com",
    "netflix.com"
  ]
}
```

# Actor output Schema

## `overview` (type: `string`):

No description

## `similar_sites` (type: `string`):

No description

## `aitdk` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchType": "similarweb",
    "domains": [
        "google.com",
        "facebook.com",
        "youtube.com",
        "twitter.com",
        "instagram.com",
        "linkedin.com",
        "amazon.com",
        "wikipedia.org",
        "reddit.com",
        "netflix.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("pro100chok/similarweb-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "searchType": "similarweb",
    "domains": [
        "google.com",
        "facebook.com",
        "youtube.com",
        "twitter.com",
        "instagram.com",
        "linkedin.com",
        "amazon.com",
        "wikipedia.org",
        "reddit.com",
        "netflix.com",
    ],
}

# Run the Actor and wait for it to finish
run = client.actor("pro100chok/similarweb-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchType": "similarweb",
  "domains": [
    "google.com",
    "facebook.com",
    "youtube.com",
    "twitter.com",
    "instagram.com",
    "linkedin.com",
    "amazon.com",
    "wikipedia.org",
    "reddit.com",
    "netflix.com"
  ]
}' |
apify call pro100chok/similarweb-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=pro100chok/similarweb-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/aqPbs3KeH9aD8b22w/builds/ecmKwpJ2S90drKoPX/openapi.json