# Ahrefs Scraper (`scraper-engine/ahrefs-scraper`) Actor

🔎 Ahrefs Scraper extracts SEO data from Ahrefs to power competitive research, backlink analysis, and content strategy. 🚀 Automate insights, save time, and scale campaigns with cleaner datasets. 📈

- **URL**: https://apify.com/scraper-engine/ahrefs-scraper.md
- **Developed by:** [Scraper Engine](https://apify.com/scraper-engine) (community)
- **Categories:** SEO tools, Automation, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $4.99 / 1,000 results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

### Ahrefs Scraper — Domain Rating, Traffic, Backlinks and Keywords

Ahrefs Scraper pulls SEO data from Ahrefs' public free tools and returns it as structured JSON: Domain Rating, backlink and referring-domain counts, estimated monthly traffic and traffic value, keyword ideas, keyword difficulty, full SERP overviews, broken links, AI-citation visibility, and the AhrefsTop trending-websites ranking. Nine report types, one dataset row each, written live as every result lands — no HTML parsing, no selectors. Pick the reports you want with checkboxes and press Start.

⚠️ **Read the captcha requirement below before your first run.** Ahrefs protects every free tool with a Cloudflare Turnstile challenge, so this Actor needs your own CapSolver or 2Captcha key. Without one it stops immediately with an error rather than returning empty rows.

### What is Ahrefs Scraper?

Ahrefs Scraper is an Apify Actor that queries Ahrefs' free public SEO tools and shapes the responses into clean, typed records. Each report type you enable produces its own dataset row, tagged with a `type` field, so one run can cover authority, traffic, backlinks and keyword research for the same target.

No Ahrefs account, login or subscription is required — these are the free tools any visitor can use. What you do need is a captcha-solver API key, because Ahrefs gates those tools behind Cloudflare Turnstile.

It is built for SEO consultants auditing sites at scale, agencies running competitor research across a client list, and developers piping authority and keyword metrics into their own dashboards.

### What Ahrefs data is publicly available to scrape?

Ahrefs publishes a set of free tools that return real metrics without an account — but each is a capped preview of what the paid product shows, and each sits behind a Turnstile challenge.

| Data Category | Free tools (what this Actor reads) | Paid Ahrefs subscription |
| ----- | ----- | ----- |
| Domain Rating, backlink and referring-domain counts | ✅ Public | Full historical series |
| Estimated monthly traffic, traffic value, top pages/countries/keywords | ✅ Public | Full breakdown and exports |
| Keyword ideas and question ideas with volumes | ✅ Public | Complete keyword database |
| Keyword Difficulty score and its SERP | ✅ Public | Full metrics per result |
| SERP overview for a keyword | ✅ Public | Historical SERP tracking |
| Where a domain ranks for a keyword | ✅ Public | Rank tracking over time |
| Broken inbound and outbound links | ✅ Public | Full Site Audit crawl |
| AI citation visibility across LLMs | ✅ Public | Brand Radar |
| AhrefsTop trending websites ranking | ✅ Public — no captcha needed | — |
| Full backlink lists, anchor text, link history | ❌ | Subscription only |
| Site Audit crawls, Rank Tracker projects | ❌ | Subscription only |

Ahrefs Scraper only returns publicly visible data — what any visitor sees on Ahrefs' free tools. Nothing behind a login wall or a paid plan.

### ⚠️ You must supply a captcha-solver key

This is the one prerequisite, and it is worth stating plainly rather than burying it in the input table.

Ahrefs protects every free tool with a **Cloudflare Turnstile** challenge. The Actor cannot fetch any of the `/v4` tool data without solving it, so it needs a solver key from **CapSolver** or **2Captcha** — accounts you create and fund yourself at `capsolver.com` or `2captcha.com`.

Supply it either way:

- Paste it into the **Captcha API key** (`captchaApiKey`) input — stored encrypted
- Or set the `AHREFS_CAPTCHA_KEY` environment variable on the Actor

If neither is present, the run logs an explicit error naming both options and exits without pushing any rows. It does not silently return empty results, and it does not charge you for rows it could not fetch.

One exception: the **AhrefsTop trending websites** report (`include_top_websites`) reads a plain server-rendered page with no captcha and no `/v4` call, so it works without a solver key.

### What data can I extract with Ahrefs Scraper?

Every row carries `data_captured_at` and a `type` field identifying which report it is, plus the identity fields for that report — `domain` and `mode`, or `keyword` and `country`, or `brand`. The rest of the row depends on the type.

| Field Name | Description |
| ----- | ----- |
| `data_captured_at` | UTC ISO timestamp for the record — on every row |
| `type` | Report type: `traffic`, `backlinks`, `broken_links`, `keywords`, `keyword_difficulty`, `keyword_ranking`, `serp`, `ai_visibility`, `top_websites` |
| `domain` | Bare host the record is about, normalised (scheme and `www.` stripped) |
| `mode` | `subdomains` or `exact` — the scope the domain report was run at |
| `keyword` | The researched keyword, on keyword-based records |
| `country` | Country code the keyword/SERP data was localised to |
| `brand` | The researched brand, on `ai_visibility` records |
| `website_authority` | Object: `domainRating`, `backlinks`, `refdomains`, `dofollowBacklinks`, `dofollowRefdomains` |
| `website_traffic` | Object: `trafficMonthlyAvg`, `costMontlyAvg` |
| `website_traffic_history` | Monthly traffic series |
| `website_traffic_top_pages` | Highest-traffic pages on the domain |
| `website_traffic_top_countries` | Traffic split by country |
| `website_traffic_top_keywords` | Keywords driving the most traffic |
| `keyword_difficulty` | Difficulty score for the keyword |
| `keyword_difficulty_results` | Decoded SERP behind the difficulty score |
| `keyword_difficulty_source` | Source label Ahrefs reported for that SERP |
| `keyword_difficulty_shortage` | Ahrefs' data-shortage indicator for the keyword |
| `serp_results` | Decoded SERP results — each with `content_type`, `data`, `pos`, `pos_with_metrics` |
| `serp_source` | Source label for the SERP snapshot |
| `top_position` | Best position the domain holds for the keyword, decoded like a SERP result |
| `last_update` | Ahrefs' own last-update stamp for the SERP data |
| `top_websites_category` | Category filter applied to the AhrefsTop ranking |
| `top_websites_country` | Country filter applied to the AhrefsTop ranking |
| `websites_rankings` | Main ranking rows: `rank`, `domain`, `search_traffic`, `change`, `category` |
| `website_rankings_trending_up` | Trending-up rows: same shape plus `position_change` |
| `website_rankings_trending_down` | Trending-down rows: same shape plus `position_change` |
| `data` | Full decoded payload on `backlinks`, `broken_links`, `keywords` and `ai_visibility` records |
| `error` | Ahrefs' error message when a report came back empty |
| `website_authority_error` | Error specific to the authority half of a `traffic` record |
| `website_traffic_error` | Error specific to the traffic half of a `traffic` record |

> `costMontlyAvg` is spelled exactly that way — it is Ahrefs' own typo in the upstream response, preserved rather than silently corrected so your field mapping matches what actually arrives.

#### Domain and authority reports

`include_web_authority` and `include_traffic` merge into a single `traffic` record per target. Authority gives you `domainRating`, `backlinks`, `refdomains` and the dofollow variants of both; traffic gives you `trafficMonthlyAvg` and `costMontlyAvg` plus four breakdown arrays — history, top pages, top countries, top keywords. `include_backlinks` and `include_broken_links` add their own rows with the full decoded payload under `data`.

`mode` controls scope: `subdomains` covers the whole site including subdomains, `exact` covers only the exact URL or host you supplied.

#### Keyword and SERP reports

`include_keywords` returns keyword ideas and question ideas with volumes. `include_keywords_difficulty` returns the difficulty score plus the SERP it was calculated from. `include_serp` returns the full SERP overview. `include_keywords_ranking` needs a keyword *and* a domain and returns `top_position` — where that domain actually sits for that keyword. All four localise to `country`.

SERP entries are decoded rather than passed through raw: each carries `content_type` (organic, AI overview, questions, and so on), the decoded `data`, `pos`, and `pos_with_metrics`.

#### AI visibility and trending websites

`include_ai_visibility` takes a brand or keyword and returns AI citation counts across LLMs, with the model breakdown and top cited domains under `data`. `include_top_websites` needs no keyword or domain at all — it returns the AhrefsTop ranking filtered by `category_top_websites` and `country_top_websites`, with separate trending-up and trending-down tables.

#### 🤖 Add-on: Need additional SEO data?

**MOZ Scraper** covers the same authority-metric job from a second data source, which is useful when you want a cross-check rather than a single vendor's score. For the search side, **Quora Search Scraper** and **Youtube Search Scraper** surface question and video intent that keyword tools tend to miss.

### How does Ahrefs Scraper differ from the official Ahrefs API?

Ahrefs publishes an official API, and if you already pay for it, use it — it is the supported route, it returns Ahrefs' full dataset rather than the free-tool preview, and it comes with terms you can rely on commercially. Ahrefs Scraper exists for the case where you do not have that subscription.

| Feature | Official Ahrefs API | Ahrefs Scraper |
| ----- | ----- | ----- |
| Paid Ahrefs subscription required | ✅ | ❌ |
| Data scope | Ahrefs' full index | The free tools' capped preview |
| Captcha-solver key required | ❌ | ✅ CapSolver or 2Captcha |
| AhrefsTop trending websites | Not part of the API | ✅ Included |
| Historical depth | Full series | What the free tools expose |
| Output shape | Ahrefs' documented API schema | One normalised row per report type |

Check Ahrefs' current published API documentation for exact plan requirements, endpoints and unit costs before choosing between them — those terms change and are not reproduced here.

Use the official API for production workloads, contractual guarantees, and full-index depth. Use Ahrefs Scraper for ad-hoc research, prospect audits, and pulling free-tool metrics for many domains without a subscription.

### How to use Ahrefs Scraper

Ahrefs Scraper runs on Apify. Start it from the Apify Console or call it through the Apify API.

1. Get a **CapSolver** or **2Captcha** API key — this is the prerequisite, not an optional extra
2. Open Ahrefs Scraper on Apify and click **Try for free**
3. Paste your solver key into **Captcha API key**
4. Give the Actor something to work on: a **URL or Domain**, a **Keyword or Brand**, or tick **Include AhrefsTop Trending Websites** — no input is marked required in the schema, but a run with none of these three logs "nothing to do" and exits
5. Tick the reports you want. `include_web_authority`, `include_traffic` and `include_backlinks` are on by default; the other six are off
6. Set **Country** and **Mode**, click **Start**, and export as JSON, CSV or Excel

Match the report to the input it needs, or it is silently skipped: authority, traffic, backlinks and broken links need a domain; keyword ideas, difficulty and SERP need a keyword; keyword ranking needs both; AI visibility needs a brand.

#### How to scale to bulk domain analysis

Use **Multiple URLs or Domains (Bulk)** (`urls`) — it is a list, and when set it overrides the single `url` field entirely. Every domain-based report you enabled runs once per entry, so ten domains with authority, traffic and backlinks ticked produce thirty rows in one run.

Keyword-based reports behave differently: `keywords`, `keyword_difficulty`, `serp` and `ai_visibility` run **once per run**, not once per URL, because they key off the single `keyword` field. `keyword_ranking` is the exception — it runs per URL, pairing your one keyword against each domain, which is exactly what you want for a share-of-voice comparison.

### What can you do with Ahrefs data?

- 🔍 **An SEO consultant** qualifying inbound leads pastes 40 prospect domains into `urls` and sorts the results by `website_authority.domainRating` and `website_traffic.trafficMonthlyAvg` to decide which are worth a proposal.
- 🏢 **An agency** running a competitive audit enables `include_keywords_ranking` with one target keyword and a list of competitor domains, then reads `top_position` from each row to build a share-of-voice table in a single run.
- 🔗 **A link-building team** enables `include_broken_links` across a prospect list and works the `data` payload's inbound broken links into a reclamation outreach queue.
- 📊 **A content strategist** pairs `include_keywords` with `include_keywords_difficulty` for a seed term, then filters keyword ideas against `keyword_difficulty` to find volume that is actually winnable.
- 🤖 **An AI engineer** building an SEO assistant indexes `website_traffic_top_keywords` and `serp_results` into a vector store so an agent can answer "which pages drive this competitor's traffic, and who outranks us for their top term" against live data rather than a stale export.
- 📈 **A brand manager** tracking AI search enables `include_ai_visibility` on a brand name and monitors citation counts by model over repeated runs.

Every one of these is callable from an agent framework over the Apify API, since the Actor is a standard HTTP-triggered run.

### How does Ahrefs Scraper handle rate limits and blocking?

Two separate defences, handled independently.

**Turnstile** is solved through your CapSolver or 2Captcha key — that is what the key is for, and it is why the run cannot proceed without one.

**IP blocking** uses an escalation ladder that starts at the cheapest rung. Every run begins on a **direct connection with no proxy at all**. Only if Ahrefs blocks a request does the Actor climb: direct → Apify datacenter → Apify residential, logging each switch. Once on residential it retries up to three times before giving up on that request. Setting a country in the proxy input localises the residential fallback. Most runs never leave the direct rung, and the final log line tells you which tier the run actually completed on.

When a single report fails, its error message is captured on the record itself — `error` on most types, or `website_authority_error` / `website_traffic_error` on the split `traffic` record — and the run continues with the remaining reports. You get a row that tells you what went wrong rather than a missing row you have to notice.

### ⬇️ Input

| Parameter | Required | Type | Description | Example Value |
| ----- | ----- | ----- | ----- | ----- |
| `keyword` | No | string | The keyword (or brand, for AI Visibility) to research on Ahrefs. | `"project management software"` |
| `url` | No | string | The single URL or domain to analyse. For many targets at once, use the bulk field. | `"make.com"` |
| `country` | No | string | Country to localise keyword, SERP and traffic results. 188 ISO alpha-2 values in the dropdown. Default `"us"`. | `"gb"` |
| `mode` | No | string | Scope of the domain analysis: `"subdomains"` (whole site incl. subdomains) or `"exact"` (just the exact URL/host). Default `"subdomains"`. | `"subdomains"` |
| `urls` | No | array | A list of URLs or domains to analyse. When set, overrides the single `url` field. | `["make.com", "zapier.com"]` |
| `include_web_authority` | No | boolean | Needs a Domain/URL. Includes Domain Rating, backlinks & referring-domain counts. Default `true`. | `true` |
| `include_traffic` | No | boolean | Needs a Domain/URL. Includes estimated monthly visits, traffic value, history, top pages/countries/keywords. Default `true`. | `true` |
| `include_ai_visibility` | No | boolean | Needs a Brand/Keyword. Includes AI citations across LLMs, top cited domains/pages & topics. Default `false`. | `false` |
| `include_keywords` | No | boolean | Needs a Keyword and Country. Includes keyword ideas + question ideas with volumes. Default `false`. | `true` |
| `include_keywords_difficulty` | No | boolean | Needs a Keyword and Country. Includes the Keyword Difficulty score + ranking SERP. Default `false`. | `true` |
| `include_keywords_ranking` | No | boolean | Needs a Keyword, Domain and Country. Includes where the domain ranks for the keyword. Default `false`. | `false` |
| `include_serp` | No | boolean | Needs a Keyword and Country. Includes the full SERP overview for the keyword. Default `false`. | `false` |
| `include_backlinks` | No | boolean | Needs a Domain/URL. Includes the backlink & referring-domain overview. Default `true`. | `true` |
| `include_broken_links` | No | boolean | Needs a Domain/URL. Includes broken inbound & outbound links. Default `false`. | `false` |
| `include_top_websites` | No | boolean | Includes the top trending websites from AhrefsTop.com — the Top 1000 (worldwide + all categories) or Top 100 when filtered by category/country. Does not need a Keyword or Domain. Default `false`. | `false` |
| `category_top_websites` | No | string | Category filter for the AhrefsTop ranking. 26 values from `all` through `autos-and-vehicles`. Default `"all"`. | `"finance"` |
| `country_top_websites` | No | string | Country filter for the AhrefsTop ranking. 189 values including `worldwide`. Default `"worldwide"`. | `"worldwide"` |
| `proxyConfiguration` | No | object | The scraper starts on a DIRECT connection and only escalates to a datacenter, then residential proxy if Ahrefs blocks it. Pick a country here to localise residential fallback. Default `{"useApifyProxy": true}`. | `{"useApifyProxy": true}` |
| `captchaApiKey` | No | string | **Required in practice.** Ahrefs protects every free tool with a Cloudflare Turnstile challenge, so this Actor needs a captcha-solver key to fetch data. Paste your CapSolver or 2Captcha API key. Stored encrypted. Can also be set via the `AHREFS_CAPTCHA_KEY` environment variable. | `"CAP-XXXXXXXXXXXX"` |

**Common pitfall:** `captchaApiKey` is not flagged required by the schema, but every `/v4` report needs it. A run without it exits with an error before pushing anything.

#### Example input

```json
{
  "urls": ["make.com", "zapier.com", "n8n.io"],
  "keyword": "workflow automation",
  "country": "us",
  "mode": "subdomains",
  "include_web_authority": true,
  "include_traffic": true,
  "include_backlinks": true,
  "include_broken_links": false,
  "include_keywords": true,
  "include_keywords_difficulty": true,
  "include_keywords_ranking": true,
  "include_serp": false,
  "include_ai_visibility": false,
  "include_top_websites": false,
  "category_top_websites": "all",
  "country_top_websites": "worldwide",
  "proxyConfiguration": { "useApifyProxy": true },
  "captchaApiKey": "CAP-XXXXXXXXXXXX"
}
```

### ⬆️ Output

Typed, normalized JSON with a consistent shape per `type` across runs. Rows are pushed live as each report lands, so the dataset fills while the run is still going. Export as JSON, CSV or Excel, or read the dataset through the Apify API.

Ahrefs encodes its responses in an OCaml/Melange style — options as `["Some", x]`, results as `["Ok", x]`, enums as single-element arrays. All of it is decoded recursively before it reaches you, so what lands in the dataset is plain JSON with `null` where Ahrefs sent `None`.

#### Example output

A `traffic` record, the type produced by the two reports that are on by default:

```json
{
  "data_captured_at": "2026-07-25T09:14:02.481230+00:00",
  "type": "traffic",
  "domain": "make.com",
  "mode": "subdomains",
  "website_authority": {
    "domainRating": 91,
    "backlinks": 4182337,
    "refdomains": 38412,
    "dofollowBacklinks": 3120884,
    "dofollowRefdomains": 29760
  },
  "website_traffic": {
    "trafficMonthlyAvg": 2841500,
    "costMontlyAvg": 1934200
  },
  "website_traffic_history": [
    { "date": "2026-05-01", "organic": 2705300, "paid": 18400 },
    { "date": "2026-06-01", "organic": 2788100, "paid": 17250 },
    { "date": "2026-07-01", "organic": 2841500, "paid": 16980 }
  ],
  "website_traffic_top_pages": [
    { "url": "https://www.make.com/en/templates", "traffic": 184300, "keywords": 6120 },
    { "url": "https://www.make.com/en/integrations", "traffic": 121750, "keywords": 4488 }
  ],
  "website_traffic_top_countries": [
    { "country": "us", "traffic": 812400, "share": 0.286 },
    { "country": "gb", "traffic": 214900, "share": 0.076 }
  ],
  "website_traffic_top_keywords": [
    { "keyword": "make automation", "position": 1, "volume": 74000, "traffic": 31200 },
    { "keyword": "integromat", "position": 1, "volume": 60500, "traffic": 24800 }
  ]
}
```

A `keyword_difficulty` record from the same run:

```json
{
  "data_captured_at": "2026-07-25T09:14:11.902847+00:00",
  "type": "keyword_difficulty",
  "keyword": "workflow automation",
  "country": "us",
  "keyword_difficulty": 68,
  "keyword_difficulty_source": "serp",
  "keyword_difficulty_shortage": null,
  "keyword_difficulty_results": [
    {
      "content_type": "organic",
      "data": { "url": "https://zapier.com/workflow-automation", "title": "Workflow Automation", "domainRating": 92 },
      "pos": 1,
      "pos_with_metrics": 1
    }
  ]
}
```

### How does it work?

Ahrefs' free tools are backed by a `/v4` JSON API, each call gated by a Cloudflare Turnstile challenge. Ahrefs Scraper solves that challenge with your CapSolver or 2Captcha key, calls the tool endpoint, and shapes the response into a flat record.

Requests start on a direct connection. If Ahrefs blocks one, the Actor escalates to an Apify datacenter proxy and then to residential, retrying up to three times on the residential rung, and logs which tier it ended on.

The responses arrive in Ahrefs' internal OCaml/Melange encoding, which is decoded recursively — options unwrapped, results unwrapped, enums flattened, signed URL wrappers reduced to a plain `url`. Because the Actor reads that structured API rather than scraping rendered HTML, an Ahrefs front-end redesign does not affect it, and your field names stay put. The AhrefsTop report is the one exception: it is a server-rendered page, parsed from HTML, and needs no captcha at all.

Only publicly visible free-tool data is collected. No Ahrefs account, cookie or session is used.

### Integrations

Ahrefs Scraper is an Apify Actor, so it works with anything that can call the Apify API or consume a dataset.

#### Calling Ahrefs Scraper from Python

```python
from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")

run = client.actor("<YOUR_USERNAME>/ahrefs-scraper").call(run_input={
    "urls": ["make.com", "zapier.com"],
    "keyword": "workflow automation",
    "country": "us",
    "include_web_authority": True,
    "include_traffic": True,
    "include_keywords_difficulty": True,
    "captchaApiKey": "<YOUR_CAPSOLVER_KEY>",
})

for rec in client.dataset(run["defaultDatasetId"]).iterate_items():
    if rec["type"] == "traffic" and not rec.get("website_authority_error"):
        wa = rec["website_authority"]
        print(rec["domain"], "DR", wa["domainRating"], "refdomains", wa["refdomains"])
```

Works in Go, Ruby, Node.js, cURL — any language that can make an HTTP request. Branch on `type` to route each report to the right handler.

#### No-code tools (n8n, Make, Zapier)

In n8n, use the Apify node — or an HTTP Request node pointed at the Apify run endpoint with your token — and pass the same JSON input shown above; a Switch node on `type` then splits the reports into separate branches. In Make, the Apify module supports run-and-wait, so a monthly authority sweep can feed a Google Sheets or Airtable step directly. Both platforms handle scheduling, so a recurring competitor audit needs no code.

### Is it legal to scrape Ahrefs data?

Scraping publicly accessible tool output is broadly treated as permissible where no authentication is bypassed, and Ahrefs Scraper reads only Ahrefs' free public tools — no account, no cookie, no paid session.

The output is aggregate SEO metrics about websites: Domain Rating, traffic estimates, backlink counts, keyword volumes, ranking positions. These are business and technical records, not personal data, so GDPR and equivalent personal-data regimes do not attach to this output the way they would to a profile scraper.

What does apply is contractual and database-related: Ahrefs' terms of service, and database or unfair-competition rules covering systematic reuse of a compiled dataset. Solving a captcha to reach the tools is a factor worth raising with counsel, since it is a deliberate access control rather than an incidental one.

Consult legal counsel before reselling extracted Ahrefs metrics as a standalone product or republishing them as a competing dataset.

### ❓ Frequently asked questions

#### What Ahrefs fields does Ahrefs Scraper return?

The five most used are `website_authority.domainRating`, `website_authority.refdomains`, `website_traffic.trafficMonthlyAvg`, `keyword_difficulty` and `serp_results`. Every row also carries `data_captured_at` and `type`. See the data fields table above for all of them.

#### Does Ahrefs Scraper require an Ahrefs account or subscription?

No. It reads Ahrefs' free public tools, which need no account, no login and no paid plan. It does require a CapSolver or 2Captcha key, because Ahrefs puts a Cloudflare Turnstile challenge in front of those tools — that is a third-party service you sign up for separately.

#### Why does my run fail immediately with a captcha error?

Because no solver key was found. The Actor checks `captchaApiKey` and then the `AHREFS_CAPTCHA_KEY` environment variable; if both are empty it logs an error naming both options and exits before pushing anything. Add a CapSolver or 2Captcha key and re-run. The AhrefsTop trending-websites report is the only one that works without a key.

#### How many reports can I get in one run?

Nine types are available, and domain-based reports repeat per target. With `urls` holding ten domains and authority, traffic, backlinks and broken links all enabled, one run produces forty rows. Keyword-based reports run once per run, so add one row each for `keywords`, `keyword_difficulty`, `serp` and `ai_visibility`.

#### What happens if a report returns nothing?

The record is still written, with Ahrefs' own error message attached — `error` on most types, or `website_authority_error` / `website_traffic_error` on the split `traffic` record — and the run continues. A failed report is visible in your dataset rather than being an absence you have to spot. Filter clean records with `rec.get("error") is None`.

#### Can I analyse multiple domains at once?

Yes. `urls` is a list and overrides the single `url` field when set. Every domain-based report runs once per entry. To pair one keyword against many domains for share-of-voice, enable `include_keywords_ranking` — it runs per URL and returns `top_position` for each.

#### Does Ahrefs Scraper work with Claude, ChatGPT and other AI agent tools?

Yes. It is callable as a standard HTTP endpoint through the Apify API, so LangChain, CrewAI, n8n or a hand-written tool definition can invoke it and receive typed JSON with no parsing step.

#### How does Ahrefs Scraper compare to other Ahrefs scrapers?

Checked on the Apify Store on 25 July 2026: `radeance/ahrefs-scraper` is the most used, with an extensive README documenting a very similar report set — website authority, traffic, AI visibility — and the same `data_captured_at` / `type` / `domain` / `mode` record shape. `pro100chok/ahrefs-seo-tools` documents a comparable spread with per-tool output examples. `scrap3r/ahrefs-domain-overview-checker` is narrower, covering the domain-overview DR/UR check only. All of these read the same free Ahrefs tools and therefore face the same Turnstile gate. What this Actor documents explicitly is the captcha prerequisite, the direct-first proxy ladder, per-report error capture on the record itself, and the AhrefsTop trending-websites report as a no-captcha extra.

#### Does Ahrefs Scraper return data in a format LLMs can use directly?

Yes. Every record is typed, normalized JSON with consistent field names across runs, decoded out of Ahrefs' internal encoding. No HTML parsing, no selectors. Pass a row directly into an LLM, index it into a vector store, or hand it to an agent tool.

#### What happens when Ahrefs changes its layout or anti-bot system?

The scraper is maintained, and because eight of the nine reports read Ahrefs' `/v4` JSON API rather than rendered markup, a front-end redesign generally does not affect them at all. The AhrefsTop report is HTML-parsed and is the one exposed to layout change. Your field names and types stay the same on your end.

#### Can I use Ahrefs Scraper without managing proxies?

Yes. The proxy ladder is automatic — direct, then Apify datacenter, then Apify residential with up to three retries — and you never create a proxy account or rotate an IP. Setting a country in `proxyConfiguration` localises the residential rung. The captcha solver is the one external account you do have to set up yourself.

#### Which Ahrefs fields work best for AI training data and RAG indexing?

For RAG indexing: `website_traffic_top_keywords`, `website_traffic_top_pages` and `serp_results` carry the most descriptive text per record and chunk cleanly by domain. For training data: `website_authority.domainRating`, `website_authority.refdomains`, `website_traffic.trafficMonthlyAvg` and `keyword_difficulty` are the most structurally consistent fields across records. All values are returned as typed primitives, arrays or plain nested objects, so no normalization pass is needed before indexing.

### 🔗 Related scrapers

| Scraper Name | What it extracts |
| ----- | ----- |
| MOZ Scraper | Domain and page authority metrics from a second SEO data source |
| Quora Search Scraper | Question-intent results for keyword and content research |
| Youtube Search Scraper | Video search results for a keyword |
| Google Play Store Reviews Scraper | App review text, ratings and versions |
| Twitter Trends Scraper | Trending topics for timely content planning |

### 💬 Your feedback

Found a bug, or need an Ahrefs free tool that is not yet covered? Open an issue on the Actor's Issues tab. Reports that include the exact input JSON and the target domain or keyword are the fastest to reproduce and fix.

# Actor input Schema

## `keyword` (type: `string`):

The keyword (or brand, for AI Visibility) to research on Ahrefs.

## `url` (type: `string`):

The single URL or domain to analyse on Ahrefs (e.g. make.com). For many targets at once, use the bulk field below.

## `country` (type: `string`):

Country to localise keyword, SERP and traffic results.

## `mode` (type: `string`):

Scope of the domain analysis: 'subdomains' (whole site incl. subdomains) or 'exact' (just the exact URL/host).

## `urls` (type: `array`):

A list of URLs or domains to analyse on Ahrefs. Add them one by one or paste in bulk. When set, this overrides the single URL field above.

## `include_web_authority` (type: `boolean`):

Needs a Domain/URL. If checked, includes Domain Rating, backlinks & referring-domain counts.

## `include_traffic` (type: `boolean`):

Needs a Domain/URL. If checked, includes estimated monthly visits, traffic value, history, top pages/countries/keywords.

## `include_ai_visibility` (type: `boolean`):

Needs a Brand/Keyword. If checked, includes AI citations across LLMs, top cited domains/pages & topics.

## `include_keywords` (type: `boolean`):

Needs a Keyword and Country. If checked, includes keyword ideas + question ideas with volumes.

## `include_keywords_difficulty` (type: `boolean`):

Needs a Keyword and Country. If checked, includes the Keyword Difficulty score + ranking SERP.

## `include_keywords_ranking` (type: `boolean`):

Needs a Keyword, Domain and Country. If checked, includes where the domain ranks for the keyword.

## `include_serp` (type: `boolean`):

Needs a Keyword and Country. If checked, includes the full SERP overview for the keyword.

## `include_backlinks` (type: `boolean`):

Needs a Domain/URL. If checked, includes the backlink & referring-domain overview.

## `include_broken_links` (type: `boolean`):

Needs a Domain/URL. If checked, includes broken inbound & outbound links.

## `include_top_websites` (type: `boolean`):

If checked, includes the top trending websites from AhrefsTop.com — the Top 1000 (worldwide + all categories) or Top 100 when filtered by category/country. Does not need a Keyword or Domain.

## `category_top_websites` (type: `string`):

Category to filter the AhrefsTop trending websites ranking.

## `country_top_websites` (type: `string`):

Country to filter the AhrefsTop trending websites ranking.

## `proxyConfiguration` (type: `object`):

The scraper starts on a DIRECT connection and only escalates to a datacenter, then residential proxy if Ahrefs blocks it. Pick a country here to localise residential fallback.

## `captchaApiKey` (type: `string`):

Required. Ahrefs protects every free tool with a Cloudflare Turnstile challenge, so this actor needs a captcha-solver key to fetch data. Paste your CapSolver or 2Captcha API key here (get one at capsolver.com or 2captcha.com). Stored encrypted. You can also set it via the AHREFS\_CAPTCHA\_KEY environment variable.

## Actor input object example

```json
{
  "keyword": "ai automation",
  "url": "make.com",
  "country": "us",
  "mode": "subdomains",
  "urls": [
    "make.com"
  ],
  "include_web_authority": true,
  "include_traffic": true,
  "include_ai_visibility": false,
  "include_keywords": false,
  "include_keywords_difficulty": false,
  "include_keywords_ranking": false,
  "include_serp": false,
  "include_backlinks": true,
  "include_broken_links": false,
  "include_top_websites": false,
  "category_top_websites": "all",
  "country_top_websites": "worldwide",
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "keyword": "ai automation",
    "url": "make.com",
    "urls": [
        "make.com"
    ],
    "proxyConfiguration": {
        "useApifyProxy": true
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("scraper-engine/ahrefs-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "keyword": "ai automation",
    "url": "make.com",
    "urls": ["make.com"],
    "proxyConfiguration": { "useApifyProxy": True },
}

# Run the Actor and wait for it to finish
run = client.actor("scraper-engine/ahrefs-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "keyword": "ai automation",
  "url": "make.com",
  "urls": [
    "make.com"
  ],
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}' |
apify call scraper-engine/ahrefs-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=scraper-engine/ahrefs-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/S4aPmOlnEx9sa69Ys/builds/N1LN1lbETyvwwAPZV/openapi.json