Ahrefs Scraper & Competitor Comparison

Ahrefs Competitor Comparison Scraper — DR, Traffic, Keyword Gap

Ahrefs Scraper & Competitor Comparison pulls the same metrics Ahrefs' free tools expose — Domain Rating, backlink and referring-domain counts, monthly traffic and traffic value, keyword ideas, keyword difficulty, SERP overviews, broken links, AI-citation visibility, and the AhrefsTop trending-websites ranking — for one domain, one keyword, or a bulk list of domains. When you give it two or more domains it adds a ranked competitor scoreboard and an approximate keyword-gap diff, computed from data it already collected — no extra Ahrefs calls. Built for SEO consultants, agencies benchmarking client sites against competitors, and analysts comparing a market by domain.

What is Ahrefs Scraper & Competitor Comparison?

Ahrefs Scraper & Competitor Comparison is an Apify Actor that queries Ahrefs' free public SEO tools and shapes the responses into typed dataset records. Each data module you enable produces its own record, tagged with a type field, so one run can cover authority, traffic, backlinks and keyword research for the same target — and, when you compare two or more domains, a comparison record ranks them and diffs their keywords.

No Ahrefs account, login or subscription is required — these are the free tools any visitor can use. A captcha-solver API key is required, because Ahrefs gates every free tool behind Cloudflare Turnstile.

• Targets are discovered by keyword/brand, by a single URL or domain, or by a bulk list of URLs/domains (urls) that overrides the single field
• Identity fields on every record: domain or keyword/brand, country, and (on domain-scope records) mode
• Ten selectable data modules: web authority, traffic, AI visibility, keyword ideas, keyword difficulty, keyword ranking, SERP overview, backlinks, broken links, and AhrefsTop trending websites
• Country localisation across 188 ISO country codes for keyword, SERP and traffic results
• Real filters: mode (subdomains vs. exact host), category_top_websites / country_top_websites for the trending-websites ranking, and rankBy for the competitor scoreboard
• Export as JSON, CSV or Excel from the Apify dataset, or read it through the Apify API

⚠️ You must supply a captcha-solver key

Ahrefs protects every /v4 free-tool endpoint with a Cloudflare Turnstile challenge. The Actor cannot fetch web authority, traffic, backlinks, broken links, keyword, SERP or AI-visibility data without solving it, so it needs a solver key from a third-party captcha-solving service — CapSolver or 2Captcha — that you create and fund yourself.

Supply it either way:

• Paste it into the 🔐 Captcha Solver API Key (captchaApiKey) input — marked isSecret, stored encrypted, never logged
• Or set the AHREFS_CAPTCHA_KEY environment variable on the Actor

If neither is present, the run logs an error naming both options and exits before pushing any rows — no charge, because nothing is pushed. One exception: AhrefsTop trending websites (include_top_websites) reads a plain server-rendered page with no captcha and no /v4 call, so it works without a solver key even if nothing else does.

What data can I extract with Ahrefs Scraper & Competitor Comparison?

Every record carries data_captured_at and a type field identifying which of the eleven record shapes it is, plus the identity fields for that type. The table below lists every field the Actor's row-building code (src/parser.py) actually writes, grouped by record type — not just the seven columns shown in the default dataset view.

Field	Example value	Use case
data_captured_at	"2026-07-25T09:14:02.481230+00:00"	On every record — when the row was collected
type	"traffic"	On every record — one of traffic, backlinks, broken_links, keyword_ranking, keyword_difficulty, keywords, serp, ai_visibility, top_websites, comparison
domain	"make.com"	Bare host, scheme and www. stripped — on traffic, backlinks, broken_links, keyword_ranking records
mode	"subdomains"	Scope of a domain report — on traffic, backlinks, broken_links records only (keyword_ranking carries domain but not mode)
keyword	"workflow automation"	On keyword_ranking, keyword_difficulty, keywords, serp records
country	"us"	On keyword_ranking, keyword_difficulty, keywords, serp records
brand	"make.com"	On ai_visibility records — the researched brand or keyword
website_authority	{"domainRating": 91, "backlinks": 4182337, "refdomains": 38412, "dofollowBacklinks": 3120884, "dofollowRefdomains": 29760}	On traffic records, present only if include_web_authority is on and the call succeeds
website_authority_error	"CaptchaRequired"	On traffic records, in place of website_authority if that half failed
website_traffic	{"trafficMonthlyAvg": 2841500, "costMontlyAvg": 1934200}	On traffic records, present only if include_traffic is on and the call succeeds — note costMontlyAvg, not a typo of ours
website_traffic_error	"CaptchaRequired"	On traffic records, in place of the four website_traffic* fields if that half failed
website_traffic_history	[{"date": "2026-06-01", "organic": 2788100}]	Monthly organic traffic series
website_traffic_top_pages	[{"url": "https://www.make.com/en/templates", "traffic": 184300}]	Highest-traffic pages on the domain
website_traffic_top_countries	[{"country": "us", "share": 0.286}]	Traffic split by country
website_traffic_top_keywords	[{"keyword": "make automation", "position": 1, "traffic": 31200}]	Keywords driving the most traffic — also the source data for the keyword-gap diff
data	(nested object)	Full decoded payload on backlinks, broken_links, keywords and ai_visibility records — see module notes below
error	"CaptchaRequired"	Ahrefs' own error/reason string when a module call failed, on any single-payload record type
top_position	{"content_type": "Organic", "data": {...}, "pos": 3, "pos_with_metrics": null}	On keyword_ranking records — best SERP slot the domain holds for the keyword, or null if it doesn't rank in the captured SERP
serp_results	[{"content_type": "Organic", "data": {...}, "pos": 1, "pos_with_metrics": null}]	On keyword_ranking and serp records — the decoded SERP, each slot carrying Ahrefs' own content_type label (organic result, AI overview, paid, questions box, etc.)
serp_source	"live"	Source label Ahrefs reported for that SERP snapshot
last_update	"2026-07-20T00:00:00Z"	Ahrefs' own last-update stamp for the SERP/ranking data
keyword_difficulty	68	On keyword_difficulty records — the 0–100 difficulty score
keyword_difficulty_results	(array of decoded SERP items)	The SERP the difficulty score was calculated from
keyword_difficulty_source	"serp"	Source label for the difficulty SERP
keyword_difficulty_shortage	null	Ahrefs' own data-shortage indicator for the keyword
top_websites_category	"finance"	On top_websites records — the category filter applied
top_websites_country	"worldwide"	On top_websites records — the country filter applied
websites_rankings	[{"rank": 1, "domain": "wikipedia.org", "search_traffic": "4.6B", "change": "+360.7M", "category": "Reference"}]	Main AhrefsTop ranking rows
website_rankings_trending_up	[{"rank": 4, "domain": "example.com", "search_traffic": "12.1M", "change": "+2.3M", "position_change": "↑12"}]	Trending-up rows — same shape as the main table with position_change instead of category
website_rankings_trending_down	(same shape as trending-up)	Trending-down rows

Competitor scoreboard and keyword gap fields

These only appear on the one extra comparison record, and only when compareDomains produces one (2+ domains with usable website_authority/website_traffic data). rankedBy names the metric used; domainsCompared is the count ranked; comparison is the scoreboard array, each row carrying domain, domainRating, trafficMonthlyAvg, trafficValue (mapped from the website_traffic.costMontlyAvg field above), refdomains and a computed rank. If keywordOverlap is on, sharedKeywords lists {keyword, rankedBy: [domains]} for keywords two or more compared domains rank for, keywordGap lists {keyword, rankedByCompetitors: [domains], position} for keywords competitors rank for that gapTargetDomain does not (within the sampled set), and comparisonNote is a plain-text string that reports how many of the compared domains actually returned a keyword sample — this is the field to read before treating an empty gap as "nothing to find," since a keyword missing from keywordGap only means it wasn't in the free sample, not that it doesn't exist.

Targeting and localisation fields

mode (subdomains or exact) sets whether a domain report covers the whole site or just the exact host you gave it. country localises keyword, SERP and traffic results across 188 country codes. category_top_websites (26 values) and country_top_websites (189 values including worldwide) narrow the AhrefsTop ranking — these two inputs are read only when include_top_websites is true; with it off, they're accepted by the schema but never touched by the code. rankBy picks which of the four scoreboard metrics orders the comparison array.

Why not build this yourself?

Ahrefs does publish an official API, but it's a separate paid subscription on top of an Ahrefs account — its own pricing tiers and quotas, which this README doesn't reproduce because they change and aren't this Actor's to quote. What this Actor covers is Ahrefs' free tools, and reaching those yourself means solving a fresh Cloudflare Turnstile challenge on every request, discovering and replaying a rotating X-Client-Version header, decoding Ahrefs' internal OCaml/Melange response envelope — options as ["Some", x], results as ["Ok", x]/["Error", e], enums as single-element arrays — and running a proxy-escalation ladder for when Ahrefs blocks the request outright. That's infrastructure to build and maintain before you've parsed a single field, and it breaks again the next time Ahrefs rotates its client version or its Turnstile sitekey.

Use the official Ahrefs API if you already pay for the subscription and need its full index. Use this Actor for ad-hoc research, prospect and competitor audits, and pulling free-tool metrics for many domains without that subscription — you supply only a captcha-solver key.

How to use data extracted from Ahrefs Scraper & Competitor Comparison?

🔍 SEO consultants and in-house teams

Paste a prospect or client domain into url, leave include_web_authority, include_traffic and include_backlinks on their defaults, and sort the resulting traffic record by website_authority.domainRating and website_traffic.trafficMonthlyAvg to decide where a site stands before pitching or planning work. Add include_keywords_difficulty with a seed keyword to check whether a target term is actually winnable before committing content budget to it.

🏢 Agencies benchmarking clients against competitors

Put the client domain and its competitors into urls, leave compareDomains and keywordOverlap on, set gapTargetDomain to the client's domain, and read the single comparison record back: a ranked scoreboard by rankBy plus a keywordGap list naming exactly which keywords the competitors rank for that the client doesn't. That's a monthly client report built from one run instead of manually cross-referencing several domain lookups.

📊 Market research and intelligence

Enable include_top_websites with category_top_websites set to your vertical and no keyword or domain at all to pull the AhrefsTop ranking for that category, then diff website_rankings_trending_up and website_rankings_trending_down run over run to see which sites are gaining or losing traffic share in the space.

🤖 AI agents and automated pipelines

Every module is a standard HTTP-triggered Apify run, so an agent framework can call it, branch on the type field, and route website_traffic_top_keywords or serp_results into a vector store to answer "which pages drive this competitor's traffic" against live data instead of a stale export.

⬇️ Input

No input is marked required by the schema — a run with none of url, urls, keyword or include_top_websites=true logs "nothing to do" and exits.

Targets and scope

Parameter	Required	Type	Description	Example value
keyword	No	string	The keyword (or brand, for AI Visibility) to research on Ahrefs.	"ai automation"
url	No	string	The single URL or domain to analyse on Ahrefs (e.g. make.com). For many targets at once, use the bulk field below.	"make.com"
country	No	string	Country to localise keyword, SERP and traffic results. Default "us". Enum of 188 ISO alpha-2 codes, e.g. "af", "al", ... "us", ... "zw".	"us"
mode	No	string	Scope of the domain analysis: 'subdomains' (whole site incl. subdomains) or 'exact' (just the exact URL/host). Default "subdomains". Enum: "subdomains", "exact".	"subdomains"
urls	No	array	The domains/URLs to scrape and rank side by side. Add 2 or more to build the competitor scoreboard and keyword gap. When set, this overrides the single URL field above.	["make.com", "zapier.com", "n8n.io"]

Data modules (include_* toggles)

All ten are independent booleans — none are mutually exclusive, all are combinable, and every one you enable adds its own API call per applicable target, so more modules on means more requests and more pushed (charged) records.

Parameter	Required	Type	Description	Example value
include_web_authority	No	boolean	Needs a Domain/URL. If checked, includes Domain Rating, backlinks & referring-domain counts. Default true.	true
include_traffic	No	boolean	Needs a Domain/URL. If checked, includes estimated monthly visits, traffic value, history, top pages/countries/keywords. Default true.	true
include_ai_visibility	No	boolean	Needs a Brand/Keyword. If checked, includes AI citations across LLMs, top cited domains/pages & topics. Default false.	false
include_keywords	No	boolean	Needs a Keyword and Country. If checked, includes keyword ideas + question ideas with volumes. Default false.	false
include_keywords_difficulty	No	boolean	Needs a Keyword and Country. If checked, includes the Keyword Difficulty score + ranking SERP. Default false.	false
include_keywords_ranking	No	boolean	Needs a Keyword, Domain and Country. If checked, includes where the domain ranks for the keyword. Default false.	false
include_serp	No	boolean	Needs a Keyword and Country. If checked, includes the full SERP overview for the keyword. Default false.	false
include_backlinks	No	boolean	Needs a Domain/URL. If checked, includes the backlink & referring-domain overview. Default true.	true
include_broken_links	No	boolean	Needs a Domain/URL. If checked, includes broken inbound & outbound links. Default false.	false
include_top_websites	No	boolean	If checked, includes the top trending websites from AhrefsTop.com — the Top 1000 (worldwide + all categories) or Top 100 when filtered by category/country. Does not need a Keyword or Domain. Default false.	false

Top Websites filters (only read when include_top_websites is true)

Parameter	Required	Type	Description	Example value
category_top_websites	No	string	Category to filter the AhrefsTop trending websites ranking. Default "all". Enum of 26 values: "all", "beauty-and-fitness", "books-and-literature", "business", "computers-and-electronics", "arts-and-entertainment", "finance", "food-and-drink", "games", "health", "hobbies-and-leisure", "home-and-garden", "internet-and-telecom", "jobs-and-education", "law-and-government", "news", "people-and-society", "pets-and-animals", "real-estate", "reference", "science", "shopping", "social-networks", "sports", "travel-and-transportation", "autos-and-vehicles".	"finance"
country_top_websites	No	string	Country to filter the AhrefsTop trending websites ranking. Default "worldwide". Enum of 189 values: "worldwide" plus 188 ISO alpha-2 codes.	"worldwide"

Competitor Comparison (post-processing — no extra Ahrefs calls)

Parameter	Required	Type	Description	Example value
compareDomains	No	boolean	Rank the bulk domains side by side (Domain Rating, monthly organic traffic, traffic value, referring domains). Needs 2+ domains in the bulk field and the Domain Overview / Web-Traffic datasets enabled. Default true.	true
rankBy	No	string	Metric used to order the competitor scoreboard (highest first). Default "trafficMonthlyAvg". Enum: "domainRating", "trafficMonthlyAvg", "trafficValue", "refdomains".	"trafficMonthlyAvg"
keywordOverlap	No	boolean	Diff each domain's free top-keyword sample into shared keywords and an approximate keyword gap for the target domain below. Approximation over Ahrefs' free top-keyword sample — not the paid Content Gap. Default true.	true
gapTargetDomain	No	string	The domain whose keyword gap you want — keywords its competitors rank for but it does not, within the sampled set. Leave empty to use the first bulk domain.	"make.com"

Captcha and proxy

Parameter	Required	Type	Description	Example value
captchaApiKey	No	string	Your CapSolver (or 2Captcha) API key, used to solve the Cloudflare Turnstile that gates every Ahrefs free tool. Required to collect data. Stored securely and never logged. Can also be supplied via the AHREFS_CAPTCHA_KEY env var.	"CAP-XXXXXXXXXXXX"
proxyConfiguration	No	object	The scraper starts on a DIRECT connection and only escalates to a datacenter, then residential proxy if Ahrefs blocks it. Pick a country here to localise residential fallback. Default {"useApifyProxy": true}.	{"useApifyProxy": true}

Common pitfall: captchaApiKey isn't flagged required by the schema, but it's required in practice — a run with no key and no AHREFS_CAPTCHA_KEY env var logs an error and exits before pushing anything, so the run doesn't fail loudly in the Console the way a schema-required field would. A second pitfall: compareDomains silently produces nothing (not an error, just a log line and a lower record count) unless urls has 2+ domains and at least one of include_traffic / include_web_authority is on — the scoreboard reads only the website_authority/website_traffic data those two modules produce.

Example input

{
  "urls": ["make.com", "zapier.com", "n8n.io"],
  "mode": "subdomains",
  "country": "us",
  "include_web_authority": true,
  "include_traffic": true,
  "include_backlinks": true,
  "compareDomains": true,
  "rankBy": "trafficMonthlyAvg",
  "keywordOverlap": true,
  "gapTargetDomain": "make.com",
  "captchaApiKey": "CAP-XXXXXXXXXXXX",
  "proxyConfiguration": { "useApifyProxy": true }
}

⬆️ Output

Typed, normalized JSON with a consistent shape per type, written live as each module's result lands — the dataset fills while the run is still going. Export as JSON, CSV or Excel, or read it through the Apify API. The Apify Console dataset view offers a separate tab per record type (Overview, Traffic & Authority, Backlinks, Broken Links, Keyword Ranking, Keyword Difficulty, Keyword Ideas, SERP Overview, AI Visibility, Top Trending Websites, Competitor Comparison) — the seven-column "Overview" tab is a subset for browsing, not the full row.

Every pushed record — including one that carries an error field because a module call failed — charges one row_result pay-per-event unit. The only free case is a run that exits before anything is pushed: no captcha key at all, or none of url / urls / keyword / include_top_websites set. To exclude failed modules from downstream analysis without touching billing, filter on: item.get("error") is None and not item.get("website_authority_error") and not item.get("website_traffic_error").

Example output — traffic record (on by default)

{
  "data_captured_at": "2026-07-25T09:14:02.481230+00:00",
  "type": "traffic",
  "domain": "make.com",
  "mode": "subdomains",
  "website_authority": {
    "domainRating": 89.0,
    "backlinks": 2569132,
    "refdomains": 35171,
    "dofollowBacklinks": 96,
    "dofollowRefdomains": 85
  },
  "website_traffic": {
    "trafficMonthlyAvg": 360938,
    "costMontlyAvg": 17315114
  },
  "website_traffic_history": [
    { "date": "2026-05-01", "organic": 338830 },
    { "date": "2026-06-01", "organic": 345835 }
  ],
  "website_traffic_top_pages": [
    { "url": "https://www.make.com/en/integrations/perplexity-ai", "traffic": 8774, "share": 10.65 }
  ],
  "website_traffic_top_countries": [
    { "country": "in", "share": 23.79 },
    { "country": "us", "share": 23.76 }
  ],
  "website_traffic_top_keywords": [
    { "keyword": "make.com", "position": 1, "traffic": 4500 },
    { "keyword": "instagram videos download", "position": 12, "traffic": 260000 }
  ]
}

Example output — comparison record (the differentiator, produced when compareDomains finds 2+ usable domains)

{
  "data_captured_at": "2026-07-25T09:14:19.002104+00:00",
  "type": "comparison",
  "rankedBy": "trafficMonthlyAvg",
  "domainsCompared": 3,
  "gapTargetDomain": "make.com",
  "comparison": [
    { "domain": "zapier.com", "domainRating": 92, "trafficMonthlyAvg": 4200000, "trafficValue": 50000000, "refdomains": 120000, "rank": 1 },
    { "domain": "n8n.io", "domainRating": 80, "trafficMonthlyAvg": 500000, "trafficValue": 1000000, "refdomains": 9000, "rank": 2 },
    { "domain": "make.com", "domainRating": 89, "trafficMonthlyAvg": 360938, "trafficValue": 17315114, "refdomains": 35171, "rank": 3 }
  ],
  "sharedKeywords": [
    { "keyword": "automation", "rankedBy": ["make.com", "n8n.io", "zapier.com"] }
  ],
  "keywordGap": [
    { "keyword": "zapier alternative", "rankedByCompetitors": ["zapier.com"], "position": 1 }
  ],
  "comparisonNote": "*Approx. Keyword overlap/gap is diffed from each domain's FREE Ahrefs top-keyword SAMPLE (a small subset of ranked keywords), not Ahrefs' paid Content Gap. A keyword missing from the gap only means it was not in the sampled set. Keyword samples available for 3/3 domains."
}

How do you filter and target specific domain and keyword reports?

Targeting here happens in two layers: which target you give the run, and which modules you turn on for it.

Single domain vs. bulk list. url analyses one target; urls is a list that, when set, overrides url entirely — put 2 or more domains there to unlock compareDomains and keywordOverlap. Keyword-based modules (include_keywords, include_keywords_difficulty, include_serp, include_ai_visibility) key off the single keyword/brand field and run once per run regardless of how many URLs are in urls; include_keywords_ranking is the one hybrid — it pairs your one keyword against each domain in urls, which is what a share-of-voice comparison needs.

Scope precision. mode decides whether a domain report covers the whole site (subdomains) or just the exact host (exact). country localises keyword, SERP and traffic data across 188 codes. category_top_websites and country_top_websites narrow the trending-websites ranking, but only take effect when include_top_websites is on.

Scoreboard precision. rankBy picks which metric orders the comparison array; gapTargetDomain picks which domain the keyword gap is computed for — leave it empty and the Actor uses the first domain in urls.

Volume. There is no maxItems/limit parameter in this Actor's schema — record volume is the product of which include_* modules you enable and how many URLs are in urls. Enabling more modules or more URLs directly increases both the request count and the number of charged row_result events.

{ "urls": ["make.com", "zapier.com", "n8n.io"], "include_web_authority": true, "include_traffic": true, "compareDomains": true, "rankBy": "domainRating", "keywordOverlap": true, "gapTargetDomain": "make.com" }

{ "url": "make.com", "keyword": "workflow automation", "country": "gb", "include_keywords_ranking": true, "include_keywords_difficulty": true }

{ "include_top_websites": true, "category_top_websites": "internet-and-telecom", "country_top_websites": "us" }

▶️ Want to try other Ahrefs scrapers?

Scraper Name	What it extracts
Moz SEO Health Scorer	Domain and page authority metrics from a second SEO data source, useful as a cross-check against Domain Rating
Quora Search Scraper — Fresh Trending Question Monitor	Question-intent search results that keyword-volume tools tend to miss
YouTube Channel Keyword Research Scraper	Keyword and topic research surfaced from YouTube search rather than Google
Google Search Autocomplete API	Long-tail keyword suggestions straight from Google's own autocomplete

How to extract Ahrefs data programmatically

Ahrefs Scraper & Competitor Comparison runs as a standard Apify Actor — one authenticated call starts the run, and structured JSON comes back through the dataset.

Python example

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")

run = client.actor("<YOUR_USERNAME>/ahrefs-scraper-competitor-comparison").call(run_input={
    "urls": ["make.com", "zapier.com", "n8n.io"],
    "include_web_authority": True,
    "include_traffic": True,
    "compareDomains": True,
    "rankBy": "trafficMonthlyAvg",
    "captchaApiKey": "<YOUR_CAPSOLVER_KEY>",
})

for rec in client.dataset(run["defaultDatasetId"]).iterate_items():
    if rec["type"] == "comparison":
        for row in rec["comparison"]:
            print(row["domain"], "DR", row["domainRating"], "traffic", row["trafficMonthlyAvg"])

Export to spreadsheets or CRM

Export the dataset as CSV directly from the Console or via the API, then map columns on type — for a traffic row, website_authority.domainRating and website_traffic.trafficMonthlyAvg are the two fields most CRMs use as lead-scoring inputs; for a comparison row, comparison[].domain and comparison[].rank map cleanly to a spreadsheet leaderboard.

Is it legal to scrape Ahrefs data?

Scraping publicly accessible tool output is broadly treated as permissible where no authentication is bypassed, and this Actor reads only Ahrefs' free public tools — no account, no cookie, no paid session. The output is aggregate SEO metrics about websites — Domain Rating, traffic estimates, backlink counts, keyword volumes, ranking positions — which are business and technical records, not personal data, so GDPR and equivalent personal-data regimes do not attach to this output the way they would to a profile scraper.

What does apply is contractual and database-related: Ahrefs' terms of service, and database or unfair-competition rules covering systematic reuse of a compiled dataset. Solving a Turnstile challenge to reach the tools is a deliberate access control worth raising with counsel before commercial use, not an incidental one.

Consult legal counsel for commercial applications involving bulk storage or resale of this data.

❓ FAQ

What if a domain has no backlink or traffic data?

The record is still written, with Ahrefs' error attached — error on backlinks/broken_links records, or website_authority_error/website_traffic_error on the split traffic record — instead of the data field, and the run continues with the rest of the modules. A failed module is a visible row you can filter on, not a silent gap.

What if a keyword has zero search volume or no ranking SERP?

keyword_difficulty, serp and keywords records are written from whatever Ahrefs' free tool actually returns for that keyword; if the call fails outright the record carries error instead of the data fields. keyword_ranking's top_position comes back null when the domain doesn't appear anywhere in the captured SERP for that keyword.

Can I get the competitor scoreboard along with the base SEO records?

Yes — that's the point of this Actor over the base Ahrefs Scraper. Put 2+ domains in urls, leave include_web_authority and/or include_traffic on, and leave compareDomains on (its default). One extra comparison record is pushed alongside the per-domain records, at no extra Ahrefs API cost since it's pure post-processing over data already collected.

How accurate is the keyword-gap diff?

It's an approximation, and the comparisonNote field says so on every comparison record. The gap is computed from each domain's free Ahrefs top-keyword sample — a small subset of that domain's actual ranked keywords — not Ahrefs' paid Content Gap tool. A keyword missing from keywordGap means it wasn't in the sampled set, not that no gap exists.

How many records can I get per run?

There's no maxItems or row-cap parameter in the schema. Volume is the product of how many include_* modules you enable and how many URLs are in urls: domain-based modules (web authority/traffic, backlinks, broken links) run once per URL, keyword-based modules (keyword ideas, keyword difficulty, SERP, AI visibility) run once per run regardless of URL count, keyword_ranking runs once per URL, and top_websites and comparison each produce at most one record per run.

What's the actual difference between the base data and the comparison feature?

The nine base modules (web authority, traffic, AI visibility, keywords, keyword difficulty, keyword ranking, SERP, backlinks, broken links) plus AhrefsTop trending websites are unchanged Ahrefs free-tool lookups, one record per module per applicable target. compareDomains and keywordOverlap are the two inputs unique to this variant — they don't call Ahrefs again, they rank and diff the website_authority/website_traffic records the run already produced.

Does this work with Claude, ChatGPT, and AI agent frameworks?

Yes. It's callable as a standard HTTP endpoint through the Apify API, so LangChain, CrewAI, n8n, or a hand-written tool definition can invoke it and receive typed JSON, branching on the type field to route each record.

How does this compare to other Ahrefs scrapers?

Checked on the Apify Store on 2026-07-25: radeance/ahrefs-scraper is the most used listing, documenting a similar module set (website authority, traffic, AI visibility) plus a separate "Competitors" search type that returns each competitor's traffic and page count, but not a user-ranked scoreboard with a selectable rankBy metric or a computed keyword-gap diff. pro100chok/ahrefs-seo-tools documents 13 search types including a "Website Details" competitors/similarly-ranked view, again without a scoreboard the caller ranks or a shared/gap keyword split. This Actor's difference is that compareDomains and keywordOverlap are computed automatically from the bulk domains you already scraped, at no extra Ahrefs API cost, rather than requiring a separate lookup per competitor.

Can I use this without an Ahrefs API key or developer account?

Yes — no Ahrefs account, login or API key is needed, since the Actor reads Ahrefs' free public tools. What you do need is a captcha-solver API key from a third-party service (CapSolver or 2Captcha), because those free tools sit behind a Cloudflare Turnstile challenge. That key is the one credential this Actor requires beyond your Apify token.

Conclusion

Ahrefs Scraper & Competitor Comparison turns Ahrefs' free-tool metrics — authority, traffic, keywords, backlinks, AI visibility, trending websites — into typed JSON for one domain, one keyword, or a bulk competitive set, then ranks and keyword-diffs that set for you without a paid Ahrefs subscription or a second API call. It's built for consultants and agencies who need a repeatable competitor snapshot, not a one-off manual lookup. Start it from the Apify Console with a captcha-solver key and a list of domains, or call it through the Apify API from your own pipeline.