Ahrefs Scraper
Pricing
from $4.99 / 1,000 results
Ahrefs Scraper
π Ahrefs Scraper extracts SEO data from Ahrefs to power competitive research, backlink analysis, and content strategy. π Automate insights, save time, and scale campaigns with cleaner datasets. π
Pricing
from $4.99 / 1,000 results
Rating
0.0
(0)
Developer
Scraper Engine
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
6 days ago
Last modified
Categories
Share
Ahrefs Scraper β Domain Rating, Traffic, Backlinks and Keywords
Ahrefs Scraper pulls SEO data from Ahrefs' public free tools and returns it as structured JSON: Domain Rating, backlink and referring-domain counts, estimated monthly traffic and traffic value, keyword ideas, keyword difficulty, full SERP overviews, broken links, AI-citation visibility, and the AhrefsTop trending-websites ranking. Nine report types, one dataset row each, written live as every result lands β no HTML parsing, no selectors. Pick the reports you want with checkboxes and press Start.
β οΈ Read the captcha requirement below before your first run. Ahrefs protects every free tool with a Cloudflare Turnstile challenge, so this Actor needs your own CapSolver or 2Captcha key. Without one it stops immediately with an error rather than returning empty rows.
What is Ahrefs Scraper?
Ahrefs Scraper is an Apify Actor that queries Ahrefs' free public SEO tools and shapes the responses into clean, typed records. Each report type you enable produces its own dataset row, tagged with a type field, so one run can cover authority, traffic, backlinks and keyword research for the same target.
No Ahrefs account, login or subscription is required β these are the free tools any visitor can use. What you do need is a captcha-solver API key, because Ahrefs gates those tools behind Cloudflare Turnstile.
It is built for SEO consultants auditing sites at scale, agencies running competitor research across a client list, and developers piping authority and keyword metrics into their own dashboards.
What Ahrefs data is publicly available to scrape?
Ahrefs publishes a set of free tools that return real metrics without an account β but each is a capped preview of what the paid product shows, and each sits behind a Turnstile challenge.
| Data Category | Free tools (what this Actor reads) | Paid Ahrefs subscription |
|---|---|---|
| Domain Rating, backlink and referring-domain counts | β Public | Full historical series |
| Estimated monthly traffic, traffic value, top pages/countries/keywords | β Public | Full breakdown and exports |
| Keyword ideas and question ideas with volumes | β Public | Complete keyword database |
| Keyword Difficulty score and its SERP | β Public | Full metrics per result |
| SERP overview for a keyword | β Public | Historical SERP tracking |
| Where a domain ranks for a keyword | β Public | Rank tracking over time |
| Broken inbound and outbound links | β Public | Full Site Audit crawl |
| AI citation visibility across LLMs | β Public | Brand Radar |
| AhrefsTop trending websites ranking | β Public β no captcha needed | β |
| Full backlink lists, anchor text, link history | β | Subscription only |
| Site Audit crawls, Rank Tracker projects | β | Subscription only |
Ahrefs Scraper only returns publicly visible data β what any visitor sees on Ahrefs' free tools. Nothing behind a login wall or a paid plan.
β οΈ You must supply a captcha-solver key
This is the one prerequisite, and it is worth stating plainly rather than burying it in the input table.
Ahrefs protects every free tool with a Cloudflare Turnstile challenge. The Actor cannot fetch any of the /v4 tool data without solving it, so it needs a solver key from CapSolver or 2Captcha β accounts you create and fund yourself at capsolver.com or 2captcha.com.
Supply it either way:
- Paste it into the Captcha API key (
captchaApiKey) input β stored encrypted - Or set the
AHREFS_CAPTCHA_KEYenvironment variable on the Actor
If neither is present, the run logs an explicit error naming both options and exits without pushing any rows. It does not silently return empty results, and it does not charge you for rows it could not fetch.
One exception: the AhrefsTop trending websites report (include_top_websites) reads a plain server-rendered page with no captcha and no /v4 call, so it works without a solver key.
What data can I extract with Ahrefs Scraper?
Every row carries data_captured_at and a type field identifying which report it is, plus the identity fields for that report β domain and mode, or keyword and country, or brand. The rest of the row depends on the type.
| Field Name | Description |
|---|---|
data_captured_at | UTC ISO timestamp for the record β on every row |
type | Report type: traffic, backlinks, broken_links, keywords, keyword_difficulty, keyword_ranking, serp, ai_visibility, top_websites |
domain | Bare host the record is about, normalised (scheme and www. stripped) |
mode | subdomains or exact β the scope the domain report was run at |
keyword | The researched keyword, on keyword-based records |
country | Country code the keyword/SERP data was localised to |
brand | The researched brand, on ai_visibility records |
website_authority | Object: domainRating, backlinks, refdomains, dofollowBacklinks, dofollowRefdomains |
website_traffic | Object: trafficMonthlyAvg, costMontlyAvg |
website_traffic_history | Monthly traffic series |
website_traffic_top_pages | Highest-traffic pages on the domain |
website_traffic_top_countries | Traffic split by country |
website_traffic_top_keywords | Keywords driving the most traffic |
keyword_difficulty | Difficulty score for the keyword |
keyword_difficulty_results | Decoded SERP behind the difficulty score |
keyword_difficulty_source | Source label Ahrefs reported for that SERP |
keyword_difficulty_shortage | Ahrefs' data-shortage indicator for the keyword |
serp_results | Decoded SERP results β each with content_type, data, pos, pos_with_metrics |
serp_source | Source label for the SERP snapshot |
top_position | Best position the domain holds for the keyword, decoded like a SERP result |
last_update | Ahrefs' own last-update stamp for the SERP data |
top_websites_category | Category filter applied to the AhrefsTop ranking |
top_websites_country | Country filter applied to the AhrefsTop ranking |
websites_rankings | Main ranking rows: rank, domain, search_traffic, change, category |
website_rankings_trending_up | Trending-up rows: same shape plus position_change |
website_rankings_trending_down | Trending-down rows: same shape plus position_change |
data | Full decoded payload on backlinks, broken_links, keywords and ai_visibility records |
error | Ahrefs' error message when a report came back empty |
website_authority_error | Error specific to the authority half of a traffic record |
website_traffic_error | Error specific to the traffic half of a traffic record |
costMontlyAvgis spelled exactly that way β it is Ahrefs' own typo in the upstream response, preserved rather than silently corrected so your field mapping matches what actually arrives.
Domain and authority reports
include_web_authority and include_traffic merge into a single traffic record per target. Authority gives you domainRating, backlinks, refdomains and the dofollow variants of both; traffic gives you trafficMonthlyAvg and costMontlyAvg plus four breakdown arrays β history, top pages, top countries, top keywords. include_backlinks and include_broken_links add their own rows with the full decoded payload under data.
mode controls scope: subdomains covers the whole site including subdomains, exact covers only the exact URL or host you supplied.
Keyword and SERP reports
include_keywords returns keyword ideas and question ideas with volumes. include_keywords_difficulty returns the difficulty score plus the SERP it was calculated from. include_serp returns the full SERP overview. include_keywords_ranking needs a keyword and a domain and returns top_position β where that domain actually sits for that keyword. All four localise to country.
SERP entries are decoded rather than passed through raw: each carries content_type (organic, AI overview, questions, and so on), the decoded data, pos, and pos_with_metrics.
AI visibility and trending websites
include_ai_visibility takes a brand or keyword and returns AI citation counts across LLMs, with the model breakdown and top cited domains under data. include_top_websites needs no keyword or domain at all β it returns the AhrefsTop ranking filtered by category_top_websites and country_top_websites, with separate trending-up and trending-down tables.
π€ Add-on: Need additional SEO data?
MOZ Scraper covers the same authority-metric job from a second data source, which is useful when you want a cross-check rather than a single vendor's score. For the search side, Quora Search Scraper and Youtube Search Scraper surface question and video intent that keyword tools tend to miss.
How does Ahrefs Scraper differ from the official Ahrefs API?
Ahrefs publishes an official API, and if you already pay for it, use it β it is the supported route, it returns Ahrefs' full dataset rather than the free-tool preview, and it comes with terms you can rely on commercially. Ahrefs Scraper exists for the case where you do not have that subscription.
| Feature | Official Ahrefs API | Ahrefs Scraper |
|---|---|---|
| Paid Ahrefs subscription required | β | β |
| Data scope | Ahrefs' full index | The free tools' capped preview |
| Captcha-solver key required | β | β CapSolver or 2Captcha |
| AhrefsTop trending websites | Not part of the API | β Included |
| Historical depth | Full series | What the free tools expose |
| Output shape | Ahrefs' documented API schema | One normalised row per report type |
Check Ahrefs' current published API documentation for exact plan requirements, endpoints and unit costs before choosing between them β those terms change and are not reproduced here.
Use the official API for production workloads, contractual guarantees, and full-index depth. Use Ahrefs Scraper for ad-hoc research, prospect audits, and pulling free-tool metrics for many domains without a subscription.
How to use Ahrefs Scraper
Ahrefs Scraper runs on Apify. Start it from the Apify Console or call it through the Apify API.
- Get a CapSolver or 2Captcha API key β this is the prerequisite, not an optional extra
- Open Ahrefs Scraper on Apify and click Try for free
- Paste your solver key into Captcha API key
- Give the Actor something to work on: a URL or Domain, a Keyword or Brand, or tick Include AhrefsTop Trending Websites β no input is marked required in the schema, but a run with none of these three logs "nothing to do" and exits
- Tick the reports you want.
include_web_authority,include_trafficandinclude_backlinksare on by default; the other six are off - Set Country and Mode, click Start, and export as JSON, CSV or Excel
Match the report to the input it needs, or it is silently skipped: authority, traffic, backlinks and broken links need a domain; keyword ideas, difficulty and SERP need a keyword; keyword ranking needs both; AI visibility needs a brand.
How to scale to bulk domain analysis
Use Multiple URLs or Domains (Bulk) (urls) β it is a list, and when set it overrides the single url field entirely. Every domain-based report you enabled runs once per entry, so ten domains with authority, traffic and backlinks ticked produce thirty rows in one run.
Keyword-based reports behave differently: keywords, keyword_difficulty, serp and ai_visibility run once per run, not once per URL, because they key off the single keyword field. keyword_ranking is the exception β it runs per URL, pairing your one keyword against each domain, which is exactly what you want for a share-of-voice comparison.
What can you do with Ahrefs data?
- π An SEO consultant qualifying inbound leads pastes 40 prospect domains into
urlsand sorts the results bywebsite_authority.domainRatingandwebsite_traffic.trafficMonthlyAvgto decide which are worth a proposal. - π’ An agency running a competitive audit enables
include_keywords_rankingwith one target keyword and a list of competitor domains, then readstop_positionfrom each row to build a share-of-voice table in a single run. - π A link-building team enables
include_broken_linksacross a prospect list and works thedatapayload's inbound broken links into a reclamation outreach queue. - π A content strategist pairs
include_keywordswithinclude_keywords_difficultyfor a seed term, then filters keyword ideas againstkeyword_difficultyto find volume that is actually winnable. - π€ An AI engineer building an SEO assistant indexes
website_traffic_top_keywordsandserp_resultsinto a vector store so an agent can answer "which pages drive this competitor's traffic, and who outranks us for their top term" against live data rather than a stale export. - π A brand manager tracking AI search enables
include_ai_visibilityon a brand name and monitors citation counts by model over repeated runs.
Every one of these is callable from an agent framework over the Apify API, since the Actor is a standard HTTP-triggered run.
How does Ahrefs Scraper handle rate limits and blocking?
Two separate defences, handled independently.
Turnstile is solved through your CapSolver or 2Captcha key β that is what the key is for, and it is why the run cannot proceed without one.
IP blocking uses an escalation ladder that starts at the cheapest rung. Every run begins on a direct connection with no proxy at all. Only if Ahrefs blocks a request does the Actor climb: direct β Apify datacenter β Apify residential, logging each switch. Once on residential it retries up to three times before giving up on that request. Setting a country in the proxy input localises the residential fallback. Most runs never leave the direct rung, and the final log line tells you which tier the run actually completed on.
When a single report fails, its error message is captured on the record itself β error on most types, or website_authority_error / website_traffic_error on the split traffic record β and the run continues with the remaining reports. You get a row that tells you what went wrong rather than a missing row you have to notice.
β¬οΈ Input
| Parameter | Required | Type | Description | Example Value |
|---|---|---|---|---|
keyword | No | string | The keyword (or brand, for AI Visibility) to research on Ahrefs. | "project management software" |
url | No | string | The single URL or domain to analyse. For many targets at once, use the bulk field. | "make.com" |
country | No | string | Country to localise keyword, SERP and traffic results. 188 ISO alpha-2 values in the dropdown. Default "us". | "gb" |
mode | No | string | Scope of the domain analysis: "subdomains" (whole site incl. subdomains) or "exact" (just the exact URL/host). Default "subdomains". | "subdomains" |
urls | No | array | A list of URLs or domains to analyse. When set, overrides the single url field. | ["make.com", "zapier.com"] |
include_web_authority | No | boolean | Needs a Domain/URL. Includes Domain Rating, backlinks & referring-domain counts. Default true. | true |
include_traffic | No | boolean | Needs a Domain/URL. Includes estimated monthly visits, traffic value, history, top pages/countries/keywords. Default true. | true |
include_ai_visibility | No | boolean | Needs a Brand/Keyword. Includes AI citations across LLMs, top cited domains/pages & topics. Default false. | false |
include_keywords | No | boolean | Needs a Keyword and Country. Includes keyword ideas + question ideas with volumes. Default false. | true |
include_keywords_difficulty | No | boolean | Needs a Keyword and Country. Includes the Keyword Difficulty score + ranking SERP. Default false. | true |
include_keywords_ranking | No | boolean | Needs a Keyword, Domain and Country. Includes where the domain ranks for the keyword. Default false. | false |
include_serp | No | boolean | Needs a Keyword and Country. Includes the full SERP overview for the keyword. Default false. | false |
include_backlinks | No | boolean | Needs a Domain/URL. Includes the backlink & referring-domain overview. Default true. | true |
include_broken_links | No | boolean | Needs a Domain/URL. Includes broken inbound & outbound links. Default false. | false |
include_top_websites | No | boolean | Includes the top trending websites from AhrefsTop.com β the Top 1000 (worldwide + all categories) or Top 100 when filtered by category/country. Does not need a Keyword or Domain. Default false. | false |
category_top_websites | No | string | Category filter for the AhrefsTop ranking. 26 values from all through autos-and-vehicles. Default "all". | "finance" |
country_top_websites | No | string | Country filter for the AhrefsTop ranking. 189 values including worldwide. Default "worldwide". | "worldwide" |
proxyConfiguration | No | object | The scraper starts on a DIRECT connection and only escalates to a datacenter, then residential proxy if Ahrefs blocks it. Pick a country here to localise residential fallback. Default {"useApifyProxy": true}. | {"useApifyProxy": true} |
captchaApiKey | No | string | Required in practice. Ahrefs protects every free tool with a Cloudflare Turnstile challenge, so this Actor needs a captcha-solver key to fetch data. Paste your CapSolver or 2Captcha API key. Stored encrypted. Can also be set via the AHREFS_CAPTCHA_KEY environment variable. | "CAP-XXXXXXXXXXXX" |
Common pitfall: captchaApiKey is not flagged required by the schema, but every /v4 report needs it. A run without it exits with an error before pushing anything.
Example input
{"urls": ["make.com", "zapier.com", "n8n.io"],"keyword": "workflow automation","country": "us","mode": "subdomains","include_web_authority": true,"include_traffic": true,"include_backlinks": true,"include_broken_links": false,"include_keywords": true,"include_keywords_difficulty": true,"include_keywords_ranking": true,"include_serp": false,"include_ai_visibility": false,"include_top_websites": false,"category_top_websites": "all","country_top_websites": "worldwide","proxyConfiguration": { "useApifyProxy": true },"captchaApiKey": "CAP-XXXXXXXXXXXX"}
β¬οΈ Output
Typed, normalized JSON with a consistent shape per type across runs. Rows are pushed live as each report lands, so the dataset fills while the run is still going. Export as JSON, CSV or Excel, or read the dataset through the Apify API.
Ahrefs encodes its responses in an OCaml/Melange style β options as ["Some", x], results as ["Ok", x], enums as single-element arrays. All of it is decoded recursively before it reaches you, so what lands in the dataset is plain JSON with null where Ahrefs sent None.
Example output
A traffic record, the type produced by the two reports that are on by default:
{"data_captured_at": "2026-07-25T09:14:02.481230+00:00","type": "traffic","domain": "make.com","mode": "subdomains","website_authority": {"domainRating": 91,"backlinks": 4182337,"refdomains": 38412,"dofollowBacklinks": 3120884,"dofollowRefdomains": 29760},"website_traffic": {"trafficMonthlyAvg": 2841500,"costMontlyAvg": 1934200},"website_traffic_history": [{ "date": "2026-05-01", "organic": 2705300, "paid": 18400 },{ "date": "2026-06-01", "organic": 2788100, "paid": 17250 },{ "date": "2026-07-01", "organic": 2841500, "paid": 16980 }],"website_traffic_top_pages": [{ "url": "https://www.make.com/en/templates", "traffic": 184300, "keywords": 6120 },{ "url": "https://www.make.com/en/integrations", "traffic": 121750, "keywords": 4488 }],"website_traffic_top_countries": [{ "country": "us", "traffic": 812400, "share": 0.286 },{ "country": "gb", "traffic": 214900, "share": 0.076 }],"website_traffic_top_keywords": [{ "keyword": "make automation", "position": 1, "volume": 74000, "traffic": 31200 },{ "keyword": "integromat", "position": 1, "volume": 60500, "traffic": 24800 }]}
A keyword_difficulty record from the same run:
{"data_captured_at": "2026-07-25T09:14:11.902847+00:00","type": "keyword_difficulty","keyword": "workflow automation","country": "us","keyword_difficulty": 68,"keyword_difficulty_source": "serp","keyword_difficulty_shortage": null,"keyword_difficulty_results": [{"content_type": "organic","data": { "url": "https://zapier.com/workflow-automation", "title": "Workflow Automation", "domainRating": 92 },"pos": 1,"pos_with_metrics": 1}]}
How does it work?
Ahrefs' free tools are backed by a /v4 JSON API, each call gated by a Cloudflare Turnstile challenge. Ahrefs Scraper solves that challenge with your CapSolver or 2Captcha key, calls the tool endpoint, and shapes the response into a flat record.
Requests start on a direct connection. If Ahrefs blocks one, the Actor escalates to an Apify datacenter proxy and then to residential, retrying up to three times on the residential rung, and logs which tier it ended on.
The responses arrive in Ahrefs' internal OCaml/Melange encoding, which is decoded recursively β options unwrapped, results unwrapped, enums flattened, signed URL wrappers reduced to a plain url. Because the Actor reads that structured API rather than scraping rendered HTML, an Ahrefs front-end redesign does not affect it, and your field names stay put. The AhrefsTop report is the one exception: it is a server-rendered page, parsed from HTML, and needs no captcha at all.
Only publicly visible free-tool data is collected. No Ahrefs account, cookie or session is used.
Integrations
Ahrefs Scraper is an Apify Actor, so it works with anything that can call the Apify API or consume a dataset.
Calling Ahrefs Scraper from Python
from apify_client import ApifyClientclient = ApifyClient("<YOUR_APIFY_TOKEN>")run = client.actor("<YOUR_USERNAME>/ahrefs-scraper").call(run_input={"urls": ["make.com", "zapier.com"],"keyword": "workflow automation","country": "us","include_web_authority": True,"include_traffic": True,"include_keywords_difficulty": True,"captchaApiKey": "<YOUR_CAPSOLVER_KEY>",})for rec in client.dataset(run["defaultDatasetId"]).iterate_items():if rec["type"] == "traffic" and not rec.get("website_authority_error"):wa = rec["website_authority"]print(rec["domain"], "DR", wa["domainRating"], "refdomains", wa["refdomains"])
Works in Go, Ruby, Node.js, cURL β any language that can make an HTTP request. Branch on type to route each report to the right handler.
No-code tools (n8n, Make, Zapier)
In n8n, use the Apify node β or an HTTP Request node pointed at the Apify run endpoint with your token β and pass the same JSON input shown above; a Switch node on type then splits the reports into separate branches. In Make, the Apify module supports run-and-wait, so a monthly authority sweep can feed a Google Sheets or Airtable step directly. Both platforms handle scheduling, so a recurring competitor audit needs no code.
Is it legal to scrape Ahrefs data?
Scraping publicly accessible tool output is broadly treated as permissible where no authentication is bypassed, and Ahrefs Scraper reads only Ahrefs' free public tools β no account, no cookie, no paid session.
The output is aggregate SEO metrics about websites: Domain Rating, traffic estimates, backlink counts, keyword volumes, ranking positions. These are business and technical records, not personal data, so GDPR and equivalent personal-data regimes do not attach to this output the way they would to a profile scraper.
What does apply is contractual and database-related: Ahrefs' terms of service, and database or unfair-competition rules covering systematic reuse of a compiled dataset. Solving a captcha to reach the tools is a factor worth raising with counsel, since it is a deliberate access control rather than an incidental one.
Consult legal counsel before reselling extracted Ahrefs metrics as a standalone product or republishing them as a competing dataset.
β Frequently asked questions
What Ahrefs fields does Ahrefs Scraper return?
The five most used are website_authority.domainRating, website_authority.refdomains, website_traffic.trafficMonthlyAvg, keyword_difficulty and serp_results. Every row also carries data_captured_at and type. See the data fields table above for all of them.
Does Ahrefs Scraper require an Ahrefs account or subscription?
No. It reads Ahrefs' free public tools, which need no account, no login and no paid plan. It does require a CapSolver or 2Captcha key, because Ahrefs puts a Cloudflare Turnstile challenge in front of those tools β that is a third-party service you sign up for separately.
Why does my run fail immediately with a captcha error?
Because no solver key was found. The Actor checks captchaApiKey and then the AHREFS_CAPTCHA_KEY environment variable; if both are empty it logs an error naming both options and exits before pushing anything. Add a CapSolver or 2Captcha key and re-run. The AhrefsTop trending-websites report is the only one that works without a key.
How many reports can I get in one run?
Nine types are available, and domain-based reports repeat per target. With urls holding ten domains and authority, traffic, backlinks and broken links all enabled, one run produces forty rows. Keyword-based reports run once per run, so add one row each for keywords, keyword_difficulty, serp and ai_visibility.
What happens if a report returns nothing?
The record is still written, with Ahrefs' own error message attached β error on most types, or website_authority_error / website_traffic_error on the split traffic record β and the run continues. A failed report is visible in your dataset rather than being an absence you have to spot. Filter clean records with rec.get("error") is None.
Can I analyse multiple domains at once?
Yes. urls is a list and overrides the single url field when set. Every domain-based report runs once per entry. To pair one keyword against many domains for share-of-voice, enable include_keywords_ranking β it runs per URL and returns top_position for each.
Does Ahrefs Scraper work with Claude, ChatGPT and other AI agent tools?
Yes. It is callable as a standard HTTP endpoint through the Apify API, so LangChain, CrewAI, n8n or a hand-written tool definition can invoke it and receive typed JSON with no parsing step.
How does Ahrefs Scraper compare to other Ahrefs scrapers?
Checked on the Apify Store on 25 July 2026: radeance/ahrefs-scraper is the most used, with an extensive README documenting a very similar report set β website authority, traffic, AI visibility β and the same data_captured_at / type / domain / mode record shape. pro100chok/ahrefs-seo-tools documents a comparable spread with per-tool output examples. scrap3r/ahrefs-domain-overview-checker is narrower, covering the domain-overview DR/UR check only. All of these read the same free Ahrefs tools and therefore face the same Turnstile gate. What this Actor documents explicitly is the captcha prerequisite, the direct-first proxy ladder, per-report error capture on the record itself, and the AhrefsTop trending-websites report as a no-captcha extra.
Does Ahrefs Scraper return data in a format LLMs can use directly?
Yes. Every record is typed, normalized JSON with consistent field names across runs, decoded out of Ahrefs' internal encoding. No HTML parsing, no selectors. Pass a row directly into an LLM, index it into a vector store, or hand it to an agent tool.
What happens when Ahrefs changes its layout or anti-bot system?
The scraper is maintained, and because eight of the nine reports read Ahrefs' /v4 JSON API rather than rendered markup, a front-end redesign generally does not affect them at all. The AhrefsTop report is HTML-parsed and is the one exposed to layout change. Your field names and types stay the same on your end.
Can I use Ahrefs Scraper without managing proxies?
Yes. The proxy ladder is automatic β direct, then Apify datacenter, then Apify residential with up to three retries β and you never create a proxy account or rotate an IP. Setting a country in proxyConfiguration localises the residential rung. The captcha solver is the one external account you do have to set up yourself.
Which Ahrefs fields work best for AI training data and RAG indexing?
For RAG indexing: website_traffic_top_keywords, website_traffic_top_pages and serp_results carry the most descriptive text per record and chunk cleanly by domain. For training data: website_authority.domainRating, website_authority.refdomains, website_traffic.trafficMonthlyAvg and keyword_difficulty are the most structurally consistent fields across records. All values are returned as typed primitives, arrays or plain nested objects, so no normalization pass is needed before indexing.
π Related scrapers
| Scraper Name | What it extracts |
|---|---|
| MOZ Scraper | Domain and page authority metrics from a second SEO data source |
| Quora Search Scraper | Question-intent results for keyword and content research |
| Youtube Search Scraper | Video search results for a keyword |
| Google Play Store Reviews Scraper | App review text, ratings and versions |
| Twitter Trends Scraper | Trending topics for timely content planning |
π¬ Your feedback
Found a bug, or need an Ahrefs free tool that is not yet covered? Open an issue on the Actor's Issues tab. Reports that include the exact input JSON and the target domain or keyword are the fastest to reproduce and fix.