Twitter Profile Scraper

Twitter/X Profile Scraper — Extract Tweets, Stats and Bio Data

Twitter/X Profile Scraper extracts tweets and profile data from any public X account — no API keys and no login required for standard runs. Point it at a profile URL or handle and get back full_text, favorite_count, retweet_count, views_count, and flattened profile stats like followersCount and bio as clean, typed JSON ready for analysis, CSV export, or a database. Paste a handle below and start your first run.

What is Twitter/X Profile Scraper?

Twitter/X Profile Scraper is an Apify Actor that pulls a public X (Twitter) account's tweet timeline — plus an optional flattened profile-stats record — into structured JSON, using X's own GraphQL endpoints instead of browser automation. No Twitter/X account or login is required for standard tweet and profile scraping; a login cookie pair (auth_token + ct0) is only needed if you turn on reply scraping. It's built for researchers, growth marketers, and developers who need public tweet and profile data pipelines without maintaining their own scraper.

What Twitter/X profile data is publicly available to scrape?

A public X profile's tweets, engagement counts, and basic bio/stats are all visible without logging in — this Actor reads them the same way a signed-out browser does.

Data Category	Publicly Available	Requires Login Cookies
Tweets from a public profile's timeline	Yes	—
Engagement counts (likes, retweets, replies, quotes, bookmarks, views)	Yes	—
Profile bio, location, avatar/banner images	Yes (via getAboutData)	—
Follower/following/tweet/listed counts	Yes (via getAboutData)	—
Verification status (legacy + Blue)	Yes (via getAboutData)	—
Replies to a specific tweet	No	requires authToken + csrfToken
Protected/private account content	No	not accessible — a login can't bypass this
Direct messages	No	never exposed publicly

Twitter/X Profile Scraper only returns publicly visible data — what any visitor sees. Nothing behind a login wall, and reply scraping is opt-in and clearly gated behind your own cookies.

What data can I extract with Twitter/X Profile Scraper?

Twitter/X Profile Scraper returns one row per tweet (plus an optional per-user "about" row), covering tweet content, engagement metrics, and profile/account fields.

Field Name	Description
id_str	Tweet's unique numeric ID
full_text	Full text of the tweet
tweetUrl	Direct link to the tweet, built from screen_name + id_str
created_at	Tweet timestamp in X's native format
screen_name	Author's handle
user_id_str	Author's numeric user ID
conversation_id_str	ID of the thread the tweet belongs to
in_reply_to_status_id_str / in_reply_to_user_id_str / in_reply_to_screen_name	Reply-target IDs, present when the tweet is a reply
lang	Detected tweet language code
entities / extended_entities	Raw hashtags, mentions, URLs, and media objects
display_text_range	Character range of the visible tweet text
is_quote_status	Whether the tweet quotes another tweet
possibly_sensitive	X's own sensitive-content flag

Engagement metrics

Field Name	Description
favorite_count	Like count
retweet_count	Retweet count
reply_count	Reply count
quote_count	Quote-tweet count
bookmark_count	Bookmark count
views_count	View count
favorited / retweeted / bookmarked	Whether the scraping session's own account performed that action (normally false without login)

Profile & account fields

These are flattened to top-level columns on the optional per-user "about" row (getAboutData: true); the raw user object with the same underlying data is also embedded on every tweet row.

Field Name	Description
user	Full raw author object, embedded on every tweet row
followersCount	Follower count
followingCount	Accounts followed
tweetsCount	Total tweets/statuses posted
favouritesCount	Total likes given by the account
listedCount	Number of public lists the account appears on
accountCreatedAt	Account creation date
verified	Legacy verified flag
isBlueVerified	X Premium (Blue) verified flag
location	Profile location text
bio	Profile biography text
profileImageUrl	Avatar image URL
profileBannerUrl	Banner image URL
scrapedAt	UTC timestamp the row was written
isReply / parentTweetId	Present on reply rows, when getReplies is enabled

🤖 Add-on: Need additional Twitter/X data?

Scraper-Engine doesn't currently publish a separate X/Twitter search, hashtag, or followers-list scraper — this Actor is the account's only Twitter/X entry point today. If you need lightweight profile monitoring across many accounts, enabling getAboutData alongside a broad twitterHandles list covers most of that need without a dedicated tool.

Why not build this yourself?

X's GraphQL query IDs and bearer tokens change without notice — this Actor re-discovers the current UserTweets and TweetDetail query IDs and bearer token from X's own client JavaScript on every run instead of hardcoding values that break the moment X ships an update. Its source code carries multiple fallback strategies for resolving a user ID (schema.org ld+json, React-flight-data, legacy JSON, banner-image IDs) specifically because earlier, simpler approaches broke when X changed its page internals. Maintaining that discovery-and-fallback layer yourself means continuously tracking X's minified bundles and re-testing extractors every time the page structure shifts, on top of handling proxy rotation for rate limits. Running it as a hosted Actor means you get endpoint discovery, header/cookie handling, and structured dataset output already built — instead of building and babysitting all of it yourself for one project.

How to use Twitter/X Profile Scraper

Twitter/X Profile Scraper runs on the Apify platform — no separate signup or Twitter/X developer approval needed.

1. Open Twitter/X Profile Scraper on the Apify Store and click "Try for free," or start it via the Apify API/CLI with your Apify API token.
2. Provide at least one profile — paste full URLs into startUrls (e.g. https://x.com/nasa) or bare usernames into twitterHandles (e.g. nasa). No Twitter/X login is needed for this step.
3. Optionally set maxItems, a start/end date range, onlyImages, or includeNativeRetweets to narrow results.
4. Start the run.
5. Download results as JSON, CSV, or Excel, or stream dataset items via the Apify API once the run finishes.

How to scale to bulk Twitter/X profile extraction

Both startUrls and twitterHandles accept arrays, so a single run can process many profiles at once — mix full URLs and bare handles freely in the same list. maxItems caps rows across the entire run, not per profile, so raise it in step with your target profile count. For very large profile lists you want to run on a recurring basis, split them across separate scheduled runs (via the Apify API or a schedule) rather than one unbounded run.

What can you do with Twitter/X profile data?

• A growth marketer tracking competitors uses full_text and views_count to see which tweet formats perform, ranking them with retweet_count and favorite_count.
• A social listening analyst uses bio, location, and followersCount from the about-data row to segment an audience list before a launch campaign.
• A research team studying posting behavior uses created_at and accountCreatedAt to build an activity timeline across a set of public accounts.
• An outreach specialist builds an enriched contact list from screen_name, bio, and profileImageUrl for accounts matching a niche, ahead of manual outreach.
• An AI engineer feeds full_text, entities, and engagement fields (favorite_count, retweet_count, views_count) into a RAG pipeline or an LLM-based sentiment/topic classifier directly, with no HTML parsing step.

Because output is typed JSON callable through the Apify API, any of these pipelines can be wired directly into an agent framework.

How does Twitter/X Profile Scraper handle rate limits and blocking?

Twitter/X Profile Scraper doesn't run a browser — it talks to X's own GraphQL API directly, discovering the current query IDs and bearer token fresh from X's client JavaScript on each run so it keeps working as X rotates its internal endpoints. Requests can be routed through Apify Proxy (proxyConfiguration) to distribute traffic across IPs and reduce rate-limit friction; this is optional and off by default. Unauthenticated runs use a guest token activated at the start of the run; reply scraping additionally uses your own authToken/csrfToken login cookies.

One real limitation: this Actor does not implement automatic retry-with-backoff. If a tweet-timeline request returns a non-200 response, that profile's pagination stops there and a warning is logged — the run moves on to the next profile rather than retrying. If resolving a profile's user ID fails entirely, that profile is skipped and logged, and the run continues with the remaining targets rather than failing the whole run.

⬇️ Input

All parameters are optional in the schema, but you must supply at least one profile via startUrls or twitterHandles for a run to produce results.

Parameter	Required	Type	Description	Example Value
startUrls	No	array	List of Twitter/X profile URLs (e.g., https://x.com/elonmusk). Required if twitterHandles is empty.	["https://x.com/nasa"]
twitterHandles	No	array	List of usernames without the @ (e.g., elonmusk). Required if startUrls is empty.	["nasa"]
maxItems	No	integer	Hard cap on total items written across the whole run. Default 10.	100
start	No	string	Lower bound for tweet date. Use absolute date (YYYY-MM-DD) or relative value (e.g. 7 days). Empty = no lower bound.	"7 days"
end	No	string	Upper bound for tweet date. Use absolute date (YYYY-MM-DD) or relative value (e.g. 1 day). Empty = no upper bound.	""
includeNativeRetweets	No	boolean	When false, native retweets (full_text starts with RT @) are dropped. Only effective for the last ~7-10 days. Default false.	false
onlyImages	No	boolean	Keep only tweets that contain at least one photo. Default false.	false
getAboutData	No	boolean	Emit one profile-about record per user (uses the embedded user object from UserTweets). Default false.	true
getReplies	No	boolean	Fetch replies to each qualifying tweet via the TweetDetail endpoint. Requires login cookies. Default false.	false
minReplyCount	No	integer	Only fetch replies for tweets whose reply_count is at least this value. Default 0.	5
authToken	No	string	Value of the auth_token cookie from x.com (devtools → Application → Cookies). Required for getReplies.	"a1b2c3..."
csrfToken	No	string	Value of the ct0 cookie from x.com. Required for getReplies.	"d4e5f6..."
proxyConfiguration	No	object	Apify proxy configuration. Recommended for reliability against rate limits.	{"useApifyProxy": true}

Example input

{
  "startUrls": ["https://x.com/nasa"],
  "twitterHandles": ["elonmusk"],
  "maxItems": 100,
  "start": "7 days",
  "end": "",
  "includeNativeRetweets": false,
  "onlyImages": false,
  "getAboutData": true,
  "getReplies": false,
  "minReplyCount": 0,
  "authToken": "",
  "csrfToken": "",
  "proxyConfiguration": { "useApifyProxy": true }
}

⬆️ Output

Results are written to the Actor's default dataset as typed, normalized JSON rows — one per tweet, plus one optional per-user "about" row — with a consistent schema across runs. Export directly from the Apify Console as JSON, CSV, Excel, or XML, or pull rows via the Apify API/dataset client.

Example output

{
  "user": {
    "screen_name": "nasa",
    "name": "NASA",
    "followers_count": 98230451,
    "friends_count": 470,
    "statuses_count": 76210,
    "favourites_count": 3812,
    "listed_count": 152400,
    "created_at": "Wed Dec 06 16:07:57 +0000 2007",
    "verified": true,
    "is_blue_verified": true,
    "profile_image_url_https": "https://pbs.twimg.com/profile_images/.../nasa_400x400.jpg",
    "profile_banner_url": "https://pbs.twimg.com/profile_banners/.../1600000000",
    "location": "",
    "description": "Explore the universe and discover our home planet."
  },
  "bookmark_count": 214,
  "bookmarked": false,
  "created_at": "Fri Jul 18 14:32:05 +0000 2026",
  "conversation_id_str": "1943821004552192000",
  "display_text_range": [0, 187],
  "entities": {
    "hashtags": [{ "text": "Artemis" }],
    "user_mentions": [],
    "urls": [],
    "timestamps": []
  },
  "extended_entities": {
    "media": [{ "type": "photo", "media_url_https": "https://pbs.twimg.com/media/example.jpg" }]
  },
  "favorite_count": 15420,
  "favorited": false,
  "full_text": "New images from the Artemis mission show the lunar south pole in stunning detail. #Artemis",
  "is_quote_status": false,
  "lang": "en",
  "quote_count": 88,
  "reply_count": 342,
  "retweet_count": 3190,
  "retweeted": false,
  "user_id_str": "11348282",
  "id_str": "1943821004552192001",
  "possibly_sensitive": false,
  "views_count": 1284730,
  "screen_name": "nasa",
  "tweetUrl": "https://x.com/nasa/status/1943821004552192001",
  "scrapedAt": "2026-07-25T09:12:44Z"
}

How does it work?

Twitter/X Profile Scraper doesn't use a browser. On each run it fetches a public X search page and extracts the current main.*.js client bundle URL, then reads that JavaScript to pull the live GraphQL queryId values for X's UserTweets and TweetDetail operations plus the bearer token — the same credentials the x.com web app uses internally. It resolves your handle to a numeric user ID from the profile page's own structured (ld+json) data, then pages through that user's tweet timeline via X's GraphQL endpoint, applying your date, retweet, and image filters as it writes rows. Because it reads the same public endpoints your browser calls when signed out, only publicly visible tweets and profile fields are ever returned, and the output schema (id_str, full_text, engagement counts, flattened profile stats) stays the same even when X changes its page layout.

Integrations

Twitter/X Profile Scraper works with the tools you already use for automation and data pipelines through the standard Apify platform integrations.

Calling Twitter/X Profile Scraper programmatically

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")

run_input = {
    "twitterHandles": ["elonmusk", "nasa"],
    "maxItems": 100,
    "getAboutData": True,
}

run = client.actor("<YOUR_USERNAME>/twitter-profile-scraper-pay-per-events").call(run_input=run_input)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item.get("full_text") or item.get("screen_name"))

Works in Go, Ruby, Node.js, cURL — any language that can make an HTTP request.

No-code tools (n8n, Make)

In n8n, use the Apify node (or a generic HTTP Request node) to call this Actor's run endpoint with your input JSON and Apify API token, then feed the returned dataset items into downstream nodes. In Make, use the Apify app's "Run Actor" module with the same JSON input and iterate over the resulting dataset items in your scenario.

Is it legal to scrape Twitter/X profiles?

Scraping publicly available social media data is generally lawful — courts (e.g., hiQ v. LinkedIn in the US) have repeatedly held that accessing public, non-login-walled pages is not itself prohibited. Twitter/X Profile Scraper only returns data already visible on a public profile page or public tweet timeline; it does not access private accounts, DMs, or anything behind a login wall, and reply scraping still only surfaces public replies. Because tweets and profile fields (bio, location, handle) can be personal data about identifiable people, GDPR and CCPA still govern how you store, process, and retain what you collect, even though the collection itself targets public data. Consult legal counsel before any bulk storage or commercial use of personal data scraped this way.

Frequently asked questions

What Twitter/X fields does Twitter/X Profile Scraper return?

It returns full_text, favorite_count, retweet_count, views_count, and created_at on every tweet row, plus follower and bio fields when getAboutData is enabled — see What data can I extract above for the full field list.

Does Twitter/X Profile Scraper require a Twitter/X account or login?

No, not for standard tweet and profile scraping — it uses a guest session activated automatically at the start of each run. A login cookie pair (authToken + csrfToken) is only required if you enable getReplies.

How many tweets can I extract in one run?

There's no fixed schema cap — maxItems (default 10) sets the total across the whole run, and internally the Actor paginates up to 40 pages per profile (about 50 tweets fetched per page) before moving to the next target.

What happens if a profile is private, suspended, or doesn't exist?

No row is written for that profile. The Actor logs a "User not found" warning and moves on to the next target in your list rather than failing the whole run.

Can I scrape multiple Twitter/X profiles at once?

Yes — both startUrls and twitterHandles accept arrays, so you can list many profile URLs and/or bare handles in a single run and they'll be processed in order until maxItems is reached.

Does Twitter/X Profile Scraper work with Claude, ChatGPT, and other AI agent tools?

Yes, as a callable HTTP endpoint through the Apify API — any agent framework that can make an authenticated HTTP request can trigger a run and read back the resulting dataset items.

How does Twitter/X Profile Scraper compare to other Twitter/X profile scrapers?

It re-discovers X's live GraphQL query IDs and bearer token from X's own client JavaScript on every run instead of relying on hardcoded values, and it flattens profile/about data (followers, bio, verification) to top-level columns rather than leaving them nested in a raw object.

Does Twitter/X Profile Scraper return data in a format LLMs can use directly?

Yes. Output is typed, normalized JSON with consistent field names across runs — no HTML parsing or CSS selectors needed. Pass it directly to an LLM, index it into a vector store, or feed it to an agent tool.

What happens when Twitter/X changes its layout or anti-bot system?

The Actor is maintained and re-discovers its GraphQL query IDs and bearer token fresh on every run, so the output schema stays stable through most front-end changes. No specific fix turnaround time is promised for deeper API changes.

Can I use Twitter/X Profile Scraper without managing proxies or browser infrastructure?

Yes. It makes direct HTTP/GraphQL calls with no browser involved, and proxy support (proxyConfiguration) is optional — you can run it with no proxy configured at all.

Which Twitter/X fields work best for AI training data and RAG indexing?

For RAG, index full_text alongside entities (hashtags/mentions) and bio for author context. For consistent-structure training data, favorite_count, retweet_count, views_count, and followersCount return as typed numeric primitives across every row.

Related scrapers

Scraper Name	What it extracts
Instagram Profile Scraper	Public Instagram profile bio, follower/following/post counts, verification, and business-account status
LinkedIn Profile & Credibility Analysis Scraper	Public LinkedIn profile data plus a 0-100 credibility score from recommendations, network reach, and experience
YouTube Channel Finder	YouTube channel name, handle, subscriber/view/video counts, and channel metadata by keyword or URL

Your feedback

Found a bug or missing a field? Let us know so we can fix it. Reach out through the Actor's Issues tab on Apify or your usual Scraper-Engine support channel — active maintenance means reported gaps get looked at.