# 🐦 Twitter / X Tweets Scraper (`scrapeengine/twitter-tweets-scraper`) Actor

Scrape public tweets from any Twitter / X profile in real time — bulk URLs, date filters, smart proxy fallback, structured JSON / CSV / Excel output.

- **URL**: https://apify.com/scrapeengine/twitter-tweets-scraper.md
- **Developed by:** [ScrapeEngine](https://apify.com/scrapeengine) (community)
- **Categories:** Automation, Lead generation, Social media
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.99 / 1,000 results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## 🐦 Twitter / X Tweets Scraper

> The fastest, friendliest way to collect public tweets from any **X (Twitter)** profile — at scale, in real time, and with built-in anti-block defences.

Paste one (or hundreds) of profile URLs, pick how many tweets per profile, and watch structured JSON / CSV / Excel rolls into your dataset live — already labelled, sorted, and ready to drop into your BI tool, spreadsheet, or LLM pipeline.

***

### ✨ Why this Actor?

| 🟢 This Actor | 🔴 Other tools |
|---|---|
| ⚡ **Rust-powered HTTP** via `impit` — looks identical to a real Chrome browser | Plain `requests` / `httpx` that anti-bot engines block on sight |
| 🛡️ **4-tier proxy escalator** (direct → datacenter → residential, sticky) | Single proxy, no fallback |
| 📡 **Live streaming output** — first tweet appears in seconds | Wait until the run ends to see any data |
| 📅 **`since_date` filter** built in | Manual post-filtering required |
| 🪵 **Per-profile section logs** with engagement preview | Wall-of-text logs |
| 🔐 **Cookie-based authentication** — bypasses guest rate limits | Limited guest quotas |

***

### 🚀 Key Features

- 🔗 **Bulk URL input** — paste an unlimited number of profile URLs or `@handles` at once
- 🎯 **Per-profile cap** — stop after *N* tweets per profile
- 📅 **Since-date filter** — only tweets posted on or after a given date
- 🌐 **Smart proxy fallback** — starts direct, automatically escalates if X pushes back
- ⏱️ **Polite, jittered pacing** — configurable inter-request delay with random jitter
- 📊 **Live dataset writes** — every tweet is pushed the moment it is parsed; nothing is lost on crash
- 🪵 **Beautiful run log** — engagement counts, previews, and section headers per profile
- 🧾 **Rich tweet schema** — text, language, full entities, all engagement metrics, full user object, view counts, and more

***

### 📥 Input

| Field | Type | Required | Description |
|---|---|---|---|
| `start_urls` | array | ✅ | List of X profile URLs (e.g. `https://x.com/peckshield`) or bare usernames |
| `auth_token` | string (secret) | ✅ | Your X.com `auth_token` cookie |
| `ct0` | string (secret) | ✅ | Your X.com `ct0` (CSRF) cookie |
| `max_items` | integer | – | Max tweets per profile (default `100`, range 1–10,000) |
| `since_date` | string | – | Apify datepicker: absolute `YYYY-MM-DD` or relative (`7 days`, `2 weeks`, …). Only tweets from the resolved date forward |
| `request_delay` | number | – | Seconds between page requests (default `1.0`, jitter added in code) |
| `proxy_configuration` | object | – | Defaults to direct connection. Fallback escalator kicks in automatically |

#### 🔐 How to get the cookies

1. Open <https://x.com> in your browser, signed in.
2. Press **F12** → **Application** tab → **Cookies → https://x.com**.
3. Copy the values of **`auth_token`** and **`ct0`** into the input fields above.
4. The Actor stores both as secrets — they never appear in the run log.

#### 📋 Example input

```json
{
  "start_urls": [
    { "url": "https://x.com/peckshield" },
    { "url": "https://x.com/elonmusk" },
    { "url": "naval" }
  ],
  "max_items": 200,
  "since_date": "2026-01-01",
  "auth_token": "REDACTED",
  "ct0": "REDACTED",
  "request_delay": 1.0,
  "proxy_configuration": { "useApifyProxy": false }
}
```

***

### 📤 Output

Every tweet is pushed to the default dataset as soon as it is parsed. Open the **Dataset → Overview** tab for a clean, sortable table.

#### 📊 Example output row

```json
{
  "id_str": "2056183051357495735",
  "url": "https://x.com/peckshield/status/2056183051357495735",
  "created_at": "Mon May 18 01:20:16 +0000 2026",
  "full_text": "@VerusCoin token has been hacked. ~$1.78M stolen…",
  "lang": "en",
  "favorite_count": 142,
  "retweet_count": 31,
  "reply_count": 12,
  "quote_count": 4,
  "bookmark_count": 24,
  "views_count": "98231",
  "is_quote_status": false,
  "possibly_sensitive": false,
  "user_screen_name": "peckshield",
  "user_name": "PeckShield Inc.",
  "user_followers_count": 312874,
  "user_is_verified": true,
  "_section": "@peckshield",
  "entities": { "media": [ … ], "urls": [ … ], "user_mentions": [ … ] },
  "extended_entities": { "media": [ … ] },
  "user": { "rest_id": "…", "legacy": { … }, "professional": { … } }
}
```

The `_section` column tags every row with the source profile so you can slice the dataset cleanly per profile in the Console table view.

#### 📂 Dataset view

The dataset comes preconfigured with an **Overview** table that shows:

`Tweet ID · Tweet URL · Posted At · Tweet · Lang · ❤️ Likes · 🔁 Retweets · 💬 Replies · 📝 Quotes · 🔖 Bookmarks · 👁 Views · @handle · Followers · Verified`

Export options: **JSON · JSONL · CSV · Excel · HTML · RSS · XML**.

***

### 🚀 How to Use (Apify Console)

1. Log in at <https://console.apify.com> → **Actors**.
2. Open **Twitter / X Tweets Scraper**.
3. Paste profile URLs / handles into **🔗 Twitter / X Profile URLs**.
4. Paste your `auth_token` and `ct0` cookies into the secret fields.
5. Pick **Max tweets per profile** and an optional **Since date**.
6. Click **Start**.
7. Watch tweets stream into the **Output** tab in real time.
8. When the run finishes, export to JSON / CSV / Excel.

***

### 🤖 Use via API / MCP

#### REST API (sync — returns dataset items in one call)

```bash
curl -X POST "https://api.apify.com/v2/acts/<ACTOR_ID>/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "start_urls": [{ "url": "https://x.com/peckshield" }],
       "max_items": 100,
       "since_date": "2026-01-01",
       "auth_token": "REDACTED",
       "ct0": "REDACTED"
     }'
```

#### Python SDK

```python
from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("<ACTOR_ID>").call(run_input={
    "start_urls": [{"url": "https://x.com/peckshield"}],
    "max_items": 50,
    "auth_token": "REDACTED",
    "ct0": "REDACTED",
})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["full_text"])
```

***

### 🎯 Best Use Cases

- 📈 **Market & competitor monitoring** — track what top accounts in your space are saying
- 🛡️ **Security / threat intel** — watch researchers like `@peckshield`, `@SlowMist_Team` for live exploit reports
- 🪙 **Crypto signal mining** — feed tweets into your sentiment model
- 📰 **News & journalism** — bulk-collect statements from public figures
- 🧠 **LLM training / fine-tuning** — curate domain-specific tweet corpora
- 🔍 **Brand mentions & PR** — pull every tweet from a brand's official handle for analysis

***

### 💲 Pricing

This Actor is monetized with **pay-per-event (PPE)** — you pay only for the **scraped tweets you actually receive**.

| Event | What you pay for |
|---|---|
| 🪙 `apify-actor-start` | Per-run startup (covered up to 5s of compute) |
| 📥 `result-item` | Per tweet pushed to the dataset (the primary value unit) |

Configure exact prices in the **Pricing** tab in Console. Platform usage (compute, proxy) is included in the per-event price. The Actor exits gracefully when your `ACTOR_MAX_TOTAL_CHARGE_USD` is reached — you never get charged beyond your cap.

***

### ❓ Frequently Asked Questions

**Q: Do I really need the `auth_token` and `ct0` cookies?**
Yes. X's guest endpoint is heavily throttled; authenticated requests are required for stable bulk scraping. The cookies are stored as Actor secrets and never logged.

**Q: My account is at risk of being banned?**
X may flag accounts that scrape aggressively. We recommend using a dedicated burner account, conservative `max_items`, and a positive `request_delay`. The Actor uses real-browser HTTP/TLS fingerprints to look like normal traffic.

**Q: What if I get rate limited?**
The Actor automatically escalates from direct → datacenter proxy → residential proxy (sticky, with 3 IP rotations) the moment X starts pushing back. No action needed from you.

**Q: Can I scrape protected / private accounts?**
No — the Actor only scrapes **public** tweets. Following a private account does not grant scrape access.

**Q: How fresh is the data?**
Live. Each tweet hits your dataset within milliseconds of being parsed.

**Q: Does this work for replies / threads / quote tweets?**
Yes — replies, quotes, retweets, media-only tweets, and longform note tweets are all extracted with full text and entities.

***

### ⚖️ Cautions & Legal

- Data is collected only from **publicly available sources** on x.com.
- **You** are responsible for legal compliance: GDPR, CCPA, anti-spam laws, X's Terms of Service, and any local regulation that applies to you.
- Do not use this Actor to scrape private accounts, automate harassment, or build profiles of private individuals.
- The Actor honours reasonable rate limits and `robots.txt` etiquette by default.

***

### 📬 Support & Feedback

- Open an issue on the Actor page if something is off.
- Feature requests welcome — we ship fast.
- For commercial / high-volume usage, reach out via the Actor profile.

# Actor input Schema

## `start_urls` (type: `array`):

List one or more **X / Twitter profile URLs** (e.g. `https://x.com/peckshield`) or plain **usernames** (e.g. `peckshield`). Supports bulk input — paste many URLs at once.

## `max_items` (type: `integer`):

Stop after collecting this many tweets **per profile**. Set high for full sweeps, low for quick samples.

## `since_date` (type: `string`):

Lower bound for tweet time. Use the picker in **absolute** mode for a calendar date (`YYYY-MM-DD`), or **relative** mode for a rolling window (e.g. `7 days`, `2 weeks`, `3 months`, `1 year`). Only tweets **on or after** the resolved date are returned. Leave empty for no lower bound.

## `auth_token` (type: `string`):

Your X.com `auth_token` cookie. Open x.com in your browser → DevTools → Application → Cookies → copy `auth_token`. **Stored securely.**

## `ct0` (type: `string`):

Your X.com `ct0` cookie (CSRF token), from the same Cookies panel as `auth_token`. **Stored securely.**

## `proxy_configuration` (type: `object`):

Expand this section to change Apify Proxy options (groups, country). The built-in escalation still applies when direct requests fail.

## Actor input object example

```json
{
  "start_urls": [
    "https://x.com/peckshield"
  ],
  "max_items": 10,
  "since_date": "",
  "proxy_configuration": {
    "useApifyProxy": false
  }
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "start_urls": [
        "https://x.com/peckshield"
    ],
    "proxy_configuration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("scrapeengine/twitter-tweets-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "start_urls": ["https://x.com/peckshield"],
    "proxy_configuration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("scrapeengine/twitter-tweets-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "start_urls": [
    "https://x.com/peckshield"
  ],
  "proxy_configuration": {
    "useApifyProxy": false
  }
}' |
apify call scrapeengine/twitter-tweets-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=scrapeengine/twitter-tweets-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/uupR2gPWwKF7vauNc/builds/Yo9fApGjwwCEZzddQ/openapi.json