Reddit Posts Search Scraper

Apify actor that searches Reddit posts by keyword and exports them as structured data. It scrapes the classic old.reddit.com search pages over plain HTTP — no browser, no login, no cookies — which keeps runs fast and cheap (256 MB of memory is enough). (Reddit hard-blocks its public .json listing endpoints for unauthenticated clients, but the old.reddit HTML search is served normally and its markup has been stable for a decade.)

Features

• One keyword (query) or many (queries) — each searched separately, results combined and de-duplicated by post.
• Sort by new (chronological), relevance, hot, top or comments.
• Reddit's own time window (timeFilter: hour/day/week/month/year/all) plus exact timeSince / timeUntil date bounds enforced actor-side.
• Optional restriction to a single subreddit.
• Cursor-based auto-pagination up to Reddit's ~1000-results-per-listing cap.
• Results are pushed page by page, so an abort or timeout keeps everything collected so far.
• Automatic block recovery: on HTTP 403/429 the actor retries with a fresh proxy IP.
• When a run ends with 0 results, the last page fetched is saved as DEBUG_HTML in the run's key-value store.

Input

{
    "queries": ["econet", "delta corporation"],
    "sort": "new",
    "numberOfPosts": 100,
    "timeSince": "2026-01-01",
    "subreddit": "zimbabwe",
    "proxyConfiguration": {
        "useApifyProxy": true,
        "apifyProxyGroups": ["RESIDENTIAL"],
        "apifyProxyCountry": "US"
    }
}

numberOfPosts applies per keyword. Reddit search operators (subreddit:, author:, self:yes, OR, quoted phrases, ...) typed into a query are passed through as-is.

Reddit blocks most datacenter IPs on its public JSON API, so keep the RESIDENTIAL proxy default.

Output

One dataset item per post:

{
    "id": "1abcd2",
    "name": "t3_1abcd2",
    "url": "https://www.reddit.com/r/zimbabwe/comments/1abcd2/econet_results/",
    "title": "Econet posts strong half-year results",
    "text": "Rendered selftext of the post (empty for link posts)...",
    "author": "some_user",
    "authorProfileUrl": "https://www.reddit.com/user/some_user",
    "subreddit": "zimbabwe",
    "subredditUrl": "https://www.reddit.com/r/zimbabwe",
    "score": 42,
    "numComments": 17,
    "linkFlairText": "Business",
    "isSelf": true,
    "over18": false,
    "externalUrl": null,
    "domain": "self.zimbabwe",
    "thumbnail": null,
    "created_at": "2026-07-01T09:30:12.000Z",
    "createdUtc": 1782898212,
    "searchQuery": "econet",
    "sort": "new",
    "timeFilter": "all"
}

score is null while Reddit still hides the vote count of a very fresh post (typically the first hour or two).

Integration (scraping-tool)

Call it like the other outspoken_strategy/* search actors:

const res = await this.scrapingService.scrape({
    url: `https://www.reddit.com/search/?q=${encodeURIComponent(keyword)}`,
    resultsLimit: requestCount,
    actor: 'outspoken_strategy/reddit-posts-search-scraper',
    timeout,
    additionalInput: {
        query: keyword,
        numberOfPosts: requestCount,
        sort: 'new',
        proxyConfiguration: { useApifyProxy: true, apifyProxyGroups: ['RESIDENTIAL'], apifyProxyCountry: 'US' }
    },
    scrapeType: 'reddit-keyword-search'
});

Normalization hints: url and id are ready to use; text holds the selftext only, so index title + text for relevance matching; likesCount → score, commentsCount → numComments, createdTime → created_at.

Local development

npm install
echo '{ "query": "artificial intelligence", "numberOfPosts": 25, "proxyConfiguration": { "useApifyProxy": false } }' > storage/key_value_stores/default/INPUT.json
npm start

Deploy with apify push.