# RSS Feed Aggregator & Article Extractor (`darknezz/rss-feed-aggregator`) Actor

Aggregate RSS/Atom feeds and extract full article content. Multi-feed ingestion, deduplication, keyword filtering, rich metadata. Returns clean JSON with full-text extraction. For news monitoring, AI training, and curation.

- **URL**: https://apify.com/darknezz/rss-feed-aggregator.md
- **Developed by:** [Oaida Adrian](https://apify.com/darknezz) (community)
- **Categories:** News, AI, Automation
- **Stats:** 2 total users, 0 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $10.00 / 1,000 extract events

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## RSS Feed Aggregator — Full Article Content Extraction

Aggregate any number of RSS/Atom feeds in one run and get the **full article text**, not just the feed summary. Every item is fetched, extracted with [trafilatura](https://github.com/adbar/trafilatura) (high-precision main-content extraction), deduplicated, and written as clean, structured JSON.

Ideal for **news monitoring, content curation, AI/RAG pipelines, media intelligence, and SEO research** — one scheduled run replaces a self-hosted feed reader plus a scraping stack.

### What it does

1. Parses every RSS/Atom feed you give it (news sites, blogs, Substack, Google News queries, podcast feeds — anything with a valid feed).
2. Optionally visits each article URL and extracts the **complete readable text**, author, date, and site name — stripping navigation, ads, and boilerplate.
3. Deduplicates across feeds by URL or title, so overlapping sources don't produce duplicate rows.
4. Filters by publish date (ISO 8601) and/or feed categories.
5. Pushes one structured item per article to the dataset, ready for export as JSON, CSV, or Excel.

### Input

```json
{
  "feedUrls": [
    { "url": "https://hnrss.org/frontpage" },
    { "url": "https://techcrunch.com/feed/" }
  ],
  "maxArticles": 100,
  "extractFullContent": true,
  "deduplicateBy": "link",
  "dateFilter": "2026-07-01"
}
```

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `feedUrls` | array | *required* | RSS/Atom feed URLs to aggregate |
| `maxArticles` | integer | `100` | Maximum articles per run (0 = unlimited) |
| `extractFullContent` | boolean | `true` | Visit each article and extract the full text |
| `deduplicateBy` | string | `"link"` | `link`, `title`, or `none` |
| `dateFilter` | string | `""` | Only keep articles published after this ISO 8601 date |
| `includeCategories` | string | `""` | Comma-separated category filter (matches feed categories) |
| `proxyConfiguration` | object | Apify proxy | Proxy settings used for article fetching |

### Output

One dataset item per article:

```json
{
  "title": "Show HN: I built a local-first RSS reader",
  "link": "https://example.com/local-first-rss",
  "guid": "https://example.com/local-first-rss",
  "pubDate": "2026-07-16T08:30:00Z",
  "source": "Hacker News: Front Page",
  "sourceUrl": "https://hnrss.org/frontpage",
  "author": "jane_doe",
  "summary": "A local-first RSS reader with offline sync…",
  "fullText": "For the last six months I've been building…",
  "wordCount": 1240,
  "extractionMethod": "trafilatura",
  "categories": ["programming"],
  "tags": ["rss", "local-first"],
  "enclosures": []
}
```

All fields are null-tolerant: if an article page can't be fetched you still get the title, link, date, and RSS summary.

### Use cases

- **News monitoring** — merge dozens of sources into one deduplicated stream; schedule hourly and pipe new items to Slack, email, or a webhook with an Apify integration.
- **AI training / RAG pipelines** — `fullText` + `wordCount` give you clean, boilerplate-free text chunks with provenance metadata.
- **Content curation** — feed newsletters or "best of" pages from niche blogs without opening a feed reader.
- **Media intelligence** — track competitor blogs and industry publications; filter with `dateFilter` to process only what's new since the last run.
- **SEO research** — monitor publications in your niche for content gaps and backlink opportunities.

### Run it on a schedule

Create an Apify **Schedule** (e.g. every hour) pointing at this actor with your feed list, and set `dateFilter` to roll forward — each run then returns only fresh articles. Combine with the dataset API to pull new rows into your own system:

```bash
curl "https://api.apify.com/v2/acts/darknezz~rss-feed-aggregator/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -X POST -H "Content-Type: application/json" \
  -d '{"feedUrls": [{"url": "https://hnrss.org/frontpage"}], "maxArticles": 50}'
```

### Pricing

Pay-per-event: **$0.01 per article extracted**, plus Apify's standard per-run and per-dataset-item micro-fees. 100 fully extracted articles ≈ $1. No subscription, no monthly minimum.

### FAQ

**Does it work with Atom feeds / podcast feeds?** Yes — anything feedparser understands: RSS 0.9x/1.0/2.0, Atom, RDF. Podcast enclosures land in `enclosures`.

**What if a site blocks full-text extraction?** You still get the feed-level data (title, link, summary, date). The item's `fullText` is null and `extractionMethod` reflects the fallback — the run does not fail.

**Can I aggregate Google News search results?** Yes — use a Google News RSS URL (`https://news.google.com/rss/search?q=your+query`) as a feed.

**How is this different from a feed reader?** A reader shows you summaries in an app. This actor gives you the complete article text as structured data you can pipe into a database, LLM, or BI tool.

**How many feeds can I pass?** No hard limit — runs process feeds sequentially; use `maxArticles` to cap cost on large lists.

# Actor input Schema

## `feedUrls` (type: `array`):

List of RSS/Atom feed URLs to aggregate.

## `maxArticles` (type: `integer`):

Maximum number of articles to extract per run. 0 = no limit.

## `extractFullContent` (type: `boolean`):

If enabled, fetches and extracts the full article text from each article URL.

## `deduplicateBy` (type: `string`):

Field to use for deduplication.

## `dateFilter` (type: `string`):

Only include articles published after this date (ISO 8601, e.g. 2025-01-01). Leave empty for no filter.

## `includeCategories` (type: `string`):

Comma-separated list of categories to include. Leave empty for all.

## `proxyConfiguration` (type: `object`):

Use Apify proxy for article content extraction.

## Actor input object example

```json
{
  "feedUrls": [
    {
      "url": "https://hnrss.org/frontpage"
    }
  ],
  "maxArticles": 100,
  "extractFullContent": true,
  "deduplicateBy": "link",
  "dateFilter": "",
  "includeCategories": "",
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "feedUrls": [
        {
            "url": "https://hnrss.org/frontpage"
        }
    ],
    "proxyConfiguration": {
        "useApifyProxy": true
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("darknezz/rss-feed-aggregator").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "feedUrls": [{ "url": "https://hnrss.org/frontpage" }],
    "proxyConfiguration": { "useApifyProxy": True },
}

# Run the Actor and wait for it to finish
run = client.actor("darknezz/rss-feed-aggregator").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "feedUrls": [
    {
      "url": "https://hnrss.org/frontpage"
    }
  ],
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}' |
apify call darknezz/rss-feed-aggregator --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=darknezz/rss-feed-aggregator",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/vpqdvmWUl5ge26kHF/builds/FTdXuHZzFPorKjd6U/openapi.json