# Google News Scraper (`lhotanova/google-news-scraper`) Actor Gets featured articles from Google News with title, link, source, publication date and image. - **URL**: https://apify.com/lhotanova/google-news-scraper.md - **Developed by:** [Kristýna Lhoťanová](https://apify.com/lhotanova) (community) - **Categories:** News - **Stats:** 3,139 total users, 44 monthly users, 99.9% runs succeeded, 79 bookmarks - **User rating**: 4.59 out of 5 stars ## Pricing $20.00/month + usage To use this Actor, you pay a monthly rental fee to the developer. The rent is subtracted from your prepaid usage every month after the free trial period.You also pay for the Apify platform usage, which gets cheaper the higher Apify subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#rental-actors ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Google News Scraper ### What data does Google News Scraper collect? Google News Scraper allows you to extract news metadata such as title, link, source, publication datetime and image. ### Query-based search The Actor provides a simple interface where you can define your search query, language, region, and date range. Unlike manually browsing Google News, which limits you to approximately 100 results per search, Google News Scraper allows you to retrieve significantly more results without restrictions. To bypass the usual result limits, the scraper can automatically filter news articles by date. If you set `maxItems` to a value greater than `100`, the scraper will fetch news articles day by day until it reaches the specified limit. For more precise control over the date range, you can use the `dateFrom` and `dateTo` fields in the `YYYY-MM-DD` format. Alternatively, you can specify an open-ended date range using the `openEndedDateRange` field with values such as: - `1h` – retrieves news from the past hour - `7d` – retrieves news from the past week - `1y` – retrieves news from the past year You can combine numbers with the `h`, `d`, or `y` shortcuts to customize your date range as needed. ![](https://i.imgur.com/LXQn2wS.png) #### Advanced search filters You can also use advanced search operators in your queries, such as `intitle`, `inurl`, `site`, exclude operator `-`, exact match with double-quotes `""`, `AND`, `OR` and more. Example queries with advanced operators: | Query | Explained | | ------------------------------------------------------ | ------------------------------------------------------------ | | `intitle:"AI" AND site:bbc.com` | Finds articles with `"AI"` in the title from BBC. | | `site:reuters.com "stock market" -crypto` | Finds stock market articles on Reuters, excluding crypto-related ones. | | `"Samsung Galaxy S25" AND (review OR comparison)` | Searches for reviews or comparisons of Samsung Galaxy S25. | | `site:nytimes.com intitle:"election" after:2025-01-01` | Retrieves recent NY Times articles with `"election"` keyword in the title. | | `inurl:blog OR inurl:news "climate change"` | Searches for climate change mentions in blog or news URLs. | You may already know some of the operators from Google search or Apify's [Google Search Scraper](https://apify.com/apify/google-search-scraper). For more information, see [Google Guide on Search Operators](https://www.googleguide.com/advanced_operators_reference.html). ### Topic-based search Google News Scraper also supports topic-based search. You can search for news from one of the topics predefined in `topics` input field: - `WORLD` 🌎 - `NATION` 🚩 - `BUSINESS` 🪙 - `TECHNOLOGY` 💻 - `ENTERTAINMENT` 🎸 - `SPORTS` 🏒 - `SCIENCE` 🧪 - `HEALTH` 🧑‍⚕️ ![Science Topic](https://i.imgur.com/0n2fPqw.png) #### Hashed Topics The scraper also supports hashed topics, allowing you to extract news based on specific topic IDs from Google News. Simply copy the topic ID from the Google News URL and paste it into the `topicsHashed` input field. For example, if you want to find news about your home country but published by foreign sources or in different languages, you can: 1. Click on your country in Google News. 2. Copy the topic ID from the URL. 3. Paste it into the `topicsHashed` array. 4. Set the `language` field to your preferred language and region (e.g., `DE:de`). This way, you can for example retrieve the latest news from the US but published by German sources like `zdf.de` or `spiegel.de`. Similarly, you can search for news about a non-English-speaking country and get results from international sources such as `reuters.com`, `bbc.co.uk`, `independent.co.uk`, or `variety.com`. ![US Country Topic ID](https://i.imgur.com/4JuFoWj.png) #### Topic Sections The scraper also allows you to refine your search by targeting specific sections within broader topics. For example, you can extract news from the `NHL` section under `Sports` or `Artificial Intelligence` within the `Technology` category. Imagine you need to scrape AI-related news from the section `Technology > Artificial Intelligence`, which is available at the following URL: https://news.google.com/topics/CAAqJggKIiBDQkFTRWdvSUwyMHZNRGRqTVhZU0FtVnVHZ0pWVXlnQVAB/sections/CAQiQ0NCQVNMQW9JTDIwdk1EZGpNWFlTQW1WdUdnSlZVeUlOQ0FRYUNRb0hMMjB2TUcxcmVpb0pFZ2N2YlM4d2JXdDZLQUEqKggAKiYICiIgQ0JBU0Vnb0lMMjB2TURkak1YWVNBbVZ1R2dKVlV5Z0FQAVAB?hl=en-US\&gl=US\&ceid=US%3Aen Simply copy the `{TOPIC_ID}/section/{SECTION_ID}` part of the URL and paste it into the `topicsHashed` array to target that specific section: ```json { "topicsHashed": [ "CAAqJggKIiBDQkFTRWdvSUwyMHZNRGRqTVhZU0FtVnVHZ0pWVXlnQVAB/sections/CAQiQ0NCQVNMQW9JTDIwdk1EZGpNWFlTQW1WdUdnSlZVeUlOQ0FRYUNRb0hMMjB2TUcxcmVpb0pFZ2N2YlM4d2JXdDZLQUEqKggAKiYICiIgQ0JBU0Vnb0lMMjB2TURkak1YWVNBbVZ1R2dKVlV5Z0FQAVAB" ] } ``` ![Technology Artificial Intelligence](https://i.imgur.com/OHyKI21.png) ### Example output To obtain a detailed output with decoded article links and image URLs, set the `fetchArticleDetails` input field to `true`. Extracted news are then stored in a following format: ```json [ { "title": "A win at last: Big blow to AI world in training data copyright scrap", "link": "https://www.theregister.com/2025/02/12/thomson_reuters_wins_ai_copyright/", "guid": "CBMifkFVX3lxTFByV0JDX09JaDR4bmNfX3AzWVFRbjMwdWNKcmtWQ255MEVGa2lCTHBpTE5JbFFWMG5nVFE1T2M0alVfY0N2N3ZFdFJDV3FrUjhVeDZ2dkQxQUNwb1FPcm9kMV9KYXk0ZUNnN1BDNDlZUXZtY3ktemgwWkpNU3Nxd9IBgwFBVV95cUxPc2tobXlKUmcwR0wyR0podklDS2MzMDhpQTd6N2NVbnRqYndPSVVmckV1Tk82TkNDck1JbmFuUU0yUjByTDE4VFRGVC1jTUc2M0VPOE1wX1RUZWpfOG9DamVmX21lWERuU0JGcHJyVDZQdEFDTHNNZjFQV1BQNkN0SjhaSQ", "source": "The Register", "sourceUrl": "https://www.theregister.com", "publishedAt": "2025-02-12T01:45:00.000Z", "loadedUrl": "https://www.theregister.com/2025/02/12/thomson_reuters_wins_ai_copyright/", "rssLink": "https://news.google.com/rss/articles/CBMifkFVX3lxTFByV0JDX09JaDR4bmNfX3AzWVFRbjMwdWNKcmtWQ255MEVGa2lCTHBpTE5JbFFWMG5nVFE1T2M0alVfY0N2N3ZFdFJDV3FrUjhVeDZ2dkQxQUNwb1FPcm9kMV9KYXk0ZUNnN1BDNDlZUXZtY3ktemgwWkpNU3Nxd9IBgwFBVV95cUxPc2tobXlKUmcwR0wyR0podklDS2MzMDhpQTd6N2NVbnRqYndPSVVmckV1Tk82TkNDck1JbmFuUU0yUjByTDE4VFRGVC1jTUc2M0VPOE1wX1RUZWpfOG9DamVmX21lWERuU0JGcHJyVDZQdEFDTHNNZjFQV1BQNkN0SjhaSQ?oc=5", "image": "https://regmedia.co.uk/2021/08/02/shutterstock_robot_justice.jpg" }, { "title": "Web Scraping Optimization: Tips for Faster, Smarter Scrapers", "link": "https://hackernoon.com/web-scraping-optimization-tips-for-faster-smarter-scrapers", "guid": "CBMiiAFBVV95cUxQUXh5WVZ2RkNpNG9ndjF6V3hMRHBRTGRSVnNkelpwZDY2TWJzejBSMGZrRC1rSm5DZ1BxanpoeFFGdDRjWGpZR0tOUG9FY0kyeWFXOE9MSzBobTg1ajRiZzVhSWhtbm5nSVNJVWExSDBSaEFjUUJkT1JRRDJHSDBrMU9jU2ZZN3RN", "source": "hackernoon.com", "sourceUrl": "https://hackernoon.com", "publishedAt": "2024-11-15T08:00:00.000Z", "loadedUrl": "https://hackernoon.com/web-scraping-optimization-tips-for-faster-smarter-scrapers", "rssLink": "https://news.google.com/rss/articles/CBMiiAFBVV95cUxQUXh5WVZ2RkNpNG9ndjF6V3hMRHBRTGRSVnNkelpwZDY2TWJzejBSMGZrRC1rSm5DZ1BxanpoeFFGdDRjWGpZR0tOUG9FY0kyeWFXOE9MSzBobTg1ajRiZzVhSWhtbm5nSVNJVWExSDBSaEFjUUJkT1JRRDJHSDBrMU9jU2ZZN3RN?oc=5", "image": "https://hackernoon.imgix.net/images/0FC9YtxD4fbD3T7mPipOt4HSxY42-7y034nb.png" } ] ``` > NOTE: Image URLs are retrieved directly from target article websites and may differ from those displayed on the Google News website. The scraper relies on the Google News RSS API, which does not include image links. However, using the RSS API instead of web browsing allows the scraper to efficiently gather all data through simple HTTP requests, making the process faster and more cost-effective. If you set `fetchArticleDetails` to `false`, the scraper will run significantly faster and at a lower cost, but the output will only include URLs in the RSS feed format, such as: https://news.google.com/rss/articles/CBMiiAFBVV95cUxQUXh5WVZ2RkNpNG9ndjF6V3hMRHBRTGRSVnNkelpwZDY2TWJzejBSMGZrRC1rSm5DZ1BxanpoeFFGdDRjWGpZR0tOUG9FY0kyeWFXOE9MSzBobTg1ajRiZzVhSWhtbm5nSVNJVWExSDBSaEFjUUJkT1JRRDJHSDBrMU9jU2ZZN3RN?oc=5 The output will be simplified to include RSS links only and exclude preview images: ```json [ { "title": "How I used Scrapy and Scrapy Playwright to take screenshot", "link": "https://news.google.com/rss/articles/CBMigAFBVV95cUxQRDJDNW1ONklDTlhJN3lsWlIwdTEzV050d284bnZ4YWo2TnQ3MU1XcUZkdHNZTTMxSTZ4NjkwRnh0UFloa0ZsZ0d5Qmx0NjRFd25qMXdBTmN6TVBmcHdKdXhGWnRCSDVMdjVJZlJkTlRTMjZSZXZwdGNYM28xbE9uTA?oc=5", "guid": "CBMigAFBVV95cUxQRDJDNW1ONklDTlhJN3lsWlIwdTEzV050d284bnZ4YWo2TnQ3MU1XcUZkdHNZTTMxSTZ4NjkwRnh0UFloa0ZsZ0d5Qmx0NjRFd25qMXdBTmN6TVBmcHdKdXhGWnRCSDVMdjVJZlJkTlRTMjZSZXZwdGNYM28xbE9uTA", "source": "substack.com", "sourceUrl": "https://substack.com", "publishedAt": "2024-06-09T07:00:00.000Z" }, { "title": "Beyond the Buzz: Making AI Work for Large-Scale Web Data Collection", "link": "https://news.google.com/rss/articles/CBMisgFBVV95cUxPVXVJeGNPTWI0bXVmOWdSc1B5MVpDRWVnX1R3QV8wdzRmb2VNU1hneU9wNEZrMGdOT2Fqc1Rremg1R0lmM1NBdjZJa3llOXc2bDVzN3RwNVppZnJ3Q1FTS1NsRUVxOTlwWl9QNlJYMUpPYUxKeFVRSWpCYXNaOVU2aFZ0RWNzd200cmRRQlJoUE8tcnU2bXNIQkZNOXUxRDBzTXAxSEMtMG5lLTRsdlVWZFRB?oc=5", "guid": "CBMisgFBVV95cUxPVXVJeGNPTWI0bXVmOWdSc1B5MVpDRWVnX1R3QV8wdzRmb2VNU1hneU9wNEZrMGdOT2Fqc1Rremg1R0lmM1NBdjZJa3llOXc2bDVzN3RwNVppZnJ3Q1FTS1NsRUVxOTlwWl9QNlJYMUpPYUxKeFVRSWpCYXNaOVU2aFZ0RWNzd200cmRRQlJoUE8tcnU2bXNIQkZNOXUxRDBzTXAxSEMtMG5lLTRsdlVWZFRB", "source": "GeekWire", "sourceUrl": "https://www.geekwire.com", "publishedAt": "2024-11-26T08:00:00.000Z" } ] ``` # Actor input Schema ## `query` (type: `string`): Enter search query as you would write it to Google News search bar. You can also use advanced search operators in your queries, such as 'intitle', 'inurl', 'site', exclude operator '-', exact match with double-quotes, 'AND', 'OR' and more. Example queries with advanced operators: 'intitle:samsung AND inurl:forbes', '"samsung galaxy s25" -ultra'. You may already know some of the operators from Google search or Apify's [Google Search Scraper](https://apify.com/apify/google-search-scraper). For more information, see [Google Guide on Search Operators](https://www.googleguide.com/advanced_operators_reference.html). ## `topics` (type: `array`): Select topics you want to search for using 'https://news.google.com/topics/' search. Note that your search query will not be combined with the selected topics but will be used as a standalone search. If you want to search for news only in the selected topics, leave the 'Search query' field empty. ## `topicsHashed` (type: `array`): Specify topics you want to search for using hashed topic URLs. This field is useful when you want to search for topics that are not available in the 'Topics' field. You can find hashed topic URLs in the URL of the topic page on 'https://news.google.com/topics/{TOPIC\_HASH}'. The Actor also supports hashed sections URLs in the format 'https://news.google.com/topics/{TOPIC\_HASH}/sections/{SECTION\_HASH}'. For example, if you'd like to scrape the news from 'Technology' topic, section 'Artificial intelligence', you can use the following value (extracted from the section's URL): 'CAAqJggKIiBDQkFTRWdvSUwyMHZNRGRqTVhZU0FtVnVHZ0pWVXlnQVAB/sections/CAQiQ0NCQVNMQW9JTDIwdk1EZGpNWFlTQW1WdUdnSlZVeUlOQ0FRYUNRb0hMMjB2TUcxcmVpb0pFZ2N2YlM4d2JXdDZLQUEqKggAKiYICiIgQ0JBU0Vnb0lMMjB2TURkak1YWVNBbVZ1R2dKVlV5Z0FQAVAB'. Note that your search query will not be combined with the topics but will be used as a standalone search. If you want to search for news only in the specified topics, leave the 'Search query' field empty. ## `language` (type: `string`): Select your language and region pair. ## `maxItems` (type: `integer`): Set the maximum number of items you want to scrape. If you leave this field unset, the actor will extract all news available for your search criteria. If you set this field to value less than or equal to 100, date filter won't be used with your search query. Otherwise, the scraper will search news per individual days. ## `fetchArticleDetails` (type: `boolean`): Check this option if you want to decode RSS links into actual article links and also fetch images from the article's page metadata. Please note that this option will increase the runtime of the actor because additional requests to article pages will be made. If you don't need decoded links and article images, you can leave this option unchecked and your run will be significantly faster and cheaper. ## `dateFrom` (type: `string`): Set the starting date. Only the news published on this or newer date will be scraped. If `Open-ended date range` field is set, `Date from` and `Date to` fields will be ignored. ## `dateTo` (type: `string`): Set the finish date. Only the news published on this date or before will be scraped. If `Open-ended date range` field is set, `Date from` and `Date to` fields will be ignored. ## `openEndedDateRange` (type: `string`): Set the open-ended date range such as '1h' to scrape news published in the last hour, '2d' for the last two days, or '1y' for the last year. You can use any combination of numbers and letters 'h', 'd', and 'y' to specify the time range. If you set this field, the `Date from` and `Date to` fields will be ignored. ## `proxyConfiguration` (type: `object`): Use either automatic Apify proxies or your own. ## Actor input object example ```json { "query": "Tesla", "topics": [], "topicsHashed": [], "language": "US:en", "maxItems": 100, "fetchArticleDetails": true, "proxyConfiguration": { "useApifyProxy": true } } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "query": "Tesla", "language": "US:en", "maxItems": 100, "proxyConfiguration": { "useApifyProxy": true } }; // Run the Actor and wait for it to finish const run = await client.actor("lhotanova/google-news-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "query": "Tesla", "language": "US:en", "maxItems": 100, "proxyConfiguration": { "useApifyProxy": True }, } # Run the Actor and wait for it to finish run = client.actor("lhotanova/google-news-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "query": "Tesla", "language": "US:en", "maxItems": 100, "proxyConfiguration": { "useApifyProxy": true } }' | apify call lhotanova/google-news-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=lhotanova/google-news-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/acts/eWUEW5YpCaCBAa0Zs/builds/bJc6J4Bouqh8OyZvb/openapi.json