# Instagram Highlights Scraper & Cover Archive (`api-empire/instagram-highlights-scraper`) Actor Instagram Highlights Scraper captures all Highlight stories from any public profile. Extract media URLs, captions, timestamps, views, and metadata. Ideal for research, content auditing, brand analysis, and workflows that need organized Instagram Highlights data. - **URL**: https://apify.com/api-empire/instagram-highlights-scraper.md - **Developed by:** [API Empire](https://apify.com/api-empire) (community) - **Categories:** Social media, Automation, Agents - **Stats:** 13 total users, 0 monthly users, 100.0% runs succeeded, 1 bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ### Instagram Scraper — Extract Highlights, Covers & Rankings Instagram Highlights Scraper & Cover Archive extracts every public story highlight from an Instagram profile — its title, ID, and cover image — then selects the highest-resolution version of that cover and archives a permanent copy so the link never expires. Each run can also add a per-profile summary row and a cross-profile leaderboard ranking every target by highlight count. Every response is typed, normalized JSON — no HTML, no selectors, no parsing required. After running it, you'll have a durable archive of highlight titles and covers that survives Instagram's own expiring CDN links. ### What is Instagram Highlights Scraper & Cover Archive? Instagram Highlights Scraper & Cover Archive is an Apify Actor that reads a public Instagram profile's story highlights tray and returns each highlight's title, ID, and cover image, upgraded to the highest resolution Instagram exposes and archived to a permanent URL. No Instagram account, login, or session cookie is required — it reads logged-out, public data only. Alongside the raw highlight rows, it can produce a per-profile analytics summary and a cross-profile leaderboard in the same run. - Scrape story highlight titles, IDs, and cover images from any public profile - Resolve profile URLs, usernames, direct highlight links, or post/reel/IGTV shortcodes to the owning profile - Upgrade covers to full resolution and archive them to a permanent key-value store URL - Generate an optional per-profile summary and cross-profile leaderboard - Export results as JSON or CSV directly from the Actor's dataset ### What data does Instagram Highlights Scraper & Cover Archive collect? The Actor returns three distinct row types in one dataset: raw highlight rows, an optional per-profile summary row, and an optional cross-profile leaderboard row. | Data Type | Key Fields | JSON Field Names | | --- | --- | --- | | Story Highlights | title, highlight ID, cover image, full-resolution cover | `title`, `id`, `cover_media`, `cover_media_full_res` | | Archived Cover Images | permanent archive URL, CDN expiry timestamp | `cover_media_archive_url`, `cover_media_expires_at` | | Per-Profile Summary | highlight count, combined titles, first/last highlight ID, success rate | `profile_highlight_count`, `profile_titles`, `profile_success_rate` | | Cross-Profile Leaderboard | rank, highlight count, success rate | `leaderboard_rank`, `profile_highlight_count`, `profile_success_rate` | #### Need more Instagram data? If you also need engagement numbers rather than highlight covers, **Instagram Profile Scraper: Per-Post Likes & Comments** returns full profile data plus a sampled engagement rate from recent posts. If you're researching content by topic instead of by profile, **Instagram Hashtag Scraper: Top Post Data** returns ranked Reels for any hashtag with audio metadata and engagement ranking. Both live in this same account and slot in alongside this Actor for a broader Instagram data pipeline. ### Why not build this yourself? Instagram's Graph API only exposes a Business or Creator account's own connected content — it has no endpoint for reading an arbitrary public profile's story highlights, which is this Actor's entire purpose. Building a logged-out scraper yourself means reverse-engineering Instagram's private GraphQL contract (relay document IDs, CSRF-cookie handling, defensive JSON parsing around its `for (;;);` anti-hijacking prefix), then maintaining it as Instagram changes that contract, plus building and paying for your own proxy escalation when requests get blocked. This Actor already does that work: it ships the resolved GraphQL query, a three-tier proxy fallback ladder, and a stable output schema, so you get the highlight data without owning the scraping infrastructure behind it. ### Why do developers and teams scrape Instagram? #### For brand and social media teams Brand and agency teams use this Actor to archive their own or a client's highlight covers before Instagram's CDN link expires, keeping a permanent visual record of campaign highlight reels for reporting or asset reuse. Because `cover_media_archive_url` never rots, a marketing team can build a rolling visual archive of a profile's "Story Highlights" bar — titles and cover art — without re-scraping every time a report is due, and use the leaderboard row to compare highlight coverage across a roster of managed or competitor accounts in one run. #### For competitive content analysis Analysts tracking competitor accounts use `enableLeaderboard` to rank a batch of profiles by highlight count in a single run, surfacing which accounts maintain the most highlight categories without opening each profile manually. Combined with `profile_titles` from the per-profile summary row, a team can see at a glance which topics (e.g., "FAQ", "Reviews", "Behind the Scenes") a competitor organizes their highlights around, then feed that structured list into a spreadsheet or BI tool for recurring competitive reporting. #### For AI engineers and agent builders Agent builders use the Actor's typed JSON output — `title`, `cover_media_full_res`, and `cover_media_archive_url` — as a stable, non-expiring image source for RAG pipelines or vision-model inputs, avoiding the broken-link failures that come from indexing Instagram's own time-limited CDN URLs. Because the archived URL is permanent, an agent can re-fetch the same image weeks later without re-running the scrape, which matters for any pipeline that stores image references rather than image bytes. #### For developers building data products Developers building monitoring dashboards or digital asset management tools use the per-profile summary and leaderboard rows as ready-made aggregates instead of computing them client-side from raw highlight rows. Since those two row types are pushed without a charged event, a product built on top of this Actor gets free, structured analytics rows bundled with every batch of billed highlight rows, simplifying the downstream schema needed to power a dashboard. ### How to scrape Instagram (step by step) 1. Open **Instagram Highlights Scraper & Cover Archive** on its Apify Store listing. 2. Provide one or more targets in `highlightCoverTargets` — a profile URL, username, `@handle`, direct highlight link, or post/reel/IGTV shortcode. 3. Configure options: leave `maxHighlightsPerProfile` at `0` for the full tray, or set a cap; toggle `archiveCoverImages`, `enableProfileSummary`, and `enableLeaderboard` as needed. 4. Start the run from the Apify Console or via the API. 5. Download results as JSON or CSV from the run's dataset, or read them via the API. #### What to do when Instagram changes its structure This Actor is maintained, and its output schema is kept stable — field names and types don't change on your end even if Instagram's internal endpoints do. No specific turnaround time is promised for any given structural change, but integrations built against the documented fields keep working without a rewrite on your side. ### ⬇️ Input Every field is optional — the Actor runs with defaults if you supply nothing but `highlightCoverTargets`. | Parameter | Required | Type | Description | Example Value | | --- | --- | --- | --- | --- | | `highlightCoverTargets` | No | array | Profile URL, username (`username` or `@username`), direct highlight link (`https://www.instagram.com/stories/highlights//`), or post/reel/IGTV shortcode — each is resolved to its owning profile via Instagram's public embed pages. Also accepts the base actor's `startUrls` key. | `["https://www.instagram.com/mrbeast/"]` | | `maxHighlightsPerProfile` | No | integer (default `0`, min `0`) | Cap on highlights fetched per profile across all pages of the tray. `0` = unlimited (fetch every page until Instagram reports no more). | `0` | | `archiveCoverImages` | No | boolean | Downloads the highest-resolution cover (max width × height across all `image_versions2` candidates) and stores a permanent copy in the run's key-value store. | `true` | | `enableProfileSummary` | No | boolean (default `true`) | Pushes one extra, uncharged summary row per profile — highlight count, combined titles, first/last highlight ID, success rate. | `true` | | `enableLeaderboard` | No | boolean (default `true`) | Pushes one extra, uncharged leaderboard row per profile at the end of the run, ranking every profile by highlight count. | `true` | | `proxyLadderConfig` | No | object (proxy editor) | Proxy configuration. On a platform block, the Actor falls back from your selection to datacenter, then residential, genuinely carrying your `apifyProxyGroups`/`apifyProxyCountry` through both fallback tiers. Also accepts the base actor's `proxyConfiguration` key. | `{ "useApifyProxy": true }` | ```json { "highlightCoverTargets": [ "https://www.instagram.com/mrbeast/", "nasa", "https://www.instagram.com/stories/highlights/17895948565602293/" ], "maxHighlightsPerProfile": 0, "archiveCoverImages": true, "enableProfileSummary": true, "enableLeaderboard": true, "proxyLadderConfig": { "useApifyProxy": true } } ``` Since no parameter is marked required, the most common mistake is starting a run with an empty `highlightCoverTargets` list — the Actor logs an error and exits immediately with no dataset rows, so always supply at least one target. ### ⬆️ Output Results are typed, normalized JSON, with a consistent 21-field schema shared across all three row types, exportable as JSON, CSV, Excel, or the other formats Apify's dataset export supports. #### Scraped Story Highlight ```json { "input_url": "https://www.instagram.com/mrbeast/", "username": "mrbeast", "user_id": "3223361110", "type": "instagram_highlights", "id": "17895948565602293", "title": "FAQ", "cover_media": "https://scontent.cdninstagram.com/v/t51.2885-19/example_150x150.jpg?oe=66B1F2A0", "cover_media_full_res": "https://scontent.cdninstagram.com/v/t51.2885-19/example_1170x2080.jpg?oe=66B1F2A0", "cover_media_expires_at": "2026-08-01T12:00:00+00:00", "cover_media_archive_url": "https://api.apify.com/v2/key-value-stores/abc123/records/cover-17895948565602293.jpg", "success": true, "error": null, "isSummary": false, "profile_highlight_count": null, "profile_titles": null, "profile_first_highlight_id": null, "profile_last_highlight_id": null, "profile_success_rate": null, "isLeaderboard": false, "leaderboard_rank": null, "timestamp": "2026-07-25T14:02:11.000000+00:00" } ``` Not every highlight guarantees a resolution upgrade: `cover_media_full_res` is only raised above the tray thumbnail when Instagram's reels endpoint returns a matching asset for that exact cover photo. When no match is found, the Actor honestly falls back to the original tray cover rather than returning an unrelated, higher-resolution image. #### Per-Profile Summary Row ```json { "input_url": "https://www.instagram.com/mrbeast/", "username": "mrbeast", "user_id": "3223361110", "type": "instagram_highlights_profile_summary", "id": null, "title": null, "cover_media": null, "cover_media_full_res": null, "cover_media_expires_at": null, "cover_media_archive_url": null, "success": true, "error": null, "isSummary": true, "profile_highlight_count": 14, "profile_titles": "FAQ | Behind the Scenes | Merch | Q&A", "profile_first_highlight_id": "17895948565602293", "profile_last_highlight_id": "17896001234567890", "profile_success_rate": 1.0, "isLeaderboard": false, "leaderboard_rank": null, "timestamp": "2026-07-25T14:02:15.000000+00:00" } ``` #### Cross-Profile Leaderboard Entry ```json { "input_url": "https://www.instagram.com/mrbeast/", "username": "mrbeast", "user_id": "3223361110", "type": "instagram_highlights_leaderboard_entry", "id": null, "title": null, "cover_media": null, "cover_media_full_res": null, "cover_media_expires_at": null, "cover_media_archive_url": null, "success": true, "error": null, "isSummary": false, "profile_highlight_count": 14, "profile_titles": null, "profile_first_highlight_id": null, "profile_last_highlight_id": null, "profile_success_rate": 1.0, "isLeaderboard": true, "leaderboard_rank": 1, "timestamp": "2026-07-25T14:05:40.000000+00:00" } ``` ### How many results can you scrape with Instagram Highlights Scraper & Cover Archive? There's no hard cap on the number of profiles you can list in `highlightCoverTargets`, and `maxHighlightsPerProfile` defaults to `0` (unlimited) — the Actor pages through the highlights tray via its `has_next_page`/`end_cursor` cursor until Instagram reports no more results. As a runaway-loop safety net (not a normal-use limit), pagination for a single profile stops after 50 GraphQL pages, a ceiling ordinary highlight trays fall well under. If a profile has no public highlights, the Actor returns a single definitive "no highlights found" row rather than exhausting retries. No benchmark run time is published, since actual duration depends on the number of targets, tray size, and whether proxy fallback is triggered. ### Integrate Instagram Highlights Scraper & Cover Archive and automate your workflow Instagram Highlights Scraper & Cover Archive works with any language or tool that can send an HTTP request, since it runs as a standard Apify Actor with a REST API. #### REST API integration ```python from apify_client import ApifyClient client = ApifyClient("YOUR_APIFY_API_TOKEN") run = client.actor("API-Empire/instagram-highlights-scraper-cover-archive").call( run_input={ "highlightCoverTargets": ["https://www.instagram.com/mrbeast/"], "archiveCoverImages": True, } ) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item["username"], item["title"], item["cover_media_archive_url"]) ``` Works in Python, Node.js, Go, Ruby, cURL, or any HTTP client — authenticate with your Apify API token as a bearer credential. #### Automation platforms (n8n, Make) In n8n, the Apify node's "Run Actor" operation can call this Actor by ID and pass `highlightCoverTargets` directly from an upstream node, then feed the resulting dataset into downstream nodes. In Make, the Apify app's actor-run module works the same way — trigger a run, wait for it to finish, then iterate the dataset items to route archived cover URLs into a spreadsheet, CMS, or storage bucket. ### Is it legal to scrape Instagram? Scraping publicly available data is generally lawful; this Actor only returns data from public Instagram profiles — no login, private-account access, or authentication bypass is involved. Because the output includes personal data such as usernames, user IDs, and profile-linked highlight content, GDPR/CCPA-style considerations around lawful basis, storage limitation, and purpose apply if you store or reuse it, particularly for EU or California-connected individuals. Consult legal counsel for commercial use cases involving bulk personal data. ### Frequently asked questions #### Does Instagram Highlights Scraper & Cover Archive work without an Instagram account? Yes. It reads public profile and highlight data logged-out — no Instagram account, login, or session cookie is required for any input parameter. #### How often is the scraped data updated? Every run fetches live from Instagram at the time it executes — nothing is served from a cache. Re-run the Actor to get current highlight data. #### What happens if a profile has no public highlights? The Actor returns one row with `success: false` and an explanatory `error` message ("No highlights found for this profile.") instead of retrying indefinitely or fabricating data — Instagram's response was accepted, it just contained an empty tray. #### Can I scrape private Instagram accounts' highlights? No. Only public profiles are supported. A private profile's page won't expose the tokens the Actor needs, and the run reports that profile as unresolved rather than bypassing the privacy setting. #### Are the per-profile summary and leaderboard rows billed the same as highlight rows? No. Highlight rows (and per-profile error rows) are billed under the `row_result` charged event; the summary and leaderboard rows are pushed without a charged event name, so they're free. To isolate only billed rows when processing output, filter out any item where `isSummary` or `isLeaderboard` is `true`. #### Does Instagram Highlights Scraper & Cover Archive work for AI agent workflows and LLM pipelines? Yes. It's callable as a standard HTTP endpoint by any agent framework, and every response is typed JSON — no HTML or selectors to parse before passing results into an LLM context window or agent tool call. #### How does Instagram Highlights Scraper & Cover Archive handle Instagram's anti-bot system? It escalates through three tiers on a block (HTTP 429/403): no proxy first, then a datacenter proxy, then a residential proxy that retries up to three times and stays sticky for the rest of the run. Requests also use exponential backoff between retries and defensively parse Instagram's GraphQL response format, including its `for (;;);` anti-hijacking prefix. #### Does Instagram Highlights Scraper & Cover Archive return data in a format LLMs can use directly? Yes. Output is typed, normalized JSON with stable field names across all three row types — pass it directly to an LLM, index it into a vector store, or route it through an agent tool without a parsing step. #### Can I use it without managing proxies? Yes. The Actor defaults to no proxy and automatically escalates to Apify's datacenter and residential proxy tiers on a block; you only need to configure `proxyLadderConfig` if you want to steer which proxy groups or country are used. #### What happens when Instagram changes its structure or blocks the scraper? The Actor is maintained, and its output schema stays stable — field names and types don't change on your end. No specific turnaround time is promised for any given fix. ### Your feedback Found a bug or missing a field? We want to know. Reach out through the Actor's Issues tab on Apify Store or the API-Empire support channel, and we'll take a look — reports like this are how the Actor stays maintained as Instagram's own structure evolves. # Actor input Schema ## `highlightCoverTargets` (type: `array`): 🔗 One or more targets — each line is one input. Accepted: full profile URL (https://www.instagram.com/username), username (username or @username), a direct highlight link (https://www.instagram.com/stories/highlights//), or a post/reel/IGTV shortcode — each shortcode/highlight-link is genuinely resolved to its owning profile via Instagram's public embed pages. Every public highlight is fetched with title, ID, cover image, and a full-resolution archived copy. (Also accepts the base actor's `startUrls` key.) ## `maxHighlightsPerProfile` (type: `integer`): Cap on how many highlights to fetch per profile, across ALL pages of the tray. 0 = unlimited (fetch every page until Instagram reports no more highlights). ## `archiveCoverImages` (type: `boolean`): When ON, downloads the highest-resolution cover image (max width×height across all image\_versions2 candidates) and stores a permanent copy in this run's key-value store, so cover\_media\_archive\_url never rots even after Instagram's CDN link (oe= param) expires. ## `enableProfileSummary` (type: `boolean`): Push one extra, uncharged summary row per profile — highlight count, combined titles, first/last highlight ID, and success rate — right after that profile's highlight rows. ## `enableLeaderboard` (type: `boolean`): Push one extra, uncharged leaderboard row per profile at the end of the run, ranking every profile in this run by highlight count (highest first). ## `proxyLadderConfig` (type: `object`): Choose which proxies to use. Strategy: start with your selection. On HTTP 429/403 (platform block) the actor fails fast and falls back — first to datacenter, then to residential — genuinely carrying your chosen apifyProxyGroups/apifyProxyCountry through to BOTH fallback tiers instead of discarding them. Residential retries up to 3 times with a fresh proxy URL, then stays sticky for the rest of the run. (Also accepts the base actor's `proxyConfiguration` key.) ## Actor input object example ```json { "highlightCoverTargets": [ "https://www.instagram.com/mrbeast/" ], "maxHighlightsPerProfile": 0, "archiveCoverImages": true, "enableProfileSummary": true, "enableLeaderboard": true, "proxyLadderConfig": { "useApifyProxy": true } } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "highlightCoverTargets": [ "https://www.instagram.com/mrbeast/" ], "archiveCoverImages": true, "proxyLadderConfig": { "useApifyProxy": true } }; // Run the Actor and wait for it to finish const run = await client.actor("api-empire/instagram-highlights-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "highlightCoverTargets": ["https://www.instagram.com/mrbeast/"], "archiveCoverImages": True, "proxyLadderConfig": { "useApifyProxy": True }, } # Run the Actor and wait for it to finish run = client.actor("api-empire/instagram-highlights-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "highlightCoverTargets": [ "https://www.instagram.com/mrbeast/" ], "archiveCoverImages": true, "proxyLadderConfig": { "useApifyProxy": true } }' | apify call api-empire/instagram-highlights-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=api-empire/instagram-highlights-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/acts/8zUbUt8Pidxj3CLvb/builds/YwKFl8JBUw7RCqkXx/openapi.json