Skip to main content

pagination Scenario 3 — Infinite Scroll Feed

Written by RTILA X Engineering Team

Intermediate ~6 min
Prerequisites: pagination/1 pagination/2

What You'll Learn

Use infinite_scroll command for lazy loading

Real-World Use Case

Scraping social media feeds
LIVE DEMO

Infinite Scroll Feed

A lazy-loading feed that appends new batches as you scroll.

Built a resilient scraping queue today.

techguru42

XPath selectors are underrated.

dataminer_sam

Playwright makes multi-tab automation easy.

scraper_queen

Remember to wait for network idle.

automation_alice

CSS attribute selectors solved my data extraction.

selector_sam

Variables and triggers are the secret sauce.

css_crusader
EXPECTED OUTPUT

Feed Items

text author
Built a resilient scraping queue today. techguru42
XPath selectors are underrated. dataminer_sam
Playwright makes multi-tab automation easy. scraper_queen
Remember to wait for network idle. automation_alice
CSS attribute selectors solved my data extraction. selector_sam
Variables and triggers are the secret sauce. css_crusader
Check return codes before parsing. json_jess
Stealth mode reduced my block rate dramatically. xpath_xavier
Using datasets keeps extraction organized. dom_dave
HTTP requests can bypass heavy DOM work. puppeteer_paul
Writing tests for scrapers saves hours. playwright_pam
Always verify your selectors against real markup. cypress_cy
Pagination loops need careful index tracking. selenium_sid
Regex transforms are worth learning. api_amy
Avoid hardcoding timeouts when possible. regex_ray
Dynamic content needs explicit waits. variable_vic
Loop pagination with a while command. loop_linda
Post-processing transforms make data clean. trigger_tom
Command chains can model complex workflows. command_carl
Store temporary values in variables. dataset_dee
Element properties often hold hidden data. techguru42
Attribute extraction is faster than text extraction. dataminer_sam
Use scroll commands to trigger lazy loading. scraper_queen
Screenshot commands help debug selectors. automation_alice
Crawl links inside a loop for category pages. selector_sam
Keep your project JSON readable. css_crusader
Respect robots.txt during automation tests. json_jess
Try/catch blocks prevent silent failures. xpath_xavier
Small delays make automations more human. dom_dave
Every demo here is deterministic. puppeteer_paul
RTILA X PROJECT JSON

RTILA X Automation Project

infinite-scroll-feed.json 
{
  "name": "Infinite_Scroll_Feed",
  "settings": {
    "urls": [
      "https://learn.rtila.com/scenarios/pagination/3"
    ]
  },
  "datasets": {
    "feed_items": {
      "item_selector": "css=.feed-item",
      "properties": [
        {
          "name": "text",
          "type": "text",
          "selector": "css=.feed-text"
        },
        {
          "name": "author",
          "type": "text",
          "selector": "css=.feed-author"
        }
      ]
    }
  },
  "commands": [
    {
      "command": "infinite_scroll",
      "params": {
        "selector": "css=.feed-container",
        "max_scrolls": 10,
        "scroll_delay": 1000,
        "do": [
          {
            "command": "extract_data",
            "params": {
              "dataset": "feed_items"
            }
          }
        ]
      }
    }
  ]
}

The infinite_scroll command repeatedly scrolls a container until no new items appear. In this scenario, the feed loads new batches after a short delay, and each scroll triggers extraction of the currently visible items.

Did you complete this scenario?

Ready to run this automation?

Copy the project JSON above, open RTILA X, create a new project, and paste it.

Download RTILA X Free → View setup guide →

This scenario covers 2 RTILA X commands and is referenced in 2 learning articles.

Continue Learning

Was this helpful?