pagination Scenario 3 — Infinite Scroll Feed

Written by RTILA X Engineering Team

Intermediate ~6 min

Prerequisites: pagination/1 pagination/2

What You'll Learn

Use infinite_scroll command for lazy loading

Real-World Use Case

Scraping social media feeds

LIVE DEMO

Built a resilient scraping queue today.

techguru42

XPath selectors are underrated.

dataminer_sam

Playwright makes multi-tab automation easy.

scraper_queen

Remember to wait for network idle.

automation_alice

CSS attribute selectors solved my data extraction.

selector_sam

Variables and triggers are the secret sauce.

css_crusader

EXPECTED OUTPUT

RTILA X PROJECT JSON

    {
  "name": "Infinite_Scroll_Feed",
  "settings": {
    "urls": [
      "https://learn.rtila.com/scenarios/pagination/3"
    ]
  },
  "datasets": {
    "feed_items": {
      "item_selector": "css=.feed-item",
      "properties": [
        {
          "name": "text",
          "type": "text",
          "selector": "css=.feed-text"
        },
        {
          "name": "author",
          "type": "text",
          "selector": "css=.feed-author"
        }
      ]
    }
  },
  "commands": [
    {
      "command": "wait_for_selector",
      "params": {
        "selector": "css=.feed-item"
      }
    },
    {
      "command": "infinite_scroll",
      "params": {
        "selector": "css=.feed-container",
        "max_scrolls": 10,
        "scroll_delay": 1000,
        "do": [
          {
            "command": "extract_data",
            "params": {
              "dataset": "feed_items"
            }
          }
        ]
      }
    }
  ]
}

The infinite_scroll command repeatedly scrolls a container until no new items appear. In this scenario, the feed loads new batches after a short delay, and each scroll triggers extraction of the currently visible items.

Did you complete this scenario?

Ready to run this automation?

Copy the project JSON above, open RTILA X, create a new project, and paste it.

Download RTILA X Free → View setup guide →

This scenario covers 2 RTILA X commands and is referenced in 2 learning articles.

Continue Learning

Was this helpful?