loops Scenario 7 — Iterate Over Dataset Items for Detail Pages

Written by RTILA X Engineering Team

Advanced ~10 min

Prerequisites: loops/3 loops/4 loops/5

What You'll Learn

Perform two-phase scraping across list and detail pages

Real-World Use Case

Building a comprehensive product database

LIVE DEMO

Dataset Detail Pages

Extract the index, then loop over each detail link.

Dataset Extraction Demo

This section demonstrates iterating over extracted dataset items and visiting each detail page.

Product 101 Details

Premium wireless over-ear headphones.

$299.99 View Details

Product 102 Details

High-speed mechanical keyboard.

$89.99 View Details

Product 103 Details

Portable Bluetooth speaker.

$149.99 View Details

Product 104 Details

4K OLED monitor with HDR.

$499.99 View Details

Product 105 Details

Ergonomic wireless mouse.

$34.99 View Details

EXPECTED OUTPUT

Detail Items

title	description
Product 101 Details	Premium wireless over-ear headphones.
Product 102 Details	High-speed mechanical keyboard.
Product 103 Details	Portable Bluetooth speaker.
Product 104 Details	4K OLED monitor with HDR.
Product 105 Details	Ergonomic wireless mouse.

RTILA X PROJECT JSON

RTILA X Automation Project

dataset-detail-pages.json

{
  "name": "Dataset_Detail_Pages",
  "settings": {
    "urls": [
      "https://learn.rtila.com/scenarios/loops/7"
    ]
  },
  "datasets": {
    "items": {
      "item_selector": "css=.detail-item",
      "properties": [
        {
          "name": "title",
          "type": "text",
          "selector": "css=.detail-title"
        },
        {
          "name": "description",
          "type": "text",
          "selector": "css=.detail-description"
        },
        {
          "name": "price",
          "type": "text",
          "selector": "css=.detail-price"
        },
        {
          "name": "url",
          "type": "attribute",
          "selector": "css=a.detail-link",
          "attribute": "href"
        }
      ]
    },
    "details": {
      "item_selector": "css=article",
      "properties": [
        {
          "name": "full_text",
          "type": "text",
          "selector": "css=p"
        }
      ]
    }
  },
  "commands": [
    {
      "command": "extract_data",
      "params": {
        "dataset": "items"
      }
    },
    {
      "command": "for_each",
      "params": {
        "source_type": "dataset",
        "dataset": "items",
        "as": "item",
        "do": [
          {
            "command": "goto",
            "params": {
              "url": "${item.url}"
            }
          },
          {
            "command": "extract_data",
            "params": {
              "dataset": "details"
            }
          }
        ]
      }
    }
  ]
}

RTILA X Concepts

A second for_each can iterate over a previously extracted dataset. Each row provides values for navigation and extraction, enabling two-phase scraping patterns.

Did you complete this scenario?

Ready to run this automation?

Copy the project JSON above, open RTILA X, create a new project, and paste it.

Download RTILA X Free → View setup guide →

This scenario covers 3 RTILA X commands and is referenced in 2 learning articles.

Continue Learning

Was this helpful?