social-media Scenario 4 — Forum Thread — Extract Nested Comments

Written by RTILA X Engineering Team

Intermediate ~6 min

Prerequisites: social-media/1 social-media/2

What You'll Learn

Handle infinite-scroll feeds on social media

Real-World Use Case

Archiving a complete timeline

LIVE DEMO

Forum Thread

Comments and nested replies collected with a list-type property.

EXPECTED OUTPUT

Forum Comments

author	time	text	replies
techguru42	2025-05-01T10:00:00Z	Comment body 1 discussing the thread topic.	[object Object][object Object]
dataminer_sam	2025-05-02T11:00:00Z	Comment body 2 discussing the thread topic.	[object Object][object Object]
scraper_queen	2025-05-03T12:00:00Z	Comment body 3 discussing the thread topic.	[object Object][object Object]
automation_alice	2025-05-04T13:00:00Z	Comment body 4 discussing the thread topic.	[object Object][object Object]
selector_sam	2025-05-05T14:00:00Z	Comment body 5 discussing the thread topic.	[object Object][object Object]
css_crusader	2025-05-06T15:00:00Z	Comment body 6 discussing the thread topic.	[object Object][object Object]
json_jess	2025-05-07T16:00:00Z	Comment body 7 discussing the thread topic.	[object Object][object Object]
xpath_xavier	2025-05-08T17:00:00Z	Comment body 8 discussing the thread topic.	[object Object][object Object]

RTILA X PROJECT JSON

RTILA X Automation Project

extract-forum-thread.json

{
  "name": "Extract_Forum_Thread",
  "settings": {
    "urls": [
      "https://learn.rtila.com/scenarios/social-media/4"
    ]
  },
  "datasets": {
    "comments": {
      "item_selector": "css=.forum-comment",
      "properties": [
        {
          "name": "author",
          "type": "text",
          "selector": "css=.comment-author"
        },
        {
          "name": "time",
          "type": "text",
          "selector": "css=.comment-time"
        },
        {
          "name": "text",
          "type": "text",
          "selector": "css=.comment-text"
        },
        {
          "name": "replies",
          "type": "list",
          "selector": "css=.forum-reply",
          "properties": [
            {
              "name": "author",
              "type": "text",
              "selector": "css=.reply-author"
            },
            {
              "name": "time",
              "type": "text",
              "selector": "css=.reply-time"
            },
            {
              "name": "text",
              "type": "text",
              "selector": "css=.reply-text"
            }
          ]
        }
      ]
    }
  }
}

Nested comment structures are represented with a list property inside the comment dataset. Each reply has its own author, timestamp, and text selectors.

Did you complete this scenario?

Ready to run this automation?

Copy the project JSON above, open RTILA X, create a new project, and paste it.

Download RTILA X Free → View setup guide →

This scenario covers 2 RTILA X commands and is referenced in 2 learning articles.

Continue Learning

Was this helpful?