complex-workflows Scenario 6 — Scrape → Script Transform → API Call → Merge

Written by RTILA X Engineering Team

Intermediate ~6 min

Prerequisites: complex-workflows/1 complex-workflows/2 complex-workflows/3

What You'll Learn

Integrate AI completion for data extraction

Real-World Use Case

Using natural language to parse unstructured data

LIVE DEMO

Name	Email
Alice	alice@example.com
Bob	bob@example.com
Carol	carol@example.com
Dan	dan@example.com
Eve	eve@example.com

{
  "api_users": [
    {
      "id": 1,
      "plan": "Free"
    },
    {
      "id": 2,
      "plan": "Pro"
    },
    {
      "id": 3,
      "plan": "Pro"
    },
    {
      "id": 4,
      "plan": "Free"
    },
    {
      "id": 5,
      "plan": "Enterprise"
    }
  ]
}

Name	Email	Plan
Alice	alice@example.com	Free
Bob	bob@example.com	Pro
Carol	carol@example.com	Pro
Dan	dan@example.com	Free
Eve	eve@example.com	Enterprise

EXPECTED OUTPUT

RTILA X PROJECT JSON

    {
  "name": "Scrape_Script_API_Merge",
  "settings": {
    "urls": [
      "https://learn.rtila.com/scenarios/complex-workflows/6"
    ]
  },
  "datasets": {
    "scraped_rows": {
      "item_selector": "css=table.cw-scrape-data tbody tr",
      "properties": [
        {
          "name": "name",
          "type": "text",
          "selector": "css=td:nth-child(1)"
        },
        {
          "name": "email",
          "type": "text",
          "selector": "css=td:nth-child(2)"
        }
      ]
    },
    "merged_data": {
      "item_selector": "css=table.cw-merged-data tbody tr",
      "properties": [
        {
          "name": "name",
          "type": "text",
          "selector": "css=td:nth-child(1)"
        },
        {
          "name": "email",
          "type": "text",
          "selector": "css=td:nth-child(2)"
        },
        {
          "name": "plan",
          "type": "text",
          "selector": "css=td:nth-child(3)"
        }
      ]
    }
  },
  "commands": [
    {
      "command": "wait_for_selector",
      "params": {
        "selector": "css=table.cw-scrape-data"
      }
    },
    {
      "command": "extract_data",
      "params": {
        "dataset": "scraped_rows"
      }
    },
    {
      "command": "comment",
      "params": {
        "text": "Trigger chain: script transform and REST API merge"
      }
    },
    {
      "command": "wait_for_selector",
      "params": {
        "selector": "css=table.cw-merged-data"
      }
    },
    {
      "command": "extract_data",
      "params": {
        "dataset": "merged_data"
      }
    }
  ]
}

Merging scraped data with API data often requires a script transform stage. Trigger chains can handle the external REST call and merge the results into one dataset.

Did you complete this scenario?

Ready to run this automation?

Copy the project JSON above, open RTILA X, create a new project, and paste it.

Download RTILA X Free → View setup guide →

This scenario covers 2 RTILA X commands and is referenced in 2 learning articles.

Continue Learning

Was this helpful?