🕷️ Build: scraping to CSV

Caption: the complete build process — Firecrawl brings in the site’s raw content, the agent organizes what matters, and saves a CSV that opens directly in a spreadsheet.

New here? Scraping is the technical term for "automatically extracting information from a web page." CSV is a simple spreadsheet file (Comma-Separated Values) that any spreadsheet app can open directly. In this build, you'll ask the agent to read a public page and deliver its data organized in a table — without you touching any scraping code.

This is the hands-on module of Track 2: so far, you’ve learned what the MCP protocol is (2.1), how an MCP server is organized internally (2.2), how to install and authenticate one (2.3), and how to keep permanent instructions in a cheat sheet (2.4). Now all of that becomes a concrete result — a data file you can open and show someone. This kind of tangible deliverable is what separates “I learned about MCP” from “I used MCP to solve a real problem.”

New here? O Firecrawl is the MCP server this build uses to read websites. It offers four operations with very different purposes — knowing which one to request (or letting the module 2.4 cheat sheet guide the agent) helps avoid vague requests and empty results.

A common mistake beginners make is asking crawl when you actually meant extract — the result is full of loose text, difficult to organize into columns. If your end goal is a table (rows and columns, like a CSV), start by thinking about extract; if the goal is to "read the content" without a fixed structure, crawl or scrape are enough.

With the Firecrawl MCP server already running (module 2.3), the entire build comes down to a well-written request—the agent chooses between map/crawl/extract based on what you asked for.

Objective: generate a CSV with real data extracted from the site. How to verify: open produtos.csv in a spreadsheet (Excel, Google Sheets, LibreOffice) and confirm that every row has a name, price, and category filled in — without too many empty cells.

Caption: you never write the three technical calls—you only describe the result you want. The agent is the one that translates the request into map, extract and saving the file.

Two errors can block this step before it even begins: invalid or expired API key — Firecrawl responds with an authentication error, and the agent can't even try; the fix is to revisit module 2.3 and reauthenticate. And address entered incorrectly — a https:// missing or a domain with a typo will cause the first call to fail before any scraping happens. Check the pasted address before running it.

Caption: the agent doesn’t give up at the first “I couldn’t find anything” — it tries a simpler approach, then falls back to a raw scrape to organize manually, and reports a real failure only if neither attempt works. It’s the self-correction from Track 1, now applied to scraping.

Sometimes the extract returns an empty list — the page may have a different structure than expected, or the requested fields may not exist in that form. This is exactly where Trilha 1's self-correction comes into play: a good agent doesn't give up at the first "nothing found" — it simplifies the request (fewer fields), tries again, and if that still doesn't work, falls back to scrape raw and manually organizes the data from the complete text.

An empty list isn’t the only way a build can fail — two other common errors show up along the way. One site that blocks scraping returns an error response (sometimes called "403" or "access denied") instead of content — in that case, insisting won't help; the site simply doesn't allow this kind of automated access, and the right thing is to check whether it offers its own public API or give up on that source. And one usage limit exceeded (your monthly Firecrawl plan allowance) makes every new request fail until the next cycle — the agent usually mentions this in the error response; if it happens in the middle of a large crawl, that's a sign you should have asked for fewer pages at once.

New here? Each Firecrawl call uses up part of your quota (a monthly usage limit on your plan), and some sites prohibit scraping in their terms of service or in a file called robots.txt. Before running a large crawl, check whether the site allows it and avoid requesting more volume than you actually need.

O robots.txt is a simple file that websites publish at the root of the address (something like exemplo.com/robots.txt) saying what automated bots can and can’t access — Firecrawl usually follows these rules on its own, but it’s useful to know they exist, because they explain why sometimes part of a site returns nothing. If the data you want involves personal information from third parties (names, email addresses, phone numbers of individuals), handle it with the same care you’d give any other sensitive data: ask yourself whether you really need that field before including it in the CSV, and never redistribute this type of data unnecessarily.

Once the MCP server and cheat sheet are ready, repeating this build on another site just means changing the address and the desired fields in the request — the rest of the workflow (map → extract → CSV, with a fallback to scrape) is already taught. This is the kind of work that's worth truly automating — a topic Track 4 (Deploy) will take further.

New here? "Automate" here means letting that same routine run on its own, without you needing to copy and paste the request again — for example, generating the updated CSV every Monday morning. This module gives you the manual, on-demand build; Track 4 teaches you how to put that same request on a fixed schedule, running on a server without depending on your laptop being on. Think of this module as the prototype that proves the request works — only after it’s been validated is it worth investing time in truly automating it.