Start free
All posts
· Roger Massana · guide, scraping

Turning GitHub repositories into clean JSON

GitHub renders most of what you need server-side, which makes it a great first target. Here's a schema for pulling repo metadata into structured data.


If you want to feel out a structured-extraction API before you point it at something hard, GitHub is a good place to start. Repository pages render their important metadata server-side, the content is stable, and the layout is consistent across millions of pages. That combination is exactly what makes an extraction predictable.

A repo schema

Here is a schema that captures the fields most people actually want off a repository page:

const repo = await client.extract({
  url: "https://github.com/facebook/react",
  schema: {
    name: z.string(),
    owner: z.string(),
    description: z.string(),
    primaryLanguage: z.string(),
    stars: z.number(),
    topics: z.array(z.string()),
  },
});

You send that, and you get back a clean object. No parsing the star count out of a string like “228k”, no guessing which heading is the description. The numbers come back as numbers because the schema said so.

Why this one is easy

A page is easy to extract from when three things are true, and a GitHub repo page hits all three:

When you are learning what a tool can do, start on pages like this. You get a clean signal about extraction quality without fighting infrastructure.

Scaling it up

Once the single-repo extraction works, the same schema runs across a list of repos. Feed it the URLs from an org, a topic page, or a starred list, and you get a uniform dataset where every row has the same fields and the same types. That is the part that makes downstream work pleasant: you are not writing defensive code for the one repo that formatted its star count differently.

A note on what is hard

Not every site behaves like GitHub. Pages that render everything client-side, sit behind aggressive anti-bot systems, or change layout constantly are a different conversation. We are honest about that distinction, and we think you should pick your early targets accordingly. Cooperative, server-rendered, structured pages first.

Try it

The live demo on the home page defaults to a GitHub repo for exactly this reason. Run it and change the URL to any repo you like, or start free to script it.


← All posts Start free with Extracto →