How to scrape DEV Community

Extract dev.to article title, author, date, tags, reactions and reading time as typed JSON validated against your schema, no CSS selectors.

Site:: dev.to
Difficulty:: Easy
Rendering:: JavaScript-rendered
Anti-bot:: Cloudflare in front of dev.to (DEV runs on Forem, fronted by Cloudflare), Per-IP rate limiting on rapid sequential requests

What you can extract from DEV Community

These are the fields teams most often pull from dev.to, and why a hand-written selector for each one tends to break over time.

Field	Type	Why selectors break
article title	string	The visible h1 on a dev.to article carries Forem's hashed utility classes (crayons-article__header__title style), and those class names change across Forem releases, so a selector pinned to the class can silently return empty after a deploy while the page still looks identical.
author	string	The author name sits inside a profile-link block that DEV also reuses for the article's organization and for co-authors, so a naive .crayons-article author selector can grab the org name or a second contributor instead of the primary author depending on how the byline rendered.
published date	string	DEV shows a human label like 'Posted on Jun 4' and hides the exact ISO date in a datetime attribute on a time tag, so a selector reading the visible text loses the year entirely and breaks on relative phrasings like 'Updated' versus 'Posted on'.
tags	array	Tags render as a list of anchor chips whose count varies from zero to four per article, and the chip markup is shared with 'related tags' widgets in the sidebar, so a tag-link selector over-collects unless it is carefully scoped to the article header block.
reaction count	number	The reaction total is injected by JavaScript after the article shell loads and is split across heart, unicorn and bookmark counters, so a selector run on raw HTML reads zero or null, and even after render the displayed value is abbreviated (for example 1.2k) rather than the exact integer.
reading time	string	Reading time appears as free text such as '5 min read' tucked next to the date in a meta row whose layout shifts on mobile breakpoints, so a positional selector that works on desktop returns the wrong sibling node when DEV serves the responsive variant.

The schema-first approach

Instead of maintaining selectors, you hand Extracto a JSON schema describing the shape you want. Here is a schema that works for DEV Community:

{
  "article_title": "string",
  "author": "string",
  "published_date": "string",
  "tags": "array",
  "reaction_count": "number",
  "reading_time": "string"
}

1

Describe the DEV Community fields you want

Write a small JSON schema listing the fields to pull from dev.to (for example article title, author, published date). No CSS selectors or XPath needed.
2

Send the URL and schema to Extracto

POST the DEV Community page URL and your schema to the Extracto API. Extracto renders the page in a real browser, handling JavaScript and dynamic content automatically.
3

Get schema-validated JSON back

Extracto returns JSON that matches your schema exactly, validated before it leaves the API, so your pipeline never receives broken or partial data.

Common pitfalls when scraping DEV Community

Reaction counts on dev.to are rendered client side and abbreviated in the UI (1.2k rather than 1234), so any approach that reads the static HTML will miss them. Extracto runs the page in a real headless browser so the post-render number is present, but expect the value to reflect the rounded display figure unless you cross-check against the Forem API's exact public_reactions_count.
DEV reuses the same byline and tag-chip markup for organizations, co-authors and sidebar 'related tags' widgets, so field bleed is the most common failure: the org name lands in the author field or sidebar tags inflate the tag array. A schema that names author and tags explicitly lets Extracto target the article's own header rather than every matching node on the page.
The published date is shown as a relative or shortened label ('Posted on Jun 4', 'Updated') while the precise timestamp lives in a hidden datetime attribute. Asking for published_date as a plain string can yield the lossy visible text; if you need a sortable value, state in the schema that you want an ISO 8601 date so the model resolves the year.
For more than a handful of articles, scraping individual pages is the wrong tool. DEV publishes a documented Forem API with the same fields (reading_time_minutes, tag_list, public_reactions_count, published_at), defaults to 30 articles per page, and enforces its own rate limits. Use the API for bulk pulls and reserve page extraction for one-off reads where you want the rendered article rather than markdown.

Is it legal to scrape DEV Community?

DEV Community (dev.to) is one of the more cooperative public targets on the web. It is built on the open-source Forem platform and publishes a documented public REST API at developers.forem.com, so for bulk or production work the API is almost always the right call and is explicitly sanctioned. Article pages themselves are public and indexable, and dev.to's robots.txt permits crawling of article paths while disallowing administrative and search endpoints. Even so, the content belongs to the individual authors who posted it, and DEV's Terms of Use govern reuse: scraping article text to republish it elsewhere can infringe the author's copyright regardless of how the page was fetched. Keep request volume modest, identify yourself, respect robots.txt and the documented API rate limits, and treat extraction as a way to read public metadata (titles, tags, reaction counts) rather than to mirror or repost authors' work. As of 2026, prefer the official Forem API for anything beyond occasional one-off page reads.

Extracto follows robots.txt by default and rate-limits politely. You are responsible for compliance with DEV Community's terms and applicable law (GDPR, CCPA) in your use case.

Scraping DEV Community: FAQ

Should I scrape dev.to article pages or use the Forem API?

For anything at scale, use the official Forem API at developers.forem.com. It returns the same structured fields (title, user, published_at, tag_list, public_reactions_count, reading_time_minutes) as clean JSON, supports pagination at 30 articles per page, and is explicitly sanctioned. Reserve page-level extraction for one-off reads where you specifically want the rendered article rather than the markdown body the API returns.

Why does the reaction count come back as null or zero when I scrape the raw HTML?

DEV injects the reaction total with JavaScript after the article shell loads, so it is simply not present in the initial server response that a plain HTTP fetch sees. Extracto renders the page in a real headless browser before reading it, so the count is populated. Keep in mind the UI abbreviates large values (1.2k), so for an exact integer you should reconcile against the Forem API's public_reactions_count field.

Do I need to handle Cloudflare to read a dev.to article?

DEV sits behind Cloudflare, but article pages are public and indexable. Extracto includes a managed rendering and anti-bot handling layer with proxies managed for you, so protected pages are fetched without any setup on your side. You still control volume: keep requests modest and respect robots.txt and the documented rate limits so you behave like a well-mannered reader rather than a crawler.

How do I get the exact publish date instead of 'Posted on Jun 4'?

The visible label is a shortened, sometimes relative string, while the precise timestamp is stored in a hidden datetime attribute on the page. In your Extracto schema, describe published_date as an ISO 8601 date so the extraction resolves the full year and time rather than returning the lossy display text. If the exact value is unavailable on the rendered page, Extracto returns null for that field instead of guessing a date.

Can I pull the full article body text with Extracto?

Yes, you can add a body field to your schema and Extracto will return the rendered article content as a typed string. Be mindful that DEV article text is copyrighted by its author, so use it for analysis or indexing rather than republishing. If you want the canonical markdown source instead of rendered HTML text, the Forem API exposes body_markdown directly.