Open Source

2 crawlers operated by Open Source

Who Open Source is, for site owners

Open Source operates 2 crawlers in our directory, filed under Data Scrapers. Of those 2, 1 rated safe, 1 rated aggressive for crawl behaviour.

Blocking Open Source carries an SEO impact score of 2/10 across its crawlers. Every crawler here carries the same blocking impact: Low - No SEO ranking impact.

Our recommendation for Open Source, set by its least reassuring crawler (news-please, rated aggressive): Block this bot - it provides no SEO benefit and wastes crawl budget.

Open Source at a glance

OperatorOpen Source
Crawlers in directory2
CategoriesData Scrapers
SEO impact if blocked2/10
Crawl behaviour1 rated safe, 1 rated aggressive

Crawlers operated by Open Source

How to block or allow Open Source in robots.txt

These rules cover all 2 crawlers Open Source operates. Add them to the robots.txt file at the root of your domain.

Block Open Source completely
User-agent: imgproxy User-agent: news-please Disallow: /
Allow Open Source explicitly
User-agent: imgproxy User-agent: news-please Allow: /

robots.txt is a request, not an enforcement mechanism. A crawler that ignores it needs a block at the server or CDN layer. Rules are case-sensitive on the path and matched against the user-agent token, not the full user-agent string.

Frequently asked questions about Open Source

What crawlers does Open Source run?
Open Source operates 2 crawlers in our directory: imgproxy (user-agent imgproxy), news-please (user-agent news-please).
Should I block Open Source?
Block this bot - it provides no SEO benefit and wastes crawl budget. That guidance follows this operator's least reassuring crawler, news-please, rated aggressive, because allowing an operator allows every crawler it runs. The SEO impact score for blocking is 2/10, where 10 means blocking costs you the most visibility.
How do I block Open Source in robots.txt?
Add the following to robots.txt at your domain root: User-agent: imgproxy then User-agent: news-please then Disallow: /. One Disallow line covers all 2 user-agent tokens.
Does blocking Open Source hurt my search rankings?
Every crawler here carries the same blocking impact: Low - No SEO ranking impact. Google's own ranking crawler is Googlebot, which this operator does not run, so blocking Open Source does not by itself remove you from Google's index. What it can change is whether AI assistants and analytics tools can read your pages.