10 Laravel packages tagged “crawlers”

Most downloaded: spatie/crawler, with 21,842,954 downloads and 2,829 GitHub stars. Ranked by total downloads — use the controls below to sort by stars, most recently updated, or name.

spatie/crawler

Crawl all internal links found on a website
2,829 21,842,954

jaybizzle/laravel-crawler-detect

A Laravel package to detect web crawlers via the user agent
326 2,977,212

lobotomised/laravel-autocrawler

A tool to crawl your own laravel installation checking your HTTP status codes
2 44,677

vlados/laravel-blade-crawler-detect

Boost Lighthouse and PageSpeed scores by hiding cookie banners, chat widgets and heavy scripts from crawlers via two Blade directives. Octane-safe.
5,961

kaishiyoku/hera-rss-crawler

Modern library to handle RSS/Atom feeds
2 4,617

acanto/crawler

Crawl all internal links found on a website
2,921

alrik11es/scrappy

A free proxy list based downloader for your crawling projects in PHP
1 2,224

tfd/laravel-well-known-traffic-advice

A Laravel package that provides a standardized .well-known/traffic-advice endpoint for communicating server load, maintenance windows, and access recommendations to automated clients, crawlers, and partner services.
1,210

hissezhaut/laravel-link-checker

Check all links in a Laravel app
16

abelhalo/zolcrawler

Zol Crawler
12