Flowpane Web Crawler

About

Flowpane is a web governance platform that helps digital agencies monitor compliance files across their client websites. Our crawler periodically fetches governance files to verify they exist, are correctly configured, and follow best practices.

We do not crawl page content, collect personal data, or index websites for search. Our checks are limited to publicly accessible governance and compliance files only.

Identification

User-Agent	Flowpane/1.0 (+https://crawler.flowpane.com)
Compatible UA	Mozilla/5.0 (compatible; Flowpane/1.0; +https://crawler.flowpane.com)
Operator	Flowpane Ltd
Contact	[email protected]
Website	flowpane.com

Behaviour

Files accessed — robots.txt, ads.txt, security.txt, llms.txt, humans.txt, sitemap.xml, sitemap_index.xml, and cookie/consent detection on the homepage
Request rate — Maximum 1 request per second per domain. Respects Crawl-delay directives in robots.txt
robots.txt compliance — Fully RFC 9309 compliant. Respects Disallow rules for both Flowpane and * user-agent groups
Redirects — Follows up to 5 redirects (301, 302, 307, 308)
Timeout — 3-second connection timeout per request
No JavaScript — Raw HTTP responses only. No headless browser execution during automated checks
Caching — Results cached per site's check schedule. Minimum 24 hours between automated checks for any given file

IP Addresses

Flowpane's crawler operates from the following IP address. You can use this to create WAF allowlist rules.

46.32.253.56

This list will be updated if additional IPs are added. For questions, contact [email protected]

Whitelisting

If your WAF is blocking Flowpane, use the instructions below for your provider.

Navigate to Security → WAF → Custom Rules
Create a rule with condition: User-Agent contains Flowpane/1.0
Set the action to Allow
Alternatively, add Flowpane's IP range to Security → IP Access Rules

In Bot Manager, navigate to your bot policy
Add Flowpane/1.0 to the allowed bot list
Or create an IP whitelist for the Flowpane IP range

Log in to the Sucuri dashboard
Go to Settings → Security → Whitelist
Add the Flowpane user agent or IP range

Add a firewall rule to allow requests where the User-Agent header contains Flowpane/1.0
Or whitelist the Flowpane IP addresses listed in the IP Addresses section above

Opting Out

If you don't want Flowpane to access your site, add the following to your robots.txt:

User-agent: Flowpane
Disallow: /

Our crawler fully respects robots.txt directives. This will prevent all automated Flowpane checks.

You can also contact us at [email protected] and we'll exclude your domain manually.

Verifying Requests

To confirm a request is genuinely from Flowpane:

Check the User-Agent header matches Flowpane/1.0 (+https://crawler.flowpane.com)
Verify the source IP is in the published range above
Perform a reverse DNS lookup — legitimate requests resolve to *.crawler.flowpane.com
Verify the cryptographic signature — Flowpane signs all requests using Web Bot Auth (HTTP Message Signatures). Our public key is published at /.well-known/http-message-signatures-directory

Requests claiming to be from Flowpane that don't match these criteria are spoofed. Report them to [email protected].