robot

Elated Elk answered on August 20, 2022 Popularity 10/10 Helpfulness 1/10

answer robot

robot

Comment

Tip Elated Elk 1 GREPCC

# Pantheon's documentation on robots.txt: https://pantheon.io/docs/bots-and-indexing/
User-agent: *
Disallow: /

User-agent: RavenCrawler
User-agent: rogerbot
User-agent: dotbot
User-agent: SemrushBot
User-agent: SemrushBot-SA
User-agent: PowerMapper
User-agent: Swiftbot
Allow: /

xxxxxxxxxx

# Pantheon's documentation on robots.txt: https://pantheon.io/docs/bots-and-indexing/

User-agent: *

Disallow: /

User-agent: RavenCrawler

User-agent: rogerbot

User-agent: dotbot

User-agent: SemrushBot

User-agent: SemrushBot-SA

User-agent: PowerMapper

User-agent: Swiftbot

Allow: /

Popularity 10/10 Helpfulness 1/10 Language whatever

Source: dev-rahatanwar.pantheonsite.io

Tags: robot whatever

Link to this answer
Share Copy Link

Contributed on Aug 20 2022

Elated Elk

0 Answers Avg Quality 2/10

robot

Contents

More Related Answers

robot

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.