The robots.txt file is used by website owners to give instructions about their site to web robots. The web robot will read the robots.txt file before visiting a site, to check whether any directories should be excluded from the visit.
In the example file below, the 'User-agent' is specifying ALL robots, and 'Disallow' is instructing them not to visit the /customers directory.
Info | ||
---|---|---|
| ||
User-agent: * Disallow: /customers |
Customer Self Service allows for easy maintenance of the robots.txt file by site Administrators. Simply follow the procedure below.
Step-by-step guide
To View and Update the robots.txt file:
- Login as an Administrator.
- Navigate to Settings → SEO Maintenance (/zSearchEngineOptimisationMaintenance.aspx).
- Click the Robots.txt tab.
- The contents of the current robots.txt file will be displayed in the dialogue box.
- Edit the file contents directly on screen, or copy and paste data from another file.
- In the event of an error, click the Reload Robots.txt button to start again.
- Click the Submit Changes button.
The revised file will be sent to Commerce Vision for review, and you will be advised once the new file has been uploaded.
Tip |
---|
For further information on robots.txt, see http://www.robotstxt.org/robotstxt.html. |
Related help
Content by Label | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|