Web scraping is used to extract content and data from a website, often via bots and other automated means. For instance, competitors may want to scrape or copy a site to gather information about your site. To discourage scraping of your site, the Customer Self Service eCommerce Platform has a setting that helps detect suspicious IP addresses and restrict them from accessing your site for a set time. Administrators can view the list of restricted IP addresses and remove them.
How it works
IP addresses are flagged as suspicious when they access a special trap route on your site since the usual browsing or purchasing customer or website visitor would not go to this 'hidden' page.
Step-by-step guide
To enable and configure the setting,
- In the CMS, navigate to Settings → Feature Management.
- Select System, then enable Honeypot.
- Click Configure.
- To enable Honeypot on your website, toggle ON Enable Honeypot.
- In Honeypot Trap Route, enter the path of the hidden page. The route can be any name. NOTE - This name should be changed from time to time to counter scrapers from detecting that it is a 'hidden' page.
- In Ip Timeout Minutes, enter the number of minutes a restricted IP address will not be permitted to access your site. After the set timeout, the IP address will be able to access your site again. However, it will remain on the list of sites that have been restricted. NOTE - The timeout minutes must be a number higher than zero. If '0' is entered, it will default to '1'. Default: 60min.
- In Honeypot Code, leave the default code as it is.
- In Response Type, select either '404 - Not Found' or 'Response Message Content'. This determines the page type returned when the trap route is accessed.
- 404 - Not Found: the route has no page so the server returns a not found error.
- Response Message Content: the route leads to a meaningless content page
If 'Response Message Content' was selected, the Response Message Content editor automatically displays. Enter the content including formatting and styling for the page.
This page should be edited to resemble other pages on your site.
- To save the settings, click Save or Save & Exit.
View restricted IP addresses list
All IP addresses that are restricted currently or in the past are listed in the Restricted IP Maintenance section of the Honeypot Settings screen. They will remain in this list unless manually deleted.
Related help
