Robotic User-Agents

JICWEBS reporting standards say that Page Impressions made by search engines, commercial robots, and other similar user-agents are invalid.

Media owners must exclude material robotic activity from any claimed statistics. Should ABC find such activity, we may apply an adjustment to claimed metrics in our final certificate.

To help in this exclusion process, the IAB/ABC Spider and Robot Exclusion List contains a list of user-agent strings found to represent material robots, formatted with both plus and space.

The list is updated monthly, and is available for download as a text file from the link below. This list may be altered without notice; however, we will show the date of alteration. The list to be used in a given month will be published by the 25th of the previous month.

Spider and Robot Exclusion List (available to members with technical access only)

Valid Browser Strings

Media owners may wish to use a more rigorous robot exclusion process by adopting the dual-pass approach recommended by the IAB. Before applying user-agent matches to the Exclusion List, this approach will include only records matching the user-agents on the Include list.

This list is updated monthly to the same schedule as the Spider and Robot exclusion list above.

User-Agent Include List (available to members with technical access only)

IP Address Exclusions

JICWEBS reporting standards consider requests coming from certain IP addresses always to be invalid. By definition, these addresses represent activity from internal maintenance processes and external site monitoring tools, and our filtering excludes these records by default.

The list is updated monthly, and is available for download in various text file formats from the link below. The list to be used in a given month will be published by the 25th of the previous month.

IP Address Exclusion List (available to members with technical access only)