SEO: 2019 | Robots.txt is changing – Is Your Website Ready?

By |2019-07-11T09:56:46-04:00July 10th, 2019|

An important piece of running a website in an SEO campaign involves directing the search engine crawlers as to where to go and what to do there. Though the system - a text file called robots.txt - has been in use for decades, there is no formal documentation detailing the directives crawlers will respect and