“Such simple devices as robots.txt, “noindex,” and password protection could wall off any news media web page from search engines. But no media companies were doing that, because they WANT the traffic delivered by search engines. So it has always been clear that the media recognized the value of being seen by search engines.” #Media

[Link]