Talk:Internet Archive

From RationalWiki
Jump to navigation Jump to search

robots.txt[edit]

Also if web archive has archived a site then the robots.txt changes to "do not crawl" will web archive delete the saved pages from existence or just hide them?

archive.is doesn't care about robots.txt! User000name (talk) 19:54, 19 June 2016 (UTC)

Retrieved from "https://rationalwiki.org/w/index.php?title=Talk:Internet_Archive&oldid=1818672"