Thejollyhellhound20 Thejollyhellhound20 07-11-2018 Computers and Technology contestada 1. How do i find a robots txt for a website? 2. Why is robots txt file used? 3. What is Webcrawling and how do i stop it