############################### # # User-agent: * # # list folders robots are not allowed to index # Disallow: /about_us/about_us.archives/ Disallow: /activities/activities.archives/ Disallow: /admin/ Disallow: /archives/ Disallow: /common/ Disallow: /contest/ Disallow: /docents/ Disallow: /email/ Disallow: /gmap/ Disallow: /include/ Disallow: /js/ Disallow: /news/news.archives/ Disallow: /news//ECDM_Channel_Bridges.asp/ Disallow: /plans_projects/plans_projects.archives/ Disallow: /preserves/preserves.archives Disallow: /renamed nav/ Disallow: /Scripts/ Disallow: /test/ Disallow: /test2/ Disallow: /vtp/ User-agent: Baidu Disallow: / # # list specific files robots are not allowed to index # Disallow: /directory/foo.html Disallow: /folder2/whatever.html # # ###############################