kforney.github.io/robots.txt

mirror of https://github.com/kforney/kforney.github.io.git synced 2024-11-14 19:10:10 -07:00
# www.robotstxt.org/

# Allow crawling of all content
User-agent: *
Disallow: /files/*
Disallow: /js/*
Add boilerplate project. 2022-12-04 22:21:44 -07:00			`# www.robotstxt.org/`

			`# Allow crawling of all content`
			`User-agent: *`
Update robots.txt 2023-03-02 14:31:13 -07:00			`Disallow: /files/*`
			`Disallow: /js/*`