在 .htaccess 中配置强制UTF-8字符集

网站配置了 robots.txt 来控制爬虫的行为，由于文件里面有中文注释，导致直接浏览器打开之后中文显示乱码。

对于 Apache 服务器，可以通过在 .htaccess 中配置文件的字符集格式解决显示乱码问题。

Set universal UTF-8 encoding:

Set UTF-8 encoding on select file types only:

Note: An .htaccess file located in a sub-directory overrides any duplicate rules from previous .htaccess files. For example, if you have a .htaccess file located in the root defining a 404 and 403 error page, and another .htaccess located in the “test” folder defining only a 404 error page, any files and folders in the “test” folder will use the 404 page defined in the "test" .htaccess file, and the 403 page defined in the root .htaccess file.

参考链接

---

Forcing UTF-8 encoding with .htaccess