diff options
Diffstat (limited to 'docs/content/en/templates/robots.md')
-rw-r--r-- | docs/content/en/templates/robots.md | 33 |
1 files changed, 13 insertions, 20 deletions
diff --git a/docs/content/en/templates/robots.md b/docs/content/en/templates/robots.md index 0e453c549..2d412d775 100644 --- a/docs/content/en/templates/robots.md +++ b/docs/content/en/templates/robots.md @@ -2,13 +2,9 @@ title: robots.txt template linkTitle: robots.txt templates description: Hugo can generate a customized robots.txt in the same way as any other template. -categories: [templates] +categories: [] keywords: [] -menu: - docs: - parent: templates - weight: 170 -weight: 170 +weight: 180 aliases: [/extras/robots-txt/] --- @@ -20,8 +16,6 @@ enableRobotsTXT = true By default, Hugo generates robots.txt using an [embedded template]. -[embedded template]: {{% eturl robots %}} - ```text User-agent: * ``` @@ -37,24 +31,23 @@ You may overwrite the internal template with a custom template. Hugo selects the ## robots.txt template example -{{< code file=layouts/robots.txt >}} +```text {file="layouts/robots.txt"} User-agent: * {{ range .Pages }} Disallow: {{ .RelPermalink }} {{ end }} -{{< /code >}} +``` This template creates a robots.txt file with a `Disallow` directive for each page on the site. Search engines that honor the Robots Exclusion Protocol will not crawl any page on the site. -{{% note %}} -To create a robots.txt file without using a template: - -1. Set `enableRobotsTXT` to `false` in the site configuration. -1. Create a robots.txt file in the `static` directory. - -Remember that Hugo copies everything in the [`static` directory][static] to the root of `publishDir` (typically `public`) when you build your site. +> [!note] +> To create a robots.txt file without using a template: +> +> 1. Set `enableRobotsTXT` to `false` in the site configuration. +> 1. Create a robots.txt file in the `static` directory. +> +> Remember that Hugo copies everything in the [`static` directory][static] to the root of `publishDir` (typically `public`) when you build your site. +[embedded template]: {{% eturl robots %}} +[site configuration]: /configuration/ [static]: /getting-started/directory-structure/ -{{% /note %}} - -[site configuration]: /getting-started/configuration/ |