<p>I ended up adding a lightswitch for do not index. I am checking if this value exists in _layout.html if it does I am adding </p>

<pre><code>{% if entry.noIndex %}&lt;meta name="robots" content="noindex,nofollow"&gt;{% endif %}
</code></pre>

<p>I also updated my search results, sitemap entry template to check for the same thing and not include it in the sitemap file or the search results. As well as the dynamic sitemap.xml file. I believe this should be good.</p>

<p>Also I extended the token expiration but I think with the above approach, we will just publish like normal and remove the token expiration.</p>

CC BY-SA 3.0
2015-07-13T20:30:58.333Z
2015-07-13T20:30:58.333Z
Bowenac
<blockquote>
  <p>Or does anyone have any suggestions on a field to prevent specific entries from being indexed by search engines.</p>
</blockquote>

<p>You could create a sitemap file that excludes those entries (based on some field in the entry), and include that sitemap in your robots.txt. That should discourage well-behaved search engines.</p>

<p>Also, you could <a href="http://buildwithcraft.com/docs/templating/requirelogin" rel="nofollow">requireLogin</a> on those pages, which would make them inaccessible to anyone not logged in (which would include bots).</p>

CC BY-SA 3.0
2015-07-13T19:16:40.257Z
That is what I was thinking, I thought of just adding a checkbox at the top of entries that just says don't index. Then if that value is true, add the noindex, nofollow meta tag to _layout.html. Then on my current sitemap just exclude drafts. This would require extending the token duration.
Bowenac
CC BY-SA 3.0
2015-07-13T19:34:37.800Z
Can't require login, most of this content users are being sent to based on if they filled out a form. Really wish there was a built in way to just create hidden entries haha.
Bowenac
CC BY-SA 3.0
2015-07-13T18:18:43.450Z
2015-07-13T18:18:43.450Z
Marion Newlevant
<blockquote>
  <p>Anyway to prevent these tokens from expiring?</p>
</blockquote>

<p>You could set the <a href="http://buildwithcraft.com/docs/config-settings#defaultTokenDuration" rel="nofollow">defaultTokenDuration</a> to some really high value.  Just keep in mind this affects <em>all</em> tokens on the site including the ones used for email account verification and password resetting.</p>

<blockquote>
  <p>Or does anyone have any suggestions on a field to prevent specific entries from being indexed by search engines.</p>
</blockquote>

<p>If you know the URLs, you could list them in <a href="https://en.wikipedia.org/wiki/Nofollow" rel="nofollow">nofollow</a>.</p>

CC BY-SA 3.0
2015-07-13T18:10:38.350Z
If you know which path they'll be published under you could disallow `/sectionName/*` in `robots.txt`
megatrond
CC BY-SA 3.0
2015-07-13T19:02:33.810Z
I still need the links to work, so I could set the tokenduration really high, then I guess if is draft nofollow. Are drafts also indexed? Or is that the purpose of the token so that the url is always different?
Bowenac
CC BY-SA 3.0
2015-07-13T19:12:03.147Z
Also a specific path wouldnt work as some in the same path need to be indexed. I think I would need to check if it's a draft, add noidex meta tag. Or would I run into issues going that route for drafts that end up published?
Bowenac
CC BY-SA 3.0
2015-07-13T18:03:52.890Z
2015-07-13T18:03:52.890Z
Brad Bell
<p>So I have some gated content entries and need to prevent these from being indexed by search engines etc. Currently using draft pages but the tokens expire which then causes a 404 if someone is to land on that entry.</p>

<p>Anyway to prevent these tokens from expiring? Or does anyone have any suggestions on a field to prevent specific entries from being indexed by search engines. Or any options for saving as hidden entry, would be nice if this was built into craft as it seems like a pretty common feature.</p>

<plugin-development><templating><entries><draft><token>
Is it possible to disable token expiration? Or prevent follow from bots?
# Is it possible to disable token expiration? Or prevent follow from bots?

(Asked by: username_0 on 2015-07-13)

So I have some gated content entries and need to prevent these from being indexed by search engines etc. Currently using draft pages but the tokens expire which then causes a 404 if someone is to land on that entry.

Anyway to prevent these tokens from expiring? Or does anyone have any suggestions on a field to prevent specific entries from being indexed by search engines. Or any options for saving as hidden entry, would be nice if this was built into craft as it seems like a pretty common feature.

**Tags:** plugin-development, templating, entries, draft, token


---

## 3 answers

### Answer by username_0 on 2015-07-13. Score: 2

I ended up adding a lightswitch for do not index. I am checking if this value exists in _layout.html if it does I am adding 
{% if entry.noIndex %}{% endif %}

I also updated my search results, sitemap entry template to check for the same thing and not include it in the sitemap file or the search results. As well as the dynamic sitemap.xml file. I believe this should be good.
Also I extended the token expiration but I think with the above approach, we will just publish like normal and remove the token expiration.


---

### Answer by username_1 on 2015-07-13. Score: 1

> Or does anyone have any suggestions on a field to prevent specific entries from being indexed by search engines.

You could create a sitemap file that excludes those entries (based on some field in the entry), and include that sitemap in your robots.txt. That should discourage well-behaved search engines.

Also, you could [requireLogin](http://buildwithcraft.com/docs/templating/requirelogin) on those pages, which would make them inaccessible to anyone not logged in (which would include bots).

> **Comments:**
> **username_0**: That is what I was thinking, I thought of just adding a checkbox at the top of entries that just says don't index. Then if that value is true, add the noindex, nofollow meta tag to _layout.html. Then on my current sitemap just exclude drafts. This would require extending the token duration.  
> **username_0**: Can't require login, most of this content users are being sent to based on if they filled out a form. Really wish there was a built in way to just create hidden entries haha.  

---

### Answer by username_2 on 2015-07-13. Score: 0

> Anyway to prevent these tokens from expiring?

You could set the [defaultTokenDuration](http://buildwithcraft.com/docs/config-settings#defaultTokenDuration) to some really high value. Just keep in mind this affects *all* tokens on the site including the ones used for email account verification and password resetting.

> Or does anyone have any suggestions on a field to prevent specific entries from being indexed by search engines.

If you know the URLs, you could list them in [nofollow](https://en.wikipedia.org/wiki/Nofollow).

> **Comments:**
> **username_3**: If you know which path they'll be published under you could disallow `/sectionName/*` in `robots.txt`  
> **username_0**: I still need the links to work, so I could set the tokenduration really high, then I guess if is draft nofollow. Are drafts also indexed? Or is that the purpose of the token so that the url is always different?  
> **username_0**: Also a specific path wouldnt work as some in the same path need to be indexed. I think I would need to check if it's a draft, add noidex meta tag. Or would I run into issues going that route for drafts that end up published?  

---

