<p>What's the reasoning behind putting all queries in a single file? That looks totally chaotic to me. You're defining a bunch of variables that you might not even need for the current request, and then presumably using those variables in other templates. A new dev looking at the codebase will be wondering where those variables are coming from. Whenever you need a new query, you add it to an ever-growing list of global query results? Who will check if you still need all of those? How will you know whether you can safely delete or modify one of those queries, if you can't tell where it might be used?</p>
<p>Looks like a great opinionated setup if you're working alone, but not something I would ever consider if there's a chance another dev will need to work on this codebase.</p>
<hr />
<p>Regarding the <code>.cache()</code> vs <code>{% cache %}</code> question, it helps to understand what both of them do:</p>
<ul>
<li><code>.cache()</code> will cache the result of that particular query with those exact parameters and return this cache in future queries using the same parameters. It will also register a cache dependency on the queries elements, so if any of those change, the cache is automatically invalidated.</li>
<li><code>{% cache %}</code> will cache the entire template output of the Twig code inside it and <em>some</em> of its side-effects. In particular, JS and CSS assets or code snippets registered through the <code>{% js %}</code> and <code>{% css %}</code> tags will be output on subsequent requests as well. The tag also automatically registers all element queries <em>executed</em> inside the tag and add cache dependencies on those. So if any elements that are used inside the cache change, the cache is invalidated as well.</li>
</ul>
<p>I would not use both in conjunction, i.e. not cache individual queries when you're already using the <code>{% cache %}</code> tag. This gets confusing, I'm not sure how those will interact or if the cache dependencies will be picked up correctly. For the most part, the element query cache won't be of much use, since the cache tag will prevent those queries from being executed at all during a cache hit.</p>
<p>I rarely if ever use <code>.cache()</code>. The savings on a single query are just lackluster compared to caching entire parts of the page with the <code>{% cache %}</code> tag.</p>
<blockquote>
<p>The second part of this question is the .cache() - do I need to use this on ALL versions of the query? If not then which one is it best to set this on initially?</p>
</blockquote>
<p>You need to call <code>.cache()</code> on all queries that you want to cache the results of and be able to return cached results on subsequent requests. Any query where you're <em>not</em> calling <code>.cache()</code> will not be able to return cached results, even if the exact same query with the same parameters has already been cached in another place or during another request. So you need to use <code>.cache()</code> on all queries that you want to utilize the cache, not just the first occurrence of the query.</p>
<blockquote>
<p>What is the best way to add these queries to keep the, as DRY but make them cacheable?</p>
</blockquote>
<p>In most cases, I would just use the <code>{% cache %}</code> tag. This works automatically since it picks up element queries inside the tag and adds them as dependencies, so you rarely have to worry about cache invalidation. It's also much more efficient. For example, say you're caching an entire navigation with a mega menu, some related content, etc. The template might have dozens of queries, and the <code>{% cache %}</code> tag will reduce that to a single DB request or file read on cache hits. Compared to that, element queries cache much less, result in more queries and more of your template being executed for every request.</p>
<p>I would consider element query caching only in some cases:</p>
<ul>
<li>You have a highly dynamic site with lots of interactive elements that depend on user accounts or cookies, so you can't cache entire areas of the page.</li>
<li>You have a site with very frequent content edits (say, a news outlet that posts frequently). The <code>{% cache %}</code> tag might be insufficient because those caches will be invalidated very frequently. Meanwhile, caches for element queries that query content that is changed less frequently can be used for a longer time.</li>
</ul>
<p>Regarding DRY templates, with the <code>{% cache %}</code> tag, sometimes you have to repeat queries. This happens most often if you want to use a query inside multiple <code>{% block %}</code> tags in a Twig template, you can't wrap the <code>{% cache %}</code> tag around a block. For example:</p>
<pre class="lang-twig prettyprint-override"><code>{% set results = craft.entries().search(query).all() %}

{% block header %}
{% cache %}
    Found {{ results|length }} results. 
{% endcache %}
{% endblock %}

{% block content %}
{% cache %}

    {% for result in results %}
        {{ result.title }}
    {% endfor %}

{% endcache %}
{% endblock %}
</code></pre>
<p>This will not work because the query is executed outside the cache tags, so it will not be picked up as a dependency, leading to stale results. But you also can't move the query inside the first cache tag, because then the second block doesn't register the dependencies. In this case, you need to execute the query twice:</p>
<pre class="lang-twig prettyprint-override"><code>{% set resultsQuery = craft.entries().search(query) %}

{% block header %}
{% cache %}
    Found {{ clone(resultsQuery).count() }} results. 
{% endcache %}
{% endblock %}

{% block content %}
{% cache %}

    {% for result in clone(resultsQuery).all() %}
        {{ result.title }}
    {% endfor %}

{% endcache %}
{% endblock %}
</code></pre>
<p>This will work because while the query is created outside the cache tags, it is only executed inside them. This is technically inefficient, because you have one superfluous query. But now both areas can be cached correctly, and as a result cache hits will load faster.</p>
<p>In general, I would keep queries in the templates where they are used and not outsource them to one god template somewhere. This is much easier to understand, and those queries will play nicer with the <code>{% cache %}</code> tag and behave predictably.</p>

CC BY-SA 4.0
2024-07-19T14:14:06.163Z
2024-07-20T12:37:12.833Z
2024-07-20T12:37:12.833Z
MoritzLost
<p>Thanks for the in depth response @MoritzLost</p>
<p>So, yes I am a solo developer so it didn't concern me too much about other developers, as per your first point. But I see what you are saying here.</p>
<p>The main reason I made the decision to put these queries into the globals was due to most of these queries being used in the navigation which is global AND some also reused in the footer (also global), but several are then also used in templates in block tags so rather than writing duplicate queries, it made sense to have the available via the global file, for easy re-use.</p>
<p>My reason for doing this was so I didn't have the inefficiency of repeating several queries multiple times, especially as I wasn't doing any caching initially. I know that this approach wasn't sensible long term and hence the main point of my question and why I am looking to change this now I want to add some caching.</p>
<p>I understand the difference between the <code>{% cache %}</code> tag and the <code>.cache()</code>
My question was perhaps not phrased too well.</p>
<p>In regards to the <code>.cache()</code> - it seems to make sense to use this for the new approach when I am repeating the element queries.<br />
What I wasn't sure about was if each repeated query should contain the <code>.cache()</code> or more-so which version of the query to add this to?
Is it a 'first seen' approach from Craft? So logically this should likely be added to the one in the <code>header.twig</code> as this is likely the one seen first.</p>
<p>I have used <code>clone()</code> before in other projects, again to reduce duplicated queries. However, I thought that  <code>.cache()</code> would be more efficient? It is interesting how you have suggested this and this would have perhaps worked better with the original globals approach; but having now moved and duplicated the queries I wonder if  <code>.cache()</code> is a better option?</p>
<blockquote>
<p>In general, I would keep queries in the templates where they are used and not outsource them to one god template somewhere. This is much easier to understand, and those queries will play nicer with the {% cache %} tag and behave predictably.</p>
</blockquote>
<p>I agree with this sentiment too and definitely an approach I am going to use moving forward. I suppose in essence, a duplicated query isn't too much to be concerned about when using  <code>{% cache %}</code> if it is efficient, then it should only affect the first load as after this it will be cached and it's a non-issue.</p>
<p>Thanks for the help. Happy to discuss in more detail here or in Discord.</p>

CC BY-SA 4.0
2024-07-20T12:38:53.923Z
Regarding where to use `.cache()`, you need it on all queries that you want to use the cache. I had mentioned this in my reply, but a bit briefly, probably got lost in the wall of text. I've expanded on that paragraph above to make it more clear!
I'll follow up on the other point on Discord.
MoritzLost
CC BY-SA 4.0
2024-07-20T11:26:34.790Z
2024-07-20T11:26:34.790Z
Terry Upton
<p>I am looking for a bit of support and guidance with caching.
Specifically where to place queries, how to re-use them and structure this across a project.</p>
<p>So the current project has no caching at all and I am looking to add it in.
My initial setup was using Andrew Welches 'Globals' approach, where a single <code>global-variables.twig</code> file is used and is extended for the whole site.</p>
<p>Within this I have the following;</p>
<pre><code>{% set aboutPages = craft.entries.section(['about']).cache().all() %}
{% set helpPages = craft.entries.section(['help']).cache().all() %}
{% set contactPage = craft.entries.section('contact').cache().one() ?? null %}
{% set newsPage = craft.entries.section('newsIndex').cache().one() ?? null %}
{% set officesPage = craft.entries.section('officesIndex').cache().one() ?? null %}
{% set insightsBusinessPage = craft.entries.section('insightsIndexBusiness').cache().one() ?? null %}
{% set insightsPersonalPage = craft.entries.section('insightsIndexPersonal').cache().one() ?? null %}
{% set notaryPublicPage = craft.entries.id('29586').cache().one() ?? null %}

{% set forBusinessNavigation = craft.entries
.section('servicesBusiness')
.level(1)
.with([
   ['children', { orderBy: &quot;title ASC&quot; }]
])
.cache()
.all() %}

{% set forIndividualsNavigation = craft.entries
.section('servicesPersonal')
.level(1)
.with([
   ['children', { orderBy: &quot;title ASC&quot; }]
])
.cache()
.all() %}

{% set aboutNavigation = aboutPages|merge([officesPage,newsPage]) %}
{% set careersNavigation = craft.entries.section(['careersIndex', 'careers']).cache().all() %}
{% set termsNavigation = craft.entries.section(['terms']).cache().all() %}
{% set helpNavigation = helpPages|merge([contactPage]) %}
</code></pre>
<p>Originally I did not have the <code>.cache()</code> but after learning about this a few months ago I added it.</p>
<p>My understanding is that this will approach will not work very well with the native <code>{% cache %}</code> tag for the following reasons:</p>
<ul>
<li>It is not caching these queries and adding a <code>{% cache %}</code> around these will not help.</li>
<li>It won't clear caches if the element queries are not sat within the <code>{% cache %}</code> tag.</li>
</ul>
<p>So do I need to move each of these lines into the necessary files?
So into the my <code>header.twig</code> and into my <code>footer.twig</code> like so:</p>
<p><code>header.twig</code></p>
<pre><code>{% cache globally using key 'header' %}
   {% set aboutPages = craft.entries.section(['about']).cache().all() %}
   {% set helpPages = craft.entries.section(['help']).cache().all() %}
   {% set contactPage = craft.entries.section('contact').cache().one() ?? null %}
   {% set newsPage = craft.entries.section('newsIndex').cache().one() ?? null %}
   {% set officesPage = craft.entries.section('officesIndex').cache().one() ?? null %}
   {% set insightsBusinessPage = craft.entries.section('insightsIndexBusiness').cache().one() ?? null %}
   {% set insightsPersonalPage = craft.entries.section('insightsIndexPersonal').cache().one() ?? null %}
   {% set notaryPublicPage = craft.entries.id('29586').cache().one() ?? null %}

   {% set forBusinessNavigation = craft.entries
      .section('servicesBusiness')
      .level(1)
      .with([
         ['children', { orderBy: &quot;title ASC&quot; }]
      ])
      .cache()
      .all() %}

   {% set forIndividualsNavigation = craft.entries
      .section('servicesPersonal')
      .level(1)
      .with([
         ['children', { orderBy: &quot;title ASC&quot; }]
      ])
      .cache()
      .all() %}

   {% set aboutNavigation = aboutPages|merge([officesPage,newsPage]) %}
   {% set careersNavigation = craft.entries.section(['careersIndex', 'careers']).cache().all() %}
   {% set termsNavigation = craft.entries.section(['terms']).cache().all() %}
   {% set helpNavigation = helpPages|merge([contactPage]) %}

HTML header code ..
{% endcache %}
</code></pre>
<p><code>footer.twig</code></p>
<pre><code>{% cache globally using key &quot;footer&quot; until lastDay %}
   {% set termsNavigation = craft.entries.section(['terms']).cache().all() %}
   {% set helpNavigation = helpPages|merge([contactPage]) %}
Footer HML `code
{% endcache %}
</code></pre>
<p>I have other templates that this will affect like the service pages where I use</p>
<pre><code>{% set insightsBusinessPage = craft.entries.section('insightsIndexBusiness').cache().one() ?? null %}
{% set insightsPersonalPage = craft.entries.section('insightsIndexPersonal').cache().one() ?? null %}
</code></pre>
<p>The second part of this question is the <code>.cache()</code> - do I need to use this on ALL versions of the query? If not then which one is it best to set this on initially?</p>
<p>What is the best way to add these queries to keep the, as DRY but make them cacheable?</p>
<p>Thanks.
Terry</p>

<templating><cache>
Re-using element queries in templates, where and how when using native Cache tags?
# Re-using element queries in templates, where and how when using native Cache tags?

(Asked by: username_0 on 2024-07-19)

I am looking for a bit of support and guidance with caching.
Specifically where to place queries, how to re-use them and structure this across a project.
So the current project has no caching at all and I am looking to add it in.
My initial setup was using Andrew Welches 'Globals' approach, where a single global-variables.twig file is used and is extended for the whole site.
Within this I have the following;
{% set aboutPages = craft.entries.section(['about']).cache().all() %}
{% set helpPages = craft.entries.section(['help']).cache().all() %}
{% set contactPage = craft.entries.section('contact').cache().one() ?? null %}
{% set newsPage = craft.entries.section('newsIndex').cache().one() ?? null %}
{% set officesPage = craft.entries.section('officesIndex').cache().one() ?? null %}
{% set insightsBusinessPage = craft.entries.section('insightsIndexBusiness').cache().one() ?? null %}
{% set insightsPersonalPage = craft.entries.section('insightsIndexPersonal').cache().one() ?? null %}
{% set notaryPublicPage = craft.entries.id('29586').cache().one() ?? null %}

{% set forBusinessNavigation = craft.entries
.section('servicesBusiness')
.level(1)
.with([
   ['children', { orderBy: "title ASC" }]
])
.cache()
.all() %}

{% set forIndividualsNavigation = craft.entries
.section('servicesPersonal')
.level(1)
.with([
   ['children', { orderBy: "title ASC" }]
])
.cache()
.all() %}

{% set aboutNavigation = aboutPages|merge([officesPage,newsPage]) %}
{% set careersNavigation = craft.entries.section(['careersIndex', 'careers']).cache().all() %}
{% set termsNavigation = craft.entries.section(['terms']).cache().all() %}
{% set helpNavigation = helpPages|merge([contactPage]) %}

Originally I did not have the .cache() but after learning about this a few months ago I added it.
My understanding is that this will approach will not work very well with the native {% cache %} tag for the following reasons:

It is not caching these queries and adding a {% cache %} around these will not help.
It won't clear caches if the element queries are not sat within the {% cache %} tag.

So do I need to move each of these lines into the necessary files?
So into the my header.twig and into my footer.twig like so:
header.twig
{% cache globally using key 'header' %}
   {% set aboutPages = craft.entries.section(['about']).cache().all() %}
   {% set helpPages = craft.entries.section(['help']).cache().all() %}
   {% set contactPage = craft.entries.section('contact').cache().one() ?? null %}
   {% set newsPage = craft.entries.section('newsIndex').cache().one() ?? null %}
   {% set officesPage = craft.entries.section('officesIndex').cache().one() ?? null %}
   {% set insightsBusinessPage = craft.entries.section('insightsIndexBusiness').cache().one() ?? null %}
   {% set insightsPersonalPage = craft.entries.section('insightsIndexPersonal').cache().one() ?? null %}
   {% set notaryPublicPage = craft.entries.id('29586').cache().one() ?? null %}

   {% set forBusinessNavigation = craft.entries
      .section('servicesBusiness')
      .level(1)
      .with([
         ['children', { orderBy: "title ASC" }]
      ])
      .cache()
      .all() %}

   {% set forIndividualsNavigation = craft.entries
      .section('servicesPersonal')
      .level(1)
      .with([
         ['children', { orderBy: "title ASC" }]
      ])
      .cache()
      .all() %}

   {% set aboutNavigation = aboutPages|merge([officesPage,newsPage]) %}
   {% set careersNavigation = craft.entries.section(['careersIndex', 'careers']).cache().all() %}
   {% set termsNavigation = craft.entries.section(['terms']).cache().all() %}
   {% set helpNavigation = helpPages|merge([contactPage]) %}

HTML header code ..
{% endcache %}

footer.twig
{% cache globally using key "footer" until lastDay %}
   {% set termsNavigation = craft.entries.section(['terms']).cache().all() %}
   {% set helpNavigation = helpPages|merge([contactPage]) %}
Footer HML `code
{% endcache %}

I have other templates that this will affect like the service pages where I use
{% set insightsBusinessPage = craft.entries.section('insightsIndexBusiness').cache().one() ?? null %}
{% set insightsPersonalPage = craft.entries.section('insightsIndexPersonal').cache().one() ?? null %}

The second part of this question is the .cache() - do I need to use this on ALL versions of the query? If not then which one is it best to set this on initially?
What is the best way to add these queries to keep the, as DRY but make them cacheable?
Thanks.
Terry

**Tags:** templating, cache


---

## 2 answers

### Answer by username_1 on 2024-07-19. Score: 3

What's the reasoning behind putting all queries in a single file? That looks totally chaotic to me. You're defining a bunch of variables that you might not even need for the current request, and then presumably using those variables in other templates. A new dev looking at the codebase will be wondering where those variables are coming from. Whenever you need a new query, you add it to an ever-growing list of global query results? Who will check if you still need all of those? How will you know whether you can safely delete or modify one of those queries, if you can't tell where it might be used?
Looks like a great opinionated setup if you're working alone, but not something I would ever consider if there's a chance another dev will need to work on this codebase.

Regarding the .cache() vs {% cache %} question, it helps to understand what both of them do:

.cache() will cache the result of that particular query with those exact parameters and return this cache in future queries using the same parameters. It will also register a cache dependency on the queries elements, so if any of those change, the cache is automatically invalidated.
{% cache %} will cache the entire template output of the Twig code inside it and some of its side-effects. In particular, JS and CSS assets or code snippets registered through the {% js %} and {% css %} tags will be output on subsequent requests as well. The tag also automatically registers all element queries executed inside the tag and add cache dependencies on those. So if any elements that are used inside the cache change, the cache is invalidated as well.

I would not use both in conjunction, i.e. not cache individual queries when you're already using the {% cache %} tag. This gets confusing, I'm not sure how those will interact or if the cache dependencies will be picked up correctly. For the most part, the element query cache won't be of much use, since the cache tag will prevent those queries from being executed at all during a cache hit.
I rarely if ever use .cache(). The savings on a single query are just lackluster compared to caching entire parts of the page with the {% cache %} tag.

The second part of this question is the .cache() - do I need to use this on ALL versions of the query? If not then which one is it best to set this on initially?

You need to call .cache() on all queries that you want to cache the results of and be able to return cached results on subsequent requests. Any query where you're not calling .cache() will not be able to return cached results, even if the exact same query with the same parameters has already been cached in another place or during another request. So you need to use .cache() on all queries that you want to utilize the cache, not just the first occurrence of the query.

What is the best way to add these queries to keep the, as DRY but make them cacheable?

In most cases, I would just use the {% cache %} tag. This works automatically since it picks up element queries inside the tag and adds them as dependencies, so you rarely have to worry about cache invalidation. It's also much more efficient. For example, say you're caching an entire navigation with a mega menu, some related content, etc. The template might have dozens of queries, and the {% cache %} tag will reduce that to a single DB request or file read on cache hits. Compared to that, element queries cache much less, result in more queries and more of your template being executed for every request.
I would consider element query caching only in some cases:

You have a highly dynamic site with lots of interactive elements that depend on user accounts or cookies, so you can't cache entire areas of the page.
You have a site with very frequent content edits (say, a news outlet that posts frequently). The {% cache %} tag might be insufficient because those caches will be invalidated very frequently. Meanwhile, caches for element queries that query content that is changed less frequently can be used for a longer time.

Regarding DRY templates, with the {% cache %} tag, sometimes you have to repeat queries. This happens most often if you want to use a query inside multiple {% block %} tags in a Twig template, you can't wrap the {% cache %} tag around a block. For example:
{% set results = craft.entries().search(query).all() %}

{% block header %}
{% cache %}
    Found {{ results|length }} results. 
{% endcache %}
{% endblock %}

{% block content %}
{% cache %}

    {% for result in results %}
        {{ result.title }}
    {% endfor %}

{% endcache %}
{% endblock %}

This will not work because the query is executed outside the cache tags, so it will not be picked up as a dependency, leading to stale results. But you also can't move the query inside the first cache tag, because then the second block doesn't register the dependencies. In this case, you need to execute the query twice:
{% set resultsQuery = craft.entries().search(query) %}

{% block header %}
{% cache %}
    Found {{ clone(resultsQuery).count() }} results. 
{% endcache %}
{% endblock %}

{% block content %}
{% cache %}

    {% for result in clone(resultsQuery).all() %}
        {{ result.title }}
    {% endfor %}

{% endcache %}
{% endblock %}

This will work because while the query is created outside the cache tags, it is only executed inside them. This is technically inefficient, because you have one superfluous query. But now both areas can be cached correctly, and as a result cache hits will load faster.
In general, I would keep queries in the templates where they are used and not outsource them to one god template somewhere. This is much easier to understand, and those queries will play nicer with the {% cache %} tag and behave predictably.


---

### Answer by username_0 on 2024-07-20. Score: 0

Thanks for the in depth response @username_1

So, yes I am a solo developer so it didn't concern me too much about other developers, as per your first point. But I see what you are saying here.

The main reason I made the decision to put these queries into the globals was due to most of these queries being used in the navigation which is global AND some also reused in the footer (also global), but several are then also used in templates in block tags so rather than writing duplicate queries, it made sense to have the available via the global file, for easy re-use.

My reason for doing this was so I didn't have the inefficiency of repeating several queries multiple times, especially as I wasn't doing any caching initially. I know that this approach wasn't sensible long term and hence the main point of my question and why I am looking to change this now I want to add some caching.

I understand the difference between the `{% cache %}` tag and the `.cache()`
My question was perhaps not phrased too well.

In regards to the `.cache()` - it seems to make sense to use this for the new approach when I am repeating the element queries.  
What I wasn't sure about was if each repeated query should contain the `.cache()` or more-so which version of the query to add this to?
Is it a 'first seen' approach from Craft? So logically this should likely be added to the one in the `header.twig` as this is likely the one seen first.

I have used `clone()` before in other projects, again to reduce duplicated queries. However, I thought that `.cache()` would be more efficient? It is interesting how you have suggested this and this would have perhaps worked better with the original globals approach; but having now moved and duplicated the queries I wonder if `.cache()` is a better option?

> In general, I would keep queries in the templates where they are used and not outsource them to one god template somewhere. This is much easier to understand, and those queries will play nicer with the {% cache %} tag and behave predictably.

I agree with this sentiment too and definitely an approach I am going to use moving forward. I suppose in essence, a duplicated query isn't too much to be concerned about when using `{% cache %}` if it is efficient, then it should only affect the first load as after this it will be cached and it's a non-issue.

Thanks for the help. Happy to discuss in more detail here or in Discord.

> **Comments:**
> **username_1**: Regarding where to use `.cache()`, you need it on all queries that you want to use the cache. I had mentioned this in my reply, but a bit briefly, probably got lost in the wall of text. I've expanded on that paragraph above to make it more clear! I'll follow up on the other point on Discord.  

---

