DrupalEasy: Understanding common cache-related HTTP response headers

Planet Drupal - 3 hours 21 min ago

Having a basic understanding of caching is a requirement of being a professional Drupal developer. Unfortunately, there can be many layers of caching which can make it challenging to figure out exactly how best to configure cache settings and troubleshoot potential caching issues.

Web page caching can be thought of as moats around the castle, where each moat is a caching layer and the castle can be thought of as the site's web, database, and other origin servers.

HTTP headers provide a mechanism for the client (usually a web browser) and a server (usually a web server) to exchange information that is not visible to the end user, nor included in the HTML of the page. While there are few types of headers, cache-related headers are generally "response headers".

This blog post isn't meant to be a comprehensive guide to all layers of web page caching, rather we'll focused on common, cache-related HTTP response headers. Information in these response headers can help developers identify which caching layers, if any, were utilized in a given page request. Specifically, we'll be looking at how to view this response header information, and what it can tell us about Drupal page caching and content delivery network (CDN) caching activity on a per-request basis.

Keep in mind that there is not a constrained list of headers that can be added to a page request or response. While there is a common set defined by the Internet Assigned Numbers Authority (IANA) additional proprietary (custom) headers can be added by developers and vendors. It is also important to note that header names are case-insensitive, so the header names "Cache-Control" and "cache-control" are equivalent.

Instructions for viewing cache-related HTTP headers is included at the bottom of this article.

Categories of cache-related response headers

Cache-related response headers can be categorized as either "standard" or "proprietary". Standard headers are those defined by IANA while proprietary headers are added by developers and vendors. Below is a sample of some of the commonly-used cache-related response headers:

Standard (non-proprietary) cache-related response headers
  • Age: the time (seconds) that the object (file) has been cached.
  • Cache-control: general directives for caching mechanisms. For example, if this is set to "no-cache", then the file won't be cached.
  • Expires: the date/time after which the response will be considered stale, and then refreshed from the source. For Drupal sites, by default, this value is set to Dries Buytaert's birth date.
Common proprietary response headers

Again, these are not industry-standard response headers, and different vendors (hosting companies, content delivery networks) may have different implementations. It is best to refer to your specific vendor's documentation for additional details.

  • x-drupal-cache: Drupal-specific header set by Drupal core, values include "HIT", "MISS", and "UNCACHEABLE". Provided by the core Internal Page Cache module, applies only to anonymous users.
  • x-drupal-dynamic-cache: Drupal-specific header set by Drupal core, values include "HIT", "MISS", and "UNCACHEABLE". Provided by the core Internal Dynamic Page Cache module, applies to both anonymous and authenticated users. Allows for partial responses. See blog post by Wim Leers for more information.
  • x-served-by: this header is added by various tools to (normally) indicate the server that provided the response. This includes some (but not all) hosting companies and content delivery networks.
  • x-cache: in general, this indicates if the file was served by a CDN or the origin server. Multiple x-cache values often correspond to multiple "x-served-by" values.
  • x-cache-hits: for some CDNs, this value closely mirrors the "x-cache" value, only with a value of "0" being equivalent to "MISS" and any positive value equivalent to a "HIT" (the value being equal to the number of tries before a hit).
  • cf-cache-status: set by Cloudflare CDN, indicates the status of a cache request. Values include "HIT", "MISS", "DYNAMIC", "EXPIRED", and others. Note that in its default configuration, Cloudflare will not cache HTML - only things like static images.
  • cf-ray, cf-request-id: set by Cloudflare CDN to help trace requests through the Cloudflare network. Used for troubleshooting.

At this point, it is important to note an important difference between two of the more widely-used content delivery networks, Fastly and Cloudflare. Fastly is primarily a content delivery network built on top of the Varnish web application accelerator. While Fastly does have some web application firewall (WAF) capabilities, Cloudflare is both a content delivery network and a full, security-focused web application firewall provider. In some cases (including for DrupalEasy.com), both Fastly and Cloudflare can be used - Fastly for its CDN and Cloudflare for its WAF.

Examples

Below are a couple of examples of cache-related response headers along with a short discussion about what they mean.

Drupal.org cache-related response header values for HTML page Authenticated user

Cache-Control: no-cache, must-revalidate x-served-by: cache-sea4481-SEA, cache-mia11376-MIA x-cache-hits: 0, 0 x-cache: MISS, MISS

Non-cached page served from the drupal.org "origin server" (located at Oregon State University) due to the "Cache-Control" value.

The "x-cache" values indicate that for authenticated users, Drupal.org's CDN (Fastly) did not serve the page. The first "x-served-by" value is the Fastly server closest to the origin server, while the second value is the Fastly server closest to the user loading the page. While "x-cache" indicates that a cached version of the page was not used, it was Fastly's servers that initially processed the request and ultimately retrieved the page from the origin server.

Anonymous user

Cache-Control: public, max-age=900 x-drupal-cache: MISS x-served-by: cache-sea4420-SEA, cache-mia11369-MIA x-cache-hits: 1, 4 x-cache: HIT, HIT

The "Cache-Control" value provided by the origin server indicates that the page can be cached for up to 900 seconds (15 minutes). The "x-cache" values indicate that the cached page was served by the Fastly server closest to the user ("cache-mia11369-MIA" in this example).

DrupalEasy.com cache-related response header values for HTML page Authenticated user

Cache-Control: no-cache x-served-by: cache-mdw17368-MDW, cache-mia11364-MIA x-drupal-dynamic-cache: UNCACHEABLE x-cache-hits: 0, 0 x-cache: MISS, MISS cf-cache-status: DYNAMIC

DrupalEasy is hosted on Pantheon (which includes the Fastly CDN) and uses the Cloudflare CDN. The "x-cache" values indicate that for authenticated users, Pantheon's default CDN (Fastly) did not serve the page. The first "x-served-by" value is the Fastly server close to the origin, while the second value is the Fastly server close to the user loading the page. While "x-cache" indicates that a cached version of the page was not used, it was Fastly's servers that processed the request and ultimately retrieved the page from the origin (Pantheon) server.

The "cf-cache-status" of "DYNAMIC" indicates that the file is not eligible for caching by Cloudflare. This is normally a result of a directive from the origin server to never cache the file ("Cache-Control: no-cache").

Anonymous user

cache-control: max-age=60, public x-served-by: cache-mdw17325-MDW, cache-mia11383-MIA x-drupal-cache: HIT x-drupal-dynamic-cache: MISS x-cache-hits: 0, 1 x-cache: MISS, HIT cf-request-id: 0775a92a710000e958291b2000000001 cf-ray: 60cfaaf0aa52e958-MIA cf-cache-status: DYNAMIC

For an anonymous user, the "x-cache" value of "MISS, HIT" indicates that the 2nd "x-served-by" value (the Fastly server close to the user) provided a cached response.

DrupalEasy.com cache-related response header values for image (site logo) Anonymous and authenticated users

Cache-Control: max-age=31622400 Expires: Fri, 31 Dec 2021 17:54:40 GMT x-served-by: cache-mdw17349-MDW, cache-mia11320-MIA x-cache-hits: 1, 1 x-cache: HIT, HIT cf-cache-status: HIT

The "cf-cache-status" value of "HIT" indicates that Cloudflare served this image. The "Cache-Control" value indicates that the image can be cached for up to 3162240 seconds (< 36 days). Note that the image was cached every step of the way on both Fastly and Cloudflare.

Manually "bust" the cache

During development, sometimes it is necessary to "bust" the cache - that is, to request an uncached version of the content. While this can be done by clearing/purging/rebuilding caches at every level of the site's infrastructure (Drupal, Fastly, Cloudflare), it can also be done by adding a unique querystring variable to the content as they are cached according to their unique URL, including querystring variables. So, if you want to request an uncached version of "logo.png", simply append a meaningless querystring variable to it - for example, "logo.png?buster=2358973430985234895".

Summary

As the previous examples show, in order to be able to make sense of cache-related response headers, it is important to know which, if any CDNs are being used as well as which, if any, Drupal-level caching modules are enabled.

Resources Viewing cache-related HTTP headers

All major browsers provide relatively easy access to HTTP header information, if you know where to look. Here's a quick guide:

Safari
  • Open Web Inspector ("Inspect Element" anywhere)
  • Go to "Network" tab.
  • Reload page.
  • Click on (usually) first entry in list of files - this will be the main HTML file for the page.
  • Click on the "Headers" sub-tab.
Firefox
  • Open the network tools via the "Tools|Web Developer|Network" menu item.
  • Reload page.
  • Click on (usually) first entry in list of files - this will be the main HTML file for the page.
  • Click on the "Headers" sub-tab.
Chrome
  • Open the network tools via the "View|Developer|Developer Tools" menu item.
  • Go to "Network" tab.
  • Reload page.
  • Click on (usually) first entry in list of files - this will be the main HTML file for the page.
  • Click on the "Headers" sub-tab.

Edge
  • Open Developer Tools (Crtl-Shift-I).
  • Go to "Network" tab.
  • Reload page.
  • Click on (usually) first entry in list of files - this will be the main HTML file for the page.
  • Click on the "Headers" sub-tab.

Thanks to Andy Giles from Blue Oak Interactive for reviewing this blog post.

Categories:

Promet Source: Documenting Web Accessibility: Do You Need a VPAT?

Planet Drupal - 4 hours 47 min ago
It has become increasingly common to find located in the footer of many websites a link to their Statement of Accessibility. In a few cases you will find some sites with a VPAT (Voluntary Product Accessibility Template) and Statement of Accessibility linked. What are these documents? What is their purpose and should your site have one or both?   
Categories:

clemens-tolboom commented on pull request boku-ilen/geodot-plugin#33

On github - Mon, 2021/01/25 - 6:50pm
clemens-tolboom commented on pull request boku-ilen/geodot-plugin#33 Jan 25, 2021 clemens-tolboom commented Jan 25, 2021

We could try manual copy the library from /usr/local/Cellar/gdal/3.2.1/lib/libgdal.dylib

clemens-tolboom commented on issue boku-ilen/geodot-plugin#36

On github - Mon, 2021/01/25 - 6:34pm
clemens-tolboom commented on issue boku-ilen/geodot-plugin#36 Jan 25, 2021 clemens-tolboom commented Jan 25, 2021

.gitattributes is key. addons/geodot should be rooted imho I prefer the demo to be in addons/geodot/demo which can be ignored by the user importing …

clemens-tolboom opened a pull request in pytorch/tutorials

On github - Mon, 2021/01/25 - 5:32pm
clemens-tolboom opened a pull request in pytorch/tutorials Jan 25, 2021 Lets make a copyright jump into 2021 #1318

It looks way better to have copyright extends into current year ;-)

+1 -1

clemens-tolboom pushed to patch-1 in clemens-tolboom/tutorials

On github - Mon, 2021/01/25 - 5:31pm
clemens-tolboom pushed to patch-1 in clemens-tolboom/tutorials Jan 25, 2021 1 commit to patch-1
  • 006cced Lets make a copyright jump into 2021

clemens-tolboom forked clemens-tolboom/tutorials from pytorch/tutorials

On github - Mon, 2021/01/25 - 5:29pm
clemens-tolboom forked clemens-tolboom/tutorials from pytorch/tutorials Jan 25, 2021 pytorch/tutorials

PyTorch tutorials.

Jupyter Notebook 4.6k Updated Jan 25

Community Working Group posts: Encouraging healthy conversations in the Drupal community

Planet Drupal - Mon, 2021/01/25 - 4:11pm

If you've spent any time in the Drupal community (specifically the issue queues, forums, and Drupal Slack workspace channels), you may have noticed that while the vast majority of discussions are positive, every now and then tempers flare up or less-than-helpful comments are posted. 

To continue to grow a healthy community, we all must work under the assumption that no one intentionally uses language to hurt others. Even so, despite our best efforts we sometimes still use words or phrases that are discouraging, harmful, or offensive to others. We are all human beings who make mistakes, but as members of a shared community, it’s our responsibility to lift each other up and encourage the best in each other. 

When observing negative behavior in the Drupal community, it can be difficult and/or uncomfortable to interject ourselves and find the right words to get conversations moving in a healthy direction. To address this, starting with an idea discussed at a Community Working Group (CWG) workshop at DrupalCon Seattle, the CWG Community Health team has been working on a communication initiative for the Drupal community. It consists of a series of de-escalation templates we have dubbed “Nudges”.

“Nudges”

The team has written five nudges that community members can use when they are faced with one of these uncomfortable situations in the Drupal community. We selected topics that we feel can have the most impact and are easy to understand. Each “nudge” provides context as to why the comment may be harmful and relevant links including our Code of Conduct and Values and Principles.

The “nudges” created to date are as follows:

Inclusive language - gendered terms

This discussion appears to include the use of gendered language in a comment. Gendered language can be harmful to our community because it can signal that we assume that people’s participation in the community is determined by gender.

The shift to gender-neutral language promotes gender equality. Please respect the pronouns that community members provide in their profiles.

For more information, please refer to Drupal’s Values and Principles about treating each other with dignity and respect.

This comment is provided as a service (currently being tested) of the Drupal Community Health Team as part of a project to encourage all participants to engage in positive discourse. For more information, please visit https://www.drupal.org/project/drupal_cwg/issues/3129687

Inclusive language - ableist terms

This discussion appears to include the use of ableist language in a comment. Ableist language can be harmful to our community because it can devalue challenges experienced by people with disabilities.

For more information, please refer to Drupal’s Values and Principles about treating each other with dignity and respect.

This comment is provided as a service (currently being tested) of the Drupal Community Health Team as part of a project to encourage all participants to engage in positive discourse. For more information, please visit https://www.drupal.org/project/drupal_cwg/issues/3129687

Gatekeeping knowledge

This discussion appears to discourage contribution from some participants by assuming a level of education or exposure to information. This can discourage folks that are new to a conversation from asking for help or getting involved. Consider providing the new contributor with links to help them learn the topic or concept.

For more information, please refer to Drupal’s Values and Principles #3 of fostering a learning environment: long-term contributors should be role models by admitting their own shortcomings and mistakes, being vulnerable, and by giving others the same respect that was once given to them. To keep learning read this article about not feigning surprise.

Principle #9 can also be useful here: be constructively honest, and relentlessly optimistic. You can be optimistic and supportive by giving suggestions for how to improve their contributions. By being helpful, you encourage people to accept feedback and act on it.

This comment is provided as a service (currently being tested) of the Drupal Community Health Team as part of a project to encourage all participants to engage in positive discourse. For more information, please visit https://www.drupal.org/project/drupal_cwg/issues/3129687

Cultural differences

This discussion appears to have escalated due to the inclusion of culturally-specific language. This may be harmful to our community as stereotypes can exclude or tokenize community members, devaluing their individual contribution and value. Please avoid language that imposes an identity on an individual or group.

We should be mindful that not all words and phrases translate to have the same meaning. Please be mindful that different cultures may appreciate different levels of directness in their communication styles and that may impact how words are presented or interpreted.

For more information, please refer to Drupal’s Values and Principles about treating each other with dignity and respect.

This comment is provided as a service (currently being tested) of the Drupal Community Health Team as part of a project to encourage all participants to engage in positive discourse. For more information, please visit https://www.drupal.org/project/drupal_cwg/issues/3129687

Escalating emotions

This discussion appears to include escalating emotions, creating the opportunity for miscommunication. The invested parties are encouraged to take a break from this discussion to help gain perspective. It is important to the community that all members are shown the appropriate amount of respect and openness when working together. Additionally, there are resources offered by the Drupal community to aid conflict resolution should those be needed.

For more information, please refer to Drupal’s Values and Principles of seeking first to understand, then to be understood. Assume best intentions of other contributors and suspend judgment until you have invested time to understand decisions, ask questions, and listen. Before expressing a disagreement, make a serious attempt to understand the reasons behind the decision.

This comment is provided as a service (currently being tested) of the Drupal Community Health Team as part of a project to encourage all participants to engage in positive discourse. For more information, please visit https://www.drupal.org/project/drupal_cwg/issues/3129687

Testing strategy

Over the next few months, members of the Community Health Team will be testing these "nudges” in Drupal.org issue queues, forums, and Drupal Slack workspace channels. For now, this will be done by copying and pasting the nudge text and formatting directly into an issue comment or Slack message. Our goal is to create a nudge widget that allows community members to insert one of the pre-written nudges via a token in issue queue and forum comments.

We feel that a gradual introduction to help prevent overuse of these nudges is necessary as overuse may produce its own sort of gatekeeping. This could lead to desensitization to the “nudge” initiative, negating our efforts. The CWG’s mission is to ensure a positive space that is inclusive to all of our members. 

These are early days for “nudges”. We are looking forward to hearing from other community members on how we can improve on this idea. We have been having weekly meetings for nudges and have been documenting everything we've been doing in the Community Working Group issue queue - let us know if you'd like to get involved!

Thanks

Current community members who have been working on this task include: AmyJune Hineline (volkswagenchick), Neil Drumm (drumm), George Matthes (watsonerror), Darren Oh, Donna Bungard, JD Flynn (dorficus), George DeMet, and Mike Anello (ultimike).
 

Categories:

clemens-tolboom commented on pull request boku-ilen/geodot-plugin#33

On github - Mon, 2021/01/25 - 2:56pm
clemens-tolboom commented on pull request boku-ilen/geodot-plugin#33 Jan 25, 2021 clemens-tolboom commented Jan 25, 2021

Hmmm ... moving in the demo root makes running the scene crash :-) Registered camera FaceTime HD Camera (Display) with id 4 position 0 at index 3 …

clemens-tolboom commented on pull request boku-ilen/geodot-plugin#33

On github - Mon, 2021/01/25 - 2:49pm
clemens-tolboom commented on pull request boku-ilen/geodot-plugin#33 Jan 25, 2021 clemens-tolboom commented Jan 25, 2021

Just a guess: Could it be that it's expecting libRasterTileExtractor.dylib to be in a directory called build for some reason, but it's actually in…

clemens-tolboom commented on pull request boku-ilen/geodot-plugin#33

On github - Mon, 2021/01/25 - 2:35pm
clemens-tolboom commented on pull request boku-ilen/geodot-plugin#33 Jan 25, 2021 clemens-tolboom commented Jan 25, 2021

I guess I should have created new issues for my remarks ... not sure what comment is what now :( See new issues #38, #39,#40

clemens-tolboom opened an issue in boku-ilen/geodot-plugin

On github - Mon, 2021/01/25 - 2:32pm
clemens-tolboom opened an issue in boku-ilen/geodot-plugin Jan 25, 2021 Why are both SConstruct files different for windows? #38

diff src/vector-extractor/SConstruct src/raster-tile-extractor/SConstruct

clemens-tolboom opened an issue in boku-ilen/geodot-plugin

On github - Mon, 2021/01/25 - 2:30pm
clemens-tolboom opened an issue in boku-ilen/geodot-plugin Jan 25, 2021 Make sure godot-cpp is build same as our target #37

scons target=debug should build godot-cpp similar. The instructions are not covering switching from debug to release. Can't we add this to main SCo…

clemens-tolboom commented on pull request boku-ilen/geodot-plugin#33

On github - Mon, 2021/01/25 - 12:30pm
clemens-tolboom commented on pull request boku-ilen/geodot-plugin#33 Jan 25, 2021 clemens-tolboom commented Jan 25, 2021

Running scons target=d osgeo_path=/usr/local/Cellar/gdal/3.2.1 solved building. Godot ERROR: open_dynamic_library: Can't open dynamic library: /Use…