trAvis - MANAGER

Edit File: changes-5.1.html

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
 "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 
 <title>Changes in Varnish 5.1 &#8212; Varnish version 5.2.1 documentation</title>
 
 <link rel="stylesheet" href="../_static/classic.css" type="text/css" />
 <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
 
 <script type="text/javascript">
 var DOCUMENTATION_OPTIONS = {
 URL_ROOT: '../',
 VERSION: '5.2.1',
 COLLAPSE_INDEX: false,
 FILE_SUFFIX: '.html',
 HAS_SOURCE: true
 };
 </script>
 <script type="text/javascript" src="../_static/jquery.js"></script>
 <script type="text/javascript" src="../_static/underscore.js"></script>
 <script type="text/javascript" src="../_static/doctools.js"></script>
 <link rel="index" title="Index" href="../genindex.html" />
 <link rel="search" title="Search" href="../search.html" />
 <link rel="top" title="Varnish version 5.2.1 documentation" href="../index.html" />
 <link rel="up" title="What&#39;s new / Upgrading" href="index.html" />
 <link rel="next" title="Upgrading to Varnish 5.1" href="upgrading-5.1.html" />
 <link rel="prev" title="Upgrading to Varnish 5.2" href="upgrading-5.2.html" /> 
 </head>
 <body role="document">
 <div class="related" role="navigation" aria-label="related navigation">
 <h3>Navigation</h3>
 <ul>
 <li class="right" style="margin-right: 10px">
 <a href="../genindex.html" title="General Index"
 accesskey="I">index</a></li>
 <li class="right" >
 <a href="upgrading-5.1.html" title="Upgrading to Varnish 5.1"
 accesskey="N">next</a> |</li>
 <li class="right" >
 <a href="upgrading-5.2.html" title="Upgrading to Varnish 5.2"
 accesskey="P">previous</a> |</li>
 <li class="nav-item nav-item-0"><a href="../index.html">Varnish version 5.2.1 documentation</a> &#187;</li>
 <li class="nav-item nav-item-1"><a href="index.html" accesskey="U">What's new / Upgrading</a> &#187;</li> 
 </ul>
 </div>

<div class="document">
 <div class="documentwrapper">
 <div class="bodywrapper">
 <div class="body" role="main">
 
 <div class="section" id="changes-in-varnish-5-1">
<h1>Changes in Varnish 5.1<a class="headerlink" href="#changes-in-varnish-5-1" title="Permalink to this headline">¶</a></h1>
We have a couple of new and interesting features in Varnish 5.1,
and we have a lot of smaller improvements and bugfixes all over
the place, in total we have made about 750 commits since Varnish 5.0,
so this is just some of the highlights.
Probably the biggest change in Varnish 5.1 is that a couple of very
significant contributors to Varnish have changed jobs, and therefore
stopped being active contributors to the Varnish Project.
Per Buer was one of the first people who realized that Varnish was
not just &quot;Some program for a couple of Nordic newspapers&quot;, and he
started the company Varnish Software, which is one of the major
sponsors of the Varnish Project.
Lasse Karstensen got roped into Varnish Software by Per, and in
addition to his other duties, he has taken care of the projects
system administration and release engineering for most of the 11
years we have been around now.
Per &amp; Lasse: &quot;Thanks&quot; doesn't even start to cover it, and we wish
you all the best for the future!
<div class="section" id="startup-cli-command-file">
<h2>Startup CLI command file<a class="headerlink" href="#startup-cli-command-file" title="Permalink to this headline">¶</a></h2>
The new '-I cli_file' option to varnishd will make it much more
practical to use the VCL labels introduced in Varnish 5.0.
The cli commands in the file will be executed before the worker
process starts, so it could for instance contain:
<div class="highlight-default"><div class="highlight"><pre>vcl.load panic /etc/varnish_panic.vcl
vcl.load siteA0 /etc/varnish_siteA.vcl
vcl.load siteB0 /etc/varnish_siteB.vcl
vcl.load siteC0 /etc/varnish_siteC.vcl
vcl.label siteA siteA0
vcl.label siteB siteB0
vcl.label siteC siteC0
vcl.load main /etc/varnish_main.vcl
vcl.use main
</pre></div>
</div>
If a command in the file is prefixed with '-', failure will not
abort the startup.
Related to this change we have reordered the argument checking so
that argument problems are reported more consistently.
In case you didn't hear about them yet, labelling VCL programs
allows you to branch out to other VCLs in the main::vcl_recv{},
which in the above example could look like:
<div class="highlight-default"><div class="highlight"><pre>sub vcl_recv {
 if (req.http.host ~ &quot;asite.example.com$&quot;) {
 return(vcl(siteA));
 }
 if (req.http.host ~ &quot;bsite.example.com$&quot;) {
 return(vcl(siteB));
 }
 if (req.http.host ~ &quot;csite.example.com$&quot;) {
 return(vcl(siteC));
 }
 // Main site processing ...
}
</pre></div>
</div>
</div>
<div class="section" id="universal-vcl-return-fail">
<h2>Universal VCL return(fail)<a class="headerlink" href="#universal-vcl-return-fail" title="Permalink to this headline">¶</a></h2>
It is now possible to <code class="docutils literal">return(fail)</code> anywhere in VCL,
including inside VMODs. This will cause VCL processing
to terminate forthright.
In addition to <code class="docutils literal">return(fail)</code>, this mechanism will be
used to handle all failure conditions without a safe
fallback, for instance workspace exhaustion, too many
headers etc. (This is a work in progress, there is a
lot of code to review before we are done.)
In <code class="docutils literal">vcl_init{}</code> failing causes the <code class="docutils literal">vcl.load</code> to fail,
this is nothing new for this sub-routine.
A failure in any of the client side VCL methods (<code class="docutils literal">vcl_recv{}</code>,
<code class="docutils literal">vcl_hash{}</code> ...) except <code class="docutils literal">vcl_synth{}</code>, sends the request
to <code class="docutils literal">vcl_synth{}</code> with a 503, and reason &quot;VCL failed&quot;.
A failure on the backend side (<code class="docutils literal">vcl_backend_*{}</code>) causes the
fetch to fail.
(VMOD writers should use the new <code class="docutils literal">VRT_fail(ctx, format_string, ...)</code>
function which logs a SLT_VCL_Error record.)
</div>
<div class="section" id="progress-on-http-2-support">
<h2>Progress on HTTP/2 support<a class="headerlink" href="#progress-on-http-2-support" title="Permalink to this headline">¶</a></h2>
HTTP/2 support is better than in 5.0, and is now enabled and survives
pretty well on our own varnish-cache.org website, but there are
still things missing, most notably windows and priority, which may
be fatal to more complex websites.
We expect HTTP/2 support to be production ready in the autumn 2017
release of Varnish-Cache, but that requires a testing and feedback
from real-world applications.
So if you have a chance to test our HTTP/2 code, by all means do
so, please report any crashes, bugs or other trouble back to us.
To enable HTTP/2 you need to <code class="docutils literal">param.set feature +http2</code> but due
to internet-politics, you will only see HTTP/2 traffic if you have
an SSL proxy in front of Varnish which advertises HTTP2 with ALPN.
For the hitch SSL proxy, add the argument <code class="docutils literal">--alpn-protos=&quot;h2,http/1.1&quot;</code>
</div>
<div class="section" id="hit-for-pass-has-returned">
<h2>Hit-For-Pass has returned<a class="headerlink" href="#hit-for-pass-has-returned" title="Permalink to this headline">¶</a></h2>
As hinted in <a class="reference internal" href="changes-5.0.html#whatsnew-changes-5-0">Changes in Varnish 5.0</a>, we have restored the
possibility of invoking the old hit-for-pass feature in VCL. The
treatment of uncacheable content that was new in version 5.0, which we
have taken to calling &quot;hit-for-miss&quot;, remains the default. Now you can
choose hit-for-pass with <code class="docutils literal">return(pass(DURATION))</code> from
<code class="docutils literal">vcl_backend_response</code>, setting the duration of the hit-for-pass
state in the argument to <code class="docutils literal">pass</code>. For example:
<code class="docutils literal">return(pass(120s))</code>.
To recap: when <code class="docutils literal">beresp.uncacheable</code> is set to <code class="docutils literal">true</code> in
<code class="docutils literal">vcl_backend_response</code>, Varnish makes a note of it with a minimal
object in the cache, and finds that information again on the next
lookup for the same object. In essence, the cache is used to remember
that the last backend response was not cacheable. In that case,
Varnish proceeds as with a cache miss, so that the response may become
cacheable on subsequent requests. The difference is that Varnish does
not perform request coalescing, as it does for ordinary misses, when a
response has been marked uncacheable. For ordinary misses, when there
are requests pending for the same object at the same time, only one
fetch is executed at a time, since the response may be cached, in
which case the cached response may be used for the remaining
requests. But this is not done for &quot;hit-for-miss&quot; objects, since they
are known to have been uncacheable on the previous fetch.
<code class="docutils literal">builtin.vcl</code> sets <code class="docutils literal">beresp.uncacheable</code> to <code class="docutils literal">true</code> when a number
of conditions hold for a backend response that indicate that it should
not be cached, for example if the TTL has been determined to be 0
(perhaps due to a <code class="docutils literal">Cache-Control</code> header), or if a <code class="docutils literal">Set-Cookie</code>
header is present in the response. So hit-for-miss is the default
for uncacheable backend responses.
A consequence of this is that fetches for uncacheable responses cannot
be conditional in the default case. That is, the backend request may
not include the headers <code class="docutils literal">If-Modified-Since</code> or <code class="docutils literal">If-None-Match</code>,
which might cause the backend to return status &quot;304 Not Modified&quot; with
no response body. Since the response to a cache miss might be cached,
there has to be a body to cache, and this is true of hit-for-miss as
well. If either of those two headers were present in the client
request, they are removed from the backend request for a miss or
hit-for-miss.
Since conditional backend requests and the 304 response may be
critical to performance for non-cacheable content, especially if the
response body is large, we have made the old hit-for-pass feature
available again, with <code class="docutils literal">return(pass(DURATION))</code> in VCL.
As with hit-for-miss, Varnish uses the cache to make a note of
hit-for-pass objects, and finds them again on subsequent lookups. The
requests are then processed as for ordinary passes (<code class="docutils literal">return(pass)</code>
from <code class="docutils literal">vcl_recv</code>) -- there is no request coalescing, and the response
will not be cached, even if it might have been otherwise.
<code class="docutils literal">If-Modified-Since</code> or <code class="docutils literal">If-None-Match</code> headers in the client
request are passed along in the backend request, and a backend
response with status 304 and no body is passed back to the client.
The hit-for-pass state of an object lasts for the time given as the
DURATION in the previous return from <code class="docutils literal">vcl_backend_response</code>. After
the &quot;hit-for-pass TTL&quot; elapses, the next request will be an ordinary
miss. So a hit-for-pass object cannot become cacheable again until
that much time has passed.
</div>
<div class="section" id="not-modified-responses-after-a-pass">
<h2>304 Not Modified responses after a pass<a class="headerlink" href="#not-modified-responses-after-a-pass" title="Permalink to this headline">¶</a></h2>
Related to the previous topic, there has been a change in the way
Varnish handles a very specific case: deciding whether to send a &quot;304
Not Modified&quot; response to the client after a pass, when the backend
had the opportunity to send a 304 response, but chose not to by
sending a 200 response status instead.
Previously, Varnish went along with the backend when this happened,
sending the 200 response together with the response body to the
client. This was the case even if the backend set the response headers
<code class="docutils literal">ETag</code> and/or <code class="docutils literal">Last-Modified</code> so that, when compared to the
request headers <code class="docutils literal">If-None-Match</code> and <code class="docutils literal">If-Modified-Since</code>, a 304
response would seem to be warranted. Since those headers are passed
back to the client, the result could appear a bit odd from the
client's perspective -- the client used the request headers to ask if
the response was unmodified, and the response headers seem to indicate
that it wasn't, and yet the response status suggests that it was.
Now the decision to send a 304 client response status is made solely
at delivery time, based on the contents of the client request headers
and the headers in the response that Varnish is preparing to send,
regardless of whether the backend fetch was a pass. So Varnish may
send a 304 client response after a pass, even though the backend chose
not to, having seen the same request headers (if the response headers
permit it).
We made this change for consistency -- for hits, misses, hit-for-miss,
hit-for-pass, and now pass, the decision to send a 304 client response
is based solely on the contents of client request headers and the
response headers.
You can restore the previous behavior -- don't send a 304 client
response on pass if the backend didn't -- with VCL means, either by
removing the <code class="docutils literal">ETag</code> or <code class="docutils literal">Last-Modified</code> headers in
<code class="docutils literal">vcl_backend_response</code>, or by removing the If-* client request
headers in <code class="docutils literal">vcl_pass</code>.
</div>
<div class="section" id="vxid-in-vsl-queries">
<h2>VXID in VSL queries<a class="headerlink" href="#vxid-in-vsl-queries" title="Permalink to this headline">¶</a></h2>
The Varnish Shared Log (VSL) became much more powerful starting Varnish
4.0 and hasn't changed much since. Changes usually consist in adding new
log records when new feature are introduced, or when we realize that some
missing piece of information could really help troubleshooting.
Varnish UTilities (VUT) relying on the VSL usually share the same <code class="docutils literal">-q</code>
option for querying, which allows to filter transactions based on log
records. For example you could be looking for figures on a specific
domain:
<div class="highlight-default"><div class="highlight"><pre>varnishtop -i ReqURL -q &#39;ReqHeader:Host eq www.example.com&#39;
</pre></div>
</div>
While options like <code class="docutils literal">-i</code> and <code class="docutils literal">-q</code> were until now both limited to log
records, it also meant you could only query a specific transaction using
the <code class="docutils literal">X-Varnish</code> header. Depending on the nature of the transaction
(client or backend side) the syntax is not the same and you can't match
a session.
For instance, we are looking for the transaction 1234 that occurred very
recently and we would like to collect everything from the same session.
We have two options:
<div class="highlight-default"><div class="highlight"><pre># client side
varnishlog -d -g session -q &#39;RespHeader:X-Varnish[1] == 1234&#39;

# backend side
varnishlog -d -g session -q &#39;BereqHeader:X-Varnish == 1234&#39;
</pre></div>
</div>
There was no simple way to match any transaction using its id until the
introduction of <code class="docutils literal">vxid</code> as a possible left-hand side of a <code class="docutils literal">-q</code> query
expression:
<div class="highlight-default"><div class="highlight"><pre># client side
varnishlog -d -g session -q &#39;vxid == 1234&#39;

# session
varnishlog -d -g session -q &#39;vxid == 1234&#39;
</pre></div>
</div>
Another use case is the collection of non-transactional logs. With raw
grouping the output is organized differently and each record starts with
its transaction id or zero for non-transactional logs:
<div class="highlight-default"><div class="highlight"><pre># before 5.1
varnishlog -g raw | awk &#39;$1 == 0&#39;

# from now on
varnishlog -g raw -q &#39;vxid == 0&#39;
</pre></div>
</div>
This should offer you a more concise, and more consistent means to filter
transactions with <code class="docutils literal">varnishlog</code> and other VUTs.
</div>
<div class="section" id="project-tool-improvements">
<h2>Project tool improvements<a class="headerlink" href="#project-tool-improvements" title="Permalink to this headline">¶</a></h2>
We have spent a fair amount of time on the tools we use internally
in the project.
The <code class="docutils literal">varnishtest</code> program has been improved in many small ways,
in particular it is now much easier to execute and examine
results from other programs with the <code class="docutils literal">shell</code> and <code class="docutils literal">process</code>
commands. It might break existing test cases if you were already
using <code class="docutils literal">varnishtest</code>.
The project now has KISS web-backend which summarizes
<code class="docutils literal">make distcheck</code> results from various platforms:
<a class="reference external" href="http://varnish-cache.org/vtest/">http://varnish-cache.org/vtest/</a>
If you want Varnish to be tested on a platform not already
covered, all you need to do is run the tools/vtest.sh script
from the source tree. We would love to see more platforms
covered (arm64, ppc, mips) and OS/X would also be nice.
We also publish our code-coverage status now:
<a class="reference external" href="http://varnish-cache.org/gcov/">http://varnish-cache.org/gcov/</a>
Our goal is 90+% coverage, but we need to start implementing
terminal emulation in <code class="docutils literal">varnishtest</code> before we can test the curses(1)
based programs (top/stat/hist) comprehensively, so they currently
drag us down.
</div>
<div class="section" id="news-for-authors-of-vmods-and-varnish-api-client-applications">
<h2>News for authors of VMODs and Varnish API client applications<a class="headerlink" href="#news-for-authors-of-vmods-and-varnish-api-client-applications" title="Permalink to this headline">¶</a></h2>
<ul>
<li>The VRT version has been bumped to 6.0, since there have been some
changes and additions to the ABI. See <code class="docutils literal">vrt.h</code> for an overview.
</li>
<li>In particular, there have been some changes to the <code class="docutils literal">WS_*</code>
interface for accessing workspaces. We are working towards fully
encapsulating workspaces with the <code class="docutils literal">WS_*</code> family of functions, so
that it should not be necessary to access the internals of a
<code class="docutils literal">struct ws</code>, which may be revised in a future release. There are
no revisions at present, so your code won't break if you're
working with the innards of a <code class="docutils literal">struct ws</code> now, but you would be
prudent to replace that code with <code class="docutils literal">WS_*</code> calls some time before
the next release. And please let us know if there's something you
need to do that the workspace interface won't handle.
</li>
<li><code class="docutils literal">libvarnishapi.so</code> now exports more symbols from Varnish internal
libraries:
<ul class="simple">
<li>All of the <code class="docutils literal">VTIM_*</code> functions -- getting clock times, formatting
and parsing date &amp; time formats, sleeping and so forth.</li>
<li>All of the <code class="docutils literal">VSB_*</code> functions for working with safe string
buffers.</li>
</ul>
</li>
<li><code class="docutils literal">varnish.m4</code> and <code class="docutils literal">varnishapi.pc</code> now expose more information about
the Varnish installation. See &quot;Since 5.1.0&quot; comments for a comprehensive
list of what was added.
</li>
<li>VMOD version coexistence improvements: In difference from executable
files, shared libraries are not protected against overwriting under
UNIX, and this has generally caused grief when VMODs were updated
by package management tools.
We have decided to bite the bullet, and now the Varnishd management
process makes a copy of the VMOD shared library to a version-unique
name inside the workdir, from which the running VCL access it. This
ensures that Varnishd can always restart the worker process, no matter
what happened to the original VMOD file.
It also means that VMODs maintaining state spanning VCL reloads might
break. It is still possible to maintain global state in a VMOD despite
VMOD caching: one solution is to move the global state into separate
shared library that won't be cached by Varnish.
</li>
</ul>
EOF
</div>
</div>

</div>
 </div>
 </div>
 <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
 <div class="sphinxsidebarwrapper">
 <h3><a href="../index.html">Table Of Contents</a></h3>
 <ul>
<li><a class="reference internal" href="#">Changes in Varnish 5.1</a><ul>
<li><a class="reference internal" href="#startup-cli-command-file">Startup CLI command file</a></li>
<li><a class="reference internal" href="#universal-vcl-return-fail">Universal VCL return(fail)</a></li>
<li><a class="reference internal" href="#progress-on-http-2-support">Progress on HTTP/2 support</a></li>
<li><a class="reference internal" href="#hit-for-pass-has-returned">Hit-For-Pass has returned</a></li>
<li><a class="reference internal" href="#not-modified-responses-after-a-pass">304 Not Modified responses after a pass</a></li>
<li><a class="reference internal" href="#vxid-in-vsl-queries">VXID in VSL queries</a></li>
<li><a class="reference internal" href="#project-tool-improvements">Project tool improvements</a></li>
<li><a class="reference internal" href="#news-for-authors-of-vmods-and-varnish-api-client-applications">News for authors of VMODs and Varnish API client applications</a></li>
</ul>
</li>
</ul>

<h4>Previous topic</h4>
 <a href="upgrading-5.2.html"
 title="previous chapter">Upgrading to Varnish 5.2</a>
 <h4>Next topic</h4>
 <a href="upgrading-5.1.html"
 title="next chapter">Upgrading to Varnish 5.1</a>
 <div role="note" aria-label="source link">
 <h3>This Page</h3>
 <ul class="this-page-menu">
 <li><a href="../_sources/whats-new/changes-5.1.txt"
 rel="nofollow">Show Source</a></li>
 </ul>
 </div>
<div id="searchbox" style="display: none" role="search">
 <h3>Quick search</h3>
 <form class="search" action="../search.html" method="get">
 <div><input type="text" name="q" /></div>
 <div><input type="submit" value="Go" /></div>
 <input type="hidden" name="check_keywords" value="yes" />
 <input type="hidden" name="area" value="default" />
 </form>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
 </div>
 </div>
 <div class="clearer"></div>
 </div>
 <div class="related" role="navigation" aria-label="related navigation">
 <h3>Navigation</h3>
 <ul>
 <li class="right" style="margin-right: 10px">
 <a href="../genindex.html" title="General Index"
 >index</a></li>
 <li class="right" >
 <a href="upgrading-5.1.html" title="Upgrading to Varnish 5.1"
 >next</a> |</li>
 <li class="right" >
 <a href="upgrading-5.2.html" title="Upgrading to Varnish 5.2"
 >previous</a> |</li>
 <li class="nav-item nav-item-0"><a href="../index.html">Varnish version 5.2.1 documentation</a> &#187;</li>
 <li class="nav-item nav-item-1"><a href="index.html" >What's new / Upgrading</a> &#187;</li> 
 </ul>
 </div>
 <div class="footer" role="contentinfo">
 &#169; Copyright 2010-2014, Varnish Software AS.
 Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.9.
 </div>
 </body>
</html>