合并回源 collapsed forwarding

https://docs.trafficserver.apache.org/en/6.2.x/admin-guide/plugins/collapsed_forwarding.en.html?highlight=ats%20internal

Collapsed Forwarding Plugin

Traffic Server plugin to allow collapsed forwarding of concurrent requests for the same object. This plugin is based on open_write_fail_action feature（该插件依赖于OWF特性）, which detects cache open write failure on a cache miss and returns a 502 error along
with a special @-header indicating the reason for 502 error. （OWF的工作流程是：请求获取写锁失败时会返回502错误，并附带一个特殊的http头用来指示错误的原因）

The plugin acts on the error by using an internal redirect follow back to itself（该插件的工作方式：收到502错误后，发起一个内部重定向）, essentially blocking the request until a response arrives, at which point, relies on read-whilewriter feature to start downloading the object to all waiting clients.

The following config parameters are assumed to be set for this plugin to work:
:ts:cv:`proxy.config.http.cache.open_write_fail_action` 1
:ts:cv:`proxy.config.cache.enable_read_while_writer` 1
:ts:cv:`proxy.config.http.redirection_enabled` 1
:ts:cv:`proxy.config.http.number_of_redirections` 10
:ts:cv:`proxy.config.http.redirect_use_orig_cache_key` 1
:ts:cv:`proxy.config.http.background_fill_active_timeout` 0
:ts:cv:`proxy.config.http.background_fill_completed_threshold` 0

Additionally, given that collapsed forwarding works based on cache write lock failure detection（基于获取写锁失败）, the plugin requires cache to be enabled and ready.

On a restart, Traffic Server typically takes a few seconds to initialize the cache depending on the cache size and number of dirents. While the cache is not ready yet, collapsed forwarding can not
detect the write lock contention and so can not work. The setting :ts:cv:‘proxy.config.http.wait_for_cache‘ may be enabled which allows blocking incoming connections from being accepted until cache is ready

 

-------

Traffic Server has been affected severely by the Thundering Herd problem caused by its inability to do effective connection collapse of multiple concurrent requests for the same segment.

This is especially critical when Traffic Server is used as a solution to use cases such as delivering a large scale video live streaming. This problem results in a specific behavior where multiple number of requests for the same file are leaked upstream to the Origin layer choking
the upstream bandwidth due to the duplicated large file downloads or process intensive file at the Origin layer（坏处）. This ultimately can cause stability problems on the origin layer disrupting the overall network performance.

Traffic Server supports several kind of connection collapse mechanisms including Read-While-Writer (RWW), StaleWhile-Revalidate (SWR)（解决手段） etc each very effective dealing with a majority of the use cases that can result in the Thundering herd problem.

For a large scale Video Streaming scenario, there’s a combination of a large number of revalidations (e.g.media playlists) and cache misses (e.g. media segments) that occur for the same file.

Traffic Server’s RWW works great in collapsing the concurrent requests in such a scenario（RWW的适用场景，如大规模视频流媒体播放）, however, as described in _admin-configuration-reducing-origin-requests, Traffic Server’s implementation of RWW has a significant limitation,

which restricts its ability to invoke RWW only when the response headers are already received.（RWW的不足：需要等到ats已经收到响应头部）

This means that any number of concurrent requests for the same file that are received before the response headers arrive are leaked upstream, which can result in a severe Thundering herd problem, depending on the network latencies
(which impact the TTFB for the response headers) at a given instant of time.（这有可能导致大量回源请求）

To address this limitation, Traffic Server supports a few Cache tuning solutions, such as Open Read Retry, and a new feature called Open Write Fail action from 6.0. （为了解决上述问题，引入了ORR, OWF机制）

To understand how these approaches work, it is important to understand the high level flow of how Traffic Server handles a GET request. （要了解这些机制的工作方式，需要掌握ats是如何处理一个get请求的）

 

ATS对GET请求的处理流程：

On receiving a HTTP GET request, Traffic Server 【1】generates the cache key (basically, a hash of the request URL) and looks up for the directory entry (dirent) using the generated index.

On a cache 【2.1】miss, the lookup fails and Traffic Server then tries to just get a write lock for the cache object and proceeds to the origin to download the object.

On the Other hand, if the lookup is 【2.2】successful, meaning, the dirent exists for the generated cache key, Traffic Server tries to obtain a read lock on the cache object to be able to serve it from the cache.

　　If the 【2.2.1】read lock is not successful (possibly, due to the fact that the object’s being written to at that same instant and the response headers are not in the cache yet),Traffic Server then moves to the next step of trying to obtain an exclusive write lock.

　　获取读锁失败（原因：刚好正在被写and响应头还没下载完成），则尝试获取排他式写锁

 

　　【2.2.2】If the write lock is already held exclusively by another request (transaction), the attempt fails and at this point Traffic Server simply disables the cache on that transaction and downloads the object in a proxy-only mode:

　　排他式的写锁获取失败，则回源下载

 

1). Cache Lookup (lookup for the dirent using the request URL as cache key).
　　1.1). If lookup fails (cache miss), goto (3).
　　1.2). If lookup succeeds, try to obtain a read lock, goto (2).
2). Open Cache Read (try to obtain read lock)
　　2.1). If read lock succeeds, serve from cache, goto (4).
　　2.2). If read lock fails, goto (3).
3). Open Cache Write (try to obtain write lock).
　　3.1). If write lock succeeds, download the object into cache and to the client in parallel
　　3.2). If write lock fails, disable cache, and download to the client in a proxy-only mode.
4). Done

As can be seen above, if a majority of concurrent requests arrive before response headers are received, they hit (2.2) and (3.2) above.

Open Read Retry can help to repeat (2) （ORR的处理时机）after a configured delay on 2.2, thereby increasing the chances for obtaining a read lock and being able to serve from the cache.

However, the Open Read Retry can not help with the concurrent requests that hit (1.1) （ORR不能处理的地方）above, jumping to (3) directly.

Only one such request will be able to obtain the exclusive write lock and all other requests are leaked upstream. 只有一个请求能成功获取写锁，其他的请求都将回源。

This is where, the recently developed Traffic Server feature Open Write Fail Action will help.  （OWF的发挥其作用的地方）

The feature detects the write lock failure and can return a stale copy for a Cache Revalidation or a 5xx status code for a Cache Miss with a special internal header <@Ats-Internal> that allows a TS plugin to take other special actions depending on the use-case.

对于获取写锁失败的concurrent请求，可以返回过期的copy或者返回5xx 的http状态码。

 

collapsed_forwarding plugin catches that error in SEND_RESPONSE_HDR_HOOK and performs an internal 3xx Redirect back to the same host, （该插件会在SEND_RESPONSE_HDR_HOOK阶段捕获上面由于获取写锁失败而返回的错误，并发发起一个内部重定向到原来的主机(原请求的主机）

the configured number of times with the configured amount of delay between consecutive retries, allowing to be able to initiate RWW, whenever the response headers are received for the request that was allowed to go to the Origin.

根据配置，循环发起内部重定向，以此来等待RWW机制开始工作(等待响应头部)，以此来减少回源请求

More details are available at
https://docs.trafficserver.apache.org/en/6.0.x/admin/http-proxy-caching.en.html#reducing-origin-server-requests-avoiding-the-thundering-herd