atomic原子编程中的Memory Order

在多核编程中，我们使用内核对象【如：事件对象（Event）、互斥量对象（Mutex，或互斥体对象）、信号量对象（Semaphore）等】来避免多个线程修改同一个数据时产生的竞争条件。

但是，基于内核对象的同步，会带来昂贵的上下文切换（用户态切换到内核态，占用1000个以上的cpu周期）。就需要使用另一种方法 —— 原子指令。

原子指令（x为std::atomic类型）	说明
x.load()	读操作返回x的值
x.store(n)	写操作把x设为n，什么都不返回
x.exchange(n)	把x设为n，返回设定之前的值
x.fetch_add(n)	原子地做x += n，返回修改之前的值
x.fetch_sub(n)	原子地做x-= n，返回修改之前的值

仅靠原子技术实现不了对资源的访问控制，即使简单计数操作，看上去正确的代码也可能会crash。

这里的关键在于编译器和cpu实施的重排指令导致了读写顺序的变化。只要没有依赖，代码中在后面的指令就可能跑到前面去，编译器和CPU都会这么做。

PowerPC和ARM等弱排序cpu会进行指令重排（依赖内存栅栏指令）；而Intel x86, x86-64强排序cpu，总能保证按顺序执行，遵从数据依赖顺序

注1：单线程代码不需要关心乱序的问题。因为乱序至少要保证这一原则：不能改变单线程程序的执行行为

注2：内核对象多线程编程在设计的时候都阻止了它们调用点中的乱序（已经隐式包含memory barrier），不需要考虑乱序的问题

注3：使用用户模式下的线程同步时，乱序的效果才会显露无疑

std::atomic 本身就具备编译期内存屏障（compiler barrier）功能，并且还具备运行时内存屏障（hardware barrier）功能。这是 C++11 及以后标准内存模型的核心特性之一。

注1：如果需要barrier行为，优先用 std::atomic，不要用裸的volatile或手写 barrier

注2：如果只需要 barrier 而不需要原子性，可以用 std::atomic_signal_fence（运行时+编译期）或 std::atomic_thread_fence（仅编译期）

注3：std::atomic 的所有成员函数（如 store, load, exchange, fetch_add 等）都会阻止编译器对相关内存操作的重排序。也就是说，编译器不会把对 std::atomic 的操作与其前后的普通内存操作随意重排

注4：对于大多数平台，std::atomic 的操作还会在必要时插入 CPU 指令级的内存屏障，确保多线程下的可见性和顺序性

可以通过 std::memory_order 参数精细控制屏障的强度（如 memory_order_relaxed, memory_order_acquire, memory_order_release, memory_order_seq_cst 等）

注5：默认的memory_order_seq_cst 是最强的，既保证编译器不重排，也保证 CPU 不重排

程序员可以使用c++11 atomic提供了6种memory order，来在编程语言层面对编译器和cpu实施的重排指令行为进行控制

memory order	作用
memory_order_relaxed	无fencing作用，cpu可以任意重排指令
memory_order_consume	后面依赖此原子变量的访存指令勿重排至此条指令之前注：性能比memory_order_acquire高
memory_order_acquire	后面访存指令勿重排至此条指令之前
memory_order_release	前面访存指令勿重排到此条指令之后
memory_order_acq_rel	acquare + release
memory_order_seq_cst	acq_rel + 所有使用seq_cst的指令有严格的全序关系

多线程编程时，通过这些标志位，来读写原子变量，可以组合出4种同步模型：

Relaxed ordering

Release-Acquire ordering

Release-Consume ordering

Sequentially-consistent ordering

默认情况下，std::atomic使用的是Sequentially-consistent ordering(最严格的同步模型)。但在某些场景下，合理使用其它3种ordering，可以让CPU优化执行的代码，从而提高性能。

Relaxed ordering

在这种模型下，std::atomic的load()和store()都要带上memory_order_relaxed参数。Relaxed ordering仅仅保证load()和store()是原子操作，除此之外，不提供任何跨线程的同步。

先看看一个简单的例子：

std::atomic<int> x = 0;     // global variable
std::atomic<int> y = 0;     // global variable
		  
Thread-1:                                  Thread-2:
r1 = y.load(memory_order_relaxed); // A    r2 = x.load(memory_order_relaxed); // C
x.store(r1, memory_order_relaxed); // B    y.store(42, memory_order_relaxed); // D

执行完上面的程序，可能出现r1 == r2 == 42。理解这一点并不难，因为CPU可能调整 C 和 D 的执行顺序。

如果程序的执行顺序是 D -> A -> B -> C，那么就会出现r1 == r2 == 42。

如果某个操作只要求是原子操作，不需要其它同步的保障，就可以使用 Relaxed ordering。程序计数器是一种典型的应用场景。

#include <cassert>
#include <vector>
#include <iostream>
#include <thread>
#include <atomic>
std::atomic<int> cnt = {0};
void f()
{
    for (int n = 0; n < 1000; ++n) {
        cnt.fetch_add(1, std::memory_order_relaxed);
    }
}
int main()
{
    std::vector<std::thread> v;
    for (int n = 0; n < 10; ++n) {
        v.emplace_back(f);
    }
    for (auto& t : v) {
        t.join();
    }
    assert(cnt == 10000);    // never failed
    return 0;
}

Release-Acquire ordering

在这种模型下，store()使用memory_order_release，而load()使用memory_order_acquire。这种模型有两种效果，第一种是可以限制 CPU 指令的重排：

（1）在store()之前的所有读写操作，不允许被移动到这个store()的后面。 // write-release语义

（2）在load()之后的所有读写操作，不允许被移动到这个load()的前面。 // read-acquire语义

该模型可以保证：如果Thread-1的store()的那个值，成功被 Thread-2的load()到了，那么 Thread-1在store()之前对内存的所有写入操作，此时对 Thread-2 来说，都是可见的。

下面的例子阐述了这种模型的原理：

#include <thread>
#include <atomic>
#include <cassert>
#include <string>
std::atomic<bool> ready{ false };
int data = 0;
void producer()
{
    data = 100;                                       // A
    ready.store(true, std::memory_order_release);     // B
}
void consumer()
{
    while (!ready.load(std::memory_order_acquire))    // C
        ;
    assert(data == 100); // never failed              // D
}
int main()
{
    std::thread t1(producer);
    std::thread t2(consumer);
    t1.join();
    t2.join();
    return 0;
}

让我们分析一下这个过程：
首先 A 不允许被移动到 B 的后面。
同样 D 也不允许被移动到 C 的前面。
当 C 从 while 循环中退出了，说明 C 读取到了 B store()的那个值，此时，Thread-2 保证能够看见 Thread-1 执行 B 之前的所有写入操作（也即是 A）。

使用Release-Acquire ordering实现双重检查锁模式(DLCP)

下面单件为例来说明：

class Singleton
{
public:
    static Singleton* get_instance() {
        Singleton* tmp = instance_.load(std::memory_order_acquire);
        if (tmp == nullptr) {
            std::unique_lock<std::mutex> lk(mutex_);
            tmp = instance_;
            if (tmp == nullptr) {
                tmp = new Singleton();
                instance_.store(std::memory_order_release);
            }
        }
        return tmp;
    }

private:
    Singleton() = default;
    static std::atomic<Singleton*> instance_;
    static std::mutex mutex_;
};

使用Release-Acquire ordering实现自旋锁(Spinlock)

获取和释放语义，是实现锁的基础(Spinlock, Mutex, RWLock, ...)，所有被[Read Acquire,Write Release]包含的区域，即构成了一个临界区，临界区里的内存操作，不会乱序到临界区之外执行。

            read-acquire(判断是否加锁，没则加锁，否则循环等待)
-------------------------------------------------------------------------

            all memory operation stay between the line（临界区）

-------------------------------------------------------------------------
                        write-release(释放锁)

实现代码如下：

#include <atomic>
class simple_spin_lock
{
public:
    simple_spin_lock() = default;
    void lock()
    {
        while (flag.test_and_set(std::memory_order_acquire))
            continue;
    }
    void unlock()
    {
        flag.clear(std::memory_order_release);
    }
private:
    simple_spin_lock(const simple_spin_lock&) = delete;
    simple_spin_lock& operator =(const simple_spin_lock&) = delete;
    std::atomic_flag flag = ATOMIC_FLAG_INIT;
};

① 对std::atomic_flag的操作具有原子性，保证了同一时间，只有一个线程能够lock成功，其余线程全部在while循环

② 使用了acquire内存屏障，所以lock具有获取语义

③ 使用了release内存屏障，所以unlock具有释放语义

Release-Consume ordering

在这种模型下，store()使用memory_order_release，而load()使用memory_order_consume。这种模型有两种效果，第一种是可以限制 CPU 指令的重排：

（1）在store()之前的所有读写操作，不允许被移动到这个store()的后面。

（2）在load()之后的所有依赖此原子变量的读写操作，不允许被移动到这个load()的前面。

注：不依赖此原子变量的读写操作可能会CPU指令重排

下面的例子阐述了这种模型的原理：

#include <thread>
#include <atomic>
#include <cassert>
#include <string>

std::atomic<std::string*> ptr;
int data;
// thread1
void producer()
{
    std::string* p  = new std::string("Hello"); // A
    data = 42; // B
    ptr.store(p, std::memory_order_release); // C
}
// thread2
void consumer()
{
    std::string* p2;
    while (!(p2 = ptr.load(std::memory_order_consume))) // D
        ;
    assert(*p2 == "Hello"); //E     always true: *p2 carries dependency from ptr
    assert(data == 42); // F     may be false: data does not carry dependency from ptr
}

int main()
{
    std::thread t1(producer);
    std::thread t2(consumer);
    t1.join(); 
    t2.join();
    return 0;
}

Sequentially-consistent ordering

所有以memory_order_seq_cst为参数的原子操作(不限于同一个原子变量)，对所有线程来说有一个全局顺序(total order)

并且两个相邻memory_order_seq_cst原子操作之间的其他操作(包括非原子变量操作)，不能reorder到这两个相邻操作之外

UE4下的std::atomic案例

FShaderMapResource::GetShader会在RenderThread和TaskGraph工作线程中并发运行

// RenderThread线程
RaiseException    (:0)[amd64:Windows NT:C85FB769730CD5340757B74BE740FD1A1]
ReportAssert(wchar_t const*, int) (UnrealEngine\Engine\Source\Runtime\Core\Private\Windows/WindowsPlatformCrashContext.cpp:1685)[amd64:Windows NT:D617C4312ED947B59C04DBEC458EFA7E1]
FWindowsErrorOutputDevice::Serialize(wchar_t const*, ELogVerbosity::Type, FName const&) (UnrealEngine\Engine\Source\Runtime\Core\Private\Windows/WindowsErrorOutputDevice.cpp:93)[amd64:Windows NT:D617C4312ED947B59C04DBEC458EFA7E1]
FOutputDevice::LogfImpl(wchar_t const*, <NoType>) (UnrealEngine\Engine\Source\Runtime\Core\Private\Misc/OutputDevice.cpp:61)[amd64:Windows NT:D617C4312ED947B59C04DBEC458EFA7E1]
FShaderLibraryInstance::GetOrCreateShader(int, bool) (UnrealEngine\Engine\Source\Runtime\RenderCore\Private/ShaderCodeLibrary.cpp:1045)[amd64:Windows NT:D617C4312ED947B59C04DBEC458EFA7E1]
FShaderMapResource_SharedCode::CreateRHIShader(int, bool) (UnrealEngine\Engine\Source\Runtime\RenderCore\Private/ShaderCodeLibrary.cpp:1173)[amd64:Windows NT:D617C4312ED947B59C04DBEC458EFA7E1]
FShaderMapResource::CreateShader(int, bool) (UnrealEngine\Engine\Source\Runtime\RenderCore\Private/ShaderResource.cpp:413)[amd64:Windows NT:D617C4312ED947B59C04DBEC458EFA7E1]
FShaderMapResource::GetShader(int, bool) (UnrealEngine\Engine\Source\Runtime\RenderCore\Private/ShaderResource.cpp:387)[amd64:Windows NT:D617C4312ED947B59C04DBEC458EFA7E1]
<lambda_668c427b06c73becabd992f95247910b>::operator()(FRHICommandList &) const (UnrealEngine\Engine\Source\Runtime\Renderer\Private/VolumetricRenderTarget.cpp:791)[amd64:Windows NT:D617C4312ED947B59C04DBEC458EFA7E1]
FRDGPass::Execute(FRHIComputeCommandList &) (UnrealEngine\Engine\Source\Runtime\RenderCore\Private/RenderGraphPass.cpp:418)[amd64:Windows NT:D617C4312ED947B59C04DBEC458EFA7E1]
FRDGBuilder::ExecutePass(FRDGPass *, bool) (UnrealEngine\Engine\Source\Runtime\RenderCore\Private/RenderGraphBuilder.cpp:2093)[amd64:Windows NT:D617C4312ED947B59C04DBEC458EFA7E1]
FRDGBuilder::ExecuteParalleled() (UnrealEngine\Engine\Source\Runtime\RenderCore\Private/RenderGraphBuilderParallel.cpp:242)[amd64:Windows NT:D617C4312ED947B59C04DBEC458EFA7E1]
FRDGBuilder::Execute() (UnrealEngine\Engine\Source\Runtime\RenderCore\Private/RenderGraphBuilder.cpp:1401)[amd64:Windows NT:D617C4312ED947B59C04DBEC458EFA7E1]
FDeferredShadingSceneRenderer::Render(FRHICommandListImmediate &) (UnrealEngine\Engine\Source\Runtime\Renderer\Private/DeferredShadingRenderer.cpp:3926)[amd64:Windows NT:D617C4312ED947B59C04DBEC458EFA7E1]
RenderViewFamily_RenderThread(FRHICommandListImmediate &, FSceneRenderer *) (UnrealEngine\Engine\Source\Runtime\Renderer\Private/SceneRendering.cpp:5415)[amd64:Windows NT:D617C4312ED947B59C04DBEC458EFA7E1]
<lambda_73b2ea47c3c5cbe1e1cd1347c54c377f>::operator()(FRHICommandListImmediate &) const (UnrealEngine\Engine\Source\Runtime\Renderer\Private/SceneRendering.cpp:5818)[amd64:Windows NT:D617C4312ED947B59C04DBEC458EFA7E1]
TGraphTask<TEnqueueUniqueRenderCommandType<`FRendererModule::BeginRenderingViewFamily'::`27'::FDrawSceneCommandName,<lambda_73b2ea47c3c5cbe1e1cd1347c54c377f> > >::ExecuteTask(TArray<FBaseGraphTask *,TSizedDefaultAllocator<32> > &, ENamedThreads::Type) (UnrealEngine\Engine\Source\Runtime\Core\Public\Async/TaskGraphInterfaces.h:907)[amd64:Windows NT:D617C4312ED947B59C04DBEC458EFA7E1]
FNamedTaskThread::ProcessTasksUntilQuit(int) (UnrealEngine\Engine\Source\Runtime\Core\Private\Async/TaskGraph.cpp:604)[amd64:Windows NT:D617C4312ED947B59C04DBEC458EFA7E1]
RenderingThreadMain(FEvent *) (UnrealEngine\Engine\Source\Runtime\RenderCore\Private/RenderingThread.cpp:391)[amd64:Windows NT:D617C4312ED947B59C04DBEC458EFA7E1]
FRenderingThread::Run() (UnrealEngine\Engine\Source\Runtime\RenderCore\Private/RenderingThread.cpp:532)[amd64:Windows NT:D617C4312ED947B59C04DBEC458EFA7E1]
FRunnableThreadWin::Run() (UnrealEngine\Engine\Source\Runtime\Core\Private\Windows/WindowsRunnableThread.cpp:90)[amd64:Windows NT:D617C4312ED947B59C04DBEC458EFA7E1]
FRunnableThreadWin::GuardedRun() (UnrealEngine\Engine\Source\Runtime\Core\Private\Windows/WindowsRunnableThread.cpp:35)[amd64:Windows NT:D617C4312ED947B59C04DBEC458EFA7E1]
BaseThreadInitThunk    (:0)[amd64:Windows NT:639B06F376030222DC25A08D1F57CB931]
RtlUserThreadStart (:0)[amd64:Windows NT:794BBCE43C345ADE6883E3F78DC2928F1]

// TaskGraph工作线程
FShaderCodeReader::FindOptionalData(unsigned char, unsigned char) const (UnrealEngine\Engine\Source\Runtime\RenderCore\Public/ShaderCore.h:720)[amd64:Windows NT:FC8E8E7614B94EB7A82674A8CDEE36561]
ReadShaderOptionalData<FD3D12DomainShader>(FShaderCodeReader &, FD3D12DomainShader &, bool &, FShaderCodeFeatures &) (UnrealEngine\Engine\Source\Runtime\D3D12RHI\Private/D3D12Shaders.cpp:13)[amd64:Windows NT:FC8E8E7614B94EB7A82674A8CDEE36561]
FD3D12DynamicRHI::RHICreatePixelShader(TArrayView<unsigned char const ,int>, FSHAHash const&) (UnrealEngine\Engine\Source\Runtime\D3D12RHI\Private/D3D12Shaders.cpp:198)[amd64:Windows NT:FC8E8E7614B94EB7A82674A8CDEE36561]
FD3D12DynamicRHI::CreatePixelShader_RenderThread(FRHICommandListImmediate &, TArrayView<unsigned char const ,int>, FSHAHash const&) (UnrealEngine\Engine\Source\Runtime\D3D12RHI\Private/D3D12RHIPrivate.h:522)[amd64:Windows NT:FC8E8E7614B94EB7A82674A8CDEE36561]
FShaderCodeArchive::CreateShader(int, bool) (UnrealEngine\Engine\Source\Runtime\RenderCore\Private/ShaderCodeArchive.cpp:661)[amd64:Windows NT:FC8E8E7614B94EB7A82674A8CDEE36561]
FShaderLibraryInstance::GetOrCreateShader(int, bool) (UnrealEngine\Engine\Source\Runtime\RenderCore\Private/ShaderCodeLibrary.cpp:1062)[amd64:Windows NT:FC8E8E7614B94EB7A82674A8CDEE36561]
FShaderMapResource_SharedCode::CreateRHIShader(int, bool) (UnrealEngine\Engine\Source\Runtime\RenderCore\Private/ShaderCodeLibrary.cpp:1172)[amd64:Windows NT:FC8E8E7614B94EB7A82674A8CDEE36561]
FShaderMapResource::CreateShader(int, bool) (UnrealEngine\Engine\Source\Runtime\RenderCore\Private/ShaderResource.cpp:413)[amd64:Windows NT:FC8E8E7614B94EB7A82674A8CDEE36561]
FGraphicsMinimalPipelineStateInitializer::AsGraphicsPipelineStateInitializer() const (UnrealEngine\Engine\Source\Runtime\Renderer\Public/MeshPassProcessor.h:414)[amd64:Windows NT:FC8E8E7614B94EB7A82674A8CDEE36561]
static FMeshDrawCommand::MFGpuDriven_SubmitDraw(FMeshDrawCommand const&, Experimental::TRobinHoodHashSet<FGraphicsMinimalPipelineStateInitializer,DefaultKeyFuncs<FGraphicsMinimalPipelineStateInitializer,0>,TInlineAllocator<1,TSizedDefaultAllocator<32> > > const&, FRHIVertexBuffer *, unsigned int, int, FRHICommandList &, bool, FMeshDrawCommandStateCache &, unsigned int) (UnrealEngine\Engine\Source\Runtime\Renderer\Private\MFGpuDriven/MeshPassProcessor_MFGpuDriven.inl:205)[amd64:Windows NT:FC8E8E7614B94EB7A82674A8CDEE36561]
MFGpuDriven::FMFGpuDrivenPassSetupContext::SubmitMeshDrawCommands(FRHICommandList &, Experimental::TRobinHoodHashSet<FGraphicsMinimalPipelineStateInitializer,DefaultKeyFuncs<FGraphicsMinimalPipelineStateInitializer,0>,TInlineAllocator<1,TSizedDefaultAllocator<32> > > const&, TArray<FVisibleMeshDrawCommand,TMemStackAllocator<0> > const&, int, int, unsigned int, bool, EMFGpuDrivenPassDrawTypes::Type) const (UnrealEngine\Engine\Source\Runtime\Renderer\Private\MFGpuDriven/MeshPassProcessor_MFGpuDriven.inl:407)[amd64:Windows NT:FC8E8E7614B94EB7A82674A8CDEE36561]
FMFGpuDrivenDrawMeshDrawCommandsAnyThreadTask::DoTask(ENamedThreads::Type, TRefCountPtr<FGraphEvent> const&) (UnrealEngine\Engine\Source\Runtime\Renderer\Private\MFGpuDriven/MeshDrawCommands_MFGpuDriven.cpp:78)[amd64:Windows NT:FC8E8E7614B94EB7A82674A8CDEE36561]
TGraphTask<FMFGpuDrivenDrawMeshDrawCommandsAnyThreadTask>::ExecuteTask(TArray<FBaseGraphTask *,TSizedDefaultAllocator<32> > &, ENamedThreads::Type) (UnrealEngine\Engine\Source\Runtime\Core\Public\Async/TaskGraphInterfaces.h:897)[amd64:Windows NT:FC8E8E7614B94EB7A82674A8CDEE36561]
FTaskThreadAnyThread::ProcessTasks() (UnrealEngine\Engine\Source\Runtime\Core\Private\Async/TaskGraph.cpp:1156)[amd64:Windows NT:FC8E8E7614B94EB7A82674A8CDEE36561]
FTaskThreadAnyThread::ProcessTasksUntilQuit(int) (UnrealEngine\Engine\Source\Runtime\Core\Private\Async/TaskGraph.cpp:978)[amd64:Windows NT:FC8E8E7614B94EB7A82674A8CDEE36561]
FTaskThreadBase::Run() (UnrealEngine\Engine\Source\Runtime\Core\Private\Async/TaskGraph.cpp:545)[amd64:Windows NT:FC8E8E7614B94EB7A82674A8CDEE36561]
FRunnableThreadWin::Run() (UnrealEngine\Engine\Source\Runtime\Core\Private\Windows/WindowsRunnableThread.cpp:90)[amd64:Windows NT:FC8E8E7614B94EB7A82674A8CDEE36561]
FRunnableThreadWin::GuardedRun() (UnrealEngine\Engine\Source\Runtime\Core\Private\Windows/WindowsRunnableThread.cpp:35)[amd64:Windows NT:FC8E8E7614B94EB7A82674A8CDEE36561]
BaseThreadInitThunk    (:0)[amd64:Windows NT:2554A4D7DED55C1D6F033011500EA7811]
RtlUserThreadStart    (:0)[amd64:Windows NT:1806222313D4104266A4820B86925E3B1]

为了保证多线程数据安全，最小化性能损耗的情况下，设计者对其成员变量RHIShaders数组进行memory order控制访问

加 memory order 的目的是：在无锁读取路径上，保证读到指针的同时也能读到指针指向对象的完整初始化数据

std::atomic 的默认操作是 memory_order_seq_cst（顺序一致），这是最强也最慢的。这里显式指定 acquire / release / relaxed，是为了在保证正确性的前提下，用尽可能弱的内存序来提升性能

/****** UnrealEngine\Engine\Source\Runtime\RenderCore\Public\Shader.h  ******/
class RENDERCORE_API FShaderMapResource : public FRenderResource, public FDeferredCleanupInterface
{
public:
    // 。。。 。。。

    inline bool HasShader(int32 ShaderIndex) const
    {
        // 。。。 。。。
    }

    inline FRHIShader* GetShader(int32 ShaderIndex)
    {
        // This is a double checked locking. This trickery arises from the fact that we're
        // synchronizing two threads: one that takes a lock and another that doesn't.
        // Without fences, there is a race between storing the shader pointer and accessing it
        // on the other (lockless) thread.

        FRHIShader* Shader = RHIShaders[ShaderIndex].load(std::memory_order_acquire); // 注：acquire 语义保证：在这个 load 之后的读操作，不会被重排到 load 之前

// 它与另一个线程的 release store 配对，形成 synchronizes-with 关系：当读线程通过 acquire 读到了 release 存进去的指针后，写线程在 store 之前的所有写入（即对象初始化 (A)）对读线程都保证可见

// 如果不用 acquire/release，可能出现这样的灾难：读线程看到了非空指针，但指针指向的对象数据还没写完/还没同步过来，导致读到未初始化的垃圾数据

if (Shader == nullptr)
        {
            // most shadermaps have <100 shaders, and less than a half of them can be created. One lock
            // for all creation seems sufficient, but if this function is often contended, per-shader
            // locks are easily possible.
            FScopeLock ScopeLock(&RHIShadersCreationGuard);

            Shader = RHIShaders[ShaderIndex].load(std::memory_order_relaxed); // 最弱的 relaxed（只保证原子性，不提供任何顺序/同步保证），原因是：此时已经拿到了 RHIShadersCreationGuard 锁
                                                                              // 锁本身（FScopeLock 的加锁操作）已经提供了 acquire 语义，足以建立必要的内存同步
                                                                              // 所以这次读取不需要额外的 acquire 开销，只要保证读取是原子的（不撕裂）即可
            if (Shader == nullptr)
            {
                Shader = CreateShader(ShaderIndex);  // (A) 构造对象、写入对象内部数据
                RHIShaders[ShaderIndex].store(Shader, std::memory_order_release); // (B) 发布指针
                // 注：release 语义保证：在 (B) 之前的所有内存写入（包括 (A) 里对 shader 对象的初始化），都不会被 CPU/编译器重排到 (B) 之后
                //    也就是说，一旦指针被存进原子变量，指针指向的对象一定已经是完整构造好的状态
            }
        }
        return Shader;
    }

    // 。。。 。。。

private:

    /** This lock is to prevent two threads creating the same RHIShaders element. It is only taken if the element is to be created. */
    FCriticalSection RHIShadersCreationGuard;

    /** An array of shader pointers (refcount is managed manually). */
    TUniquePtr<std::atomic<FRHIShader*>[]> RHIShaders;

    // 。。。 。。。
};

GetShader函数采用了double-checked locking的设计：

（1）第一次检查（无锁 + acquire）：绝大多数情况下 shader 已存在，直接命中返回，避免加锁开销。这是高频路径，所以用 acquire 而非默认的 seq_cst
（2）加锁：只有指针为空时才加锁，避免多个线程重复创建同一个 shader
（3）第二次检查（持锁 + relaxed）：进锁后再查一次，因为可能在"第一次检查失败"到"拿到锁"之间，另一个线程已经创建好了。持锁下用 relaxed 即可
（4）发布（release）：创建完成后用 release store 安全发布给其他无锁读线程

总结：acquire/release 配对建立了跨线程的"happens-before"关系，确保无锁读线程看到非空指针时，指针指向的 shader 对象一定是完全构造好的

而 relaxed 用在已被锁保护的路径上以省去多余的同步开销。这是在正确性和性能之间做的精细权衡 —— 比无脑用默认的 seq_cst 更快，比不加同步（可能读到半初始化对象）更安全

UE4下的Memory Order

enum class EMemoryOrder
{
    // Provides no guarantees that the operation will be ordered relative to any other operation.
    Relaxed,  // 对应c++标准中的Relaxed ordering：仅仅保证load()和store()是原子操作

    // Establishes a single total order of all other atomic operations marked with this.
    SequentiallyConsistent  // 对应c++标准中的Sequentially-consistent ordering。UE4中Load和Store函数缺省为该类型
};

详见：UnrealEngine\Engine\Source\Runtime\Core\Public\Templates\Atomic.h

Atomic相关的测试代码见：UnrealEngine\Engine\Source\Runtime\Core\Private\Tests\Misc\AtomicTest.cpp

TAtomic<T>

/** UnrealEngine\Engine\Source\Runtime\Core\Public\Async\TaskGraphInterfaces.h */
namespace ENamedThreads 
{
    enum Type : int32
    {
        UnusedAnchor = -1,
        /** The always-present, named threads are listed next **/
#if STATS
        StatsThread, 
#endif
        RHIThread,
        AudioThread,
        GameThread,
        // The render thread is sometimes the game thread and is sometimes the actual rendering thread
        ActualRenderingThread = GameThread + 1,
        // CAUTION ThreadedRenderingThread must be the last named thread, insert new named threads before it

        /** not actually a thread index. Means "Unknown Thread" or "Any Unnamed Thread" **/
        AnyThread = 0xff, 

        /** High bits are used for a queue index and priority**/

        MainQueue =            0x000,
        LocalQueue =        0x100,

        NumQueues =            2,
        ThreadIndexMask =    0xff,
        QueueIndexMask =    0x100,
        QueueIndexShift =    8,

        /** High bits are used for a queue index task priority and thread priority**/

        NormalTaskPriority =    0x000,
        HighTaskPriority =        0x200,

        NumTaskPriorities =        2,
        TaskPriorityMask =        0x200,
        TaskPriorityShift =        9,

        NormalThreadPriority = 0x000,
        HighThreadPriority = 0x400,
        BackgroundThreadPriority = 0x800,

        NumThreadPriorities = 3,
        ThreadPriorityMask = 0xC00,
        ThreadPriorityShift = 10,

        /** Combinations **/
#if STATS
        StatsThread_Local = StatsThread | LocalQueue,
#endif
        GameThread_Local = GameThread | LocalQueue,
        ActualRenderingThread_Local = ActualRenderingThread | LocalQueue,

        AnyHiPriThreadNormalTask = AnyThread | HighThreadPriority | NormalTaskPriority,
        AnyHiPriThreadHiPriTask = AnyThread | HighThreadPriority | HighTaskPriority,

        AnyNormalThreadNormalTask = AnyThread | NormalThreadPriority | NormalTaskPriority,
        AnyNormalThreadHiPriTask = AnyThread | NormalThreadPriority | HighTaskPriority,

        AnyBackgroundThreadNormalTask = AnyThread | BackgroundThreadPriority | NormalTaskPriority,
        AnyBackgroundHiPriTask = AnyThread | BackgroundThreadPriority | HighTaskPriority,
    };
    
    struct FRenderThreadStatics
    {
    private:
        // These are private to prevent direct access by anything except the friend functions below
        static CORE_API TAtomic<Type> RenderThread;
        static CORE_API TAtomic<Type> RenderThread_Local;

        // 友元函数，表明在以下函数中可以访问当前结构体FRenderThreadStatics中的私有成员
        friend Type GetRenderThread();
        friend Type GetRenderThread_Local();
        friend void SetRenderThread(Type Thread);
        friend void SetRenderThread_Local(Type Thread);
    };
    
    FORCEINLINE Type GetRenderThread()
    {
        return FRenderThreadStatics::RenderThread.Load(EMemoryOrder::Relaxed);
    }

    FORCEINLINE Type GetRenderThread_Local()
    {
        return FRenderThreadStatics::RenderThread_Local.Load(EMemoryOrder::Relaxed);
    }

    FORCEINLINE void SetRenderThread(Type Thread)
    {
        FRenderThreadStatics::RenderThread.Store(Thread, EMemoryOrder::Relaxed);
    }

    FORCEINLINE void SetRenderThread_Local(Type Thread)
    {
        FRenderThreadStatics::RenderThread_Local.Store(Thread, EMemoryOrder::Relaxed);
    }
}

/** UnrealEngine\Engine\Source\Runtime\Core\Private\Async\TaskGraph.cpp */
// 在cpp中定义static变量并赋初值
namespace ENamedThreads
{
    CORE_API TAtomic<Type> FRenderThreadStatics::RenderThread(ENamedThreads::GameThread); // defaults to game and is set and reset by the render thread itself
    CORE_API TAtomic<Type> FRenderThreadStatics::RenderThread_Local(ENamedThreads::GameThread_Local); // defaults to game local and is set and reset by the render thread itself
}