io_uring, barrier, and lock-free

With some clever use of shared ring buffers, io_uring performance is really memory-bound, since in polling mode, we can do away with system calls altogether. It is important to remember that performance benchmarking is a relative process with some kind of a common point of reference. According to the io_uring paper, on a reference machine, in polling mode, io_uring managed to clock 1.7M 4k IOPS, while aio(7) manages 608k. Although much more than double, this isn’t a fair comparison since aio(7) doesn’t feature a polled mode. But even when polled mode is disabled, io_uring hits 1.2M IOPS, close to double that of aio(7).

The Low-level io_uring Interface — Lord of the io_uring documentation (unixism.net)

// Read completion events, get the data buffer and print it to the console.
void read_from_completion_quque (submitter* s) {
    ...
    head = *cring->head;
    do {
        read_barrier();
        if (head == *cring->tail) break; // The queue is empty
        cqe = &cring->cqes[head & *s->cq_ring.ring_mask];
        fi = (struct file_info*) cqe->user_data;
        int blocks = (int) fi->file_sz / BLOCK_SZ;
        if (fi->file_sz % BLOCK_SZ) blocks++;
        for (int i = 0; i < blocks; i++) print_to_console(fi->iovecs[i].iov_base, fi->iovecs[i].iov_len);
        head++;
    } while (1);
    *cring->head = head;
    write_barrier();
}

barrier用作轻量级(light weight)同步：lwsync.

Memory Barriers Are Like Source Control Operations (preshing.com)

Like compiler reordering, processor reordering is invisible to a single-threaded program. It only becomes apparent when lock-free techniques are used – that is, when shared memory is manipulated without any mutual exclusion between threads.

Memory Ordering at Compile Time (preshing.com)

Lockless Programming Considerations for Xbox 360 and Microsoft Windows - Win32 apps | Microsoft Docs

_ReadWriteBarrier | Microsoft Docs

The _ReadBarrier, _WriteBarrier, and _ReadWriteBarrier compiler intrinsics and the MemoryBarrier macro are all deprecated and should not be used. For inter-thread communication, use mechanisms such as atomic_thread_fence and std::atomic<T>, which are defined in the C++ Standard Library. For hardware access, use the /volatile:iso compiler option together with the volatile keyword.

Memory Barrier Instruction - an overview | ScienceDirect Topics

Intel® 64 Architecture Memory Ordering White Paper

memory barrier, interlocked operations - Intel Communities