Netty

 

Outline of Netty

In distributed architectures, network communication is of paramount importance. Regardless of how sophisticated the architecture is, the performance of network communication directly affects user experience. In Java, there are numerous frameworks for handling network operations, along with fundamental technologies such as NIO, BIO, and Socket, which we have discussed previously. However, when developing based on these technologies, we find that the APIs they provide are relatively complex, giving rise to open-source frameworks like Mina and Netty. Almost all middleware involves network communication, such as Zookeeper, Dubbo, Redis, RocketMQ, etc. Some of them leverage Netty at their core. In the context of distributed architectures, the significance of network communication is self-evident. Therefore, I intend to engage in a discussion with you about Netty.

The development of Network IO

BIO Synchronous Blocking IO: When a client sends a request to the server, the server must block until the client writes data to it. This means the server can only handle a limited number of concurrent requests (due to queuing).

[NIO (Non-blocking IO)]: Utilizes thread polling; if no connections are available, it returns immediately. This introduces performance issues (context switching between user and kernel modes).

[Multiplexing (epoll)]: When a client connects, its connection is registered with a Selector. The Selector responds to connection or IO events from any registered Channel (client connection). A worker thread polls the list of ready channels and processes them accordingly.

[Reactor Pattern]: A high-performance design pattern built on NIO. It separates IO handling from business logic, allowing IO events to be processed by one or more threads, enabling flexible scaling. Redis uses a single-threaded single-reactor model, which can bottleneck when handling multiple connections.

[Multi-threaded Single Reactor Model]: Connections are processed asynchronously using a thread pool, eliminating blocking.

[Multi-threaded Multi-Reactor Model]: A main reactor accepts client connections and distributes them to multiple sub-reactors for IO processing.

Key Distinctions:

• Synchronous Non-blocking: IO operations do not block, but the application must actively check for completion.
• Asynchronous Non-blocking: IO operations complete independently; the application is notified via callbacks.

Netty: A high-level NIO framework that simplifies development. What might require multiple classes in raw NIO can be achieved in just a few lines with Netty.

Outline of Netty

NettyIt provides support for the above three Reactor models, and compared with the native API of NIO, it has the following characteristics：

• It provides an efficient I/O model, thread model and time processing mechanism. It offers a very simple and easy-to-use API. Compared with NIO, it provides a higher-level encapsulation for basic apis such as Channel, Selector, Sockets and Buffers. Shielding the complexity of NIO provides excellent support stability for data protocols and serialization.
• Netty has fixed many issues of JDK NIO, such as 100% CPU consumption caused by idle select, TCP reconnection after disconnection, and keep-alive detection.
• In terms of performance optimization, as a network communication framework, it needs to handle a large number of network requests, and thus inevitably faces the problem of creating and destroying network objects. This is not very friendly to the JVM's GC. To reduce the pressure of JVM garbage collection, two optimization mechanisms are introduced: object pool reuse and zero-copy technology.

Usage of Netty

Introducing Netty's package

<dependency>
    <groupId>io.netty</groupId>
    <artifactId>netty-all</artifactId>
</dependency>

Use Netty to implement a multi-threaded multi-reactor model

Overall process:

• When a client comes in, it first receives the client's connection through the mainGroup
• The connection is then registered in an EventLoopGroup, which contains multiple eventLoops, each representing a thread, i.e. the client registers into one of the eventloops.
• When an eventLoop has an I/O event, it sends the request to the channel pipeline for processing
• Channel pipelines can be multiple, we can add them all the time, and they will be executed in the order in which we added them。　　
  • 　　channel  pipeline There are two types of inbound and outbound, that is, read and output.

// Handlers that specifically handle IOs
public class NormalMessageHandler extends ChannelInboundHandlerAdapter {
    //When the request comes, this method is called to read the data
    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        ByteBuf in = (ByteBuf) msg;
        byte[] req=new byte[in.readableBytes()];
        in.readBytes(req);
        System.out.println("Data received by the server"+new String(req, StandardCharsets.UTF_8));
        ByteBuf resp= Unpooled.copiedBuffer("Service receive message success".getBytes());
        ctx.write(resp);
    }
</span><span style="color: #008000;">//</span><span style="color: #008000;">Write the data back to the client and listen for the client shutdown event</span>

    @Override

public void channelReadComplete(ChannelHandlerContext ctx) {

ctx.write(Unpooled.EMPTY_BUFFER).addListener(ChannelFutureListener.CLOSE);

}
@Override
</span><span style="color: #0000ff;">public</span> <span style="color: #0000ff;">void</span> exceptionCaught(ChannelHandlerContext ctx, Throwable cause) <span style="color: #0000ff;">throws</span><span style="color: #000000;"> Exception {
    </span><span style="color: #0000ff;">super</span><span style="color: #000000;">.exceptionCaught(ctx, cause);
}

}

public class NettyBasicServerExample {
    // Develop a multi-reactor, multi-threaded model
    public static void main(String[] args) {
        // Main thread (equivalent to our main reactor)
        EventLoopGroup mainGroup=new NioEventLoopGroup();
        //Represents multiple groups of workers (equivalent to our sub reactors) registering our events
        EventLoopGroup workGroup=new NioEventLoopGroup(4);
        //Build the API for the netty server
        ServerBootstrap serverBootstrap=new ServerBootstrap();
        // Use Nio's API and build a handler (this handler will handle the messages)
        serverBootstrap.group(mainGroup,workGroup)
                // Specifies the model to use
                .channel(NioServerSocketChannel.class)
                //Specific job processing classes that handle related channel io events
                .childHandler(new ChannelInitializer<SocketChannel>() {
                    @Override
                    protected void initChannel(SocketChannel socketChannel) {
                        //Handling I/O events
                        socketChannel.pipeline().addLast(new NormalMessageHandler());
                    }
                });
        // Synchronous blocking waits until the client callbacks
        try {
            ChannelFuture sync = serverBootstrap.bind(8080).sync();
            System.out.println("Netty server started successfully and the port 8080 has been listened");
            // Synchronization waits for the server port to be closed
            sync.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }finally {
            // Free up resources
            mainGroup.shutdownGracefully();
            workGroup.shutdownGracefully();
        }
    }
}

more detail about netty

Event Scheduler（It is through the Reactor thread model to aggregate and process various events, and through the Selector main loop thread to integrate a variety of events (I/O time, signal time), when these events are triggered, the specific processing of the event needs to be processed by the relevant Handler in the service orchestration layer)

There are two core components of an event scheduler:

• EventLoopGroup：It's actually the equivalent of a thread pool
• EventLoop：This is equivalent to a thread in a thread pool

Generally:

• An EventLoopGroup contains multiple eventloops, which are used to handle all I/O events in the channel lifecycle, such as accept, connect, read, write, etc
• EventLoops are bound to one thread at a time, and each EventLoop is responsible for handling multiple Channel
• Each time a new channel is created, the EventLoopGroup selects an EventLoop to bind, and the channel can bind and unbind the EventLoop multiple times during its lifetime.
• We can simply think of EventLoopGroup as a concrete implementation of the Reactor thread model in Netty, and we can configure different EventLoopGroups to make Netty support a variety of different Reactor models。
  • 【Single-threaded model】：The EventLoopGroup contains only one EventLoop，Boss and Worker use the same EventLoopGroup
  • 【Multi-threaded model】：EventLoopGroup 包含多个 EventLoop,Boss和 Worker 使用同一个 EventLoopGroup。
  • 【Master-slave multithreaded model】：EventLoopGroup包含多个EventLoop，Boss是主Reactor，Worker是从 Reactor。They each use a different EventLoopGroup，The primary reactor is responsible for the creation of new network connection channels (i.e., connection events)，After the primary reactor receives the connection from the client, it is handled by the secondary reactor。

Service orchestration layer（That is, after the I/O event is triggered, a Handler is required to deal with it, and the service orchestration layer can realize the dynamic orchestration and orderly propagation of network events through a Handler processing chain)

It consists of three components：

【ChannelPipeline】：It adoptedA bibievously linked list links multiple Channelhandlers together, and when an I/O event is triggered, the ChannelPipeline doesInvoke the assembled ChannelHandlers in turn，Implement data processing on the channel. The ChannelPipeline is thread-safe，Because of eachA new channel is bound to a new ChannelPipeline。A ChannelPipeline is associated with an EventLoop，An EventLoop will only bind one thread，

【ChannelHandler】： For processors with I/O data, the data is received and processed by a designated handler

【ChannelHandlerContext】，ChannelHandlerContextContextual information for the ChannelHandler，That is, when the event is triggered，Data is passed between multiple handlers through the ChannelHandlerContext。Each ChannelHandler corresponds to its own ChannelHandlerContext，It retains the contextual information that the ChannelHandler needs，Data transfer between multiple ChannelHandlers is achieved through the ChannelHandlerContext。

General working mechanism：

The service order starts to initialize the Boss and Worker thread group, the Boss thread group is responsible for listening for network connection events, when a new connection is established, the Boss thread will register the connection Channel and bind it to the Worker thread, and the Worker thread group will assign an EventLoop to handle the read and write events of the Channel, and each EventLoop is equivalent to a thread. Use the selector to listen to events in a loop. When the client initiates an I/O event, the server's EventLoop is ready to be distributed to the Pipeline for data processing After the data is transmitted to the ChannelPipeline, it is processed from the first ChannelInBoundHandler and passed one by one according to the pipeline chain After the server processing is completed, the data should be written back to the client. This writeback data is propagated through the chain of ChannelOutboundHandlers and finally reaches the client