Netty源码学习（四）Netty服务器是如何启动的？
本文会分析Netty服务器的启动过程，采用的范例代码是Netty编写的Echo Server。
0. 声明acceptor与worker
由于Netty采用的reactor模型，所以需要声明两组线程，一组作为boss/acceptor，另一组作为worker
boss/acceptor负责监听绑定的端口，accept新接入的连接，然后将这些连接转交给worker，worker会处理这些连接上的读写事件。
也就是下面的代码：
EventLoopGroup bossGroup = new NioEventLoopGroup(1);//boss/acceptor线程组
EventLoopGroup workerGroup = new NioEventLoopGroup();//worker线程组
1. 声明ServerBootstrap，并调用bind方法开始监听端口
ServerBootstrap.bind()方法最终会调用到AbstractBootstrap.doBind()方法，其源码如下：
private ChannelFuture doBind(final SocketAddress localAddress) { final ChannelFuture regFuture = initAndRegister(); final Channel channel = regFuture.channel(); if (regFuture.cause() != null) { return regFuture; } if (regFuture.isDone()) {//如果Channel已经register成功，则直接调用doBind0方法 // At this point we know that the registration was complete and successful. ChannelPromise promise = channel.newPromise(); doBind0(regFuture, channel, localAddress, promise);//doBind0方法后文再做分析 return promise; } else {//否则添加一个回调函数，在Channel register成功后再调用doBind0方法 // Registration future is almost always fulfilled already, but just in case it's not. final PendingRegistrationPromise promise = new PendingRegistrationPromise(channel); regFuture.addListener(new ChannelFutureListener() { @Override public void operationComplete(ChannelFuture future) throws Exception { Throwable cause = future.cause(); if (cause != null) { // Registration on the EventLoop failed so fail the ChannelPromise directly to not cause an // IllegalStateException once we try to access the EventLoop of the Channel. promise.setFailure(cause); } else { // Registration was successful, so set the correct executor to use. // See https://github.com/netty/netty/issues/2586 promise.registered(); doBind0(regFuture, channel, localAddress, promise); } } }); return promise; } } final ChannelFuture initAndRegister() { Channel channel = null; try { channel = channelFactory.newChannel();//创建一个新的Channel，这里的channelFactory对象是在初始化ServerBootstrap时调用channel()方法的时候被设置的，在Echo Server范例中，此处会构造一个NioServerSocketChannel init(channel);//见下文分析，初始化Channel，设置属性与选项，以及为这个NioServerSocketChannel添加一个ChannelInitializer，其目的是在这个Channel被register到boss EventLoopGroup的时候，自动添加一个SerrverBootstrapAcceptor } catch (Throwable t) { if (channel != null) { // channel can be null if newChannel crashed (eg SocketException("too many open files")) channel.unsafe().closeForcibly(); } // as the Channel is not registered yet we need to force the usage of the GlobalEventExecutor return new DefaultChannelPromise(channel, GlobalEventExecutor.INSTANCE).setFailure(t);//如果初始化Channel的时候发生异常，则将其包装一下然后返回 } ChannelFuture regFuture = config().group().register(channel);//group()方法得到的是boss EventLoopGroup，register方法很重要，后文再做分析 if (regFuture.cause() != null) { if (channel.isRegistered()) { channel.close(); } else { channel.unsafe().closeForcibly(); } } // If we are here and the promise is not failed, it's one of the following cases: // 1) If we attempted registration from the event loop, the registration has been completed at this point. // i.e. It's safe to attempt bind() or connect() now because the channel has been registered. // 2) If we attempted registration from the other thread, the registration request has been successfully // added to the event loop's task queue for later execution. // i.e. It's safe to attempt bind() or connect() now: // because bind() or connect() will be executed *after* the scheduled registration task is executed // because register(), bind(), and connect() are all bound to the same thread. return regFuture; } @Override void init(Channel channel) throws Exception { final Map<ChannelOption<?>, Object> options = options0(); synchronized (options) { setChannelOptions(channel, options, logger);//设置的channel的options } final Map<AttributeKey<?>, Object> attrs = attrs0(); synchronized (attrs) { for (Entry<AttributeKey<?>, Object> e: attrs.entrySet()) { @SuppressWarnings("unchecked") AttributeKey<Object> key = (AttributeKey<Object>) e.getKey(); channel.attr(key).set(e.getValue());//设置Channel的attribute } } ChannelPipeline p = channel.pipeline();//获取Channel的pipeline，此时为DefaultChannelPipeline，且只有head与tail两个节点 final EventLoopGroup currentChildGroup = childGroup; final ChannelHandler currentChildHandler = childHandler; final Entry<ChannelOption<?>, Object>[] currentChildOptions; final Entry<AttributeKey<?>, Object>[] currentChildAttrs; synchronized (childOptions) { currentChildOptions = childOptions.entrySet().toArray(newOptionArray(childOptions.size())); } synchronized (childAttrs) { currentChildAttrs = childAttrs.entrySet().toArray(newAttrArray(childAttrs.size())); } p.addLast(new ChannelInitializer<Channel>() {//向pipeline中添加一个ChannelInitializer对象 @Override public void initChannel(final Channel ch) throws Exception { final ChannelPipeline pipeline = ch.pipeline(); ChannelHandler handler = config.handler(); if (handler != null) { pipeline.addLast(handler); } ch.eventLoop().execute(new Runnable() { @Override public void run() { pipeline.addLast(new ServerBootstrapAcceptor(//为当前Channel添加一个ServerBootstrapAcceptor，其目的后文会做分析 ch, currentChildGroup, currentChildHandler, currentChildOptions, currentChildAttrs)); } }); } }); }
主要干的事情：
a. 声明了一个Channel对象（实际上是NioServerSocketChannel对象），这个Channel会被register到之前声明的boss NioEventLoopGroup里，然后再调用doBind0方法绑定端口，后文会分析这两个方法的调用链
b. 向这个Channel的pipeline里添加一个ChannelInitializer对象，这个对象继承于ChannelInboundHandler接口，后续初始化的时候会向pipeline里再加一个ServerBootstrapAcceptor对象，它也继承于ChannelInboundHandler接口，这个对象的作用我们会在文末介绍。
2. register方法
register方法的实际调用者为前文声明的boss NioEventLoopGroup，其register实现位于MultithreadEventLoopGroup中，调用链如下：
MultithreadEventLoopGroup.register() @Override public ChannelFuture register(Channel channel) { return next().register(channel);//next方法会从boss NioEventLoopGroup管理的NioEventLoop中挑一个并返回，所以调用的是NioEventLoop.register()方法。而其实际实现位于SingleThreadEventLoop中 } SingleThreadEventLoop.register() @Override public ChannelFuture register(Channel channel) { return register(new DefaultChannelPromise(channel, this)); } @Override public ChannelFuture register(final ChannelPromise promise) { ObjectUtil.checkNotNull(promise, "promise"); promise.channel().unsafe().register(this, promise);//调用channel的unsafe()方法，通过单步可以知道，这里会获得一个AbstractNioMessageChannel.NioMessageUnsafe对象，这个对象是在NioServerSocketChannel初始化时创建的。但是register()方法的实际实现位于AbstractUnsafe中 return promise; } AbstractUnsafe.register() @Override public final void register(EventLoop eventLoop, final ChannelPromise promise) { if (eventLoop == null) { throw new NullPointerException("eventLoop"); } if (isRegistered()) { promise.setFailure(new IllegalStateException("registered to an event loop already")); return; } if (!isCompatible(eventLoop)) { promise.setFailure( new IllegalStateException("incompatible event loop type: " + eventLoop.getClass().getName())); return; } AbstractChannel.this.eventLoop = eventLoop; if (eventLoop.inEventLoop()) { register0(promise);//如果当前线程是EventLoop线程，则直接调用register0方法 } else { try { eventLoop.execute(new Runnable() {//否则给EventLoop提交一个task，最终还是调用register0方法 @Override public void run() { register0(promise); } }); } catch (Throwable t) { logger.warn( "Force-closing a channel whose registration task was not accepted by an event loop: {}", AbstractChannel.this, t); closeForcibly(); closeFuture.setClosed(); safeSetFailure(promise, t); } } } private void register0(ChannelPromise promise) { try { // check if the channel is still open as it could be closed in the mean time when the register // call was outside of the eventLoop if (!promise.setUncancellable() || !ensureOpen(promise)) { return; } boolean firstRegistration = neverRegistered; doRegister();//将channel注册到Selector，具体实现位于AbstractNioChannel中 neverRegistered = false; registered = true; // Ensure we call handlerAdded(...) before we actually notify the promise. This is needed as the // user may already fire events through the pipeline in the ChannelFutureListener. pipeline.invokeHandlerAddedIfNeeded();//这里会将ServerBootstrapAcceptor添加到当前Channel中 safeSetSuccess(promise); pipeline.fireChannelRegistered();//触发pipeline的channelRegistered事件 // Only fire a channelActive if the channel has never been registered. This prevents firing // multiple channel actives if the channel is deregistered and re-registered. if (isActive()) { if (firstRegistration) { pipeline.fireChannelActive(); } else if (config().isAutoRead()) { // This channel was registered before and autoRead() is set. This means we need to begin read // again so that we process inbound data. // // See https://github.com/netty/netty/issues/4805 beginRead(); } } } catch (Throwable t) { // Close the channel directly to avoid FD leak. closeForcibly(); closeFuture.setClosed(); safeSetFailure(promise, t); } } AbstractNioChannel.doRegister() @Override protected void doRegister() throws Exception { boolean selected = false; for (;;) { try { selectionKey = javaChannel().register(eventLoop().unwrappedSelector(), 0, this);//终于，调用JDK NIO的方法，将传入的channel绑定到event loop关联的Selector上，这个Selector是在NioEventLoop初始化时构造的。需要注意的是这里设置的interest ops是0，此时没有任何作用，后续会在doBind0方法里做具体设置 return; } catch (CancelledKeyException e) { if (!selected) { // Force the Selector to select now as the "canceled" SelectionKey may still be // cached and not removed because no Select.select(..) operation was called yet. eventLoop().selectNow(); selected = true; } else { // We forced a select operation on the selector before but the SelectionKey is still cached // for whatever reason. JDK bug ? throw e; } } } }
调用链很长，干的事情倒是比较简单：将传入的Channel注册到某个EventLoop关联的Selector上，然后触发一些相关的回调函数。
3. doBind0方法
调用链如下：
AbstractBootstrap.doBind0() private static void doBind0( final ChannelFuture regFuture, final Channel channel, final SocketAddress localAddress, final ChannelPromise promise) { // This method is invoked before channelRegistered() is triggered. Give user handlers a chance to set up // the pipeline in its channelRegistered() implementation. channel.eventLoop().execute(new Runnable() { @Override public void run() { if (regFuture.isSuccess()) { channel.bind(localAddress, promise).addListener(ChannelFutureListener.CLOSE_ON_FAILURE); } else { promise.setFailure(regFuture.cause()); } } }); } AbstractChannel.bind() @Override public ChannelFuture bind(SocketAddress localAddress, ChannelPromise promise) { return pipeline.bind(localAddress, promise); } DefaultChannelPipeline.bind() @Override public final ChannelFuture bind(SocketAddress localAddress, ChannelPromise promise) { return tail.bind(localAddress, promise); } AbstractChannelHandlerContext.bind() @Override public ChannelFuture bind(final SocketAddress localAddress, final ChannelPromise promise) { if (localAddress == null) { throw new NullPointerException("localAddress"); } if (isNotValidPromise(promise, false)) { // cancelled return promise; } final AbstractChannelHandlerContext next = findContextOutbound(); EventExecutor executor = next.executor(); if (executor.inEventLoop()) {//根据当前线程是否为event loop线程而采取策略 next.invokeBind(localAddress, promise); } else { safeExecute(executor, new Runnable() { @Override public void run() { next.invokeBind(localAddress, promise); } }, promise, null); } return promise; } private void invokeBind(SocketAddress localAddress, ChannelPromise promise) { if (invokeHandler()) { try { ((ChannelOutboundHandler) handler()).bind(this, localAddress, promise); } catch (Throwable t) { notifyOutboundHandlerException(t, promise); } } else { bind(localAddress, promise); } } DefaultChannelPipeline.bind() @Override public void bind( ChannelHandlerContext ctx, SocketAddress localAddress, ChannelPromise promise) throws Exception { unsafe.bind(localAddress, promise);//这里的unsafe是AbstractNioMessageChannel.NioMessageUnsafe } AbstractChannel.bind() @Override public final void bind(final SocketAddress localAddress, final ChannelPromise promise) { assertEventLoop(); if (!promise.setUncancellable() || !ensureOpen(promise)) { return; } // See: https://github.com/netty/netty/issues/576 if (Boolean.TRUE.equals(config().getOption(ChannelOption.SO_BROADCAST)) && localAddress instanceof InetSocketAddress && !((InetSocketAddress) localAddress).getAddress().isAnyLocalAddress() && !PlatformDependent.isWindows() && !PlatformDependent.maybeSuperUser()) { // Warn a user about the fact that a non-root user can't receive a // broadcast packet on *nix if the socket is bound on non-wildcard address. logger.warn( "A non-root user can't receive a broadcast packet if the socket " + "is not bound to a wildcard address; binding to a non-wildcard " + "address (" + localAddress + ") anyway as requested."); } boolean wasActive = isActive(); try { doBind(localAddress); } catch (Throwable t) { safeSetFailure(promise, t); closeIfClosed(); return; } if (!wasActive && isActive()) { invokeLater(new Runnable() { @Override public void run() { pipeline.fireChannelActive();//这里会设置对ACCEPT事件的监听，后文再做分析 } }); } safeSetSuccess(promise); } NioServerSocketChannel.bind() @Override protected void doBind(SocketAddress localAddress) throws Exception { if (PlatformDependent.javaVersion() >= 7) { javaChannel().bind(localAddress, config.getBacklog());//调用JDK的NIO提供的方法，将传入的channel与指定的端口绑定，并设置backlog大小 } else { javaChannel().socket().bind(localAddress, config.getBacklog()); } }
很长很长的调用链，最终做的事情是调用JDK NIO提供的bind函数，将channel与指定的端口绑定。
在AbstractChannel.bind()方法中，提交了一个异步任务，里面只有一行代码：pipeline.fireChannelActive()，这行代码经过一系列调用之后，会执行AbstractChannel.AbstractUnsafe.doBeginRead()方法
然后又会执行到AbstractNioChannel.doBeginRead()方法，其代码如下：
@Override protected void doBeginRead() throws Exception { // Channel.read() or ChannelHandlerContext.read() was called final SelectionKey selectionKey = this.selectionKey; if (!selectionKey.isValid()) { return; } readPending = true; final int interestOps = selectionKey.interestOps(); if ((interestOps & readInterestOp) == 0) { selectionKey.interestOps(interestOps | readInterestOp);//设置interestOps } }
虽然此时selectionKey.interestOps()还没有被设置，但是readInterestOp是一个全局变量，在NioServerSocketChannel初始化的时候就会被设置为SelectionKey.OP_ACCEPT。
于是在doBeginRead方法中，Channel所关注的IO事件就会被设置为ACCEPT事件了。
这样如果有客户端连接进来，就会触发关联的EventLoop里的相关代码，并作出处理了。
4. ServerBootstrapAcceptor
ServerBootstrapAcceptor的关键代码是channelRead方法
@Override @SuppressWarnings("unchecked") public void channelRead(ChannelHandlerContext ctx, Object msg) { final Channel child = (Channel) msg; child.pipeline().addLast(childHandler);//给child channel的pipeline添加用户自定义的handler setChannelOptions(child, childOptions, logger);//设置child channel的options for (Entry<AttributeKey<?>, Object> e: childAttrs) { child.attr((AttributeKey<Object>) e.getKey()).set(e.getValue());//设置child channel的attribute } try {//将child channel注册到worker event loop group里，Netty会根据round-robin算法选择一个worker线程来做绑定 childGroup.register(child).addListener(new ChannelFutureListener() { @Override public void operationComplete(ChannelFuture future) throws Exception { if (!future.isSuccess()) { forceClose(child, future.cause()); } } }); } catch (Throwable t) { forceClose(child, t); } }
在服务器收到新链接的时候，这个函数会被触发，然后设置Channel的各种属性与关联的pipeline
这个Channel接着会被交付给worker eventloop group里的一个worker，然后这个Channel上发生的任何读写事件都是由这个worker来处理了
