RocketMQ存储篇三:追加消息

第一篇介绍了存储的概览,定位到消息追加消息处细看一下
获取mappedFile ——> 追加消息 ——> 处理结果
image

@DefaultMappedFile#appendMsgInner()

public AppendMessageResult appendMessagesInner(final MessageExt messageExt, final AppendMessageCallback cb,
        PutMessageContext putMessageContext) {
        assert messageExt != null;
        assert cb != null;

        int currentPos = WROTE_POSITION_UPDATER.get(this);

        if (currentPos < this.fileSize) {
            ByteBuffer byteBuffer = appendMessageBuffer().slice();
            byteBuffer.position(currentPos);
            AppendMessageResult result;
            if (messageExt instanceof MessageExtBatch && !((MessageExtBatch) messageExt).isInnerBatch()) {
                // traditional batch message
                result = cb.doAppend(this.getFileFromOffset(), byteBuffer, this.fileSize - currentPos,
                    (MessageExtBatch) messageExt, putMessageContext);
            } else if (messageExt instanceof MessageExtBrokerInner) {
                // traditional single message or newly introduced inner-batch message
				// 使用回调函数来处理append消息的过程
                result = cb.doAppend(this.getFileFromOffset(), byteBuffer, this.fileSize - currentPos,
                    (MessageExtBrokerInner) messageExt, putMessageContext);
            } else {
                return new AppendMessageResult(AppendMessageStatus.UNKNOWN_ERROR);
            }
            WROTE_POSITION_UPDATER.addAndGet(this, result.getWroteBytes());
            this.storeTimestamp = result.getStoreTimestamp();
            return result;
        }
        log.error("MappedFile.appendMessage return null, wrotePosition: {} fileSize: {}", currentPos, this.fileSize);
        return new AppendMessageResult(AppendMessageStatus.UNKNOWN_ERROR);
    }

@DefaultAppendMessageCallback#doAppend()
处理逻辑

  1. 根据bytebuffer中的wroteposition以及mappedfile开头获取物理offset
    long wroteOffset = fileFromOffset + byteBuffer.position();
  2. 构造msgId
  3. 判断mappedfile是否有足够的空间写msg, 如果没有,就返回一个mappedFile用完的状态,重新分配mappedfile并写入,如果空间足够,就修改encode阶段消息中的一些假值并返回PUT_OK状态
    if ((msgLen + END_FILE_MIN_BLANK_LENGTH) > maxBlank)
  4. 根据put的状态来做处理(刷盘以及ha)

那么,消息到底追加到了哪里呢?
如果开启了trasientPoolEnable,就是writeBuffer,否则是mappedByteBuffer
writeBuffer是从trasientPool中borrow的,应用了池化的思想,在刷盘时有作用,后面叙述
mappedByteBuffer就是commitlog文件的映射

    protected ByteBuffer appendMessageBuffer() {
        this.mappedByteBufferAccessCountSinceLastSwap++;
        return writeBuffer != null ? writeBuffer : this.mappedByteBuffer;
    }
posted @ 2023-01-05 21:35  风卷红旗过大江  阅读(84)  评论(0)    收藏  举报