Redis源码学习--AOF日志重写01

AOF Rewrite触发场景

函数rewriteAppendOnlyFileBackground实现了AOF Rewrite操作，并在下列三种场景中被调用：

执行bgrewriteaof命令触发，通过bgrewriteaofCommand函数来调用。
开启appendonly参数触发，通过startAppendOnly函数来调用。
满足AOF Rewrite条件触发，通过周期执行函数serverCron来调用。

bgrewriteaofCommand函数

当客户端执行bgrewriteaof命令时，会交由函数bgrewriteaofCommand进行处理：

void bgrewriteaofCommand(client *c) {
    if (server.aof_child_pid != -1) {
        addReplyError(c,"Background append only file rewriting already in progress");
    } else if (server.rdb_child_pid != -1) {
        server.aof_rewrite_scheduled = 1;
        addReplyStatus(c,"Background append only file rewriting scheduled");
    } else if (rewriteAppendOnlyFileBackground() == C_OK) {
        addReplyStatus(c,"Background append only file rewriting started");
    } else {
        addReply(c,shared.err);
    }
}

startAppendOnly函数

当客户端设置参数appendonly yes来开启AOF日志，或则主从复制重新同步后开启AOF日志，都会调用startAppendOnly函数执行：

/* Called when the user switches from "appendonly no" to "appendonly yes"
 * at runtime using the CONFIG command. */
int startAppendOnly(void) {
    char cwd[MAXPATHLEN]; /* Current working dir path for error messages. */
    int newfd;

    newfd = open(server.aof_filename,O_WRONLY|O_APPEND|O_CREAT,0644);
    serverAssert(server.aof_state == AOF_OFF);
    if (newfd == -1) {
        char *cwdp = getcwd(cwd,MAXPATHLEN);

        serverLog(LL_WARNING,
            "Redis needs to enable the AOF but can't open the "
            "append only file %s (in server root dir %s): %s",
            server.aof_filename,
            cwdp ? cwdp : "unknown",
            strerror(errno));
        return C_ERR;
    }
    if (server.rdb_child_pid != -1) {
        server.aof_rewrite_scheduled = 1;
        serverLog(LL_WARNING,"AOF was enabled but there is already a child process saving an RDB file on disk. An AOF background was scheduled to start when possible.");
    } else {
        /* If there is a pending AOF rewrite, we need to switch it off and
         * start a new one: the old one cannot be reused because it is not
         * accumulating the AOF buffer. */
        if (server.aof_child_pid != -1) {
            serverLog(LL_WARNING,"AOF was enabled but there is already an AOF rewriting in background. Stopping background AOF and starting a rewrite now.");
            killAppendOnlyChild();
        }
        if (rewriteAppendOnlyFileBackground() == C_ERR) {
            close(newfd);
            serverLog(LL_WARNING,"Redis needs to enable the AOF but can't trigger a background AOF rewrite operation. Check the above logs for more info about the error.");
            return C_ERR;
        }
    }
    /* We correctly switched on AOF, now wait for the rewrite to be complete
     * in order to append data on disk. */
    server.aof_state = AOF_WAIT_REWRITE;
    server.aof_last_fsync = server.unixtime;
    server.aof_fd = newfd;
    return C_OK;
}

在Redis主从复制全量同步过程中，从节点加载解析RDB文件过程中，会临时性关闭AOF日志，等RDB文件加载结束后，无论加载成功或失败，都会通过restartAOFAfterSYNC函数来恢复临时关闭的AOF日志。

/* This function will try to re-enable the AOF file after the
 * master-replica synchronization: if it fails after multiple attempts
 * the replica cannot be considered reliable and exists with an
 * error. */
void restartAOFAfterSYNC() {
    unsigned int tries, max_tries = 10;
    for (tries = 0; tries < max_tries; ++tries) {
        if (startAppendOnly() == C_OK) break;
        serverLog(LL_WARNING,
            "Failed enabling the AOF after successful master synchronization! "
            "Trying it again in one second.");
        sleep(1);
    }
    if (tries == max_tries) {
        serverLog(LL_WARNING,
            "FATAL: this replica instance finished the synchronization with "
            "its master, but the AOF can't be turned on. Exiting now.");
        exit(1);
    }
}

serverCron函数

Redis会周期检查AOF文件大小来判断是否需要进行AOF Rewrite操作：

/* This is our timer interrupt, called server.hz times per second.
 * Here is where we do a number of things that need to be done asynchronously.
 * For instance:
 *
 * - Triggering BGSAVE / AOF rewrite, and handling of terminated children.

 */

int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {

    /* Start a scheduled AOF rewrite if this was requested by the user while
     * a BGSAVE was in progress. */
    if (server.rdb_child_pid == -1 && server.aof_child_pid == -1 &&
        server.aof_rewrite_scheduled)
    {
        rewriteAppendOnlyFileBackground();
    }

    /* Check if a background saving or AOF rewrite in progress terminated. */
    if (server.rdb_child_pid != -1 || server.aof_child_pid != -1 ||
        ldbPendingChildren())
    {
        /* */
    } else {

        /* Trigger an AOF rewrite if needed. */
        if (server.aof_state == AOF_ON &&
            server.rdb_child_pid == -1 &&
            server.aof_child_pid == -1 &&
            server.aof_rewrite_perc &&
            server.aof_current_size > server.aof_rewrite_min_size)
        {
            long long base = server.aof_rewrite_base_size ?
                server.aof_rewrite_base_size : 1;
            long long growth = (server.aof_current_size*100/base) - 100;
            if (growth >= server.aof_rewrite_perc) {
                serverLog(LL_NOTICE,"Starting automatic rewriting of AOF on %lld%% growth",growth);
                rewriteAppendOnlyFileBackground();
            }
        }
    }
}

可以通过下面两个参数来控制AOF Rewrite操作：

参数auto-aof-rewrite-min-size，当AOF文件超过该阈值后才可能触发AOF重写，避免对较小的AOF文件进行重写，默认为64MB。
参数auto-aof-rewrite-percentage，当AOF文件增长比例超过该阈值后才会触发AOF重写，基准值是上一次AOF重写后的AOF文件大小，默认值为100，即"当前AOF文件大小"是"上一次AOF重写后文件大小"的2倍时触发AOF重写。

当AOF重写完成时，会设置AOF重写的基准值：

/* A background append only file rewriting (BGREWRITEAOF) terminated its work.
 * Handle this. */
void backgroundRewriteDoneHandler(int exitcode, int bysignal) {
            /* 更新AOF重写后AOF文件大小 */
            aofUpdateCurrentSize();
           /* 设置下一次AOF重写的基准值 */
            server.aof_rewrite_base_size = server.aof_current_size;
            server.aof_fsync_offset = server.aof_current_size;
}

AOF重写信息查看

可以通过客户端命令info persistence来查看AOF相关信息：

# 执行命令
redis-cli -p 6566 info persistence

# 输出结果
# Persistence
loading:0
rdb_changes_since_last_save:174
rdb_bgsave_in_progress:0
rdb_last_save_time:1654524706
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:52
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:970752
# 是否开启AOF日志
aof_enabled:1
# 是否正在AOF重写
aof_rewrite_in_progress:0
# AOF重写调度设置
aof_rewrite_scheduled:0
# 最近一次AOF重写消耗时间
aof_last_rewrite_time_sec:52
# 当前AOF重写消耗时间，-1表示当前未处于重写状态
aof_current_rewrite_time_sec:-1
# 最近一次AOF重写状态
aof_last_bgrewrite_status:ok
# 最近一次AOF写日志状态
aof_last_write_status:ok
# 最后一次AOF重写过程中，分配给copy-on-write的内存大小(单元byte)
aof_last_cow_size:937984
# 当前AOF文件大小(单元byte)
aof_current_size:2460435827
# 上一次AOF重写后的文件大小(单元byte)
aof_base_size:2459418938
# 是否等待AOF重写
aof_pending_rewrite:0
aof_buffer_length:0
aof_rewrite_buffer_length:0
aof_pending_bio_fsync:0
aof_delayed_fsync:0

AOF重写并发控制

为保证AOF重写的准确性，需要保证同一时间只能有一个AOF Rewrite线程执行，为保证AOF重写性能和避免AOF重写消耗过多服务器资源，需要避免AOF重写和RDB备份同时进行，因此在开启AOF重写调度(server.aof_rewrite_scheduled = 1) 和创建AOF重写线程时，都会检查当前是否有RDB备份进程和AOF重写进程。

posted @ 2022-06-06 23:08 TeyGao 阅读(188) 评论(0) 收藏举报

刷新页面返回顶部