09 7个检验软件系统稳定性场景的混沌工程故障注入开源工具ChaosBlade

ChaosBlade可针对多达7个场景开展故障注入实验，但网上官方的中英文文档质量欠佳，内容缺失，真心没有站在一般用户的角度来写，只能通过运行blade命令的help了解究竟有什么功能。

官网

https://github.com/chaosblade-io/chaosblade

一句话介绍

ChaosBlade是阿里巴巴开源的针对7个检验软件系统稳定性场景的混沌工程故障注入开源工具：主机基础资源、CRI容器、K8S平台、Java应用、C++应用、阿里云平台、其他服务。

关键特性

以下关键特性，是我下载ChaosBlade v1.7.2，在运行blade时带上help参数，一点点试出来的。官网github的readme文档并没有写这么细。

场景1：主机基础资源

可注入故障包括CPU负载爆满，磁盘占用和读写爆满，内存爆满，网络延迟/丢包/坏包/乱序，端口被占用，DNS配置错误，杀进程。

场景2：CRI容器

可注入故障包括向容器内的基础资源注入故障，删除容器，以及向容器内各种服务注入故障。

场景3：K8S平台

可注入故障包括向K8S平台内容器、node和pod注入故障。

场景4：Java应用

可注入故障包括代码缓存爆满，内存不足，增加延迟，返回特定值，动态执行脚本，抛异常等。

场景5：C++应用

可注入故障包括增加延迟，运行中修改变量值，返回错误。

场景6：阿里云平台

可注入故障包括卸载磁盘，挂载或卸载网络接口等。

场景7：其他服务

可注入故障包括增加延迟和抛异常，可以针对如elastic search, hbase, http, mongodb, mysql等近30种服务注入故障。

具体特性，可以参考文后我根据命令行help输出，整理的特性列表。需要对里面的英文语法错误保持宽容态度。

适用平台

主机，CRI容器，K8S平台，Java应用，C++应用，阿里云平台，其他服务。

适用场景

需要向主机、CRI容器、K8S平台、阿里云平台、Java应用和C++应用注入故障的场景。

优势

支持多达7个故障注入场景。

劣势

网上的中英文文档描述过于简略。每个功能往往就一句话。只能把工具装上，运行起来，通过help参数，逆向工程来发现有什么功能。

案例

国内有金融行业的用户，已经基于ChaosBlade开发了混沌工程工具平台，在测试环境注入故障，检验基础设施和容器平台的稳定性。

开源许可证

Apache-2.0 license

Github点赞数

5.4k

最近发布日期与版本

2023.05: v1.7.2

所属项目

CNCF云原生计算基金会沙箱项目

参考资料

ChaosBlade文档：https://chaosblade.io/docs/

ChaosBlade使用入门：https://developer.aliyun.com/article/1196882

ChaosBlade使用案例：https://juejin.cn/post/6996923672564858916?searchId=2023080922085236FCE1AB509C0489C977

混沌工程开源工具 Chaosblade 探究：https://juejin.cn/post/7037154744225234981?searchId=2023080922085236FCE1AB509C0489C977#heading-5

---

你还知道有什么好用的开源故障注入工具？你还希望我聊有关混沌工程的其他什么新话题？欢迎在评论区留言。我会仔细阅读每一条留言。期待听到你的声音。

如果觉得本文对你有帮助，欢迎*点赞*，【点击在读，】并*转发*给其他志同道合的小伙伴。

企业生意好，系统运行稳。你所阅读的文章，来自“吾真本说混沌工程”知乎专栏。

附录：ChaosBlade支持7个故障注入场景

1 主机基础资源

  cpu         Cpu experiment, for example full load

    fullload    cpu load

  disk        Disk experiment contains fill disk or burn io

    burn        Increase disk read and write io load

    fill        Fill the specified directory path

  mem         Mem experiment, for example load

    load        mem load

  network     Network experiment

    corrupt     Corrupt experiment

    delay       Delay experiment

    dns         Dns experiment

    dns_down    Make DNS is not accessible

    drop        Drop experiment

    duplicate   Duplicate experiment

    loss        Loss network package

    occupy      Occupy the specify port

    reorder     Reorder experiment

  process     Process experiment

    kill        Kill process

    stop        process fake death

  file        File experiment contains file content append, permission modification so on

    add         File or path add

    append      File content append

    chmod       File permission modification.

    delete      File delete

    move        File move

2 CRI容器

  cri         CRI experiment, for example remove container. If container-runtime is containerd, the container-id shoud be full id.

    ck          Clickhouse experiment 

    container   Execute a container experiment

    cpu         Cpu experiment

    disk        Disk experiment

    druid       Experiment with the Druid

    dubbo       Experiment with the Dubbo

    es          ElasticSearch experiment!

    feign       feign experiment

    file        File experiment

    gateway     gateway experiment!

    hbase       hbase experiment!

    http        http experiment

    jedis       jedis experiment

    jvm         Experiment with the JVM

    kafka       kafka experiment

    lettuce     redis client lettuce experiment

    log         log experiment

    mem         Mem experiment

    mongodb     MongoDB experiment

    mysql       mysql experiment

    network     Network experiment

    process     Process experiment

    psql        Postgrelsql experiment

    rabbitmq    rabbitmq experiment

    redisson    redisson experiment

    rocketmq    Rocketmq experiment,can make message send or pull delay and exception

    security    SpringSecurity login experiment

    servlet     java servlet experiment

    tars        tars experiment

    zk          zk experiment

3 K8S平台

  k8s         Kubernetes experiment, for example kill pod

    container-ck        Clickhouse experiment 

    container-container Execute a container experiment

    container-cpu       Cpu experiment

    container-disk      Disk experiment

    container-druid     Experiment with the Druid

    container-dubbo     Experiment with the Dubbo

    container-es        ElasticSearch experiment!

    container-feign     feign experiment

    container-file      File experiment

    container-gateway   gateway experiment!

    container-hbase     hbase experiment!

    container-http      http experiment

    container-jedis     jedis experiment

    container-jvm       Experiment with the JVM

    container-kafka     kafka experiment

    container-lettuce   redis client lettuce experiment

    container-log       log experiment

    container-mem       Mem experiment

    container-mongodb   MongoDB experiment

    container-mysql     mysql experiment

    container-network   Network experiment

    container-process   Process experiment

    container-psql      Postgrelsql experiment

    container-rabbitmq  rabbitmq experiment

    container-redisson  redisson experiment

    container-rocketmq  Rocketmq experiment,can make message send or pull delay and exception

    container-script    Script chaos experiment

    container-security  SpringSecurity login experiment

    container-servlet   java servlet experiment

    container-tars      tars experiment

    container-zk        zk experiment

    node-cpu            Cpu experiment

    node-disk           Disk experiment

    node-file           File experiment

    node-mem            Mem experiment

    node-network        Network experiment

    node-process        Process experiment

    node-script         Script chaos experiment

    pod-cpu             Cpu experiment

    pod-disk            Disk experiment

    pod-file            File experiment

    pod-mem             Mem experiment

    pod-network         Network experiment

    pod-pod             Pod experiments

    pod-process         Process experiment

    pod-script          Script chaos experiment

4 Java应用

  jvm         Experiment with the JVM, and you can specify classes, method injection delays, return values, exception failure scenarios, or write Groovy and Java scripts to implement complex scenarios

    CodeCacheFilling       Fill up code cache.

    OutOfMemoryError       JVM out of memory

    cpufullload            Process occupied cpu full load

    delay                  delay time

    full-gc                JVM full gc

    return                 Return the specify value

    script                 Dynamically execute custom scripts

    threadfull             Specifies that the application thread is soaring

    throwCustomException   throw custom exception

    throwDeclaredException Throw the first declared exception of method

5 C++应用

  cplus       C++ chaos experiments contain code line delayed, variable modified and err returned

    delay       Code line delayed

    modify      Modify value of the variable in source code when program running

    return      error returned

6 阿里云

  aliyun      Aliyun experiment contains ecs, public ip, private ip, networkInterface, securityGroup, VSwitch, disk

    disk             do some aliyun diskId Operations, like detach

    ecs              do some aliyun ecs Operations, like stop, start, reboot

    networkInterface do some aliyun networkInterfaceId Operations, like detach, attach

    privateIp        do some aliyun private ip Operations, like unassign

    publicIp         do some aliyun publicIp Operations, like release, unassociate

    securityGroup    do some aliyun securityGroupId Operations, like join, remove

    vSwitch          do some aliyun vSwitchId Operations, like delete

7 其他服务

  ck          Clickhouse experiment contains delay and exception by table name and so on

    delay                delay time

    throwCustomException throw custom exception

  druid       Experiment with the Druid database connection pool, For example `blade create druid connectionpoolfull

    connectionpoolfull Connection pool full

  dubbo       Dubbo experiment for testing service delay and exception

    delay                delay time

    threadpoolfull       Thread pool full

    throwCustomException throw custom exception

  es          ElasticSearch experiment contains delay and exception by table name and so on

    delay                delay time

    throwCustomException throw custom exception

  feign       feign experiment for testing service api delay and exception

    delay                delay time

    throwCustomException throw custom exception

  gateway     gateway experiment contains delay and exception

    delay                delay time

    throwCustomException throw custom exception

  hbase       hbase experiment contains delay and exception by table name and so on

    delay                delay time

    throwCustomException throw custom exception

  http        http experiment for testing service delay and exception

    delay                delay time

    throwCustomException throw custom exception

  jedis       jedis experiment contains delay and exception by command and so on

    delay                delay time

    throwCustomException throw custom exception

  kafka       kafka experiment for testing service delay and exception

    delay                delay time

    throwCustomException throw custom exception

  lettuce     redis client lettuce experiment

    delay                delay time

    throwCustomException throw custom exception

    update               update action spec

  log         log experiment for testing service delay

    delay                delay time

    throwCustomException throw custom exception

  mongodb     MongoDB experiment contains delay and exception by Collection name and so on

    delay                delay time

    throwCustomException throw custom exception

  mysql       Mysql experiment contains delay and exception by table name and so on

    delay                delay time

    throwCustomException throw custom exception

  nginx       Nginx experiment

    config      Config experiment

    crash       Crash experiment

    response    Response experiment

    restart     Restart experiment

  psql        Postgrelsql experiment contains delay and exception by table name and so on

    delay                delay time

    throwCustomException throw custom exception

  rabbitmq    rabbitmq experiment

    delay                delay time

    throwCustomException throw custom exception

  redis       Redis experiment

    cache-expire Cache Expire

    cache-limit  Cache Memory Limit

  redisson    redisson experiment

    delay                delay time

    throwCustomException throw custom exception

  rocketmq    Rocketmq experiment,can make message send or pull delay and exception,default if you not set [producerGroup,consumerGroup],will effect both send and pull message,if you only set producerGroup for specific group,will only effect on sendMessage,if you only set consumerGroup,will only effect pullMessage for specific group

    delay                delay time

    throwCustomException throw custom exception

  script      Script chaos experiment

    delay       Script executed delay

    exit        Exit script

  security    SpringSecurity login experiment contains delay and exception by command and so on

    delay                delay time

    throwCustomException throw custom exception

  servlet     Java servlet experiment, support path, query string, request method matcher

    delay                delay time

    modifyCode           update action spec

    throwCustomException throw custom exception

  strace      strace experiment contains syscall delay or syscall error

    delay       Delay the syscall of the target pid

    error       change the syscall's return value of the target pid

  systemd     Systemd experiment

    stop        Stop systemd

  tars        tars experiment

    delay                delay time

    throwCustomException throw custom exception

  time        Time experiment

    travel      Time Travel

  zk          zk experiment

    delay                delay time

    throwCustomException throw custom exception