使用GPT工具对极狐GitLab做性能与可用性测试
使用GPT工具对极狐GitLab做性能与可用性测试
1. 测试工具简介
GitLab Performance Tool（GPT）是由 GitLab 质量保证团队构建和维护的工具，可以对任何 GitLab 实例（包括极狐 GitLab）进行性能和可用性测试。默认测试以下内容：
出于安全考虑，默认不测试 HTTP POST、PUT、DELETE 和 PATCH，如果确实需要测试，可以在测试命令后面加 --unsafe 标识；
2. 准备测试环境
2.1 准备电脑和网络
- 安装有 Docker 的电脑或工作站：
- 操作系统为 Linux 或 MacOS
- 通常只需要 4 到 5G 内存
- 能网络访问到被测试的极狐 GitLab；
- 尽可能的靠近目标 GitLab，以免受到网络环境的干扰；
- 能网络访问到 https://gitlab.com
- 测试过程需要从 https://gitlab.com 下载测试工具和测试数据
2.2 初始化环境
# 克隆 GPT 项目到本地电脑；
git clone https://gitlab.com/gitlab-org/quality/performance.git
cd performance
#创建 results 目录
mkdir results
2.3 准备测试环境
- 以极狐 GitLab 管理员的身份创建个人访问 Token，scope 包含
API - 修改 ./k6/config/environments/1k.json 中的下面两行：
- “url”: “your gitlab url”,
- “user”: “admin username that the access token belong to”,
- 生成测试所需的数据
docker run --rm -it -e ACCESS_TOKEN=<token> -v $(pwd)/k6/config:/config -v $(pwd)/results:/results gitlab/gpt-data-generator -f --max-wait-for-delete=360000 --environment 1k.json
# 查看帮助 docker run -it --rm gitlab/gpt-data-generator --help
如果您的目标 GitLab 额定用户数量大于 1k，请调整上述参数中 1k ，替换为下列值中的一个：1k, 2k, 3k, 5k, 10k, 25k, 50k
注意：生成数据需要一定时间，请耐心等候
3. 运行测试
docker run --rm -it -e ACCESS_TOKEN=<TOKEN> -v $(pwd)/k6/config:/config -v $(pwd)/k6/tests:/tests -v $(pwd)/results:/results gitlab/gitlab-performance-tool --environment 1k.json --options 60s_20rps.json
# 查看帮助 docker run -it --rm gitlab/gitlab-performance-tool --help
如果您的目标 GitLab 额定用户数量大于 1k，请将上述参数中的 1k 和 60s_20rps.json 替换为下列参数中的一对：
1k - 60s_20rps.json
2k - 60s_40rps.json
3k - 60s_60rps.json
5k - 60s_100rps.json
10k - 60s_200rps.json
25k - 60s_500rps.json
50k - 60s_1000rps.json
4. 查看测试结果
完成所有测试后，您将看到结果摘要。例如：
| Vertical data: importing large projects for GPT...
Group gpt already exists
Creating group gpt/large_projects
Checking if project gitlabhq1 already exists in gpt/large_projects/gitlabhq1...
Disabling Max Import Size limit on environment...
Updating application settings: {:max_import_size=>10240}
Tarball is remote, downloading...
Starting import of Project 'gitlabhq1' from tarball 'https://gitlab.com/gitlab-org/quality/performance-data/-/raw/main/projects_export/gitlabhq_export_14.0.0.tar.gz' under namespace 'gpt/large_projects' to GitLab environment 'https://gitlab.leffss.cn'
Checking that GitLab environment 'https://gitlab.leffss.cn' is available, supported and that provided Access Token works...
Environment and Access Token check complete - URL: https://gitlab.leffss.cn, Version: 14.7.3-ee dfd1d9f3e51
Importing project gitlabhq1...
Note that this may take some time to upload a file to the target environment.
If project import takes more than an hour please refer to the troubleshooting docs https://gitlab.com/gitlab-org/quality/performance/-/blob/main/docs/environment_prep.md#import-looks-to-have-hanged.
Start time: 09:49:34 2022-03-04 UTC
Uploading project tarball to the target environment Import API...
Project tarball has successfully uploaded and started to be imported with ID '31786'
Waiting until Project '31786' has imported successfully....
GPT data generation failed:
Project has failed to import.
GitLab import error:
Error importing repository into gpt/large_projects/gitlabhq1 - No such file or directory @ rb_file_s_lstat - [FILTERED]
Correlation ID: 01FXA5BJBJ82QZA9CR6V1V99JQ
This is very likely to be an issue with the target environment. To troubleshoot please refer to https://gitlab.com/gitlab-org/quality/performance/-/blob/main/docs/environment_prep.md#import-has-failed
Traceback:["/performance/lib/import_project.rb:84:in `block in wait_for_import'", "/performance/lib/import_project.rb:75:in `loop'", "/performance/lib/import_project.rb:75:in `wait_for_import'", "/performance/lib/import_project.rb:111:in `import_project'", "/performance/lib/gpt_test_data.rb:522:in `block in create_vertical_test_data'", "/performance/lib/gpt_test_data.rb:492:in `each'", "/performance/lib/gpt_test_data.rb:492:in `with_index'", "/performance/lib/gpt_test_data.rb:492:in `create_vertical_test_data'", "./bin/generate-gpt-data:153:in `<main>'"]
Restoring the original 'max_import_size' setting...
Updating application settings: {:max_import_size=>0}
█ Logs: /results/generate-gpt-data_gitlab.leffss.cn_2022-03-04_081346.log
█ Results summary
* Environment: 3k
* Environment Version: 14.7.3-ee `dfd1d9f3e51`
* Option: 60s_60rps
* Date: 2022-03-04
* Run Time: 1h 27m 57.87s (Start: 14:48:44 UTC, End: 16:16:42 UTC)
* GPT Version: v2.9.0
❯ Overall Results Score: 90.76%
NAME | RPS | RPS RESULT | TTFB AVG | TTFB P90 | REQ STATUS | RESULT
---------------------------------------------------------|------|--------------------|-----------|----------------------|----------------|-----------------
api_v4_groups | 60/s | 54.63/s (>48.00/s) | 979.41ms | 1228.10ms (<200ms) | 100.00% (>99%) | FAILED²
api_v4_groups_group | 60/s | 17.03/s (>4.80/s) | 3246.66ms | 4141.72ms (<7500ms) | 100.00% (>99%) | Passed
api_v4_groups_group_subgroups | 60/s | 52.36/s (>48.00/s) | 1025.22ms | 1264.96ms (<200ms) | 100.00% (>99%) | FAILED²
api_v4_groups_issues | 60/s | 11.27/s (>14.40/s) | 4917.08ms | 5926.70ms (<3500ms) | 100.00% (>99%) | FAILED²
api_v4_groups_merge_requests | 60/s | 12.54/s (>14.40/s) | 4437.41ms | 5380.08ms (<3500ms) | 100.00% (>99%) | FAILED²
api_v4_groups_projects | 60/s | 17.38/s (>24.00/s) | 3184.20ms | 4073.62ms (<3500ms) | 100.00% (>99%) | FAILED²
api_v4_projects | 60/s | 6.94/s (>7.20/s) | 7735.07ms | 9847.03ms (<7000ms) | 100.00% (>99%) | FAILED²
api_v4_projects_deploy_keys | 60/s | 59.82/s (>48.00/s) | 34.08ms | 39.71ms (<200ms) | 100.00% (>99%) | Passed
api_v4_projects_issues | 60/s | 35.14/s (>48.00/s) | 1567.84ms | 1936.28ms (<500ms) | 100.00% (>99%) | FAILED²
api_v4_projects_issues_issue | 60/s | 26.53/s (>48.00/s) | 2069.81ms | 2994.28ms (<1500ms) | 100.00% (>99%) | FAILED²
api_v4_projects_issues_search | 60/s | 24.15/s (>7.20/s) | 2308.81ms | 2950.61ms (<12000ms) | 100.00% (>99%) | Passed
api_v4_projects_languages | 60/s | 59.83/s (>48.00/s) | 33.11ms | 37.62ms (<200ms) | 100.00% (>99%) | Passed
api_v4_projects_merge_requests | 60/s | 30.44/s (>48.00/s) | 1822.94ms | 2308.80ms (<400ms) | 100.00% (>99%) | FAILED²
api_v4_projects_merge_requests_merge_request | 60/s | 24.26/s (>24.00/s) | 2293.25ms | 3412.42ms (<2750ms) | 100.00% (>99%) | FAILED²
api_v4_projects_merge_requests_merge_request_changes | 60/s | 13.49/s (>24.00/s) | 4026.31ms | 5293.36ms (<3500ms) | 100.00% (>99%) | FAILED²
api_v4_projects_merge_requests_merge_request_commits | 60/s | 59.43/s (>48.00/s) | 382.75ms | 602.31ms (<200ms) | 100.00% (>99%) | FAILED²
api_v4_projects_merge_requests_merge_request_discussions | 60/s | 34.07/s (>48.00/s) | 1621.57ms | 2143.18ms (<400ms) | 100.00% (>99%) | FAILED²
api_v4_projects_project | 60/s | 54.45/s (>48.00/s) | 967.49ms | 1217.32ms (<200ms) | 100.00% (>99%) | FAILED²
api_v4_projects_project_pipelines | 60/s | 59.81/s (>48.00/s) | 47.63ms | 55.15ms (<200ms) | 100.00% (>99%) | Passed
api_v4_projects_project_pipelines_pipeline | 60/s | 59.5/s (>48.00/s) | 71.98ms | 94.99ms (<200ms) | 100.00% (>99%) | Passed
api_v4_projects_project_services | 60/s | 59.69/s (>48.00/s) | 31.49ms | 35.10ms (<200ms) | 100.00% (>99%) | Passed
api_v4_projects_releases | 60/s | 59.55/s (>48.00/s) | 93.71ms | 143.33ms (<200ms) | 100.00% (>99%) | Passed
api_v4_projects_repository_branches | 60/s | 59.57/s (>48.00/s) | 34.10ms | 38.60ms (<200ms) | 100.00% (>99%) | Passed
api_v4_projects_repository_branches_branch | 60/s | 59.66/s (>48.00/s) | 71.99ms | 85.61ms (<200ms) | 100.00% (>99%) | Passed
api_v4_projects_repository_branches_search | 60/s | 59.55/s (>14.40/s) | 33.92ms | 37.60ms (<6000ms) | 100.00% (>99%) | Passed
api_v4_projects_repository_commits | 60/s | 59.76/s (>48.00/s) | 63.64ms | 75.94ms (<200ms) | 100.00% (>99%) | Passed
api_v4_projects_repository_commits_commit | 60/s | 59.79/s (>48.00/s) | 50.79ms | 60.65ms (<200ms) | 100.00% (>99%) | Passed
api_v4_projects_repository_commits_commit_diff | 60/s | 59.49/s (>48.00/s) | 121.58ms | 149.81ms (<200ms) | 100.00% (>99%) | Passed
api_v4_projects_repository_compare | 60/s | 59.35/s (>48.00/s) | 62.50ms | 75.70ms (<200ms) | 100.00% (>99%) | Passed
api_v4_projects_repository_files_file | 60/s | 59.78/s (>48.00/s) | 48.31ms | 56.16ms (<200ms) | 100.00% (>99%) | Passed
api_v4_projects_repository_files_file_blame | 60/s | 6.93/s (>0.48/s) | 7557.77ms | 9728.95ms (<35000ms) | 100.00% (>15%) | Passed
api_v4_projects_repository_files_file_raw | 60/s | 59.78/s (>48.00/s) | 53.64ms | 63.23ms (<200ms) | 100.00% (>99%) | Passed
api_v4_projects_repository_tags | 60/s | 30.09/s (>9.60/s) | 1822.55ms | 2346.19ms (<10000ms) | 100.00% (>99%) | Passed
api_v4_projects_repository_tree | 60/s | 59.62/s (>48.00/s) | 77.74ms | 92.38ms (<200ms) | 100.00% (>99%) | Passed
api_v4_user | 60/s | 59.73/s (>48.00/s) | 38.84ms | 41.11ms (<200ms) | 100.00% (>99%) | Passed
api_v4_users | 60/s | 59.64/s (>48.00/s) | 107.68ms | 143.85ms (<200ms) | 100.00% (>99%) | Passed
git_clone | 3/s | 0.6/s (>0.08/s) | 272.55ms | 569.47ms (<800ms) | 100.00% (>99%) | Passed
git_ls_remote | 6/s | 6.01/s (>4.80/s) | 45.74ms | 49.62ms (<200ms) | 100.00% (>99%) | Passed
git_pull | 6/s | 5.98/s (>4.80/s) | 94.30ms | 142.93ms (<400ms) | 100.00% (>99%) | Passed
web_group | 6/s | 5.92/s (>4.80/s) | 198.32ms | 231.85ms (<400ms) | 100.00% (>99%) | Passed
web_group_issues | 6/s | 5.81/s (>4.80/s) | 361.17ms | 445.35ms (<500ms) | 100.00% (>99%) | Passed
web_group_merge_requests | 6/s | 5.87/s (>4.80/s) | 319.44ms | 366.55ms (<500ms) | 100.00% (>99%) | Passed
web_project | 6/s | 5.88/s (>4.80/s) | 265.22ms | 311.79ms (<500ms) | 100.00% (>99%) | Passed
web_project_branches | 6/s | 5.77/s (>4.80/s) | 387.54ms | 443.38ms (<800ms) | 100.00% (>99%) | Passed
web_project_branches_search | 6/s | 5.21/s (>4.80/s) | 944.08ms | 1287.76ms (<1300ms) | 100.00% (>99%) | Passed
web_project_commit | 6/s | 4.47/s (>1.92/s) | 1289.10ms | 3590.99ms (<2900ms) | 100.00% (>99%) | FAILED²
web_project_commits | 6/s | 5.88/s (>4.80/s) | 442.50ms | 559.24ms (<750ms) | 100.00% (>99%) | Passed
web_project_file_blame | 6/s | 1.55/s (>0.05/s) | 3384.21ms | 4146.46ms (<7000ms) | 100.00% (>99%) | Passed
web_project_file_rendered | 6/s | 5.73/s (>3.84/s) | 591.58ms | 1229.19ms (<1700ms) | 100.00% (>99%) | Passed
web_project_file_source | 6/s | 5.56/s (>0.48/s) | 920.62ms | 1685.88ms (<1700ms) | 100.00% (>99%) | FAILED²
web_project_files | 6/s | 5.87/s (>4.80/s) | 140.26ms | 229.72ms (<800ms) | 100.00% (>99%) | Passed
web_project_issue | 6/s | 5.93/s (>4.80/s) | 332.95ms | 817.04ms (<2000ms) | 100.00% (>99%) | Passed
web_project_issues | 6/s | 5.78/s (>4.80/s) | 316.03ms | 377.60ms (<500ms) | 100.00% (>99%) | Passed
web_project_issues_search | 6/s | 5.92/s (>4.80/s) | 293.99ms | 338.29ms (<500ms) | 100.00% (>99%) | Passed
web_project_merge_request | 6/s | 5.91/s (>1.92/s) | 286.60ms | 483.22ms (<7500ms) | 100.00% (>99%) | Passed
web_project_merge_request_changes | 6/s | 5.73/s (>4.80/s) | 265.37ms | 303.72ms (<1500ms) | 100.00% (>99%) | Passed
web_project_merge_request_commits | 6/s | 4.63/s (>2.88/s) | 1091.44ms | 1677.95ms (<1750ms) | 100.00% (>99%) | Passed
web_project_merge_requests | 6/s | 5.8/s (>4.80/s) | 296.95ms | 356.22ms (<500ms) | 100.00% (>99%) | Passed
web_project_pipelines | 6/s | 5.92/s (>2.88/s) | 287.96ms | 427.97ms (<1000ms) | 100.00% (>99%) | Passed
web_project_pipelines_pipeline | 6/s | 5.86/s (>4.80/s) | 617.61ms | 1286.55ms (<2500ms) | 100.00% (>99%) | Passed
web_project_repository_compare | 6/s | 0.79/s (>0.24/s) | 6919.88ms | 8741.90ms (<7500ms) | 100.00% (>99%) | FAILED²
web_project_tags | 6/s | 5.4/s (>3.84/s) | 786.82ms | 1133.29ms (<1500ms) | 100.00% (>99%) | Passed
web_user | 6/s | 5.96/s (>2.88/s) | 223.77ms | 349.72ms (<4000ms) | 100.00% (>99%) | Passed
² Failure may not be clear from summary alone. Refer to the individual test's full output for further debugging.
█ Results files
/results/3k_v14-7-3-ee_2022-03-04_144844/3k_v14-7-3-ee_2022-03-04_144844_results_output.log
/results/3k_v14-7-3-ee_2022-03-04_144844/3k_v14-7-3-ee_2022-03-04_144844_results.json
/results/3k_v14-7-3-ee_2022-03-04_144844/3k_v14-7-3-ee_2022-03-04_144844_results.csv
/results/3k_v14-7-3-ee_2022-03-04_144844/3k_v14-7-3-ee_2022-03-04_144844_results.txt
One or more tests have failed...
- 其中 failed 的项是测试数据未低于预期值
- 关于结果输出的更多信息可以在k6的文档中找到。
- 除了终端中的标准输出外，results/文件夹中还收集了各种文件格式的结果。
- 在可用性测试的场景下，上述APi能够正常访问，即说明满足可用性测试要求。
5. 清理极狐GitLab上的测试数据
#方法 1:
docker run --rm -it -e ACCESS_TOKEN=<TOKEN> -v $(pwd)/k6/config:/config -v $(pwd)/results:/results gitlab/gpt-data-generator -f --max-wait-for-delete=360000 --environment 1k.json --clean-up
#方法 2:
#在极狐 GitLab UI 中删除顶层的 gpt 组（或者您在环境配置文件中设置的组名）.
