搭建DragonflyClient Debug环境

1、安装colima，以containerd启动

》brew install colima

  如果单实例启动colima，

   # colima start --runtime containerd --cpu 4 --memory 8 --disk=100

  如果多实例启动colima，支持同时启动docker和containerd实例。

Colima 支持多实例，可以分别创建不同 runtime 的实例：

# 创建 Docker 实例
colima start docker-instance

# 创建 containerd 实例
colima start --runtime containerd containerd-instance

2、安装并启动minikube（containerd）

》minikube start --driver=qemu --container-runtime=containerd

3. 配置containerd

》minikube ssh

3.1 如果通过kubelet调用crt接口，需要配置config_path，如下：（如果直接通过ctr命令，不需要配置，带--hosts-dir "/etc/containerd/certs.d"即可）

》sudo vi /etc/containerd/config.toml

加入

[plugins]
[plugins."io.containerd.grpc.v1.cri"]
[plugins."io.containerd.grpc.v1.cri".registry]
config_path = "/etc/containerd/certs.d"

3.2 对docker.io镜像仓库加跳转到dragonfly请求的转发配置

》sudo vi /etc/containerd/certs.d/docker.io/hosts.toml，配置如下：（host.minikube.internal是minikube访问物理机服务的域名）

server = "https://registry-1.docker.io"

[host."http://host.minikube.internal:4001"]
capabilities = ["pull", "resolve"]
[host."http://host.minikube.internal:4001".header]
X-Dragonfly-Registry = "https://registry-1.docker.io"

4. 验证 （可待dragonfly安装后执行）

验证是否跳转dragonfly：

》sudo ctr images pull --skip-verify --hosts-dir /etc/containerd/certs.d  docker.io/library/nginx:latest

验证dragonfly服务：

1）获取token：

》TOKEN=$(curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:library/nginx:pull" | jq -r .token)

2）拉取manifest

》curl -H "Authorization: Bearer $TOKEN" \
-H "Accept: application/vnd.docker.distribution.manifest.v2+json" \
-H "Accept: application/vnd.docker.distribution.manifest.list.v2+json" \
-H "Accept: application/vnd.oci.image.manifest.v1+json" \
-H "Accept: application/vnd.oci.image.index.v1+json" \
"https://registry-1.docker.io/v2/library/nginx/manifests/latest" \
-o nginx-manifest.json

或

》curl -H "X-Dragonfly-Registry: https://registry-1.docker.io" -v http://host.minikube.internal:4001/v2/library/nginx/manifests/latest

3）获取各层digest

》LAYERS=$(cat nginx-manifest.json | jq -r '.layers[].digest')

4）下载各层

》for LAYER in $LAYERS; do
echo "Downloading layer: $LAYER"
curl -H "Authorization: Bearer $TOKEN" \
-L "https://registry-1.docker.io/v2/library/nginx/blobs/$LAYER" \
-o "${LAYER//:/_}.tar.gz"
done

---

5. 安装各种crd、validating&mutating webhookconfiguration

注意webhookconfiguration,鉴于minikube通过host.minikube.internal访问物理机服务，需要特别注意配置如下：

启动控制器服务的时候的配置得与 webhookconfiguration里面的url一致

 和

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  annotations:
    meta.helm.sh/release-name: kservice
    meta.helm.sh/release-namespace: jdcapp
  labels:
    app.kubernetes.io/instance: kservice
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kservice
    app.kubernetes.io/version: 0.2.2
  name: kservice-validating-webhook-configuration
webhooks:
- admissionReviewVersions:
  - v1
  clientConfig:
    caBundle: ''
    url: https://host.minikube.internal:8443/kservice-validation
  failurePolicy: Fail
  name: kservice.validating.webhook.configuration
  namespaceSelector: {}
  rules:
  - apiGroups:
    - fed.jd.com
    apiVersions:
    - v1
    operations:
    - CREATE
    - UPDATE
    - DELETE
    resources:
    - kservices
  sideEffects: None
  timeoutSeconds: 15

对应

5. helm安装dragonfly

》helm repo add dragonfly https://dragonflyoss.github.io/helm-charts/

》helm install --create-namespace --namespace dragonfly-system dragonfly dragonfly/dragonfly

》kubectl edit sts -ndragonfly-system dragonfly-scheduler =》 replicas=0

》kubectl edit deploy -ndragonfly-system dragonfly-manager =》replicas=0

如果资源优先，也可以把sts里面mysql和redis设置为单副本

6. 暴露redis和mysql服务

# mysql服务forward
》kubectl port-forward svc/dragonfly-mysql -ndragonfly-system 53306:3306
# redis服务forward
》kubectl port-forward svc/dragonfly-redis-master -ndragonfly-system 56379:6379

7. 配置configmap

7.1 Manager的configmap yaml文件manager.yaml

server:
  rest:
    addr: :8080
  grpc:
    advertiseIP:
    port:
      start: 65003
      end: 65003
  #requestRateLimit: 4000
  workHome:
  cacheDir:
  pluginDir:

  # 日志目录
  logDir: /Users/xxx/cyy/dragonfly/log/manager
  # 日志级别: debug, info, warn, error, panic, fatal
  logLevel: debug
  # 单个日志文件最大大小 (MB)
  logMaxSize: 200
  # 日志文件最大保留天数
  logMaxAge: 3
  # 最大备份文件数量
  logMaxBackups: 5
database:
  mysql:
    user: dragonfly
    password: dragonfly
    #host: dragonfly-mysql.dragonfly-system.svc.cluster.local
    host: 127.0.0.1
    port: 53306
    dbname: manager
    migrate: true
  redis:
    addrs:
    #  - dragonfly-redis-master.dragonfly-system.svc.cluster.local:6379
      - 127.0.0.1:56379
    masterName:
    username:
    password: dragonfly
cache:
  local:
    size: 200000
    ttl: 3m
  redis:
    ttl: 5m
job:
  preheat:
    tls:
      insecureSkipVerify: true
  syncPeers:
    interval: 24h
    timeout: 10m
network:
  enableIPv6: false
metrics:
  enable: true
  addr: ":8000"
  enablePeerGauge: true
tracing:
  protocol: grpc
console: true
verbose: true
pprofPort: 8004

7.2 Scheduler的configmap yaml文件scheduler.yaml

server:
  advertiseIP: ""
  advertisePort: 8002
  cacheDir: ""
  dataDir: ""
  listenIP: 0.0.0.0
  logDir: ""
  logLevel: debug
  pluginDir: ""
  port: 8002
  #requestRateLimit: 4000
  workHome: ""
scheduler:
  algorithm: default
  backToSourceCount: 20
  gc:
    hostGCInterval: 6h
    hostTTL: 1h
    peerGCInterval: 10s
    peerTTL: 24h
    pieceDownloadTimeout: 30m
    taskGCInterval: 30m
  retryBackToSourceLimit: 30
  retryInterval: 500ms
  retryLimit: 50
database:
  redis:
    addrs:
    #  - tpredis-dragonfly-mnq-svc.tpredis-dragonfly-mnq.glb.jd.local:6379
      - 127.0.0.1:56379
    masterName:
    username:
    password: dragonfly
dynconfig:
  refreshInterval: 1m
  type: manager
host:
  idc: "qa"
  location: "qa"
manager:
  #addr: dragonfly-manager.jddb.com:2000
  addr: 127.0.0.1:65003
  schedulerClusterID: 1
  keepAlive:
    interval: 5s
seedPeer:
  enable: true
job:
  enable: true
  globalWorkerNum: 500
  schedulerWorkerNum: 500
  localWorkerNum: 1000

storage:
  bufferSize: 100
  maxBackups: 10
  maxSize: 100
network:
  enableIPv6: false
metrics:
  enable: false
  addr: ":8000"
  enableHost: false
console: true
verbose: false

7.3 peer/seedpeer 的configmap yaml文件seedpeer.yaml

verbose: true
log:
  level: debug

host:
  idc: 'qa'
  location: 'qa'

server:
  pluginDir: /Users/xxx/cyy/dragonfly/seedpeer/
  cacheDir: /Users/xxx/cyy/dragonfly/seedpeer/

download:
  server:
    socketPath: /Users/xxx/cyy/dragonfly/seedpeer/dfdaemon.sock
  rateLimit: 2048Mi
  pieceTimeout: 300s
  concurrentPieceCount: 5

upload:
  server:
    port: 4000
  disableShared: false
  rateLimit: 2048Mi

manager:
  addr: http://127.0.0.1:65003

scheduler:
  announceInterval: 30s
  scheduleTimeout: 3h
  maxScheduleCount: 10
  enableBackToSource: true

seedPeer:
  enable: true
  type: super
  clusterID: 1
  keepaliveInterval: 15s
dynconfig:
  refreshInterval: 10m

storage:
  dir: /Users/xxx/cyy/dragonfly/seedpeer/
  keep: true
  writeBufferSize: 131072
  readBufferSize: 131072

gc:
  interval: 300s
  policy:
    taskTTL: 172800s
    distHighThresholdPercent: 60
    distLowThresholdPercent: 40
proxy:
  server:
    port: 4001
  rules:
    - regex: https://registry-1\.docker\.io/.*/blobs/sha256:.*
      useTLS: false
    - regex: https://docker\.io/.*/blobs/sha256:.*
      useTLS: false   
  registryMirror:
    addr: https://index.docker.io
  disableBackToSource: false
  prefetch: false
  readBufferSize: 32768
metrics:
  server:
    port: 4002

8. 起debug服务

8.1 debug manager

d7y.io/dragonfly/v2/cmd/manager
--config=manager.yaml

8.2 debug scheduler

d7y.io/dragonfly/v2/cmd/scheduler
--config=scheduler.yaml

8.2 debug seedpeer/peer （IDEA+RUST plugin + cargo包管理）

 1）安装IDEA，并在plugins里面选择Rust和Native Debugging Support插件安装，并且开启。

2）用IDEA打开dragonfly-client工程,

配置dragonfly-client工程/rust-toolchain.toml如下：

[toolchain]
channel = "1.95.0"

配置dragonfly-client工程/Cargo.toml，添加或修改如下配置：

[profile.dev]
opt-level = 0
debug = true
incremental = true
strip = false

下载依赖包在IDEA terminal操作

》cargo clean

3）在IDEA terminal安装python

4）在IDEA terminal安装rust-toolchain 1.95.0版本

》rustup toolchain install 1.95.0

》rustup default 1.95.0

》rustup --version

   rustc --version

5）配置IDEA setting

 

 6）配置Configuration

Command:
run  --bin dfdaemon -- --config /Users/xxx/cyy/dragonfly/seedpeer.yaml --log-level debug