SAIO - Swift All In One Diablo版 安装指南 Alpha

Note

在阅读本文之外,你可以参考来自于zzcase精心制作的简明swift安装指南和使用已经打包好的配置文件

以及笨笨blog所写的Swift在Ubuntu系统上的安装与配置

按照本人的一贯风格,采用注解和部分翻译的方式来介绍Swift的SAIO安装

Alpha:Created in 2011.9.24

Updated:

1st 2011.9.27    修正了调试结果

2rd 2011.9.29   增加一些注释

 

Instructions for setting up a development VM  搭建开发虚拟机

This documents setting up a virtual machine for doing Swift development. The virtual machine will emulate running a four node Swift cluster.

Additional information about setting up a Swift development snapshot on other distributions is available on the wiki at http://wiki.openstack.org/SAIOInstructions.

Installing dependencies and the core code  安装依赖性包和核心代码


  • As root on guest (you’ll have to log in as you, then sudo su -):
    1. apt-get install python-software-properties
    2. add-apt-repository ppa:swift-core/ppa
    3. apt-get update
    4. apt-get install curl gcc bzr memcached python-configobj python-coverage python-dev python-nose python-setuptools python-simplejson python-xattr sqlite3 xfsprogs python-webob python-eventlet python-greenlet python-pastedeploy python-netifaces
    5. Install anything else you want, like screen, ssh, vim, etc.
    6. Next, choose either Using a partition for storage or Using a loopback device for storage.
    7. 本文选择的是使用环回设备用于存储,哥没第二块硬盘,也无空余分区...

Using a partition for storage    使用一个新分区作为存储(跳过此段)

If you are going to use a separate partition for Swift data, be sure to add another device when creating the VM, and follow these instructions.
  1. fdisk /dev/sdb (set up a single partition)

  2. mkfs.xfs -i size=1024 /dev/sdb1

  3. Edit /etc/fstab and add

    /dev/sdb1 /mnt/sdb1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0

  4. mkdir /mnt/sdb1

  5. mount /mnt/sdb1

  6. mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4

  7. chown <your-user-name>:<your-group-name> /mnt/sdb1/*

  8. mkdir /srv

  9. for x in {1..4}; do ln -s /mnt/sdb1/$x /srv/$x; done

  10. mkdir -p /etc/swift/object-server /etc/swift/container-server /etc/swift/account-server /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4 /var/run/swift

  11. chown -R <your-user-name>:<your-group-name> /etc/swift /srv/[1-4]/ /var/run/swiftMake sure to include the trailing slash after /srv/[1-4]/

  12. Add to /etc/rc.local (before the exit 0):

    mkdir /var/run/swift chown <your-user-name>:<your-group-name> /var/run/swift
  13. Next, skip to Setting up rsync.

Using a loopback device for storage  使用环回设备用于存储

关于环回设备

环回设备 (loopback device)

Linux内核用结构体struct net_device表示一个网络设备接口,该结构体的成员hard_start_xmit是个函数指针,用于完成数据报在网络上的发送工作,其原型是:
   
int (*hard_start_xmit)( struct sk_buff *skb, struct net_device *dev );
    skb
是待发送的数据缓冲区,dev是该网络设备接口本身的一个指针。
   
环回设备接口由于是把数据报发给本机,所以其发送数据报函数比较特别,他把skb稍加处理后,又转回给协议栈的数据报接收函数netif_rx。其发送函数的函数名是loopback_xmit
   

  首先,loopback_xmit调用skb_orphanskb孤立使他跟发送socket和协议栈不再有任何联系,也即对本机来说,这个skb的数据内容已发送出去了,而skb相当于已被释放掉了。skb_orphan所做的实际事情是,首先从skb->sk(发送这个skb的那个socket)sk_wmem_alloc减去skb->truesize,也即从socket的已提交发送队列的字节数中减去这个skb,表示这个skb已发送出去了,同时,假如有进程在这个socket上写等待,则唤醒这些进程继续发送数据报,然后把socket的引用计数减1,最后,令sk->destructorskb->sk都为NULL,使skb完全孤立。实际上,对于环回设备接口来说,数据的发送工作至此已全部完成,接下来,只要把这个实际上还未被释放的skb传回给协议栈的接收函数即可。
   
在传回去之前,还需要做一些工作,因为协议栈在发送数据报时,为数据报打以太网首部时,skb->data是指向以太网首部的开始位置的,网络设备接口传回的数据报是需要已剥离了以太网首部的,所以令skb->data加上ETH_HLEN(14),令skb->mac.raw指向原来data的位置,即以太网首部。然后,需要确定skb->pkt_type的值,即该数据报的类型,假如以太网首部中的目的mac地址是个组播或广播地址,则skb->pkt_typePACKET_BROADCASTPACKET_MULTICAST,关于MAC地址类型的判断方法以后再周详分析。否则,假如目的mac地址不等于本接口设备的mac地址,则skb->pkt_typePACKET_OTHERHOST,否则就为默认值PACKET_HOST。最后,配置skb->protocol(ETH_P_IPETH_P_ARP或其他),配置skb->dev,再更新一下统计值,把skb交回给协议协栈即可。
    struct net_device更有一些跟协议栈的处理流程关系比较密切的成员函数,下面一一介绍。
    hard_header,该成员被以太网设备驱动程式用于为每一个待发送数据报构建以太网首部,系统中任何以太网设备驱动程式共享一个函数即eth_header。任何数据报在传递给该函数之前,其skb头部预留了以太网首部的空间,data成员指向网络层首部,eth_headerdata成员指向以太网首部,并为以太网首部填入目的以太网地址,源以太网地址和网络层协议类型(ETH_P_IPETH_P_ARP),该函数在协议栈中主要有两处被用到,一是ARP模块中,发送ARP请求或应答前,构建以太网首部用;另一处是在IP数据发送过程中,构建以太网首部用。
    hard_header_cache,用于创建以太网首部的高速缓冲,每一个邻居节点都有一个struct hh_cache *hh成员,用于缓冲该邻居节点的以太网首部,有了这个缓冲,以后再向这个邻居发数IP数据的时候,不必再调用hard_header构建以太网首部,而是直接从hh中拷贝即可。

If you want to use a loopback device instead of another partition, follow these instructions.

  1. mkdir /srv  

  2. dd if=/dev/zero of=/srv/swift-disk bs=1024 count=0 seek=1000000

    (modify seek to make a larger or smaller partition

    dd是转换并复制文件命令,参数if指输入文件,of指输出文件,bs是每次读写bs大小的bytes,count是指复制count个输入块。调整seek参数的值以分配分区大小,swift-disk的大小=1024*1000000)
  3. mkfs.xfs -i size=1024 /srv/swift-disk    #i是指inode_options,size是指文件系统的inode大小是1024bytes

  4. 编辑 /etc/fstab 增加   /srv/swift-disk /mnt/sdb1 xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0   
    /etc/fstab是用来存放文件系统的静态信息的文件。位于/etc/目录下,可以用命令less /etc/fstab 来查看。当系统启动的时候,系统会自动地从这个文件读取信息,并且会自动将此文件中指定的文件系统挂载到指定的目录,详细介绍
    参数loop:表示loop设备
    参数noatime:Do not update inode access times on this  filesystem  (e.g,  for faster access on the news spool to speed up news servers).
    参数 nodiratime: Do not update directory inode access times on this filesystem.
    参数nobarrier:This disables the use of write barriers in the jbd code.Write barriers enforce proper on-disk ordering of journal commits, making volatile disk write caches safe  to  use,  at some performance penalty.If  your  disks  are  battery-backed  in  one  way or another, disabling barriers may safely improve performance.
    参数logbufs:Set  the  number  of in-memory log buffers. 默认值是8,表示文件系统的块大小是64KiB。 **Kibibyte是一种资讯计量单位,代表1024字节,即210字节,一般简称为KiB
  5. mkdir /mnt/sdb1
  6. mount /mnt/sdb1
  7. mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4

  8. chown -R swift:swift /mnt/sdb1/*

  9. for x in {1..4}; do ln -s /mnt/sdb1/$x /srv/$x; done

  10. mkdir -p /etc/swift/object-server /etc/swift/container-server /etc/swift/account-server /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4 /var/run/swift

  11. chown -R swift:swift /etc/swift /srv/[1-4]/ /var/run/swift – Make sure to include the trailing slash after /srv/[1-4]/

  12. Add to /etc/rc.local (before the exit 0):  mkdir /var/run/swift chown swift:swift /var/run/swift   #/etc/rc.local 是一个开机可自动执行的任务脚本,也就是说,如果你想让电脑在开机时自动执行的任务、程序,只要写进其中即可/var/run 目录中存放的是自系统启动以来描述系统信息的文件

    
    

Setting up rsync  设置rsync

  1. Create /etc/rsyncd.conf:

         uid = swift
    gid = swift
    log file = /var/log/rsyncd.log
    pid file = /var/run/rsyncd.pid
    address = 127.0.0.1

    [account6012]
    max connections = 25
    path = /srv/1/node/
    read only = false
    lock file = /var/lock/account6012.lock

    [account6022]
    max connections = 25
    path = /srv/2/node/
    read only = false
    lock file = /var/lock/account6022.lock

    [account6032]
    max connections = 25
    path = /srv/3/node/
    read only = false
    lock file = /var/lock/account6032.lock

    [account6042]
    max connections = 25
    path = /srv/4/node/
    read only = false
    lock file = /var/lock/account6042.lock

    [container6011]
    max connections = 25
    path = /srv/1/node/
    read only = false
    lock file = /var/lock/container6011.lock

    [container6021]
    max connections = 25
    path = /srv/2/node/
    read only = false
    lock file = /var/lock/container6021.lock

    [container6031]
    max connections = 25
    path = /srv/3/node/
    read only = false
    lock file = /var/lock/container6031.lock

    [container6041]
    max connections = 25
    path = /srv/4/node/
    read only = false
    lock file = /var/lock/container6041.lock


    [object6010]
    max connections = 25
    path = /srv/1/node/
    read only = false
    lock file = /var/lock/object6010.lock

    [object6020]
    max connections = 25
    path = /srv/2/node/
    read only = false
    lock file = /var/lock/object6020.lock

    [object6030]
    max connections = 25
    path = /srv/3/node/
    read only = false
    lock file = /var/lock/object6030.lock

    [object6040]
    max connections = 25
    path = /srv/4/node/
    read only = false
    lock file = /var/lock/object6040.lock
  2. Edit the following line in /etc/default/rsync:

    RSYNC_ENABLE=true    #这步别忘了
  3. service rsync restart

Optional: Setting up rsyslog for individual logging  设置rsylog来生成独立日志

  1. Create /etc/rsyslog.d/10-swift.conf:

        # Uncomment the following to have a log containing all logs together
    #local1,local2,local3,local4,local5.* /var/log/swift/all.log

    # Uncomment the following to have hourly proxy logs for stats processing
    #$template HourlyProxyLog,"/var/log/swift/hourly/%$YEAR%%$MONTH%%$DAY%%$HOUR%"
    #local1.*;local1.!notice ?HourlyProxyLog

    local1.*;local1.!notice /var/log/swift/proxy.log
    local1.notice /var/log/swift/proxy.error
    local1.* ~

    local2.*;local2.!notice /var/log/swift/storage1.log
    local2.notice /var/log/swift/storage1.error
    local2.* ~

    local3.*;local3.!notice /var/log/swift/storage2.log
    local3.notice /var/log/swift/storage2.error
    local3.* ~

    local4.*;local4.!notice /var/log/swift/storage3.log
    local4.notice /var/log/swift/storage3.error
    local4.* ~

    local5.*;local5.!notice /var/log/swift/storage4.log
    local5.notice /var/log/swift/storage4.error
    local5.* ~
  2. Edit /etc/rsyslog.conf and make the following change:

    $PrivDropToGroup adm
  3. mkdir -p /var/log/swift/hourly

  4. chown -R syslog.adm /var/log/swift

  5. service rsyslog restart

Getting the code and setting up test environment  获得代码和设置测试环境

Sample configuration files are provided with all defaults in line-by-line comments.

Do these commands as you on guest. The bazaar configuration is optional; you can always do a bzr branch command regardless of whether you have a Launchpad account(这里我没有执行成功,研究了好久的bzr和LaunchPad啊...哪位同学可以指点一下...更新,已经搞定,具体见下):

  1. mkdir ~/bin

  2. (optional) mkdir ~/.bazaar

  3. (optional) Create ~/.bazaar/bazaar.conf:

    [DEFAULT] email = Your Name <your-email-address>
  4. (optional) If you are using launchpad to get the code or make changes, run bzr launchpad-login <launchpad_id>

  5. Create the swift repo with bzr init-repo swift

  6. Check out a bzr branch of swift, for example: cd ~/swift; bzr branch lp:swift trunk (我卡在这里了,后来我去注册来Launchpad帐号,依然没有成功,错误原因 bzr: ERROR: Not a branch: "bzr+ssh://bazaar.launchpad.net/%2Bbranch/swift/"。去官网下了最新的Diablo版,自行安装。更新,已经解决。办法有两种:1.可以使用bzr branch http://bazaar.launchpad.net/~hudson-openstack/swift/trunk/                                                                                                                         2.由于openstack的代码整体已迁到github上了,可以使用git clone https://github.com/openstack/swift.git

  7. Build a development installation of swift, for example: cd ~/swift/trunk; sudo python setup.py develop

  8. Edit ~/.bashrc and add to the end:

    export SWIFT_TEST_CONFIG_FILE=/etc/swift/func_test.conf 
    export PATH=${PATH}:~/bin
  9. . ~/.bashrc

Configuring each node  配置各个节点

Sample configuration files are provided with all defaults in line-by-line comments.

  1. Create /etc/swift/proxy-server.conf:   #这个文件请确认无误,当前使用的认证方式是tempauth

        [DEFAULT]
    bind_port = 8080
    user = swift
    log_facility = LOG_LOCAL1

    [pipeline:main]
    pipeline = healthcheck cache tempauth proxy-server

    [app:proxy-server]
    use = egg:swift#proxy
    allow_account_management = true

    [filter:tempauth]
    use = egg:swift#tempauth
    user_admin_admin = admin .admin .reseller_admin
    user_test_tester = testing .admin
    user_test2_tester2 = testing2 .admin
    user_test_tester3 = testing3

    [filter:healthcheck]
    use = egg:swift#healthcheck

    [filter:cache]
    use = egg:swift#memcache
  2. Create /etc/swift/swift.conf:  

         [swift-hash] # random unique string that can never change (DO NOT LOSE) 
    swift_hash_path_suffix = changeme #这个字段你可以随意设置
  3. Create /etc/swift/account-server/1.conf:

         [DEFAULT]
    devices = /srv/1/node
    mount_check = false
    bind_port = 6012
    user = <your-user-name>
    log_facility = LOG_LOCAL2

    [pipeline:main]
    pipeline = account-server

    [app:account-server]
    use = egg:swift#account

    [account-replicator]
    vm_test_mode = yes

    [account-auditor]

    [account-reaper]
  4. Create /etc/swift/account-server/2.conf:

         [DEFAULT]
    devices = /srv/2/node
    mount_check = false
    bind_port = 6022
    user = <your-user-name>
    log_facility = LOG_LOCAL3

    [pipeline:main]
    pipeline = account-server

    [app:account-server]
    use = egg:swift#account

    [account-replicator]
    vm_test_mode = yes

    [account-auditor]

    [account-reaper]
  5. Create /etc/swift/account-server/3.conf:

         [DEFAULT]
    devices = /srv/3/node
    mount_check = false
    bind_port = 6032
    user = <your-user-name>
    log_facility = LOG_LOCAL4

    [pipeline:main]
    pipeline = account-server

    [app:account-server]
    use = egg:swift#account

    [account-replicator]
    vm_test_mode = yes

    [account-auditor]

    [account-reaper]
  6. Create /etc/swift/account-server/4.conf:

         [DEFAULT]
    devices = /srv/4/node
    mount_check = false
    bind_port = 6042
    user = <your-user-name>
    log_facility = LOG_LOCAL5

    [pipeline:main]
    pipeline = account-server

    [app:account-server]
    use = egg:swift#account

    [account-replicator]
    vm_test_mode = yes

    [account-auditor]

    [account-reaper]
  7. Create /etc/swift/container-server/1.conf:

         [DEFAULT]
    devices = /srv/1/node
    mount_check = false
    bind_port = 6011
    user = <your-user-name>
    log_facility = LOG_LOCAL2

    [pipeline:main]
    pipeline = container-server

    [app:container-server]
    use = egg:swift#container

    [container-replicator]
    vm_test_mode = yes

    [container-updater]

    [container-auditor]

    [container-sync]
  8. Create /etc/swift/container-server/2.conf:

         [DEFAULT]
    devices = /srv/2/node
    mount_check = false
    bind_port = 6021
    user = <your-user-name>
    log_facility = LOG_LOCAL3

    [pipeline:main]
    pipeline = container-server

    [app:container-server]
    use = egg:swift#container

    [container-replicator]
    vm_test_mode = yes

    [container-updater]

    [container-auditor]

    [container-sync]
  9. Create /etc/swift/container-server/3.conf:

         [DEFAULT]
    devices = /srv/3/node
    mount_check = false
    bind_port = 6031
    user = <your-user-name>
    log_facility = LOG_LOCAL4

    [pipeline:main]
    pipeline = container-server

    [app:container-server]
    use = egg:swift#container

    [container-replicator]
    vm_test_mode = yes

    [container-updater]

    [container-auditor]

    [container-sync]
  10. Create /etc/swift/container-server/4.conf:

         [DEFAULT]
    devices = /srv/4/node
    mount_check = false
    bind_port = 6041
    user = <your-user-name>
    log_facility = LOG_LOCAL5

    [pipeline:main]
    pipeline = container-server

    [app:container-server]
    use = egg:swift#container

    [container-replicator]
    vm_test_mode = yes

    [container-updater]

    [container-auditor]

    [container-sync]
  11. Create /etc/swift/object-server/1.conf:

         [DEFAULT]
    devices = /srv/1/node
    mount_check = false
    bind_port = 6010
    user = <your-user-name>
    log_facility = LOG_LOCAL2

    [pipeline:main]
    pipeline = object-server

    [app:object-server]
    use = egg:swift#object

    [object-replicator]
    vm_test_mode = yes

    [object-updater]

    [object-auditor]
  12. Create /etc/swift/object-server/2.conf:

         [DEFAULT]
    devices = /srv/2/node
    mount_check = false
    bind_port = 6020
    user = <your-user-name>
    log_facility = LOG_LOCAL3

    [pipeline:main]
    pipeline = object-server

    [app:object-server]
    use = egg:swift#object

    [object-replicator]
    vm_test_mode = yes

    [object-updater]

    [object-auditor]
  13. Create /etc/swift/object-server/3.conf:

         [DEFAULT]
    devices = /srv/3/node
    mount_check = false
    bind_port = 6030
    user = <your-user-name>
    log_facility = LOG_LOCAL4

    [pipeline:main]
    pipeline = object-server

    [app:object-server]
    use = egg:swift#object

    [object-replicator]
    vm_test_mode = yes

    [object-updater]

    [object-auditor]
  14. Create /etc/swift/object-server/4.conf:

         [DEFAULT]
    devices = /srv/4/node
    mount_check = false
    bind_port = 6040
    user = <your-user-name>
    log_facility = LOG_LOCAL5

    [pipeline:main]
    pipeline = object-server

    [app:object-server]
    use = egg:swift#object

    [object-replicator]
    vm_test_mode = yes

    [object-updater]

    [object-auditor]

Setting up scripts for running Swift 设置运行Swift的脚本

  1. Create ~/bin/resetswift.

    If you are using a loopback device substitute /dev/sdb1 with /srv/swift-disk.

    If you did not set up rsyslog for individual logging, remove the find /var/log/swift... line:

         #!/bin/bash
    swift-init all stop
    find /var/log/swift -type f -exec rm -f {} \;
    sudo umount /mnt/sdb1
    sudo mkfs.xfs -f -i size=1024 /dev/sdb1
    sudo mount /mnt/sdb1
    sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
    sudo chown <your-user-name>:<your-group-name> /mnt/sdb1/*
    mkdir -p /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4
    sudo rm -f /var/log/debug /var/log/messages /var/log/rsyncd.log /var/log/syslog
    sudo service rsyslog restart
    sudo service memcached restart
  2. Create ~/bin/remakerings:

         #!/bin/bash
    cd /etc/swift

    rm -f *.builder *.ring.gz backups/*.builder backups/*.ring.gz

    swift-ring-builder object.builder create 18 3 1
    swift-ring-builder object.builder add z1-127.0.0.1:6010/sdb1 1
    swift-ring-builder object.builder add z2-127.0.0.1:6020/sdb2 1
    swift-ring-builder object.builder add z3-127.0.0.1:6030/sdb3 1
    swift-ring-builder object.builder add z4-127.0.0.1:6040/sdb4 1
    swift-ring-builder object.builder rebalance
    swift-ring-builder container.builder create 18 3 1
    swift-ring-builder container.builder add z1-127.0.0.1:6011/sdb1 1
    swift-ring-builder container.builder add z2-127.0.0.1:6021/sdb2 1
    swift-ring-builder container.builder add z3-127.0.0.1:6031/sdb3 1
    swift-ring-builder container.builder add z4-127.0.0.1:6041/sdb4 1
    swift-ring-builder container.builder rebalance
    swift-ring-builder account.builder create 18 3 1
    swift-ring-builder account.builder add z1-127.0.0.1:6012/sdb1 1
    swift-ring-builder account.builder add z2-127.0.0.1:6022/sdb2 1
    swift-ring-builder account.builder add z3-127.0.0.1:6032/sdb3 1
    swift-ring-builder account.builder add z4-127.0.0.1:6042/sdb4 1
    swift-ring-builder account.builder rebalance
  3. Create ~/bin/startmain:

    #!/bin/bash 
    swift-init main start
  4. Create ~/bin/startrest:

    #!/bin/bash 
    swift-init rest start
  5. chmod +x ~/bin/*

  6. remakerings    运行结果如下:

    Device z1-127.0.0.1:6010/sdb1_"" with 1.0 weight got id 0
    Device z2-127.0.0.1:6020/sdb2_"" with 1.0 weight got id 1
    Device z3-127.0.0.1:6030/sdb3_"" with 1.0 weight got id 2
    Device z4-127.0.0.1:6040/sdb4_"" with 1.0 weight got id 3
    Reassigned 262144 (100.00%) partitions. Balance is now 0.00.
    Device z1-127.0.0.1:6011/sdb1_"" with 1.0 weight got id 0
    Device z2-127.0.0.1:6021/sdb2_"" with 1.0 weight got id 1
    Device z3-127.0.0.1:6031/sdb3_"" with 1.0 weight got id 2
    Device z4-127.0.0.1:6041/sdb4_"" with 1.0 weight got id 3
    Reassigned 262144 (100.00%) partitions. Balance is now 0.00.
    Device z1-127.0.0.1:6012/sdb1_"" with 1.0 weight got id 0
    Device z2-127.0.0.1:6022/sdb2_"" with 1.0 weight got id 1
    Device z3-127.0.0.1:6032/sdb3_"" with 1.0 weight got id 2
    Device z4-127.0.0.1:6042/sdb4_"" with 1.0 weight got id 3
    Reassigned 262144 (100.00%) partitions. Balance is now 0.00.
  7. cd ~/swift/trunk; ./.unittests  这步单元测试,出错,出错原因是找不到目录下测试用的两个db数据库,我暂时把test_db.py的相应代码注释掉了(现在已经解决,注意 你若使用的是zzcase提供的配置文件,需要注意到1.4使用的是tempauth,和1.1是不同的,请按本文修改你的配置文件)

    swift@ubuntu:~/swift-1.4.3$ ./.unittests 
    ..........................................SSSSSSSS.........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
    Name                                   Stmts   Exec  Cover   Missing
    --------------------------------------------------------------------
    swift                                     13     13   100%   
    swift.account                              1      1   100%   
    swift.account.auditor                     57     12    21%   30-37, 41-62, 66-84, 93-103
    swift.account.reaper                     204     23    11%   55-74, 78-82, 86-90, 94-98, 106-113, 122-132, 147-174, 213-266, 304-364, 387-415
    swift.account.replicator                   8      8   100%   
    swift.account.server                     215    171    79%   62-63, 66, 69, 82-83, 86, 114, 137-138, 141, 156-158, 172, 255-269, 275-276, 280, 283, 286, 301, 307-311, 315, 327, 335-337
    swift.common                               0      0   100%   
    swift.common.bench                       177     31    17%   32-33, 36, 42-78, 83-84, 91-104, 107-116, 119, 125-129, 132-139, 145-148, 151-157, 161-177, 183-186, 189-205, 211-218, 222-249
    swift.common.bufferedhttp                 65     52    80%   63-71, 92, 122, 156, 160, 162
    swift.common.client                      454    359    79%   28-29, 33-34, 39-43, 82, 124, 129, 134, 160, 193-196, 222-223, 228-230, 234, 236, 238, 254, 298-307, 337-349, 354, 356, 358, 360, 375, 404-407, 460, 525-538, 567-570, 614, 620, 622, 626, 628, 630, 634, 641-652, 791, 793, 797, 818, 840, 871
    swift.common.constraints                  73     73   100%   
    swift.common.daemon                       38     35    92%   46-48
    swift.common.db                          809    675    83%   135-136, 231, 242, 259, 273-295, 304-305, 315-319, 345-348, 383-389, 401-403, 454-456, 472, 513, 519-521, 547, 576-582, 594, 629, 641, 644, 698, 781-783, 799, 811-812, 819-821, 849-852, 884, 890, 892, 915-917, 924, 945-947, 969, 972-975, 994, 1016-1022, 1066-1068, 1093, 1189, 1323, 1338-1339, 1346-1348, 1358-1360, 1393-1396, 1410-1412, 1458, 1460, 1468-1482, 1496-1498, 1525-1527, 1552-1554, 1576, 1637
    swift.common.db_replicator               338    194    57%   61, 183, 191, 196-198, 223-227, 236, 269, 288-290, 296, 298-299, 301, 309-315, 339, 347-348, 352-353, 357-360, 367-377, 394-401, 404-408, 416-417, 419-430, 433, 443-448, 463-482, 485-532, 543-551, 557-568
    swift.common.direct_client               193     20    10%   34-36, 57-87, 104-123, 145-175, 180-189, 214-233, 253-282, 308-338, 356-365, 389-398, 421-454
    swift.common.exceptions                   25     24    96%   26
    swift.common.manager                     321    319    99%   335-336
    swift.common.memcached                   188    172    91%   81-85, 101, 170-171, 206-208, 245-246, 275-276, 311-312
    swift.common.middleware                    1      1   100%   
    swift.common.middleware.acl               47     47   100%   
    swift.common.middleware.catch_errors      30     22    73%   40-42, 54-59
    swift.common.middleware.domain_remap      56     51    91%   112-117
    swift.common.middleware.healthcheck       15     12    80%   42-44
    swift.common.middleware.memcache          14      9    64%   36-42
    swift.common.middleware.ratelimit        124    124   100%   
    swift.common.middleware.staticweb        220    214    97%   114-115, 150, 511, 516, 519
    swift.common.middleware.swift3           237    229    96%   225, 347, 405-408, 456-457, 470-471
    swift.common.middleware.tempauth         279    184    65%   82, 85, 90, 121-137, 147-154, 206, 210-212, 215-233, 244-245, 306, 308-310, 319-323, 337-338, 343, 376-377, 400, 409-448, 454, 457, 461, 466, 469, 474
    swift.common.ring                          2      2   100%   
    swift.common.ring.builder                258    226    87%   73-99, 102, 130, 147, 312, 341, 364
    swift.common.ring.ring                    68     64    94%   33, 58, 67, 81
    swift.common.utils                       572    478    83%   39-40, 76-77, 103, 139, 157, 266, 333, 632-635, 653-678, 704-708, 735-750, 761-767, 898-900, 936-939, 947, 952, 957, 1025, 1087-1106
    swift.common.wsgi                        124     51    41%   79-81, 83, 102-191
    swift.container                            1      1   100%   
    swift.container.auditor                   58     12    20%   30-38, 42-64, 68-87, 96-106
    swift.container.replicator                 8      8   100%   
    swift.container.server                   269    223    82%   106, 126-127, 138-139, 143, 146, 161, 164, 167, 174-175, 179, 185, 187, 204, 230-231, 234, 262, 361, 365-374, 380-381, 385, 391, 393, 396, 417, 423-427, 439, 447-449
    swift.container.sync                     196    195    99%   383
    swift.container.updater                  166    119    71%   78-79, 89-93, 99, 108, 110, 113-121, 124, 126-133, 136-138, 149, 156-157, 203, 237-240, 265-278
    swift.obj                                  1      1   100%   
    swift.obj.auditor                        126    124    98%   145-146
    swift.obj.replicator                     371    254    68%   66, 93, 190-192, 206-209, 267-270, 275-280, 282, 285, 306, 319, 329, 355, 362-379, 394-448, 478, 483-487, 496, 506-509, 524-525, 528, 538-539, 563-565, 570-576, 588-592, 595-613
    swift.obj.server                         462    428    92%   206-207, 255-256, 339, 431, 434, 467, 544, 619, 659, 686-703, 711, 718-721, 734, 740, 746-748
    swift.obj.updater                        140     95    67%   63-101, 112-114, 134, 139, 142-146, 167-172, 186, 188-191
    swift.proxy                                1      1   100%   
    swift.proxy.server                      1066    945    88%   163-164, 258-264, 392, 468, 470-471, 477-478, 617, 634-635, 639-640, 642-643, 749-757, 774-796, 806-819, 824-847, 850, 855-865, 899, 976-990, 1043, 1090, 1092-1105, 1108, 1114-1118, 1178-1180, 1190, 1212, 1264, 1353, 1391, 1411-1414, 1422, 1471-1474, 1480, 1607, 1641, 1683, 1696, 1719-1721
    --------------------------------------------------------------------
    TOTAL                                   8091   6302    77%   
    ----------------------------------------------------------------------
    Ran 683 tests in 22.676s

    OK (SKIP=8)

  8. startmain (The Unable to increase file descriptor limit.  Running as non-root? warnings are expected and ok.出现该警告为预期结果,正常)

    swift@ubuntu:~/bin$ ./startmain        #注意WARNING是正常信息
    WARNING: Unable to increase file descriptor limit.  Running as non-root?
    auth-server running (6473 - /etc/swift/auth-server.conf)
    auth-server already started...
    WARNING: Unable to increase file descriptor limit.  Running as non-root?
    proxy-server running (6475 - /etc/swift/proxy-server.conf)
    proxy-server already started...
    WARNING: Unable to increase file descriptor limit.  Running as non-root?
    account-server running (6480 - /etc/swift/account-server/3.conf)
    account-server running (6477 - /var/run/swift/account-server/0.pid)
    account-server running (6478 - /etc/swift/account-server/1.conf)
    account-server running (6479 - /etc/swift/account-server/2.conf)
    account-server already started...
    WARNING: Unable to increase file descriptor limit.  Running as non-root?
    container-server running (6488 - /etc/swift/container-server/2.conf)
    container-server running (6489 - /etc/swift/container-server/3.conf)
    container-server running (6486 - /var/run/swift/container-server/0.pid)
    container-server running (6487 - /etc/swift/container-server/1.conf)
    container-server already started...
    WARNING: Unable to increase file descriptor limit.  Running as non-root?
    object-server running (6498 - /var/run/swift/object-server/0.pid)
    object-server running (6499 - /etc/swift/object-server/1.conf)
    object-server running (6500 - /etc/swift/object-server/2.conf)
    object-server running (6501 - /etc/swift/object-server/3.conf)
    object-server already started...

  9. Get an X-Storage-Url and X-Auth-Token: curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' http://127.0.0.1:8080/auth/v1.0  

    运行结果如下:

    swift@ubuntu:~/bin$ curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' http://127.0.0.1:8080/auth/v1.0
    * About to connect() to 127.0.0.1 port 8080 (#0)
    *   Trying 127.0.0.1... connected
    * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
    > GET /auth/v1.0 HTTP/1.1
    > User-Agent: curl/7.21.3 (i686-pc-linux-gnu) libcurl/7.21.3 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.18
    > Host: 127.0.0.1:8080
    > Accept: */*
    > X-Storage-User: test:tester
    > X-Storage-Pass: testing

    < HTTP/1.1 204 No Content
    < X-Storage-Url: http://127.0.0.1:8080/v1/AUTH_98a7125e8bb84ded9cfe15d8dd19bc6b   #下一步要用到的数据
    < X-Storage-Token: AUTH_tkf4f5252a030c4640b8fb04942849148c
    < X-Auth-Token: AUTH_tkf4f5252a030c4640b8fb04942849148c    #这个是我们下一步需要用的
    < Content-Length: 0
    < Date: Sun, 25 Sep 2011 03:09:52 GMT

    * Connection #0 to host 127.0.0.1 left intact
    * Closing connection #0

  10. Check that you can GET account: curl -v -H 'X-Auth-Token: <token-from-x-auth-token-above>' <url-from-x-storage-url-above> 请注意这条语句中的空格,如果'X-Auth-Token: <token-from-x-auth-token-above>'间忽略了空格,将产生500 Internal Server Error。

    swift@ubuntu:~/bin$ curl -v -H 'X-Auth-Token: AUTH_tkf4f5252a030c4640b8fb04942849148c' http://127.0.0.1:8080/v1/AUTH_98a7125e8bb84ded9cfe15d8dd19bc6b
    * About to connect() to 127.0.0.1 port 8080 (#0)
    *   Trying 127.0.0.1... connected
    * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
    > GET /v1/AUTH_98a7125e8bb84ded9cfe15d8dd19bc6b HTTP/1.1
    > User-Agent: curl/7.21.3 (i686-pc-linux-gnu) libcurl/7.21.3 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.18
    > Host: 127.0.0.1:8080
    > Accept: */*
    > X-Auth-Token: AUTH_tkf4f5252a030c4640b8fb04942849148c

    < HTTP/1.1 204 No Content
    < X-Account-Object-Count: 0
    < X-Account-Bytes-Used: 0
    < X-Account-Container-Count: 0
    < Content-Length: 0
    < Date: Sun, 25 Sep 2011 03:15:33 GMT

    * Connection #0 to host 127.0.0.1 left intact
    * Closing connection #0

  11. Check that swift works: swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat    运行结果如下:

    swift@ubuntu:~/bin$ swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat
       Account: AUTH_98a7125e8bb84ded9cfe15d8dd19bc6b
    Containers: 0
       Objects: 0
         Bytes: 0
  12. cp ~/swift/trunk/test/functional/sample.conf /etc/swift/func_test.conf

  13. cd ~/swift/trunk; ./.functests (Note: functional tests will first delete everything in the configured accounts.)  

  14. cd ~/swift/trunk; ./.probetests (Note: probe tests will reset your environment as they call resetswift for each test.)  本步仍存在问题

If you plan to work on documentation (and who doesn’t?!老外好搞...):

On Ubuntu:
  1. sudo apt-get install python-sphinx   installs Sphinx.
  2. python setup.py build_sphinx   builds the documentation.
  3. 文档可以在/doc/build/html/index.html处浏览
On MacOS:
  1. sudo easy_install -U sphinx installs Sphinx.
  2. python setup.py build_sphinx builds the documentation.

Debugging Issues 调试问题

If all doesn’t go as planned, and tests fail, or you can’t auth, or something doesn’t work, here are some good starting places to look for issues:

  1. Everything is logged in /var/log/syslog, so that is a good first place to look for errors (most likely python tracebacks 确切地说会把出错记录和Python的trackback记录在此,调试的好东西).
  2. Make sure all of the server processes are running. For the base functionality, the Proxy, Account, Container, and Object servers should be running
  3. If one of the servers are not running, and no errors are logged to syslog, it may be useful to try to start the server manually, for example: swift-object-server /etc/swift/object-server/1.conf will start the object server. If there are problems not showing up in syslog, then you will likely see the traceback on startup.

 

 

posted @ 2011-09-24 11:35  牛皮糖NewPtone  阅读(...)  评论(...编辑  收藏