2011年12月27日

公欲利其事，必先利其器。同样在软件开发测试的世界里面，正确的使用工具，大大提高效率！

工作中用到不少工具，最近学到不少新工具CI， git， soupUI。 总结一下自己工具的心得：

1. 没有最好的工具，只有最适合自己的工具。

    这条告诉我们，学习新工具之前，先了解一下自己的问题或需求。先回答自己想解决什么问题？

2. 工具越强大，暴露的接口越复杂，其学习花的时间和精力越多，长期开看收益也大。

    试想我们工厂里面一个复杂机器，我们是如何学习的？同样学习软件工具，也需要花时间和精力去学习。

3. 如何找工具。

   如果不知道工具的名字，问身边人或者stackoverflow 不错，Google一下试试。 好的工具，两条：比较久，人气旺！

   找人 --》stackoverflow --》google--->文章，书！ 很多问题，都有很好的工具支持开解决，有开源的，和也有闭源的。

4. 学习工具。

   找官方文档，找身边的人！工具是自己自己问题的解决方案的外化，学习工具先学习工具背后的原理！同时考虑：

    模型是什么？输入是什么？输出是什么？可定制吗？

5. 解决自己的问题，同时集成到自己的系统里面。

   最好是自动化起来，一劳永逸！

希望大家分享自己心得！

  2011年12月22日

学习Git

1. Open mind .这样才不会墨守陈规，才可以好奇心，接受新观点，新方法，尝试不一样，创新。跑题了。

2. Learn tool in serious.  When you learn complex machine, you need and couch , take time ,spend effort to master it. So It is to learn new software tool.

3. Use the tool correct and efficiency.  Read the official document and tutorial with community.

4. Practice.   book + practice.  tool come from the mental model, so learn the concept , idea, or model behind the tool before.

5. Share it with community.   It will make you understand the tool much more deeply. 

Git :

Doc: http://progit.org/book/

Difference for merge and rebase:     http://www.jarrodspillers.com/2009/08/19/git-merge-vs-git-rebase-avoiding-rebase-hell/ ( see the first comment)

webinfo:  http://blog.longwin.com.tw/2009/05/git-learn-initial-document-2009/

Content:

1. Install git

2. Work with git in local

3. Work with git in branch

4. Work with git in remote branch

5. Git collabration model in open source.

6. Github

7. CI integration baseon github.

8. P4-git 

  2011年11月17日

c

The canonical URL of this page should be  http://pigtail.net/LRP/printsrv/cygwin-sshd.htm

This page has been tranlated to Bulgarian by Albert Ward, see this link: http://www.fatcow.com/edu/cygwin-sshd-bl/

Disclaimer

According to some cygwin gods, the only official document that you should use is  /usr/share/doc/Cygwin/openssh.README which is probably valid, but it seems to aim at users with a fair bit of Linux/Unix knowledge.
The purpose of this tutorial is for Windows users who are perhaps less familiar with Unix commands wanting to try out the famous Open Source ssh server (openSSH) on a desktop Windows XP , Windows Vista or Windows 7  .

The behavior of Windows 2003 Server is different. Follow this link to install OpenSSH on Windows 2003 Server, by Stephen Pillinger of the School of Computer Science, University of Birmingham. Or this link to install Cygwin SSH server on Windows 2003 Server, by  Kevin Scully at the University of Waterloo, Ontario, Canada.

There seems to be some problems when using cygwin ssh with McAfee 8.0i. A work around to that problem is available below.

Please don't send any questions to the cygwin mailing list to ask questions about this page as it seems to provoke them severely. They considered and declared the instructions on this page "broken" and "random" but won't constructively say what is "broken"; instead some went on launching personal attacks. The information here is provided "as is, in good faith" with no guarantee it will work. If it doesn't work, then it doesn't work. Don't send any questions to the Cygwin mailing list to ask why to provoke them. If you must go to Cygwin mailing list to ask, you better off completely remove Cygwin before you go to the mailing list to ask questions and don't even mention that you have looked at this web page (to avoid provoking them off).

Read this Disclaimer. 

Constructive comments are of course welcome, in the original spirit of the Internet, sharing experience and knowledge regarding bug fixes and improvements to benefit other users of the Internet community. My email address is

How to install a ssh server (called sshd, from OpenSSH) on a Windows 2000 or XP
How to install a sftp server on a Windows 2000 or XP

If you need a PDF converter, the post powerful pdf converter package available is Investintech's pdf server software bundle.

After you installed OpenSSH, you will find it simple to install application software on your Windows XP/Vista that will effectively turn your PC into a complete security camera system. Install security cameras into the back of your computer and you will have the ability to remotely access your security system over a network or Internet.

The ssh server is an emulation of the UNIX environment and OpenSSH for Windows, by Redhat, called cygwin.
The file system on your target machine should be journalled (e.g. NTFS) because FAT file system has bugs in file access.

(1a) Login as Administrator
Windows XP - login as a user with Administrator privilege;
Windows 2003 Server: login as local admin, it will not work for domain users or domain admin.

(1b) Make sure the current admin/user has a Windows password set.
If not, use Control Panel...User Accounts to create a password.
Just to be on the safe side, after you created a password, logoff and then log in again.

(2a) Create a folder c:\cygwin

(2b) Download cygwin's setup.exe from http://www.cygwin.com/ and save setup.exe in c:\cygwin

Cygwin's setup.exe has some uncommon properties, click here to find out more.

(2c) For Windows 7 and Windows Vista, navigate to c:\cygwin in explorer,  right click the setup.exe icon, and select "Run as Administrator". Thanks to David Shanks for his contributions.

(2d) For Windows XP: Click Start...Run...and type c:\cygwin\setup.exe

If you are asked to select "Just Me" or "All Users", choose "All Users"

When it asks for "Local Package Directory", type c:\cygwin

Choose a download site that is "close" to you.
When a selection screen comes up (you can resize the windows to see better),
click the little View button for "Full" view  ,
find the package "openssh", click on the word "skip" so that an appears in Column B,
see this illustration.
(optional) find the package "tcp_wrappers", click on the word "skip" so that an appears in Column B,
if you add "tcp_wrapper", you will most likely get "ssh-exchange-identification: Connectiion closed by remote host" error.
If you get that error, edit the file  /etc/hosts.allow and add these two lines
ALL: 127.0.0.1/32 : allow  
ALL: [::1]/128: allow
before the PARANOID line.
(optional) find the package "diffutils", click on the word "skip" so that an appears in Column B, 
find the package "zlib", click on the word "skip" (it should be already selected) so that an appears in Column B.

Notes:
tcp_wrappers provides host-based access control and possible need you to edit "/etc/hosts.allow" 
zlib is the compression and decompression library that is used by many programs.
Thanks to Lex Sheehan on the diffutils tips, and Thomas Braun for the hosts.allow tips.

Click to start installing cygwin and ssh.
Size of the basic cygwin system is more than 50 Meg, this may take a while.

Take a coffee break and wait.

While you wait, take a look at SherWeb's Exchange server hosting.

(3) This paragraph is no longer valid for newer versions of Cygwin 1.7.2 and later.
REF:  http://www.cygwin.com/cygwin-ug-net/ov-new1.7.html
Right click My Computer, Properties, Advanced, Environment Variables
See this illustration (red dots)
Click the "New" button to add a new entry to System variables:
variable name is CYGWIN
variable value is ntsec tty

Note: by setting variable ntsec, it will break sshd on MinGW by generating the error "WARNING: UNPROTECTED PRIVATE KEY FILE".
Thanks to Peter Rust for this issue.

(4) Right click My Computer, Properties, Advanced, Environment Variables
See this illustration (green dots)
Select the Path variable and click the "Edit" button:
append  ;c:\cygwin\bin   to the end of the existing variable string.

(5) 

For Windows 7 and Windows Vista  you may need to do additional uninstall steps if previous attempts to install sshd fails. Click here on how to do clean uninstall.

For Windwos XP, you should login as a user with admin privilege and that user belongs to a Windows "Users" group.

For Windows XP , open a cygwin window by double clicking the icon; a black screen pops open,
For Windows Vista and Windows 7 , right click the icon and choose "run as administrator"; a black screen pops open, type

With recent releases of cygwin, there are many permission problems. Add these 6 commands as work around:

chmod +r  /etc/passwd
chmod u+w /etc/passwd
chmod +r  /etc/group
chmod u+w /etc/group
chmod  755  /var  
touch /var/log/sshd.log
chmod 664 /var/log/sshd.log

ssh-host-config    (manuall answer Yes to questions except)
If the script says "This script plans to use cyg_server, Do you want to use a different name?  Answer Yes, and type sshd as the user name.

cyglsa-config

reboot the computer.

Thanks to David Spillett of Londdon, UK on the permission tips.
Thanks to Dave Lennert of Portland Oregon for the WIndows 7 tips.
Thanks to Frank Martin for the cyglsa-config tips.

When the script stops and asks you for "environment variable CYGWIN="     your answer is ntsec tty 
(click here for an explanation of ntsec)
(click here for an explanation of tty )
(thanks to Peter Reutemann of New Zealand and Ron Dozier of University of Delaware)
(thanks to Mike and Michael Pechner for the Windows Vista tip)
(thanks to Kevin Hilton on the Vista tips in the ssh-host-config section)

See Note 25 near the end of this web page if you need to run ssh-host-config again.

Run "rebaseall" as desribed in http://www.acooke.org/cute/CygwinSSHS0.html

(6) While you are still in the (black) cygwin screen, start the sshd service, type
net start sshd
or
cygrunsrv  --start  sshd

Click here on how to stop the sshd service.

If the service fails to start, try (thanks to Ross Beveridge of HP for this tip)
chown system /etc/ssh*
chown system /var/empty  or chown sshd_server /var/empty
net start sshd

If you get "ssh-exchange-identification: Connectiion closed by remote host" error.
edit the file  /etc/hosts.allow and add these two lines
ALL: 127.0.0.1/32 : allow  
ALL: [::1]/128: allow
before the PARANOID line. Thanks to Thomas Braun for the hosts.allow tips.

(7) Make sure every Windows user has a password set, if not, 
go to  Control Panel....User Accounts and create a password.

(7a) Make sure every Windows user has done the following at least once:
Login in as the Windows user, pop a console command screen by clicking Start...Run....cmd
Thanks to Magno Corrêa of Brazil for the tip in (7a)

(8) Pop a cygwin window, harmonize Windows user information with cygwin, otherwise they cannot login
mkpasswd   -cl   >   /etc/passwd
mkgroup   --local    >   /etc/group

If your XP logs on to a domain, you most likely have to manually edit /etc/group. See this page.

If  your local account name is the same as the domain name, then you would need to use Windows's User admin function to
rename the loacal account from name to name.local
Then rebuild the passwd and group files as shown above, then edit the /etc/group file as shown above. Open firewall's TCP port 22
 (Thanks to Christopher Poda of Venturi Wireless, Sunnyvale, California)

If your XP logs on to a domain, you may want to edit /etc/passwd to replace /home/username by //unc_server/path_to_home 
(thanks to Geoff Thomas)

mkpasswd creates a password file from Windows' user list, click here for more details.
mkgroup creates a group file from Windows' user list, click here for more details.
Thanks to John Skiggn of Cingular Wireless in Redmond, Washington for his tweak on domain user /etc/group

Test to see if sshd is working, pop a cygwin window (note: the command below is case sensitive)
whoami
ssh    localhost
or
ssh  -vvv  localhost
or
ssh    "$USERNAME@127.0.0.1"

if ssh complains "The authenticity of host xx.xx.xx.xx can't be established .... Are you sure you want to continue connecting (yes/no)?"  Answer yes  (Thanks to Daniel Griscom of Suitable Systems)

If you get an error message like "ssh-exchange-identification: Connection closed by remote host",
it is probably caused by McAfee 8.0i, see this page about the fix. (Thanks to Ron Dozier of University of Delaware, USA)

Error is also related to /etc/hosts.allow file, see http://www.cygwin.com/ml/cygwin/2008-12/msg00678.html

If you get an error message like "entry point _getreent", or "QuerryService Status: Win32 error 1062", it is probably
caused by the existance of an older version of "cygwin1.dll" located in the search path.
Do a full serarch of "cygwin1.dll" and remove the old version, except the current version at c:\cygwin\bin  (Thanks to Joe britton)

If you get a prompt without error messages, type
cd   /cygdrive/c
lsc
if you see a directory listing, success!  
(type exit to end the cygwin ssh session)  
Thanks to Roger Pack for his tips clarifying between Microsoft's ls.exe (installed by MS compilers) and cygwin's ls.exe

If you have a Windows username that contains space, expand the space into \ [space],
e.g. if the Windows login name is  Mickey mouse
ssh  Mickey\  mouse@127.0.0.1

If you have a Unix system that does not know what to do with TERM cygwin, add these scripts to .login

If you have troubles ssh into the server, try run ssh-user-configc

  2011年11月14日

Hudson 里面可以创建Job，去执行自动部署，需要让远程的服务器执行一些脚本。 问题出来了：CI如何告诉远程机器执行脚本？

在Hudson里面有插件，支持ssh,和Powershell 可以做到。

目前环境是window7作为hudson 的服务器， window2003 sp2 II6 为需要部署的环境。 下面用不同的办法去尝试：

第一个当然是ssh。服务器开启sshd 服务，CIserver 作为客户端。

服务器可以用cygwin作为一个sshd服务器，或者安装sshd一个单独服务器opensshd。但是window下面的IIS的命令行管理工具是居然是一个VBscript。 

在cygin 里面调用vbscript 的解析器Cscript去执行，类似 Script iisweb.vbs /start website. 居然执行不了。 排除路径错误，还是执行不了。 没有解决，网上一搜问题一片，目前没有解决。应该是Script 解释器引擎，与cygwin的环境不一致。

转换思路，尝试找其他exe工具，发现 appcmd.exe.  在window2008上面可以运行,不错，但是window2003不能运行。典型window不向后兼容。放弃!

发现webdeply.exe, 同样只支持window2003以上版本.悲摧！

时间不等人，再想办法.

IISRESET,也可以工作！居然支持完成调用。但是杀伤力气太大，每次都重启IIS。先用这个，项目要升级到2008/II7，到时问题自动解决！

第二个办事powershell2.0。 

首先powershell2.0 在window2003上面没有缺省安装。

安装powershell2.0： powershell2.0是window2003 的一个补丁包WindowsServer2003-KB968930-x64-ENG（http://www.microsoft.com/download/en/confirmation.aspx?id=4045）. 需要在administrator权限下面安装。

尝试远程登录：

wiki： powershell http://en.wikipedia.org/wiki/Windows_PowerShell

powershell 远程访问： http://tech.ccidnet.com/art/3539/20100801/2102911_1.html

记录下来，下次更新。

  2011年9月2日

https://wangyan.org/blog/s3cmd-how-to-use.html

s3cmd 安装使用指南

s3cmd 是一款 Amazon S3 命令行工具。它不仅能上传、下载、同步，还能设置权限，下面是完整的安装使用指南。

一、安装方法

方法一：（Debian/Ubuntu ）

1
2
3

wget -O- -q http://s3tools.org/repo/deb-all/stable/s3tools.key | sudo apt-key add -
wget -O/etc/apt/sources.list.d/s3tools.list http://s3tools.org/repo/deb-all/stable/s3tools.list
apt-get update && sudo apt-get install s3cmd

方法二：

1
2
3
4

wget http://nchc.dl.sourceforge.net/project/s3tools/s3cmd/1.0.0/s3cmd-1.0.0.tar.gz
tar -zxf s3cmd-1.0.0.tar.gz -C /usr/local/
mv /usr/local/s3cmd-1.0.0/ /usr/local/s3cmd/
ln -s /usr/local/s3cmd/s3cmd /usr/bin/s3cmd

二、使用方法

1、配置，主要是 Access Key ID 和 Secret Access Key

1

s3cmd --configure

2、列举所有 Buckets。（bucket 相当于根文件夹）

1

s3cmd ls

3、创建 bucket，且 bucket 名称是唯一的，不能重复。

1

s3cmd mb s3://my-bucket-name

4、删除空 bucket

1

s3cmd rb s3://my-bucket-name

5、列举 Bucket 中的内容

1

s3cmd ls s3://my-bucket-name

6、上传 file.txt 到某个 bucket，

1

s3cmd put file.txt s3://my-bucket-name/file.txt

7、上传并将权限设置为所有人可读

1

s3cmd put --acl-public file.txt s3://my-bucket-name/file.txt

8、批量上传文件

1

s3cmd put ./* s3://my-bucket-name/

9、下载文件

1

s3cmd get s3://my-bucket-name/file.txt file.txt

10、批量下载

1

s3cmd get s3://my-bucket-name/* ./

11、删除文件

1

s3cmd del s3://my-bucket-name/file.txt

12、来获得对应的bucket所占用的空间大小

1

s3cmd du -H s3://my-bucket-name

三、目录处理规则

以下命令都能将dir1 中的文件上传至my-bucket-name，但效果只截然不同的。

1）dir1 不带"/"斜杠，那么dir1会作为文件路径的一部分，相当于上传整个dir1目录，即类似 "cp -r dir1/"

1
2

~/demo$ s3cmd put -r dir1 s3://my-bucket-name/
dir1/file1-1.txt -> s3://my-bucket-name/dir1/file1-1.txt  [1 of 1]

2）带"/"斜杠的 dir1，相当于上传dir1目录下的所有文件，即类似 "cp ./* "

1
2

~/demo$ s3cmd put -r dir1/ s3://my-bucket-name/
dir1/file1-1.txt -> s3://my-bucket-name/file1-1.txt  [1 of 1]

四、同步方法

这是s3cmd 使用难点，但却是最实用的功能。官方使用说明见《s3cmd sync HowTo》

首先明确，同步操作是要进行MD5校验的，只有当文件不同时，才会被传输。

4.1、常规同步操作

1、同步当前目录下所有文件

1

s3cmd sync  ./  s3://my-bucket-name/

2、加 "--dry-run"参数后，仅列出需要同步的项目，不实际进行同步。

1

s3cmd sync  --dry-run ./  s3://my-bucket-name/

3、加 " --delete-removed"参数后，会删除本地不存在的文件。

1

s3cmd sync  --delete-removed ./  s3://my-bucket-name/

4、加 " --skip-existing"参数后，不进行MD5校验，直接跳过本地已存在的文件。

1

s3cmd sync  --skip-existing ./  s3://my-bucket-name/

4.2、高级同步操作

4.2.1、排除、包含规则（--exclude 、--include）

file1-1.txt被排除，file2-2.txt同样是txt格式却能被包含。

1
2
3

~/demo$ s3cmd sync --dry-run --exclude '*.txt' --include 'dir2/*' ./  s3://my-bucket-name/
exclude: dir1/file1-1.txt
upload: ./dir2/file2-2.txt -> s3://my-bucket-name/dir2/file2-2.txt

4.2.2、从文件中载入排除或包含规则。(--exclude-from、--include-from)

1

s3cmd sync  --exclude-from pictures.exclude ./  s3://my-bucket-name/

pictures.exclude 文件内容

1
2
3

# Hey, comments are allowed here ;-)
*.jpg
*.gif

4.2.3、排除或包含规则支持正则表达式

1

--rexclude 、--rinclude、--rexclude-from、--rinclude-from