一步一步学习网络编程

1.python客户端程序

#!/usr/bin/env python
# Simple Gopher Client - Chapter 1 - gopherclient.py

import socket, sys

port = 70
host = sys.argv[1]
filename = sys.argv[2]

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host, port))

s.sendall(filename + "\r\n")

while 1:
    buf = s.recv(2048)
    if not len(buf):
        break
    sys.stdout.write(buf)
    

在命令行下输入python gopherclient.py quux.org /

运行结果：

该程序实现的是Gopher协议，需要两个命令行参数：主机名和文件名，实现从主机上请求相关文档的功能。

程序简单分析：通过调用socket.socket()来建立一个Socket；参数告诉系统需要一个Internet socket来进行TCP通信；接着程序连接远程主机并提供文件名；这以后获得响应后，在屏幕上打印出来。

注意看程序可知没有错误检查机制，当有错误发生并产生异常时，例如给出一个不存在的主机名，就会出现如下结果：在命令行下输入python gopherclient.py nonexitant.example.com /

python会检测到错误并产生一个socket.error（注意此名字可能不一样）异常如此不够友好，故加入异常处理机制，代码如下：

#!/usr/bin/env python
# Simple Gopher Client with basic error handling - Chapter 1
# gopherclient2.py

import socket, sys

port = 70
host = sys.argv[1]
filename = sys.argv[2]

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
    s.connect((host, port))
#except socket.gaierror, e:
except socket.error, e:
    print "Error connecting to server: %s" % e
    sys.exit(1)

s.sendall(filename + "\r\n")

while 1:
    buf = s.recv(2048)
    if not len(buf):
        break
    sys.stdout.write(buf)
    

Now如果试图连接一个不存在的server，程序终止并得到一个友好的错误提示如下：

附录：

python的socket模块实际上定义了4种可能出现的异常：

1.与一般I/O和通信问题有关的socket.error;

2.与查询地址有关的socket.gaierror;

3.与其他地址错误有关的socket.herror;

4.与在一个socket上调用settimeout()后，处理超时有关的socket.timeout

下面采用文件类对象重写,代码如下：

#!/usr/bin/env python
# Simple Gopher Client with file-like interface - Chapter 1
# gopherclient3.py

import socket, sys

port = 70
host = sys.argv[1]
filename = sys.argv[2]

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host, port))
fd = s.makefile('rw', 0)

fd.write(filename + "\r\n")

for line in fd.readlines():
    sys.stdout.write(line)
    

不同之处：1.对makefile()函数的调用。此函数有两个可选参数：操作文件类的模式和缓存(buffering)的模式。操作文件类的模式表明是只读、只写或既写又读；本例中需要既读又写，所以是'rw'；缓存主要使用自磁盘文件，但是对于交互式的网络程序，它可能会阻碍程序的运行，故设置为0来关闭它。得到了文件类对象，于是就可以用以下方法：write()和readlines()

2.服务端程序

Servers basically do the following :

1. Open a socket
2. Bind to a address(and port).
3. Listen for incoming connections.
4. Accept connections
5. Read/Send

#!/usr/bin/env python
# Simple Server - Chapter 1 - server.py
import socket

host = ''                               # Bind to all interfaces
port = 51423

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((host, port))
s.listen(1)

print "Server is running on port %d; press Ctrl-C to terminate." \
      % port

while 1:
    clientsock, clientaddr = s.accept()
    clientfile = clientsock.makefile('rw', 0)
    clientfile.write("Welcome, " + str(clientaddr) + "\n")
    clientfile.write("Please enter a string: ")
    line = clientfile.readline().strip()
    clientfile.write("You entered %d characters.\n" % len(line))
    clientfile.close()
    clientsock.close()

代码解释：

第9行：s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
此行代码表示设置和得到socket选项；

python定义了setsockopt()和getsockopt()，一个是设置选项，一个是得到设置。这里主要使用setsockopt()，具体结构如下：

setsockopt(level,optname,value)

level定义了哪个选项将被使用。通常情况下是SOL_SOCKET，意思是正在使用的socket选项。它还可以通过设置一个特殊协议号码来设置协议选项，然而对于一个给定的操作系统，大多数协议选项都是明确的，所以为了简便，它们很少用于为移动设备设计的应用程序。

optname参数提供使用的特殊选项。关于可用选项的设置，会因为操作系统的不同而有少许不同。如果level选定了SOL_SOCKET，那么一些常用的选项见下表：

 

选项	意义	期望值
SO_BINDTODEVICE	可以使socket只在某个特殊的网络接口（网卡）有效。也许不能是移动便携设备	一个字符串给出设备的名称或者一个空字符串返回默认值
SO_BROADCAST	允许广播地址发送和接收信息包。只对UDP有效。如何发送和接收广播信息包	布尔型整数
SO_DONTROUTE	禁止通过路由器和网关往外发送信息包。这主要是为了安全而用在以太网上UDP通信的一种方法。不管目的地址使用什么IP地址，都可以防止数据离开本地网络	布尔型整数
SO_KEEPALIVE	可以使TCP通信的信息包保持连续性。这些信息包可以在没有信息传输的时候，使通信的双方确定连接是保持的	布尔型整数
SO_OOBINLINE	可以把收到的不正常数据看成是正常的数据，也就是说会通过一个标准的对recv()的调用来接收这些数据	布尔型整数
SO_REUSEADDR	当socket关闭后，本地端用于该socket的端口号立刻就可以被重用。通常来说，只有经过系统定义一段时间后，才能被重用。	布尔型整数

 此外还有一些与所使用的OS相关的选项：对于UNIX/LINUX系统的用户可以通过查看socket(7) (运行man 7 socket)或 setsocket(2)得到帮助。windows用户可以通过http://msdn.microsoft.com/library/default.asp?...来获得相关信息

下面给出一个python程序来给出你所使用机器上安装的python所支持的socket选项列表：

import socket
solist = [x for x in dir(socket) if x.startswith('SO_')]
solist.sort()
for x in solist:
    print x

本节在学习时，用到了SO_REUSEADDR选项，具体写法是：

s.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1) 这里value设置为1，表示将SO_REUSEADDR标记为TRUE，操作系统会在服务器socket被关闭或服务器进程终止后马上释放该服务器的端口，否则操作系统会保留几分钟该端口。

首先在命令行cd到server.py所在的目录执行python server.py,则出现：

再打开一个终端，直接执行telnet localhost 51423    (注：此处对于win7默认没有安装telnet，依次打开“开始”→“控制面板”→“打开或关闭Windows功能”，在打开的窗口处，寻找并勾选“Telnet客户端”，然后点击“确定”。顺利安装后，再在运行下输入此命令就OK了)，执行结果如下：

上述server代码中的绑定bind没有实现异常处理，故如下bind.py实现异常处理，更加友好：

import socket
import sys

HOST = ''    # Symbolic name meaning all available interfaces
PORT = 8888    # Arbitrary non-privileged port

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print 'Socket created'

try:
    s.bind((HOST, PORT))
except socket.error , msg:
    print 'Bind failed. Error Code : ' + str(msg[0]) + ' Message ' + msg[1]
    sys.exit()
    
print 'Socket bind complete'

We bind a socket to a particular IP address and a certain port number. By doing this we ensure that all incoming data which is directed towards this port number is received by this application.

然后就是监听函数：

s.listen(10)
print 'Socket now listening'

函数listen的参数叫做backlog，它控制传入连接的数量，如果程序繁忙的时 候，则保持“等待”，所以参数10就意味着，如果已经有10个连接等待处理，之后 第11个连接请求将会被拒绝，这对于后面的检查socket_accept就更清晰了。

最后就是接受等待了：

#wait to accept a connection - blocking call
conn, addr = s.accept()

#display client information
print 'Connected with ' + addr[0] + ':' + str(addr[1])
总程序就是：

import socket
import sys

HOST = ''    # Symbolic name meaning all available interfaces
PORT = 8888    # Arbitrary non-privileged port

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print 'Socket created'

try:
    s.bind((HOST, PORT))
except socket.error , msg:
    print 'Bind failed. Error Code : ' + str(msg[0]) + ' Message ' + msg[1]
    sys.exit()
    
print 'Socket bind complete'

s.listen(10)
print 'Socket now listening'

#wait to accept a connection - blocking call
conn, addr = s.accept()

#display client information
print 'Connected with ' + addr[0] + ':' + str(addr[1])

Output

Run the program. It should show

$ python server.py
Socket created
Socket bind complete
Socket now listening

So now this program is waiting for incoming connections on port 8888. Dont close this program , keep it running.
Now a client can connect to it on this port. We shall use the telnet client for testing this. Open a terminal and type

$ telnet localhost 8888

It will immediately show

$ telnet localhost 8888
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Connection closed by foreign host.

And the server output will show

$ python server.py
Socket created
Socket bind complete
Socket now listening
Connected with 127.0.0.1:59954

So we can see that the client connected to the server. Try the above steps till you get it working perfect.

We accepted an incoming connection but closed it immediately. This was not very productive. There are lots of things that can be done after an incoming connection is established. Afterall the connection was established for the purpose of communication. So lets reply to the client.

下面发送一些东东：

import socket
import sys

HOST = ''    # Symbolic name meaning all available interfaces
PORT = 8888    # Arbitrary non-privileged port

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print 'Socket created'

try:
    s.bind((HOST, PORT))
except socket.error , msg:
    print 'Bind failed. Error Code : ' + str(msg[0]) + ' Message ' + msg[1]
    sys.exit()
    
print 'Socket bind complete'

s.listen(10)
print 'Socket now listening'

#wait to accept a connection - blocking call
conn, addr = s.accept()

print 'Connected with ' + addr[0] + ':' + str(addr[1])

#now keep talking with the client
data = conn.recv(1024)
conn.sendall(data)

conn.close()
s.close()

在一个终端中运行上面的程序，然后在另一个终端里用telnet连接到这个server，会看到：

$ telnet localhost 8888
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
happy
happy
Connection closed by foreign host.
可以看到client(telnet)接收到一个来自server的回复

我们可以看到，连接后立刻关闭了，仅仅是因为server结束接收和发送回复，server像www.google.com的话就一直可以接收传入连接。

这意味着server得支持时时刻刻运行，毕竟server是为了服务， 我需要保持我们的 server不停止的运行，一个简单的办法可以到做到，就是把accept放到一个循 环中，这样它就可以时刻接收传入连接了。

Live Server(实时server)

import socket
import sys
 
HOST = ''   # Symbolic name meaning all available interfaces
PORT = 8888 # Arbitrary non-privileged port
 
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print 'Socket created'
 
try:
    s.bind((HOST, PORT))
except socket.error , msg:
    print 'Bind failed. Error Code : ' + str(msg[0]) + ' Message ' + msg[1]
    sys.exit()
     
print 'Socket bind complete'
 
s.listen(10)
print 'Socket now listening'
 
#now keep talking with the client
while 1:
    #wait to accept a connection - blocking call
    conn, addr = s.accept()
    print 'Connected with ' + addr[0] + ':' + str(addr[1])
     
    data = conn.recv(1024)
    reply = 'OK...' + data
    if not data: 
        break
     
    conn.sendall(reply)
 
conn.close()
s.close()

We havent done a lot there. Just put the socket_accept in a loop.

Now run the server program in 1 terminal , and open 3 other terminals.
From each of the 3 terminal do a telnet to the server port.

Each of the telnet terminal would show :

$ telnet localhost 5000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
happy
OK .. happy
Connection closed by foreign host.

And the server terminal would show

$ python server.py
Socket created
Socket bind complete
Socket now listening
Connected with 127.0.0.1:60225
Connected with 127.0.0.1:60237
Connected with 127.0.0.1:60239

现在server以不停的状态运行着，而telnet的连接也没有停止，现在关闭server 程序，所有telnet终端里将会显示"Connection closed by foreign host."

到目前一切都还不错，但server和client之间还没有有效的通信，server程序在一个循环中接受连接，然后只发给client一个回复，然后就对client不理不踩了， 也无法同时处理处理超过一个连接，看来得想办法处理连接，想办法让多连接处理也一起搞上。

到此处便是关键了：

为了处理每一个连接，我们需要分离处理部分代码，让它在主server中单独运行以接受连接，一个达到这个目的的办法是使用线程，主server程序接受一个连接并创建一个新线程来处理连接的通信，然后主server返回接受更多的连接；此处涉及到线程的知识，创建线程或进程来处理客户请求时一个很常用的手段。

理解：

一旦接收到一个连接，accept()函数就会返回一个单独的客户的套接字用于后续的通讯。使用
新的客户套接字就像把客户的电话转给一个客户服务人员。当一个客户打电话进来的时候，总机接
了电话，然后把电话转到合适的人那里来处理客户的需求。

这样就可以空出总机，也就是最初的那个服务器套接字，于是，话务员就可以等待下一个电话
（客户的请求），与此同时，前一个客户与对应的客户服务人员在另一条线路上进行着他们自己的对
话。同样的，当一个请求到来时，要创建一个新的端口，然后直接在那个端口上与客户对话，这样
就可以空出主端口来接受其它客户的连接。

We shall now use threads to create handlers for each connection the server accepts.

import socket
import sys
from thread import *
 
HOST = ''   # Symbolic name meaning all available interfaces
PORT = 8888 # Arbitrary non-privileged port
 
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print 'Socket created'
 
#Bind socket to local host and port
try:
    s.bind((HOST, PORT))
except socket.error , msg:
    print 'Bind failed. Error Code : ' + str(msg[0]) + ' Message ' + msg[1]
    sys.exit()
     
print 'Socket bind complete'
 
#Start listening on socket
s.listen(10)
print 'Socket now listening'
 
#Function for handling connections. This will be used to create threads
def clientthread(conn):
    #Sending message to connected client
    conn.send('Welcome to the server. Type something and hit enter\n') #send only takes string
     
    #infinite loop so that function do not terminate and thread do not end.
    while True:
         
        #Receiving from client
        data = conn.recv(1024)
        reply = 'OK...' + data
        if not data: 
            break
     
        conn.sendall(reply)
     
    #came out of loop
    conn.close()
 
#now keep talking with the client
while 1:
    #wait to accept a connection - blocking call
    conn, addr = s.accept()
    print 'Connected with ' + addr[0] + ':' + str(addr[1])
     
    #start new thread takes 1st argument as a function name to be run, second is the tuple of arguments to the function.
    start_new_thread(clientthread ,(conn,))
 
s.close()

 

Run the above server and open 3 terminals like before. Now the server will create a thread for each client connecting to it.

The telnet terminals would show :

$ telnet localhost 8888
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Welcome to the server. Type something and hit enter
hi
OK...hi
asd
OK...asd
cv
OK...cv

The server terminal might look like this

$ python server.py
Socket created
Socket bind complete
Socket now listening
Connected with 127.0.0.1:60730
Connected with 127.0.0.1:60731

The above connection handler takes some input from the client and replies back with the same.

So now we have a server thats communicative. Thats useful now.

 

3.连接到一个server：另外一种实现代码：

client

import socket    #for sockets
import sys    #for exit

try:
    #create an AF_INET, STREAM socket (TCP)
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error, msg:
    print 'Failed to create socket. Error code: ' + str(msg[0]) + ' , Error message : ' + msg[1]
    sys.exit();

print 'Socket Created'

host = 'www.google.com'
port = 80

try:
    remote_ip = socket.gethostbyname( host )

except socket.gaierror:
    #could not resolve
    print 'Hostname could not be resolved. Exiting'
    sys.exit()
    
print 'Ip address of ' + host + ' is ' + remote_ip

#Connect to remote server
s.connect((remote_ip , port))

print 'Socket Connected to ' + host + ' on ip ' + remote_ip

此程序中第一步首先要获取远程主机/地址的ip地址，即16行代码所示；然后用connect函数连接到ip的指定端口上，运行程序

此处事先已经知道了http的端口号是80，如果事先不知道可以调用socket.getservbyname()来获取，具体事先如下：

port = socket.getservbyname('http', 'tcp')

此函数需要两个参数：协议名和端口名。端口名是一个字符串，注意不是端口号

“连接”的概念应用到SOCK_STREAM/TCP类型的sockets，连接意味一个可靠的数 据流，这样就可以有多个这样的数据流，每一个具有它自己的通信。可以把它想 象成一个管道，这个管道不会影响其他管道的数据，流连接的另一个重要特性就 是数据包有一个“顺序”或“序列”。
其他sockets像UDP，ICMP，ARP没有连接的概念，它们是以非连接为基础的通信。

接下来发送一些数据到google上，在后面添加如下代码：

#Send some data to remote server
message = "GET / HTTP/1.1\r\n\r\n"

try :
    #Set the whole string
    s.sendall(message)
except socket.error:
    #Send failed
    print 'Send failed'
    sys.exit()

print 'Message send successfully'

In the above example , we first connect to an ip address and then send the string message "GET / HTTP/1.1\r\n\r\n" to it. The message is actually an "http command" to fetch the mainpage of a website.

4.下面开始接收发送的数据(即来自server的回复)

#Socket client example in python

import socket    #for sockets
import sys    #for exit

#create an INET, STREAMing socket
try:
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error:
    print 'Failed to create socket'
    sys.exit()
    
print 'Socket Created'

host = 'www.google.com';
port = 80;

try:
    remote_ip = socket.gethostbyname( host )

except socket.gaierror:
    #could not resolve
    print 'Hostname could not be resolved. Exiting'
    sys.exit()

#Connect to remote server
s.connect((remote_ip , port))

print 'Socket Connected to ' + host + ' on ip ' + remote_ip

#Send some data to remote server
message = "GET / HTTP/1.1\r\n\r\n"

try :
    #Set the whole string
    s.sendall(message)
except socket.error:
    #Send failed
    print 'Send failed'
    sys.exit()

print 'Message send successfully'

#Now receive data
reply = s.recv(4096)

print reply

Google.com replied with the content of the page we requested. Quite simple!
Now that we have received our reply, its time to close the socket.

函数close用于关闭socket

s.close()

如此便结束了

Lets Revise

 

So in the above example we learned how to :
1. Create a socket
2. Connect to remote server
3. Send some data
4. Receive a reply

 

Its useful to know that your web browser also does the same thing when you open www.google.com
This kind of socket activity represents a CLIENT. A client is a system that connects to a remote system to fetch data.

 

The other kind of socket activity is called a SERVER. A server is a system that uses sockets to receive incoming connections and provide them with data. It is just the opposite of Client. So www.google.com is a server and your web browser is a client. Or more technically www.google.com is a HTTP Server and your web browser is an HTTP client.

 接下来进一步学习：首先列出一些socket模块中常用的函数用法如下：

Here is documentation in-depth on the functions available through the socket module.

    gethostbyaddr(...)    

    gethostbyaddr(host) -> (name, aliaslist, addresslist)                  Return the true host name, a list of aliases, and a list of IP addresses, for a host.  The host   argument is a string giving a host name or IP number.     

  gethostbyname(...)        

 gethostbyname(host) -> address              Return the IP address (a string of the form '255.255.255.255') for a host.    

 gethostbyname_ex(...)         

gethostbyname_ex(host) -> (name, aliaslist, addresslist)               Return the true host name, a list of aliases, and a list of IP addresses,         for a host.  The host argument is a string giving a host name or IP number.     

gethostname(...)       

  gethostname() -> string                         Return the current host name.    

 getnameinfo(...)         

getnameinfo(sockaddr, flags) --> (host, port)         Get host and port for a sockaddr.     

getprotobyname(...)         

getprotobyname(name) -> integer            Return the protocol number for the named protocol.  (Rarely used.)   

  getservbyname(...)        

 getservbyname(servicename[, protocolname]) -> integer              Return a port number from a service name and protocol name.         The optional protocol name, if given, should be 'tcp' or 'udp',         otherwise any protocol will match.    

 getservbyport(...)         

getservbyport(port[, protocolname]) -> string                               Return the service name from a port number and protocol name.         The optional protocol name, if given, should be 'tcp' or 'udp', otherwise any protocol will match.     

getservbyport(...)        

 getservbyport(port[, protocolname]) -> string                             Return the service name from a port number and protocol name.         The optional protocol name, if given, should be 'tcp' or 'udp',         otherwise any protocol will match.

 此外还有很多其他的module，详细参见http://python.about.com/od/pythonstandardlibrary/a/sock_doc3_3.htm

下面再附加一些module的用法，如getsockname():获取本机的IP及端口号,getpeername()：获取远程机器的IP及端口号，代码如下

import socket

print "Creating socket..."
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print 'done.'

print "Looking up port number..."
port = socket.getservbyname('http', 'tcp')
print 'done.'

print "Connecting to remote host on port %d..." % port
s.connect(("www.baidu.com", port))
print 'done.'

print "Connected from", s.getsockname()
print "Connected to", s.getpeername()

运行结果：

Creating socket...
done.
Looking up port number...
done.
Connecting to remote host on port 80...
done.
Connected from ('192.168.**.**', 51881)
Connected to ('115.239.210.26', 80)

概念：

python提供了两种方法：socket对象和文件类对象；socket对象提供了操作系统的send(),sendto(),recv()和recvfrom()调用的接口。文件类对象提供了send(),write()和readline()这些更加典型的接口。文件类对象一般是面向线性的协议，因为它可以通过提供readline()函数自动处理大多数的解析；

 注：此处附录一个如何在windows和linux下获取本地IP：然而文件类对象一般只对TCP连接较好，对UDP连接反而不好，这时因为TCP连接的行为更像是标准的文件，可以保证数据接收的准确性，并且和文件一样是以字节流形式运转的；而UDP并不像文件那样以字节流形式运转，相反，踏实一种基于信息包的通信，文件类对象无法操作每个基本的信息包。

Windows下获得IP地址的方法

方法一

使用拨号上网的话，一般都有一个本地ip和一个外网ip，使用python可以很容易的得到这两个ip 使用gethostbyname和gethostbyname_ex两个函数可以实现

import socket
localIP = socket.gethostbyname(socket.gethostname())#得到本地ip
print "local ip:%s "%localIP
 
ipList = socket.gethostbyname_ex(socket.gethostname())
for i in ipList:
    if i != localIP:
       print "external IP:%s"%i

方法二

import socket
 
myname = socket.getfqdn(socket.gethostname())
myaddr = socket.gethostbyname(myname)

Linux下获得IP地址的方法

上面的方法在Linux下也可以使用，除此之外，Linux下还可以用下面的方法得到本机IP地址。

Uses the Linux SIOCGIFADDR ioctl to find the IP address associated with a network interface, given the name of that interface, e.g. “eth0”. The address is returned as a string containing a dotted quad.

import socket
import fcntl
import struct
 
def get_ip_address(ifname):
    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    return socket.inet_ntoa(fcntl.ioctl(
        s.fileno(),
        0x8915,  # SIOCGIFADDR
        struct.pack('256s', ifname[:15])
    )[20:24])