Fork me on GitHub

fastdfs安装+python客户端

fastdfs安装+python客户端

1.环境：

  • Centos7.6

2.安装：

  • 根据版本下载

    https://github.com/happyfish100/fastdfs/releases

  • gcc安装

    yum install gcc-c++

  • libevent安装

    yum -y install libevent

  • 安装libfastcommon

    • libfastcommon从开源项目FastDFS中提取的常用函数库。这个库非常简单和稳定。功能包括：字符串，记录器，链，哈希，套接字，ini文件阅读器，base64编码/解码，url编码/解码，快速计时器，跳过列表，对象池等。详细信息请参阅c头文件。

    下载地址：https://github.com/happyfish100/libfastcommon/releases

    • 安装

      //解压文件
tar -zxvf libfastcommon-1.0.39.tar.gz
cd libfastcommon-1.0.39
//编译文件
./make.sh
//安装依赖
./make.sh install

    • 安装出错：

      // 安装所需依赖
yum -y install zlib zlib-devel pcre pcre-devel gcc gcc-c++ openssl openssl-devel libevent libevent-devel perl unzip net-tools wget

  • Tracker编译安装

    //解压FastDFS文件
tar -zxvf fastdfs-5.11.tar.gz
cd fastdfs-5.11
//编译源码
./make.sh
//安装Tracker
./make.sh install

    • 安装成功将安装目录下的conf下的文件拷贝到/etc/fdfs/下。

      //复制配置文件
cp -r conf/* /etc/fdfs/

    • 配置Tracker

      mkdir /home/FastDFS

//拷贝一份新的配置文件
cp tracker.conf.sample  tracker.conf
//修改tracker.conf
vi tracker.conf  
 
//配置基本路径
base_path=/home/yuqing/FastDFS    
//改为 
//没有此目录需要创建此目录，个人一般使用FastDFS作为存放数据的基本目录
base_path=/home/FastDFS   
 
//配置http端口
http.server_port=80

    • 启动Tracker

      /usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf start 启动
/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart 重启
/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf stop  停止

  • Storage安装

    • 安装Tracker已经安装过Storage,直接配置Storage就好

      //重命名storage配置文件
cp storage.conf.sample storage.conf
 
//修改storage.conf
vim storage.conf
 
 
//配置base_path
base_path=/home/fastdfs
//配置store_path0
store_path0=/home/fastdfs/fdfs_storage
//配置Tracker_Server
tracker_server=你的ip:22122
 
//配置http端口
http.server_port=8888

    • 启动Storage

      mkdir /home/fastdfs
mkdir /home/fastdfs/fdfs_storage


/usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart 启动Storage
/usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart 重启
/usr/bin/fdfs_storaged /etc/fdfs/storage.conf stop 关闭
 
 
//正常启动过程
[root@localhost home]# /usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart
waiting for pid [23896] exit ...
starting ...
[root@localhost home]#

  • 文件上传测试

    • 客户端配置

      mkdir /home/fastdfsi

//在/etc/fdfs下配置client.conf文件
cp client.conf.sample client.conf
vim client.conf
 
//根据自己虚拟机的IP配置，目录没有必须手动创建
base_path=/home/fastdfsi
tracker_server=你的ip:22122
//如果在同一台虚拟机不能和TrackerServer使用相同的端口
http.tracker_server_port=8080

    • 上传文件

      //个人测试如下
/usr/bin/fdfs_test /etc/fdfs/client.conf upload /opt/mp3/阿刁.mp3
 //上传格式
/usr/bin/fdfs_test /etc/fdfs/client.conf  upload  上传文件

    • 上传日志正常打印

      This is FastDFS client test program v5.11

Copyright (C) 2008, Happy Fish / YuQing

FastDFS may be copied only under the terms of the GNU General
Public License V3, which may be found in the FastDFS source kit.
Please visit the FastDFS Home Page http://www.csource.org/ 
for more detail.

[2021-01-17 21:57:42] DEBUG - base_path=/home/fastdfsi, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0

tracker_query_storage_store_list_without_group: 
	server 1. group_name=, ip_addr=10.0.0.134, port=23000

group_name=group1, ip_addr=10.0.0.134, port=23000
storage_upload_by_filename
group_name=group1, remote_filename=M00/00/00/CgAAhmAEQlaAC7dJAMLF83jkALE177.mp3
source ip address: 10.0.0.134
file timestamp=2021-01-17 21:57:42
file size=12764659
file crc32=2028208305
example file url: http://10.0.0.134:8080/group1/M00/00/00/CgAAhmAEQlaAC7dJAMLF83jkALE177.mp3
storage_upload_slave_by_filename
group_name=group1, remote_filename=M00/00/00/CgAAhmAEQlaAC7dJAMLF83jkALE177_big.mp3
source ip address: 10.0.0.134
file timestamp=2021-01-17 21:57:42
file size=12764659
file crc32=2028208305
example file url: http://10.0.0.134:8080/group1/M00/00/00/CgAAhmAEQlaAC7dJAMLF83jkALE177_big.mp3

3.FastDFS和Nginx

tar -zxvf fastdfs-nginx-module-1.20.tar.gz

//修改config文件
vim fastdfs-nginx-module-1.20/src/config
 
ngx_module_incs="/usr/local/include"
修改为：
ngx_module_incs="/usr/include/fastdfs /usr/include/fastcommon/"
 
CORE_INCS="$CORE_INCS /usr/local/include"
修改为：
CORE_INCS="$CORE_INCS /usr/include/fastdfs /usr/include/fastcommon/"
  • 拷贝mod_fastdfs.conf并进行配置
cp /opt/xjk_fastdfs/fastdfs-nginx-module-1.20/src/mod_fastdfs.conf /etc/fdfs/
//修改配置文件
vim /etc/fdfs/mod_fastdfs.conf
 
base_path=/home/fastdfs  //基本路径
tracker_server=192.168.168.138:22122  //Tracker的服务IP，根据个人配置
url_have_group_name = true   //是否含有组名
store_path0=/home/fastdfs/fdfs_storage  //存储目录

//解压Nginx
tar -zxvf nginx-1.16.1.tar.gz
//配置安装路径
cd nginx-1.16.1

//定义nginx的安装目录 和fastdfs第三方模块
./configure --prefix=/usr/soft/nginx --add-module=/opt/xjk_fastdfs/fastdfs-nginx-module-1.20/src

//编译安装Nginx
make && make install
  • 配置nginx
//编辑nginx配置文件
vim /usr/soft/nginx/conf/nginx.conf

# FastDFS配置
location /group1/M00/ {
	root /home/fastdfs/fdfs_storage/data;
	ngx_fastdfs_module;
}
  • 启动nginx
/usr/soft/nginx/sbin/nginx
  • 重新加载
/usr/soft/nginx/sbin/nginx -s reload
  • 访问：
http://10.0.0.134/group1/M00/00/00/CgAAhmAESjuAHX0GAAHkf3qQ_Y8975_big.jpg
http://10.0.0.134/group1/M00/00/00/CgAAhmAEQlaAC7dJAMLF83jkALE177_big.mp3

4.python使用fastdfs客户端

  • 下载

    https://github.com/JaceHo/fdfs_client-py

  • windows修改：

    1. fdfs_client-py-master.zip 中setup.py
	33,34注释掉
	# 'ext_modules': [Extension('fdfs_client.sendfile',   
    #                           sources=['fdfs_client/sendfilemodule.c'])],
2、fdfs_client 目录下的storage_client.py
    注释掉12行
    # from fdfs_client.sendfile import *

  • 重新打回zip包

  • 安装依赖

    pip install mutagen
pip install requests

  • 安装包

H:\env\Scripts\python3.exe -m pip install J:\fdfs_client-py-master.zip
  • 将服务client.conf复制本地文件夹
  • 配置client.conf
tracker_server=10.0.0.134:22122
base_path=/home/fastdfsi
  • 上传代码演示：
from fdfs_client.client import Fdfs_client
# 加载配置文件
client = Fdfs_client('./client.conf')
ret = client.upload_by_filename(r"G:\a.txt")
print(ret)
# {'Group name': 'group1', 'Remote file_id': 'group1\\M00/00/00/CgAAhmAEVpiAZf3wAAAAQSbW53M734.txt', 'Status': 'Upload successed.', 'Local file name': 'G:\\a.txt', 'Uploaded size': '65B', 'Storage IP': '10.0.0.134'}
  • 常见报错
1、ModuleNotFoundError: No module named 'mutagen._compat'
# 根据报错找到utils.py
将from mutagen._compat import StringIO改成from mutagen._senf._compat import StringIO
2、fdfs_client.exceptions.DataError: [-] Error: 22, Invalid argument
上传文件写入绝对路径
posted @ 2021-01-17 23:33  是阿凯啊  阅读(541)  评论(0)  编辑  收藏  举报