免费注册 查看新帖 |

Chinaunix

  平台 论坛 博客 文库
最近访问板块 发新帖
查看: 3508 | 回复: 1

[FastDFS] storage存储节点同步问题和tracker之间调度问题 [复制链接]

论坛徽章:
0
发表于 2014-02-28 17:44 |显示全部楼层
请教下,我的测试tracker server
192.168.4.253
group1 中2个storage
192.168.4.252 nginx+mod_fastdfs
192.168.4.254 nginx+mod_fastdfs

2个问题
1 假设上传文件较大,上传到192.168.4.252 此时还没没有同步到192.168.4.254
那么如何保证用户请求不落在192.168.4.254
  我这边想在4.252和4.254之前搞一个nginx+upstream 分发请求到252、254
2 我从存储节点上下载文件的时候,存储节点和tracker server是否有数据交互?
  看到 nginx+mod_fastdfs有模块调用mod_fastdfs.conf
tracker_server=192.168.4.253:22122
   但我抓包没有发现有数据交互
谢谢


如下上传文件
# /usr/local/bin/fdfs_test /etc/fdfs/client.conf upload install.log
This is FastDFS client test program v5.01

Copyright (C) 2008, Happy Fish / YuQing

FastDFS may be copied only under the terms of the GNU General
Public License V3, which may be found in the FastDFS source kit.
Please visit the FastDFS Home Page http://www.csource.org/
for more detail.

[2014-02-28 17:17:26] DEBUG - base_path=/home/yuqing/fastdfs, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0

tracker_query_storage_store_list_without_group:
        server 1. group_name=, ip_addr=192.168.4.252, port=23000
        server 2. group_name=, ip_addr=192.168.4.254, port=23000

group_name=group1, ip_addr=192.168.4.252, port=23000
storage_upload_by_filename
group_name=group1, remote_filename=M00/00/00/wKgE_FMQVCWAMR4mAAAk8o19T2w416.log
source ip address: 192.168.4.252
file timestamp=2014-02-28 17:17:25
file size=9458
file crc32=2373799788
example file url: http://192.168.4.252:8080/group1 ... mAAAk8o19T2w416.log
storage_upload_slave_by_filename
group_name=group1, remote_filename=M00/00/00/wKgE_FMQVCWAMR4mAAAk8o19T2w416_big.log
source ip address: 192.168.4.252
file timestamp=2014-02-28 17:17:25
file size=9458
file crc32=2373799788
example file url: http://192.168.4.252:8080/group1 ... k8o19T2w416_big.log
假设我上传的文件比较大
当时在http://192.168.4.252:8080/group1 ... k8o19T2w416_big.log
已经可以访问
但是同步到同组另一个主机
http://192.168.4.254:8080/group1 ... k8o19T2w416_big.log 还需要时间
我如何解决 这个问题?

论坛徽章:
0
发表于 2014-03-03 17:14 |显示全部楼层
自己更新下:
只有在本机并没有需要获取的资源的情况下,才会进行特殊的处理。
当同组内其他Storeage新上传的图片本机并没有及时同步的时候,Nginx会根据url寻找上传的Storeage源服务器,进行redirect或者proxy两种方式之一的转发。
测试如下:
删除其中一个节点中的数据【非源数据192.168.4.254】然后通过http请求的这个不在本地磁盘的文件
会回源去请求文件【192.168.4.252】
在192.168.4.252上抓包tcpdump -i eth0 port 8080 分析
GET /group1/M00/00/00/wKgE_FMT_SmABZFXAAAE6F4aJyc89_big.conf?redirect=1  HTTP/1.0
User-Agent: Wget/1.11.4 Red Hat modified
Accept: */*
Host: 192.168.4.254:8080
Connection: Keep-Alive

HTTP/1.1 200 OK
Server: nginx/1.4.4
Date: Mon, 03 Mar 2014 09:05:05 GMT
Content-Length: 1256
Last-Modified: Mon, 03 Mar 2014 03:55:21 GMT
Connection: keep-alive
Accept-Ranges: bytes

disabled=false
group_name=group1
bind_addr=192.168.4.252
client_bind=true
port=23000
connect_timeout=30
network_timeout=60
heart_beat_interval=30
stat_report_interval=60
base_path=/home/yuqing/fastdfs
max_connections=256
buff_size = 256KB
accept_threads=1
work_threads=4
disk_rw_separated = true
disk_reader_threads = 1
disk_writer_threads = 1
sync_wait_msec=50
sync_interval=0
sync_start_time=00:00
sync_end_time=23:59
write_mark_file_freq=500
store_path_count=1
store_path0=/home/yuqing/fastdfs
subdir_count_per_path=256
tracker_server=192.168.4.253:22122
log_level=info
run_by_group=
run_by_user=
allow_hosts=*
file_distribute_path_mode=0
file_distribute_rotate_count=100
fsync_after_written_bytes=0
sync_log_buff_interval=10
sync_binlog_buff_interval=10
sync_stat_file_interval=300
thread_stack_size=512KB
upload_priority=10
if_alias_prefix=
check_file_duplicate=0
file_signature_method=hash
key_namespace=FastDFS
keep_alive=0
use_access_log = false
rotate_access_log = false
access_log_rotate_time=00:00
rotate_error_log = false
error_log_rotate_time=00:00
rotate_access_log_size = 0
rotate_error_log_size = 0
file_sync_skip_invalid_record=false
use_connection_pool = false
connection_pool_max_idle_time = 3600
http.domain_name=
http.server_port=8888
您需要登录后才可以回帖 登录 | 注册

本版积分规则 发表回复

  

北京盛拓优讯信息技术有限公司. 版权所有 京ICP备16024965号-6 北京市公安局海淀分局网监中心备案编号:11010802020122 niuxiaotong@pcpop.com 17352615567
未成年举报专区
中国互联网协会会员  联系我们:huangweiwei@itpub.net
感谢所有关心和支持过ChinaUnix的朋友们 转载本站内容请注明原作者名及出处

清除 Cookies - ChinaUnix - Archiver - WAP - TOP