免费注册 查看新帖 |

Chinaunix

  平台 论坛 博客 文库
最近访问板块 发新帖
楼主: happy_fish100

[FastDFS] FastDFS FAQ (欢迎反馈,我将及时整理) [复制链接]

论坛徽章:
0
发表于 2014-06-20 23:54 |显示全部楼层
今天启动storage时遇到这样的问题,下面是启动时的storaged.log 日志内容:

[2014-06-20 22:09:10] INFO - FastDFS v4.05, base_path=/fddata/pan/fastdfs, store_path_count=1, subdir_count_per_path=32, group_name=group6, run_by_group=, run_by_user=, connect_timeout=30s, network_timeout=60s, port=23000, bind_addr=, client_bind=1, max_connections=256, work_threads=4, disk_rw_separated=1, disk_reader_threads=1, disk_writer_threads=1, buff_size=256KB, heart_beat_interval=30s, stat_report_interval=60s, tracker_server_count=2, sync_wait_msec=50ms, sync_interval=0ms, sync_start_time=00:00, sync_end_time=23:59, write_mark_file_freq=500, allow_ip_count=-1, file_distribute_path_mode=0, file_distribute_rotate_count=100, fsync_after_written_bytes=0, sync_log_buff_interval=10s, sync_binlog_buff_interval=10s, sync_stat_file_interval=300s, thread_stack_size=512 KB, upload_priority=10, if_alias_prefix=, check_file_duplicate=0, file_signature_method=hash, FDHT group count=0, FDHT server count=0, FDHT key_namespace=, FDHT keep_alive=0, HTTP server port=8888, domain name=, use_access_log=0, rotate_access_log=0, access_log_rotate_time=00:00, rotate_error_log=0, error_log_rotate_time=00:00, rotate_access_log_size=0, rotate_error_log_size=0, file_sync_skip_invalid_record=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s
[2014-06-20 22:09:10] INFO - file: storage_param_getter.c, line: 187, use_storage_id=0, id_type_in_filename=ip, storage_ip_changed_auto_adjust=1, store_path=0, reserved_storage_space=10.00%, use_trunk_file=1, slot_min_size=256, slot_max_size=16 MB, trunk_file_size=64 MB, trunk_create_file_advance=0, trunk_create_file_time_base=02:00, trunk_create_file_interval=86400, trunk_create_file_space_threshold=20 GB, trunk_init_check_occupying=0, trunk_init_reload_from_binlog=0, store_slave_file_use_link=0
[2014-06-20 22:09:10] INFO - file: storage_func.c, line: 174, tracker_client_ip: 192.168.124.72, my_server_id_str: 192.168.124.72, g_server_id_in_filename: 1216129216
[2014-06-20 22:09:10] INFO - local_host_ip_count: 2,  127.0.0.1  192.168.124.72
[2014-06-20 22:09:10] INFO - file: tracker_client_thread.c, line: 308, successfully connect to tracker server 192.168.147.123:22122, as a tracker client, my ip is 192.168.124.72
[2014-06-20 22:09:10] INFO - file: tracker_client_thread.c, line: 1124, tracker server 192.168.147.123:22122, set tracker leader: 192.168.147.123:22122
[2014-06-20 22:09:10] INFO - file: tracker_client_thread.c, line: 1185, I am the the trunk server 192.168.124.72:23000
[2014-06-20 22:09:10] ERROR - file: trunk_mgr/trunk_sync.c, line: 837, read data from binlog file "/fddata/pan/fastdfs/data/trunk/binlog" fail, file offset: 27429432, read item count: 1 < 8
[2014-06-20 22:09:10] ERROR - file: trunk_mgr/trunk_sync.c, line: 779, read from binlog file "/fddata/pan/fastdfs/data/trunk/binlog" fail, file offset: 27429433, line buffer size: 128 is too small! <= line length: 550
[2014-06-20 22:09:10] ERROR - file: trunk_mgr/trunk_mem.c, line: 777, trunk load fail, errno: 28, error info: No space left on device
[2014-06-20 22:09:11] INFO - file: tracker_client_thread.c, line: 308, successfully connect to tracker server 192.168.147.122:22122, as a tracker client, my ip is 192.168.124.72
[2014-06-20 22:09:11] INFO - file: trunk_mgr/trunk_sync.c, line: 1068, trunk sync thread to storage server 192.168.124.73:23000 started
[2014-06-20 22:09:11] INFO - file: tracker_client_thread.c, line: 308, successfully connect to tracker server 192.168.147.123:22122, as a tracker client, my ip is 192.168.124.72
[2014-06-20 22:09:11] ERROR - file: trunk_mgr/trunk_sync.c, line: 1141, connect to storage server 192.168.124.73:23000 fail, errno: 111, error info: Connection refused
[2014-06-20 22:09:11] INFO - file: tracker_client_thread.c, line: 1124, tracker server 192.168.147.123:22122, set tracker leader: 192.168.147.123:22122
[2014-06-20 22:09:11] WARNING - file: tracker_client_thread.c, line: 1179, I am already the trunk server 192.168.124.72:23000, may be the tracker server restart
[2014-06-20 22:09:17] ERROR - file: trunk_mgr/trunk_mem.c, line: 1378, I am not inited!
[2014-06-20 22:09:17] ERROR - file: storage_nio.c, line: 314, client ip: 127.0.0.1, pkg length: -4431937177564820894 < 0
[2014-06-20 22:09:22] ERROR - file: trunk_mgr/trunk_mem.c, line: 1378, I am not inited!
[2014-06-20 22:09:31] ERROR - file: trunk_mgr/trunk_mem.c, line: 1378, I am not inited!
[2014-06-20 22:09:31] ERROR - file: trunk_mgr/trunk_mem.c, line: 1378, I am not inited!

期初看到 trunk load fail, errno: 28, error info: No space left on device 以为是磁盘空间的问题
分别用df查看了磁盘空间和inode节点使用情况,都是正常的,没有到不足的情况

后来又查看了下日志中报错读取失败的 binlog文件(UTF8打开),发现和其他正常的storage server不同,除了有一些记录时间,文件等信息的内容,多出很多乱码的内容,如下

1401804499 D 0 0 22 871 48095760 1051468
1401804501 D 0 0 22 871 49147228 17961636
1401804501 A 0 0 22 871 55574092 11534772
1401804501 A 0 0 22 871 49147228 6426864
1401804501 D 0 0 22 871 49147228 6426864
1401804505 D 0 0 22 869 67107883 981
1401804505 A 0 0 22 869 67107995 869
1401804505 A 0 0 22 869 67107883 112
1401804505 D 0 0 22 869 67107883 112
�mbZn�Ȉ���t�n�v��-���Q�U �屶�\��j�0K7Z3o�Ah���6���e        ���΄S�Z���N��2��J��53��uML⒖�srX���?��Y籶e���IY�1pn8g�8'�sL:���*ʉ�X���CLf����y��_'%�l�ӖRB
�ת:V=V�R69��+%�.QXֺa��Ԉ�Na�?��[�i��K���7�gq?o��5bx���0���w�: c�]��c����U��##~��,�gQ�8}b�m��0��~�zj��hzSv��t8�?X=���4�
��:�K%��7z��6á���+^�}�lb��(��{��i��f��(\�ǣ`,�M�t�������k:�yaXEd�p�.��9�a��cm��

不知道是不是binlog文件损坏导致启动时不能读取的,还是应该通过其他什么思路去进一步定位?

另外,如果是binlog文件损坏了,这个文件能不能重建,因为之前看到的资料说,这个binlog记录的是文件的操作记录,如果删除,组内新增节点会不能同步已有文件

不知道有没有其他人遇到过类似情况。。。

请余大指导

论坛徽章:
0
发表于 2014-06-23 17:53 |显示全部楼层
请问版主:FastDFS支持断点续传功能吗?

论坛徽章:
0
发表于 2014-06-30 16:41 |显示全部楼层
请问怎么删除一个group ?不是删除group下的storage哦!

论坛徽章:
0
发表于 2014-06-30 17:46 |显示全部楼层
请问怎么删除一个group ?不是删除group下的storage哦!

论坛徽章:
0
发表于 2014-07-03 14:01 |显示全部楼层
我ip 192.168.1.41  安装好后nginx,http://192.168.1.41  正常打开网站 ,但一安装fastdfs nginx扩展后导致网站再也打不开了,nginx重启也成功,不报任何错误,但网站就是再也打不开了
安装fastdfs完全按照INSTALL 说明文件安装

论坛徽章:
0
发表于 2014-07-16 14:34 |显示全部楼层
Fish大哥,请问我从tracker可以上传,然后再storage上面也可以看到上传的文件,但是在从tracker访问那个文件显示404,请问是什么问题?或者在哪里可以看到日志文件

论坛徽章:
0
发表于 2014-07-23 18:01 |显示全部楼层
本帖最后由 wojiushilzq 于 2014-07-23 18:08 编辑

我是新手,能给我一个您觉得不错的搭建FastDFS系统的相关教程的帖子地址吗?
我用的是最新的5.01版本

论坛徽章:
0
发表于 2014-08-02 15:44 |显示全部楼层
fdfs版本5.0.1 启动tracker报错。有谁能提供帮助啊
错误信息如下:
ERROR - file: ../common/process_ctrl.c, line: 189, "/fdfs/tracker" can't be accessed, error info: No such file or directory

论坛徽章:
0
发表于 2014-09-01 21:24 |显示全部楼层
FASTDFS采用的什么负载均衡算法?比如选择存储路径和选择上传文件的数据节点时,采用负载均衡模式时,采用啥负载均衡策略?

论坛徽章:
0
发表于 2014-09-02 09:32 |显示全部楼层
余大,我们现在考虑做FastDFS的异地IDC容灾,使用的是trunk文件,目前有两种方案,请帮忙回答一下下面的问题。
1. 一种是写程序定期同步新增的trunk文件和dat文件,但如果trunk文件中产生了小文件的删除和新增,则需要重传trunk块。问下目前有没有配置可以使得旧trunk块中的小文件被删除后,新的文件只在新的trunk中存,不在旧的中存?
2. 另一种是把异地的storage加到同一个组里,依靠FastDFS自身的binlong进行同步。为了使得异地storage完全不参与上传,是否可以利用upload_priority参数实现?另外,如果想实现主机房故障以后的failover,是故障后在异地机房新起一个tracker好还是初期部署的时候就在异地部署一个同等的tracker好?
您需要登录后才可以回帖 登录 | 注册

本版积分规则 发表回复

  

北京盛拓优讯信息技术有限公司. 版权所有 京ICP备16024965号-6 北京市公安局海淀分局网监中心备案编号:11010802020122 niuxiaotong@pcpop.com 17352615567
未成年举报专区
中国互联网协会会员  联系我们:huangweiwei@itpub.net
感谢所有关心和支持过ChinaUnix的朋友们 转载本站内容请注明原作者名及出处

清除 Cookies - ChinaUnix - Archiver - WAP - TOP