可恶的 发表于 2014-05-15 18:09

FastDFS的storager server如何手动同步?

请教一下,storager server如何手动同步呢?谢谢!
我用的FastDFS_v4.06,一共两个group,奇怪的是storager server无法同步之前的数据。
期间更换过一台tracker server和一台storager server。
在更换tracker server之前,我测试过,可以同步历史数据到新增storager的服务器上面,但是现在无法同步上历史数据,只能同步新增的数据。

以下是fdfs_monitor所获取的信息,last_source_update、last_sync_update和last_synced_timestamp的值都为1970-01-01 08:00:00。 DEBUG - base_path=/usr/local/fastdfs, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0

server_count=1, server_index=0

tracker server is 172.16.0.60:22122

group count: 2

Group 1:
group name = group1
disk total space = 4031 MB
disk free space = 2452 MB
trunk free space = 0 MB
storage server count = 2
active server count = 2
storage server port = 23000
storage HTTP port = 8090
store path count = 1
subdir count per path = 256
current write server index = 0
current trunk file id = 0

        Storage 1:
                id = 172.16.0.56
                ip_addr = 172.16.0.56ACTIVE
                http domain =
                version = 4.06
                join time = 2014-05-15 09:48:41
                up time = 2014-05-15 09:48:41
                total storage = 4031 MB
                free storage = 2452 MB
                upload priority = 10
                store_path_count = 1
                subdir_count_per_path = 256
                storage_port = 23000
                storage_http_port = 8090
                current_write_path = 0
                source storage id= 172.16.0.59
                if_trunk_server= 0
                total_upload_count = 0
                success_upload_count = 0
                total_append_count = 0
                success_append_count = 0
                total_modify_count = 0
                success_modify_count = 0
                total_truncate_count = 0
                success_truncate_count = 0
                total_set_meta_count = 0
                success_set_meta_count = 0
                total_delete_count = 0
                success_delete_count = 0
                total_download_count = 0
                success_download_count = 0
                total_get_meta_count = 0
                success_get_meta_count = 0
                total_create_link_count = 0
                success_create_link_count = 0
                total_delete_link_count = 0
                success_delete_link_count = 0
                total_upload_bytes = 0
                success_upload_bytes = 0
                total_append_bytes = 0
                success_append_bytes = 0
                total_modify_bytes = 0
                success_modify_bytes = 0
                stotal_download_bytes = 0
                success_download_bytes = 0
                total_sync_in_bytes = 0
                success_sync_in_bytes = 0
                total_sync_out_bytes = 0
                success_sync_out_bytes = 0
                total_file_open_count = 0
                success_file_open_count = 0
                total_file_read_count = 0
                success_file_read_count = 0
                total_file_write_count = 0
                success_file_write_count = 0
                last_heart_beat_time = 2014-05-15 10:01:10
                last_source_update = 1970-01-01 08:00:00
                last_sync_update = 1970-01-01 08:00:00
                last_synced_timestamp = 1970-01-01 08:00:00
        Storage 2:
                id = 172.16.0.59
                ip_addr = 172.16.0.59ACTIVE
                http domain =
                version = 4.06
                join time = 2014-05-15 09:48:28
                up time = 2014-05-15 09:48:28
                total storage = 4031 MB
                free storage = 2489 MB
                upload priority = 10
                store_path_count = 1
                subdir_count_per_path = 256
                storage_port = 23000
                storage_http_port = 8090
                current_write_path = 0
                source storage id=
                if_trunk_server= 0
                total_upload_count = 0
                success_upload_count = 0
                total_append_count = 0
                success_append_count = 0
                total_modify_count = 0
                success_modify_count = 0
                total_truncate_count = 0
                success_truncate_count = 0
                total_set_meta_count = 0
                success_set_meta_count = 0
                total_delete_count = 0
                success_delete_count = 0
                total_download_count = 0
                success_download_count = 0
                total_get_meta_count = 0
                success_get_meta_count = 0
                total_create_link_count = 0
                success_create_link_count = 0
                total_delete_link_count = 0
                success_delete_link_count = 0
                total_upload_bytes = 0
                success_upload_bytes = 0
                total_append_bytes = 0
                success_append_bytes = 0
                total_modify_bytes = 0
                success_modify_bytes = 0
                stotal_download_bytes = 0
                success_download_bytes = 0
                total_sync_in_bytes = 0
                success_sync_in_bytes = 0
                total_sync_out_bytes = 0
                success_sync_out_bytes = 0
                total_file_open_count = 0
                success_file_open_count = 0
                total_file_read_count = 0
                success_file_read_count = 0
                total_file_write_count = 0
                success_file_write_count = 0
                last_heart_beat_time = 2014-05-15 10:01:27
                last_source_update = 1970-01-01 08:00:00
                last_sync_update = 1970-01-01 08:00:00
                last_synced_timestamp = 1970-01-01 08:00:00

Group 2:
group name = group2
disk total space = 4031 MB
disk free space = 2487 MB
trunk free space = 0 MB
storage server count = 2
active server count = 2
storage server port = 23000
storage HTTP port = 8090
store path count = 1
subdir count per path = 256
current write server index = 0
current trunk file id = 0

        Storage 1:
                id = 172.16.0.54
                ip_addr = 172.16.0.54ACTIVE
                http domain =
                version = 4.06
                join time = 2014-05-15 09:49:22
                up time = 2014-05-15 09:49:22
                total storage = 4031 MB
                free storage = 2487 MB
                upload priority = 10
                store_path_count = 1
                subdir_count_per_path = 256
                storage_port = 23000
                storage_http_port = 8090
                current_write_path = 0
                source storage id= 172.16.0.55
                if_trunk_server= 0
                total_upload_count = 0
                success_upload_count = 0
                total_append_count = 0
                success_append_count = 0
                total_modify_count = 0
                success_modify_count = 0
                total_truncate_count = 0
                success_truncate_count = 0
                total_set_meta_count = 0
                success_set_meta_count = 0
                total_delete_count = 0
                success_delete_count = 0
                total_download_count = 0
                success_download_count = 0
                total_get_meta_count = 0
                success_get_meta_count = 0
                total_create_link_count = 0
                success_create_link_count = 0
                total_delete_link_count = 0
                success_delete_link_count = 0
                total_upload_bytes = 0
                success_upload_bytes = 0
                total_append_bytes = 0
                success_append_bytes = 0
                total_modify_bytes = 0
                success_modify_bytes = 0
                stotal_download_bytes = 0
                success_download_bytes = 0
                total_sync_in_bytes = 0
                success_sync_in_bytes = 0
                total_sync_out_bytes = 0
                success_sync_out_bytes = 0
                total_file_open_count = 0
                success_file_open_count = 0
                total_file_read_count = 0
                success_file_read_count = 0
                total_file_write_count = 0
                success_file_write_count = 0
                last_heart_beat_time = 2014-05-15 10:01:20
                last_source_update = 1970-01-01 08:00:00
                last_sync_update = 1970-01-01 08:00:00
                last_synced_timestamp = 1970-01-01 08:00:00
        Storage 2:
                id = 172.16.0.55
                ip_addr = 172.16.0.55ACTIVE
                http domain =
                version = 4.06
                join time = 2014-05-15 09:49:04
                up time = 2014-05-15 09:49:04
                total storage = 4031 MB
                free storage = 2510 MB
                upload priority = 10
                store_path_count = 1
                subdir_count_per_path = 256
                storage_port = 23000
                storage_http_port = 8090
                current_write_path = 0
                source storage id=
                if_trunk_server= 0
                total_upload_count = 0
                success_upload_count = 0
                total_append_count = 0
                success_append_count = 0
                total_modify_count = 0
                success_modify_count = 0
                total_truncate_count = 0
                success_truncate_count = 0
                total_set_meta_count = 0
                success_set_meta_count = 0
                total_delete_count = 0
                success_delete_count = 0
                total_download_count = 0
                success_download_count = 0
                total_get_meta_count = 0
                success_get_meta_count = 0
                total_create_link_count = 0
                success_create_link_count = 0
                total_delete_link_count = 0
                success_delete_link_count = 0
                total_upload_bytes = 0
                success_upload_bytes = 0
                total_append_bytes = 0
                success_append_bytes = 0
                total_modify_bytes = 0
                success_modify_bytes = 0
                stotal_download_bytes = 0
                success_download_bytes = 0
                total_sync_in_bytes = 0
                success_sync_in_bytes = 0
                total_sync_out_bytes = 0
                success_sync_out_bytes = 0
                total_file_open_count = 0
                success_file_open_count = 0
                total_file_read_count = 0
                success_file_read_count = 0
                total_file_write_count = 0
                success_file_write_count = 0
                last_heart_beat_time = 2014-05-15 10:01:32
                last_source_update = 1970-01-01 08:00:00
                last_sync_update = 1970-01-01 08:00:00
                last_synced_timestamp = 1970-01-01 08:00:00group1中的一台storager server日志如下。 INFO - FastDFS v4.06, base_path=/usr/local/fastdfs, store_path_count=1, subdir_count_per_path=256, group_name=group1, run_by_group=, run_by_user=, connect_timeout=30s, network_timeout=60s, port=23000, bind_addr=, client_bind=1, max_connections=256, work_threads=4, disk_rw_separated=1, disk_reader_threads=1, disk_writer_threads=1, buff_size=256KB, heart_beat_interval=30s, stat_report_interval=60s, tracker_server_count=1, sync_wait_msec=50ms, sync_interval=0ms, sync_start_time=00:00, sync_end_time=23:59, write_mark_file_freq=500, allow_ip_count=-1, file_distribute_path_mode=0, file_distribute_rotate_count=100, fsync_after_written_bytes=0, sync_log_buff_interval=10s, sync_binlog_buff_interval=10s, sync_stat_file_interval=300s, thread_stack_size=512 KB, upload_priority=10, if_alias_prefix=, check_file_duplicate=0, file_signature_method=hash, FDHT group count=0, FDHT server count=0, FDHT key_namespace=, FDHT keep_alive=0, HTTP server port=8090, domain name=, use_access_log=0, rotate_access_log=0, access_log_rotate_time=00:00, rotate_error_log=0, error_log_rotate_time=00:00, rotate_access_log_size=0, rotate_error_log_size=0, file_sync_skip_invalid_record=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s
INFO - file: storage_param_getter.c, line: 187, use_storage_id=0, id_type_in_filename=ip, storage_ip_changed_auto_adjust=1, store_path=0, reserved_storage_space=10.00%, use_trunk_file=0, slot_min_size=256, slot_max_size=16 MB, trunk_file_size=64 MB, trunk_create_file_advance=0, trunk_create_file_time_base=02:00, trunk_create_file_interval=86400, trunk_create_file_space_threshold=20 GB, trunk_init_check_occupying=0, trunk_init_reload_from_binlog=0, store_slave_file_use_link=0
INFO - file: storage_func.c, line: 174, tracker_client_ip: 172.16.0.59, my_server_id_str: 172.16.0.59, g_server_id_in_filename: 989860012
INFO - local_host_ip_count: 2,127.0.0.1172.16.0.59
INFO - file: tracker_client_thread.c, line: 308, successfully connect to tracker server 172.16.0.60:22122, as a tracker client, my ip is 172.16.0.59
INFO - file: tracker_client_thread.c, line: 1124, tracker server 172.16.0.60:22122, set tracker leader: 172.16.0.60:22122
INFO - file: storage_sync.c, line: 2698, successfully connect to storage server 172.16.0.56:23000
INFO - file: storage_sync.c, line: 2698, successfully connect to storage server 172.16.0.56:23000tracker server的日志也一并贴上来。 INFO - FastDFS v4.06, base_path=/usr/local/fastdfs, run_by_group=, run_by_user=, connect_timeout=30s, network_timeout=60s, port=22122, bind_addr=, max_connections=256, work_threads=4, store_lookup=2, store_group=, store_server=0, store_path=0, reserved_storage_space=10.00%, download_server=0, allow_ip_count=-1, sync_log_buff_interval=10s, check_active_interval=120s, thread_stack_size=64 KB, storage_ip_changed_auto_adjust=1, storage_sync_file_max_delay=86400s, storage_sync_file_max_time=300s, use_trunk_file=0, slot_min_size=256, slot_max_size=16 MB, trunk_file_size=64 MB, trunk_create_file_advance=0, trunk_create_file_time_base=02:00, trunk_create_file_interval=86400, trunk_create_file_space_threshold=20 GB, trunk_init_check_occupying=0, trunk_init_reload_from_binlog=0, use_storage_id=0, id_type_in_filename=ip, storage_id_count=0, rotate_error_log=0, error_log_rotate_time=00:00, rotate_error_log_size=0, store_slave_file_use_link=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s
INFO - local_host_ip_count: 2,127.0.0.1172.16.0.60
INFO - file: tracker_relationship.c, line: 374, selecting leader...
INFO - file: tracker_relationship.c, line: 392, I am the new tracker leader 172.16.0.60:22122

fxysun 发表于 2014-05-19 16:36

我也碰到这个问题, 请问happy fish, 这个怎么解决啊?

fxysun 发表于 2014-05-20 10:54

解决方法是: 同组里的storage都重启一下, 如果还不行, tracker也都重启一下

可恶的 发表于 2014-05-21 11:52

回复 3# fxysun

感谢您的回复,您说的方法,我之前都试过了,依旧无法同步历史数据。
   

可恶的 发表于 2014-05-21 11:54

@happy_fish100
麻烦您有空的话,告知一下接近方法,谢谢。

happy_fish100 发表于 2014-05-22 16:01

确认文件没有同步?

可恶的 发表于 2014-05-23 11:10

happy_fish100 发表于 2014-05-22 16:01 static/image/common/back.gif
确认文件没有同步?

是的呀。
历史文件没有自动同步,而且很奇怪的是除了last_heart_beat_time是系统时间,其他三个选项都是1970.01.01.的时间戳。
请问能否实现手头同步历史文件呢?谢谢!                last_source_update = 1970-01-01 08:00:00
                last_sync_update = 1970-01-01 08:00:00
                last_synced_timestamp = 1970-01-01 08:00:00

happy_fish100 发表于 2014-05-23 17:39

你后来上传过文件么?

可恶的 发表于 2014-05-23 17:54

happy_fish100 发表于 2014-05-23 17:39 static/image/common/back.gif
你后来上传过文件么?

后来上传过文件,也可以同步后来上传的文件,但是历史文件无法同步。
场景的话,在帖子开头有说明:期间更换过一台tracker server和一台storager server。

shaneyuan 发表于 2014-05-29 18:30

重点在你是怎么换的,还有是否影响了binlog的完整性。回复 9# 可恶的


   
页: [1] 2
查看完整版本: FastDFS的storager server如何手动同步?