免费注册 查看新帖 |

Chinaunix

  平台 论坛 博客 文库
最近访问板块 发新帖
查看: 3553 | 回复: 0
打印 上一主题 下一主题

[FastDFS] Java客户端上传问题 [复制链接]

论坛徽章:
0
跳转到指定楼层
1 [收藏(0)] [报告]
发表于 2013-07-03 14:21 |只看该作者 |倒序浏览
本帖最后由 yasenagat 于 2013-07-03 14:45 编辑

版本4.06
一:155,160两台tracker
二:160启动,选举155为leader
[2013-07-03 14:15:15] INFO - FastDFS v4.06, base_path=/home/fdfs/tracker, run_by_group=, run_by_user=, connect_timeout=60s, network_timeout=60s, port=22122, bind_addr=, max_connections=1800, work_threads=500, store_lookup=1, store_group=group1, store_server=0, store_path=0, reserved_storage_space=2048 MB, download_server=0, allow_ip_count=-1, sync_log_buff_interval=10s, check_active_interval=120s, thread_stack_size=64 KB, storage_ip_changed_auto_adjust=1, storage_sync_file_max_delay=86400s, storage_sync_file_max_time=300s, use_trunk_file=0, slot_min_size=256, slot_max_size=16 MB, trunk_file_size=64 MB, trunk_create_file_advance=0, trunk_create_file_time_base=02:00, trunk_create_file_interval=86400, trunk_create_file_space_threshold=20 GB, trunk_init_check_occupying=0, trunk_init_reload_from_binlog=0, use_storage_id=0, id_type_in_filename=ip, storage_id_count=0, rotate_error_log=0, error_log_rotate_time=00:00, rotate_error_log_size=0, store_slave_file_use_link=0, use_connection_pool=1, g_connection_pool_max_idle_time=3600s
[2013-07-03 14:15:15] INFO - local_host_ip_count: 2,  127.0.0.1  192.168.20.160
[2013-07-03 14:15:31] INFO - file: tracker_relationship.c, line: 374, selecting leader...
[2013-07-03 14:15:32] INFO - file: tracker_relationship.c, line: 413, the tracker leader 192.168.20.155:22122

三:两个线程同时上传文件
每个线程
上传文件: 67753个
上传时间:7698秒!

上传代码和testClient1中的上传方法,几乎一模一样!不过配置文件中2个tracker而已!也配置过1个tracker,同样的问题!

PS:启动一个线程的时候,无错误!!!!

四:错误如下

刚开始上传的时候,很正常,大概有个2,3分钟,然后就开始报错!
这个错误不是一直出现,因为上传很多文件,上传的时候,有一定频率的出现这个错误,但是还有很多,都上传成功了!举例说就是上传1W文件的时候,错误了几十个!后来我就停了程序!

connect to server 192.168.20.155:22122 fail
java.net.BindException: Address already in use: connect
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
        at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
        at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
        at java.net.Socket.connect(Socket.java:519)
        at org.csource.fastdfs.TrackerGroup.getConnection(TrackerGroup.java:47)
        at org.csource.fastdfs.TrackerGroup.getConnection(TrackerGroup.java:72)
        at org.csource.fastdfs.TrackerClient.getConnection(TrackerClient.java:5
        at org.csource.fastdfs.TrackerClient.getStoreStorage(TrackerClient.java:91)
        at org.csource.fastdfs.StorageClient.newWritableStorageConnection(StorageClient.java:193
        at org.csource.fastdfs.StorageClient.do_upload_file(StorageClient.java:703)
        at org.csource.fastdfs.StorageClient.upload_file(StorageClient.java:164)
        at org.csource.fastdfs.StorageClient.upload_file(StorageClient.java:132)
        at org.csource.fastdfs.StorageClient1.upload_file1(StorageClient1.java:90)
        at com.yasenagat.test.FastDFSUtil.upload1(FastDFSUtil.java:57)
        at com.yasenagat.test.BigTest.testUpload(BigTest.java:65)
        at com.yasenagat.test.Runner.run(Runner.java:14)
        at java.lang.Thread.run(Thread.java:619)


connect to server 192.168.20.160:22122 fail
java.net.BindException: Address already in use: connect
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
        at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
        at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
        at java.net.Socket.connect(Socket.java:519)
        at org.csource.fastdfs.TrackerGroup.getConnection(TrackerGroup.java:47)
        at org.csource.fastdfs.TrackerGroup.getConnection(TrackerGroup.java:72)
        at org.csource.fastdfs.TrackerClient.getConnection(TrackerClient.java:5
        at org.csource.fastdfs.TrackerClient.getStoreStorage(TrackerClient.java:91)
        at org.csource.fastdfs.StorageClient.newWritableStorageConnection(StorageClient.java:193
        at org.csource.fastdfs.StorageClient.do_upload_file(StorageClient.java:703)
        at org.csource.fastdfs.StorageClient.upload_file(StorageClient.java:164)
        at org.csource.fastdfs.StorageClient.upload_file(StorageClient.java:132)
        at org.csource.fastdfs.StorageClient1.upload_file1(StorageClient1.java:90)
        at com.yasenagat.test.FastDFSUtil.upload1(FastDFSUtil.java:57)
        at com.yasenagat.test.BigTest.testUpload(BigTest.java:65)
        at com.yasenagat.test.Runner.run(Runner.java:14)
        at java.lang.Thread.run(Thread.java:619)

五:请问如何破

# max concurrent connections this server supported
max_connections=1800

max_connections=1800, work_threads=500,这两个参数我改了N次,也不管用!难道是我机器不行?还是说并发太高了?我觉得是代码有问题!求大神看看!

报错过程中tracker的日志,storage的日志木有任何错误!!!

单线程上传,1个tracker,OK!
2个线程上传,1个tracker,错误!



core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 14962
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 8196
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 8196
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

您需要登录后才可以回帖 登录 | 注册

本版积分规则 发表回复

  

北京盛拓优讯信息技术有限公司. 版权所有 京ICP备16024965号-6 北京市公安局海淀分局网监中心备案编号:11010802020122 niuxiaotong@pcpop.com 17352615567
未成年举报专区
中国互联网协会会员  联系我们:huangweiwei@itpub.net
感谢所有关心和支持过ChinaUnix的朋友们 转载本站内容请注明原作者名及出处

清除 Cookies - ChinaUnix - Archiver - WAP - TOP