【求助】java客户端并发访问报错。

sun_wanx 发表于 2016-05-06 15:17

环境：
服务器FastDFS版本5.0.7
java客户端版本1.24
window7 64位、ubuntu、centos

部署结构：
2台trackage，2台storage

问题：
并发访问单个或者多个文件，少数可以下载成功，大部分报如下多种错误：
1.java.io.IOException: Socket Closed
at java.net.AbstractPlainSocketImpl.getOutputStream(AbstractPlainSocketImpl.java:429)
at java.net.PlainSocketImpl.getOutputStream(PlainSocketImpl.java:232)
at java.net.Socket$3.run(Socket.java:927)
at java.net.Socket$3.run(Socket.java:925)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.Socket.getOutputStream(Socket.java:924)
at org.csource.fastdfs.ProtoCommon.closeSocket(ProtoCommon.java:350)
at org.csource.fastdfs.TrackerServer.close(TrackerServer.java:76)
at org.csource.fastdfs.StorageClient.download_file(StorageClient.java:1318)
at org.csource.fastdfs.StorageClient.download_file(StorageClient.java:1280)
=========================================================================
2.java.io.IOException: recv cmd: 64 is not correct, expect cmd: 100
at org.csource.fastdfs.ProtoCommon.recvHeader(ProtoCommon.java:219) ~
at org.csource.fastdfs.ProtoCommon.recvPackage(ProtoCommon.java:250) ~
at org.csource.fastdfs.StorageClient.download_file(StorageClient.java:1301) ~
at org.csource.fastdfs.StorageClient.download_file(StorageClient.java:1280) ~
at com.github.mtdp.dfs.server.service.impl.FastDFSFileSystemServiceImpl.download(FastDFSFileSystemServiceImpl.java:106) ~
at sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source) ~
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~
at java.lang.reflect.Method.invoke(Method.java:606) ~
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at com.alibaba.druid.support.spring.stat.DruidStatInterceptor.invoke(DruidStatInterceptor.java:72)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
at com.sun.proxy.$Proxy31.download(Unknown Source)
==================================================================================================================

sun_wanx 发表于 2016-05-06 15:21

补充服务端没有报错

yxs1112003 发表于 2016-05-11 08:17

同问
centos7.1，使用5.05，1台tracker，2台storage，javaclient低并发上传，有几率报timeout错误

查询日志，在tracker的server中报
DEBUG - file: tracker_nio.c, line: 226, client ip: 172.27.12.86, recv error event: 25, close connection
DEBUG - file: tracker_nio.c, line: 226, client ip: 172.27.12.86, recv error event: 25, close connection
DEBUG - file: tracker_nio.c, line: 226, client ip: 172.27.12.86, recv error event: 25, close connection

看到以前的帖子说升级版本，不是很确定升级是否能解决问题

yxs1112003 发表于 2016-05-11 13:53

yxs1112003 发表于 2016-05-11 08:17 static/image/common/back.gif
同问
centos7.1，使用5.05，1台tracker，2台storage，javaclient低并发上传，有几率报timeout错误

问题解决了，升级到5.08，不在出现timeout异常了。{:qq23:}

sun_wanx 发表于 2016-05-12 15:31

回复 4# yxs1112003

感谢分享，
如果在上传/下载文件时添加syc锁,就不会报错，我的问题应该不是服务端问题。
我的代码是：

yxs1112003 发表于 2016-05-31 19:08

本帖最后由 yxs1112003 于 2016-05-31 19:11 编辑

这么传好像特别慢啊,我是上传的时候每次都获取一个新的storageClient，我是这样多线程上传的，不过这样并发数大一些就报错了。。。在网上看到说需要写线程池。。。
/**
* 每次执行上传下载动作时需要获取一个新的StorageClient，因为旧的StorageClient会被关闭
*
* @return 返回一个新的StorageClient1实例
*/
StorageClient getStorageClient() {
   return new StorageClient1();
}
/ **
* 上传小文件
*
* @param nvp 文件数据元
* @return Meta类型的数据元，保存了保存组和上传路径
*/
public Meta upload(NameValuePair[] nvp) {

   String[] fileIds;
   Meta meta = null;
   StorageClient client;
   try {
         // nameValuePair是一个记录上传信息的类，可以存储name，value字符串
         String local_filename = configManager.getLocal_filename();
         client = getStorageClient();
         fileIds = client.upload_file(local_filename, null, nvp);
         meta = new Meta(fileIds, fileIds);

         // 将上传的文件路径记录到文件
         recorder.recordUploadedFileList(meta);
         logger.info("组名：" + fileIds + " 路径: " + fileIds + "\t");

   } catch (Exception e) {
         logger.error(null, e);
   }
   return meta;
}

yxs1112003 发表于 2016-06-01 10:43

哪位大大帮着看看可好？

yxs1112003 发表于 2016-06-01 11:42

本帖最后由 yxs1112003 于 2016-06-01 11:47 编辑

yxs1112003 发表于 2016-05-31 19:08 static/image/common/back.gif
这么传好像特别慢啊,我是上传的时候每次都获取一个新的storageClient，我是这样多线程上传的，不过这样并发 ...
报的错误，异常解释如下http:// frenchmay.iteye.com/ blog/ 253123
2016-06-0111:39:41,554 pool-1-thread-49:com.common.impl.FDFSBaseDaoImpl.upload(87)
java.net.SocketException: Software caused connection abort: socket write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
at org.csource.fastdfs.UploadStream.send(UploadStream.java:55)
at org.csource.fastdfs.StorageClient.do_upload_file(StorageClient.java:972)
at org.csource.fastdfs.StorageClient.upload_file(StorageClient.java:192)
at org.csource.fastdfs.StorageClient.upload_file(StorageClient.java:153)
at org.csource.fastdfs.StorageClient.upload_file(StorageClient.java:126)
at com.common.impl.FDFSBaseDaoImpl.upload(FDFSBaseDaoImpl.java:80)
at com.common.impl.FDFSBaseDaoImpl.upload(FDFSBaseDaoImpl.java:63)
at com.service.impl.FDFSServiceImpl.upload(FDFSServiceImpl.java:20)
at com.multi.UploadThread.run(UploadThread.java:21)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2016-06-0111:39:41,554 pool-1-thread-285:com.common.impl.FDFSBaseDaoImpl.upload(87)
java.io.IOException: recv package size -1 != 10
at org.csource.fastdfs.ProtoCommon.recvHeader(ProtoCommon.java:214)
at org.csource.fastdfs.ProtoCommon.recvPackage(ProtoCommon.java:250)
at org.csource.fastdfs.StorageClient.do_upload_file(StorageClient.java:976)
at org.csource.fastdfs.StorageClient.upload_file(StorageClient.java:192)
at org.csource.fastdfs.StorageClient.upload_file(StorageClient.java:153)
at org.csource.fastdfs.StorageClient.upload_file(StorageClient.java:126)
at com.common.impl.FDFSBaseDaoImpl.upload(FDFSBaseDaoImpl.java:80)
at com.common.impl.FDFSBaseDaoImpl.upload(FDFSBaseDaoImpl.java:63)
at com.service.impl.FDFSServiceImpl.upload(FDFSServiceImpl.java:20)
at com.multi.UploadThread.run(UploadThread.java:21)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2016-06-0111:39:41,950 pool-1-thread-585:com.common.impl.FDFSBaseDaoImpl.upload(85) Meta
2016-06-0111:39:41,554 pool-1-thread-335:com.common.impl.FDFSBaseDaoImpl.upload(87)
java.net.SocketException: Software caused connection abort: socket write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
at org.csource.fastdfs.UploadStream.send(UploadStream.java:55)
at org.csource.fastdfs.StorageClient.do_upload_file(StorageClient.java:972)
at org.csource.fastdfs.StorageClient.upload_file(StorageClient.java:192)
at org.csource.fastdfs.StorageClient.upload_file(StorageClient.java:153)
at org.csource.fastdfs.StorageClient.upload_file(StorageClient.java:126)
at com.common.impl.FDFSBaseDaoImpl.upload(FDFSBaseDaoImpl.java:80)
at com.common.impl.FDFSBaseDaoImpl.upload(FDFSBaseDaoImpl.java:63)
at com.service.impl.FDFSServiceImpl.upload(FDFSServiceImpl.java:20)
at com.multi.UploadThread.run(UploadThread.java:21)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2016-06-0111:39:41,553 pool-1-thread-540:com.common.impl.FDFSBaseDaoImpl.upload(85) Meta
2016-06-0111:39:41,553 pool-1-thread-224:com.common.impl.FDFSBaseDaoImpl.upload(87)
java.net.SocketException: Software caused connection abort: socket write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
at org.csource.fastdfs.UploadStream.send(UploadStream.java:55)
at org.csource.fastdfs.StorageClient.do_upload_file(StorageClient.java:972)
at org.csource.fastdfs.StorageClient.upload_file(StorageClient.java:192)
at org.csource.fastdfs.StorageClient.upload_file(StorageClient.java:153)
at org.csource.fastdfs.StorageClient.upload_file(StorageClient.java:126)
at com.common.impl.FDFSBaseDaoImpl.upload(FDFSBaseDaoImpl.java:80)
at com.common.impl.FDFSBaseDaoImpl.upload(FDFSBaseDaoImpl.java:63)
at com.service.impl.FDFSServiceImpl.upload(FDFSServiceImpl.java:20)
at com.multi.UploadThread.run(UploadThread.java:21)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2016-06-0111:39:41,553 pool-1-thread-550:com.common.impl.FDFSBaseDaoImpl.upload(85) Meta
2016-06-0111:39:41,552 pool-1-thread-542:com.common.impl.FDFSBaseDaoImpl.upload(85) Meta
2016-06-0111:39:41,552 pool-1-thread-346:com.common.impl.FDFSBaseDaoImpl.upload(87)
java.io.IOException: recv package size -1 != 10

sun_wanx 发表于 2016-06-12 15:01

上传还行，不是很慢。回复 6# yxs1112003

chushiyun 发表于 2016-06-24 14:52

客户端上传时需要添加一个连接池，本人测试过了，可以解决多线程上传导致的问题。有需要的可以发邮件给我841439685@qq.com

页: [1]

Chinaunix's Archiver

【求助】java客户端并发访问报错。