免费注册 查看新帖 |

Chinaunix

  平台 论坛 博客 文库
最近访问板块 发新帖
查看: 125257 | 回复: 246
打印 上一主题 下一主题

浩存 - 面向数据库,虚拟机等海量数据可同时提供NFS/iSCSI访问的集群存储系统 [复制链接]

论坛徽章:
0
跳转到指定楼层
1 [收藏(0)] [报告]
发表于 2005-05-13 14:41 |只看该作者 |倒序浏览
本帖最后由 yftty 于 2012-02-13 18:17 编辑

:Google是当前最有影响的Web搜索引擎,它利用一万多台廉价PC机构造了一个高性能、超大存储容量、稳定、实用的巨型Linux集群。
http://bbs.chinaunix.net/forum/v ... 9&show_type=old

其分布式分布式文件系统的实现方法,用低成本实现了高可用、高性能集群的方法是并行机设计、开发的一个成功典范,这种严格追求性价比的设计方法值得借鉴。

请大家参与到这一工作中来

发件人:         Eric Anderson
收件人:         FreeBSD Clustering List
主题:         FreeBSD Clustering wishlist - Was: Introduction & RE: Clustering with Freebsd
日期:         Wed, 11 May 2005 22:45:55 -0500  (星期四,11:45 CST)
邮件程序:         Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.7) Gecko/20050504


Ok - I'm changing the subject here in an attempt to gather information.

Here's my wishlist:

FreeBSD have a 'native' clustered filesystem.  This is different than
shared media (we already can do that over fiber channel, ggated, soon
iscsi and AOE).  This would allow multiple servers to access the same
data read/write - highly important for load balancing applications like
web servers, mail servers, and NFS servers.

Online growable filesystem.  I know I can growfs a filesystem now, but
doing online while data is being used is *insanely* useful.  Reiserfs
and Polyserve's FS (a clustered filesystem, not open-source) do this well.

FreeBSD's UFS2 made to do journaling.  There's already someone working
on this.

I believe the above mean that we need a distributed lock manager too, so
might as well add that to my wishlist.

Single filesystem limits set very high - 16TB would be a good minimum.

Vinum/geom (?) made to allow added a couple more 'disks' - be it a real
scsi device, or another vinum device - to existing vinum's, so I can
extend my vinum stripe, raid, concat, etc to a larger volume size,
without worrying about which disk is where.  I want to stripe mirrors of
raids, and raid striped mirrors of stripes.  I know it sounds crazy, but
I really *do* have uses for all this.

We currently pay lots of money every year (enough to pay an engineers
salary) for support and maintenance with Polyserve.  They make a good
product (we need it for the clustered filesystem and NFS distributed
lock manager stuff) - I'd much rather see that go to FreeBSD.

Eric         

[ 本帖最后由 yftty 于 2008-4-12 13:03 编辑 ]

论坛徽章:
0
2 [报告]
发表于 2005-05-13 14:46 |只看该作者

Unix下针对邮件,搜索,网络硬盘等海量存储的分布式文件系统项目

在 2005-05-11三的 22:45 -0500,Eric Anderson写道:
>; Ok - I'm changing the subject here in an attempt to gather information.
>;
>; Here's my wishlist:
>;
>; FreeBSD have a 'native' clustered filesystem.  This is different than
>; shared media (we already can do that over fiber channel, ggated, soon

Yes, the clustered filesystem will not run on SAN, since that will give
a high cost.

>; iscsi and AOE).  This would allow multiple servers to access the same
>; data read/write - highly important for load balancing applications like
>; web servers, mail servers, and NFS servers.

http://www.netapp.com/tech_library/3022.html <-- this article give some
info about the small file operations among the web, mail, IM, netdisk,
blog, etc. service. and that's our DFS targets at

>;
>; Online growable filesystem.  I know I can growfs a filesystem now, but
>; doing online while data is being used is *insanely* useful.  Reiserfs
>; and Polyserve's FS (a clustered filesystem, not open-source) do this well.

Yes, we also support that with our insanely mechanism.

And you know in the current clustered fs, as GoogleFS, Lustre, etc.
which can be built on online growfs. That's also our way to do it.

>;
>; FreeBSD's UFS2 made to do journaling.  There's already someone working
>; on this.

Good news.

>;
>; I believe the above mean that we need a distributed lock manager too, so
>; might as well add that to my wishlist.

By the specific application & services, we can easily remove the
distributed lock manager easily with upper layer way. You can read the
GoogleFS paper to get some further info.

>;
>; Single filesystem limits set very high - 16TB would be a good minimum.

The limits can be removed.

>;
>; Vinum/geom (?) made to allow added a couple more 'disks' - be it a real
>; scsi device, or another vinum device - to existing vinum's, so I can
>; extend my vinum stripe, raid, concat, etc to a larger volume size,
>; without worrying about which disk is where.  I want to stripe mirrors of
>; raids, and raid striped mirrors of stripes.  I know it sounds crazy, but
>; I really *do* have uses for all this.

Yes, that's Lustre's way, and we also add a logical disk layer to
support it.

>;
>; We currently pay lots of money every year (enough to pay an engineers
>; salary) for support and maintenance with Polyserve.  They make a good

Would you like to persuade your company to sponse the developing

>;  
>; product (we need it for the clustered filesystem and NFS distributed
>; lock manager stuff) - I'd much rather see that go to FreeBSD.

At last, any help & donate & contribute among the requirements & tech.
domains are great appreciated !

>;
>; Eric
>;
>;
>;
--
yf-263
Unix-driver.org

论坛徽章:
0
3 [报告]
发表于 2005-05-13 15:00 |只看该作者

Unix下针对邮件,搜索,网络硬盘等海量存储的分布式文件系统项目

yftty愿意为BSD做贡献啦。。。哈哈。
而且还有钱拿。。。。

论坛徽章:
0
4 [报告]
发表于 2005-05-13 15:05 |只看该作者

Unix下针对邮件,搜索,网络硬盘等海量存储的分布式文件系统项目

我也想写fs!

论坛徽章:
1
数据库技术版块每日发帖之星
日期:2016-04-10 06:20:00
5 [报告]
发表于 2005-05-13 15:19 |只看该作者

Unix下针对邮件,搜索,网络硬盘等海量存储的分布式文件系统项目

largeness   project

论坛徽章:
0
6 [报告]
发表于 2005-05-13 16:33 |只看该作者

Unix下针对邮件,搜索,网络硬盘等海量存储的分布式文件系统项目

though i can not understand it completely, i think it's a good idea.

论坛徽章:
0
7 [报告]
发表于 2005-05-14 11:22 |只看该作者

Unix下针对邮件,搜索,网络硬盘等海量存储的分布式文件系统项目

need to learn more advanced tech for understanding this artical

论坛徽章:
0
8 [报告]
发表于 2005-05-15 21:54 |只看该作者

Unix下针对邮件,搜索,网络硬盘等海量存储的分布式文件系统项目

在 2005-05-11三的 22:45 -0500,Eric Anderson写道:
>; Ok - I'm changing the subject here in an attempt to gather information.
>;
>; Here's my wishlist:

As for your wishlists, how about the MogileFS of
http://www.danga.com/mogilefs/

And what do you think about our GoogleFS like && MogileFS features
Clustre FS ?

Any comments are quite welcomed

论坛徽章:
0
9 [报告]
发表于 2005-05-16 13:11 |只看该作者

Unix下针对邮件,搜索,网络硬盘等海量存储的分布式文件系统项目

看样子大家对英文不是很感冒,给点我们头人的中文的乐呵乐呵

我一直在构想一个基于类SMPP协议的公开协议的分布式网络存储系统。大家可以看到google发布过一个google
fs的白皮书。实质上就是一个将fs的操作变为网络协议的操作的做法。最近手头上在帮助一个朋友完成了一个相关系统的设计的考虑。不知各位是否有兴趣一起来完成这样的一个项目,并且将它一直维护下去,也许将来它不止是一个python的实现,还会有c、java的实现。但是我相信python的实现会是最好的,就像现在的bt一样。
这样的分布式网络存储的用处会非常的多,如现在大家常在使用的大容量网络硬盘、gmail这样的大容量邮件系统、NNTP这样的大容量信息交互系统、Blog这样的大容量信息存储系统。
它的特点在于存储的内容多样化、存储的数据不能集中化、存储的数据会以用户/组/系统等为中心进行存储。
相关内容大家可以看看google fs。如果找不到我可以提供相关的pdf白皮书。
另:项目会开源(GPL或BSD)、项目会有实质的用所来证明我们的想法的正确性(我来解决测试环境的问题)。
----HD

论坛徽章:
0
10 [报告]
发表于 2005-05-16 13:47 |只看该作者

Unix下针对邮件,搜索,网络硬盘等海量存储的分布式文件系统项目

为何要基于类SMPP协议,不要基于bt
您需要登录后才可以回帖 登录 | 注册

本版积分规则 发表回复

  

北京盛拓优讯信息技术有限公司. 版权所有 京ICP备16024965号-6 北京市公安局海淀分局网监中心备案编号:11010802020122 niuxiaotong@pcpop.com 17352615567
未成年举报专区
中国互联网协会会员  联系我们:huangweiwei@itpub.net
感谢所有关心和支持过ChinaUnix的朋友们 转载本站内容请注明原作者名及出处

清除 Cookies - ChinaUnix - Archiver - WAP - TOP