浩存 - 面向数据库，虚拟机等海量数据可同时提供NFS/iSCSI访问的集群存储系统

nizvoo

ok, i know it. I need learn more FS knowledge. keep touch.my mail : nizvoo"AT"gmail.com.

deltali

what's the role of locks in a distributed filesystem？

thanks!

yftty

原帖由 "deltali" 发表：
what's the role of locks in a distributed filesystem？

thanks!

The locks in a distributed filesystem is managed by Distributed Lock Manager (DLM),

A distributed filesystem need to addressing the problem of delivering aggregate performance to a large number of clients.

DLM is the basis of scalable clusters. In a DLM based cluster all nodes can write to all shared resources and co-ordinate their action using the DLM.

This sort of technology is mainly intended for CPU and/or ram intensive processing, not for disc intensive operations nor for reliblity.

Digital >; Compaq >; HP...  HP own the Digital DLM technology, available in Tru64 Unix (was Digital Unix and OpenVMS 8.)

Compaq/HP licensed the DLM technology to Oracle who have base their cluster/grid software on the DLM

Sun Solaris also has a DLM based cluster technology.

Now Sun and HP are fighting blog wars...
http://blogs.zdnet.com/index.php?p=661&tag=nl.e539
http://www.chillingeffects.org/responses/notice.cgi?NoticeID=1460

Where I see DLM being good is for rendering and scientific calculation. These processes could really benifit from having a central data store but will not put a huge load on the DLM hardware..

Some more deeply knowledge:

http://kerneltrap.org/mailarchive/1/message/56956/thread

http://kerneltrap.org/mailarchive/1/message/66678/thread








http://lwn.net/Articles/135686/

Clusters and distributed lock management
The creation of tightly-connected clusters requires a great deal of supporting infrastructure. One of the necessary pieces is a lock manager - a system which can arbitrate access to resources which are shared across the cluster. The lock manager provides functions similar to those found in the locking calls on a single-user system - it can give a process read-only or write access to parts of files. The lock management task is complicated by the cluster environment, though; a lock manager must operate correctly regardless of network latencies, cope with the addition and removal of nodes, recover from the failure of nodes which hold locks, etc. It is a non-trivial problem, and Linux does not currently have a working, distributed lock manager in the mainline kernel.

David Teigland (of Red Hat) recently posted a set of distributed lock manager patches (called "dlm", with a request for inclusion into the mainline. This code, which was originally developed at Sistina, is said to be influenced primarily by the venerable VMS lock manager. An initial look at the code confirms this statement: callbacks are called "ASTs" (asynchronous system traps, in VMS-speak), and the core locking call is an eleven-parameter monster:

    int dlm_lock(dlm_lockspace_t *lockspace,
        int mode,
        struct dlm_lksb *lksb,
        uint32_t flags,
        void *name,
        unsigned int namelen,
        uint32_t parent_lkid,
        void (*lockast) (void *astarg),
        void *astarg,
        void (*bast) (void *astarg, int mode),
        struct dlm_range *range);

Most of the discussion has not been concerned with the technical issues, however. There are some disagreements over issues like how nodes should be identified, but most of the developers who are interested in this area seem to think that this implementation is at least a reasonable starting point. The harder issue is figuring out just how a general infrastructure for cluster support can be created for the Linux kernel. At least two other projects have their own distributed lock managers and are likely to want to be a part of this discussion; an Oracle developer recently described the posting of dlm as "a preemptive strike." Lock management is a function needed by most tightly-coupled clustering and clustered filesystem projects; wouldn't it be nice if they could all use the same implementation?

The fact is that the clustering community still needs to work these issues out; Andrew Morton doesn't want to have to make these decisions for them:

Not only do I not know whether this stuff should be merged: I don't even know how to find that out. Unless I'm prepared to become a full-on cluster/dlm person, which isn't looking likely.

The usual fallback is to identify all the stakeholders and get them to say "yes Andrew, this code is cool and we can use it", but I don't think the clustering teams have sufficent act-togetherness to be able to do that.

Clustering will be discussed at the kernel summit in July. A month prior to that, there will also be a clustering workshop held in Germany. In the hopes that these two events will help bring some clarity to this issue, Andrew has said that he will hold off on any decisions for now.

wolfg

关注

ufoor

看的有些晕了,还得多学
相关的东西还是先看中文的比较好些,效率高些.如果中文的没有再看英文的

Zer4tul

好像是HD想出的主意吧？不错啊……可惜我水平不够……就在一边加油好了……过两天仔细看看Google FS的文档。

yftty

原帖由 "ufoor" 发表：
看的有些晕了,还得多学
相关的东西还是先看中文的比较好些,效率高些.如果中文的没有再看英文的

看中文的有利于迅速建立相关的概念, 但几个概念建立起来之后, 就不要看中文的了, 否则会越看越糊涂.

yftty

[quote]原帖由 "Zer4tul"]好像是HD想出的主意吧？不错啊……可惜我水平不够……就在一边加油好了……过两天仔细看看Google FS的文档。[/quote 发表：


hehe, HD can be considerred the Godfather of the Project !

Also great project need great man. Do you want to let me know and merge your brilliant ideas as what to do or how to do.

Let's inspiring each to other !

akadoc

up，up，up。关注中。。。

yftty

[quote]原帖由 "akadoc"]up，up，up。关注中。。。[/quote 发表：


您想关注那一点或哪一部分呢,是组织还是技术呢,还是技术的哪一部分呢

请看与我们类似的MogileFS提供的Features.

http://www.danga.com/mogilefs/

MogileFS is our open source distributed filesystem. Its properties and features include:

    * Application level -- no special kernel modules required.
    * No single point of failure -- all three components of a MogileFS setup (storage nodes, trackers, and the tracker's database(s)) can be run on multiple machines, so there's no single point of failure. (you can run trackers on the same machines as storage nodes, too, so you don't need 4 machines...) A minimum of 2 machines is recommended.
    * Automatic file replication -- files, based on their "class", are automatically replicated between enough different storage nodes as to satisfy the minimum replica count as requested by their class. For instance, for a photo hosting site you can make original JPEGs have a minimum replica count of 3, but thumbnails and scaled versions only have a replica count of 1 or 2. If you lose the only copy of a thumbnail, the application can just rebuild it. In this way, MogileFS (without RAID) can save money on disks that would otherwise be storing multiple copies of data unnecessarily.
    * "Better than RAID" -- in a non-SAN RAID setup, the disks are redundant, but the host isn't. If you lose the entire machine, the files are inaccessible. MogileFS replicates the files between devices which are on different hosts, so files are always available.
    * Transport Neutral -- MogileFS clients can communicate with MogileFS storage nodes (after talking to a tracker) via either NFS or HTTP, but we strongly recommend HTTP.
    * Flat Namespace -- Files are identified by named keys in a flat, global namespace. You can create as many namespaces as you'd like, so multiple applications with potentially conflicting keys can run on the same MogileFS installation.
    * Shared-Nothing -- MogileFS doesn't depend on a pricey SAN with shared disks. Every machine maintains its own local disks.
    * No RAID required -- Local disks on MogileFS storage nodes can be in a RAID, or not. It's cheaper not to, as RAID doesn't buy you any safety that MogileFS doesn't already provide.
    * Local filesystem agnostic -- Local disks on MogileFS storage nodes can be formatted with your filesystem of choice (ext3, ReiserFS, etc..). MogileFS does its own internal directory hashing so it doesn't hit filesystem limits such as "max files per directory" or "max directories per directory". Use what you're comfortable with.

MogileFS is not:

    * POSIX Compliant -- you don't run regular Unix applications or databases against MogileFS. It's meant for archiving write-once files and doing only sequential reads. (though you can modify a file by way of overwriting it with a new version) Notes:
          o Yes, this means your application has to specifically use a MogileFS client library to store and retrieve files. The steps in general are 1) talk to a tracker about what you want to put or get, 2) read/write to the NFS path for that storage node (the tracker will tell you where) or do an HTTP GET/PUT to the storage node, if you're running with an HTTP transport instead of NFS (which is highly recommended)
          o We've briefly tinkered with using FUSE, which lets Linux filesystems be implemented in userspace, to provide a Linux filesystem interface to MogileFS, but we haven't worked on it much.
    * Completely portable ... yet -- we have some Linux-isms in our code, at least in the HTTP transport code. Our plan is to scrap that and make it portable, though.