Chinaunix

标题: RHEL4U7下的9TB mkfs.ext3分区问题 [打印本页]

作者: ldren 时间: 2009-05-31 22:19
标题: RHEL4U7下的9TB mkfs.ext3分区问题
硬件设备：DL 380G5一台  内装两块A8002A HBA卡（EMULEX芯片）  MSA2000FC AJ745
操作系统：RHEL4U7
问题如下：
(parted) mkfs
分区编号？ 1
文件系统？  [ext2]? ext3
未实现: 关于创建 ext3 文件系统的支持尚未实现。
(parted) print
/dev/mapper/mpath0 的磁盘几何结构：0.000-8584682.000 兆字节
磁盘标签类型：gpt
Minor 起始点       终止点  文件系统名称                标志
1       0.017 8584681.983
(parted) quit
[root@localhost mapper]# fdisk -l

Disk /dev/cciss/c0d0: 146.7 GB, 146778685440 bytes
255 heads, 63 sectors/track, 17844 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

         Device Boot    Start       End    Blocks Id  System
/dev/cciss/c0d0p1 *          1       13    104391 83  Linux
/dev/cciss/c0d0p2             14    17844 143227507+  8e  Linux LVM

WARNING: GPT (GUID Partition Table) detected on '/dev/sda'! The util fdisk doesn't support GPT. Use GNU Parted.

Disk /dev/sda: 9001.6 GB, 9001691512832 bytes
255 heads, 63 sectors/track, 1094393 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot    Start       End    Blocks Id  System
/dev/sda1             1    267350  2147483647+  ee  EFI GPT

WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted.

Disk /dev/sdb: 9001.6 GB, 9001691512832 bytes
255 heads, 63 sectors/track, 1094393 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot    Start       End    Blocks Id  System
/dev/sdb1             1    267350  2147483647+  ee  EFI GPT

WARNING: GPT (GUID Partition Table) detected on '/dev/dm-2'! The util fdisk doesn't support GPT. Use GNU Parted.

Disk /dev/dm-2: 9001.6 GB, 9001691512832 bytes
255 heads, 63 sectors/track, 1094393 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot    Start       End    Blocks Id  System
/dev/dm-2p1             1    267350  2147483647+  ee  EFI GPT

Disk /dev/dm-3: 205.5 GB, 205595800576 bytes
255 heads, 63 sectors/track, 24995 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/dm-3 doesn't contain a valid partition table
[root@localhost mapper]# mkfs.ext3 /dev/mapper/mpath0
mke2fs 1.35 (28-Feb-2004)
mkfs.ext3: Filesystem too large.  No more than 2**31-1 blocks
      (8TB using a blocksize of 4k) are currently supported.
不让创建文件系统。那里出现错误？

作者: kns1024wh 时间: 2009-06-01 08:53
标题: 回复 #1 ldren 的帖子
建议考虑ext4文件系统

作者: ldren 时间: 2009-06-01 10:40
我使用的方式是gpt+ext3的方式。这个相对稳定些。感谢答复。

作者: ipaddr 时间: 2009-06-01 10:50
超过2T,,需要使用GPT.

作者: ldren 时间: 2009-06-01 18:14
目前情况如下：能否看出可以做出7.5TB?
[root@localhost ~]# cd /dev/mapper/
[root@localhost mapper]# ls
control  mpath0  mpath0p1  VolGroup00-LogVol00  VolGroup00-LogVol01
[root@localhost mapper]# cat /proc/partitions
major minor  #blocks  name

104    0  143338560 cciss/c0d0
104    1    104391 cciss/c0d0p1
104    2  143227507 cciss/c0d0p2
8    0 8790714368 sda
8    1 7679999983 sda1
8 16 8790714368 sdb
8 17 7679999983 sdb1
253    0  137756672 dm-0
253    1 5406720 dm-1
253    2 8790714368 dm-2
253    3 1237549039 dm-3

[root@localhost mapper]# mkfs.ext3 /dev/mapper/mpath0
mke2fs 1.35 (28-Feb-2004)
mkfs.ext3: Filesystem too large.  No more than 2**31-1 blocks
      (8TB using a blocksize of 4k) are currently supported.
[root@localhost mapper]# mkfs.ext3  /dev/mapper/mpath0p1
mke2fs 1.35 (28-Feb-2004)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
154697728 inodes, 309387259 blocks
15469362 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=310378496
9442 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
      32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
      4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
      102400000, 214990848
Writing inode table xxxx/9442

作者: ldren 时间: 2009-06-01 21:47
[root@localhost mapper]# mkfs.ext3  /dev/mapper/mpath0p1
mke2fs 1.35 (28-Feb-2004)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
154697728 inodes, 309387259 blocks
15469362 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=310378496
9442 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
      32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
      4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
      102400000, 214990848

Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 26 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
[root@localhost mapper]#
[root@localhost mapper]# mount /dev/mapper/mpath0p1 /var/ftp/pub
[root@localhost mapper]# dh -f
bash: dh: command not found
[root@localhost mapper]# df -h
Filesystem          容量  已用可用已用% 挂载点
/dev/mapper/VolGroup00-LogVol00
                  130G  4.5G  119G 4% /
/dev/cciss/c0d0p1    99M 15M 79M  16% /boot
none                16G    0 16G 0% /dev/shm
/dev/mapper/mpath0p1  1.2T  109M  1.1T 1% /var/ftp/pub
[root@localhost mapper]#

结果为1.2T。看来gpt+ext3的方式难成功。各位能看出那个步骤错误了？

作者: straw 时间: 2009-06-02 10:02
ext3支持超过8T的文件系统么？

作者: hbwangting 时间: 2009-06-02 11:38
标题: 回复 #7 straw 的帖子
EXT3 Size Limits  :

Block size Max file size Max filesystem size
1KiB                16GiB                2TiB
2KiB                256GiB             8TiB
4KiB                2TiB                16TiB
8KiB             16TiB                32TiB

决定单个文件大小取决于文件系统大小的限制，而决定文件系统大小则跟block的大小有关系。

[ 本帖最后由 hbwangting 于 2009-6-2 11:41 编辑 ]

作者: hbwangting 时间: 2009-06-02 11:54

原帖由 ldren 于 2009-6-1 18:14 发表
目前情况如下：能否看出可以做出7.5TB?
[root@localhost ~]# cd /dev/mapper/
[root@localhost mapper]# ls
control mpath0 mpath0p1 VolGroup00-LogVol00 VolGroup00-LogVol01
[root@localhost mapper ...

/dev/mapper/里面为什么看不到dm-2设备？怀疑楼主的mutipath没做成功，lz可以贴出multipath -ll命令的输出结果，你mkfs.ext3 /dev/mapper/mpath0时的mpath0设备如果不是EVA阵列柜的空间，而是本地阵列的空间，就可以解释为什么不时8TB而是1.2TB啦，当然这些都是我个人的猜测。

作者: ldren 时间: 2009-06-03 11:01
以下是链路，没有问题的。

[root@localhost ~]# multipath -ll
mpath0 (3600c0ff000d7b45dd84cf94901000000)
[size=8383 GB][features="1 queue_if_no_path"][hwhandler="0"]
\_ round-robin 0 [active]
\_ 0:0:0:0    sda 8:0  [active][ready]
\_ 1:0:0:0    sdb 8:16 [active][ready]
[root@localhost ~]# cd /dev/mapper/
[root@localhost mapper]# ls
control  mpath0  VolGroup00-LogVol00  VolGroup00-LogVol01
本地磁盘也不会是。因为我指定了路径，mkfs.ext3 /dev/mapper/mpath0/mapth0p1

作者: hbwangting 时间: 2009-06-03 13:27

原帖由 ldren 于 2009-6-1 18:14 发表
目前情况如下：能否看出可以做出7.5TB?
[root@localhost ~]# cd /dev/mapper/
[root@localhost mapper]# ls
control mpath0 mpath0p1 VolGroup00-LogVol00 VolGroup00-LogVol01
[root@localhost mapper ...

我看到dm2的大小约是8.4T，dm3的大小大约1.18T（按1 block=1kb估算），刚好你mount上的/dev/mapper/mpath0p1是约1.2TB，所以我再一次猜测是你分区时只分出了1.2TB，另8.4TB你并没有用fdisk分出来，导致生成的文件只有mpath0p1一个分区，也就是1.2TB的分区，所以你mount上后，df -h也就看到1.2TB。

我建议LZ使用multipath –F 命令清除多路径设备缓存后，再用multipath –v3 命令重新加载，然后parted命令mklable gpt，最后再pvcreate /dev/mapper/mpath0标记为物理卷.

然后对磁盘进行分区fdisk /dev/mapper/mpath0，Kpartx - a / dev/mapper/mpath0 ,分区之后会在dev/mapper/目录下创建新的块设备，/dev/mapper/mpath0p1 这个表示mpath0设备下面的分区1。之后使用fdisk –l 命令查看磁盘会看到dm-2 磁盘下面已经有分区的信息了,就像这样子：

Device Boot Start End Blocks Id System
/dev/dm-2p1 1 121577 976567221 83 Linux

最后再格式化：mkfs.ext3 /dev/mapper/mpath0p1

个人意见，仅供参考。

[ 本帖最后由 hbwangting 于 2009-6-3 14:05 编辑 ]

作者: ldren 时间: 2009-06-03 14:19
Disk /dev/dm-2: 9001.6 GB, 9001691512832 bytes
255 heads, 63 sectors/track, 1094393 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/dm-2p1 1 267350 2147483647+ ee EFI GPT
区已经出来了。

作者: ldren 时间: 2009-06-03 14:23
Device Boot    Start       End    Blocks Id  System
/dev/cciss/c0d0p1 *          1       13    104391 83  Linux
/dev/cciss/c0d0p2             14    17844 143227507+  8e  Linux LVM

WARNING: GPT (GUID Partition Table) detected on '/dev/sda'! The util fdisk doesn't support GPT. Use GNU Parted.

Disk /dev/sda: 9001.6 GB, 9001691512832 bytes
255 heads, 63 sectors/track, 1094393 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot    Start       End    Blocks Id  System
/dev/sda1             1    267350  2147483647+  ee  EFI GPT

WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted.

Disk /dev/sdb: 9001.6 GB, 9001691512832 bytes
255 heads, 63 sectors/track, 1094393 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot    Start       End    Blocks Id  System
/dev/sdb1             1    267350  2147483647+  ee  EFI GPT

WARNING: GPT (GUID Partition Table) detected on '/dev/dm-2'! The util fdisk doesn't support GPT. Use GNU Parted.

Disk /dev/dm-2: 9001.6 GB, 9001691512832 bytes
255 heads, 63 sectors/track, 1094393 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
这个是分区前的情况。

作者: hbwangting 时间: 2009-06-03 14:50
我想确认下mpath0p1的大小，parted /dev/mapper/mpath0 ,(parted) print

普通的linux的mkfs.ext3只能设置有效的 block size 为1024, 2048 and 4096 bytes ，也就是说ext3的block 4kb大小时，文件系统不能超过8TB，单个文件不能超过2TB，但网上说有些系统可以将block size设为更大，来支持更大的文件系统和单个文件大小，不过更多的人通过放弃ext3格式而选择其它的文件系统，来达到支持超大分区。

[ 本帖最后由 hbwangting 于 2009-6-3 15:35 编辑 ]

作者: ldren 时间: 2009-06-03 15:01

原帖由 hbwangting 于 2009-6-3 13:27 发表

我看到dm2的大小约是8.4T，dm3的大小大约1.18T（按1 block=1kb估算），刚好你mount上的/dev/mapper/mpath0p1是约1.2TB，所以我再一次猜测是你分区时只分出了1.2TB，另8.4TB你并没有用fdisk分出来，导致生 ...

这个命令能做超过2TB 容量的分区吗？应该不行吧。前面你使用的是parted这个命令啊。

作者: hbwangting 时间: 2009-06-03 15:40
标题: 回复 #15 ldren 的帖子
我只是觉的分区混乱，特别是想确认/dev/mapper/mpath0p1这个分区的真实大小，所以最好用parted重做一遍

作者: ldren 时间: 2009-06-03 16:14
[root@localhost ~]# fdisk -l

Disk /dev/cciss/c0d0: 146.7 GB, 146778685440 bytes
255 heads, 63 sectors/track, 17844 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

         Device Boot    Start       End    Blocks Id  System
/dev/cciss/c0d0p1 *          1       13    104391 83  Linux
/dev/cciss/c0d0p2             14    17844 143227507+  8e  Linux LVM

WARNING: GPT (GUID Partition Table) detected on '/dev/sda'! The util fdisk doesn 't support GPT. Use GNU Parted.

Disk /dev/sda: 9001.6 GB, 9001691512832 bytes
255 heads, 63 sectors/track, 1094393 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot    Start       End    Blocks Id  System
/dev/sda1             1    267350  2147483647+  ee  EFI GPT

WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn 't support GPT. Use GNU Parted.

Disk /dev/sdb: 9001.6 GB, 9001691512832 bytes
255 heads, 63 sectors/track, 1094393 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot    Start       End    Blocks Id  System
/dev/sdb1             1    267350  2147483647+  ee  EFI GPT

WARNING: GPT (GUID Partition Table) detected on '/dev/dm-2'! The util fdisk does n't support GPT. Use GNU Parted.

Disk /dev/dm-2: 9001.6 GB, 9001691512832 bytes
255 heads, 63 sectors/track, 1094393 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot    Start       End    Blocks Id  System
/dev/dm-2p1             1    267350  2147483647+  ee  EFI GPT

Disk /dev/dm-3: 1267.2 GB, 1267250216448 bytes
255 heads, 63 sectors/track, 154067 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/dm-3 doesn't contain a valid partition table
[root@localhost ~]# cd /dev/mapper/
[root@localhost mapper]# ls
control  mpath0  mpath0p1  VolGroup00-LogVol00  VolGroup00-LogVol01
[root@localhost mapper]# cat /proc/partitions
major minor  #blocks  name

104    0  143338560 cciss/c0d0
104    1    104391 cciss/c0d0p1
104    2  143227507 cciss/c0d0p2
8    0 8790714368 sda
8    1 7679999983 sda1
8 16 8790714368 sdb
8 17 7679999983 sdb1
253    0  137756672 dm-0
253    1 5406720 dm-1
253    2 8790714368 dm-2
253    3 1237549039 dm-3

这样你可以看清楚分区的情况了。我使用了cat /proc/partitions  sda1和sdb1是两个链路指向
dm-3是我没有做分区那个。我给9TB分了7.5TB这样。余下的大概就是1.2TB啦。我也很奇怪为什么他只分成功1.2TB的。但是1.2TB的我根本就没有分区啊。

作者: hbwangting 时间: 2009-06-03 16:55
从上面得到的信息可以看到/dev/sda 和/dev/sdb是不同路径的同一设备，/dev/dm2和/dev/mapper/mpath0都是自动生成的多路径设备，大小都是9TB，超过了8TB限制，是不是这个引起的？

作者: chinaorg 时间: 2009-11-14 18:39
mark

欢迎光临 Chinaunix (http://bbs.chinaunix.net/)