• 隐藏侧边栏
  • 展开分类目录
  • 关注微信公众号
  • 我的GitHub
  • QQ:1753970025
Chen Jiehua

Linux ext4文件系统划分磁盘inode数量 

最近因特殊场景,需要往磁盘上写入大量小文件,然而在操作过程中磁盘空间未满但是却提示无法写入……

错误分析

在本次操作过程,需要往磁盘上写入大概 150w 个小文件,文件大小约为 1~100KB,大概预估了一下需要 50G 的磁盘空间。当操作到一半时,磁盘报错:无法写入!

  • 通过 df -h 查看磁盘,发现还有剩余空间;
  • 通过 du -i 查看磁盘,发现 inode 已经使用了 100%;

继续观察磁盘文件系统:

dumpe2fs -h /dev/vdc1

这里列出了几个比较关键的数据:

Inode count:              3932160
Block count:              15728384
Block size:               4096
Inode size:	          256
Inodes per group:         8192
Inode blocks per group:   512

其中,

  • 每个 inode 大小为 256byte,block 大小为 4k byte;
  • 根据 block count 和 inode count,我们也可以算出 16k bytes-per-inode(15728384*4096/3932160);

也就是文件系统在创建的时候每16k空间自动划分一个inode,而我们需要写入大量小文件,虽然磁盘空间(block)还有剩余,但是 inode 已经分配完了。

磁盘规划

因此针对上面的情况,我们需要划分更多的 inode 用于记录文件。通过查看文档:

man mkfs.ext4
OPTIONS
       -b block-size
              Specify  the  size  of blocks in bytes.  Valid block-size values are 1024, 2048 and 4096 bytes per block.  If omitted, block-
              size is heuristically determined by the filesystem size and the expected usage of the filesystem (see  the  -T  option).   If
              block-size  is  preceded  by  a negative sign ('-'), then mke2fs will use heuristics to determine the appropriate block size,
              with the constraint that the block size will be at least block-size bytes.  This is useful for certain hardware devices which
              require that the blocksize be a multiple of 2k.

       -i bytes-per-inode
              Specify the bytes/inode ratio.  mke2fs creates an inode for every bytes-per-inode bytes of space on the disk.  The larger the
              bytes-per-inode ratio, the fewer inodes will be created.  This value generally shouldn't be smaller than the blocksize of the
              filesystem,  since in that case more inodes would be made than can ever be used.  Be warned that it is not possible to change
              this ratio on a filesystem after it is created, so be careful deciding the correct  value  for  this  parameter.   Note  that
              resizing a filesystem changes the numer of inodes to maintain this ratio.

       -I inode-size
              Specify  the  size of each inode in bytes.  The inode-size value must be a power of 2 larger or equal to 128.  The larger the
              inode-size the more space the inode table will consume, and this reduces the usable space in the filesystem and can also neg‐
              atively impact performance.  It is not possible to change this value after the filesystem is created.

              In  kernels  after  2.6.10  and  some  earlier vendor kernels it is possible to utilize inodes larger than 128 bytes to store
              extended attributes for improved performance.  Extended attributes stored in large inodes are not visible with older kernels,
              and such filesystems will not be mountable with 2.4 kernels at all.

              The default inode size is controlled by the mke2fs.conf(5) file.  In the mke2fs.conf file shipped with e2fsprogs, the default
              inode size is 256 bytes for most file systems, except for small file systems where the inode size will be 128 bytes.

        -N number-of-inodes
              Overrides  the  default calculation of the number of inodes that should be reserved for the filesystem (which is based on the
              number of blocks and the bytes-per-inode ratio).  This allows the user to specify the number of desired inodes directly.

一般情况下, block-size inode-size 我们都不需要去更改;而 number-of-inodes 则应该由 bytes-per-inode 自动计算而出,也不应该手动指定。

因此,我们重新格式化磁盘:

mkfs.ext4 -i 8192 /dev/vdc1

查看磁盘文件系统:

dumpe2fs -h /dev/vdc1
Inode count:              7864320
Block count:              15728384
Block size:               4096
Inode size:	          256
Inodes per group:         16384 
Inode blocks per group:   1024

可以看到,inode整体数量以及在每个group的数量都翻了一倍。虽然重新划分更多 inode 占用了磁盘空间,不过这才更符合我们的实际使用需求。

此外,bytes-per-inode 在文件系统创建之后则无法修改,因此我们需要在使用前格式化的时候就明确下来,避免导致后期数据迁移等麻烦。

码字很辛苦,转载请注明来自ChenJiehua《Linux ext4文件系统划分磁盘inode数量》

评论

评论关闭