Linux ext4文件系统划分磁盘inode数量
最近因特殊场景,需要往磁盘上写入大量小文件,然而在操作过程中磁盘空间未满但是却提示无法写入……
错误分析
在本次操作过程,需要往磁盘上写入大概 150w 个小文件,文件大小约为 1~100KB,大概预估了一下需要 50G 的磁盘空间。当操作到一半时,磁盘报错:无法写入!
- 通过 df -h 查看磁盘,发现还有剩余空间;
- 通过 du -i 查看磁盘,发现 inode 已经使用了 100%;
继续观察磁盘文件系统:
dumpe2fs -h /dev/vdc1
这里列出了几个比较关键的数据:
Inode count: 3932160 Block count: 15728384 Block size: 4096 Inode size: 256 Inodes per group: 8192 Inode blocks per group: 512
其中,
- 每个 inode 大小为 256byte,block 大小为 4k byte;
- 根据 block count 和 inode count,我们也可以算出 16k bytes-per-inode(15728384*4096/3932160);
也就是文件系统在创建的时候每16k空间自动划分一个inode,而我们需要写入大量小文件,虽然磁盘空间(block)还有剩余,但是 inode 已经分配完了。
磁盘规划
因此针对上面的情况,我们需要划分更多的 inode 用于记录文件。通过查看文档:
man mkfs.ext4
OPTIONS -b block-size Specify the size of blocks in bytes. Valid block-size values are 1024, 2048 and 4096 bytes per block. If omitted, block- size is heuristically determined by the filesystem size and the expected usage of the filesystem (see the -T option). If block-size is preceded by a negative sign ('-'), then mke2fs will use heuristics to determine the appropriate block size, with the constraint that the block size will be at least block-size bytes. This is useful for certain hardware devices which require that the blocksize be a multiple of 2k. -i bytes-per-inode Specify the bytes/inode ratio. mke2fs creates an inode for every bytes-per-inode bytes of space on the disk. The larger the bytes-per-inode ratio, the fewer inodes will be created. This value generally shouldn't be smaller than the blocksize of the filesystem, since in that case more inodes would be made than can ever be used. Be warned that it is not possible to change this ratio on a filesystem after it is created, so be careful deciding the correct value for this parameter. Note that resizing a filesystem changes the numer of inodes to maintain this ratio. -I inode-size Specify the size of each inode in bytes. The inode-size value must be a power of 2 larger or equal to 128. The larger the inode-size the more space the inode table will consume, and this reduces the usable space in the filesystem and can also neg‐ atively impact performance. It is not possible to change this value after the filesystem is created. In kernels after 2.6.10 and some earlier vendor kernels it is possible to utilize inodes larger than 128 bytes to store extended attributes for improved performance. Extended attributes stored in large inodes are not visible with older kernels, and such filesystems will not be mountable with 2.4 kernels at all. The default inode size is controlled by the mke2fs.conf(5) file. In the mke2fs.conf file shipped with e2fsprogs, the default inode size is 256 bytes for most file systems, except for small file systems where the inode size will be 128 bytes. -N number-of-inodes Overrides the default calculation of the number of inodes that should be reserved for the filesystem (which is based on the number of blocks and the bytes-per-inode ratio). This allows the user to specify the number of desired inodes directly.
一般情况下, block-size 和 inode-size 我们都不需要去更改;而 number-of-inodes 则应该由 bytes-per-inode 自动计算而出,也不应该手动指定。
因此,我们重新格式化磁盘:
mkfs.ext4 -i 8192 /dev/vdc1
查看磁盘文件系统:
dumpe2fs -h /dev/vdc1
Inode count: 7864320 Block count: 15728384 Block size: 4096 Inode size: 256 Inodes per group: 16384 Inode blocks per group: 1024
可以看到,inode整体数量以及在每个group的数量都翻了一倍。虽然重新划分更多 inode 占用了磁盘空间,不过这才更符合我们的实际使用需求。
此外,bytes-per-inode 在文件系统创建之后则无法修改,因此我们需要在使用前格式化的时候就明确下来,避免导致后期数据迁移等麻烦。
码字很辛苦,转载请注明来自ChenJiehua的《Linux ext4文件系统划分磁盘inode数量》
评论
评论关闭