Download the authoritative guide: Enterprise Data Storage 2018: Optimizing Your Storage Infrastructure
This month we take an in-depth look at the internal workings and implementations of file systems, the next stage in the data path as part of our series on storage.
In the last column we briefly introduced some of the internals of file system and volume managers; this month we will review and go into more detail on the important concepts as well as cover several new topics.
File System Services
A file system provides a management framework for the external storage, whereas memory (internal storage) is managed by the operating system itself. The file system allows you to manage data via a framework called files (shocking, huh?). File systems:
- Manage the available space on the devices that are under the file system's control
- Allocation maps
- Where the files reside on the storage
- Removal of files
- Support in some cases via hierarchical storage management (HSM) the placement of those files on secondary storage
- Provide access control via:
- Standard UNIX/POSIX permissions
- Access Control Lists (ACL)
- Support standard UNIX/POSIX interfaces like read/write, fread/fwrite, and aioread/aiowrite
- Support (in some cases) for feature functions like homogenous and/or heterogeneous file sharing and special I/O for databases
- File locking
- Support for NFS
- Special functions which allow the creation of a file without writing the data sequentially (from the start of the file to the end)
File System Functions
Because of data consistency issues, file systems have historically been checked after system crashes. As file systems grew and disk performance did not, given the physical limitations, the time to check the file system after a crash became longer and longer. I remember back in 1992 waiting for 11 hours to fsck(1M) a Cray file system that I personally crashed (the customer was not happy and to this day still remembers -- sorry, Bill). A number of technologies have been developed since that time to help.
How Does This All Work
When you mount(1m) a file system, what really happens? The mount command is a special command used for each file system type. The basic structure of the file system is written to the device(s), and when you type
mount, this basic structure is read from the raw device and processed. What is read is usually called the superblock for the file system. This is generally a special area for the file system that contains information on things like:
- What devices are used
- In what order the devices are used
- Characteristics of the file system such as allocation, performance, and strategies
- Allocation maps
- Location of the inodes and directory blocks
Some file systems are bootable, meaning the system can know about them at boot. This usually requires that the system BIOS or boot bios knows about the structure of the superblock and how to read it.
Occasionally, the file system must be checked for consistency before it can be mounted.