A Trip Down the Data Path: I/O and Performance Page 3 - EnterpriseStorageForum.com

A Trip Down the Data Path: I/O and Performance Page 3

Download the authoritative guide: Enterprise Data Storage 2018: Optimizing Your Storage Infrastructure

By default, all requests made to the system will be the sizes listed in Table 1, regardless of the request size made from a fread(3), fwrite(3) and/or fprint(3) call. This is often called buffered I/O, as the I/O moves not from the user application space to the device but to the C library buffer first. This library buffer size can be changed by a call to the POSIX setvbuf(3). Sometimes making the buffer size over 128 KB (or in the Microsoft case, over 32 KB) does not ensure that the system will perform larger I/O because some operating systems and other software and hardware limitations do not allow those larger I/O requests.

Making the buffers bigger almost always improves performance when doing sequential I/O. When using open(2) and read/write system and/or asynchronous I/O ,the data bypasses the C library buffer that exists when using fopen(3). The request size is usually written in one system call, not withstanding the other software and hardware limitations.

The data path looks like this for buffered I/O:


Application<->Library Buffer<->Operation System Cache<->File System/Volume Manager<->Device


The data path for non-buffered I/O looks like:


Application<-> Operation System Cache <->File System/Volume Manager<->Device

Sometimes the data moves from the C library buffer to the system cache and/or other file system caches or from the user system cache for systems calls. This is because if I/O is not read and/or written on disk hardware sector boundaries (currently multiples of 512 byte blocks on most disk drives), then the request is moved into the system cache to ensure that all I/O done to the system is on 512 byte boundaries.

As you will see in future columns, the amount of data moved does not always depend on the request size from the application or the library buffer, but sometimes depends on the file system instead.

Page 4: Why Does this Matter


Page 3 of 5

Previous Page
1 2 3 4 5
Next Page

Comment and Contribute


(Maximum characters: 1200). You have characters left.



Storage Daily
Don't miss an article. Subscribe to our newsletter below.

By submitting your information, you agree that enterprisestorageforum.com may send you ENTERPRISEStorageFORUM offers via email, phone and text message, as well as email offers about other products and services that ENTERPRISEStorageFORUM believes may be of interest to you. ENTERPRISEStorageFORUM will process your information in accordance with the Quinstreet Privacy Policy.

We have made updates to our Privacy Policy to reflect the implementation of the General Data Protection Regulation.
Thanks for your registration, follow us on our social networks to keep up-to-date