File System Evolution With Memory-Like Storage

--By Manager On June 11, 2017


Developers are always faced with a lot of variations and similarities among tasks they encounter. One of them is the difference between rest and soap webservices in c#, while others include changing of file systems. However, when it comes to file systems, tech companies like Intel, Sun Microsystems, and IBM are looking into further development of their products that could change the way we save and manage files.

File System Evolution

Eric Barton, one of the people who developed early parallel file systems that were used in supercomputers has become curious and amused about the term software-defined storage. As a result, he dedicated his time and energy doing software-defined storage.

Diversification and Commercialization of File Systems

Barton has already left Intel, but is serious in providing help to diversify and commercialize the file system DDNs IME. Now, he is the chief technical officer SDN in one of the biggest storage companies.


Eric Barton isn’t just an expert but also an experienced developer of file system and storage. His three decades of career in storage had made him realize the need to reinvent file systems.

During his career, he observed that there are also some good file systems from other companies like that in Solaris that can be adopted and striped across other file systems by using a namespace; however, this is more easily said than done. This is because scaling a namespace and providing access to a chunk of data is tricky.

Consequently, the Lustre file system entered the scene, and this further pushed tech giants like Sun Microsystems, IBM, and SGI to develop parallel file systems through the years. Now, file systems are diversified and commercialized unlike before where they could only be used in supercomputers.

The Need to Change File Systems with Memory-Like Storage

Companies like Lustre are geared toward creating a non-volatile memory technology that can offer huge amount of bandwidth. This is something which was unimaginable many years ago. Lustre file system is not different from PFS, and according to Eric Barton, the only difference is that Lustre is the one that is done properly.

On the other hand, 3D XPoint as a popular memory is not actually a memory but a disk or a flash storage. Barton added that there are a lot of controversies and issues when it comes to using flash and 3DXPoint. To optimize them, you will have to change the storage software layer.

According to Barton, the kind of change that is needed is like what the NVM Express protocol has displayed. This protocol allows direct connection to flash rather than allowing the disk to emulate the flash. This can also be done in larger file systems. Barton insists that the thickness of software between an application, and a hardware device is what hampers the efficiency of file systems.

The change that we can expect is not the improvement of the existing device or tools that are used for storage. This is not another disk or flash technology but a shortcut process; moreover, speed also matters. It’s not the storage space that is only important to be considered but also the speed at which data is processed.

Final Thoughts

Technology continues to surprise not only the novice users but also developers. In the world of information technology, time is of the essence.

Every year, there must be a change not only in the design of technology, but also in the way it works with other technologies. This includes file system that needs to be changed and improved out of its old form that has been around through the years.