PI Name: Zhengchun Liu
PI Institution: argonne-national-laboratory
Collaborating ANL Division: data-science-and-learning-dsl

Description: Presently, many applications need to process lots of small files, which are organized using file system directories and frequently transferred between compute centres for different computing purposes. Moving data in a wide area network (WAN) environment has been studied extensively, especially for large files. However, many present data movement solutions are not efficient for handling lots of small files (LOSF). We examine the storage impact on the performance of LOSF movement. Furthermore, we improve the efficiency of moving small files by extending GridFTP to optimize the consumption of system resources.

Testbed: IT and Thing