Open Science Research Excellence

Open Science Index

Commenced in January 2007 Frequency: Monthly Edition: International Publications Count: 29858


Select areas to restrict search in scientific publication database:
14259
Design and Implementation of Shared Memory based Parallel File System Logging Method for High Performance Computing
Abstract:
I/O workload is a critical and important factor to analyze I/O pattern and file system performance. However tracing I/O operations on the fly distributed parallel file system is non-trivial due to collection overhead and a large volume of data. In this paper, we design and implement a parallel file system logging method for high performance computing using shared memory-based multi-layer scheme. It minimizes the overhead with reduced logging operation response time and provides efficient post-processing scheme through shared memory. Separated logging server can collect sequential logs from multiple clients in a cluster through packet communication. Implementation and evaluation result shows low overhead and high scalability of this architecture for high performance parallel logging analysis.
Digital Object Identifier (DOI):

References:

[1] John K. Ousterhout, Hervg Da Costa, David Harrison, John A. Kunze, Mike Kupfer, and James G. Thompson, " A Trace-Driven Analysis of the UNIX 4.2 BSD File System," ACM SIGOPS Operating Systems Review archive, Volume 19, Issue 5, pp. 15~24, 1985.
[2] Drew Roselli, Jacob R. Lorch, and Thomas E. Anderson, "A comparison of file system workloads," Proc. of USENIX Annual Technical Conference, pp. 41~54, 2000.
[3] Akshat Aranya, Charles P. Wright, and Erez Zadok, "Tracefs: A File System to Trace Them All," FAST 2004.
[4] Pin Lu and Kai Shen, "Multi-Layer Event Trace Analysis for Parallel I/O Performance Tuning," Proceedings of the 2007 International Conference on Parallel Processing, 2007.
[5] Anthony Chan, William Gropp, and Ewing Lusk, "User-s Guide for MPE: Extensions for MPI Programs", from MPICH2 web site, http://www.mcs.anl.gov/research/projects/mpich2
[6] MPICH2 web site, http://www.mcs.anl.gov /research/projects/mpich2
[7] Rajeev Thakur, William Gropp, Ewing Lusk, "On Implementing MPI-IO Portably and with High Performance," In Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems, pp. 23-32, 1999.
[8] PVFS web size, http://www.pvfs.org
[9] Lustre web site, http://wiki.lustre.org
[10] GPFS Wikipedia, http://en.wikipedia.org/wiki/GPFS
[11] Chris Ruemmler and John Wilkes, "A trace-driven analysis of disk working set sizes", Technical Report HPL-OSR-93-23, Hewlett- Packard Laboratories, April 1993.
[12] sourceforge starce home page, http://sourceforge.net/projects/strace/
[13] Akshat Aranya, Charles P. Wright, Erez Zadok, "Tracefs: A File System to Trace Them All," Proceedings of the 3rd USENIX Conference on File and Storage Technologies, pp. 129 - 145, 2004.
[14] L. Mummert, M. Satyanarayanan, "Long term distributed file reference tracing: Implementation and experience," SoftwareÔÇöPractice & Experience, Volume 26, Issue 6, pp. 705-736, 1996.
[15] tcpdump/libcap homepage, http://www.tcpdump.org/
[16] M. Blaze, "NFS Tracing by Passive Network Monitoring, " In Proceedings of the USENIX Winter Conference, January 1992.
[17] D. Ellard, J. Ledlie, P. Malkani, and M. Seltzer. Passive NFS Tracing of Email and Research Workloads. In Proceedings of the Annual USENIX Conference on File and Storage Technologies, March 2003.
[18] Andrew W. Leung, Shankar Pasupathy, Garth Goodson, Ethan L. Miller "Measurement and Analysis of Large-Scale Network File System Workloads," In the proceedings of the 2008 USENIX Annual Technical Conference, June 2008.
[19] Ibrahim F. Haddad inSysAdmin, "PVFS: A Parallel Virtual File System for Linux Cluster," Linux Journal, 2000.
[20] Mohan Rajagopalan, Matti Hiltunen, Trevor Jim, and Richard Schlichting, "Authenticated System Calls," DSN-2005: The International Conference on `Dependable Systems and Networks, June 2005.
[21] Intel web site, http://software.intel.com/en-us/forums/watercoolercatchall /topic/54276
[22] iozone web site, http://www.iozone.org/
Vol:13 No:07 2019Vol:13 No:06 2019Vol:13 No:05 2019Vol:13 No:04 2019Vol:13 No:03 2019Vol:13 No:02 2019Vol:13 No:01 2019
Vol:12 No:12 2018Vol:12 No:11 2018Vol:12 No:10 2018Vol:12 No:09 2018Vol:12 No:08 2018Vol:12 No:07 2018Vol:12 No:06 2018Vol:12 No:05 2018Vol:12 No:04 2018Vol:12 No:03 2018Vol:12 No:02 2018Vol:12 No:01 2018
Vol:11 No:12 2017Vol:11 No:11 2017Vol:11 No:10 2017Vol:11 No:09 2017Vol:11 No:08 2017Vol:11 No:07 2017Vol:11 No:06 2017Vol:11 No:05 2017Vol:11 No:04 2017Vol:11 No:03 2017Vol:11 No:02 2017Vol:11 No:01 2017
Vol:10 No:12 2016Vol:10 No:11 2016Vol:10 No:10 2016Vol:10 No:09 2016Vol:10 No:08 2016Vol:10 No:07 2016Vol:10 No:06 2016Vol:10 No:05 2016Vol:10 No:04 2016Vol:10 No:03 2016Vol:10 No:02 2016Vol:10 No:01 2016
Vol:9 No:12 2015Vol:9 No:11 2015Vol:9 No:10 2015Vol:9 No:09 2015Vol:9 No:08 2015Vol:9 No:07 2015Vol:9 No:06 2015Vol:9 No:05 2015Vol:9 No:04 2015Vol:9 No:03 2015Vol:9 No:02 2015Vol:9 No:01 2015
Vol:8 No:12 2014Vol:8 No:11 2014Vol:8 No:10 2014Vol:8 No:09 2014Vol:8 No:08 2014Vol:8 No:07 2014Vol:8 No:06 2014Vol:8 No:05 2014Vol:8 No:04 2014Vol:8 No:03 2014Vol:8 No:02 2014Vol:8 No:01 2014
Vol:7 No:12 2013Vol:7 No:11 2013Vol:7 No:10 2013Vol:7 No:09 2013Vol:7 No:08 2013Vol:7 No:07 2013Vol:7 No:06 2013Vol:7 No:05 2013Vol:7 No:04 2013Vol:7 No:03 2013Vol:7 No:02 2013Vol:7 No:01 2013
Vol:6 No:12 2012Vol:6 No:11 2012Vol:6 No:10 2012Vol:6 No:09 2012Vol:6 No:08 2012Vol:6 No:07 2012Vol:6 No:06 2012Vol:6 No:05 2012Vol:6 No:04 2012Vol:6 No:03 2012Vol:6 No:02 2012Vol:6 No:01 2012
Vol:5 No:12 2011Vol:5 No:11 2011Vol:5 No:10 2011Vol:5 No:09 2011Vol:5 No:08 2011Vol:5 No:07 2011Vol:5 No:06 2011Vol:5 No:05 2011Vol:5 No:04 2011Vol:5 No:03 2011Vol:5 No:02 2011Vol:5 No:01 2011
Vol:4 No:12 2010Vol:4 No:11 2010Vol:4 No:10 2010Vol:4 No:09 2010Vol:4 No:08 2010Vol:4 No:07 2010Vol:4 No:06 2010Vol:4 No:05 2010Vol:4 No:04 2010Vol:4 No:03 2010Vol:4 No:02 2010Vol:4 No:01 2010
Vol:3 No:12 2009Vol:3 No:11 2009Vol:3 No:10 2009Vol:3 No:09 2009Vol:3 No:08 2009Vol:3 No:07 2009Vol:3 No:06 2009Vol:3 No:05 2009Vol:3 No:04 2009Vol:3 No:03 2009Vol:3 No:02 2009Vol:3 No:01 2009
Vol:2 No:12 2008Vol:2 No:11 2008Vol:2 No:10 2008Vol:2 No:09 2008Vol:2 No:08 2008Vol:2 No:07 2008Vol:2 No:06 2008Vol:2 No:05 2008Vol:2 No:04 2008Vol:2 No:03 2008Vol:2 No:02 2008Vol:2 No:01 2008
Vol:1 No:12 2007Vol:1 No:11 2007Vol:1 No:10 2007Vol:1 No:09 2007Vol:1 No:08 2007Vol:1 No:07 2007Vol:1 No:06 2007Vol:1 No:05 2007Vol:1 No:04 2007Vol:1 No:03 2007Vol:1 No:02 2007Vol:1 No:01 2007