Along the way, added a bunch of diagnostic output in the HDFS calls, which
only shows up at verbosity >= 4. I'll probably remove this stuff before
merging with master. Also, there's an #ifdef'ed-out sleep() in place,
which I used to attach gdb to a running MPI task. I'll get rid of that
later, too.
Also, added another hdfs-related parameter to the IOR_param_t structure;
hdfs_user_name gets the value of the USER environment-variable as the
default HDFS user for connections. Does this cause portability problems?
I saw run in which I caught an MPI task hanging in ctime() here. Swiching
to ctime_r() fixes that. This function is only called form rank==0, but it
hangs anyway.
This is not a problem for most backends, but HDFS doesn't support opening
RDWR. If you use only write-oriented or read-oriented flags on the
command-line, CheckRunSettings() will undo the default IOR_RDWR flag and
install the appropriate IOR_WRONLY or IO_RDONLY open-flags, respectively.
This provides an HDFS back-end, allowing IOR to exercise a Hadoop
Distributed File-System, plus corresponding changes throughout, to
integrate the new module into the build. The commit compiles at LANL, but
hasn't been run yet. We're currently waiting for some configuration on
machines that will eventually provide HDFS. By default, configure ignores
the HDFS module. You have to explicitly add --with-hdfs.
GPFS supports a "gpfs_fcntl" method for hinting various things,
including "i'm about to write this block of data". Let's see if, for
the cost of a few system calls, we can wrangle the GPFS locking system
into allowing concurrent access with less overhead. (new IOR parameter
gpfsHintAccess)
Also, drop all locks on a file immediately after open/creation in the
shared file case, since we know all processes will touch unique regions
of the file. It may or may not be a good idea to release all file locks
after opening. Processes will then have to re-acquire locks already
held. (new IOR parameter gpfsReleaseToken)
Improve the scalabilit of CountTasksPerNode() by using
a Broadcast and AllReduce, rather than flooding task zero
with MPI_Send() messages.
Also change the hostname lookup function from MPI_Get_processor_name
to gethostname(), which should work on most systems that I know of,
including BlueGene/Q.
Removing AC_FUNC_MALLOC from configure.ac, to allow compilation
on BG/P systems. This check can fail in cross-compilation environments,
which unnecessarily forces autoconf to require an rpl_malloc()
replacement for malloc(). We could implement the conditional addition
of rpl_malloc(), but removing AC_FUNC_MALLOC is a quite work-around.
fixes#4
Allows every task to allocate a specified amount of memory as
a rough simulation of a real application's memory usage.
Every page of the allocated memory is touch to defeat lazy
memory allocation.
Original patch by Michael Kluge <michael.kluge@tu-dresden.de>