It shares the create/open/delete/set_version/get_file_size
functions with POSIX backend.
The mmap backend also supports fsync and fsyncPerWrite options,
and it will use msync() instead and fsync().
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Context: Some file systems require a prefix in the path.
The POSIX 'access' call fails and consequently files are
never deleted.
This patch implements an access function in the MPIIO
backend using MPI_File_open. Prefixes can now be parsed
by ROMIO.
The O_DIRECT option was not working as set_o_direct_flag() were moved to
utilities.c but there the #define _GNU_SOURCE where missing. This lead
to not the Waring "cannot use O_DIRECT".
From Pnetcdf 1.7, the MPI datatype corresponding to NC_BYTE is change from MPI_BYTE to MPI_SIGNED_CHAR. If not change, running IOR with NCMPI will cause a fatal error as below:
ERROR in aiori-NCMPI.c (line 287): cannot write to data set.
ERROR: NetCDF: Not a valid data type or _FillValue type mismatch.
The file "src/aiori-NCMPI.c" uses numTasksWorld as the process which is declared in "src/ior.h". In "src/mdtest.c", NCMPI backend will be called if IOR configured with ncmpi enabled but numTasksWorld was not defined in "src/mdtest.c". So it will cause a compiler error like below:
mdtest-aiori-NCMPI.o: In function `NCMPI_Xfer':
/home/parallels/Documents/ior/src/aiori-NCMPI.c:272: undefined reference to `numTasksWorld'
Once a process hits the stonewall (timelimit), they all figure out the maximum pair read/written.
Each proces continues to read/write until the maximum number of pairs is reached, this simulates the wear out.
This commit makes changes to the AIORI backends to add support for
abstacting statfs, mkdir, rmdir, stat, and access. These new
abstractions are used by a modified mdtest. Some changes:
- Require C99. Its 2017 and most compilers now support C11. The
benefits of using C99 include subobject naming (for aiori backend
structs), and fixed size integers (uint64_t). There is no reason to
use the non-standard long long type.
- Moved some of the aiori code into aiori.c so it can be used by both
mdtest and ior.
- Code cleanup of mdtest. This is mostly due to the usage of the IOR
backends rather than a mess of #if code.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
Previously -R did run another read phase and compared if the output of both reads is identical.
-R now checks if the data matches the expected signature (as set using -G <NUMBER>), so reads data once and then
directly compares the read data with the expected buffer.
This allows to first run IOR with a write only phase, then later with a read phase that checks if the data still is correct.
Since the read can be repeated multiple times, there is no need for the old -R semantics.
If a hintfile contains e.g. cb_buffer_size = 1234, IOR will try to set
the hint "cb_buffer_size " (note trailing space), a hint that no MPI
implementation actually supports.
IOR was leaking a hint structure in MPI-IO case, two groups in
common code, and when we get the info member from the file we were
losing our reference to the hints we just passed in.
This makes it so that the buffers are only allocated once per test instead
of once per transfer. This also removes initial buffer set-up from the
timing window.
Added a new struct into ior.h IOR_io_buffers for the buffer, checkbuffer, and readcheckbuffer
so only one pointer needed to be passed to XferBuffersSetup(), XferBuffersFree(),
and WriteOrRead().
Changed the logic in XferBuffersSetup() and XferBuffersFree() to not be transfer
dependent. If a test includes a write check or read check the checkBuffer
and readcheckBuffer will be created once per test in TestIoSys(). The
argument now taken by both function has changed from the access type to
a pointer to IOR_param_t.
Changed WriteOrRead to take as an additional parameter
the IOR_io_buffers struct, since it was no longer creating those
buffers.
Changed how the -l option works. Now you choose the type of datapacket
-l i incompressible data packets
-l incompressible incompressible data packets
-l timestamp timestamped data packets
-l t timestamped data packets
-l offset offset data packets
-l o offset data packets
-G option now is either the seed for the incompressible random packets
or the timestamp, depending on the input to the -l option.
-G will no long timestamp packets on its own without the additon of -l timestamp or -l t
I kept shorter versions of the options for the sake of typing sanity.
Julian M. Kunkel