Commit Graph

102 Commits (19a0102181a6fbfa7ea71768a3c2b6660e886274)

Author SHA1 Message Date
Julian M. Kunkel 205b20f0f4 Bugfix merge 2018-07-12 16:13:40 +01:00
Julian Kunkel 060c47e80a
Merge branch 'testing' into rados-hacks 2018-07-12 15:53:04 +01:00
Julian M. Kunkel 2bc30410cc JSON output works. 2018-07-08 16:47:23 +01:00
Julian M. Kunkel 20ebeb71b8 Further adaptions to generic output, support of JSON output (partially complete). 2018-07-08 14:59:54 +01:00
Julian M. Kunkel 8525f9734c Extraction of output completed. 2018-07-08 13:50:20 +01:00
Julian M. Kunkel 02883d4a63 Moved most output to ior-output.c 2018-07-08 13:38:05 +01:00
Julian M. Kunkel 0f7a1f14b9 Support IO redirect from main() 2018-07-08 13:12:18 +01:00
Shane Snyder 0052bff3e9 use backend->access in IOR source
Signed-off-by: Julian M. Kunkel <juliankunkel@googlemail.com>
2018-07-08 12:13:38 +01:00
Enno Zickler 9e97a16fe4 fix output to be allinge with column headers
Signed-off-by: Julian M. Kunkel <juliankunkel@googlemail.com>
2018-07-08 11:54:49 +01:00
Julian M. Kunkel d938f0efe1 MPIIO support fsync. 2018-07-08 00:16:30 +01:00
Julian M. Kunkel ed5787811c Extracted max pathlen variable. 2018-07-07 22:39:14 +01:00
Julian M. Kunkel 48c17b4d81 Extracted (nearly identical) function DelaySeconds. 2018-07-07 22:26:57 +01:00
Julian M. Kunkel 6f8d2e8845 Some simplification / unification between IOR and MDTest.
MDTest needs more refactoring to be really maintainable.
2018-07-07 22:19:42 +01:00
Julian M. Kunkel b4f356a7b2 Bugfix: StoneWallingStatusFile. 2018-07-07 14:14:55 +01:00
Julian M. Kunkel 812b798f05 IOR allows to store actual performaned stonewalling count into a status file. 2018-07-07 14:01:11 +01:00
Julian M. Kunkel 15c383b0d1 Extended description for #44 2018-07-07 12:20:29 +01:00
Julian M. Kunkel e10bebe653 MDTest supports only POSIX at the moment. Added command line check. 2018-07-07 12:14:34 +01:00
Julian M. Kunkel 21b8cca8cf IOR library version. 2018-07-07 11:42:21 +01:00
Julian M. Kunkel a6bfa0f94b MDTest library option.
Moved core functionality / variables used by both implementations to utilities
2018-07-07 10:42:48 +01:00
Li Dongyang 6ba22a19e4 Add mmap as a new AIORI backend
It shares the create/open/delete/set_version/get_file_size
functions with POSIX backend.
The mmap backend also supports fsync and fsyncPerWrite options,
and it will use msync() instead and fsync().

Signed-off-by: Li Dongyang <dongyangli@ddn.com>
2018-05-11 12:13:42 +10:00
Shane Snyder 92dfb67e5c allow fsync in POSIX, MPIIO, and HDFS 2018-04-27 12:44:20 -05:00
Shane Snyder 9b75f071f4 use backend->access in IOR source 2018-04-25 17:35:10 -05:00
Glenn K. Lockwood 7a371cfeda
Merge pull request #46 from lidongyang/iops
Add OPs result columns back
2018-04-23 13:12:43 -07:00
Glenn K. Lockwood a80cc44fb1 fixes #47 2018-04-23 13:10:04 -07:00
Li Dongyang 0d2c9e5451 Add OPs result columns back
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
2018-04-18 14:24:42 +10:00
Enno Zickler 16c8ef6a23 Doc: src comments for getOffsetArray functions 2018-03-15 21:52:34 +01:00
Enno Zickler d5507955af Doc: Initial setup for Sphinx and Doxygen 2018-03-15 21:52:10 +01:00
Glenn K. Lockwood 3887668bda
Merge pull request #19 from VI4IO/master
Various bug fixes and code cleanup
2017-12-12 10:56:09 -08:00
Julian M. Kunkel cdc93da3a4 Check for command line slight naming change. 2017-12-09 17:24:31 +01:00
Julian M. Kunkel 93355314de Adjusted command line checks for recent changes in semantics for readCheck/writeCheck. 2017-12-09 17:22:58 +01:00
Julian M. Kunkel 1f6d062f7f Bugfix #22 2017-12-09 11:52:13 +01:00
Glenn K. Lockwood 10e2b2efa2 make all references to API name case-insensitive (fix for 6ba323b) 2017-12-04 15:20:31 -08:00
Julian M. Kunkel 2c224e0ee8 Bugfix for #14, storeFileOffset=1 2017-11-30 11:56:26 +01:00
Julian Kunkel b67c5ce62c Bugfix for #14
WriteCheck did not work properly due to shifting of patterns.
It is possible that one buffer could be saved by refactoring the code properly.
2017-11-29 10:17:02 +01:00
Julian Kunkel 6436165edb For debugging of issues: added IOR_FAKE_TASK_PER_NODES env variable to set tasksPerNode. 2017-11-29 09:54:22 +01:00
Glenn K. Lockwood 3aa1f4c9b7
Merge pull request #11 from VI4IO/master
StoneWalling with WearOutPhase
2017-11-15 00:09:22 -07:00
John Bent 34cb63c1b7 Fixed the bug in the read check code 2017-10-25 07:57:50 -06:00
Julian M. Kunkel 072eb452fd Stonewalling wear-out only with long options. 2017-10-21 11:59:09 +02:00
Julian Kunkel 96e75cc832 stoneWallingWearOutIterations allows to read data back using the same number of iterations as used for the write with stoneWallingWearOut. 2017-10-20 18:14:06 +02:00
Julian Kunkel aa0b8c92d2 New stonewalling option to create a wear out phase.
Once a process hits the stonewall (timelimit), they all figure out the maximum pair read/written.
Each proces continues to read/write until the maximum number of pairs is reached, this simulates the wear out.
2017-10-20 18:02:24 +02:00
Nathan Hjelm 4dc0081628 First cut at using the IOR backends for mdtest
This commit makes changes to the AIORI backends to add support for
abstacting statfs, mkdir, rmdir, stat, and access. These new
abstractions are used by a modified mdtest. Some changes:

 - Require C99. Its 2017 and most compilers now support C11. The
   benefits of using C99 include subobject naming (for aiori backend
   structs), and fixed size integers (uint64_t). There is no reason to
   use the non-standard long long type.

 - Moved some of the aiori code into aiori.c so it can be used by both
   mdtest and ior.

 - Code cleanup of mdtest. This is mostly due to the usage of the IOR
   backends rather than a mess of #if code.

Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
2017-10-19 15:29:12 -06:00
Alfred Torrez 2541bfea7b Merge pull request #10 from ThinkParQ/master
add support for tuning BeeGFS parameters
2017-10-19 13:40:41 -06:00
Oliver Steffen 738190bd79 add support for tuning BeeGFS parameters 2017-10-04 14:14:36 +02:00
Julian Kunkel 2ce37ae90a Changed the semantics of -R to compare the data with the expected data buffer.
Previously -R did run another read phase and compared if the output of both reads is identical.
-R now checks if the data matches the expected signature (as set using -G <NUMBER>), so reads data once and then
directly compares the read data with the expected buffer.
This allows to first run IOR with a write only phase, then later with a read phase that checks if the data still is correct.
Since the read can be repeated multiple times, there is no need for the old -R semantics.
2017-09-27 18:45:47 +02:00
Brett Kettering 73c1183d8c Merge pull request #2 from roblatham00/memory_cleanup
assorted memory leak cleanups
2015-12-07 14:20:43 -07:00
Rob Latham d2a8aa0d2d assorted memory leak cleanups
tidy up a few stray memory allocations.  Note: these changes make
HumanReadable no longer re-entrant.
2015-12-07 15:10:30 -06:00
Rob Latham 343ad076a3 clean up MPI resources
IOR was leaking a hint structure in MPI-IO case, two groups in
common code, and when we get the info member from the file we were
losing our reference to the hints we just passed in.
2015-12-07 14:54:22 -06:00
Blair Crossman f297c7b0d5 Moved the calls to XferBuffersSetup() and XferBuffersFree to TestIoSys()
This makes it so that the buffers are only allocated once per test instead
of once per transfer.  This also removes initial buffer set-up from the
timing window.

Added a new struct into ior.h IOR_io_buffers for the buffer, checkbuffer, and readcheckbuffer
so only one pointer needed to be passed to XferBuffersSetup(), XferBuffersFree(),
and WriteOrRead().

Changed the logic in XferBuffersSetup() and XferBuffersFree() to not be transfer
dependent.  If a test includes a write check or read check the checkBuffer
 and readcheckBuffer will be created once per test in TestIoSys().  The
argument now taken by both function has changed from the access type to
a pointer to IOR_param_t.

Changed WriteOrRead to take as an additional  parameter
the IOR_io_buffers struct, since it was no longer creating those
buffers.
2015-05-27 10:24:52 -06:00
Blair Crossman 08d14e986d Added an uncompressible data packet to IOR.
Changed how the -l option works.  Now you choose the type of datapacket
-l i  			incompressible data packets
-l incompressible  	incompressible data packets
-l timestamp 		timestamped data packets
-l t			timestamped data packets
-l offset		offset data packets
-l o			offset data packets

-G option now is either the seed for the incompressible random packets
   or the timestamp, depending on the input to the -l option.

-G will no long timestamp packets on its own without the additon of -l timestamp or -l t

I kept shorter versions of the options for the sake of typing sanity.
2015-05-21 12:05:56 -06:00
Jeff Inman 37738dab26 Numerous changes to file-modes, small build-tweaks, and a tweak to aiori-S3.c
(Only rank-0 should create the bucket, if it doesn't already exist.)

Prepping this for a push (as an experimental S3 build) to github.
2015-05-19 09:36:28 -06:00