Commit Graph

68 Commits (master)

Author SHA1 Message Date
Julian Kunkel 310fd37427
Feature mdtest dir rename #306 (#311)
* MDTest: Support for directory renaming.
* Refactored some MDTest variables from hardcoded numbers to ENUM symbols.
2021-01-21 14:10:23 +00:00
Julian M. Kunkel 4898badf48 Fix compiler issue with redefinition via typedef. 2020-10-29 10:38:33 +00:00
Mohamad Chaarawi 7c201c0d9c
HDF5: fix segfault in HDF5 driver after latest AIORI changes (#255)
- the hints structure was not getting initialized in MPIIO when HDF5
backend is used. Since HDF5 utilizes the MPIIO backend, this causes
a segfault.

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2020-09-02 18:12:17 +01:00
Julian M. Kunkel f275671cc9 AIORI POSIX AIO support. Collect ops until granularity is reached, then submit pending IOs. Synchronize latest on close. Doesn't work with data verification and reuses the existing buffer. The implementation shows the potential AIO may have.
Extract also the POSIX header from AIORI to allow better reuse. #240
2020-07-21 16:18:50 +01:00
Julian M. Kunkel 81a7a3ab81 Renamed existing S3 impl. which uses (apparently) outdated library. Added dummy support for libS3 with library search. 2020-06-30 13:41:59 +01:00
Julian M. Kunkel 82417128cd Extracted file size routine that is replicated in each module into IOR removing the MPI dependency from AIORI. 2020-06-30 12:33:56 +01:00
Julian M. Kunkel 145c71f7c3 Trivial cleanup: Extracted debug-related stuff into new header. 2020-06-28 17:16:56 +01:00
Julian Kunkel d5de9bcc77
Merge branch 'master' into master 2020-06-24 09:50:03 +01:00
Julian M. Kunkel eb2cd27fac Removed offset from IOR structure. 2020-06-10 17:47:07 +01:00
Julian M. Kunkel 5663593919 Moved generic IOR information to "hint" structure. Backends should only use the "hints" if set. 2020-05-31 12:50:03 +01:00
Julian M. Kunkel 8098c3740a Refactored fd into aiori_fd_t for type safety. 2020-05-31 12:12:49 +01:00
Julian M. Kunkel c4c19d3245 Fixed type typo in name. 2020-05-31 11:58:34 +01:00
Julian M. Kunkel 294b8891e6 Created dummy option type to provide type safety. Fixed various call issues for options. 2020-05-31 11:50:15 +01:00
Julian M. Kunkel e91b79cea0 Moved Lustre/GPFS/BeeGFS options into POSIX backend. Needs testing. Additional minor fixes. 2020-05-30 20:09:37 +01:00
Julian M. Kunkel 1890aaaeb0 Started to move IOR specific knowledge out of AIORI.
For now, and compatibility, IOR options can still be set/internally accessed using the backends init_xfer_options.
This should be removed in the long run to strip away this dependency.
2020-05-30 18:19:48 +01:00
Mark Nelson 9649a0c520 src/aiori-CEPHFS: New libcephfs backend
Signed-off-by: Mark Nelson <mnelson@redhat.com>
2020-03-10 10:19:41 -05:00
Julian M. Kunkel c702a98376 Strict prototypes for AIORI 2019-12-22 11:21:40 +00:00
Jean-Yves VET 9da36abaca Fix confusing output of mknod
Context: mknod does not return a file descriptor.
2019-12-16 14:55:26 +01:00
Afrian Jackson fa25c31994 Adding PMDK functionality 2019-10-31 15:29:09 +00:00
Mohamad Chaarawi e9dd5fe15e Merge branch 'master' into daos-devel
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>

Conflicts:
	configure.ac
	src/Makefile.am
	src/aiori-MPIIO.c
	src/aiori.c
	src/aiori.h
	src/ior.c
	src/mdtest.c
	src/option.c
2019-09-09 19:55:56 +00:00
Julian Kunkel 0921556456
Merge pull request #184 from hpc/feature-aiori
Feature aiori
2019-09-09 15:20:43 +01:00
Julian M. Kunkel c83edfe39b Extracted check function into aiori. #24. #177 2019-09-01 15:59:52 +01:00
Julian M. Kunkel e3db1759b2 Moded sync() to aiori backend. 2019-09-01 15:47:42 +01:00
Mohamad Chaarawi 262d35d87e - Merge branch 'master' into daos-devel
- fix bugs with cont destroy.
- add destroy option to DFS driver
- share pool and container handle with DFS driver, and allow multi rank access

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>

Conflicts:
	src/ior.c
	src/mdtest.c
	src/option.c
	src/parse_options.c
2019-06-24 21:26:15 +00:00
Osamu Tatebe 6cae44b512 abstract I/O interface for Gfarm file system 2019-06-21 21:05:41 +09:00
Gu Zheng a6f8388f9f add mknod support
Add new option '-k' to support file creation test with mknod,
which is widely used in lustre.

Signed-off-by: Gu Zheng <gzheng@ddn.com>
2019-05-23 13:41:46 +08:00
Julian M. Kunkel 21405ed924 Updated MMap module to option parser. 2019-03-27 22:32:59 +00:00
Julian M. Kunkel 20e960d020 Support the setting of module-specific options per test. 2019-03-27 20:04:48 +00:00
Jean-Yves VET 110153cf3a mdtest: Enable IME native interface
This patch makes mdtest allow IME native interface.
It also defines a generic way to enable mdtest with each
backend.
2019-02-12 15:50:51 +01:00
Mohamad Chaarawi 2c87b5e0f5 Merge remote-tracking branch 'upstream/master' into daos
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>

Conflicts:
	src/aiori.c
	src/aiori.h
	src/ior.c
	src/mdtest-main.c
	src/mdtest.c
	src/option.c
2019-01-24 00:31:12 +00:00
Julian M. Kunkel 6f7576aa8a Changed the parser.
The parser now supports concurrent parsing of all plugin options.
Moved HDF5 collective_md option into the backend as an example.
Example: ./src/ior -a dummy --dummy.delay-xfer=50000
2018-10-11 18:31:21 +01:00
Jean-Yves VET f472162784 Support old legacy name in backends
Context: Some backends may have used different names in
the past (like IME backend use to be IM). Legacy scripts
may break.

This patch adds a legacy name option in the aiori structure.
Both name and legacy name work to select the interface.
But the following warning is printed if the legacy name is used:

ior WARNING: [legacy name] backend is deprecated use [name] instead.
2018-09-24 11:51:00 +02:00
Jean-Yves VET 2bcaea564e Initialize only the required backends
Context: IOR initializes all available backends. If one
backend fails to initialize IOR cannot be used.

This patch makes IOR initialize only the backends
which will be used. The initialization is done after
that the parameters are checked so that the help message
can still be dispayed is something goes wrong.
2018-09-18 19:24:29 +02:00
Mohamad Chaarawi 46ff4e26be Merge remote branch 'daos-stack/daos'
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>

Conflicts:
	configure.ac
	src/Makefile.am
	src/aiori.c
	src/aiori.h
	src/ior.c
	src/mdtest.c
	src/parse_options.c
2018-08-29 21:14:19 +00:00
Mohamad Chaarawi f4b03efd72 Add the DAOS ior driver.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-08-27 17:22:38 +00:00
Mohamad Chaarawi 5fb850c810 - update the DFS driver to latest DFS API.
- update cmd line options to add DAOS Pool and Container uuid and SVCL
- Add init/finalize backend functions.

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-08-23 21:58:53 +00:00
Julian M. Kunkel 4dcf36c1a4 Help text now outputs *compiled* APIs. 2018-08-15 17:07:17 +01:00
Jean-Yves VET 44ce8e5aaf Add backend for DDN's Infinite Memory Engine (IME)
This patch adds the support of IME Native interface
as a new AIORI backend.
2018-08-07 11:41:07 +01:00
Julian M. Kunkel 5ef95675bf AIORI initialize() and finalize() functions. 2018-07-14 09:29:08 +01:00
Julian M. Kunkel 54e47cf729 IOR: use new option parser.
TODO: Parse "-O" options
2018-07-14 08:41:35 +01:00
Julian M. Kunkel 0835191372 Option parser replaced.
Benefits: shows currently set options when used with -h
Supports options for child modules, example for aiori-dummy works, RADOS untested.
2018-07-12 18:09:13 +01:00
Julian M. Kunkel 205b20f0f4 Bugfix merge 2018-07-12 16:13:40 +01:00
Julian Kunkel 060c47e80a
Merge branch 'testing' into rados-hacks 2018-07-12 15:53:04 +01:00
Shane Snyder 7981691a63 add a new AIORI backend for RADOS
inform aiori interface about RADOS backend

stubbed out aiori backend for rados

additions to get RADOS backend compiling/linking

first cut at rados create/open patha

make sure to return RADOS oid on open/create

implement rados xfer path for WRITE

refactor + implement getfilesize and close

remember to use read_op interface for stat

implement RADOS delete function

don't error in RADOS_Delete for now

implement RADOS set_version

handle open/create flags appropriately

cleanup RADOS error handling

implement read/readcheck/writecheck for RADOS

rados doesn't support directio

implement unsupported aiori ops for RADOS

implement RADOS access call

define rados types if no rados support
2018-07-12 09:36:49 -05:00
Julian M. Kunkel 1a3c92718b Fix merge 2018-07-08 12:13:08 +01:00
Shane Snyder 7d3c7e901e expose generic aiori_ calls for access, etc.
Signed-off-by: Julian M. Kunkel <juliankunkel@googlemail.com>
2018-07-08 12:09:44 +01:00
Julian M. Kunkel 378789737f Dummy AIORI driver 2018-07-07 13:25:17 +01:00
Mohamad Chaarawi 8d0cddd21e Add DFS ior/mdtest driver
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-06-13 18:37:37 +00:00
Li Dongyang 6ba22a19e4 Add mmap as a new AIORI backend
It shares the create/open/delete/set_version/get_file_size
functions with POSIX backend.
The mmap backend also supports fsync and fsyncPerWrite options,
and it will use msync() instead and fsync().

Signed-off-by: Li Dongyang <dongyangli@ddn.com>
2018-05-11 12:13:42 +10:00
Shane Snyder 60243e0277 expose MPIIO access and delete calls 2018-04-25 17:32:16 -05:00