Commit Graph

30 Commits (master)

Author SHA1 Message Date
Mohamad Chaarawi 7c201c0d9c
HDF5: fix segfault in HDF5 driver after latest AIORI changes (#255)
- the hints structure was not getting initialized in MPIIO when HDF5
backend is used. Since HDF5 utilizes the MPIIO backend, this causes
a segfault.

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2020-09-02 18:12:17 +01:00
Julian M. Kunkel 82417128cd Extracted file size routine that is replicated in each module into IOR removing the MPI dependency from AIORI. 2020-06-30 12:33:56 +01:00
Julian M. Kunkel eb2cd27fac Removed offset from IOR structure. 2020-06-10 17:47:07 +01:00
Julian M. Kunkel 5663593919 Moved generic IOR information to "hint" structure. Backends should only use the "hints" if set. 2020-05-31 12:50:03 +01:00
Julian M. Kunkel 8098c3740a Refactored fd into aiori_fd_t for type safety. 2020-05-31 12:12:49 +01:00
Julian M. Kunkel c4c19d3245 Fixed type typo in name. 2020-05-31 11:58:34 +01:00
Julian M. Kunkel 294b8891e6 Created dummy option type to provide type safety. Fixed various call issues for options. 2020-05-31 11:50:15 +01:00
Julian M. Kunkel 930ccdc68d Updated HDF5 to new module specification. 2020-05-30 19:01:20 +01:00
Julian M. Kunkel cf197d5e38 Moved broken collectiveMetadata option into HDF5 backend module. #154 2019-05-22 10:21:18 +01:00
Julian M. Kunkel 6c5952906b Dry-run option to test the configuration / scripts. 2018-10-11 19:58:30 +01:00
Julian Kunkel acee84bedc
Merge pull request #88 from roblatham00/hdf5-coll-md
HDF5: option to enable collective metadata
2018-10-02 16:13:05 +01:00
Jean-Yves VET f472162784 Support old legacy name in backends
Context: Some backends may have used different names in
the past (like IME backend use to be IM). Legacy scripts
may break.

This patch adds a legacy name option in the aiori structure.
Both name and legacy name work to select the interface.
But the following warning is printed if the legacy name is used:

ior WARNING: [legacy name] backend is deprecated use [name] instead.
2018-09-24 11:51:00 +02:00
Rob Latham 7d2464f733 HDF5: option to enable collective metadata
In HDF5-1.10 the library has an option to read and write the metadata of a file collectively.  One has to opt-in to this optimization with a property list but particularly at large scale it can have tremendous performance implications.

Further information:
- https://support.hdfgroup.org/HDF5/docNewFeatures/CollMDataIO/RFC-CollectiveMetadataReads.pdf
- https://support.hdfgroup.org/HDF5/doc/RM/RM_H5P.html#Property-SetAllCollMetadataOps
2018-09-20 11:12:08 -05:00
Rob Latham 982f266a73 clean up hints in HDF5 error cases 2018-08-28 13:30:51 -05:00
Julian M. Kunkel b613bf9396 AIORI update to initialization and fix HDF5 with changed API. 2018-07-14 10:47:25 +01:00
Shane Snyder 8069e018e3 make ncmpi/hdf5 use mpiio access/delete
Signed-off-by: Julian M. Kunkel <juliankunkel@googlemail.com>
2018-07-08 12:13:38 +01:00
Nathan Hjelm 4dc0081628 First cut at using the IOR backends for mdtest
This commit makes changes to the AIORI backends to add support for
abstacting statfs, mkdir, rmdir, stat, and access. These new
abstractions are used by a modified mdtest. Some changes:

 - Require C99. Its 2017 and most compilers now support C11. The
   benefits of using C99 include subobject naming (for aiori backend
   structs), and fixed size integers (uint64_t). There is no reason to
   use the non-standard long long type.

 - Moved some of the aiori code into aiori.c so it can be used by both
   mdtest and ior.

 - Code cleanup of mdtest. This is mostly due to the usage of the IOR
   backends rather than a mess of #if code.

Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
2017-10-19 15:29:12 -06:00
Rob Latham a4a5576858 housekeeping: remove unused array
the "report an error" macros defined an array that was not used.
2015-12-07 15:40:43 -06:00
Jeff Inman 37738dab26 Numerous changes to file-modes, small build-tweaks, and a tweak to aiori-S3.c
(Only rank-0 should create the bucket, if it doesn't already exist.)

Prepping this for a push (as an experimental S3 build) to github.
2015-05-19 09:36:28 -06:00
Rob Latham 6a7ddfe60c Define H5_USER_16_API to enable HDF5 backwards compatibility
Since aiori-HDF5 uses old 1.6-style function calls (create, open, etc),
turn on the backwards compatiblity here (the one place HDF5 calls are
made).
2013-07-18 15:01:15 -07:00
Robert Latham b76238bbd5 Remove bad "static static" function declaration
Fix some obvious compile errors for IOR hdf5 driver.

fixes #5
2012-09-12 14:43:24 -07:00
Christopher J. Morrone 8e06f07b28 Add header to set code preferences for vi and emacs. 2012-01-08 19:43:41 -08:00
Christopher J. Morrone a60ad0b088 Impose proper file scoping and header usage.
Clean up the header files to only contain those things that
need to be shared between .c files.

Functions that are not shared are now declared static to
make their file scope explicit.  Functions that ARE shared
are declared in appropriate headers.

I am not going to claim that I caugh everything, but at
least it is a good start.
2011-11-11 17:30:20 -08:00
Christopher J. Morrone e7fea4f102 Fix prototype locations 2011-11-11 15:18:17 -08:00
Christopher J. Morrone 0072f26946 More simple code cleanup 2011-11-11 15:11:45 -08:00
Christopher J. Morrone 240e5ce93f Change the the One True Formatting (Linux style)
Ran the main ior code through "indent -linux --no-tabs".
2011-11-11 15:11:45 -08:00
Christopher J. Morrone 4cdbf3aad7 Improve error messages.
Other minor code style fixes.
2011-11-10 19:01:34 -08:00
Christopher J. Morrone ca832bb46e Give ior a more reasonable plugin system. 2011-10-27 18:46:18 -07:00
Christopher J. Morrone 5eed1723f0 Remove comment cruft from beginnings of files. 2011-10-27 17:45:14 -07:00
Christopher J. Morrone 02fda88fd0 Begin conversion to autoconf 2011-10-27 17:45:09 -07:00