diff --git a/ChangeLog b/ChangeLog deleted file mode 100755 index f30050a..0000000 --- a/ChangeLog +++ /dev/null @@ -1,137 +0,0 @@ -Changes in IOR-3.0.0 - - * Reorganization of the build system. Now uses autoconf/automake. - N.B. Windows suport is not included. Patches welcome. - * Much code refactoring. - * Final summary table is printed after all tests have finished. - * Error messages significantly improved. - * Drop all "undocumented changes". If they are worth having, they - need to be implemented well and documented. - -Changes in IOR-2.10.3 - * bug 2962326 "Segmentation Fault When Summarizing Results" fixed - * bug 2786285 "-Wrong number of parameters to function H5Dcreate" fixed - (NOTE: to compile for HDF5 1.6 libs use "-D H5_USE_16_API") - * bug 1992514 "delay (-d) doesn't work" fixed -Contributed by demyn@users.sourceforge.net - * Ported to Windows. Required changes related to 'long' types, which on Windows - are always 32-bits, even on 64-bit systems. Missing system headers and - functions acount for most of the remaining changes. - New files for Windows: - - IOR/ior.vcproj - Visual C project file - - IOR/src/C/win/getopt.{h,c} - GNU getopt() support - See updates in the USER_GUIDE for build instructions on Windows. - * Fixed bug in incrementing transferCount - * Fixed bugs in SummarizeResults with mismatched format specifiers - * Fixed inconsistencies between option names, -h output, and the USER_GUIDE. - -Changes in IOR-2.10.2: -Hodson, 8/18/2008: - - * extend existing random I/O capabilities and enhance performance - output statistics. - -Fixes in IOR-2.10.1: - * added '-J' setAlignment option for HDF5 alignment in bytes; default value - is 1, which does not set alignment - * changed how HDF5 and PnetCDF calculate performance -- formerly each used - the size of the stat()'ed file; changed it to be number of data bytes - transferred. these library-generated files can have large headers and - filler as well as sparse file content - * found potential overflow error in cast -- using IOR_offset_t, not int now - * replaced HAVE_HDF5_NO_FILL macro to instead directly check if H5_VERS_MAJOR - H5_VERS_MINOR are defined; if so, then must be HDF5-1.6.x or higher for - no-fill usage. - * corrected IOR_GetFileSize() function to point to HDF5 and NCMPI versions of - IOR_GetFileSize() calls - * changed the netcdf dataset from 1D array to 4D array, where the 4 dimensions - are: [segmentCount][numTasksWorld][numTransfers][transferSize] - This patch from Wei-keng Liao allows for file sizes > 4GB (provided no - single dimension is > 4GB). - * finalized random-capability release - * changed statvfs() to be for __sun (SunOS) only - * retired Python GUI - -Fixes in IOR-2.10.0.1: - * Cleaned up WriteOrRead(), reducing much to smaller functions. - * Added random capability for transfer offset. - * modified strtok(NULL, " \t\r\n") in ExtractHints() so no trailing characters - * added capability to set hints in NCMPI - -Fixes in IOR-2.9.6.1: - * for 'pvfs2:' filename prefix, now skips DisplayFreeSpace(); formerly this - caused a problem with statvfs() - * changed gethostname() to MPI_Get_processor_name() so for certain cases - when gethostname() would only return the frontend node name - * added SunOS compiler settings for makefile - * updated O_DIRECT usage for SunOS compliance - * changed statfs() to instead use statvfs() for SunOS compliance - * renamed compiler directive _USE_LUSTRE to _MANUALLY_SET_LUSTRE_STRIPING - -Fixes in IOR-2.9.5: - * Wall clock deviation time relabeled to be "Start time skew across all tasks". - * Added notification for "Using reorderTasks '-C' (expecting block, not cyclic, - task assignment)" - * Corrected bug with read performance with stonewalling (was using full size, - stat'ed file instead of bytes transfered). - -Fixes in IOR-2.9.4: - * Now using IOR_offset_t instead of int for tmpOffset in IOR.c:WriteOrRead(). - Formerly, this would cause error in file(s) > 2GB for ReadCheck. The - more-commonly-used WriteCheck option was not affected by this. - -Fixes in IOR-2.9.3: - * Changed FILE_DELIMITER from ':' to '@'. - * Any time skew between nodes is automatically adjusted so that all nodes - are calibrated to the root node's time. - * Wall clock deviation time is still reported, but have removed the warning - message that was generated for this. (Didn't add much new information, - just annoyed folks.) - * The '-j' outlierThreshold option now is set to 0 (off) as default. To use - this, set to a positive integer N, and any task who's time (for open, - access, etc.) is not within N seconds of the mean of all the tasks will show - up as an outlier. - -Fixes in IOR-2.9.2: - * Simple cleanup of error messages, format, etc. - * Changed error reporting so that with VERBOSITY=2 (-vv) any error encountered - is displayed. Previously, this was with VERBOSITY=3, along with full test - parameters, environment, and all access times of operations. - * Added deadlineForStonewalling option (-D). This option is used for measuring - the amount of data moved in a fixed time. After the barrier, each task - starts its own timer, begins moving data, and the stops moving data at a - prearranged time. Instead of measuring the amount of time to move a fixed - amount of data, this option measures the amount of data moved in a fixed - amount of time. The objective is to prevent tasks slow to complete from - skewing the performance. - -Fixes in IOR-2.9.1: - * Updated test script to run through file1:file2 cases. - * Program exit code is now total numbers of errors (both writecheck and - readcheck for all iterations), unless quitOnError (-q) is set. - * For various failure situations, replace abort with warning, including: - - failed uname() for platform name now gives warning - - failed unlink() of data file now gives warning - - failed fsync() of data file now gives warning - - failed open() of nonexistent script file now gives warning - * Changed labelling for error checking output to be (hopefully) more - clear in details on errors for diagnostics. - * Another fix for -o file1:file2 option. - * Corrected bug in GetTestFileName() -- now possible to handle -o file1:file2 - cases for file-per-proc. - -Fixes in IOR-2.9.0: - * Improved checkRead option to reread data from different node (to avoid - cache) and then compare both reads. - * Added outlierThreshold (-j) option to give warning if any task is more than - this number of seconds from the mean of all participating tasks. If so, the - task is identified, its time (start, elapsed create, elapsed transfer, - elapsed close, or end) is reported, as is the mean and standard deviation for - all tasks. The default for this is 5, i.e. any task not within 5 seconds of - the mean for those times is displayed. This value can be set with - outlierThreshold= or -j . - * Correct for clock skew between nodes - if skew greater than wallclock - deviation threshold (WC_DEV_THRESHOLD) in seconds, then broadcast root - node's timestamp and adjust by the difference. WC_DEV_THRESHOLD currently - set to 5. - * Added a Users Guide. diff --git a/META b/META index e7b78ff..a4ea873 100755 --- a/META +++ b/META @@ -1,3 +1,3 @@ Package: ior -Version: 3.1.0 +Version: 3.2.0 Release: 0 diff --git a/NEWS b/NEWS index 368697a..dedb82a 100644 --- a/NEWS +++ b/NEWS @@ -1,9 +1,199 @@ IOR NEWS -======== +================================================================================ -Last updated 2017-06 +Last updated 2018-08 -3.0.2 +Version 3.2 +-------------------------------------------------------------------------------- +New major features: + +- mdtest now included as a frontend for the IOR aiori backend (Nathan Hjelm, + LANL) +- Added mmap AIORI (Li Dongyang, DDN) +- Added RADOS AIORI (Shane Snyder, ANL) +- Added IME AIORI (Jean-Yves Vet, DDN) +- Added stonewalling for mdtest (Julian Kunkel, U Reading) + +New minor features: + +- Dropped support for PLFS AIORI (John Bent) +- Added stoneWallingWearOut functionality to allow stonewalling to ensure that + each MPI rank writes the same amount of data and captures the effects of slow + processes (Julian Kunkel, DKRZ) +- Added support for JSON output (Enno Zickler, U Hamburg; Julian Kunkel, U + Reading) +- Added dummy AIORI (Julian Kunkel, U Reading) +- 0870ad78b3b022bcd30392a8990ba4e5090d6e34 + +General user improvements: + +- BeeGFS parameter support (Oliver Steffen, ThinkParQ) +- Semantics of `-R` now compares to expected signature (`-G`) (Julian Kunkel, + DKRZ) +- Improved macOS support for ncmpi (Vinson Leung) +- Added more complete documentation (Enno Zickler, U Hamburg) +- Assorted bugfixes and code refactoring (Adam Moody, LLNL; Julian Kunkel, U + Reading; Enno Zickler, U Hamburg; Nathan Hjelm, LANL; Rob Latham, ANL; + Jean-Yves Vet, DDN) +- More robust support for non-POSIX-backed AIORIs (Shane Snyder, ANL) + +General developer improvements: + +- Improvements to build process (Nathan Hjelm, LANL; Ian Kirker, UCL) +- Added continuous integration support (Enno Zickler, U Hamburg) +- Added better support for automated testing (Julian Kunkel, U Reading) + +Version 3.0.2 +-------------------------------------------------------------------------------- - IOR and mdtest now share a common codebase. This will make it easier -run performance benchmarks on new hardware. + run performance benchmarks on new hardware. +- Note: this version was never properly released + +Version 3.0.0 +-------------------------------------------------------------------------------- +- Reorganization of the build system. Now uses autoconf/automake. N.B. Windows + support is not included. Patches welcome. +- Much code refactoring. +- Final summary table is printed after all tests have finished. +- Error messages significantly improved. +- Drop all "undocumented changes". If they are worth having, they need to be + implemented well and documented. + +Version 2.10.3 +-------------------------------------------------------------------------------- +- bug 2962326 "Segmentation Fault When Summarizing Results" fixed +- bug 2786285 "-Wrong number of parameters to function H5Dcreate" fixed + (NOTE: to compile for HDF5 1.6 libs use "-D H5_USE_16_API") +- bug 1992514 "delay (-d) doesn't work" fixed + +Contributed by demyn@users.sourceforge.net +- Ported to Windows. Required changes related to 'long' types, which on Windows + are always 32-bits, even on 64-bit systems. Missing system headers and + functions acount for most of the remaining changes. + New files for Windows: + - IOR/ior.vcproj - Visual C project file + - IOR/src/C/win/getopt.{h,c} - GNU getopt() support + See updates in the USER_GUIDE for build instructions on Windows. +- Fixed bug in incrementing transferCount +- Fixed bugs in SummarizeResults with mismatched format specifiers +- Fixed inconsistencies between option names, -h output, and the USER_GUIDE. + +Version 2.10.2 +-------------------------------------------------------------------------------- +- Extend existing random I/O capabilities and enhance performance + output statistics. (Hodson, 8/18/2008) + +Version 2.10.1 +-------------------------------------------------------------------------------- +- Added '-J' setAlignment option for HDF5 alignment in bytes; default value + is 1, which does not set alignment +- Changed how HDF5 and PnetCDF calculate performance -- formerly each used + the size of the stat()'ed file; changed it to be number of data bytes + transferred. these library-generated files can have large headers and + filler as well as sparse file content +- Found potential overflow error in cast -- using IOR_offset_t, not int now +- Replaced HAVE_HDF5_NO_FILL macro to instead directly check if H5_VERS_MAJOR + H5_VERS_MINOR are defined; if so, then must be HDF5-1.6.x or higher for + no-fill usage. +- Corrected IOR_GetFileSize() function to point to HDF5 and NCMPI versions of + IOR_GetFileSize() calls +- Changed the netcdf dataset from 1D array to 4D array, where the 4 dimensions + are: [segmentCount][numTasksWorld][numTransfers][transferSize] + This patch from Wei-keng Liao allows for file sizes > 4GB (provided no + single dimension is > 4GB). +- Finalized random-capability release +- Changed statvfs() to be for __sun (SunOS) only +- Retired Python GUI + +Version 2.10.0.1 +-------------------------------------------------------------------------------- +- Cleaned up WriteOrRead(), reducing much to smaller functions. +- Added random capability for transfer offset. +- Modified strtok(NULL, " \t\r\n") in ExtractHints() so no trailing characters +- Added capability to set hints in NCMPI + +Version 2.9.6.1 +-------------------------------------------------------------------------------- +- For 'pvfs2:' filename prefix, now skips DisplayFreeSpace(); formerly this + caused a problem with statvfs() +- Changed gethostname() to MPI_Get_processor_name() so for certain cases + when gethostname() would only return the frontend node name +- Added SunOS compiler settings for makefile +- Updated O_DIRECT usage for SunOS compliance +- Changed statfs() to instead use statvfs() for SunOS compliance +- Renamed compiler directive _USE_LUSTRE to _MANUALLY_SET_LUSTRE_STRIPING + +Version 2.9.5 +-------------------------------------------------------------------------------- +- Wall clock deviation time relabeled to be "Start time skew across all tasks". +- Added notification for "Using reorderTasks '-C' (expecting block, not cyclic, + task assignment)" +- Corrected bug with read performance with stonewalling (was using full size, + stat'ed file instead of bytes transfered). + +Version 2.9.4 +-------------------------------------------------------------------------------- +- Now using IOR_offset_t instead of int for tmpOffset in IOR.c:WriteOrRead(). + Formerly, this would cause error in file(s) > 2GB for ReadCheck. The + more-commonly-used WriteCheck option was not affected by this. + +Version 2.9.3 +-------------------------------------------------------------------------------- +- Changed FILE_DELIMITER from ':' to '@'. +- Any time skew between nodes is automatically adjusted so that all nodes + are calibrated to the root node's time. +- Wall clock deviation time is still reported, but have removed the warning + message that was generated for this. (Didn't add much new information, + just annoyed folks.) +- The '-j' outlierThreshold option now is set to 0 (off) as default. To use + this, set to a positive integer N, and any task who's time (for open, + access, etc.) is not within N seconds of the mean of all the tasks will show + up as an outlier. + +Version 2.9.2 +-------------------------------------------------------------------------------- +- Simple cleanup of error messages, format, etc. +- Changed error reporting so that with VERBOSITY=2 (-vv) any error encountered + is displayed. Previously, this was with VERBOSITY=3, along with full test + parameters, environment, and all access times of operations. +- Added deadlineForStonewalling option (-D). This option is used for measuring + the amount of data moved in a fixed time. After the barrier, each task + starts its own timer, begins moving data, and the stops moving data at a + prearranged time. Instead of measuring the amount of time to move a fixed + amount of data, this option measures the amount of data moved in a fixed + amount of time. The objective is to prevent tasks slow to complete from + skewing the performance. + +Version 2.9.1 +-------------------------------------------------------------------------------- +- Updated test script to run through file1:file2 cases. +- Program exit code is now total numbers of errors (both writecheck and + readcheck for all iterations), unless quitOnError (-q) is set. +- For various failure situations, replace abort with warning, including: + - failed uname() for platform name now gives warning + - failed unlink() of data file now gives warning + - failed fsync() of data file now gives warning + - failed open() of nonexistent script file now gives warning +- Changed labelling for error checking output to be (hopefully) more + clear in details on errors for diagnostics. +- Another fix for -o file1:file2 option. +- Corrected bug in GetTestFileName() -- now possible to handle -o file1:file2 + cases for file-per-proc. + +Version 2.9.0 +-------------------------------------------------------------------------------- +- Improved checkRead option to reread data from different node (to avoid cache) + and then compare both reads. +- Added outlierThreshold (-j) option to give warning if any task is more than + this number of seconds from the mean of all participating tasks. If so, the + task is identified, its time (start, elapsed create, elapsed transfer, + elapsed close, or end) is reported, as is the mean and standard deviation for + all tasks. The default for this is 5, i.e. any task not within 5 seconds of + the mean for those times is displayed. This value can be set with + outlierThreshold= or -j . +- Correct for clock skew between nodes - if skew greater than wallclock + deviation threshold (WC_DEV_THRESHOLD) in seconds, then broadcast root + node's timestamp and adjust by the difference. WC_DEV_THRESHOLD currently + set to 5. +- Added a Users Guide.