Commit Graph

146 Commits (d4658f22be1d1b0a2e3b3f169489d6533da27ef7)

Author SHA1 Message Date
timchenxiaoyu d4658f22be fix example error (#53) 2017-06-06 22:26:01 +02:00
Klaus Post dde6ad55c5 Set correct field in WithMinSplitSize
Fixes #51
2017-05-28 12:38:06 +02:00
Klaus Post 5abf0ee302 Add options (#46)
* Add options

Make constants changeable as options.

The API remains backwards compatible.

* Update documentation.

* Fix line endings

* fmt

* fmt

* Use functions for parameters.

Much neater.
2017-02-19 11:13:22 +01:00
Klaus Post c056598956 Merge pull request #39 from jesselucas/patch-1
Update README.md to fix small typos.
2017-01-05 16:18:16 +01:00
Jesse Lucas ff2f89b6ca Update README.md to fix small typos. 2017-01-05 00:16:24 -05:00
Klaus Post d0a56f72c0 Update README.md 2016-10-28 09:13:20 +02:00
Klaus Post 9998b4cb21 Update README.md 2016-10-28 09:00:26 +02:00
Peter C c54154da9e Add Inverse Matrix caching in a Thread-Safe Lookup Tree (#36)
* Add matrix inversion caching
* Benchmark and Parallel Benchmark tests for Reconstruct
2016-09-12 21:31:07 +02:00
Klaus Post fac1884d47 Merge pull request #34 from muesli/master
Make Join return an error if a reconstruction is required first
2016-08-22 14:13:59 +02:00
Christian Muehlhaeuser b1c8b4b073 Make Join return an error if a reconstruction is required first
If one or more required data shards are still nil and we can't correctly join
them before a reconstruction, return ErrReconstructRequired.
2016-08-05 19:23:08 +02:00
Klaus Post 9b772b54b3 Merge pull request #30 from hackintoshrao/matrix-test
Tests: Coverage and enhancement for matrix_test.go
2016-08-01 19:20:09 +02:00
Klaus Post 5eab34543e Merge pull request #31 from klauspost/use-gofmt-for-tests
Use gofmt instead of goimports for format teting
2016-08-01 19:19:22 +02:00
Klaus Post 6978874dc1 Use gofmt instead of goimports for format teting. 2016-08-01 18:15:38 +02:00
Karthic Rao f89ef0c3a3 Coverage and enhancement for matrix_test.go 2016-07-26 15:41:13 +05:30
Frank 467733eb9c Add generated byte assembler using asm2plan9s
Add recompilable assembler using asm2plan9s
2016-07-06 21:06:00 +02:00
Klaus Post cebd11778b Merge pull request #26 from fwessels/removeJMP
Removed unnecessary JMP instruction
2016-07-06 11:08:32 +02:00
frankw d4000061f2 Removed unnecessary JMP instruction 2016-07-06 09:39:02 +02:00
Klaus Post dd5af9e1ff Merge pull request #25 from gyuho/slice_allocation
matrix.go: preallocate slice for String() method
2016-07-05 16:08:46 +02:00
Gyu-Ho Lee 7a64c70cbc matrix.go: preallocate slice for String() method
To avoid slice growth with append operations.
2016-06-04 22:12:17 -07:00
Klaus Post 2fde63d288 Merge pull request #24 from harshavardhana/parity
Add checks for data and parity to not exceed 255 shards in total.
2016-06-04 08:05:20 +02:00
Klaus Post b118551f15 Merge pull request #23 from harshavardhana/appveyor
Submit a new appveyor CI config.
2016-06-04 08:04:55 +02:00
Harshavardhana ba30981088 Add checks for data and parity to not exceed 255 shards in total.
Fixes #16
2016-06-03 01:31:01 -07:00
Harshavardhana 25fa063ea5 Submit a new appveyor CI config.
Fixes #21
2016-06-03 00:57:56 -07:00
Klaus Post 4fadad8564 Update reedsolomon.go
Fix comment
2016-05-01 12:00:51 +02:00
Klaus Post ed06f926b9 Merge pull request #20 from harshavardhana/fix
ErrShortData shouldn't be returned for data less than dataShards.
2016-05-01 11:58:47 +02:00
Klaus Post d61a5ff262 Merge pull request #19 from harshavardhana/simple
use bytes.Equal rather than bytes.Compare
2016-05-01 11:57:20 +02:00
Harshavardhana df175d2921 ErrShortData shouldn't be returned for data less than dataShards.
The reasoning behind this is that if we have a data block number
of 10, and parity of 10.  Restricting input such that files of
size < 10Bytes should be errored out doesn't seem like the right
approach.

Most erasure subsystems will have static data and parity blocks,
in such case erroring out is not correct since reedsolomon itself
doesn't provide this limitation (please correct me here if i am
wrong :-)).

So removing the check itself is not a problem since most of the
data after the split would be padded with zeros, which is okay
and should be left as application optimization if they wish to
pack small files in this range.

ErrShortData will be still returned in case if the size of data
is empty, or in case of streaming if the size == 0.
2016-04-29 20:38:45 -07:00
Harshavardhana 0b630aea27 use bytes.Equal rather than bytes.Compare 2016-04-29 14:12:03 -07:00
Klaus Post 55f0000302 Merge pull request #18 from xiaost/optimized-goroutines-number
Optimized encoding & decoding goroutines number
2016-04-12 10:54:07 +02:00
xiaost 4048a541c8 Optimized encoding & decoding goroutines number
hardware: E5-2630 v2  (Intel x86-64 with ssse3)
software: linux, go1.6, GOMAXPROCS=2

Performances                          before          after         change

BenchmarkEncode10x2x10000-2           2884.95 MB/s    2837.93 MB/s  0.98x
BenchmarkEncode100x20x10000-2          593.93 MB/s     577.17 MB/s  0.97x
BenchmarkEncode17x3x1M-2              2903.74 MB/s    5197.99 MB/s  1.80x
BenchmarkEncode10x4x16M-2             1992.13 MB/s    3689.69 MB/s  1.85x
BenchmarkEncode5x2x1M-2               2883.78 MB/s    7506.19 MB/s  2.60x
BenchmarkEncode10x2x1M-2              3205.63 MB/s    7848.12 MB/s  2.45x
BenchmarkEncode10x4x1M-2              2218.35 MB/s    3998.35 MB/s  1.80x
BenchmarkEncode50x20x1M-2              579.24 MB/s     641.08 MB/s  1.11x
BenchmarkEncode17x3x16M-2             2652.36 MB/s    4775.41 MB/s  1.80x
BenchmarkVerify10x2x10000-2           1327.27 MB/s    1837.41 MB/s  1.38x
BenchmarkVerify50x5x50000-2           1481.89 MB/s    2684.57 MB/s  1.81x
BenchmarkVerify10x2x1M-2              1553.91 MB/s    5704.71 MB/s  3.67x
BenchmarkVerify5x2x1M-2                939.90 MB/s    4949.30 MB/s  5.26x
BenchmarkVerify10x4x1M-2               956.89 MB/s    3191.01 MB/s  3.33x
BenchmarkVerify50x20x1M-2              490.49 MB/s     823.46 MB/s  1.68x
BenchmarkVerify10x4x16M-2             1078.03 MB/s    3196.97 MB/s  2.97x
BenchmarkStreamEncode10x2x10000-2        2.40 MB/s      12.10 MB/s  5.04x
BenchmarkStreamEncode100x20x10000-2      6.72 MB/s      10.72 MB/s  1.60x
BenchmarkStreamEncode17x3x1M-2         390.75 MB/s     845.08 MB/s  2.16x
BenchmarkStreamEncode10x4x16M-2       1175.93 MB/s    1803.71 MB/s  1.53x
BenchmarkStreamEncode5x2x1M-2          207.85 MB/s     790.02 MB/s  3.80x
BenchmarkStreamEncode10x2x1M-2         296.77 MB/s     872.41 MB/s  2.94x
BenchmarkStreamEncode10x4x1M-2         264.43 MB/s     699.25 MB/s  2.64x
BenchmarkStreamEncode50x20x1M-2        284.93 MB/s     414.65 MB/s  1.46x
BenchmarkStreamEncode17x3x16M-2       1439.13 MB/s    1933.42 MB/s  1.34x
BenchmarkStreamVerify10x2x10000-2        2.33 MB/s      12.07 MB/s  5.18x
BenchmarkStreamVerify50x5x50000-2       86.53 MB/s     136.02 MB/s  1.57x
BenchmarkStreamVerify10x2x1M-2         315.65 MB/s     909.44 MB/s  2.88x
BenchmarkStreamVerify5x2x1M-2          180.45 MB/s     772.42 MB/s  4.28x
BenchmarkStreamVerify10x4x1M-2         310.35 MB/s     779.26 MB/s  2.51x
BenchmarkStreamVerify50x20x1M-2        547.23 MB/s     773.74 MB/s  1.41x
BenchmarkStreamVerify10x4x16M-2       4128.01 MB/s    6606.43 MB/s  1.60x
2016-04-12 15:41:22 +08:00
Klaus Post ff5a6511bc Merge pull request #17 from xiaost/master
Tests: backport go1.6 rand.Read for speedup tests
2016-04-07 13:02:14 +02:00
xiaost 9f0bea8a29 Tests: backport go1.6 rand.Read for speedup tests 2016-04-07 18:34:47 +08:00
Klaus Post 58c5128227 Update .travis.yml
Remove golint
2016-03-21 18:45:58 +01:00
Klaus Post 458f5ce468 Update .travis.yml
Add OSX test. Don't use sudo
2016-02-19 15:32:27 +01:00
Klaus Post 56791bf235 Update .travis.yml
Add Go 1.6 to CI test.
2016-02-19 15:22:04 +01:00
klauspost d1fe8adc28 Merge branch 'master' of https://github.com/klauspost/reedsolomon 2016-01-11 14:44:57 +01:00
klauspost efb98c83c7 Update asmfmt. 2016-01-11 14:44:44 +01:00
Klaus Post fab3ee4030 Update README.md 2016-01-11 14:39:25 +01:00
klauspost a3ee8967cb asmfmt assembler. 2015-12-14 14:57:49 +01:00
Klaus Post 75cae61a5b Merge pull request #13 from klauspost/avx2
Add AVX2 assembler
2015-12-14 14:21:20 +01:00
klauspost 627f48f59e Add AVX2 assembler functions.
Benchmarks on a VM (therefore a bit more noisy)

benchmark                             old ns/op     new ns/op     delta
BenchmarkEncode10x2x10000-8           58372         47421         -18.76%
BenchmarkEncode100x20x10000-8         2635444       1550511       -41.17%
BenchmarkEncode17x3x1M-8              3885495       2231034       -42.58%
BenchmarkEncode10x4x16M-8             24180221      21467661      -11.22%
BenchmarkEncode5x2x1M-8               2395287       2261452       -5.59%
BenchmarkEncode10x2x1M-8              2571278       2566560       -0.18%
BenchmarkEncode10x4x1M-8              3396774       3431916       +1.03%
BenchmarkEncode50x20x1M-8             27004601      20325731      -24.73%
BenchmarkEncode17x3x16M-8             29671393      23668596      -20.23%
BenchmarkVerify10x2x10000-8           109730        101519        -7.48%
BenchmarkVerify50x5x50000-8           3904166       3101568       -20.56%
BenchmarkVerify10x2x1M-8              4398490       4721719       +7.35%
BenchmarkVerify5x2x1M-8               3174574       3296440       +3.84%
BenchmarkVerify10x4x1M-8              5247394       5346667       +1.89%
BenchmarkVerify50x20x1M-8             35742777      26154681      -26.83%
BenchmarkVerify10x4x16M-8             52873512      54931253      +3.89%

benchmark                             old MB/s     new MB/s     speedup
BenchmarkEncode10x2x10000-8           1713.14      2108.73      1.23x
BenchmarkEncode100x20x10000-8         379.44       644.95       1.70x
BenchmarkEncode17x3x1M-8              4587.78      7989.92      1.74x
BenchmarkEncode10x4x16M-8             6938.40      7815.11      1.13x
BenchmarkEncode5x2x1M-8               2188.83      2318.37      1.06x
BenchmarkEncode10x2x1M-8              4078.03      4085.53      1.00x
BenchmarkEncode10x4x1M-8              3086.98      3055.37      0.99x
BenchmarkEncode50x20x1M-8             1941.48      2579.43      1.33x
BenchmarkEncode17x3x16M-8             9612.38      12050.26     1.25x
BenchmarkVerify10x2x10000-8           911.32       985.03       1.08x
BenchmarkVerify50x5x50000-8           1280.68      1612.09      1.26x
BenchmarkVerify10x2x1M-8              2383.94      2220.75      0.93x
BenchmarkVerify5x2x1M-8               1651.52      1590.47      0.96x
BenchmarkVerify10x4x1M-8              1998.28      1961.18      0.98x
BenchmarkVerify50x20x1M-8             1466.84      2004.57      1.37x
BenchmarkVerify10x4x16M-8             3173.09      3054.22      0.96x
2015-12-14 14:12:09 +01:00
klauspost eef4ac6aa3 Use goimports instead of gofmt. 2015-11-03 12:50:54 +01:00
klauspost e125c8bc8c Fix comment. 2015-11-03 12:36:28 +01:00
klauspost df20f9a1a9 Manually fetch dependencies. 2015-11-03 12:17:06 +01:00
klauspost d70b66fe1a Add golint and vet tests. 2015-11-03 12:13:11 +01:00
klauspost 976a24f33b Move examples to separate file/package
This makes the reedsolomon package prefix show up in the documentation examples.

+ StreamEncoder example.
2015-11-03 12:12:42 +01:00
klauspost 94aef4a146 Add stream order information. 2015-11-03 12:10:37 +01:00
klauspost 180472d98f Make documentation conform to go vet. 2015-11-03 12:09:36 +01:00
klauspost 5dedbfc3ff Update streaming documentation. 2015-10-27 15:59:47 +01:00
Klaus Post 4c33f7e599 Remove duplicated package documentation. 2015-10-27 15:39:19 +01:00