klauspost
|
5c2ef3ae72
|
Always check/return errors.
|
2015-06-23 12:16:26 +02:00 |
klauspost
|
7381e0b7b5
|
- Only run multiple goroutines if size is bigger than splitsize.
- Update docs
|
2015-06-23 11:18:29 +02:00 |
klauspost
|
83703c37ac
|
Add package documentation and clarify interface docs.
|
2015-06-22 15:12:05 +02:00 |
Klaus Post
|
5aa37c3492
|
Add AMD64 SSE3 Galois multiplication. Approximately 5-10x faster.
BenchmarkEncode10x2x10000 333.31 5827.17 17.48x
BenchmarkEncode10x2x10000-2 431.20 2802.53 6.50x
BenchmarkEncode10x2x10000-4 553.98 2432.95 4.39x
BenchmarkEncode10x2x10000-8 585.79 3469.61 5.92x
BenchmarkEncode100x20x10000 32.59 583.40 17.90x
BenchmarkEncode100x20x10000-2 59.52 726.70 12.21x
BenchmarkEncode100x20x10000-4 108.04 1363.25 12.62x
BenchmarkEncode100x20x10000-8 113.76 1274.62 11.20x
BenchmarkEncode17x3x1M 215.28 3141.85 14.59x
BenchmarkEncode17x3x1M-2 398.76 3650.12 9.15x
BenchmarkEncode17x3x1M-4 655.32 6071.11 9.26x
BenchmarkEncode17x3x1M-8 832.16 6616.47 7.95x
BenchmarkEncode10x4x16M 154.48 1357.30 8.79x
BenchmarkEncode10x4x16M-2 295.62 2377.92 8.04x
BenchmarkEncode10x4x16M-4 529.89 3519.49 6.64x
BenchmarkEncode10x4x16M-8 632.11 4521.90 7.15x
BenchmarkEncode5x2x1M 327.87 4879.09 14.88x
BenchmarkEncode5x2x1M-2 576.11 2599.20 4.51x
BenchmarkEncode5x2x1M-4 1043.65 3559.12 3.41x
BenchmarkEncode5x2x1M-8 1227.77 4255.34 3.47x
BenchmarkEncode10x2x1M 321.24 4574.68 14.24x
BenchmarkEncode10x2x1M-2 587.73 3100.28 5.28x
BenchmarkEncode10x2x1M-4 1101.96 4770.32 4.33x
BenchmarkEncode10x2x1M-8 1217.08 5812.17 4.78x
BenchmarkEncode10x4x1M 155.34 2037.27 13.11x
BenchmarkEncode10x4x1M-2 298.38 2470.97 8.28x
BenchmarkEncode10x4x1M-4 548.67 3603.15 6.57x
BenchmarkEncode10x4x1M-8 625.23 4827.42 7.72x
BenchmarkEncode50x20x1M 31.37 347.65 11.08x
BenchmarkEncode50x20x1M-2 59.81 713.28 11.93x
BenchmarkEncode50x20x1M-4 105.34 1175.47 11.16x
BenchmarkEncode50x20x1M-8 123.84 1491.91 12.05x
BenchmarkEncode17x3x16M 209.55 1861.59 8.88x
BenchmarkEncode17x3x16M-2 394.19 3331.73 8.45x
BenchmarkEncode17x3x16M-4 643.30 4942.74 7.68x
BenchmarkEncode17x3x16M-8 839.64 6213.43 7.40x
|
2015-06-21 21:23:22 +02:00 |
Klaus Post
|
17e9fa30f0
|
Add Join function for join data shards.
|
2015-06-21 13:25:12 +02:00 |
Klaus Post
|
437e364842
|
Adjust splitsize:
benchmark old ns/op new ns/op delta
BenchmarkEncode10x2x10000-2 243613 229413 -5.83%
BenchmarkEncode100x20x10000-2 23041318 19311104 -16.19%
BenchmarkEncode17x3x1M-2 54469780 49602836 -8.94%
BenchmarkEncode10x4x16M-2 674538600 647037000 -4.08%
Bigger sizes (1024) yeilds slower less speedup.
|
2015-06-20 20:32:52 +02:00 |
Klaus Post
|
9f6744582c
|
Also refactor Verify as well as multithreaded options.
benchmark old MB/s new MB/s speedup
BenchmarkEncode10x2x10000 182.29 308.59 1.69x
BenchmarkEncode100x20x10000 14.41 30.29 2.10x
BenchmarkEncode17x3x1M 38.52 196.43 5.10x
BenchmarkEncode10x4x16M 23.78 148.58 6.25x
|
2015-06-20 20:00:25 +02:00 |
Klaus Post
|
50a83296f4
|
Restructure to make one of the galois multiplication parts constant for the main loop.
|
2015-06-20 18:46:06 +02:00 |
Klaus Post
|
921adcb5d5
|
Use range to avoid one bound check per galMultiply:
benchmark old MB/s new MB/s speedup
BenchmarkVerify10x2x10000-2 235.24 253.36 1.08x
BenchmarkVerify50x5x50000-2 76.78 94.87 1.24x
BenchmarkVerify10x2x1M-2 180.90 209.73 1.16x
BenchmarkVerify5x2x1M-2 173.22 202.89 1.17x
BenchmarkVerify10x4x1M-2 71.51 118.20 1.65x
BenchmarkVerify50x20x1M-2 11.27 12.84 1.14x
BenchmarkVerify10x4x16M-2 44.74 50.07 1.12x
|
2015-06-20 14:51:11 +02:00 |
Klaus Post
|
419c6cc9e9
|
Add Splitter to help split data into shards.
|
2015-06-20 11:27:03 +02:00 |
Klaus Post
|
c5de03551c
|
Minor adjustments for golint.
|
2015-06-20 10:11:33 +02:00 |
Klaus Post
|
fdb7664a06
|
Hide internal matrix errors and document the used ones.
|
2015-06-20 10:08:59 +02:00 |
Klaus Post
|
481b39ba1c
|
Use a mutex to check if other goroutines have detected a difference. This avoid rare "close of closed channel" panic.
|
2015-06-19 19:15:31 +02:00 |
Klaus Post
|
e3aca6cd9d
|
Shorten the variable names and make an encoder interface, so it isn't possible to create it without calling New.
|
2015-06-19 18:54:58 +02:00 |
Klaus Post
|
8296e44d17
|
Update documentation, and test if all shards are zero-sized.
|
2015-06-19 18:21:38 +02:00 |
Klaus Post
|
755a781502
|
Remove unused types.
|
2015-06-19 17:27:22 +02:00 |
klauspost
|
2b4171b9e6
|
Initial version
|
2015-06-19 16:31:24 +02:00 |