Compare commits

...

9 Commits

Author SHA1 Message Date
Rached Ben Mustapha a068ad7354 chore: bump utapi version 2021-11-01 22:25:12 +00:00
Jonathan Gramain c494cd05da S3C-3778: new test for non-versioned complete MPU replay
Add a unit test to make sure we skip the data deletion on
non-versioned buckets when a complete MPU is replayed. It was already
the case but it should help to catch any regression.

(cherry picked from commit 60d55c0a6c)
2021-08-30 14:48:22 -07:00
Jonathan Gramain 7f188b97d4 bugfix: S3C-3778 prevent MPU with duplicate data keys
Sometimes, complete MPU request may be replayed due to a previous
failure leaving the original MPU metadata untouched but with a valid
completed object as well. This could lead to a new version of the
object being created, pointing to the same data keys than the previous
version, created from the same MPU that failed from the client
perspective.

The fix consists of checking whether we are trying to update the
metadata with the same data location keys than the existing ones, and
skip the metadata update and data location deletions if this is the
case.

It's a generalization of the previous fix S3C-1959 that consisted in
skipping data deletions if they were already in the object
metadata. It worked on non-versioned buckets because there, the keys
to delete are the original master key locations because the master key
gets overwritten, but it did not work on versioned buckets since there
is no key to delete as we are creating a new version.

(cherry picked from commit 3464487c2b)
2021-08-30 14:48:22 -07:00
Jonathan Gramain 43e8d4c315 S3C-3778 optim: do not call getObjectMD(mpuOverviewKey) twice
Since metadataValidateMultipart already retrieves the metadata of the
overview key, we can piggy-back it and avoid retrieving it twice
during the completeMultipartUpload processing.

(cherry picked from commit 2b1f87511b)
2021-08-30 14:48:22 -07:00
Jonathan Gramain 627028a7f6 bugfix: S3C-3778 unit test showing a duplicate version
Create a unit test showing that a duplicate version is created after a
complete-multipart-upload fails and is retried. Having more than one
version in this case creates shared data keys that cause a risk of
data loss once one of the versions is deleted.

(cherry picked from commit 826d2fe2e8)
2021-08-30 14:48:22 -07:00
Ronnie Smith 9c57c4e630
bugfix: CLDSRV-5 Update arsenal for hotfix
(cherry picked from commit 1a544a02fe)
2021-07-30 22:11:51 -07:00
Taylor McKinnon 40ff3d11df bf(S3C-4276): Update KMIP test certificates
(cherry picked from commit 10f8e5a125)
2021-04-14 14:16:46 -07:00
Taylor McKinnon 565f82169b bf(S3C-4245): Enforce x-amz-bypass-governance-retention header and s:BypassGovernanceRetention user policy
(cherry picked from commit fcd2aa8218)
2021-04-13 17:43:19 -07:00
Taylor McKinnon e0c2cf7045 bf(S3C-4245): Don't allow COMPLIANCE mode to be shortened or disabled
(cherry picked from commit 11ddeffe40)
2021-04-13 14:11:49 -07:00
18 changed files with 728 additions and 351 deletions

View File

@ -1,6 +1,5 @@
FROM python:3-alpine
RUN apk add --no-cache \
libressl && \
apk add --no-cache --virtual .build-deps \
@ -8,7 +7,9 @@ RUN apk add --no-cache \
libffi-dev \
libressl-dev \
sqlite-dev \
build-base && \
build-base \
rust \
cargo && \
pip install pykmip requests && \
apk del .build-deps && \
mkdir /pykmip

View File

@ -1,18 +1,18 @@
-----BEGIN CERTIFICATE-----
MIIC6zCCAdOgAwIBAgIUOf68qXxLlhpqa94YFjWARaY09VIwDQYJKoZIhvcNAQEL
BQAwJDEQMA4GA1UECgwHU2NhbGl0eTEQMA4GA1UEAwwHUm9vdCBDQTAgFw0yMDA0
MDcxNzUwNThaGA8yMTIwMDMxNDE3NTA1OFowJDEQMA4GA1UECgwHU2NhbGl0eTEQ
MIIC6zCCAdOgAwIBAgIUPIpMY95b4HjKAk+FyydZApAEFskwDQYJKoZIhvcNAQEL
BQAwJDEQMA4GA1UECgwHU2NhbGl0eTEQMA4GA1UEAwwHUm9vdCBDQTAgFw0yMTA0
MDkwMDI4MTFaGA8yMTIxMDMxNjAwMjgxMVowJDEQMA4GA1UECgwHU2NhbGl0eTEQ
MA4GA1UEAwwHUm9vdCBDQTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB
AOPoCXWg+Mpwd2ltSvgicehcaeW6BoNRm3+icx6OmJHjQL/iApfBJJ54A9ef/Myw
NjBDd2RQBdu2nnEKRar06p/w+ubTbRU189mfAAOGC9m1gkl16gNlbByYikmC0grQ
UDeQ3uKnJ8LFwpu/A+MLpK4Os1CH8fqFIse5w2AT9BscwzF0aHlgO5vm48P5rR9d
EKYTsPlfuCNYIWQ6x75fk//2o9/mCRc9bFdY/ASNwZTTlnSPZ9DQ+g7zaExGG5ah
eJXoKK4skups+GdzLMQj53/lR6fZjn8pX5Mmv4Ex9OvlpFeV85HyZcKnjGqM7eGl
6TcbGZlHw0GvebXIeg4VHe8CAwEAAaMTMBEwDwYDVR0TAQH/BAUwAwEB/zANBgkq
hkiG9w0BAQsFAAOCAQEApYYIRlu8VWPCm2HJ/k119/ADPgFI6fAmbojNoZo9lLW+
i6JWkohptV5c0oMUjxTrzBRYWyMWdScHUfxjV6AY7QFoIrNBqEDTARfRu+ekFkfH
qfnSp45aLi0gf53WJNTCQJAILGxcXkvSBuw50Fm5aBSgG70Oczf/bgJY4uJTVUra
3GdmDXqLZ7WpSCS5EyzJchDo75TFFRnNAy7+YoKh+11TIwdrrs2qEiAjJ/2zAuIg
r5Vnx6AEEnhqFNbyTLOfk2/MN87FEKqH07vSAHA6M4tprWU8kArcTjMlmVaH8cgO
47lVJ6blCUiJTJLnzNw83dzJTfFmm9CQx7JyTpEBag==
AKqLFEsWtfRTxnoZrQe63tq+rQnVgninHMahRmXkzyjK/uNhoKnIh8bXdTC/eCZ6
FBROqBYNL0TJb0HDv1FzcZS1UCUldRqTlvr6wZb0pfrp40fvztsqQgAh1t/Blg5i
Zv5+ESSlNs5rWbFTxtq+FbMW/ERYTrVfnMkBiLg4Gq0HwID9a5jvJatzrrno2s1m
OfZCT3HaE3tMZ6vvYuoamvLNdvdH+9KeTmBCursfNejt0rSGjIqfi6DvFJSayydQ
is5DMSTbCLGdKQmA85VfEQmlQ8v0232WDSd6gVfp2tthDEDHnCbgWkEd1vsTyS85
ubdt5v4CWGOWV+mu3bf8xM0CAwEAAaMTMBEwDwYDVR0TAQH/BAUwAwEB/zANBgkq
hkiG9w0BAQsFAAOCAQEARTjc2zV/ol1/LsSzZy6l1R0uFBmR2KumH+Se1Yq2vKpY
Dv6xmrvmjOUr5RBO77nRhIgdcQA+LyAg8ii2Dfzc8r1RTD+j1bYOxESXctBOBcXM
Chy6FEBydR6m7S8qQyL+caJWO1WZWp2tapcm6sUG1oRVznWtK1/SHKIzOBwsmJ07
79KsCJ6wf9tzD05EDTI2QhAObE9/thy+zc8l8cmv9A6p3jKkx9rwXUttSUqTn0CW
w45bgKg6+DDcrhZ+MATbzuTfhuA4NFUTzK7KeX9sMuOV03Zs8SA3VhAOXmu063M3
0f9X7P/0RmGTTp7GGCqEINcZdbLh3k7CpFb2Ox998Q==
-----END CERTIFICATE-----

View File

@ -1,18 +1,18 @@
-----BEGIN CERTIFICATE-----
MIIC2TCCAcGgAwIBAgIUYXclvoYJK/U2Gh4ji0yRevvj3F0wDQYJKoZIhvcNAQEL
BQAwJDEQMA4GA1UECgwHU2NhbGl0eTEQMA4GA1UEAwwHUm9vdCBDQTAeFw0yMDA0
MDcxNzUwNThaFw0yMTA0MDcxNzUwNThaMCkxEDAOBgNVBAoMB1NjYWxpdHkxFTAT
BgNVBAMMDHB5a21pcC5sb2NhbDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
ggEBALtrAMYY1VaaJYWPIF630dPH/Pt4kefSjvr0NlKyKv2gpUy3L3Dm1CTd0Ay6
yQWR6jVyKvGWFdyA4GfP2+/96i7DLTI4ePZq/catho8hOSuPBOK1h+N153LlZ9Yu
YKm5lNRjchQ/Di4JykhAgQlNHh7ziR3hrV1QrgjDAgDHKvumKBcRc8oOYuW4ATYl
35C/4h5hzMucnlmwZjq03wBZyhuLKgMT6dmrXv67ZexIWo881ANaBpiUq36wPDJo
FoeS3tXvEJRy3nFL+AEEHpqEpP6GMgt8rnPVJJgUb0OFIpx3kpAaEY7I9hNQsjl3
B5okt1cRHONFOl25KXO+QKf8KuECAwEAATANBgkqhkiG9w0BAQsFAAOCAQEAkesK
fVNo2pGaOybzf9sbmUf/MJx6vPZzqkfsED3KStjBaTKndjvqBvXmxI7hczLvs8Jo
ALbwRJxjLwT3NbxqEwj0nCtid3wzl70f9fSYLh/FC0Nus/1kYJA/JKiZKbEiiaCM
ZULLqkrK26baCxoVpa3cWSo3zR2F3h0Px54abLXuBFoq6QNt5u/3+WIPY1wJYHUk
iCcjLaL7VM9snmK0jF3CNz4+6ZFPAT6H54ILuJ/V427zB1TYMDLOVZumzPLg6DEz
YxMyz99Rjwf4BHEI186csIoZZx6GOVauRoiw9KAct2KOKdcOoiyyKINL/NGlH04e
GDazMx7Yt7xHIjFmlw==
MIIC2zCCAcOgAwIBAgIUIlE8UAkqQ+6mbJDtrt9kkmi8aJYwDQYJKoZIhvcNAQEL
BQAwJDEQMA4GA1UECgwHU2NhbGl0eTEQMA4GA1UEAwwHUm9vdCBDQTAgFw0yMTA0
MDkwMDI4MTFaGA8yMTIxMDMxNjAwMjgxMVowKTEQMA4GA1UECgwHU2NhbGl0eTEV
MBMGA1UEAwwMcHlrbWlwLmxvY2FsMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB
CgKCAQEAtxr7pq/lnzVeZz4z52Yc3DeaPqjNfRSyW5cPUlT7ABXFb7+tja7K2C7u
DYVK+Q+2yJCQwYJY47aKJB++ewam9t2V8Xy0Z8S+0I2ImCwuyeihaD/f6uJZRzms
ycdECH22BA6tCPlQLnlboRiZzI6rcIvXAbUMvLvFm3nyYIs9qidExRnfyMjISknM
V+83LT5QW4IcHgKYqzdz2ZmOnk+f4wmMmitcivTdIZCL8Z0cxr7BJlOh5JZ/V5uj
WUXeNa+ttW0RKKBlg9T+wj0JvwoJBPZTmsMAy3tI9tjLg3DwGYKsflbFeU2tebXI
gncGFZ/dFxj331GGtq3kz1PzAUYf2wIDAQABMA0GCSqGSIb3DQEBCwUAA4IBAQB1
8HgJ0fu6/pCrDxAm90eESFjmaTFyTN8q00zhq4Cb3zAT9KMWzAygkZ9n4ZFgELPo
7kBE2H6RcDdoBmjVYd8HnBloDdYzYbncKgt5YBvxRaMSF4/l65BM8wjatyXErqnH
QLLTRe5AuF0/F0KtPeDQ2JFVu8dZ35W3fyKGPRsEdVOSCTHROmqpGhZCpscyUP4W
Hb0dBTESQ9mQHw14OCaaahARd0X5WdcA/E+m0fpGqj1rQCXS+PrRcSLe1E1hqPlK
q/hXSXD5nybwipktELvJCbB7l4HmJr2pIpldeR5+ef68Cs8hqs6DRlsJX9sK2ng+
TFe5v6SCarqZ9kFvr6Yp
-----END CERTIFICATE-----

View File

@ -1,18 +1,18 @@
-----BEGIN CERTIFICATE-----
MIIC8TCCAdmgAwIBAgIUOuxeluKdf49aeSl3tZ+BJEZilaswDQYJKoZIhvcNAQEL
BQAwJDEQMA4GA1UECgwHU2NhbGl0eTEQMA4GA1UEAwwHUm9vdCBDQTAeFw0yMDA0
MDcxNzUwNThaFw0yMTA0MDcxNzUwNThaMCUxEDAOBgNVBAoMB1NjYWxpdHkxETAP
BgNVBAMMCEpvaG4gRG9lMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA
xk8i8m8YEyAqT8S1Hh3VzKLt8EQf6YqnjgunCuTi5PvYd5JtKnneIBu5+JsIXt4/
TlaLUjtT8RiXLtXxDglcxKcEX1M8wwQR35dj3Jx3xQbYv6Pr1Mqp6+egIvxyuJYW
9pc7N5geQAATbJ6iqttY/+6l8KUKF8V28kocgaxhvVUlf62SKaFcau1DCBQu8TwB
sk7JCQ/kUt75cBDV6EDpiShZ7XYAo9hy3eyumMufKO5tTIl1/T2HYaVabbzYZBbY
W2kr4stBDLjls1W0JVmt4V9pn+TrrwRfhWPIXo3KlxYRUbsW2Z223QB+jCndGqcf
7dZSaoi7pVS+tKzIn3P+HQIDAQABoxowGDAWBgNVHSUBAf8EDDAKBggrBgEFBQcD
AjANBgkqhkiG9w0BAQsFAAOCAQEAXCAJBfm4/SUFJr5jgrerHNfDFS+PUo5PHexs
q1Yie1Q6MDqlPTc/ZcWT/DoqLy3aq+tqqr66OY+2oARq+SgVnsSzgWQ/LosSQikl
3fwcHZfelbbvPIofM0DKtGJLyTTGvtwjj9t3tzx0+7b6PXXWm91p0Xzpp9i0llJ3
YBAxIW43LAFaIBJonxgnR9TLVlP+XatnYBzWbOg44uxeNDQkudbEUsGb2j4SShk4
Pw1fgyKmyQRikvgst7uCgS5S9Vzu6O9CC6nvqnEfzPLWMA9GRUiTjOeZ0BlMqrlH
orzpJpoFNelpqgh9LxA026a4TeK/pdfx/NJE6gTN+CVxHU40iQ==
MIIC8zCCAdugAwIBAgIUBs6nVXQXhrFbClub3aSLg72/DiYwDQYJKoZIhvcNAQEL
BQAwJDEQMA4GA1UECgwHU2NhbGl0eTEQMA4GA1UEAwwHUm9vdCBDQTAgFw0yMTA0
MDkwMDI4MTFaGA8yMTIxMDMxNjAwMjgxMVowJTEQMA4GA1UECgwHU2NhbGl0eTER
MA8GA1UEAwwISm9obiBEb2UwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIB
AQC6neSYoBoWh/i2mBpduJnTlXacpJ0iQqLezvcGy8qR0s/48mtfV2IRGTNVsq4L
jLLRsPGt9KkJlUhHGWhG00cBGEsIiJiBUr+WrEsO04ME/Sk76kX8wk/t9Oljl7jt
UDnQUwshj+hRFe0iKAyE65JIutu5EiiNtOqMzbVgPNfNniAaGlrgwByJaS9arzsH
PVju9yZBYzYhwAMyYFcXUGrgvHRCHKmxBi4QmV7DX4TeN4l9TrCyEmqDev4PRFip
yR2Fh3WGSwWh45HgMT+Jp6Uv6yI4wMXWJAcNkHdx1OhjBoUQrkavvdeVEnCwjQ+p
SMLm0T4iNxedQWBtDM7ts4EjAgMBAAGjGjAYMBYGA1UdJQEB/wQMMAoGCCsGAQUF
BwMCMA0GCSqGSIb3DQEBCwUAA4IBAQCMi9HEhZc5jHJMj18Wq00fZy4O9XtjCe0J
nntW9tzi3rTQcQWKA7i9uVdDoCg+gMFVxWMvV7luFEUc/VYV1v8hFfbIFygzFsZY
xwv4GQaIwbsgzD+oziia53w0FSuNL0uE0MeKvrt3yzHxCxylHyl+TQd/UdAtAo+k
RL1sI0mBZx5qo6d1J7ZMCxzAGaT7KjnJvziFr/UbfSNnwDsxsUwGaI1ZeAxJN8DI
zTrg3f3lrrmHcauEgKnuQwIqaMZR6veG6RkjtcYSlJYID1irkE6njs7+wivOAkzt
fBt/0PD76FmAI0VArgU/zDB8dGyYzrq39W749LuEfm1TPmlnUtDr
-----END CERTIFICATE-----

View File

@ -1,28 +1,28 @@
-----BEGIN PRIVATE KEY-----
MIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQDGTyLybxgTICpP
xLUeHdXMou3wRB/piqeOC6cK5OLk+9h3km0qed4gG7n4mwhe3j9OVotSO1PxGJcu
1fEOCVzEpwRfUzzDBBHfl2PcnHfFBti/o+vUyqnr56Ai/HK4lhb2lzs3mB5AABNs
nqKq21j/7qXwpQoXxXbyShyBrGG9VSV/rZIpoVxq7UMIFC7xPAGyTskJD+RS3vlw
ENXoQOmJKFntdgCj2HLd7K6Yy58o7m1MiXX9PYdhpVptvNhkFthbaSviy0EMuOWz
VbQlWa3hX2mf5OuvBF+FY8hejcqXFhFRuxbZnbbdAH6MKd0apx/t1lJqiLulVL60
rMifc/4dAgMBAAECggEABbdIOZasKfj2X0A7PDf97p0PoKpGBTRC6hw5312DkLgV
oDSvQtcqaOCDtr+5OQrM5lQmReOB4uQjj20JOq9YZi6uOJUsni5i2YACl9xGs34k
BzoRVRvWU/9kJT4DjIB+/vKS+WJAFPYrmSjlZWlXImFdlRccuFyvtgIe2jn+wzdB
ErYirqHNnSJdiNBA58VmLR2ge3Ilo/S9YTgB/+aKtElhitGqSkVB8QKNWt2u5oFw
tJ0n1ikz1fiD16/kcpaOT9TWm1kkhIthSwZd65+HMSuxpO+KqmwgmXPXhDnTrlSd
y5r//+ggAnDhuUmNmFwxHn8prThYGLKlLpUFtivJIQKBgQDqWMOf8Y4Im7nW7b+r
r00gSu+XpvBBnC2Dpr0bw90OQYdLhDcdeqeJ2fFpQqXaevNhmqnAy6PFeqdoSscu
gc1XkejZsPIQNSa7e0RC7kveVp3gkzcQ222vgJY7++R46/S3Du3oQ3FAlB7/0VOi
+9CBgdSI+d3u1rcXOW63uQRb1QKBgQDYofBl2a/SQmWcMFA20fN+z1ft4j3YXb1X
KgluiJpBFGP1lWXEZ/+v7L2PI5fGAl1FJgjYYuzVH3T1f0kgZmuIUuZPMB7d8gLw
1qyfKH4MKTItwh38K28/dRckBlMj10Uc+aXMe1LTcl15jp6LLI4Mx09qy1QF/c/y
76Cvkc2lKQKBgERpfVJn9grVSz9PULESD/XpamBfP6wnp7HTL0m3uAS9ZH3LLzvP
3rEDitIrvrc1RW+s8vlxeXHhCJYNnnAZPJTf55YFbeUFXzVEGv1fC47wwk4ZK+4j
4LVnWHRSaLRUTbBTD2jKp3kuxI3x0fS2hnwIJr+GEh/zVqfVAFlqDbexAoGABf1U
PDysk8+qJ4tebGWZqePptnYO57CPz50l7ZxxR2Nc8ClVSvzlIOQWyaJeS+c81PCc
Rf9WNP5NqYv/ZZnvVzGTlJTsBY7vbeFBnJTuB0AMVx+K3LIGvWZrYV+bZN5K1uZA
I0s1mwsKcpXy5D4zHz9TfsxoYlIGMd1WQARz/yECgYB5JH9wFDKjMAhCGG9hxMSx
drMeZ/Ypya/DTyN+1p3DnG+Tw0Mh6Hmpm04D6iDKrafLkmlD+ZduzhzfEqn0lDDg
uIFFvWO6BOo2KALtMpcuDGYFKjJJ0S47EiWhKHUn/OsnAzk2lKlEOcPfrc9lY5AM
6P6Jg/Q21kr1j5gQedj7Cw==
MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQC6neSYoBoWh/i2
mBpduJnTlXacpJ0iQqLezvcGy8qR0s/48mtfV2IRGTNVsq4LjLLRsPGt9KkJlUhH
GWhG00cBGEsIiJiBUr+WrEsO04ME/Sk76kX8wk/t9Oljl7jtUDnQUwshj+hRFe0i
KAyE65JIutu5EiiNtOqMzbVgPNfNniAaGlrgwByJaS9arzsHPVju9yZBYzYhwAMy
YFcXUGrgvHRCHKmxBi4QmV7DX4TeN4l9TrCyEmqDev4PRFipyR2Fh3WGSwWh45Hg
MT+Jp6Uv6yI4wMXWJAcNkHdx1OhjBoUQrkavvdeVEnCwjQ+pSMLm0T4iNxedQWBt
DM7ts4EjAgMBAAECggEANNXdUeUKXdSzcycPV/ea/c+0XFcy8e9B46lfQTpTqQOx
xD8GbWD1L/gdk6baJgT43+ukEWdSsJbmdtLXti29Ta8OF2VtIDhIbCVtvs3dq3zt
vrvugsiVDr8nkP306qOrKrNIVIFE+igmEmSaXsu/h/33ladxeeV9/s2DC7NOOjWN
Mu4KYr5BBbu3qAavdzbrcz7Sch+GzsYqK/pBounCTQu3o9E4TSUcmcsasWmtHN3u
e6G2UjObdzEW7J0wWvvtJ0wHQUVRueHfqwqKf0dymcZ3xOlx3ZPhKPz5n4F1UGUt
RQaNazqs5SzZpUgDuPw4k8h/aCHK21Yexw/l4+O9KQKBgQD1WZSRK54zFoExBQgt
OZSBNZW3Ibti5lSiF0M0g+66yNZSWfPuABEH0tu5CXopdPDXo4kW8NLGEqQStWTX
RGK0DE9buEL3eebOfjIdS2IZ3t3dX3lMypplVCj4HzAgITlweSH1LLTyAtaaOpwa
jksqfcn5Zw+XGkyc6GBBVaZetQKBgQDCt6Xf/g26+zjvHscjdzsfBhnYvTOrr6+F
xqFFxOEOocGr+mL7UTAs+a9m/6lOWhlagk+m+TIZNL8o3IN7KFTYxPYPxTiewgVE
rIm3JBmPxRiPn01P3HrtjaqfzsXF30j3ele7ix5OxieZq4vsW7ZXP3GZE34a08Ov
12sE1DlvdwKBgQDzpYQOLhyqazzcqzyVfMrnDYmiFVN7QXTmiudobWRUBUIhAcdl
oJdJB7K/rJOuO704x+RJ7dnCbZyWH6EGzZifaGIemXuXO21jvpqR0NyZCGOXhUp2
YfS1j8AntwEZxyS9du2sBjui4gKvomiHTquChOxgSmKHEcznPTTpbN8MyQKBgF5F
LVCZniolkLXsL7tS8VOez4qoZ0i6wP7CYLf3joJX+/z4N023S9yqcaorItvlMRsp
tciAIyoi6F2vDRTmPNXJ3dtav4PVKVnLMs1w89MwOCjoljSQ6Q7zpGTEZenbpWbz
W2BYBS9cLjXu4MpoyInLFINo9YeleLs8TvrCiKAXAoGBANsduqLnlUW/f5zDb5Fe
SB51+KhBjsVIeYmU+8xtur9Z7IxZXK28wpoEsm7LmX7Va5dERjI+tItBiJ5+Unu1
Xs2ljDg35ARKHs0dWBJGpbnZg4dbT6xpIL4YMPXm1Zu++PgRpxPIMn646xqd8GlH
bavm6Km/fXNG58xus+EeLpV5
-----END PRIVATE KEY-----

View File

@ -1,28 +1,28 @@
-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC7awDGGNVWmiWF
jyBet9HTx/z7eJHn0o769DZSsir9oKVMty9w5tQk3dAMuskFkeo1cirxlhXcgOBn
z9vv/eouwy0yOHj2av3GrYaPITkrjwTitYfjdedy5WfWLmCpuZTUY3IUPw4uCcpI
QIEJTR4e84kd4a1dUK4IwwIAxyr7pigXEXPKDmLluAE2Jd+Qv+IeYczLnJ5ZsGY6
tN8AWcobiyoDE+nZq17+u2XsSFqPPNQDWgaYlKt+sDwyaBaHkt7V7xCUct5xS/gB
BB6ahKT+hjILfK5z1SSYFG9DhSKcd5KQGhGOyPYTULI5dweaJLdXERzjRTpduSlz
vkCn/CrhAgMBAAECggEAVo7o4JT/kuvGgJTF4nkLU8B9urbIzESW/JhlrnPTHyhe
r7u7EW3KdOxs9jQeO5BUlzKPWtxyZFCxU8DQV1ryGX7TFOq4Ezb+1g+2ocw6Vz/K
hdpJVGhT6ODCwEzTIBAyCJWVTnyA1Ap5fj0sW1temfToUwCzzPFCL5HBDxLtayNb
DvAdqP6nvzF85oGlyJlwdY9Zz8bWxd+kDeFAp+23rcLrwPwUASilLwp9v1erc6Hs
Mg1UkF7liuIIaSQahOF93YTMJAOMNft8wN+QQqQ12zMCD/fOcZTRBrLwHSaGJ+/Y
ACq9yzVYpzRnNOgkmpBkxXUzddkPPom/LdGaY5aE8QKBgQDwpbQaIf1VQZRyVEVL
VeW9fh1f4rzI0Yd+/BvuSzVLAuZCWDWjTC5zqpLeQ9mk7ziF0ck/MlcSlumZ+f0Q
aMUNxjtpGeKmAygCx5mZRCqKz+oxP0vdHkP5PwQQIUAwWXnWiSAuCKRX1TnjW9Dk
7Qq4Qfz2FS/PmY7Z3sS4ugId9wKBgQDHX/LYRuaoGS8gqR72DOqCJMkMduz5DGtD
fTUvSP+OZHAMRgTnANAOagy/drj6nNM3zDHTocEMfcC4HbiRU7c6Ys1iuiJ4bP4l
OkJajl9O3LJsJwLsHeMnZM27/9/mD8mrPzbNUHXZCU26uah9qLHwAwexp+rzJJNX
iTHkT/Gn5wKBgQCCNseblGTGKzQuIRdVymcEACfY6JGKgIY22igq6xstOaZqo9xy
PhiskdHi3wf3zVHiZz/kKFMhRfOlU7XxmR93cppXJqCTgAW4a1TbsBzs+9AXUc61
GVlilwyVxcg74U6iHZUCE78Jn+Ew+0+vb+xrA5njdldmmArKLVZ5Nn1KxQKBgFZs
COAnG6SSBhOqO3l8b8qqF1wH0QDDmVtP0tYEVoJqlwc68rUPbSBSZ+Q2mkhH4ma1
ZIPQAdZgTEGC4JZeK3ZrjYvWE0sQM7n/XvPR8w5ELDMlVebzrZtN3sA3Ud5vyYMp
i5/D2NGTbtYZ1CdkEH1xUsx3dSigGh4/ohjNbnrRAoGANWNVP+WK/iC6H49UGbtC
f+0Q5UL1hEh+9W8WqYiY38mA1xO3A07RQhBjk004CSqcd/ZJgniWYTZOhfd9lwBp
5FIg95xaJbklGOxL8jYAYqO4D/QURXuYpLVpVS/Xvy0d5n2U1tISSAvhs4u6/7Fv
M0iIzoBaJY36ZSK4LRgiHjM=
MIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQC3Gvumr+WfNV5n
PjPnZhzcN5o+qM19FLJblw9SVPsAFcVvv62NrsrYLu4NhUr5D7bIkJDBgljjtook
H757Bqb23ZXxfLRnxL7QjYiYLC7J6KFoP9/q4llHOazJx0QIfbYEDq0I+VAueVuh
GJnMjqtwi9cBtQy8u8WbefJgiz2qJ0TFGd/IyMhKScxX7zctPlBbghweApirN3PZ
mY6eT5/jCYyaK1yK9N0hkIvxnRzGvsEmU6Hkln9Xm6NZRd41r621bREooGWD1P7C
PQm/CgkE9lOawwDLe0j22MuDcPAZgqx+VsV5Ta15tciCdwYVn90XGPffUYa2reTP
U/MBRh/bAgMBAAECggEABCvcMcbuDztzBB0Zp5re63Fk1SqZS9Et4wJE+hYvhaf5
UHtoY8LoohYnnC0+MQBXpKgOdCoZBk8BRKNofnr/UL5pjQ/POFH2GuAujXDsO/NN
wgc6fapcaE/7DLm6ZgsfG2aOMJclaXmgScI6trtFUpIM+t/6A06vyMP1bpeddwPW
Fqu7NvpDiEcTRUGd+z1JooYgUhGgC7peYUx5+9zqFrwoDBKxnUOnz3BkDsXBy3qm
65Vu0BSjuJzf6vVMpNGUHY6JXjopVNWku+JAX0wD+iikOd5sziNVdIj1fnZ+IHIf
7G5h5owHpvSGzJFQ18/g5VHtJdCm+4WQSnbSJRsCAQKBgQDu4IH8yspyeH44fhoS
PAp/OtILqSP+Da0zAp2LbhrOgyzyuSTdEAYyptqjqHS6QkB1Bu1H44FS0BYUxRXc
iu2e9AndiLVCGngsE7TpA/ZVLN1B0LEZEHjM6p4d6zZM6iveKVnPAOkTWTBAgzCt
b31nj4jL8PdlPKQil1AMrOlRAQKBgQDEOwshzIdr2Iy6B/n4CuBViEtwnbAd5f/c
atA9bcfF8kCahokJsI4eCCLgBwDZpYKD+v0AwOBlacF6t6TX+vdlJsi5EP7uxZ22
ILsuWqVm/0H77PACuckc5/qLZoGGC81l0DhnpoeMEb6r/TKOo5xAK1gxdlwNNrq+
nP1zdZnU2wKBgBAS92xFUR4m0YeHpMV5WNN658t1FEDyNqdqE6PgQtmGpi2nG73s
aB5cb/X3TfOCpce6MZlWy8sAyZuYL4Jprte1YDySCHBsS43bvZ64b4kHvdPB8UjY
fOh9GSq2Oy8tysnmSm7NhuGQbNjKeyoQiIXBeNkQW/VqATl6qR5RPFoBAoGACNqV
JQBCd/Y8W0Ry3eM3vgQ5SyqCQMcY5UwYez0Rz3efvJknY72InAhH8o2+VxOlsOjJ
M5iAR3MfHLdeg7Q6J2E5m0gOCJ34ALi3WV8TqXMI+iH1rlnNnjVFU7bbTz4HFXnw
oZSc9w/x53a0KkVtjmOmRg0OGDaI9ILG2MfMmhMCgYB8ZqJtX8qZ2TqKU3XdLZ4z
T2N7xMFuKohWP420r5jKm3Xw85IC+y1SUTB9XGcL79r2eJzmzmdKQ3A3sf3oyUH3
RdYWxtKcZ5PAE8hVRtn1ETZqUgxASGOUn/6w0npkYSOXPU5bc0W6RSLkjES0i+c3
fv3OMNI8qpmQhEjpHHQS1g==
-----END PRIVATE KEY-----

View File

@ -4,7 +4,8 @@
* instability from the metadata layer. The check returns true if there was no
* match and false if at least one key from the previous list exists in the
* current list
* @param {array|string} prev - list of keys from the object being overwritten
* @param {array|string|null} prev - list of keys from the object being
* overwritten
* @param {array} curr - list of keys to be used in composing current object
* @returns {array} list of keys that can be deleted
*/

View File

@ -144,10 +144,37 @@ function isObjectLocked(bucket, objectMD, headers) {
return false;
}
function validateObjectLockUpdate(objectMD, retentionInfo, bypassGovernance) {
const { retentionMode: existingMode, retentionDate: existingDateISO } = objectMD;
if (!existingMode) {
return null;
}
const existingDate = new Date(existingDateISO);
const isExpired = existingDate < Date.now();
if (existingMode === 'GOVERNANCE' && !isExpired && !bypassGovernance) {
return errors.AccessDenied;
}
if (existingMode === 'COMPLIANCE') {
if (retentionInfo.mode === 'GOVERNANCE' && !isExpired) {
return errors.AccessDenied;
}
if (new Date(retentionInfo.date) < existingDate) {
return errors.AccessDenied;
}
}
return null;
}
module.exports = {
calculateRetainUntilDate,
compareObjectLockInformation,
setObjectLockInformation,
isObjectLocked,
validateHeaders,
validateObjectLockUpdate,
};

View File

@ -15,7 +15,6 @@ const collectCorsHeaders = require('../utilities/collectCorsHeaders');
const constants = require('../../constants');
const { versioningPreprocessing, checkQueryVersionId }
= require('./apiUtils/object/versioning');
const metadata = require('../metadata/wrapper');
const services = require('../services');
const { metadataValidateBucketAndObj } = require('../metadata/metadataUtils');
const { skipMpuPartProcessing } = require('../data/external/utils');
@ -126,44 +125,36 @@ function completeMultipartUpload(authInfo, request, log, callback) {
oldByteLength = objMD['content-length'];
}
services.metadataValidateMultipart(metadataValParams,
(err, mpuBucket) => {
(err, mpuBucket, mpuOverview, storedMetadata) => {
if (err) {
return next(err, destBucket);
}
return next(null, destBucket, objMD, mpuBucket);
return next(null, destBucket, objMD, mpuBucket,
storedMetadata);
});
},
function parsePartsList(destBucket, objMD, mpuBucket, next) {
function parsePartsList(destBucket, objMD, mpuBucket,
storedMetadata, next) {
const location = storedMetadata.controllingLocationConstraint;
// BACKWARD: Remove to remove the old splitter
if (mpuBucket.getMdBucketModelVersion() < 2) {
splitter = constants.oldSplitter;
}
// Reconstruct mpuOverviewKey to point to metadata
// originally stored when mpu initiated
const mpuOverviewKey =
`overview${splitter}${objectKey}${splitter}${uploadId}`;
if (request.post) {
return parseXml(request.post, (err, jsonList) => {
if (err) {
return next(err, destBucket);
}
return next(err, destBucket, objMD, mpuBucket, jsonList);
return next(null, destBucket, objMD, mpuBucket,
jsonList, storedMetadata, location, mpuOverviewKey);
});
}
return next(errors.MalformedXML, destBucket);
},
function getMPUMetadata(destBucket, objMD, mpuBucket, jsonList, next) {
// BACKWARD: Remove to remove the old splitter
if (mpuBucket.getMdBucketModelVersion() < 2) {
splitter = constants.oldSplitter;
}
// Reconstruct mpuOverviewKey to serve
// as key to pull metadata originally stored when mpu initiated
const mpuOverviewKey =
`overview${splitter}${objectKey}${splitter}${uploadId}`;
return metadata.getObjectMD(mpuBucket.getName(), mpuOverviewKey,
{}, log, (err, storedMetadata) => {
if (err) {
return next(err, destBucket);
}
const location = storedMetadata.controllingLocationConstraint;
return next(null, destBucket, objMD, mpuBucket, jsonList,
storedMetadata, location, mpuOverviewKey);
});
},
function retrieveParts(destBucket, objMD, mpuBucket, jsonList,
storedMetadata, location, mpuOverviewKey, next) {
return services.getMPUparts(mpuBucket.getName(), uploadId, log,
@ -354,6 +345,46 @@ function completeMultipartUpload(authInfo, request, log, callback) {
metaStoreParams, mpuBucket, keysToDelete, aggregateETag, objMD,
extraPartLocations, pseudoCipherBundle, dataToDelete,
completeObjData, next) {
if (objMD) {
// An object with the same key already exists, check
// if it has been created by the same MPU upload by
// checking if any of its internal location keys match
// the new keys. In such case, it must be a duplicate
// from a retry of a previous failed completion
// attempt, hence do the following:
//
// - skip writing the new metadata key to avoid
// creating a new version pointing to the same data
// keys
//
// - skip old data locations deletion since the old
// data location keys overlap the new ones (in
// principle they should be fully identical as there
// is no reuse of previous versions' data keys in
// the normal process) - note that the previous
// failed completion attempt may have left orphan
// data keys but we lost track of them so we cannot
// delete them now
//
// - proceed to the deletion of overview and part
// metadata keys, which are likely to have failed in
// the previous MPU completion attempt
//
const onlyDifferentLocationKeys = locationKeysSanityCheck(
objMD.location, dataLocations);
if (!onlyDifferentLocationKeys) {
log.info('MPU complete request replay detected', {
method: 'completeMultipartUpload.storeAsNewObj',
bucketName: destinationBucket.getName(),
objectKey: metaStoreParams.objectKey,
uploadId: metaStoreParams.uploadId,
});
return next(null, mpuBucket, keysToDelete, aggregateETag,
extraPartLocations, destinationBucket,
// pass the original version ID as generatedVersionId
objMD.versionId);
}
}
return services.metadataStoreObject(destinationBucket.getName(),
dataLocations, pseudoCipherBundle, metaStoreParams,
(err, res) => {
@ -367,31 +398,22 @@ function completeMultipartUpload(authInfo, request, log, callback) {
// unless the preexisting object and the completed mpu
// are on external backends
if (dataToDelete) {
// check keys against accidental deletion, if any
// existing key is found in the current object,
// deletion is skipped
const sanityCheckPassed =
locationKeysSanityCheck(dataToDelete,
dataLocations);
const newDataStoreName =
Array.isArray(dataLocations) && dataLocations[0] ?
dataLocations[0].dataStoreName : null;
if (sanityCheckPassed) {
const delLog =
logger.newRequestLoggerFromSerializedUids(log
.getSerializedUids());
return data.batchDelete(dataToDelete,
request.method,
newDataStoreName, delLog, err => {
if (err) {
return next(err);
}
return next(null, mpuBucket, keysToDelete,
aggregateETag, extraPartLocations,
destinationBucket, generatedVersionId);
});
}
const delLog =
logger.newRequestLoggerFromSerializedUids(log
.getSerializedUids());
return data.batchDelete(dataToDelete,
request.method,
newDataStoreName, delLog, err => {
if (err) {
return next(err);
}
return next(null, mpuBucket, keysToDelete,
aggregateETag, extraPartLocations,
destinationBucket, generatedVersionId);
});
}
return next(null, mpuBucket, keysToDelete, aggregateETag,
extraPartLocations, destinationBucket,

View File

@ -1,14 +1,17 @@
const async = require('async');
const { errors, s3middleware } = require('arsenal');
const { errors, s3middleware, auth, policies } = require('arsenal');
const vault = require('../auth/vault');
const { decodeVersionId, getVersionIdResHeader } =
require('./apiUtils/object/versioning');
const { validateObjectLockUpdate } =
require('./apiUtils/object/objectLockHelpers');
const { metadataValidateBucketAndObj } = require('../metadata/metadataUtils');
const { pushMetric } = require('../utapi/utilities');
const getReplicationInfo = require('./apiUtils/object/getReplicationInfo');
const collectCorsHeaders = require('../utilities/collectCorsHeaders');
const metadata = require('../metadata/wrapper');
const { config } = require('../Config');
const { parseRetentionXml } = s3middleware.retention;
const REPLICATION_ACTION = 'PUT_RETENTION';
@ -78,6 +81,57 @@ function objectPutRetention(authInfo, request, log, callback) {
parseRetentionXml(request.post, log,
(err, retentionInfo) => next(err, bucket, retentionInfo, objectMD));
},
(bucket, retentionInfo, objectMD, next) => {
if (objectMD.retentionMode === 'GOVERNANCE' && authInfo.isRequesterAnIAMUser()) {
log.trace('object in GOVERNANCE mode and is user, checking for attached policies',
{ method: 'objectPutRetention' });
const authParams = auth.server.extractParams(request, log, 's3',
request.query);
const ip = policies.requestUtils.getClientIp(request, config);
const requestContextParams = {
constantParams: {
headers: request.headers,
query: request.query,
generalResource: bucketName,
specificResource: { key: objectKey },
requesterIp: ip,
sslEnabled: request.connection.encrypted,
apiMethod: 'bypassGovernanceRetention',
awsService: 's3',
locationConstraint: bucket.getLocationConstraint(),
requesterInfo: authInfo,
signatureVersion: authParams.params.data.signatureVersion,
authType: authParams.params.data.authType,
signatureAge: authParams.params.data.signatureAge,
},
};
return vault.checkPolicies(requestContextParams,
authInfo.getArn(), log, (err, authorizationResults) => {
if (err) {
return next(err);
}
if (authorizationResults[0].isAllowed !== true) {
log.trace('authorization check failed for user',
{
'method': 'objectPutRetention',
's3:BypassGovernanceRetention': false,
});
return next(errors.AccessDenied);
}
return next(null, bucket, retentionInfo, objectMD);
});
}
return next(null, bucket, retentionInfo, objectMD);
},
(bucket, retentionInfo, objectMD, next) => {
const bypassHeader = request.headers['x-amz-bypass-governance-retention'] || '';
const bypassGovernance = bypassHeader.toLowerCase() === 'true';
const validationError = validateObjectLockUpdate(objectMD, retentionInfo, bypassGovernance);
if (validationError) {
return next(validationError, bucket, objectMD);
}
return next(null, bucket, retentionInfo, objectMD);
},
(bucket, retentionInfo, objectMD, next) => {
/* eslint-disable no-param-reassign */
objectMD.retentionMode = retentionInfo.mode;

View File

@ -449,13 +449,16 @@ const services = {
/**
* Checks whether bucket exists, multipart upload
* has been initatied and the user is authorized
* has been initiated and the user is authorized
* @param {object} params - custom built object containing
* bucket name, uploadId, authInfo etc.
* @param {function} cb - callback containing error and
* bucket reference for the next task
* @return {function} calls callback with arguments:
* error, bucket and the multipart upload metadata
* @return {undefined} calls callback with arguments:
* - error
* - bucket
* - the multipart upload metadata
* - the overview key stored metadata
*/
metadataValidateMultipart(params, cb) {
const { bucketName, uploadId, authInfo,
@ -522,7 +525,7 @@ const services = {
// If access was provided by the destination bucket's
// bucket policies, go ahead.
if (requestType === 'bucketPolicyGoAhead') {
return cb(null, mpuBucket, mpuOverview);
return cb(null, mpuBucket, mpuOverview, storedMetadata);
}
const requesterID = authInfo.isRequesterAnIAMUser() ?
@ -558,7 +561,7 @@ const services = {
return cb(errors.AccessDenied);
}
}
return cb(null, mpuBucket, mpuOverview);
return cb(null, mpuBucket, mpuOverview, storedMetadata);
});
return undefined;
});

View File

@ -20,7 +20,7 @@
"homepage": "https://github.com/scality/S3#readme",
"dependencies": {
"@hapi/joi": "^17.1.0",
"arsenal": "github:scality/Arsenal#65966f5",
"arsenal": "github:scality/Arsenal#0277677",
"async": "~2.5.0",
"aws-sdk": "2.831.0",
"azure-storage": "^2.1.0",
@ -37,7 +37,7 @@
"npm-run-all": "~4.1.5",
"sinon": "^9.0.2",
"sproxydclient": "scality/sproxydclient#44f025b",
"utapi": "scality/utapi#1af6532",
"utapi": "scality/utapi#db5315b",
"utf8": "~2.1.1",
"uuid": "^3.0.1",
"vaultclient": "scality/vaultclient#21d03b1",
@ -96,4 +96,4 @@
"unit_coverage": "CI=true mkdir -p coverage/unit/ && S3BACKEND=mem istanbul cover --dir coverage/unit _mocha -- --reporter mocha-multi-reporters --reporter-options configFile=$INIT_CWD/tests/reporter-config.json --recursive tests/unit",
"unit_coverage_legacy_location": "CI=true mkdir -p coverage/unitlegacylocation/ && S3_LOCATION_FILE=tests/locationConfig/locationConfigLegacy.json S3BACKEND=mem istanbul cover --dir coverage/unitlegacylocation _mocha -- --reporter mocha-multi-reporters --reporter-options configFile=$INIT_CWD/tests/reporter-config.json --reporter mocha-junit-reporter --recursive tests/unit"
}
}
}

View File

@ -662,13 +662,19 @@ describe('GET object', () => {
it('If-None-Match & If-Modified-Since: returns NotModified when ' +
'Etag does not match and lastModified is greater',
done => {
requestGet({
const req = s3.getObject({
Bucket: bucketName,
Key: objectName,
IfNoneMatch: etagTrim,
IfModifiedSince: dateFromNow(-1),
IfModifiedSince: dateFromNow(1),
}, err => {
checkError(err, 'NotModified');
done();
});
req.on('httpHeaders', (code, headers) => {
assert(!headers['content-type']);
assert(!headers['content-length']);
});
});
it('If-None-Match not match & If-Modified-Since not match',

View File

@ -32,4 +32,10 @@ describe('Sanity check for location keys', () => {
const curr = [{ key: 'ddd' }, { key: 'eee' }, { key: 'fff' }];
assert.strictEqual(locationKeysSanityCheck(prev, curr), true);
});
it('should return true if prev location is null', () => {
const prev = null;
const curr = [{ key: 'ddd' }, { key: 'eee' }, { key: 'fff' }];
assert.strictEqual(locationKeysSanityCheck(prev, curr), true);
});
});

View File

@ -6,6 +6,7 @@ const { DummyRequestLogger } = require('../../helpers');
const {
calculateRetainUntilDate,
validateHeaders,
validateObjectLockUpdate,
} = require('../../../../lib/api/apiUtils/object/objectLockHelpers');
const mockName = 'testbucket';
@ -176,3 +177,110 @@ describe('objectLockHelpers: calculateRetainUntilDate', () => {
expectedRetainUntilDate.toISOString().slice(0, 16));
});
});
describe('objectLockHelpers: validateObjectLockUpdate', () => {
it('should allow GOVERNANCE => COMPLIANCE if bypassGovernanceRetention is true', () => {
const objMD = {
retentionMode: 'GOVERNANCE',
retentionDate: moment().add(1, 'days').toISOString(),
};
const retentionInfo = {
mode: 'COMPLIANCE',
date: moment().add(1, 'days').toISOString(),
};
const error = validateObjectLockUpdate(objMD, retentionInfo, true);
assert.strictEqual(error, null);
});
it('should disallow GOVERNANCE => COMPLIANCE if bypassGovernanceRetention is false', () => {
const objMD = {
retentionMode: 'GOVERNANCE',
retentionDate: moment().add(1, 'days').toISOString(),
};
const retentionInfo = {
mode: 'COMPLIANCE',
date: moment().add(1, 'days').toISOString(),
};
const error = validateObjectLockUpdate(objMD, retentionInfo, false);
assert.deepStrictEqual(error, errors.AccessDenied);
});
it('should disallow COMPLIANCE => GOVERNANCE if retention is not expired', () => {
const objMD = {
retentionMode: 'COMPLIANCE',
retentionDate: moment().add(1, 'days').toISOString(),
};
const retentionInfo = {
mode: 'GOVERNANCE',
date: moment().add(1, 'days').toISOString(),
};
const error = validateObjectLockUpdate(objMD, retentionInfo);
assert.deepStrictEqual(error, errors.AccessDenied);
});
it('should allow COMPLIANCE => GOVERNANCE if retention is expired', () => {
const objMD = {
retentionMode: 'COMPLIANCE',
retentionDate: moment().subtract(1, 'days').toISOString(),
};
const retentionInfo = {
mode: 'GOVERNANCE',
date: moment().add(1, 'days').toISOString(),
};
const error = validateObjectLockUpdate(objMD, retentionInfo);
assert.strictEqual(error, null);
});
it('should allow extending retention period if in COMPLIANCE', () => {
const objMD = {
retentionMode: 'COMPLIANCE',
retentionDate: moment().add(1, 'days').toISOString(),
};
const retentionInfo = {
mode: 'COMPLIANCE',
date: moment().add(2, 'days').toISOString(),
};
const error = validateObjectLockUpdate(objMD, retentionInfo);
assert.strictEqual(error, null);
});
it('should disallow shortening retention period if in COMPLIANCE', () => {
const objMD = {
retentionMode: 'COMPLIANCE',
retentionDate: moment().add(2, 'days').toISOString(),
};
const retentionInfo = {
mode: 'COMPLIANCE',
date: moment().add(1, 'days').toISOString(),
};
const error = validateObjectLockUpdate(objMD, retentionInfo);
assert.deepStrictEqual(error, errors.AccessDenied);
});
it('should allow shortening retention period if in GOVERNANCE', () => {
const objMD = {
retentionMode: 'GOVERNANCE',
retentionDate: moment().add(2, 'days').toISOString(),
};
const retentionInfo = {
mode: 'GOVERNANCE',
date: moment().add(1, 'days').toISOString(),
};
const error = validateObjectLockUpdate(objMD, retentionInfo, true);
assert.strictEqual(error, null);
});
});

View File

@ -20,6 +20,7 @@ const getObjectRetention = require('../../../lib/api/objectGetRetention');
const initiateMultipartUpload
= require('../../../lib/api/initiateMultipartUpload');
const { metadata } = require('../../../lib/metadata/in_memory/metadata');
const metadataBackend = require('../../../lib/metadata/in_memory/backend');
const multipartDelete = require('../../../lib/api/multipartDelete');
const objectPutPart = require('../../../lib/api/objectPutPart');
const DummyRequest = require('../DummyRequest');
@ -1720,6 +1721,68 @@ describe('Multipart Upload API', () => {
});
});
});
it('should not delete data locations on completeMultipartUpload retry',
done => {
const partBody = Buffer.from('foo', 'utf8');
let origDeleteObject;
async.waterfall([
next =>
bucketPut(authInfo, bucketPutRequest, log, err => next(err)),
next =>
initiateMultipartUpload(authInfo, initiateRequest, log, next),
(result, corsHeaders, next) => parseString(result, next),
(json, next) => {
const testUploadId =
json.InitiateMultipartUploadResult.UploadId[0];
const partRequest = _createPutPartRequest(testUploadId, 1,
partBody);
objectPutPart(authInfo, partRequest, undefined, log,
(err, eTag) => next(err, eTag, testUploadId));
},
(eTag, testUploadId, next) => {
origDeleteObject = metadataBackend.deleteObject;
metadataBackend.deleteObject = (
bucketName, objName, params, log, cb) => {
// prevent deletions from MPU bucket only
if (bucketName === mpuBucket) {
return process.nextTick(
() => cb(errors.InternalError));
}
return origDeleteObject(
bucketName, objName, params, log, cb);
};
const parts = [{ partNumber: 1, eTag }];
const completeRequest = _createCompleteMpuRequest(
testUploadId, parts);
completeMultipartUpload(authInfo, completeRequest, log, err => {
// expect a failure here because we could not
// remove the overview key
assert.strictEqual(err, errors.InternalError);
next(null, eTag, testUploadId);
});
},
(eTag, testUploadId, next) => {
// allow MPU bucket metadata deletions to happen again
metadataBackend.deleteObject = origDeleteObject;
// retry the completeMultipartUpload with the same
// metadata, as an application would normally do after
// a failure
const parts = [{ partNumber: 1, eTag }];
const completeRequest = _createCompleteMpuRequest(
testUploadId, parts);
completeMultipartUpload(authInfo, completeRequest, log, next);
},
], err => {
assert.ifError(err);
// check that the original data has not been deleted
// during the replay
assert.strictEqual(ds[0], undefined);
assert.notStrictEqual(ds[1], undefined);
assert.deepStrictEqual(ds[1].value, partBody);
done();
});
});
});
describe('complete mpu with versioning', () => {
@ -1734,28 +1797,9 @@ describe('complete mpu with versioning', () => {
versioningTestUtils.createPutObjectRequest(bucketName, objectKey,
data));
before(done => {
beforeEach(done => {
cleanup();
async.series([
callback => bucketPut(authInfo, bucketPutRequest, log,
callback),
// putting null version: put obj before versioning configured
callback => objectPut(authInfo, testPutObjectRequests[0],
undefined, log, callback),
callback => bucketPutVersioning(authInfo,
enableVersioningRequest, log, callback),
// put another version:
callback => objectPut(authInfo, testPutObjectRequests[1],
undefined, log, callback),
callback => bucketPutVersioning(authInfo,
suspendVersioningRequest, log, callback),
], err => {
if (err) {
return done(err);
}
versioningTestUtils.assertDataStoreValues(ds, objData.slice(0, 2));
return done();
});
bucketPut(authInfo, bucketPutRequest, log, done);
});
after(done => {
@ -1766,8 +1810,21 @@ describe('complete mpu with versioning', () => {
it('should delete null version when creating new null version, ' +
'even when null version is not the latest version', done => {
async.waterfall([
next =>
initiateMultipartUpload(authInfo, initiateRequest, log, next),
// putting null version: put obj before versioning configured
next => objectPut(authInfo, testPutObjectRequests[0],
undefined, log, err => next(err)),
next => bucketPutVersioning(authInfo,
enableVersioningRequest, log, err => next(err)),
// put another version:
next => objectPut(authInfo, testPutObjectRequests[1],
undefined, log, err => next(err)),
next => bucketPutVersioning(authInfo,
suspendVersioningRequest, log, err => next(err)),
next => {
versioningTestUtils.assertDataStoreValues(
ds, objData.slice(0, 2));
initiateMultipartUpload(authInfo, initiateRequest, log, next);
},
(result, corsHeaders, next) => parseString(result, next),
(json, next) => {
const partBody = objData[2];
@ -1793,6 +1850,76 @@ describe('complete mpu with versioning', () => {
});
});
});
it('should finish deleting metadata on completeMultipartUpload retry',
done => {
let origDeleteObject;
async.waterfall([
next => bucketPutVersioning(authInfo,
enableVersioningRequest, log, err => next(err)),
next =>
initiateMultipartUpload(authInfo, initiateRequest, log, next),
(result, corsHeaders, next) => parseString(result, next),
(json, next) => {
const partBody = objData[2];
const testUploadId =
json.InitiateMultipartUploadResult.UploadId[0];
const partRequest = _createPutPartRequest(testUploadId, 1,
partBody);
objectPutPart(authInfo, partRequest, undefined, log,
(err, eTag) => next(err, eTag, testUploadId));
},
(eTag, testUploadId, next) => {
origDeleteObject = metadataBackend.deleteObject;
metadataBackend.deleteObject = (
bucketName, objName, params, log, cb) => {
// prevent deletions from MPU bucket only
if (bucketName === mpuBucket) {
return process.nextTick(
() => cb(errors.InternalError));
}
return origDeleteObject(
bucketName, objName, params, log, cb);
};
const parts = [{ partNumber: 1, eTag }];
const completeRequest = _createCompleteMpuRequest(
testUploadId, parts);
completeMultipartUpload(authInfo, completeRequest, log, err => {
// expect a failure here because we could not
// remove the overview key
assert.strictEqual(err, errors.InternalError);
next(null, eTag, testUploadId);
});
},
(eTag, testUploadId, next) => {
// allow MPU bucket metadata deletions to happen again
metadataBackend.deleteObject = origDeleteObject;
// retry the completeMultipartUpload with the same
// metadata, as an application would normally do after
// a failure
const parts = [{ partNumber: 1, eTag }];
const completeRequest = _createCompleteMpuRequest(
testUploadId, parts);
completeMultipartUpload(authInfo, completeRequest, log, next);
},
], err => {
assert.ifError(err);
let nbVersions = 0;
for (const key of metadata.keyMaps.get(bucketName).keys()) {
if (key !== objectKey && key.startsWith(objectKey)) {
nbVersions += 1;
}
}
// There should be only one version of the object, since
// the second call should not have created a new version
assert.strictEqual(nbVersions, 1);
for (const key of metadata.keyMaps.get(mpuBucket).keys()) {
assert.fail('There should be no more keys in MPU bucket, ' +
`found "${key}"`);
}
done();
});
});
});
describe('multipart upload with object lock', () => {

View File

@ -1,5 +1,7 @@
const assert = require('assert');
const moment = require('moment');
const { errors } = require('arsenal');
const { bucketPut } = require('../../../lib/api/bucketPut');
const objectPut = require('../../../lib/api/objectPut');
const objectPutRetention = require('../../../lib/api/objectPutRetention');
@ -31,17 +33,53 @@ const putObjectRequest = new DummyRequest({
url: `/${bucketName}/${objectName}`,
}, postBody);
const objectRetentionXml = '<Retention ' +
const objectRetentionXmlGovernance = '<Retention ' +
'xmlns="http://s3.amazonaws.com/doc/2006-03-01/">' +
'<Mode>GOVERNANCE</Mode>' +
`<RetainUntilDate>${date.toISOString()}</RetainUntilDate>` +
'</Retention>';
const putObjRetRequest = {
const objectRetentionXmlCompliance = '<Retention ' +
'xmlns="http://s3.amazonaws.com/doc/2006-03-01/">' +
'<Mode>COMPLIANCE</Mode>' +
`<RetainUntilDate>${moment().add(2, 'days').toISOString()}</RetainUntilDate>` +
'</Retention>';
const objectRetentionXmlComplianceShorter = '<Retention ' +
'xmlns="http://s3.amazonaws.com/doc/2006-03-01/">' +
'<Mode>COMPLIANCE</Mode>' +
`<RetainUntilDate>${moment().add(1, 'days').toISOString()}</RetainUntilDate>` +
'</Retention>';
const putObjRetRequestGovernance = {
bucketName,
objectKey: objectName,
headers: { host: `${bucketName}.s3.amazonaws.com` },
post: objectRetentionXml,
post: objectRetentionXmlGovernance,
};
const putObjRetRequestGovernanceWithHeader = {
bucketName,
objectKey: objectName,
headers: {
'host': `${bucketName}.s3.amazonaws.com`,
'x-amz-bypass-governance-retention': 'true',
},
post: objectRetentionXmlGovernance,
};
const putObjRetRequestCompliance = {
bucketName,
objectKey: objectName,
headers: { host: `${bucketName}.s3.amazonaws.com` },
post: objectRetentionXmlCompliance,
};
const putObjRetRequestComplianceShorter = {
bucketName,
objectKey: objectName,
headers: { host: `${bucketName}.s3.amazonaws.com` },
post: objectRetentionXmlComplianceShorter,
};
const expectedMode = 'GOVERNANCE';
@ -60,7 +98,7 @@ describe('putObjectRetention API', () => {
afterEach(() => cleanup());
it('should return InvalidRequest error', done => {
objectPutRetention(authInfo, putObjRetRequest, log, err => {
objectPutRetention(authInfo, putObjRetRequestGovernance, log, err => {
assert.strictEqual(err.InvalidRequest, true);
done();
});
@ -80,7 +118,7 @@ describe('putObjectRetention API', () => {
afterEach(() => cleanup());
it('should update an object\'s metadata with retention info', done => {
objectPutRetention(authInfo, putObjRetRequest, log, err => {
objectPutRetention(authInfo, putObjRetRequestGovernance, log, err => {
assert.ifError(err);
return metadata.getObjectMD(bucketName, objectName, {}, log,
(err, objMD) => {
@ -91,5 +129,48 @@ describe('putObjectRetention API', () => {
});
});
});
it('should disallow COMPLIANCE => GOVERNANCE', done => {
objectPutRetention(authInfo, putObjRetRequestCompliance, log, err => {
assert.ifError(err);
return objectPutRetention(authInfo, putObjRetRequestGovernance, log, err => {
assert.deepStrictEqual(err, errors.AccessDenied);
done();
});
});
});
it('should disallow shortening of COMPLIANCE retention', done => {
objectPutRetention(authInfo, putObjRetRequestCompliance, log, err => {
assert.ifError(err);
return objectPutRetention(authInfo, putObjRetRequestComplianceShorter, log, err => {
assert.deepStrictEqual(err, errors.AccessDenied);
done();
});
});
});
it('should disallow update if the x-amz-bypass-governance-retention header is missing and'
+ 'GOVERNANCE mode is enabled', done => {
objectPutRetention(authInfo, putObjRetRequestGovernance, log, err => {
assert.ifError(err);
return objectPutRetention(authInfo, putObjRetRequestGovernance, log, err => {
assert.deepStrictEqual(err, errors.AccessDenied);
done();
});
});
});
it('should allow update if the x-amz-bypass-governance-retention header is present and'
+ 'GOVERNANCE mode is enabled', done => {
objectPutRetention(authInfo, putObjRetRequestGovernance, log, err => {
assert.ifError(err);
return objectPutRetention(authInfo, putObjRetRequestGovernanceWithHeader, log, err => {
assert.ifError(err);
done();
});
});
});
});
});

265
yarn.lock
View File

@ -656,9 +656,9 @@ arraybuffer.slice@~0.0.7:
resolved "https://registry.yarnpkg.com/arraybuffer.slice/-/arraybuffer.slice-0.0.7.tgz#3bbc4275dd584cc1b10809b89d4e8b63a69e7675"
integrity sha512-wGUIVQXuehL5TCqQun8OW81jGzAWycqzFF8lFp+GOM5BXLYj3bKNsYC4daB7n6XjCqxQA/qgTJ+8ANR3acjrog==
"arsenal@github:scality/Arsenal#65966f5", arsenal@scality/Arsenal#65966f5:
"arsenal@github:scality/Arsenal#0277677":
version "7.7.0"
resolved "https://codeload.github.com/scality/Arsenal/tar.gz/65966f5ddf93b048906d14a8c26056abfd4c22ba"
resolved "https://codeload.github.com/scality/Arsenal/tar.gz/0277677320677c146a08d0fa65bad547e95a4765"
dependencies:
"@hapi/joi" "^15.1.0"
JSONStream "^1.0.0"
@ -707,6 +707,67 @@ arsenal@scality/Arsenal#580e25a:
optionalDependencies:
ioctl "2.0.0"
arsenal@scality/Arsenal#65966f5:
version "7.7.0"
resolved "https://codeload.github.com/scality/Arsenal/tar.gz/65966f5ddf93b048906d14a8c26056abfd4c22ba"
dependencies:
"@hapi/joi" "^15.1.0"
JSONStream "^1.0.0"
agentkeepalive "^4.1.3"
ajv "6.12.2"
async "~2.1.5"
debug "~2.6.9"
diskusage "^1.1.1"
ioredis "4.9.5"
ipaddr.js "1.9.1"
level "~5.0.1"
level-sublevel "~6.6.5"
node-forge "^0.7.1"
simple-glob "^0.2"
socket.io "~2.3.0"
socket.io-client "~2.3.0"
utf8 "2.1.2"
uuid "^3.0.1"
werelogs scality/werelogs#0ff7ec82
xml2js "~0.4.23"
optionalDependencies:
ioctl "2.0.0"
arsenal@scality/Arsenal#8ed8478:
version "8.2.1"
resolved "https://codeload.github.com/scality/Arsenal/tar.gz/8ed84786fce31f603b1e8cd641b3b44b8f715b0b"
dependencies:
"@hapi/joi" "^15.1.0"
JSONStream "^1.0.0"
ajv "6.12.2"
async "~2.6.1"
aws-sdk "2.80.0"
azure-storage "^2.1.0"
backo "^1.1.0"
bson "4.0.0"
debug "~4.1.0"
diskusage "^1.1.1"
fcntl "github:scality/node-fcntl"
hdclient scality/hdclient#5145e04e5ed33e85106765b1caa90cd245ef482b
https-proxy-agent "^2.2.0"
ioredis "4.9.5"
ipaddr.js "1.9.1"
level "~5.0.1"
level-sublevel "~6.6.5"
mongodb "^3.0.1"
node-forge "^0.7.1"
prom-client "10.2.3"
simple-glob "^0.2.0"
socket.io "~2.3.0"
socket.io-client "~2.3.0"
sproxydclient "github:scality/sproxydclient#30e7115"
utf8 "3.0.0"
uuid "^3.0.1"
werelogs scality/werelogs#0ff7ec82
xml2js "~0.4.23"
optionalDependencies:
ioctl "2.0.1"
arsenal@scality/Arsenal#9f2e74e:
version "7.4.3"
resolved "https://codeload.github.com/scality/Arsenal/tar.gz/9f2e74ec6972527c2a9ca6ecb4155618f123fc19"
@ -757,41 +818,6 @@ arsenal@scality/Arsenal#b03f5b8:
optionalDependencies:
ioctl "2.0.0"
arsenal@scality/Arsenal#c57cde8:
version "8.1.4"
resolved "https://codeload.github.com/scality/Arsenal/tar.gz/c57cde88bb04fe9803ec08c3a883f6eb986e4149"
dependencies:
JSONStream "^1.0.0"
ajv "4.10.0"
async "~2.6.1"
aws-sdk "2.80.0"
azure-storage "^2.1.0"
backo "^1.1.0"
bson "4.0.0"
debug "~4.1.0"
diskusage "^1.1.1"
fcntl "github:scality/node-fcntl"
hdclient scality/hdclient#5145e04e5ed33e85106765b1caa90cd245ef482b
https-proxy-agent "^2.2.0"
ioredis "4.9.5"
ipaddr.js "1.8.1"
joi "^14.3.0"
level "~5.0.1"
level-sublevel "~6.6.5"
mongodb "^3.0.1"
node-forge "^0.7.1"
prom-client "10.2.3"
simple-glob "^0.2.0"
socket.io "~2.2.0"
socket.io-client "~2.2.0"
sproxydclient "github:scality/sproxydclient#a6ec980"
utf8 "3.0.0"
uuid "^3.0.1"
werelogs scality/werelogs#0ff7ec82
xml2js "~0.4.16"
optionalDependencies:
ioctl "2.0.1"
asap@~2.0.3:
version "2.0.6"
resolved "https://registry.yarnpkg.com/asap/-/asap-2.0.6.tgz#e50347611d7e690943208bbdafebcbc2fb866d46"
@ -1206,7 +1232,7 @@ bucketclient@scality/bucketclient:
resolved "https://codeload.github.com/scality/bucketclient/tar.gz/07d5a3813878b2746ed00e41c177bc2a0460e243"
dependencies:
agentkeepalive "^4.1.3"
arsenal scality/Arsenal#c57cde8
arsenal scality/Arsenal#8ed8478
werelogs scality/werelogs#351a2a3
bucketclient@scality/bucketclient#6d2d5a4:
@ -1761,6 +1787,13 @@ debug@^3.1.0, debug@^3.2.5, debug@^3.2.6:
dependencies:
ms "^2.1.1"
debug@^4.3.1:
version "4.3.2"
resolved "https://registry.yarnpkg.com/debug/-/debug-4.3.2.tgz#f0a49c18ac8779e31d4a0c6029dfb76873c7428b"
integrity sha512-mOp8wKcvj7XxC78zLgw/ZA+6TSgkoE2C/ienthhRD298T7UNwAg9diBpLRxC0mOezLl4B0xV7M0cCO6P/O0Xhw==
dependencies:
ms "2.1.2"
debug@~4.1.0:
version "4.1.1"
resolved "https://registry.yarnpkg.com/debug/-/debug-4.1.1.tgz#3b72260255109c6b589cee050f1d516139664791"
@ -2049,23 +2082,6 @@ engine.io-client@~1.8.4:
xmlhttprequest-ssl "1.5.3"
yeast "0.1.2"
engine.io-client@~3.3.1:
version "3.3.2"
resolved "https://registry.yarnpkg.com/engine.io-client/-/engine.io-client-3.3.2.tgz#04e068798d75beda14375a264bb3d742d7bc33aa"
integrity sha512-y0CPINnhMvPuwtqXfsGuWE8BB66+B6wTtCofQDRecMQPYX3MYUZXFNKDhdrSe3EVjgOu4V3rxdeqN/Tr91IgbQ==
dependencies:
component-emitter "1.2.1"
component-inherit "0.0.3"
debug "~3.1.0"
engine.io-parser "~2.1.1"
has-cors "1.1.0"
indexof "0.0.1"
parseqs "0.0.5"
parseuri "0.0.5"
ws "~6.1.0"
xmlhttprequest-ssl "~1.5.4"
yeast "0.1.2"
engine.io-client@~3.4.0:
version "3.4.4"
resolved "https://registry.yarnpkg.com/engine.io-client/-/engine.io-client-3.4.4.tgz#77d8003f502b0782dd792b073a4d2cf7ca5ab967"
@ -2095,17 +2111,6 @@ engine.io-parser@1.3.2:
has-binary "0.1.7"
wtf-8 "1.0.0"
engine.io-parser@~2.1.0, engine.io-parser@~2.1.1:
version "2.1.3"
resolved "https://registry.yarnpkg.com/engine.io-parser/-/engine.io-parser-2.1.3.tgz#757ab970fbf2dfb32c7b74b033216d5739ef79a6"
integrity sha512-6HXPre2O4Houl7c4g7Ic/XzPnHBvaEmN90vtRO9uLmwtRqQmTOw0QMevL1TOfL2Cpu1VzsaTmMotQgMdkzGkVA==
dependencies:
after "0.8.2"
arraybuffer.slice "~0.0.7"
base64-arraybuffer "0.1.5"
blob "0.0.5"
has-binary2 "~1.0.2"
engine.io-parser@~2.2.0:
version "2.2.1"
resolved "https://registry.yarnpkg.com/engine.io-parser/-/engine.io-parser-2.2.1.tgz#57ce5611d9370ee94f99641b589f94c97e4f5da7"
@ -2129,18 +2134,6 @@ engine.io@~1.8.4:
engine.io-parser "1.3.2"
ws "~1.1.5"
engine.io@~3.3.1:
version "3.3.2"
resolved "https://registry.yarnpkg.com/engine.io/-/engine.io-3.3.2.tgz#18cbc8b6f36e9461c5c0f81df2b830de16058a59"
integrity sha512-AsaA9KG7cWPXWHp5FvHdDWY3AMWeZ8x+2pUVLcn71qE5AtAzgGbxuclOytygskw8XGmiQafTmnI9Bix3uihu2w==
dependencies:
accepts "~1.3.4"
base64id "1.0.0"
cookie "0.3.1"
debug "~3.1.0"
engine.io-parser "~2.1.0"
ws "~6.1.0"
engine.io@~3.4.0:
version "3.4.2"
resolved "https://registry.yarnpkg.com/engine.io/-/engine.io-3.4.2.tgz#8fc84ee00388e3e228645e0a7d3dfaeed5bd122c"
@ -3072,11 +3065,6 @@ hoek@4.x.x:
resolved "https://registry.yarnpkg.com/hoek/-/hoek-4.2.1.tgz#9634502aa12c445dd5a7c5734b572bb8738aacbb"
integrity sha512-QLg82fGkfnJ/4iy1xZ81/9SIJiq1NGFUMGs6ParyjBZr6jW2Ufj/snDqTHixNlHdPNwN2RLVD0Pi3igeK9+JfA==
hoek@6.x.x:
version "6.1.3"
resolved "https://registry.yarnpkg.com/hoek/-/hoek-6.1.3.tgz#73b7d33952e01fe27a38b0457294b79dd8da242c"
integrity sha512-YXXAAhmF9zpQbC7LEcREFtXfGq5K1fmd+4PHkBq8NUqmzW3G+Dq10bI/i0KucLRwss3YYFQ0fSfoxBZYiGUqtQ==
hosted-git-info@^2.1.4, hosted-git-info@^2.7.1:
version "2.8.8"
resolved "https://registry.yarnpkg.com/hosted-git-info/-/hosted-git-info-2.8.8.tgz#7539bd4bc1e0e0a895815a2e0262420b12858488"
@ -3329,21 +3317,22 @@ ioredis@4.9.5:
redis-parser "^3.0.0"
standard-as-callback "^2.0.1"
ioredis@^4.9.5:
version "4.19.4"
resolved "https://registry.yarnpkg.com/ioredis/-/ioredis-4.19.4.tgz#11112005f87ad3acac247ada3b22eb31b947f7c7"
integrity sha512-3haQWw9dpEjcfVcRktXlayVNrrqvvc2io7Q/uiV2UsYw8/HC2YwwJr78Wql7zu5bzwci0x9bZYA69U7KkevAvw==
ioredis@^4.28.0:
version "4.28.0"
resolved "https://registry.yarnpkg.com/ioredis/-/ioredis-4.28.0.tgz#5a2be3f37ff2075e2332f280eaeb02ab4d9ff0d3"
integrity sha512-I+zkeeWp3XFgPT2CtJKxvaF5FjGBGt4yGYljRjQecdQKteThuAsKqffeF1lgHVlYnuNeozRbPOCDNZ7tDWPeig==
dependencies:
cluster-key-slot "^1.1.0"
debug "^4.1.1"
debug "^4.3.1"
denque "^1.1.0"
lodash.defaults "^4.2.0"
lodash.flatten "^4.4.0"
lodash.isarguments "^3.1.0"
p-map "^2.1.0"
redis-commands "1.6.0"
redis-commands "1.7.0"
redis-errors "^1.2.0"
redis-parser "^3.0.0"
standard-as-callback "^2.0.1"
standard-as-callback "^2.1.0"
ip@1.1.5, ip@^1.1.5:
version "1.1.5"
@ -3355,11 +3344,6 @@ ipaddr.js@1.2.0:
resolved "https://registry.yarnpkg.com/ipaddr.js/-/ipaddr.js-1.2.0.tgz#8aba49c9192799585bdd643e0ccb50e8ae777ba4"
integrity sha1-irpJyRknmVhb3WQ+DMtQ6K53e6Q=
ipaddr.js@1.8.1:
version "1.8.1"
resolved "https://registry.yarnpkg.com/ipaddr.js/-/ipaddr.js-1.8.1.tgz#fa4b79fa47fd3def5e3b159825161c0a519c9427"
integrity sha1-+kt5+kf9Pe9eOxWYJRYcClGclCc=
ipaddr.js@1.9.1:
version "1.9.1"
resolved "https://registry.yarnpkg.com/ipaddr.js/-/ipaddr.js-1.9.1.tgz#bff38543eeb8984825079ff3a2a8e6cbd46781b3"
@ -3604,13 +3588,6 @@ isemail@2.x.x:
resolved "https://registry.yarnpkg.com/isemail/-/isemail-2.2.1.tgz#0353d3d9a62951080c262c2aa0a42b8ea8e9e2a6"
integrity sha1-A1PT2aYpUQgMJiwqoKQrjqjp4qY=
isemail@3.x.x:
version "3.2.0"
resolved "https://registry.yarnpkg.com/isemail/-/isemail-3.2.0.tgz#59310a021931a9fb06bbb51e155ce0b3f236832c"
integrity sha512-zKqkK+O+dGqevc93KNsbZ/TqTUFd46MwWjYOoMrjIMZ51eU7DtQG3Wmd9SQQT7i7RVnuTPEiYEWHU3MSbxC1Tg==
dependencies:
punycode "2.x.x"
isexe@^2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/isexe/-/isexe-2.0.0.tgz#e8fbf374dc556ff8947a10dcb0572d633f2cfa10"
@ -3750,15 +3727,6 @@ joi@^10.6:
items "2.x.x"
topo "2.x.x"
joi@^14.3.0:
version "14.3.1"
resolved "https://registry.yarnpkg.com/joi/-/joi-14.3.1.tgz#164a262ec0b855466e0c35eea2a885ae8b6c703c"
integrity sha512-LQDdM+pkOrpAn4Lp+neNIFV3axv1Vna3j38bisbQhETPMANYRbFJFUyOZcOClYvM/hppMhGWuKSFEK9vjrB+bQ==
dependencies:
hoek "6.x.x"
isemail "3.x.x"
topo "3.x.x"
"js-tokens@^3.0.0 || ^4.0.0":
version "4.0.0"
resolved "https://registry.yarnpkg.com/js-tokens/-/js-tokens-4.0.0.tgz#19203fb59991df98e3a287050d4647cdeaf32499"
@ -4210,6 +4178,11 @@ lodash.invert@^4.3.0:
resolved "https://registry.yarnpkg.com/lodash.invert/-/lodash.invert-4.3.0.tgz#8ffe20d4b616f56bea8f1aa0c6ebd80dcf742aee"
integrity sha1-j/4g1LYW9WvqjxqgxuvYDc90Ku4=
lodash.isarguments@^3.1.0:
version "3.1.0"
resolved "https://registry.yarnpkg.com/lodash.isarguments/-/lodash.isarguments-3.1.0.tgz#2f573d85c6a24289ff00663b491c1d338ff3458a"
integrity sha1-L1c9hcaiQon/AGY7SRwdM4/zRYo=
lodash.isboolean@^3.0.3:
version "3.0.3"
resolved "https://registry.yarnpkg.com/lodash.isboolean/-/lodash.isboolean-3.0.3.tgz#6c2e171db2a257cd96802fd43b01b20d5f5870f6"
@ -5404,7 +5377,7 @@ punycode@1.3.2:
resolved "https://registry.yarnpkg.com/punycode/-/punycode-1.3.2.tgz#9653a036fb7c1ee42342f2325cceefea3926c48d"
integrity sha1-llOgNvt8HuQjQvIyXM7v6jkmxI0=
punycode@2.x.x, punycode@^2.1.0, punycode@^2.1.1:
punycode@^2.1.0, punycode@^2.1.1:
version "2.1.1"
resolved "https://registry.yarnpkg.com/punycode/-/punycode-2.1.1.tgz#b58b010ac40c22c5657616c8d2c2c02c7bf479ec"
integrity sha512-XRsRjdf+j5ml+y/6GKHPZbrF/8p2Yga0JPtdqTIY2Xe5ohJPD9saDJJLPvp9+NSBprVvevdXZybnj2cv8OEd0A==
@ -5553,10 +5526,10 @@ redis-commands@1.4.0:
resolved "https://registry.yarnpkg.com/redis-commands/-/redis-commands-1.4.0.tgz#52f9cf99153efcce56a8f86af986bd04e988602f"
integrity sha512-cu8EF+MtkwI4DLIT0x9P8qNTLFhQD4jLfxLR0cCNkeGzs87FN6879JOJwNQR/1zD7aSYNbU0hgsV9zGY71Itvw==
redis-commands@1.6.0:
version "1.6.0"
resolved "https://registry.yarnpkg.com/redis-commands/-/redis-commands-1.6.0.tgz#36d4ca42ae9ed29815cdb30ad9f97982eba1ce23"
integrity sha512-2jnZ0IkjZxvguITjFTrGiLyzQZcTvaw8DAaCXxZq/dsHXz7KfMQ3OUJy7Tz9vnRtZRVz6VRCPDvruvU8Ts44wQ==
redis-commands@1.7.0:
version "1.7.0"
resolved "https://registry.yarnpkg.com/redis-commands/-/redis-commands-1.7.0.tgz#15a6fea2d58281e27b1cd1acfb4b293e278c3a89"
integrity sha512-nJWqw3bTFy21hX/CPKHth6sfhZbdiHP6bTawSgQBlKOVRG7EZkfHbbHwQJnrE4vsQf0CMNE+3gJ4Fmm16vdVlQ==
redis-errors@^1.0.0, redis-errors@^1.2.0:
version "1.2.0"
@ -6306,26 +6279,6 @@ socket.io-client@1.7.4, socket.io-client@~1.7.3:
socket.io-parser "2.3.1"
to-array "0.1.4"
socket.io-client@2.2.0, socket.io-client@~2.2.0:
version "2.2.0"
resolved "https://registry.yarnpkg.com/socket.io-client/-/socket.io-client-2.2.0.tgz#84e73ee3c43d5020ccc1a258faeeb9aec2723af7"
integrity sha512-56ZrkTDbdTLmBIyfFYesgOxsjcLnwAKoN4CiPyTVkMQj3zTUh0QAx3GbvIvLpFEOvQWu92yyWICxB0u7wkVbYA==
dependencies:
backo2 "1.0.2"
base64-arraybuffer "0.1.5"
component-bind "1.0.0"
component-emitter "1.2.1"
debug "~3.1.0"
engine.io-client "~3.3.1"
has-binary2 "~1.0.2"
has-cors "1.1.0"
indexof "0.0.1"
object-component "0.0.3"
parseqs "0.0.5"
parseuri "0.0.5"
socket.io-parser "~3.3.0"
to-array "0.1.4"
socket.io-client@2.3.0:
version "2.3.0"
resolved "https://registry.yarnpkg.com/socket.io-client/-/socket.io-client-2.3.0.tgz#14d5ba2e00b9bcd145ae443ab96b3f86cbcc1bb4"
@ -6404,18 +6357,6 @@ socket.io@~1.7.3:
socket.io-client "1.7.4"
socket.io-parser "2.3.1"
socket.io@~2.2.0:
version "2.2.0"
resolved "https://registry.yarnpkg.com/socket.io/-/socket.io-2.2.0.tgz#f0f633161ef6712c972b307598ecd08c9b1b4d5b"
integrity sha512-wxXrIuZ8AILcn+f1B4ez4hJTPG24iNgxBBDaJfT6MsyOhVYiTXWexGoPkd87ktJG8kQEcL/NBvRi64+9k4Kc0w==
dependencies:
debug "~4.1.0"
engine.io "~3.3.1"
has-binary2 "~1.0.2"
socket.io-adapter "~1.1.0"
socket.io-client "2.2.0"
socket.io-parser "~3.3.0"
socket.io@~2.3.0:
version "2.3.0"
resolved "https://registry.yarnpkg.com/socket.io/-/socket.io-2.3.0.tgz#cd762ed6a4faeca59bc1f3e243c0969311eb73fb"
@ -6510,11 +6451,12 @@ sprintf-js@~1.0.2:
resolved "https://registry.yarnpkg.com/sprintf-js/-/sprintf-js-1.0.3.tgz#04e6926f662895354f3dd015203633b857297e2c"
integrity sha1-BOaSb2YolTVPPdAVIDYzuFcpfiw=
"sproxydclient@github:scality/sproxydclient#a6ec980":
version "7.4.0"
resolved "https://codeload.github.com/scality/sproxydclient/tar.gz/a6ec98079fcbfde113de3f3afdcb57835d2ac55f"
"sproxydclient@github:scality/sproxydclient#30e7115":
version "8.0.2"
resolved "https://codeload.github.com/scality/sproxydclient/tar.gz/30e7115668bc7e10b4ec3cfdbaa7a124cdc21cc5"
dependencies:
werelogs scality/werelogs#0ff7ec82
async "^3.1.0"
werelogs scality/werelogs#351a2a3
sproxydclient@scality/sproxydclient#44f025b:
version "7.4.7"
@ -6580,6 +6522,11 @@ standard-as-callback@^2.0.1:
resolved "https://registry.yarnpkg.com/standard-as-callback/-/standard-as-callback-2.0.1.tgz#ed8bb25648e15831759b6023bdb87e6b60b38126"
integrity sha512-NQOxSeB8gOI5WjSaxjBgog2QFw55FV8TkS6Y07BiB3VJ8xNTvUYm0wl0s8ObgQ5NhdpnNfigMIKjgPESzgr4tg==
standard-as-callback@^2.1.0:
version "2.1.0"
resolved "https://registry.yarnpkg.com/standard-as-callback/-/standard-as-callback-2.1.0.tgz#8953fc05359868a77b5b9739a665c5977bb7df45"
integrity sha512-qoRRSyROncaz1z0mvYqIE4lCd9p2R90i6GxW3uZv5ucSu8tU7B5HXUP1gG8pVZsYNVaXjk8ClXHPttLyxAL48A==
"statuses@>= 1.5.0 < 2", statuses@~1.5.0:
version "1.5.0"
resolved "https://registry.yarnpkg.com/statuses/-/statuses-1.5.0.tgz#161c7dac177659fd9811f43771fa99381478628c"
@ -6926,13 +6873,6 @@ topo@2.x.x:
dependencies:
hoek "4.x.x"
topo@3.x.x:
version "3.0.3"
resolved "https://registry.yarnpkg.com/topo/-/topo-3.0.3.tgz#d5a67fb2e69307ebeeb08402ec2a2a6f5f7ad95c"
integrity sha512-IgpPtvD4kjrJ7CRA3ov2FhWQADwv+Tdqbsf1ZnPUSAtCJ9e1Z44MmoSGDXGk4IppoZA7jd/QRkNddlLJWlUZsQ==
dependencies:
hoek "6.x.x"
tough-cookie@~2.5.0:
version "2.5.0"
resolved "https://registry.yarnpkg.com/tough-cookie/-/tough-cookie-2.5.0.tgz#cd9fb2a0aa1d5a12b473bd9fb96fa3dcff65ade2"
@ -7154,15 +7094,16 @@ user-home@^2.0.0:
dependencies:
os-homedir "^1.0.0"
utapi@scality/utapi#1af6532:
utapi@scality/utapi#db5315b:
version "7.8.0"
resolved "https://codeload.github.com/scality/utapi/tar.gz/1af6532d83b7e323a62d9048cd6478cab12dfa71"
resolved "https://codeload.github.com/scality/utapi/tar.gz/db5315bf30a7faa9c1075562fe85d84c87d6cd03"
dependencies:
"@hapi/joi" "^17.1.1"
"@senx/warp10" "^1.0.14"
arsenal scality/Arsenal#65966f5
async "^3.2.0"
aws4 "^1.8.0"
backo "^1.1.0"
body-parser "^1.19.0"
bucketclient scality/bucketclient
byte-size "^7.0.0"
@ -7171,7 +7112,7 @@ utapi@scality/utapi#1af6532:
diskusage "^1.1.3"
express "^4.17.1"
get-folder-size "^2.0.1"
ioredis "^4.9.5"
ioredis "^4.28.0"
js-yaml "^3.14.0"
level-mem "^5.0.1"
needle "^2.5.0"