Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[full-ci] Assimilate clean command into sessions #9041

Merged
merged 4 commits into from
May 3, 2024
Merged

[full-ci] Assimilate clean command into sessions #9041

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4941620
feat(storage-users): assimilate clean command into sessions
kobergj Apr 30, 2024
3097a69
feat(storage-users): add virusfilter to sessions command
kobergj May 2, 2024
4b6d1f6
feat(reva): bump reva
kobergj May 2, 2024
62b1286
feat(storage-users): document sessions command
kobergj May 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions changelog/unreleased/add-virus-filter.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Enhancement: Add virus filter to sessions command

Allow filtering upload session by virus status (has-virus=true/false)

https://github.com/owncloud/ocis/pull/9041
5 changes: 5 additions & 0 deletions changelog/unreleased/assimilate-clean-into-sessions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Enhancement: Assimilate `clean` into `sessions` command

We deprecated `ocis storage-user uploads clean` and added the same logic to `ocis storage-users uploads session --clean`

https://github.com/owncloud/ocis/pull/9041
1 change: 1 addition & 0 deletions changelog/unreleased/bump-reva.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ Enhancement: Bump Reva

bumps reva version

https://github.com/owncloud/ocis/pull/9041
https://github.com/owncloud/ocis/pull/9002
https://github.com/owncloud/ocis/pull/8917
https://github.com/owncloud/ocis/pull/8795
Expand Down
3 changes: 2 additions & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ require (
github.com/cenkalti/backoff v2.2.1+incompatible
github.com/coreos/go-oidc/v3 v3.10.0
github.com/cs3org/go-cs3apis v0.0.0-20231023073225-7748710e0781
github.com/cs3org/reva/v2 v2.19.2-0.20240429085656-5faad8dad61e
github.com/cs3org/reva/v2 v2.19.2-0.20240502102837-7e48a5145133
github.com/dhowden/tag v0.0.0-20230630033851-978a0926ee25
github.com/dutchcoders/go-clamd v0.0.0-20170520113014-b970184f4d9e
github.com/egirna/icap-client v0.1.1
Expand Down Expand Up @@ -85,6 +85,7 @@ require (
github.com/spf13/afero v1.11.0
github.com/spf13/cobra v1.8.0
github.com/stretchr/testify v1.9.0
github.com/test-go/testify v1.1.4
github.com/thejerf/suture/v4 v4.0.5
github.com/tidwall/gjson v1.17.1
github.com/tus/tusd v1.13.0
Expand Down
4 changes: 2 additions & 2 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1027,8 +1027,8 @@ github.com/crewjam/saml v0.4.14 h1:g9FBNx62osKusnFzs3QTN5L9CVA/Egfgm+stJShzw/c=
github.com/crewjam/saml v0.4.14/go.mod h1:UVSZCf18jJkk6GpWNVqcyQJMD5HsRugBPf4I1nl2mME=
github.com/cs3org/go-cs3apis v0.0.0-20231023073225-7748710e0781 h1:BUdwkIlf8IS2FasrrPg8gGPHQPOrQ18MS1Oew2tmGtY=
github.com/cs3org/go-cs3apis v0.0.0-20231023073225-7748710e0781/go.mod h1:UXha4TguuB52H14EMoSsCqDj7k8a/t7g4gVP+bgY5LY=
github.com/cs3org/reva/v2 v2.19.2-0.20240429085656-5faad8dad61e h1:4Z9nfRWM2VnTX0EOBaIv0Rn27Fw7VYqOrcCntXZ3b40=
github.com/cs3org/reva/v2 v2.19.2-0.20240429085656-5faad8dad61e/go.mod h1:GRUrOp5HbFVwZTgR9bVrMZ/MvVy+Jhxw1PdMmhhKP9E=
github.com/cs3org/reva/v2 v2.19.2-0.20240502102837-7e48a5145133 h1:zuqfsPUYjJjIiq4wx2xTDxlJH+Sh3H3m8ynSjbQ/e8w=
github.com/cs3org/reva/v2 v2.19.2-0.20240502102837-7e48a5145133/go.mod h1:GRUrOp5HbFVwZTgR9bVrMZ/MvVy+Jhxw1PdMmhhKP9E=
github.com/cyberdelia/templates v0.0.0-20141128023046-ca7fffd4298c/go.mod h1:GyV+0YP4qX0UQ7r2MoYZ+AvYDp12OF5yg4q8rGnyNh4=
github.com/cyphar/filepath-securejoin v0.2.4 h1:Ugdm7cg7i6ZK6x3xDF1oEu1nfkyfH53EtKeQYTC3kyg=
github.com/cyphar/filepath-securejoin v0.2.4/go.mod h1:aPGpWjXOXUn2NCNjFvBE6aRxGGx79pTxQpKOJNYHHl4=
Expand Down
76 changes: 54 additions & 22 deletions services/storage-users/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,58 +60,90 @@ ocis storage-users uploads <command>
```plaintext
COMMANDS:
sessions Print a list of upload sessions
clean Clean up leftovers from expired uploads
clean Clean up leftovers from expired uploads (deprecated)
list Print a list of all incomplete uploads (deprecated)
```

#### Sessions command

The `sessions` command is the entry point for listing, restarting and cleaning unfinished uploads.

```bash
ocis storage-users uploads sessions <command>
```

```
NAME:
ocis storage-users uploads sessions - Print a list of upload sessions
USAGE:
ocis storage-users uploads sessions [command options]
OPTIONS:
--id value filter sessions by upload session id (default: unset)
--processing filter sessions by processing status (default: unset)
--expired filter sessions by expired status (default: unset)
--has-virus filter sessions by virus scan result (default: unset)
--json output as json (default: false)
--restart send restart event for all listed sessions (default: false)
--clean remove uploads (default: false)
--help, -h show help
```

This will always output a list of uploads that match the criteria. See Command Examples section.

Some additional information on returned information:
- `Offset` is the amount of bytes the server has already received. If `Offset` == `Size` the server has reveived all bytes of the upload.
- `Processing` indicates if the uploaded file is currently going through postprocessing.
- `Scan Date` and `Scan Result` indicate the scanning status. If `Scan Date` is set and `Scan Result` is empty the file is not virus infected.

#### Command Examples

Command to list ongoing upload sessions

```bash
ocis storage-users sessions --expired=false
ocis storage-users uploads sessions --expired=false --processing=false
```

```plaintext
Not expired sessions:
+--------------------------------------+--------------------------------------+---------+--------+------+--------------------------------------+--------------------------------------+---------------------------+------------+
| Space | Upload Id | Name | Offset | Size | Executant | Owner | Expires | Processing |
+--------------------------------------+--------------------------------------+---------+--------+------+--------------------------------------+--------------------------------------+---------------------------+------------+
| f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c | 5e387954-7313-4223-a904-bf996da6ec0b | foo.txt | 0 | 1234 | f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c | f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c | 2024-01-26T13:04:31+01:00 | false |
| f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c | f066244d-97b2-48e7-a30d-b40fcb60cec6 | bar.txt | 0 | 4321 | f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c | f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c | 2024-01-26T13:18:47+01:00 | false |
+--------------------------------------+--------------------------------------+---------+--------+------+--------------------------------------+--------------------------------------+---------------------------+------------+
+--------------------------------------+--------------------------------------+---------+--------+------+--------------------------------------+--------------------------------------+---------------------------+------------+---------------------------+-----------------------+
| Space | Upload Id | Name | Offset | Size | Executant | Owner | Expires | Processing | Scan Date | Scan Result |
+--------------------------------------+--------------------------------------+---------+--------+------+--------------------------------------+--------------------------------------+---------------------------+------------+---------------------------+-----------------------+
| f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c | 5e387954-7313-4223-a904-bf996da6ec0b | foo.txt | 0 | 1234 | f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c | f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c | 2024-01-26T13:04:31+01:00 | false | 2024-04-24T11:24:14+02:00 | infected: virus A |
| f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c | f066244d-97b2-48e7-a30d-b40fcb60cec6 | bar.txt | 0 | 4321 | f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c | f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c | 2024-01-26T13:18:47+01:00 | false | 2024-04-24T14:38:29+02:00 | |
+--------------------------------------+--------------------------------------+---------+--------+------+--------------------------------------+--------------------------------------+---------------------------+------------+---------------------------+-----------------------+
```

The sessions command can also output json

```bash
ocis storage-users sessions --expired=false --json
ocis storage-users uploads sessions --expired=false --processing=false --json
```

```json
{"id":"5e387954-7313-4223-a904-bf996da6ec0b","space":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c","filename":"foo.txt","offset":0,"size":1234,"executant":{"idp":"https://cloud.ocis.test","opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"spaceowner":{"opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"expires":"2024-01-26T13:04:31+01:00","processing":false}
{"id":"f066244d-97b2-48e7-a30d-b40fcb60cec6","space":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c","filename":"bar.txt","offset":0,"size":4321,"executant":{"idp":"https://cloud.ocis.test","opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"spaceowner":{"opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"expires":"2024-01-26T13:18:47+01:00","processing":false}
{"id":"5e387954-7313-4223-a904-bf996da6ec0b","space":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c","filename":"foo.txt","offset":0,"size":1234,"executant":{"idp":"https://cloud.ocis.test","opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"spaceowner":{"opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"expires":"2024-01-26T13:04:31+01:00","processing":false, "scanDate": "2024-04-24T11:24:14+02:00", "scanResult": "infected: virus A"}
{"id":"f066244d-97b2-48e7-a30d-b40fcb60cec6","space":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c","filename":"bar.txt","offset":0,"size":4321,"executant":{"idp":"https://cloud.ocis.test","opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"spaceowner":{"opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"expires":"2024-01-26T13:18:47+01:00","processing":false, "scanDate": "2024-04-24T14:38:29+02:00", "scanResult": ""}
```

Command to clear expired uploads
The sessions command can also clear and restart uploads. The output is the same as if run without `--clean` or `--restart` flag.
Note: It is recommended to run to command first without the `--clean` (`--processing`) flag to double check which uploads get cleaned (restarted).
```bash
ocis storage-users uploads clean
```
# cleans all expired uploads regardless of processing and virus state.
ocis storage-users uploads sessions --expired=true --clean

```plaintext
Cleaned uploads:
- 455bd640-cd08-46e8-a5a0-9304908bd40a (Filename: file_example_PPT_1MB.ppt, Size: 1028608, Expires: 2022-08-17T12:35:34+02:00)
# restarts all uploads that are processing and are not virus infected
ocis storage-users uploads sessions --processing=false --has-virus=false --restart
```

Deprecated list command to identify unfinished uploads
IMPOTANT: `list` and `clean` commands are deprecated. Do not use them.

```bash
# deprecated
ocis storage-users uploads list
```

```plaintext
Incomplete uploads:
- 455bd640-cd08-46e8-a5a0-9304908bd40a (file_example_PPT_1MB.ppt, Size: 1028608, Expires: 2022-08-17T12:35:34+02:00)
# deprecated
ocis storage-users uploads clean
```

### Manage Trash-Bin Items
Expand Down
Loading