Config pitfalls after v4 upgrade

[xoxys]

Describe the bug

• OCIS_LDAP_BIND_PASSWORD not working and must use LDAP_BIND_PASSWORD while all other LDAP env vars were prefixed with OCIS_
• OCIS_LDAP_USER_ENABLED_ATTRIBUTE and OCIS_LDAP_DISABLE_USER_MECHANISM not documented for v4 and results in an ugly breaking change while using an external LDAP
• APP_PROVIDER_SERVICE_NAME and APP_PROVIDER_EXTERNAL_ADDR breaks office integrations and are not part of the upgrade guide
• APP_PROVIDER_SERVICE_NAME=ocis-provider-collabora and APP_PROVIDER_EXTERNAL_ADDR=com.owncloud.api.ocis-provider-collabora need to match which is not documented and should be handled automatically
• MICRO_REGISTRY is not mentioned in the upgrade guide
• MICRO_REGISTRY env var is not namespaced or prefixed with OCIS_
• documented deprecated env vars for v4 are not accurate (see above)/helpful

[micbar]

@ScharfViktor Could we just run an new upgrade from 3.0.0 to 4.0.0 with the WOPI example and docuement the step-by-step process.

@dragonchaser We should prefix that LDAP_BIND_PASSWORD with ocis and include it in the 4.0.1 patch release.

[micbar]

Outcome should be an updated / enhaced upgrade guide https://doc.owncloud.com/ocis/next/migration/upgrading-ocis.html

Please ping @xoxys for review.

[micbar]

@mmattel FYI

[ScharfViktor]

Upgrade test process

for upgrade test I use https://github.com/owncloud/ocis/blob/stable-3.0/deployments/examples/ocis_wopi/docker-compose.yml for 3.0.0 version and https://github.com/owncloud/ocis/blob/stable-4.0/deployments/examples/ocis_wopi/docker-compose.yml for 4.0.0 version and of course read docs: https://doc.owncloud.com/ocis/next/migration/upgrading_3.0.0_4.0.0.html

• git checkout stable-3.0

• cd ocis/deployments/examples/ocis_wopi

• OCIS_DOCKER_TAG=3.0.0 docker-compose up -d

• create data according to checklist https://github.com/owncloud/QA/blob/master/Server/Test_Plan_OCIS_Compatibility_Test.md

• git checkout stable-4.0

• upgrade ocis version OCIS_DOCKER_TAG=4.0.0 docker-compose up -d

• ensure that migrations are succefuly (see ocis log)

• check that all data, shares, public links and created users etc exist

v3.0 and v4.0 comparison

• MICRO_REGISTRY: "mdns"(v4) - specified in the docs
• FRONTEND_FULL_TEXT_SEARCH_ENABLED: true(v4) - doesn't specified in the docs
• REVA_GATEWAY changed ocis:9142 -> com.owncloud.api.gateway - specified in the docs
• APP_PROVIDER_EXTERNAL_ADDR changed: ocis-appprovider-collabora:9164-> com.owncloud.api.app-provider-collabora - doesn't specified in the docs
• APP_PROVIDER_SERVICE_NAME: added app-provider-collabora/onlyoffice - doesn't specified in the docs
• upgrated
  • [] wopiserver image: 9.4.0 -> 10.1.0
  • [] onlyoffice image: onlyoffice/documentserver:7.3.0 -> onlyoffice/documentserver:7.3.3
  • [] collabora image: collabora/code:22.05.10.1.1 -> collabora/code:22.05.14.3.1

remarks:

file couldn't open in collabora using cs3org/wopiserver:v10.0.1 version. v9.4.0 works correct

log:

ERR  WOPI::GetFile [https://wopiserver.owncloud.test/wopi/files/3bb824e3-c20c-4fed-87d9-feca2c22888f%21ZTcwNmI3YzAtZDI0OS00ZTRiLWFhMTctNGQ2ZjVlMDNjYWJk/contents?access_token=&access_token_ttl=1693655965000&permission=edit] failed with Status Code: 500| wsd/Storage.cpp:1172

wsd-00001-00128 2023-09-01 11:59:30.630312 +0000 [ docbroker_00f ] ERR  Cannot download document from WOPI storage uri [https://wopiserver.owncloud.test/wopi/files/3bb824e3-c20c-4fed-87d9-feca2c22888f%21ZTcwNmI3YzAtZDI0OS00ZTRiLWFhMTctNGQ2ZjVlMDNjYWJk/contents?access_token=&access_token_ttl=1693655965000&permission=edit]. Error: WOPI::GetFile [https://wopiserver.owncloud.test/wopi/files/3bb824e3-c20c-4fed-87d9-feca2c22888f%21ZTcwNmI3YzAtZDI0OS00ZTRiLWFhMTctNGQ2ZjVlMDNjYWJk/contents?access_token=&access_token_ttl=1693655965000&permission=edit] failed: {"message": "Failed to fetch file from storage"}| wsd/Storage.cpp:1103

wsd-00001-00128 2023-09-01 11:59:30.630381 +0000 [ docbroker_00f ] ERR  loading document exception: WOPI::GetFile [https://wopiserver.owncloud.test/wopi/files/3bb824e3-c20c-4fed-87d9-feca2c22888f%21ZTcwNmI3YzAtZDI0OS00ZTRiLWFhMTctNGQ2ZjVlMDNjYWJk/contents?access_token=&access_token_ttl=1693655965000&permission=edit] failed: {"message": "Failed to fetch file from storage"}| wsd/DocumentBroker.cpp:2510

wsd-00001-00128 2023-09-01 11:59:30.630403 +0000 [ docbroker_00f ] ERR  Failed to add session to [https://wopiserver.owncloud.test:443/wopi/files/3bb824e3-c20c-4fed-87d9-feca2c22888f%21ZTcwNmI3YzAtZDI0OS00ZTRiLWFhMTctNGQ2ZjVlMDNjYWJk] with URI [https://wopiserver.owncloud.test/wopi/files/3bb824e3-c20c-4fed-87d9-feca2c22888f%21ZTcwNmI3YzAtZDI0OS00ZTRiLWFhMTctNGQ2ZjVlMDNjYWJk?access_token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyaWQiOiJleUpoYkdjaU9pSklVekkxTmlJc0luUjVjQ0k2SWtwWFZDSjkuZXlKaGRXUWlPaUp5WlhaaElpd2laWGh3SWpveE5qa3pOalUxT1RZMUxDSnBZWFFpT2pFMk9UTTFOamsxTmpVc0ltbHpjeUk2SW1oMGRIQnpPaTh2YjJOcGN5NXZkMjVqYkc5MVpDNTBaWE4wSWl3aWRYTmxjaUk2ZXlKcFpDSTZleUpwWkhBaU9pSm9kSFJ3Y3pvdkwyOWphWE11YjNkdVkyeHZkV1F1ZEdWemRDSXNJbTl3WVhGMVpWOXBaQ0k2SWpZMU1tUXdaVFl6TFRjeVptRXRORFl4TVMxaE5EazNMVGxsWkRWbVlqWmhNek01WlNJc0luUjVjR1VpT2pGOUxDSjFjMlZ5Ym1GdFpTSTZJbUZrYldsdUlpd2liV0ZwYkNJNkltRmtiV2x1UUdWNFlXMXdiR1V1YjNKbklpd2laR2x6Y0d4aGVWOXVZVzFsSWpvaVFXUnRhVzRpTENKMWFXUmZiblZ0WW1WeUlqbzVPU3dpWjJsa1gyNTFiV0psY2lJNk9UbDlMQ0p6WTI5d1pTSTZleUoxYzJWeUlqcDdJbkpsYzI5MWNtTmxJanA3SW1SbFkyOWtaWElpT2lKcWMyOXVJaXdpZG1Gc2RXVWlPaUpsZVVwM1dWaFNiMGxxYjJsTWVVbzVJbjBzSW5KdmJHVWlPakY5ZlgwLlBPdnpKbklrV01CNWl0bFk1T05qRFpBYzFRTUdKdTRxbjhLNnktZ0ROTVUiLCJ3b3BpdXNlciI6IjY1MmQwZTYzLTcyZmEtNDYxMS1hNDk3LTllZDVmYjZhMzM5ZSE2NTJkMGU2My03MmZhLTQ2MTEtYTQ5Ny05ZWQ1ZmI2YTMzOWVAaHR0cHM6Ly9vY2lzLm93bmNsb3VkLnRlc3QiLCJ1c2VydHlwZSI6InJlZ3VsYXIiLCJmaWxlbmFtZSI6IjY1MmQwZTYzLTcyZmEtNDYxMS1hNDk3LTllZDVmYjZhMzM5ZS9OZXVlIERhdGVpLm9kcyIsImZpbGVpZCI6ImU3MDZiN2MwLWQyNDktNGU0Yi1hYTE3LTRkNmY1ZTAzY2FiZCIsInVzZXJuYW1lIjoiQWRtaW4iLCJ2aWV3bW9kZSI6IlZJRVdfTU9ERV9SRUFEX1dSSVRFIiwiZm9sZGVydXJsIjoiaHR0cHM6Ly9vY2lzLm93bmNsb3VkLnRlc3QvZi8zYmI4MjRlMy1jMjBjLTRmZWQtODdkOS1mZWNhMmMyMjg4OGYkNjUyZDBlNjMtNzJmYS00NjExLWE0OTctOWVkNWZiNmEzMzllIWU3MDZiN2MwLWQyNDktNGU0Yi1hYTE3LTRkNmY1ZTAzY2FiZCIsImVuZHBvaW50IjoiM2JiODI0ZTMtYzIwYy00ZmVkLTg3ZDktZmVjYTJjMjI4ODhmJDY1MmQwZTYzLTcyZmEtNDYxMS1hNDk3LTllZDVmYjZhMzM5ZSIsImFwcG5hbWUiOiJDb2xsYWJvcmEiLCJhcHBlZGl0dXJsIjoiaHR0cHM6Ly9jb2xsYWJvcmEub3duY2xvdWQudGVzdC9icm93c2VyL2MzNjAwMTE4MTUvY29vbC5odG1sPyIsImFwcHZpZXd1cmwiOiJodHRwczovL2NvbGxhYm9yYS5vd25jbG91ZC50ZXN0L2Jyb3dzZXIvYzM2MDAxMTgxNS9jb29sLmh0bWw_IiwiZXhwIjoxNjkzNjU1OTY1LCJpc3MiOiJjczNvcmc6d29waXNlcnZlcjoifQ.9kncuBXc981T0LGjEVn042r48Y1dhfe3aM1UMbjgcr8&access_token_ttl=1693655965000&permission=edit]: WOPI::GetFile [https://wopiserver.owncloud.test/wopi/files/3bb824e3-c20c-4fed-87d9-feca2c22888f%21ZTcwNmI3YzAtZDI0OS00ZTRiLWFhMTctNGQ2ZjVlMDNjYWJk/contents?access_token=&access_token_ttl=1693655965000&permission=edit] failed: {"message": "Failed to fetch file from storage"}| wsd/DocumentBroker.cpp:2472

wsd-00001-00128 2023-09-01 11:59:30.630486 +0000 [ docbroker_00f ] ERR  Storage error while starting session on https://wopiserver.owncloud.test:443/wopi/files/3bb824e3-c20c-4fed-87d9-feca2c22888f%21ZTcwNmI3YzAtZDI0OS00ZTRiLWFhMTctNGQ2ZjVlMDNjYWJk for socket #28. Terminating connection. Error: WOPI::GetFile [https://wopiserver.owncloud.test/wopi/files/3bb824e3-c20c-4fed-87d9-feca2c22888f%21ZTcwNmI3YzAtZDI0OS00ZTRiLWFhMTctNGQ2ZjVlMDNjYWJk/contents?access_token=&access_token_ttl=1693655965000&permission=edit] failed: {"message": "Failed to fetch file from storage"}| wsd/COOLWSD.cpp:4929

wsd-00001-00128 2023-09-01 11:59:30.631016 +0000 [ docbroker_00f ] ERR  #28: Socket write returned -1 (EPIPE: Broken pipe)| net/Socket.hpp:1442

wsd-00001-00128 2023-09-01 11:59:30.644172 +0000 [ docbroker_00f ] ERR  #29: Read failed, have 0 buffered bytes (ECONNRESET: Connection reset by peer)| net/Socket.hpp:1136

wsd-00001-00128 2023-09-01 11:59:30.644265 +0000 [ docbroker_00f ] WRN  #29: Unassociated Kit (127) disconnected unexpectedly| wsd/COOLWSD.cpp:3409

[xoxys]

Where is MICRO_REGISTRY specified in the docs? Its not in https://doc.owncloud.com/ocis/next/migration/upgrading_3.0.0_4.0.0.html

[ScharfViktor]

yes, on this page we haven't.
indicated and added here https://doc.owncloud.com/ocis/next/deployment/services/env-vars-special-scope.html

[xoxys]

Ok, but this page doesn't even mention what this env var is used for. And things like Only change on supervision of ownCloud Support. in the docs are just bad... No chance that a user knows this is required for office integrations now and according to the docs MICRO_REGISTRY doesn't have a default, so user have to explicitly set it.

[zjean]

any updates on this? I want to migrate from 3.0 to 4.0.1, but am hesitant to do so.

[ScharfViktor]

OCIS_LDAP_BIND_PASSWORD not working and must use LDAP_BIND_PASSWORD while all other LDAP env vars were prefixed with OCIS_

fixed here #7313

It will be available again in ocis 4.0.2