discuss: everything is a resource?

[butonic]

Straigt from gitter: https://gitter.im/cs3org/REVA?at=5d722b48ae44a841248d3e73

Hm thinking about the list of incoming, outgoing, trashed and favorite files (and folders) they all look like special storages to me ... the difference is that they only have a single collection with references to other resources in it ...
they could be known to the storage registry ... which presents the list of storages a user has access to ... one of them being the current user home (or his private files) under /home, incoming shares as /shares/incoming, outgoing shares as /shares/outgoing, accepted shares as /shares/accepted ..., trash as /trash, favorites as /favorites, tagged files as /tagged///.../

Jörn Friedrich Dreyer @butonic 11:53
but they actually all just render a collection of references
this is what we did when moving the different custom api calls under the dav endpoint in owncloud
same for versions ... there we actually introduced a meta namespace that we could use for other things as well. it uses /dav/meta//v to list versions end uses /dav/meta//v/ to reference specific versions

Jörn Friedrich Dreyer @butonic 11:58
hm
probably worth discussing at the conference

Hugo Labrador @labkode 13:47
I see a concern in your model: you are tyring to model every piece of the CS3 apis to webdav resources

Jörn Friedrich Dreyer @butonic 14:17
no to storage providers ... the question is if we want to expose metadata as a filesystem or if we want to ad dedicated apis for them

Hugo Labrador @labkode 15:27
the cs3 apis expose APIS for trashbin and versioning
you are mapping those to webdav using custom paths: files-trashbin, files-versions
am I right?

Jörn Friedrich Dreyer @butonic 15:27
yes ...
it pulls the metadata into the virtual filesystem tree

Hugo Labrador @labkode 15:27
I don't think for long term that will best approach for you to follow
you are exposing these functionality as webdav resources, that people can "sync"

Jörn Friedrich Dreyer @butonic 15:28
no

Hugo Labrador @labkode 15:28
i.e they are normal webdav endpoints
if I do a propfind from Finder there, I will get the files locally
same if you want to expose favourites or other infomation
In our case we won't use the phonix webdav endpoint bur rather the grpc directly
and if you want to stick to http, the easiest woudl be to consume a grpc-http JSON gateway

Jörn Friedrich Dreyer @butonic 15:31
people only sync the /dav/files/ tree

Hugo Labrador @labkode 15:32
But these same people have access to the trashbin endpoint, if I got it correctly

Jörn Friedrich Dreyer @butonic 15:32
right so in the future we need to extend the cs3 api should we eg want to implement a service that lists metadata for a user

Hugo Labrador @labkode 15:33
yes, and I think you have already an issue open that listcontainer could return metadata associated with that resource
I think that should be enough
for listing favourites, comments, any metadta attached to a resource

Jörn Friedrich Dreyer @butonic 15:33
why not represent metadata as resources and use storage providers that implement the necessary functionality
it may not work for everything, but it certainly is flexible enough to handle anything I can currently think of.
even search could be implemented that way
versions and trashed files are resources ... we should treat them that way.
maybe something to discuss at the conference
it is definitely somthing we think we did right when revisiting the apis in owcloud

Hugo Labrador @labkode 15:36
In your case that will mean that resource operations needs to be namespaced and not enforce by the semantic of the API
by that I mean, that currenlty you can call ListVersions and you know that you receive the versions of a file
in you case, you will call ListContainer(/versions) and you will return resource types with the type set to "version", that enforces that /versions must be enfornced across the implemtors of the APIs for getting versions

Jörn Friedrich Dreyer @butonic 15:37
yes ... in reva we also have an implicit virtual file tree where we mount storages for a user

Jörn Friedrich Dreyer @butonic 15:52
so for the new /dav endpoint in owncloud we hav several namespaces:

/dav/files// - which is the users home
/dav/meta/ - which allows accessing metadate, currently only versions using /dav/meta//v
/dav/trash-bin// to list the trashbin. in the cs3 api we could return references to paths
/dav/avatars//. to get an avatar for the user - I think here a /dav/users//avatar/. would have been the better choice. It would allow listing users and their properties
I am not saying that we did everything correct, but instead of defining a protobuf spec for eg listing all the above we could repurpose the storage provider ... as a resource provider ...
Then we only need to agree on the type of resource that is listed but avatar pictures are just images. trashbin and versions also list files. users and groups would be interesting because if you want to list the properties of a user ... you could model each property as a (text) file/folder / reference

same goes for user roles ...

Hugo Labrador @labkode 15:55
I have two questions:

Jörn Friedrich Dreyer @butonic 15:55
could be handled with a resource provider

Hugo Labrador @labkode 15:55

1. Why /dav/meta/fileid does not follow the convetion

Jörn Friedrich Dreyer @butonic 15:56
because it needs to uniquely identify a resource ... and the fileid is stable

Hugo Labrador @labkode 15:56
2) Those urls work for user-based paths, but imagine a project space that is a not attached to a user, how would you reprensent it?
/dav/project/<project_name>/trash-bin?
and so on?
so for 1) that means that a storage provider for a user will store the favourite information for another user?

Jörn Friedrich Dreyer @butonic 15:57
/dav/projects/ would probably be a more stable way ... but in general i think username and a unique id are interchangeable
oh cool idea
/dav/users/jfd/favorites would be a collection of references
and yes it would be implemented by a separate resource provider, one for listing users and their metadata
you can change the implementation as an admin by mounting a different resource provider to different locations in the virtual tree
that I am using the /dav prefix here is just legacy
but I think we should be able to com up with a good namespace especially to get the group storages or project spaces mounted properly
IIRC some solutions call it places

Jörn Friedrich Dreyer @butonic 16:03
so we could use that as a namespace for storages, eg /places/home/, /place/
same for shares? /shares//incoming outgoing ... well if they are different from places ...
anyway ... this feels right but it just erupted out of me ... so I may need a good round of discussion about this ... at the conference ;-)

[butonic]

I think a certain amount of confusion comes from the different namespaces in every api:
ocdav uses /dav/files/<username or id?> to point to the users home folder
cs3 uses a Reference, which can be a path or a ResourceId, which by design consists of an opaqe storageid and an opaqueid. the storageid is used to rout requests to the correct storage provider without having to do an additional lookup as is the case with the path based reference. reva has a storage registry that allows mounting different storage providers in the namespace.
The question is what do you see when doing a ListCollection with / as the path?
at cern you see /eos, /afs and /projects or something in that regard
The result of ListCollection(ReferencePath('/')) is the root of the CS3 virtual namespace.
we dont have that in webdav
we directly start at the users home folder, which contains a virtual tree that uses mount points to present a single big tree over multiple storages
the ocdav service translates between these two namespaces

maybe we can clarify by adding a few definitions to the cs3 api ... give names to things?

[butonic]

The cs3 api uses a namespace for its path based references. If we can agree that the first path segment is reserved to namespaces like /eos, /afs or other storage implementations (?) I can see two other namespaces:

1. /home for a posix native storage that respects the freedesktop.org conventions for eg trashbin and thumbnails. PR welcome
2. /som - Storage Object Model: used to expose metadata about files, users and groups.

• contains sub namespaces for different resources:
  • /som/storages/<base62(storageid)>/node/<base62(nodeid)> to access metadata of a resource, maybe call it resource? but I'd like to keep the graph terminology ;-)
    • can be used to list the existing chunks, eg. /som/storages/<base62(storageid)>/node/<base62(nodeid)>/chunks?
    • can be used to list the queue progress, /som/storages/<base62(storageid)>/queue/<base62(queueid)>?
  • /som/users/<base62(userid)> to access user metadata
    • can be used to list user notifications, eg. /som/users/<base62(userid)>/events?
    • can be used to acces the user avatar, eg. /som/users/<base62(userid)>/avatar?
      • size as optional request parameters to the GET request

[butonic]

[butonic]

• it could be used by applications / extensions to get a private namespacce for app specific data, eg. avatars and previews, cc @TheOneRing

[labkode]

@butonic I've discussed with the team, we'll write down here the proposal :)

[butonic]

The Microsoft Graph is the direction I would like to go. We need all kinds of entities, eg users and groups for sharing. Using this kind of generic (as in ) api would allow apps to provide new namespaces and introduce concepts without us having to change the available api operations.

I see two different ways of introducing a more graph based api:

1. implement a storage provider for the graph and mount it in the cs3 namespace as eg. /graph or /.graph oif you want it to be 'hidden'
2. make the cs3 api serve the graph by default and mounting storage providers somewhere inside the tree. maybe even reusing the namespaces from the Microsoft Graph, eg /me/drivefor the users home, /me/drives for a list of the storages a user has access to. See https://docs.microsoft.com/en-us/graph/api/resources/onedrive?view=graph-rest-1.0#commonly-accessed-resources

I kind of want to keep reva separate of the graph, because they are two different things. Actually, the cs3 api is a set of operations that cloud providers should implement, in order to have an interchangeable platform.

[butonic]

The Kopano guys have a KAPI, which is a subset of Microsoft Graph. AFAICT the 'drive' part is missing ...

So, AFAICT it makes more sense to put the KAPI infront of reva / the CS3 api ... or we could expose that graph in the cs3 api ...

too late ... need to hit the 🛏️

[labkode]

The Microsoft Graph API doesn't provide a priori anything that is not provided by the CS3APIS.

Microsoft Graph is REST-based and as such you can identify resources by an URL, in CS3APIS, you can identify resources by the payload message.

Microsoft Graph can be just another protocol like WebDAV is built on top of the CS3APIS.

[DeepDiver1975]

But the Graph API provides more then the CS3API (at least at the moment) - thinking about users and groups and may more.

Furthermore from an implementation point of view these elements don't need to me implemented as we could use synergies with Kopano kapi/grapi and do some joint development.

[refs]

archiving this discussion.