autoprovisioning: Manage group memberships

changelog/unreleased/autoprovsion-groups.md

@@ -0,0 +1,7 @@
+Enhancement: Autoprovision group memberships
+
+When PROXY_AUTOPROVISION_ACCOUNTS is enabled it is now possible to automatically
+maintain the group memberships of users via a configurable OIDC claim.
+
+https://github.com/owncloud/ocis/pull/9458
+https://github.com/owncloud/ocis/issues/5538

services/proxy/README.md

@@ -50,6 +50,79 @@ unprotected: false # with false (default), calling the endpoint requires authori
                    # with true, anyone can call the endpoint without authorisation.
 ```
 
+## Automatic User and Group Provisioning
+
+When using an external OpenID Connect IDP, the proxy can be configured to automatically provision
+users upon their first login.
+
+### Prequisites
+
+A number of prerequisites must be met for automatic user provisioning to work:
+
+* ownCloud Infinite Scale must be configured to use an external OpenID Connect IDP
+* The `graph` service must be configured to allow updating users and groups
+  (`GRAPH_LDAP_SERVER_WRITE_ENABLED`).
+* The IDP must return a unique value in the user's claims (as part of the
+  userinfo response and/or the access tokens) that can be used to identify
+  the user. This claim needs to be stable and cannot be changed for the whole
+  lifetime of the user. That means, if a claim like `email` or
+  `preferred_username` is used, you must ensure that the user's email address or
+  username never changes.
+
+### Configuration
+
+To enable automatic user provisioning, the following environment variables must
+be set for the proxy service:
+
+* `PROXY_AUTOPROVISION_ACCOUNTS`\
+Set to `true` to enable automatic user provisioning.
+* `PROXY_AUTOPROVISION_CLAIM_USERNAME`\
+The name of an OIDC claim whose value should be used as the username for the
+autoprovsioned user in ownCloud Infinite Scale. Defaults to `preferred_username`.
+Can also be set to e.g. `sub` to guarantee a unique and stable username.
+* `PROXY_AUTOPROVISION_CLAIM_EMAIL`\
+The name of an OIDC claim whose value should be used for the `mail` attribute
+of the autoprovisioned user in ownCloud Infinite Scale. Defaults to `email`.
+* `PROXY_AUTOPROVISION_CLAIM_DISPLAYNAME`\
+The name of an OIDC claim whose value should be used for the `displayname`
+attribute of the autoprovisioned user in ownCloud Infinite Scale. Defaults to `name`.
+* `PROXY_AUTOPROVISION_CLAIM_GROUPS`\
+The name of an OIDC claim whose value should be used to maintain a user's group
+membership. The claim value should contain a list of group names the user should
+be a member of. Defaults to `groups`.
+* `PROXY_USER_OIDC_CLAIM`\
+When resolving and authenticated OIDC user, the value of this claims is used to
+lookup the user in the users service. For auto provisioning setups this usually is the
+same claims as set via `PROXY_AUTOPROVISION_CLAIM_USERNAME`.
+* `PROXY_USER_CS3_CLAIM`\
+This is the name of the user attribute in ocis that is used to lookup the user by the
+value of the `PROXY_USER_OIDC_CLAIM`. For auto provisioning setups this usually
+needs to be set to `username`.
+
+### How it Works
+
+When a user logs into ownCloud Infinite Scale for the first time, the proxy
+checks if that user already exists. This is done by querying the `users` service for users,
+where the attribute set in `PROXY_USER_CS3_CLAIM` matches the value of the OIDC
+claim configured in `PROXY_USER_OIDC_CLAIM`.
+
+If the users does not exist, the proxy will create a new user via the `graph`
+service using the claim values configured in
+`PROXY_AUTOPROVISION_CLAIM_USERNAME`, `PROXY_AUTOPROVISION_CLAIM_EMAIL` and
+`PROXY_AUTOPROVISION_CLAIM_DISPLAYNAME`.
+
+If the user does already exist, the proxy will check if the user's email or
+displayname has changed and updates those accordingly via `graph` service.
+
+Next, the proxy will check if the user is a member of the groups configured in
+`PROXY_AUTOPROVISION_CLAIM_GROUPS`. It will add the user to the groups listed
+via the OIDC claim that holds the groups defined in the envvar and removes it from
+all other groups that he is currently a member of.
+Groups that do not exist in the external IDP yet will be created. Note: This can be a
+somewhat costly operation, especially if the user is a member of a large number of
+groups. If the group memberships of a user are changed in the IDP after the
+first login, it can take up to 5 minutes until the changes are reflected in Infinite Scale.
+
 ## Automatic Quota Assignments
 
 It is possible to automatically assign a specific quota to new users depending on their role.
@@ -78,8 +151,7 @@ When `PROXY_ROLE_ASSIGNMENT_DRIVER` is set to `oidc` the role assignment for a u
 based on the values of an OpenID Connect Claim of that user. The name of the OpenID Connect Claim to
 be used for the role assignment can be configured via the `PROXY_ROLE_ASSIGNMENT_OIDC_CLAIM`
 environment variable. It is also possible to define a mapping of claim values to role names defined
-in ownCloud Infinite Scale via a `yaml` configuration. See the following `proxy.yaml` snippet for an
-example.
+in Infinite Scale via a `yaml` configuration. See the following `proxy.yaml` snippet for an example.
 
 ```yaml
 role_assignment:
@@ -153,7 +225,7 @@ Store specific notes:
   -   When using `nats-js-kv` it is recommended to set `OCIS_CACHE_STORE_NODES` to the same value as `OCIS_EVENTS_ENDPOINT`. That way the cache uses the same nats instance as the event bus.
   -   When using the `nats-js-kv` store, it is possible to set `OCIS_CACHE_DISABLE_PERSISTENCE` to instruct nats to not persist cache data on disc.
 
-  
+
 ## Presigned Urls
 
 To authenticate presigned URLs the proxy service needs to read signing keys from a store that is populated by the ocs service. Possible stores are:

services/proxy/pkg/config/config.go

@@ -160,6 +160,7 @@ type AutoProvisionClaims struct {
 	Username    string `yaml:"username" env:"PROXY_AUTOPROVISION_CLAIM_USERNAME" desc:"The name of the OIDC claim that holds the username." introductionVersion:"6.0.0"`
 	Email       string `yaml:"email" env:"PROXY_AUTOPROVISION_CLAIM_EMAIL" desc:"The name of the OIDC claim that holds the email." introductionVersion:"6.0.0"`
 	DisplayName string `yaml:"display_name" env:"PROXY_AUTOPROVISION_CLAIM_DISPLAYNAME" desc:"The name of the OIDC claim that holds the display name." introductionVersion:"6.0.0"`
+	Groups      string `yaml:"groups" env:"PROXY_AUTOPROVISION_CLAIM_GROUPS" desc:"The name of the OIDC claim that holds the groups." introductionVersion:"6.1.0"`
 }
 
 // PolicySelector is the toplevel-configuration for different selectors

services/proxy/pkg/config/defaults/defaultconfig.go

@@ -88,6 +88,7 @@ func DefaultConfig() *config.Config {
 			Username:    "preferred_username",
 			Email:       "email",
 			DisplayName: "name",
+			Groups:      "groups",
 		},
 		EnableBasicAuth:       false,
 		InsecureBackends:      false,

services/proxy/pkg/middleware/account_resolver.go

@@ -4,7 +4,9 @@ import (
 	"errors"
 	"fmt"
 	"net/http"
+	"time"
 
+	"github.com/jellydator/ttlcache/v3"
 	"github.com/owncloud/ocis/v2/services/proxy/pkg/user/backend"
 	"github.com/owncloud/ocis/v2/services/proxy/pkg/userroles"
 
@@ -19,6 +21,12 @@ func AccountResolver(optionSetters ...Option) func(next http.Handler) http.Handl
 	options := newOptions(optionSetters...)
 	logger := options.Logger
 
+	lastGroupSyncCache := ttlcache.New(
+		ttlcache.WithTTL[string, struct{}](5*time.Minute),
+		ttlcache.WithDisableTouchOnHit[string, struct{}](),
+	)
+	go lastGroupSyncCache.Start()
+
 	return func(next http.Handler) http.Handler {
 		return &accountResolver{
 			next:                  next,
@@ -28,6 +36,7 @@ func AccountResolver(optionSetters ...Option) func(next http.Handler) http.Handl
 			userCS3Claim:          options.UserCS3Claim,
 			userRoleAssigner:      options.UserRoleAssigner,
 			autoProvisionAccounts: options.AutoprovisionAccounts,
+			lastGroupSyncCache:    lastGroupSyncCache,
 		}
 	}
 }
@@ -40,6 +49,10 @@ type accountResolver struct {
 	autoProvisionAccounts bool
 	userOIDCClaim         string
 	userCS3Claim          string
+	// lastGroupSyncCache is used to keep track of when the last sync of group
+	// memberships was done for a specific user. This is used to trigger a sync
+	// with every single request.
+	lastGroupSyncCache *ttlcache.Cache[string, struct{}]
 }
 
 func readUserIDClaim(path string, claims map[string]interface{}) (string, error) {
@@ -140,6 +153,15 @@ func (m accountResolver) ServeHTTP(w http.ResponseWriter, req *http.Request) {
 				w.WriteHeader(http.StatusInternalServerError)
 				return
 			}
+			// Only	sync group memberships if the user has not been synced since the last cache invalidation
+			if !m.lastGroupSyncCache.Has(user.GetId().GetOpaqueId()) {
+				if err = m.userProvider.SyncGroupMemberships(req.Context(), user, claims); err != nil {
+					m.logger.Error().Err(err).Str("userid", user.GetId().GetOpaqueId()).Interface("claims", claims).Msg("Failed to sync group memberships for autoprovisioned user")
+					w.WriteHeader(http.StatusInternalServerError)
+					return
+				}
+				m.lastGroupSyncCache.Set(user.GetId().GetOpaqueId(), struct{}{}, ttlcache.DefaultTTL)
+			}
 		}
 
 		// resolve the user's roles

services/proxy/pkg/user/backend/backend.go

@@ -22,4 +22,5 @@ type UserBackend interface {
 	Authenticate(ctx context.Context, username string, password string) (*cs3.User, string, error)
 	CreateUserFromClaims(ctx context.Context, claims map[string]interface{}) (*cs3.User, error)
 	UpdateUserIfNeeded(ctx context.Context, user *cs3.User, claims map[string]interface{}) error
+	SyncGroupMemberships(ctx context.Context, user *cs3.User, claims map[string]interface{}) error
 }

services/proxy/pkg/user/backend/cs3.go

@@ -3,6 +3,7 @@ package backend
 import (
 	"context"
 	"encoding/json"
+	"errors"
 	"fmt"
 	"io"
 	"net/http"
@@ -12,6 +13,7 @@ import (
 	rpcv1beta1 "github.com/cs3org/go-cs3apis/cs3/rpc/v1beta1"
 	revactx "github.com/cs3org/reva/v2/pkg/ctx"
 	"github.com/cs3org/reva/v2/pkg/rgrpc/todo/pool"
+	utils "github.com/cs3org/reva/v2/pkg/utils"
 	libregraph "github.com/owncloud/libre-graph-api-go"
 	"go-micro.dev/v4/selector"
 
@@ -40,6 +42,10 @@ type Options struct {
 	autoProvisionClaims config.AutoProvisionClaims
 }
 
+var (
+	errGroupNotFound = errors.New("group not found")
+)
+
 // WithLogger sets the logger option
 func WithLogger(l log.Logger) Option {
 	return func(o *Options) {
@@ -243,26 +249,157 @@ func (c cs3backend) UpdateUserIfNeeded(ctx context.Context, user *cs3.User, clai
 	return nil
 }
 
-func (c cs3backend) updateLibregraphUser(userid string, user libregraph.User) error {
+// SyncGroupMemberships maintains a users group memberships based on an OIDC claim
+func (c cs3backend) SyncGroupMemberships(ctx context.Context, user *cs3.User, claims map[string]interface{}) error {
 	gatewayClient, err := c.gatewaySelector.Next()
 	if err != nil {
 		c.logger.Error().Err(err).Msg("could not select next gateway client")
 		return err
 	}
 	newctx := context.Background()
-	authRes, err := gatewayClient.Authenticate(newctx, &gateway.AuthenticateRequest{
-		Type:         "serviceaccounts",
-		ClientId:     c.serviceAccount.ServiceAccountID,
-		ClientSecret: c.serviceAccount.ServiceAccountSecret,
-	})
+	token, err := utils.GetServiceUserToken(newctx, gatewayClient, c.serviceAccount.ServiceAccountID, c.serviceAccount.ServiceAccountSecret)
 	if err != nil {
+		c.logger.Error().Err(err).Msg("Error getting token for service user")
 		return err
 	}
-	if authRes.GetStatus().GetCode() != rpcv1beta1.Code_CODE_OK {
-		return fmt.Errorf("error authenticating service user: %s", authRes.GetStatus().GetMessage())
+
+	lgClient, err := c.setupLibregraphClient(newctx, token)
+	if err != nil {
+		c.logger.Error().Err(err).Msg("Error setting up libregraph client")
+		return err
 	}
 
-	lgClient, err := c.setupLibregraphClient(newctx, authRes.GetToken())
+	lgUser, resp, err := lgClient.UserApi.GetUser(newctx, user.GetId().GetOpaqueId()).Expand([]string{"memberOf"}).Execute()
+	if resp != nil {
+		defer resp.Body.Close()
+	}
+	if err != nil {
+		c.logger.Error().Err(err).Msg("Failed to lookup user via libregraph")
+		return err
+	}
+
+	currentGroups := lgUser.GetMemberOf()
+	currentGroupSet := make(map[string]struct{})
+	for _, group := range currentGroups {
+		currentGroupSet[group.GetDisplayName()] = struct{}{}
+	}
+
+	newGroupSet := make(map[string]struct{})
+	if groups, ok := claims[c.autoProvisionClaims.Groups].([]interface{}); ok {
+		for _, g := range groups {
+			if group, ok := g.(string); ok {
+				newGroupSet[group] = struct{}{}
+			}
+		}
+	}
+
+	for group := range newGroupSet {
+		if _, exists := currentGroupSet[group]; !exists {
+			c.logger.Debug().Str("group", group).Msg("adding user to group")
+			// Check if group exists
+			lgGroup, err := c.getLibregraphGroup(newctx, lgClient, group)
+			switch {
+			case errors.Is(err, errGroupNotFound):
+				newGroup := libregraph.Group{}
+				newGroup.SetDisplayName(group)
+				req := lgClient.GroupsApi.CreateGroup(newctx).Group(newGroup)
+				var resp *http.Response
+				lgGroup, resp, err = req.Execute()
+				if resp != nil {
+					defer resp.Body.Close()
+				}
+				switch {
+				case err == nil:
+				// all good
+				case resp == nil:
+					return err
+				default:
+					// Ignore error if group already exists
+					exists, lerr := c.isAlreadyExists(resp)
+					switch {
+					case lerr != nil:
+						c.logger.Error().Err(lerr).Msg("extracting error from ibregraph response body failed.")
+						return err
+					case !exists:
+						c.logger.Error().Err(err).Msg("Failed to create group via libregraph")
+						return err
+					default:
+						// group has been created meanwhile, re-read it to get the group id
+						lgGroup, err = c.getLibregraphGroup(newctx, lgClient, group)
+						if err != nil {
+							return err
+						}
+					}
+				}
+			case err != nil:
+				return err
+			}
+
+			memberref := "https://localhost/graph/v1.0/users/" + user.GetId().GetOpaqueId()
+			resp, err := lgClient.GroupApi.AddMember(newctx, lgGroup.GetId()).MemberReference(
+				libregraph.MemberReference{
+					OdataId: &memberref,
+				},
+			).Execute()
+			if resp != nil {
+				defer resp.Body.Close()
+			}
+			if err != nil {
+				c.logger.Error().Err(err).Msg("Failed to add user to group via libregraph")
+			}
+		}
+	}
+	for current := range currentGroupSet {
+		if _, exists := newGroupSet[current]; !exists {
+			c.logger.Debug().Str("group", current).Msg("deleting user from group")
+			lgGroup, err := c.getLibregraphGroup(newctx, lgClient, current)
+			if err != nil {
+				return err
+			}
+			resp, err := lgClient.GroupApi.DeleteMember(newctx, lgGroup.GetId(), user.GetId().GetOpaqueId()).Execute()
+			if resp != nil {
+				defer resp.Body.Close()
+			}
+			if err != nil {
+				return err
+			}
+		}
+	}
+	return nil
+}
+
+func (c cs3backend) getLibregraphGroup(ctx context.Context, client *libregraph.APIClient, group string) (*libregraph.Group, error) {
+	lgGroup, resp, err := client.GroupApi.GetGroup(ctx, group).Execute()
+	if resp != nil {
+		defer resp.Body.Close()
+	}
+	if err != nil {
+		switch {
+		case resp == nil:
+			return nil, err
+		case resp.StatusCode == http.StatusNotFound:
+			return nil, errGroupNotFound
+		case resp.StatusCode != http.StatusOK:
+			return nil, err
+		}
+	}
+	return lgGroup, nil
+}
+
+func (c cs3backend) updateLibregraphUser(userid string, user libregraph.User) error {
+	gatewayClient, err := c.gatewaySelector.Next()
+	if err != nil {
+		c.logger.Error().Err(err).Msg("could not select next gateway client")
+		return err
+	}
+	newctx := context.Background()
+	token, err := utils.GetServiceUserToken(newctx, gatewayClient, c.serviceAccount.ServiceAccountID, c.serviceAccount.ServiceAccountSecret)
+	if err != nil {
+		c.logger.Error().Err(err).Msg("Error getting token for service user")
+		return err
+	}
+
+	lgClient, err := c.setupLibregraphClient(newctx, token)
 	if err != nil {
 		c.logger.Error().Err(err).Msg("Error setting up libregraph client")
 		return err