Distinguish ids from multiple providers #9896

3link · 2023-05-03T15:10:32Z

Type of change

Other

Description of change

The goal of this PR is to sketch an implementation proposal for allowing multiple user id modules to expose the same kind of identifiers. E.g. two modules exposing UID2. The idea is that a config - like the one below - provides priority in case a collision happens.

pbjs.setConfig({
    userSync: {
        idPriority: {
            uid2: ['uid2', 'liveIntentId']
        }
...

modules/liveIntentIdSystem.js

dgirardi

I do have some concerns about this approach - it seems to care only about what ends up in the EIDs array; with no support for the old userId map. I think this means that we need to deprecate it, or if this catches on adapters that are using it will effectively become second class citizens (but I'll defer to @patmmccann for the decision).

By that I mean that for example, if your adapter uses userIdAsEids you can get uid2 from liveintent; but if you're using userId you won't.

It does have the advantage of being much simpler than a comprehensive solution.

dgirardi · 2023-05-31T16:49:07Z

modules/userId/index.js

@@ -453,7 +453,7 @@ function getSubmoduleId(submodules, sourceName) {
    return {};
  }
  const submodule = submodules.filter(sub => isPlainObject(sub.idObj) &&
-    Object.keys(sub.idObj).length && USER_IDS_CONFIG[Object.keys(sub.idObj)[0]]?.source === sourceName);
+    Object.keys(sub.idObj).length && USER_IDS_CONFIG[Object.keys(sub.idObj)[0]]?.find(conf => conf.source === sourceName));


I think this says "the module that corresponds to this source is the first one I find that can provide IDs from this source". That's not necessarily true is it? I think we need a way to mark the "owner" of a source outside of the EID object. It could be as simple as the first one declared is the one you own.

actually, since this is only used in here:

Prebid.js/modules/userId/index.js

Lines 656 to 658 in d1b228c

function getUserIdsAsEidBySource(sourceName) {

return createEidsArray(getSubmoduleId(initializedSubmodules, sourceName))[0];

}

I think what we should do is reverse the filtering - instead of finding the module for a given source to get its EIDs, it makes more sense now to get all EIDs and filter them by source (since they can come from different modules).

exactly, "get all EIDs and filter them by source". If there is a collision, i.e., 2 or more eids with the same source check the provider and choose the one where the provider and source are the same. The question then becomes what do we do when one eid has no provider value?

The eids#USER_IDS_CONFIG could be adjusted to always have a global provider for each user id that is then used as provider for all uids extracted based on it. That way every eid/uid would have a provider. For example:

'lipb': { provider: 'liveintent.com', extractors: [ // liveIntentId { getValue: function(data) { return data.lipbid; }, source: 'liveintent.com', atype: 3, getEidExt: function(data) { if (Array.isArray(data.segments) && data.segments.length) { return { segments: data.segments }; } } }, // uid2 { source: 'uidapi.com', atype: 3, getValue: function(data) { return data.uid2; } } ] }

And eids#createEidObjects would take care of building the uid.ext using the global provider value and output

[{ source: 'liveintent.com', uids: [{ id: 'some-random-id-value', atype: 1 }], ext: { provider: 'liveintent.com', segments: ['s1', 's2'] } }, { source: 'uidapi.com', uids: [{ id: 'some-random-id-value', atype: 3, ext: { provider: 'liveintent.com' } }] }]

One related question: what if there is a collision and neither of the providers is the owner of the id?

Maybe it would make sense to return an eid with multiple uids when getUserIdsAsEidBySource is called and there is a collision?

For example: getUserIdsAsEidBySource('uidapi.com') would then return

[{ source: 'uidapi.com', uids: [{ id: 'some-random-id-value', atype: 3, ext: { provider: 'uidapi.com' } }, { id: 'some-random-id-value', atype: 3, ext: { provider: 'liveintent.com' } }] }]

when a UID2 could be resolved via LI's and the UID2 adapter. The caller then decides which uid to pick if there are multiple. We also can make sure that the uid of the id owner is always at index 0 in the uids array.

Same can be applied when building the eids array in general.

currently getUserIdsAsEidBySource returns a single object, and it's part of the public API, so that's not really up for change. if there's more than one EID we have to pick one.

I agree that the API should not be changed in this regard. getUserIdsAsEidBySource should return one single object - one eid. An eid, however, can have an array of uids. My proposal is rather about - in case of collisions - exposing multiple uids with their providers within the eid that is returned. By providing the "provider" field the uids become distinguishable and the caller can decide what id to use in case there are multiple based on the provider.

I think to do that you have to make the assumption that all providers for a source will always generate "compatible" EID objects; for example, ~~they share atype and~~ (edit - cancel that, atype is part of UID) everything they put in ext (through getEidExt) always applies equally no matter the provider. I would be very hesitant to rely on this especially since the EID objects are only likely to get more complicated over time. Choosing one when there's a conflict is an acceptable compromise IMO.

lets not change the API, that will require changes to bid adapters and the servers behind them, that would be a massive undertaking. Can we make this configurable? Can the pub declare a list of priorities in the core user id module?

========================
In case of user id source collisions

liveintent.com

pubcid.org

google.com

If this prioritization still doesn't produce a winner, pick one at random.

Can we make this configurable? Can the pub declare a list of priorities in the core user id module?

@jdwieland8282 Sounds good.

@jdwieland8282 What do you think, should that config allow to provide priority per identifier type - e.g. uid2 from uidapi.com has higher priority than uid2 from liveintent.com? Or should it be a config that defines that all identifiers from one user id module have priority over same types of identifiers exposed by any other module with lower priority? I personally tend towards the first option as it is more fine grained - but it is also more complex in the implementation and when writing the config.

Also, I would see if it is feasible to apply this prioritization logic when building the eids array, not only in getUserIdsAsEidBySource. Are we on the same page here?

We met today in Identity PMC call and made the following decisions after vote:
https://docs.google.com/document/d/1dXrTaA-gFXayP-QWhFZN_91kLWp7qFO1CH1GmgafUdc/edit?usp=sharing

We want to go with Option I.I which is to give publisher the ability to provide the priority of the modules for each identifier. In the event where no configuration is provided, the first id/source that gets populated from any provider will get picked.

We want this configuration defined in the core user id module so that publishers can define which. sub adapters should override the other when there are id conflicts

The priority field should appear as an optional parameter right next to name here for each id: https://github.com/prebid/Prebid.js/blob/master/modules/userId/userId.md

patmmccann · 2023-05-31T22:01:05Z

I'm pretty okay with this approach; we should also remember that the user ID method is in the public api and encourage the 57 adapters not using the eids method of the advantages to changing. It isn't really discriminating against an adapter if they can all change implementations at any time.

We hope live intent may help in this, letting their ssp partners know to prefer the eids method.

We should make sure this information is well published.

deepthivenkat · 2023-06-02T01:42:37Z

@patmmccann I can help come up with a public facing documentation explaining this.

3link · 2023-06-14T15:11:03Z

@jdwieland8282 @dgirardi @deepthivenkat

Adjusted the PR to make submodule priority configurable per user id. Also, closed the magnite id PR and added magnite id handling here.

3link · 2023-06-14T16:03:12Z

The e2e test failures seem not to be caused by the changes in this PR. Ran the e2e test from local machine successfully.

jdwieland8282 · 2023-06-15T16:57:24Z

I'm not qualified to code review this, but maybe @dgirardi can when he has a free moment. I can review the docs pr does that exist yet @deepthivenkat ?

dgirardi · 2023-06-15T20:44:29Z

modules/userId/index.js

+        const currentIdPriorityByName = idPriority[key]?.reverse().indexOf(i.submodule.name);
+        const currentIdPriorityByAlias = idPriority[key]?.reverse().indexOf(i.submodule.aliasName);
+        // eslint-disable-next-line
+        const currentIdPriority = currentIdPriorityByName ? currentIdPriorityByName : currentIdPriorityByAlias ? currentIdPriorityByAlias : - 1


I don't think this works (and I don't see it in a test case) - assuming you meant "use the name first, then the alias".
Because of how boolean casting works, this will always use the name (every value except 0 is truthy), except when it was the last element in the priority array (reversed becomes 0), in which case it will always use the alias, even if it was not in the array.

I am not sure that considering aliases here is necessary, but if we do, my approach would be to "resolve" them when the id priority is passed to setConfig; that would also help with performance. By that I mean, instead of just taking in idPriority verbatim from user input, do some processing so that when you get here idPriority[key] is already reversed, and does not contain any aliases - only the proper names.

@dgirardi Good catch! Addressed both concerns - flipping priority and alias handling - in 2256f92.

Kept the aliases handling for now - but if there are no strong arguments for keeping it, I can remove it.

dgirardi · 2023-06-15T20:54:08Z

modules/userId/index.js

    .filter(i => isPlainObject(i.idObj) && Object.keys(i.idObj).length)
    .reduce((carry, i) => {
      Object.keys(i.idObj).forEach(key => {
-        carry[key] = i.idObj[key];
+        const currentIdPriorityByName = idPriority[key]?.reverse().indexOf(i.submodule.name);


actually, doing the reverse only once may be a requirement and not just a suggestion - because IIRC it modifies the array in place so on the second call you'd actually be restoring it to the original order.

patmmccann · 2023-06-21T15:38:09Z

@deepthivenkat This looks ready to merge once we have a docs pull drafted?

deepthivenkat · 2023-06-21T15:59:35Z

Here is the docs pull: prebid/prebid.github.io#4675

…

On Wed, Jun 21, 2023 at 10:38 AM Patrick McCann ***@***.***> wrote: @deepthivenkat <https://github.com/deepthivenkat> This looks ready to merge once we have a docs pull drafted? — Reply to this email directly, view it on GitHub <#9896 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABYICH5AZWAJNJIQNXJYZWLXMMIOZANCNFSM6AAAAAAXURURWI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

* Make id priority configurable * Use mocked modules * Support provided ext for medianet, uid2 and bidswitch eid.uid * Add support for magnite id * Delete unused findEid function * Add a tests for getEncryptedEidsForSource * Extract constant for the provider domain * Make updatePPID take priority into account * Compute effective priority only once and handle aliases * Fix after merge * Use Map#has instead of Map#get * Remove duplicate test * Add tests * Run linter

3link commented May 3, 2023

View reviewed changes

modules/liveIntentIdSystem.js Outdated Show resolved Hide resolved

3link mentioned this pull request May 3, 2023

LiveIntent UserId module: Add support for magnite id #9828

Closed

1 task

patmmccann assigned jdwieland8282 May 18, 2023

skocheri1 approved these changes May 30, 2023

View reviewed changes

3link changed the title ~~DRAFT: Distinguish ids from multiple providers~~ Distinguish ids from multiple providers May 31, 2023

3link marked this pull request as ready for review May 31, 2023 14:38

skocheri1 approved these changes May 31, 2023

View reviewed changes

dgirardi reviewed May 31, 2023

View reviewed changes

3link added 7 commits June 12, 2023 13:49

Make id priority configurable

05177ef

Use mocked modules

0df1b84

Support provided ext for medianet, uid2 and bidswitch eid.uid

df941f0

Add support for magnite id

e17bfe9

Delete unused findEid function

0a6f6f3

Add a tests for getEncryptedEidsForSource

d3ade42

Extract constant for the provider domain

c0e5c62

3link force-pushed the distinguish_ids_from_multiple_providers branch from b8b0d17 to c0e5c62 Compare June 14, 2023 11:35

Make updatePPID take priority into account

1975504

muuki88 mentioned this pull request Jun 15, 2023

Add magnite id to examples prebid/prebid.github.io#4498

Merged

3 tasks

dgirardi reviewed Jun 15, 2023

View reviewed changes

Compute effective priority only once and handle aliases

2256f92

3link force-pushed the distinguish_ids_from_multiple_providers branch from edb0498 to 2256f92 Compare June 16, 2023 12:46

3link added 4 commits June 16, 2023 15:02

Merge branch 'master' into distinguish_ids_from_multiple_providers

a962852

Fix after merge

d40a4c0

Use Map#has instead of Map#get

a569d4a

Remove duplicate test

6feb33e

3link added 2 commits June 16, 2023 17:56

Add tests

8a33af3

Run linter

baa4d83

dgirardi approved these changes Jun 19, 2023

View reviewed changes

This was referenced Jun 23, 2023

Update userId.md to document scenario when user id submodule populate… LiveIntent/prebid.github.io#15

Closed

Update userId.md to document scenario when user id submodule populate… prebid/prebid.github.io#4675

Merged

patmmccann merged commit 1016e89 into prebid:master Jun 23, 2023
2 checks passed

patmmccann added the feature label Jun 27, 2023

This was referenced Jul 20, 2023

LiveIntent UserId module: magnite id documentation prebid/prebid.github.io#4741

Merged

LiveIntent UserId module: add magnite id support to legacy branch #10261

Merged

patmmccann mentioned this pull request Aug 24, 2023

Concerns with multiple ids coming from one id submodule response #9730

Closed

3link mentioned this pull request Mar 27, 2024

LiveIntent UserId module: Expose shared id #11118

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Distinguish ids from multiple providers #9896

Distinguish ids from multiple providers #9896

3link commented May 3, 2023 •

edited

Loading

dgirardi left a comment

dgirardi May 31, 2023

dgirardi May 31, 2023 •

edited

Loading

jdwieland8282 Jun 1, 2023

3link Jun 2, 2023

3link Jun 2, 2023 •

edited

Loading

3link Jun 5, 2023

dgirardi Jun 5, 2023 •

edited

Loading

jdwieland8282 Jun 5, 2023

3link Jun 6, 2023 •

edited

Loading

deepthivenkat Jun 7, 2023 •

edited

Loading

patmmccann commented May 31, 2023

deepthivenkat commented Jun 2, 2023

3link commented Jun 14, 2023

3link commented Jun 14, 2023

jdwieland8282 commented Jun 15, 2023

dgirardi Jun 15, 2023

3link Jun 16, 2023 •

edited

Loading

dgirardi Jun 15, 2023

patmmccann commented Jun 21, 2023

deepthivenkat commented Jun 21, 2023 via email •

edited

Loading

	function getUserIdsAsEidBySource(sourceName) {
	return createEidsArray(getSubmoduleId(initializedSubmodules, sourceName))[0];
	}

Distinguish ids from multiple providers #9896

Distinguish ids from multiple providers #9896

Conversation

3link commented May 3, 2023 • edited Loading

Type of change

Description of change

dgirardi left a comment

Choose a reason for hiding this comment

dgirardi May 31, 2023

Choose a reason for hiding this comment

dgirardi May 31, 2023 • edited Loading

Choose a reason for hiding this comment

jdwieland8282 Jun 1, 2023

Choose a reason for hiding this comment

3link Jun 2, 2023

Choose a reason for hiding this comment

3link Jun 2, 2023 • edited Loading

Choose a reason for hiding this comment

3link Jun 5, 2023

Choose a reason for hiding this comment

dgirardi Jun 5, 2023 • edited Loading

Choose a reason for hiding this comment

jdwieland8282 Jun 5, 2023

Choose a reason for hiding this comment

======================== In case of user id source collisions

3link Jun 6, 2023 • edited Loading

Choose a reason for hiding this comment

deepthivenkat Jun 7, 2023 • edited Loading

Choose a reason for hiding this comment

patmmccann commented May 31, 2023

deepthivenkat commented Jun 2, 2023

3link commented Jun 14, 2023

3link commented Jun 14, 2023

jdwieland8282 commented Jun 15, 2023

dgirardi Jun 15, 2023

Choose a reason for hiding this comment

3link Jun 16, 2023 • edited Loading

Choose a reason for hiding this comment

dgirardi Jun 15, 2023

Choose a reason for hiding this comment

patmmccann commented Jun 21, 2023

deepthivenkat commented Jun 21, 2023 via email • edited Loading

3link commented May 3, 2023 •

edited

Loading

dgirardi May 31, 2023 •

edited

Loading

3link Jun 2, 2023 •

edited

Loading

dgirardi Jun 5, 2023 •

edited

Loading

========================
In case of user id source collisions

3link Jun 6, 2023 •

edited

Loading

deepthivenkat Jun 7, 2023 •

edited

Loading

3link Jun 16, 2023 •

edited

Loading

deepthivenkat commented Jun 21, 2023 via email •

edited

Loading