Using Spektral to Link Prediction

[ecastillot]

Good morning!
I am interested to do link prediction task using Graph Neural Networks, Spektral supports this problem? If it's possible, do you have a basic example for it ?

[danielegrattarola]

Hi,

sure, it's possible to do link prediction with Spektral.
There are no examples yet, but you model should use some combination of convolutional layers and spektral.layers.InnerProduct as final layer to compute a similarity between the nodes.

I have put the example on my TODO list, I will reply here once it's done.

[ecastillot]

Hi @danielegrattarola ,

Do you have the basic example to do link prediction?
I understand the use of the final layer to calculate the similarity between the nodes. However, I am not very clear how we can use this schema to build the dataset with spektral.data.Dataset or how to use the generators, for example DisjointLoader.
This time the labels are links/edges between each possible combination between the nodes of the graph.

Again thank you very much.

[danielegrattarola]

Hi, you wouldn't use a Loader to implement link prediction because it's not designed for it.
Have you taken a look at the example posted below?

[ecastillot]

I have tried to follow the example. However, it is somewhat difficult for me to understand how he constructs the input parameters of the model.

[joewilaj]

I have tried to follow the example. However, it is somewhat difficult for me to understand how he constructs the input parameters of the model.

My apologies for the lack of clarity in the model inputs. Let me try to clarify the 'nba_gen' model in models.py. The other GNNs used are constructed similarly . The model input is the Adjacency Matrix (A_input) and a feature representation of the nodes (node2vec_input), as well as the indices of the two nodes (stored in 'team_inputs') for which the model will predict a link . (There is also another version of the graph and feature representation - A_veg_input and vnode2vec_veg - that is processed exactly as A, so I will ignore those layers).

The Adjacency Matrix and features are fed to a spektral layer 'conv' to get a representation of all nodes. Then, this representation is passed to the custom layer (game_vec in extract_team_GAT.py) along with the 2 node indices, say i and j, stored in 'team_inputs'. The custom layer uses tf.gather to extract only the representation corresponding to the two relevant nodes, which are just the rows i and j of the feature representation. (Note: my example gathers 4 rows from the representation of A because the graph has an an offense and defense node for each team, as well as 2 rows from the A_veg representation. In a more typical graph, you can just extract the representations of the two nodes here. ). Then these representations of the two nodes are fed to a sequence of dense layers to make a prediction about the direction and weight of the link between i and j.

the Game_vec custom layer is geared toward a specific use case. However, writing a similar custom layer that uses tf.gather is the simplest way to isolate two nodes for link predicition.

[joewilaj]

Hi,

I just posted an implementation of link prediction with spektral here: https://github.com/joewilaj/nbaGNNs. See models.py. These models are used to predict score differential in nba basketball games. In order to do link prediction, the outputs of the chosen GCN layer are passed to a custom layer, 'game_vec' that extracts the relevent node representations to use for link prediction.

Let me know if you have any questions

Thanks,
Joe

[danielegrattarola]

Looks super cool, and I love ML applications to sports!

Thanks for sharing
Daniele

[danielegrattarola]

To anyone finding this discussion looking for a link prediction example, check out this example too

https://github.com/FilippoMB/Variational-Graph-Auto-encoders-Tensorflow-2-Spektral-

[ecastillot]

Thanks, It is helping me a lot!!
I suppose that it could also work with Disjoint mode. If I have a supergraph of disconnected graphs and I want to do link prediction task on each disconnected graph, could it work?

[danielegrattarola]

You need to be careful to only reconstruct the block diagonal adjacency matrix, otherwise you'll be predicting meaningless edges between graphs.
I have never done it but it should be possible with masking

[ecastillot]

Ok, only one more thing before I start my project. Does the following make sense?

Given a supergraph G with disconnected subgraphs (A,B,C..), but the nodes in each subgraph are connected. I want to train a model to predict edge labels (different to the initial edges) in each subgraph. After that, now if we use a new subgraph, could do the model predict the edge labels in this new subgraph?

I hope to make myself understood.
And again, thank you for your constant help.

[danielegrattarola]

Yes, that would work. It depends on the task, data, etc., but you can design a model for inductive link prediction.

[sac3tf]

Following up on a similar question. If I wanted to make predictions on missing link features and node features (i.e. I know a link and node exist but the features on them are missing) within a graph, are there any examples out there for this? Do the predictions have to be done in a specific order (node and then link or link and then node) or can they be predicted at the same time/in the same process? I apologize if I am not asking this properly; GNNs are extremely new to me.

[danielegrattarola]

Are there any examples out there for this?

Not for this specific use case I'm afraid, but it should be possible to use standard building blocks in Spektral to achieve what you want.
I doubt the default Keras training loop will be of much use, but you can easily write your own

Do the predictions have to be done in a specific order (node and then link or link and then node) or can they be predicted at the same time/in the same process?

It depends on whether you want to use the predicted node/link features to then predict the link/node features, so I guess it depends on the data/task. There are no technical limitations in either case, it's just a matter of preference.