Getting NBA Shots in Context: Analysing Basketball Shots with Graph Embeddings

Graph neural networks are consolidating tools to perform tasks, including link prediction, node classification or graph classification. Bronstein et al. (2017) surveys the problems in applying deep learning to graph-structured data. Under Gilmer et al. (2017), different GNN structures are homogenized under the term message passing neural networks (MPNN) and the connections between different aggregation functions, nodes and edges. In contrast to previous work (Kipf & Welling, 2017; Schütt et al., 2017) that builds on concepts and ideas around spectral filtering and graph Fourier transformation, Gilmer et al. (2017) worked with a more geometrical, graphical understanding of the underlying algorithms. Recent work shows extensive progress in the performance of MPNN in quantum chemistry (Unke et al., 2021), social networks (Tan et al., 2019), traffic forecasting (Li et al., 2018) or football pass completion prediction (Stöckl et al., 2021). All those implementations used the naturally graphical representation of the data in spatial or temporal dimensions. In traffic forecasting, spatio-temporal graph convolutional networks (STGCN) (Yu et al., 2018) and diffusion convolutional recurrent neural networks (DCRNN) (Li et al., 2018) are dominant approaches. STGCN layers are “sandwiched” layers consisting of spatial and temporal convolutions that can be optimized in parallel and more effectively than recurrent units. In contrast, DCRNNs spatial structure is based on bidirectional graph random walks and decouples the temporal dependencies with encoder-decoder structures. Veličkovic et al. (2018) introduced a graph attention mechanism that allows generalizing well on different data sets and across multiple special networks with implemented attention mechanisms (Duan et al., 2017). Dick and Brefeld (Dick & Brefeld, 2023) use recurrent graph neural networks to model inter-dependencies between players for football. Rahimian et al. (Rahimian et al., 2024) use GNNs to predict pass success probabilities in football. Eventually, graph networks are well suited for modeling interactions between unordered sets of datapoints that preserve a structural integrity like player trajectories, the interaction between the players or the geometrical dependencies between them.

The earliest models of field goal attempts in basketball relied on a straightforward division of the court into broad areas, such as the two-point and three-point zones, and aggregated basic statistics of made shots within these regions (Jaime Sampaio & Alberto, 2006). Although these early models provided some insights that were expected to aid in player recruitment, the granularity of data was quite limited, restricting their practical application.

As the field progressed, the level of detail in the data increased with more precise annotations of shooter positions, enabling richer analyses of shooting performance (Goldsberry, 2012; Reich et al., 2006). While these advancements resulted in visually engaging graphics for fans, sports analysts and researchers continued to push beyond visualization by developing more sophisticated models that incorporated additional variables, such as the distance to defenders, remaining time on the shot clock, and various other contextual features.

In the publication by Chang et al. (2014), extensive heatmaps are generated for count statistics, multiple statistical models are fitted with the generated features and the evaluation of expected shot quality models (ESQ) is discussed. Here, a specific valuation of the decrease in Brier-loss is discussed with respect to an improvement in points per game. Several models estimate the shot quality or the FG% (Franks et al., 2015) (Skinner, 2012). In Daly-Grafstein and Bornn (2018), it is investigated how additional information about the shot (such as shooting angle, entry angle and shot depth) helps to infer FG% and stabilize this information earlier in the season, corresponding to a smaller sample size.

In Rolland et al. (2020), the origin of 3-point shots is characterized by a physical space occupation map and time occupation model based on point-mass moving equations. They show catch-and-shoot events need more time than pull-ups for the same ESQ. Pelechrinis and Goldsberry (2021) built on this work by analyzing FG% in conjunction with player tracking data, demonstrating that the efficiency of corner three-pointers is not solely due to their proximity to the basket but also to the high frequency of assisted attempts—marking an important step in disentangling ESQ from underlying factors contributing to shot success. Sandholtz and Bornn (2020) developed a more detailed time-dependent ESQ model that incorporates dynamic transition models based on Markov Decision Processes (MDP) to investigate shot selection and simulate single episodes of the game.

Cervone et al. (2014) uses the ESQ to build the game’s expected point value map. This was recently extended by (Sa-Freire et al., 2024) using a GNN to predict the immediate expected points a team can score from a specific action combined with a more granular action subdivision, by building different models for different actions e.g. passing, turnover or ESQ.

In basketball, ESQ can be described in a probabilistic fashion by predicting the probability that a shot is successful with p(y = t|x) where x describes the game’s current state and y denotes if the shot was a success (t) or not (¬t). The feature vector x can contain multiple physical quantities since the emergence of tracking data. We use a tracking data set provided by SportsVU, which includes player and ball coordinates at 25 Hz for event sequences of all 2015/16 NBA season games and annotations like shot type, shot outcome, player names, and player positions over time. As the shot time stamps of annotations are inaccurate in the original data, they are recomputed based on the acceleration of the ball and the shooter coordinates. The shot time is chosen when the ball is closest to the shooter and accelerates fast towards the hoop. Tracking data is converted from full-court coordinate space into half-court coordinate space by mirroring along the half-court line.

For each time frame, players are represented as nodes with respective features. Each node is connected with spatial edges to their k closest neighbours. Spatial edge features indicate the distance between the nodes. Per-frame spatial graphs are connected over time into single graphs by adding temporal edges between a node and its corresponding neighbours in the time axis (Figure 2 left). We use more frames closer to the time of the shot than further back in time to have both short-term and long-term information while keeping the size of the graph small. Temporal edge features indicate the time difference in the number of frames between nodes. Figure 2 displays the tracking data in its graphical representation.

Figure 2:

Left: Spatio-temporal subgraph of tracking data from multiple time frames. Spatial edges are in grey (respective on the x and y plane), and temporal edges are in lime (in the time dimension). Right: Spatio-temporal graph for a game situation with seven frames before the shot. The red and blue nodes are for attackers and defenders, respectively. Green nodes are for the ball. Playerposition is annotated with their position (G: Guard, F: Forward, C: Center).Similar colored rings around the players indicate the players’ identity. Spatialedges are grey and indicate the distance between the nodes. The shooter nodehas an outline in magenta. Temporal edges are green and connect nodes from neighboring frames. The numbers indicate the temporal delay in time frames.

To avoid the possibility that the model can calculate the shot’s outcome based on the ball’s motion, no frame after the shot time stamp is added. The model should learn to focus on shot selection, defensive pressure and other aspects of the game state; otherwise, we could model the shot by methods that model the physics of the basketball (Daly-Grafstein & Bornn, 2018). Predicting if a shot goes in (or not) boils down to a binary classification problem, resulting in heavy feature engineering in previous works. The GNN approach allows us to exploit the graph structure of tracking data, enabling us to extract new information from the data with similarity-search in embedding space and interpret the data with visualization of node and feature characteristics in new ways.

A graph G = (V, E) is structured in nodes V = {v₁, ..., v_n} and edges E = {(x, y)|x, y ∈ V ∧ x ≠ y}. Graphs can either be classified as homogeneous or heterogeneous graphs. Heterogeneous means that the edges in the graph are different, while all nodes and edges have the same type in a homogeneous graph. Our graph is heterogeneous; we denote spatial edges as E^S that model geometric dependencies between players and temporal edges as E^T, modeling the temporal evolution of the players’ positions. The task is to classify the given computational graph, made shot. This results in optimizing either the Brier loss or the log loss.

We process the graph through a graph neural network with multiple spatio-temporal graph layers. These layers, help to process information and counter the effect of oversmoothing, as using a homogenous graph, results in a large neighborhood (Rusch et al., 2023). For a spatiotemporal graph layer L, each node v_i has node features Fi,LV F_{i,L}^{V} , each spatial edge ex,ys e_{x,y}^{s} from a node x to a node y has features Fx,y,Ls F_{x,y,L}^{s} and each temporal edge ex,yT e_{x,y}^{T} has features Fx,y,LT F_{x,y,L}^{T} . We use shared node and edge functions to update the features from one layer to the next. In our case, these are represented by neural networks NLV N_{L}^{V} , NLS N_{L}^{S} and NLT N_{L}^{T} , corresponding to the node and edge features. Additionally, a per-edge attention value ax,ys|t a_{x,y}^{s|t} for spatial and temporal edges are evaluated with the help of a neural network A^s|t. This is highly related to GAT and adapted to our specific problem.

First, we update spatial and temporal edge features by applying the respective neural networks over the concatenation of the current edge features and the connected node features. Given an edge from node x to node y, the update is performed as follows

Fx,y,L+1s|t=NLs|tFx,y,L+1s|t,FxV,FyV F_{\left( x,y \right),L+1}^{s|t}=N_{L}^{s|t}\left( F_{x,y,L+1}^{s|t},F_{x}^{V},~F_{y}^{V} \right)

For a node V_x the per-edge attention value is calculated for all k edges {(y₁,x), (y₂,x),...,(y_k,x)} that connect nodes v_yi with i = {1,...,k} to node v_x based on the already updated edge features.

ayi,x,Ls|t=σALs|tFx,yi,L+1s|t=eayi,x,Ls|t∑j=1keaj,x,Ls|t \begin{array}{*{35}{l}} a_{\left( {{y}_{i}},x \right),L}^{s|t} & =\sigma \left( A_{L}^{s|t}\left( F_{x,{{y}_{i}},L+1}^{s|t} \right) \right) \\ {} & =\frac{{{e}^{a_{\left( {{y}_{i}},x \right),L}^{s|t}}}}{\mathop{\sum }_{j=1}^{k}{{e}^{a_{\left( j,x \right),L}^{s|t}}}} \\\end{array}

The node v_x collects all connected spatial and temporal edge features from all nodes into a spatial or temporal reduced message mx,Ls|t m_{x,L}^{s|t} by applying the attention weighted sum over the respective updated edge features mx,Ls|t=∑j=1kayj,x,Ls|t∘Fyj,x,L+1s|t m_{x,L}^{s|t}=\sum\limits_{j=1}^{k}{a_{\left( {{y}_{j}},x \right),L}^{s|t}\circ F_{\left( {{y}_{j}},x \right),L+1}^{s|t}}

Finally, we obtain the updated node features of a node v_x by applying the neural network over the concatenation of node features, its spatial reduced message and its temporal reduced message

Fx,L+1V=NLVFx,LV,mx,Ls,mx,Lt F_{x,L+1}^{V}=N_{L}^{V}\left( \left[ F_{x,L}^{V},m_{x,L}^{s},m_{x,L}^{t} \right] \right)

In order to get a scalar value in the range [0,1] for the probabilisitc prediction, we first apply attention-weighted reduction over all the nodes and edges in the graph. This operation also uses attention functions similar to the per-edge attention calculation in the spatio-temporal layer. After iteratively applying all the spatio-temporal layers to the input graph, we obtain the final node, spatial edge and temporal edge features. The result is a single node, a single spatial edge and a single temporal edge with respective aggregated information. We then evaluate a final linear layer with a sigmoid activation function over the concatenation of the aggregated features to obtain the classification prediction.

The training of the graph models is a graph classification task. Robberechts et al. (2021) argue that training with a Brier loss is superior when optimizing for probabilities that are further used for models that are summing over probabilities. At the same time, cross-entropy is an advantage when further multiplying the probabilities. Both losses return different optimization surfaces, leading to a superior loss for a specific problem. To investigate this behaviour with respect to calibration, we trained the same model with Brier- and cross-entropy-loss.

In basketball, models must be well-calibrated. When a model outputs a probability, this corresponds to the actual number of outcomes this event might occur. If a well-calibrated model predicts 80% success probability, meaning 80 of 100 shots lead to a field goal in the reference data. This turned out to be especially challenging in basketball regarding the distribution’s tails at zero and one. We inspect the calibration of our models with the expected calibration error (ECE) and reliability diagrams (Niculescu-Mizil & Caruana, 2005). The ECE takes the weighted average over the difference between n binned accuracy and binned confidence. The binned (B) confidence (conf) and accuracy (acc) correspond to confBi=1Bi∑n∈Bipn conf\left( {{B}_{i}} \right)=\frac{1}{\left| {{B}_{i}} \right|}\sum\nolimits_{n\in {{B}_{i}}}{{{p}_{n}}} accBi=1Bi∑n∈Bi𝕀yn=y^n acc\left( {{B}_{i}} \right)=\frac{1}{\left| {{B}_{i}} \right|}\sum\nolimits_{n\in {{B}_{i}}}{{\mathbb{I}}\left( {{y}_{n}}={{{\hat{y}}}_{n}} \right)} and the ECE is the sum over the bins.

ECEB=∑n∈BiBinaccBi−confBi ECE\left( B \right)=\sum\limits_{n\in {{B}_{i}}}{\frac{\left| {{B}_{i}} \right|}{n}\left| acc\left( {{B}_{i}} \right)-conf\left( {{B}_{i}} \right) \right|}

In our results, we used 30 bins to calculate the ECE, as in basketball, the quality of a shot can vary by around three percent and makes the difference between a bad, average, good and excellent shooter.

A common problem with applying neural networks is the explainability or interpretability of the data. In logistic regression, tree models or gradient boosting models, this is inherent in the model due to feature weights, splitting or feature importance, respectively. In neural networks it started to get investigated with LIME (Ribeiro et al., 2016), SHAP framework (Lundberg et al., 2017) or newly for graphs there are multiple models proposed (Duval & Malliaros, 2021; Huang et al., 2020; Ying et al., 2019) to explain single predictions. All those models focus on the interpretability of features and structures in the graph. The attention mechanism is a proxy for the interpretability of the edge importance (Ying et al., 2019). Using attention in our model as a proxy for the interpretability corresponds to the temporal and spatial evolution of the nodes. The proximity of the nodes is hence captured by the attention weight and can be visualized as described in Figure 6. We can see that the temporal edge attention is mainly on the previous positions of the shooters and the closest defenders’ previous positions. The spatial edge attention contributes with large weights to the shooters from the closest players around him, including defence and offence. The same behaviour is expected from other classifiers (and common sense) that the closest defenders contribute to the shooter’s performance.

To analyse the impact of the respective features, we used the GNNExplainer to give some interpretability to the features. We iterate over the test set of shot predictions and take the mean of the respective node feature mask with respect to the shooter node. When using all nodes for computation of the node feature mask, the values of the mask collapse at approximately the same values for different observations. As our prediction is yet made on the shooter node, we take the respective features of the shooter node to evaluate the associate graph features. This increases diversity in node feature mask values, yet the obvious happens: Node class features are less critical than positional features. Respectively, we receive much more interpretable results from the attention mechanism to give us additional information about the connectivity of the respective nodes.

In Figure 4 we can see the node feature importance for the shot taken in Figure 6. This helps coaches to interpret and communicate specific shot outcomes first through the attention mechanism, which gives insights into the strength of the signal from different nodes and secondly through the weighting of the respective node features. Hence, storytelling is made easy with the interpretability of the model.

Figure 4.

Normalized values of the learned feature mask by the GNNExplainer model for the shot classification task with output node shooter and mean of all embeddings (orange). The graph shows the difference in importance for an average aggregate over all nodes, and the average shooter node.

Figure 5.

In the left frame, we can see the embeddings for the neural network predictions with a mean over different shooters with coloring in positions. The pure guards are colored in turquoise, the pure forwards are colored in pink, and the pure centers are colored in red. Guard-Forwards are colored in blue and the Forward-Centers are colored in green. On the right, the embeddings of every single situation are taken with coloring in 2-point (red) and 3-point shots (blue), both displayed with the TSNE-Embedding visualization.

Since the GNN only updates the features of the input graph and keeps the nodes and edges, we can investigate the features after the last layer (embeddings). Using the knowledge about the structure of the graph before passing it through the network, we can derive various kinds of embeddings by focusing on certain nodes/edges. We compute and compare three kinds of embeddings:

• Shooter embeddings: a vector containing the average of all temporal shooter node embeddings, averaged over all shots in the dataset, which results in one embedding per player. In the following, for clarity, player embedding corresponds in the wider sense to shooter embedding.

• Attacker or defender embeddings: a vector containing the average of all temporal attacker/defender node embeddings, averaged over all shots in the dataset (attacker/defender team), which results in one embedding per team

• Situation embeddings: a vector containing the attention weighted node, spatial edge and temporal edge embedding, resulting in one embedding per shot

The respective embeddings are averaged by the equation: hp|t=1Hp|t∑i∈Hp|thi {{h}_{p|t}}=\frac{1}{\left| {{H}_{p|t}} \right|}\sum\limits_{i\in {{H}_{p|t}}}{{{h}_{i}}} where |H_p| and |H_t| are the number of plays and H_p and H_t the number of sets corresponding to the respective player or team. Especially the situation and shooter embeddings are interesting. For situation embeddings, we can use nearest neighbors search to find similar plays to compare to and visualize the corresponding graph with attention and the respective feature importance (see Figure 6).

Player similarity analysis holds significant value in the NBA, especially for team managers when building rosters, replacing departing players, or devising defensive strategies. Popular similarity measures, such as FiveThirtyEight’s RAPTOR score (Silver, 2023) and Regularized Adjusted Plus-Minus (RAPM), are often used to assess player contributions (Gong & Chen, 2024). These measures evaluate a player’s impact on their team’s success, either by looking at their performance over 100 possessions or by comparing their plus-minus score using three seasons of NBA data. However, these traditional metrics focus primarily on team success rather than individual shot-making patterns. Miller et al. (2014) uses non-negative matrix factorization (NMF) to compute base vectors for spatial shot selection. Our approach introduces an alternative method by comparing players based on shot-making similarities, which can offer a more nuanced understanding of their offensive play style. We validate our similarity analysis using the well-known “Messi” case study or also known as the “face validity test” (Davis et al., 2024), which illustrates the effectiveness of evaluating individual contributions beyond standard metrics, especially for offensive contributions. In Figure 5, we visualize player embeddings using the t-SNE model. The color coding in the visualization highlights player positions, revealing that guards, forwards, and centers tend to cluster together, indicating that these positions often exhibit similar shooting patterns. Nonetheless, there are instances of players classified as guards who perform more like forwards, and vice versa, as well as overlapping characteristics between forwards and centers. Table 2 provides a comparison of Stephen Curry’s nearest neighbors according to different similarity metrics. Our embeddings emphasize shot-making similarity, which tends to highlight players with a similar offensive playing style. For instance, basketball experts would generally agree that Kyle Lowry and Damian Lillard are more similar to Stephen Curry in terms of offensive skill set compared to players like LeBron James, Dwyane Wade, or Rudy Gobert. This observation suggests that utilizing seasonal or advanced statistics, combined with our embeddings, could provide a more accurate measure of player similarity. Table 2 also includes the L2-distance between players’ offensive statistics based on data from Basketball-Reference. This distance measure averages season statistics without necessarily factoring in variables like playing time or total points scored. Additionally, NMF-weight-correlation is the correlation coefficient between the decomposed player weight matrix from Miller et al. (2014). This approach focuses on comparing shot location from specific players and less on the volume. Our approach, therefore, extends beyond existing statistical methods by focusing on specific skills, and pre-shot behavior, making it more useful for identifying players with similar capabilities. A detailed calculation of “Normalized Stats” can be found in the appendix.

Figure 6.

Left: Shot of Serge Ibaka. Right: Shot from Blake Griffin. The attacking team is displayed in red with the shooter in magenta. The defending team is displayed in blue and the ball in dark green. Spatial edges are drawn in black/grey and temporal edges in lime. The thickness of the lines and nodes indicates the attention values, while the darker colour indicates the stronger attention of the network.

While the similarity in player shot type chart is quite important for managers to support decision making, either for role players or superstars, similarity in plays with the help of attention mechanism, might help coaches to understand specific shot types in more detail. Assuming we have two shots that have similar last seconds, this should result in similar embeddings. Hence, comparing these shots can lead to insights into how to defend specific situations or create space for a good bucket. These shots are already labelled with a lot of semantics (Action Type, Shot Zone etc.), yet the positional data gives another dimension of semantics to the shot, as labels might be slightly different but the shots have been created similarly.

In Figure 6 we compare shots of Serge Ibaka (left) and Blake Griffin (right). Those shots are closest to each other in the play embeddings with a Euclidean distance of 0.197. Serge Ibakas shot was classified as a jump shot from the left side center, while Blake Griffins attempt was classified as a center Jump shot. From tracking data we can see that the interesting part is not the shot classification of center or left side center. In both possessions, the network focuses on the closest defender as well as on the open shooter on the right, while the position of the other players is less important to the network in this situation, based on the edge and node attention of the graph.

A practical application, which comes with that, is the possibility to use the embeddings to automatically select videos for match analysis. E.g., coaches who want to create certain (promising) shot opportunities for their players, can look at how similar plays have been created against the upcoming opponent. Such data-driven detection and labelling of game situations is a valuable tool for coaches and performance analysis, as it could already be shown in other sports. Vice versa, coaches can be equipped with reports that show the most common (successful) similar plays, which can be used to adapt the defensive game plan. This could even be combined with previously suggested tools that allow coaches to simulate the behaviour of the opposing team against specific plays (Bauer & Anzer, 2021; Seidl et al., 2018).