Beyond Difference Scores: Unlocking Insights with Polynomial Regression in Studies on the Effects of Implicit-Explicit Congruency

1 Introduction

Individuals with low self-esteem think and behave in ways that diminish their quality of life (Swann Jr et al., 2007), whereas high self-esteem predicts success and well-being in life domains of relationships, work, and health (Orth & Robins, 2014). Positive psychology, building on Maslov’s conception of self-actualization has long posited that the self-valuing process is dependent on the quality of the self-concept (Ryan & Deci, 2000). Central to positive psychology’s paradigm is the position of Rogers who believed that for a person to achieve self-actualization they must be in a state of congruence (Rogers, 1957). The idea is that self-actualization occurs when a person’s “ideal self” (i.e., whom they would like to be) is congruent with their actual behavior (self-image). The closer self-image and ideal self are to each other, the more consistent or congruent the person is which increases self-esteem.

The interplay between implicit and explicit manifestations of one’s own self-esteem is often explained from the perspective of dual-process theory. Dual-process theory (Evans, 2008; Kahneman & Frederick, 2005) states that information is processed along two distinct pathways, a deliberate (explicit) and an automatic (implicit) pathway. The implication of this theory in the context of self-esteem would be the existence of two different self-esteem dimensions that play distinct roles in controlled and automatic behaviors: explicit self-esteem and implicit self-esteem (Buhrmester et al., 2011; Gawronski & Bodenhausen, 2006; Spalding & Hardin, 1999; Strack & Deutsch, 2004). Explicit self-esteem is thought to represent a conscious, cognitive and controllable reflection on one’s own self-worth (Schröder-Abé et al., 2007). Implicit self-esteem, on the other hand, prevails in situations in which the individual does not have particular goals, has little time to think, and/or is not aware of the outcome of the cognitive process (De Houwer et al., 2009). Implicit does not necessarily mean unconscious: participants may be aware of the attribute being assessed, but they have minimal control over the measurement outcome (Steffens, 2004). Kernis (2003) hypothesized that when implicit and explicit self-esteem differ, the direction of the discrepancy changes its meaning. When low implicit self-esteem concurs with high explicit self-esteem it is referred to as fragile self-esteem, whereas the combination of high implicit self-esteem with low explicit self-esteem is referred to as damaged self-esteem. Several studies that explored the effects of the discrepancy between explicit and implicit self-esteem have shown associations of discrepancy-size with depression (Creemers et al., 2013; Creemers et al., 2012; Franck et al., 2007; Leeuwis et al., 2014; Valiente et al., 2011), anxiety (Rudman & Ashmore, 2007; Schreiber et al., 2012; van Tuijl et al., 2014) and aggression (Jordan et al., 2005; Sandstrom & Jordan, 2008; Schröder-Abé et al., 2007).

Dual-process theory has met with well-reasoned critique in the past decade, mainly because of concerns with the idea that the two systems are sufficiently independent. Keren & Schul (2009) provide a thorough summary of key critical points, and assert that testing the theory of dual-process requires a re-evaluation of research methodologies used in studies that assume dual-processes take place. These critical points are sufficiently meaningful to warrant our attention. For example, in the above-mentioned studies, implicit and explicit pathways were operationalized as a discrepancy score. The discrepancy score was operationalized as a difference score between explicit and implicit self-esteem, representing the degree and direction of congruence between explicit and implicit self-esteem. However, difference scores are rife with methodological problems, often resulting in misleading conclusions. Edwards (2001) outlined many of the problems with difference-scores, like reliability issues, reduced effect sizes and an increased risk of errors. One such problem is the attenuation of error in difference scores; the reliability of each component part is attenuated by the correlation between them (Cohen & Cohen 1983), which means that the reliability of a difference score measure is usually less than the reliability of each component.

One primary concern associated with difference scores is their tendency to reduce true score variance (May and Hittner, 2003). True score variance reflects the variability in scores attributable to actual differences in ability or attitudes, while error variance encompasses variability introduced by random factors, such as measurement error. When calculating a difference score like d600, the true score variance is diminished by an amount corresponding to the square of the correlation between the pretest and posttest scores. This reduction occurs because the difference score is sensitive only to changes in ability or attitudes that are shared between the pretest and posttest measurements. Consequently, the reliance on the common variance may compromise the reliability of difference scores compared to the individual reliability of pretest and posttest scores. In addition, as in the above mentioned studies, implicit and explicit self-esteem are fundamentally differenced constructs and the method of their measurement necessarily imply that they are conceptualized and operationalized in different ways, which affects reliability and content validity, especially when these two constructs are directly compared to each other. Self-report scales of Self-Esteem often show a higher test-retest correlation and a lower situational variability compared to implicit measures, which means that explicit measures tend to be more stable over time compared to implicit measures. (Dentale et al., 2019). The IAT variable is susceptible to errors due to its reliance on reaction times, which can be affected by various factors like fatigue, concentration, and motivation. In contrast, the Rosenberg Self-esteem is a questionnaire specifically crafted to yield a dependable and valid assessment of explicit self-esteem. Dentale et al.’s (2019) findings imply the presence of residual noise in the discrepancy score between explicit and implicit self-esteem. This is attributed to the inherent error-proneness of the IAT variable, whereas the Rosenberg Self-esteem, being an explicit measurement, is not prone to such errors.

Perhaps most importantly, the use of difference scores implies an intuitive, but specific, and restrictive hypothesis, namely that the components of the difference have equal but opposite effects (Edwards, 1994). This model might be a poor reflection of the underlying reality, and several alternative models for difference scores that are often better able to describe the relationship between components have been proposed (Humberg et al., 2018; Mota et al., 2020). One important problem with the model underlying the classic use of difference scores is that it obfuscates the relative weight of each component in relation to a criterion variable, i.e. it hides whether both factors pull equal weight, or if one variable is doing most of the legwork. For example, a significant association between a difference score (such as the difference between explicit and implicit self-esteem) and an outcome measure (such as depression) may reflect only a main effect of one of the core variables (explicit self-esteem or implicit self-esteem) and not per se the effect of the degree of discrepancy. In the extreme case that one of these variables has zero effect on the criterion, the use of a difference score amounts to little more than centering the main effect to zero with a random residual deviation, turning the difference score into a worse predictor than the main effect would have been in itself.

To address methodological challenges, alternative approaches have been proposed in the literature. Edwards (1993) drew inspiration from surface response methodology and advocated for the utilization of polynomial regression equations to explore the intricate interplay between component parts in a difference score concerning a criterion variable. Edwards’s (1993) method involves eschewing the use of difference scores altogether and constructing polynomial regression models with each component as a separate predictor. This enables the assessment of the relative importance of each component. The incorporation of polynomials facilitates the depiction of a curved surface, describing the relationship between components and the criterion. This approach allows for the examination of more complex relationships that are beyond the scope of traditional difference scores.

In a subsequent work, Edwards (1994) delves into two specific congruency hypotheses within the realm of RSA, articulated as the X = Y and X = –Y lines. The X = Y line epitomizes perfect congruence, signifying an ideal alignment between two constructs. Conversely, the X = –Y line represents the extreme of incongruence, where the two constructs are diametrically opposed, leading to maximal incongruence and potential negative outcomes such as job dissatisfaction, stress, and turnover. These extremes serve as crucial benchmarks for testing congruency-related hypotheses directly through RSA.

For instance, if the observed congruence score aligns closer to the X = Y line than the X = –Y line, it implies a positive relationship between congruence and the outcome. In this scenario, an increase in congruence corresponds to an increased likelihood of a positive outcome. Conversely, if the observed congruence score is closer to the X = –Y line than the X = Y line, it suggests a negative relationship between congruence and the outcome. Here, an increase in congruence is associated with a decrease in the expected outcome.

Rather than solely examining the effect of the difference on a criterion, the combination of polynomial regression with response surface methods enables the testing of more nuanced hypotheses. By analyzing the curvature along the X = Y and X = –Y lines, one can empirically test (instead of assuming) classic difference score hypotheses, such as whether (a) each component part carries equal weight, (b) the criterion is optimized at a constant value when the two components are equal, and (c) the criterion changes systematically when deviating from these congruency lines. Therefore, the primary advantages of polynomial regression lie in its ability to test a priori hypotheses derived from theory, its resistance to attenuated reliability, and its capacity to determine the contribution of each component part in predicting a criterion.

In short, these equations get around many problems that come along with the use of difference scores and at the same time allow to draw valid conclusions about congruence effects. Nevertheless, polynomial regression in lieu of difference scores is not common practice, probably because the results of such equations are not so easy to interpret. The polynomial models and their constraints can be quite arcane to anyone not already familiar with advanced calculus, and disentangling the unique and joint effects expressed by main effects, squared main effects and interaction terms is challenging. It is more practical to plot the polynomial model as a 3-D plot, a response surface, either accompanied by a contour plot or by using colours to express the height of the surface. In addition, polynomial regression is not without limitations. This approach is only useful when the differences between variables are the predictors, and a different approach is needed when discrepancy is the dependent variable (Edwards, 1995). Polynomial regression is susceptible to outliers given that it squares the main effects, which has an exponential effect on outliers, requiring rigorous data screening.

There are two research questions here that need to be addressed. The first question is: Should implicit and explicit self-esteem be merged in one variable (difference scores) or not? The second question is: Should the effect of self-esteem (either difference scores, either the two original variables) be modelled as polynomial? The main goal of this study is to demonstrate polynomial regression as an approach that allows to investigate the associations between explicit self-esteem, implicit self-esteem and respectively depression, anxiety and aggression through specific hypotheses derived from fit patterns. We will provide an empirical example to demonstrate the differences between the approaches, and how to interpret their results. In the first approach, numerical congruence as expressed in a difference score between explicit and implicit self-esteem will be used. In the second approach, polynomial regression equations will be applied in order to analyse fit patterns based on the relative importance of each predictor, but also the shape of the curved surface, and meaningful axes along these shapes. The overall aim of this paper is to illustrate the difference in results and conclusions, depending on the approach that is used, and why it is important to move away from numerical congruence (i.e. difference scores) as predictors, especially in research concerning the congruence between explicit and implicit constructs. This paper also aims to make knowledge more accessible by providing a step by step explanation of both methods, illustrating differences between them, and making materials openly available for other researchers.

In the method section the empirical example will first be introduced and explained. In the Analysis subsection a more detailed explanation will be given regarding Polynomial Regression, such as the specific hypotheses that can be tested, how to interpret the 3D representation of the model, the most important axes in the model and their interpretation. The results section will demonstrate the outcomes from the two approaches and their differences will be discussed in the discussion section. The purpose of this paper was to be easy to understand for researchers who have little background in advanced calculus or linear algebra. For this reason both the methods and the introduction include little to no equations and proofs, unless absolutely necessary. In the appendix we will offer the R-code that was used to conduct the polynomial regression analyses, as well as a mathematical explanation for those interested in it.

2 Disclosure

The study is not preregistered. This study involved an analysis of existing data rather than new data collection. All simulations and other analyses conducted are reported. Participation in the study was voluntary and participants gave digital informed consent after being fully informed about the study, and having had the opportunity to have any questions answered. The study was carried out in accordance with The Code of Ethics of the World Medical Association (Declaration of Helsinki) for medical research involving humans. Information regarding all procedures applied and all measures assessed in this study as well as the data and the data analysis scripts needed to reproduce the results are openly accessible: https://osf.io/q7c5f/.

3 Method

3.3 Data analyses

Four different approaches were used to demonstrate the accordance and discordance in the results and conclusions between the different approaches. Regardless of the approach, in order to examine the associations between implicit and explicit self-esteem on the one hand, and depressive symptoms, anxiety and aggression on the other, hierarchical multiple regression analyses were performed. Prior to analyses the implicit and explicit self-esteem scores were standardized to help their interpretation. The four analyses will be outlined in the next subsections.

Analysis 1: Continuous Difference Scores

The self-esteem discrepancy was first computed as the difference between the standardized scores of explicit and implicit self-esteem. The difference score was calculated by subtracting the implicit (IAT) score from the standardized explicit (Rosenberg SE) score, and entered into a linear regression model. The regression model can be expressed as:

1

$\widehat{Y}\text{=}{\beta }_0\text{+}{\beta }_{1}X\text{+}ϵ$

Here, Y ̂ is the predicted dependent variable, β₀ is the intercept, β₁ is the coefficient for the self-esteem discrepancy, X, and ϵ represents the error term.

In a linear difference score model it is possible to add polynomials such as quadratic terms to model more complex patterns of difference scores. An example quadratic relationship can be expressed as:

2

$\widehat{Y}\text{=}{\beta }_0\text{+}{\beta }_{1}X \text{+}{\beta }_2{X}^2\text{+}ϵ$

Where β₂ is the coefficient for the square of the difference score X², which models the effect of the difference score on the dependent variable as a parabola.

In the continuous approach it is difficult to test specific hypotheses regarding the exact effect of the type of difference (negative or positive) on the dependent variable. Rather, the direction and strength of the association are used to ascertain whether a positive or negative difference score is positively related to the outcome variables and to what degree.

Analysis 2: Categorical Difference Scores Some

researchers believe that the problems with continuous difference scores can be solved by reducing the measurement-level of the difference score by creating subgroups based on the congruence between two component measures (e.g. Mersman & Donaldson, 2000). This belief is wrong (e.g., Edwards, 2001) and subgroup Polynomial regression’s foremost goal is to test hypotheses derived from congruenceing adds new problems to the difference score measure without alleviating the old ones, such as the attenuation of error in difference scores, the proneness of the implicit measurement to contextual factors, residual noise in the difference scores and the obfuscating of the relative weight of each component in relation to a criterion variable. The main problem with subgrouping is that it still remains a difference score, albeit one which now carries less information and reduced explained variance. To demonstrate the subgrouping approach the difference score between explicit and implicit self-esteem was split into three groups based on Kernis (2003) classification. Respondents with difference scores 0.5 SD below zero were classified as fragile (combination of low implicit self-esteem with high explicit self-esteem) and respondents with difference scores 0.5 SD above zero were classified as damaged (high implicit self-esteem combined with low explicit self-esteem). The remaining respondents were classified as congruent in explicit and implicit self-esteem. These three groups were subsequently split into two dummy variables representing fragile and damaged self-esteem, in order to compare their regression weights to the congruent group. These dummies were entered into a linear regression model. In the categorical approach it is slightly easier than in the continuous case to test specific hypotheses regarding the effect of the type of difference (negative or positive) on the dependent variable. The regression equation for this model is expressed as:

3

$\widehat{Y}\text{=}{\beta }_0\text{+}{\beta }_1{X}_{\mathrm{damaged}} \text{+} {\beta }_2{X}_{\mathrm{fragile}}\text{+}ϵ$

Here, Y ̂ is the predicted dependent variable, β₀ is the intercept, β₁ and β₂ are the coefficients for the dummy variables X_damaged and X_fragile respectively, and ϵ represents the error term. In this model the intercept takes on the value of the mean of the ‘congruent’ category, and the coefficient for the dummy variables denote the deviation from this mean.

One critique of the continuous approach can be that even though the explained variance is retained by using the high measurement-level, the interpretation of the regression weights amounts to a subgrouping approach without a congruent group, i.e. everyone is classified as incongruent in some way. In contrast, the categorical approach involves a user-defined range of difference scores that are deemed congruent scores, with tails in the distributions of the difference score serving as incongruent scores. Incongruency in the continuous and categorical approach in effect are modelled differently, as the categorical approach foregoes the less extreme deviances from perfect congruity. The categorical approach, by categorizing a range of scores as congruent and focusing on relatively extreme departures, introduces a trade-off between increased power for detecting extreme incongruencies and potential loss of sensitivity to smaller departures from congruence.

Just like in the continuous approach, the direction and strength of the association of the dummy variables are used to ascertain whether a positive or negative difference score is positively related to the outcome variables and to what degree. The regression weights are the mean differences of the subgroups relative to the intercept, which is the mean of the congruent group.

Analysis 3: Mixed Categorical and Continuous

In addition to the classical difference score approaches outlined above we will demonstrate the mixed approach followed by Creemers et al. (2012), who propose a method that splits a continuous difference score into two variables: (1) The absolute difference between the standardized score on implicit and explicit self-esteem, to indicated the size of discrepancy, i.e. a continuous difference score with only positive values; and (2) a dichotomous variable that indicates the direction of the discrepancy between implicit and explicit self-esteem (implicit < explicit or implicit > explicit). Their method makes no explicit allowance for respondents with congruent implicit and explicit scores. The size of the discrepancy (continuous) and the direction of the discrepancy (dichotomous) were entered in a hierarchical regression in step 1 and their interaction in step 2. The regression model is expressed as:

4

$\widehat{Y}\text{=}{\beta }_0\text{+}{\beta }_1{X}_{\mathrm{fragile}} \text{+} {\beta }_2{X}_2\text{+}{\beta }_3{X}_{\mathrm{fragile}}{X}_2 \text{+} ϵ$

Here, Y ̂ is the predicted dependent variable, β₀ is the intercept, β₁ is the coefficient for the dummy variable X_fragile, which functions as a direction variable which is attained through dummy coding: a value of 0 for damaged self-esteem and 1 for fragile self-esteem. β₂ is the coefficient for continuous dependent variable X₂, which represents the size of the difference. β₃X_fragile X₂ represents the interaction between the direction and size, and ϵ represents the error term. In this model the intercept takes on the value of the mean of the ‘damaged’ category, and the coefficient for the dummy variable denotes the deviation from this mean.

Analysis 4: Response Surface Analysis and Polynomial Regression

RSA (Response Surface Analysis) is a statistical technique used to study relationships between multiple independent variables and a response variable. RSA often involves modeling the response surface using mathematical models, and these models may include polynomial terms to capture non-linearities. Polynomial Regression is a specific type of regression analysis focused on modeling relationships using polynomial functions.

Difference score theories have some key similarities to this original goal of RSA. The theory behind difference scores is that congruent values on the variables (i.e., no difference between explicit and implicit) optimize a response. For example, one could hypothesize that individuals are most happy, or are least depressed when implicit and explicit self-esteem are perfectly congruent. The problem, which RSA solves, is that the theory underlying difference scores is severely restrictive. Most important among its many restrictions, is the assumption that perfect congruence optimizes the response. ‘Optimize’ here implies a global optimum (the best of all possible solutions, e.g., highest possible happiness, or lowest possible sadness) rather than a local optimum (the best in a particular neighbourhood of solutions). By visualizing the 3D response surface the researcher is able to visually determine the way in which the implicit and explicit measures affect the response variable, if, and if so where, it optimizes the response. To make these determinations it is important to be familiar with the following key terms in RSA.

Stationary Point. The global optimum, either the overall minimum or maximum, of the surface is called the stationary point. It is the point where the slope of the surface (Z) (its derivative) is zero.

Principal Axes. These are the axes on which the surface is said to rotate. Simply put: the lines of steepest and least steep ascend. The principal axes are the perpendicular lines that intersect at the stationary point and where at least one of the principal axes optimizes the amount of explained variance along its axis. The other principal axis subsequently, sometimes, minimizes the explained variance. If the surface is convex (peaks downward) the greatest ascend occurs along the first principal axis and the least occurs along the second principal axis. The reverse is true for concave surfaces (peaks upwards). In the case of a saddle point, both axes indicate paths of greatest ascend.

X = Y line. The points where two independent variables have equal value, i.e. perfect congruence. An important part of RSA is to examine the size of the slope on the surface along either the principal axes or the X = Y line and how close the principal axes and the X = Y line are to each other. For example, when perfect congruence between explicit and implicit self-esteem minimizes depression, and any incongruence adversely effects depression, the X = Y line and the principal axis would be identical. Finding the principal axes in a response surface can be difficult if one is unfamiliar with eigenbases. In the case of quadratic polynomial regression, which will be discussed in this paper, the easiest way to determine them is given in Edwards (1993).

RSA goes beyond linear regression. In linear regression, the mean of the dependent variable is modelled to be conditional on a linear set of independent variables. In its most basic form it constitutes an addition of main effects, which takes the form of:

5

$\widehat{ Y}\text{=}{\beta }_0\text{+}{\beta }_1{X}_1\text{+}{\beta }_2{X}_2\cdots \text{+}{\beta }_n{X}_n\text{+}ϵ$

Y being the value of the dependent variable, X representing the set of independent variables, Beta representing the regression weight with Beta-zero representing the intercept and epsilon to indicate normally distributed ~ N(0, σ²).

Sometimes linear regression contains a combination of main- and interaction effects. The addition of an interaction-term allows for the modelling of nonparallel lines when used with a categorical predictor, or for rotated flat surfaces when used on solely numerical predictors. Interaction implies that two variables have a multiplicative effect, which is why it is added to a regression model in the form of the multiplication between two variables, with a regression weight to determine the strength and direction of the multiplier. For example:

6

$\widehat{Y}\text{=}{\beta }_0\text{+}{\beta }_1{X}_1\text{+}{\beta }_2{X}_2\text{+}{\beta }_3{X}_1{X}_2\text{+}ϵ\hspace{0.17em}$

In polynomial regression, this regression model is expanded to allow for curved surfaces, which adds a new layer of complex ways in which variables are allowed to interact. The most basic polynomial regression is one in which the squared main effects are added to a model. It is possible to add higher-order polynomials to a model, and just like with standard linear regression the user is free to choose the form of the regression equation. In response surface methodology however the most parsimonious and widely used form is a variation of model (2):

7

$\widehat{Y}\text{=}{\beta }_0\text{+}{\beta }_1{X}_1\text{+}{\beta }_2{X}_2\text{+}{\beta }_3{X}_1^2\text{+}{\beta }_4{X}_1{X}_2\text{+}{\beta }_5{X}_2^2\text{+}ϵ$

Response surface analysis (RSA) was originally developed by Box and Wilson (1951) to determine optimal conditions for experimentation in a chemical investigation. Contrary to what social scientists usually call response, i.e. the score of a respondent on some variable, the response in response surface methodology refers to the set predicted scores on the criterion variable. The idea behind this was to optimize (to find the maximum or minimum) a set of responses, such as yield, purity or cost of the product, based on factors that influenced these responses, such as temperature and pressure. The advantage of modelling the interplay of the influential factors on the responses as a curved surface expressed by a simplified regression equation is that the region where changes in the factors would have the most pronounced effect can be efficiently determined.

In Figure 1, six visualizations are employed to elucidate key points of interest and patterns, arranged from left-to-right and top-to-bottom.

Figure 1

Figure a depicts a stationary point representing the optimum, where the values of implicit and explicit self-esteem converge to yield the lowest predicted anxiety.

Moving to Figure b, the congruence line (X = Y) and the incongruence line (X = –Y) are presented. In a hypothesis positing that congruency between predictors optimizes a given variable, one would anticipate the optimal anxiety value aligning with the X = Y line, with the least favorable values occurring along its inverse, the X = –Y line.

Figure c illustrates a hypothetical scenario of perfect congruence, where anxiety consistently is lowest when values of implicit and explicit self-esteem are congruent, accompanied by a rapid increase in anxiety when incongruence arises.

Figure d visualizes the relationship between the first principal axis and the steepest ascent, where movement along this axis at the base of the graph corresponds to a substantial change in anxiety. Figure e demonstrates how the second principal axis aligns with the second-largest change in anxiety, which is more gradual than the change along the first principal axis.

The final Figure f showcases a pattern known as a saddle point, challenging to interpret due to its stationary nature (it has a stationary point at the center of its ‘saddle’) without a clear optimum (lowest or highest value). Saddle points denote complex relationships; for instance, in Figure f, steepest increases in anxiety occur when implicit self-esteem is high and explicit self-esteem is low. Conversely, the direction of steepest descent points to low implicit self-esteem and high explicit self-esteem, suggesting that individuals with high implicit self-esteem and low explicit self-esteem are more prone to anxiety, while those with low implicit self-esteem and high explicit self-esteem are less likely to experience anxiety.

To demonstrate an RSA approach to difference score analysis, polynomial regression analysis was performed in R (version 3.6.1) with respectively depression, anxiety and aggression as dependent variables. Hierarchically explicit self-esteem, implicit self-esteem were added to the model. In step 1 the main effects were entered. In step 2 the quadratic terms were added, and the interaction between the main effects were added in step 3. Even though standardizing the predictors avoids multicollinearity in the interaction term, to avoid further multicollinearity between polynomials we calculated orthogonal polynomials with the poly() option in the lm() function in base-R (v4.2.2; R core team, 2022), which means that these parameters were scaled so that the second-order polynomial is orthogonal (i.e., uncorrelated) to the first-order polynomial. This has the advantage that it can be tested whether adding a second-order polynomial significantly improves the regression over the lower order.

4 Results

Prior to analysis, scores for depression, anxiety, aggression, explicit and implicit self-esteem were examined for accuracy of data entry, distributions, univariate and multivariate outliers. Two cases with extremely high z-scores (z > 3.29) on depression and one case with an extremely low z-score (z < –3.29) on explicit self-esteem were found to be univariate outliers. No cases were identified as multivariate outliers based on their Mahalanobis distance (p < .001). All three univariate outliers were winsorized, meaning that they were replaced with the first non-outlying value above or below it. Thirteen missing values in depression and anxiety, eleven in aggression, and nine in explicit self-esteem were found and imputed with predictive mean matching with the mice package (Van Buuren, 2011) (with ten mutations) based on the following variables: depression, anxiety, aggression, explicit self-esteem, and implicit self-esteem (D600 all, D600 practice, D600 trial).

The completed data from the research variables are summarized in Tables 1 and 2. Shapiro-Wilk’s tests for normality of distributions indicated that depression, anxiety, and aggression deviated from the normal distribution and were positively skewed (p < .001). Correlations between implicit self-esteem and other measures were close to zero and not significant (p > .05). Correlations between explicit self-esteem and other measures were negative and ranged from moderate (Depression r = –0.68, p < .01; Anxiety r = –0.60, p < 0.1) to weak (Aggression r = –0.27, p < 0.1).

Analysis 1: Continuous Difference Scores

Three hierarchical multiple regressions were used to test the effect of the continuous difference score (implicit – explicit self-esteem) on standardized scores for depression, anxiety, and aggression. The results of the regression analyses are summarized in Table 3. The continuous discrepancy score between implicit and explicit self-esteem showed a positive and significant association with depression and anxiety. The difference score explained about 22.4% in depression, 22.6%, in anxiety, but only 2.6% in aggression. The discrepancy between implicit and explicit self-esteem, expressed as a continuous difference score indicates that depressive symptoms and anxiety increase change by circa one-third of a standard deviation when the discrepancy between implicit and explicit self-esteem increases by a whole point.

In a second step of the regression analysis, we added a quadratic term for the difference score. These quadratic terms were found to be statistically significant (p < 0.01), indicating curvilinear relationships. The models with quadratics terms showed markedly higher R², namely 29.3% in depression, 29.9% in anxiety, and 5.8% in aggression.

For standardized depression, the positive coefficient of 3.045 suggests that as the difference score increases, reflecting a shift from lower explicit to higher implicit self-esteem, the rate of change in standardized depression initially rises until reaching a minimum point at –0.90. Subsequently, the rate of change decreases.

Similarly, in the case of standardized anxiety, the quadratic term yielded a positive coefficient of 3.146, signifying an analogous curvilinear relationship with a minimum point at –0.87. This suggests that the transition from lower to higher implicit self-esteem is associated with an initial increase in anxiety, peaking at –0.87 on the difference score.

Additionally, for standardized aggression, the positive coefficient of 2.085 indicates a curvilinear association with the difference score, peaking at –0.44. This implies that the shift from lower to higher implicit self-esteem is linked to an initial increase in aggression, reaching its minimum point at –0.44 on the difference score.

Analysis 2: Categorical Difference Scores

Three hierarchical multiple regressions were used to test the effect of the categorical difference score (implicit – explicit self-esteem) on standardized scores for depression, anxiety, and aggression. The results of the regression analyses are summarized in Table 4. The three types of difference were split into two dummy variables. Kernis’ terms for these dummy variables (Kernis, 2003) were used for labelling the categories as damaged (I > E) and fragile (E > I) so that congruency was expressed as the intercept to test the two types of discrepancy against. Similar to continuous discrepancy, the categorical difference score shows a positive, albeit weaker relationship with, respectively, depression and anxiety. The categorical difference score explained about 16.6% in depression, 18.3% in anxiety, but only 1.9% in aggression.

The effect of the discrepancy between implicit and explicit self-esteem, expressed as three categories, i.e. damaged, congruent and fragile, differs from the effect of the continuous difference score. The overall pattern of a positive linear relationship is maintained, but indicates that the effect is mostly, if not only due to the positive effect of damaged self-esteem on depressive symptoms (β = 0.80; p < 0.001) and anxiety (β = 0.86; p < 0.001). Fragile self-esteem had no significant effect on the three outcome variables. Neither fragile nor damaged self-esteem had a significant effect on aggression. A damaged self-esteem (high implicit self-esteem combined with low explicit self-esteem) increased mean depression and anxiety by around two thirds of a standard deviation and fragile self-esteem (combination of low implicit self-esteem with high explicit self-esteem) decreased mean depression by almost a quarter of a standard deviation. The results suggest that a damaged self-esteem might be associated with a higher mean depression and anxiety, but that the mean is still below average, and that the effect of the discrepancy is largely linear, meaning that congruent self-esteem is associated with both higher and lower depression and anxiety than incongruent self-esteem, with fragile self-esteem (E > I) seeming most beneficial.

Analysis 3: Mixed Categorical and Continuous Scores

The approach demonstrated by Creemers et al. (2012) was tested by using three hierarchical multiple regressions on standardized scores for depression, anxiety, and aggression. The absolute value of the difference between implicit and explicit self-esteem (I-E) and the direction of the difference (negative vs. positive) were added to the models. The results of the regression analyses are summarized in Table 5. Comparable to the categorical approach, this approach labels the continuous discrepancy as either damaged (difference is positive, i.e., I > E) or fragile (difference is negative, i.e., E > I), but instead of a congruent group it defines two incongruent groups who differ in strength of incongruency (with a strength of zero for a perfect congruency). So the definitions differ a little from the categorical and continuous approach, and tries to offer what Creemers (2012) considers the best of both models. Without an interaction between size and direction (models 1, 3, and 5), the direction of the difference was only significantly related to depression and anxiety. None of these effects remained significant after adding the interaction between size and discrepancy. The interaction between size and direction was significant for depression and anxiety, but not for aggression. The positive interaction term (Table 5, Model 2, 4, and 6) coupled with the negative main effect of the direction of the discrepancy indicates that the effect of the discrepancy on depressive symptoms and anxiety is stronger in the damaged group, than in the fragile group. The mixed difference score explained about 29,3% in depression, 30,3% in anxiety, and 5,8% in aggression.

Analysis 4: Polynomial Regression

The results of the regressions are summarized in Table 6 and visualized in Figures 2, 3, and 4.

Table 6

Polynomial regression of orthogonal factors based on the standardized predictors explicit and implicit self-esteem as predictors of standardized depression, anxiety and aggression. For each dependent variable three models are shown; the first contains only main effects, the second adds polynomial terms, and the final, third model adds an interaction term.

	DEPENDENT VARIABLE:
	DEPRESSION			ANXIETY			AGGRESSION
	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)
Explicit self-esteem	–0.660*** (0.065)	–7.660*** (0.741)	–7.638*** (0.731)	–0.591*** (0.070)	–6.625*** (0.799)	–6.863*** (0.785)	–0.266** (0.083)	–3.169*** (0.974)	–3.149*** (0.969)
Implicit self-esteem	0.019 (0.065)	0.167 (0.739)	0.310 (0.733)	0.089 (0.070)	1.254 (0.801)	1.173 (0.787)	–0.037 (0.084)	–0.426 (0.972)	–0.291 (0.971)
Explicit self-esteem2		2.149*** (0.739)	2.232*** (0.731)		1.634 (0.799)	1.854** (0.785)		–0.009 (0.972)	0.069 (0.968)
Implicit self-esteem2		0.387	0.420		1.254	0.711		1.297	1.328
Explicit self-esteem * Implicit self-esteem			–0.135** (0.064)			–0.170** (0.069)			–0.127 (0.085)
Intercept	0.000 (0.065)	0.000 (0.064)	0.003 (0.063)	0.000 (0.070)	0.001 (0.069)	0.004 (0.067)	–0.000 (0.084)	–0.000 (0.084)	0.003 (0.083)
R2	0.435	0.471	0.488	0.355	0.380	0.410	0.073	0.085	0.101
Adjusted R2	0.426	0.454	0.468	0.345	0.366	0.387	0.059	0.057	0.066

[i] Note: *p < 0.1; **p < 0.05; ***p < 0.01.

Figure 2

Figure 3

Figure 4

The relationship between explicit, and implicit self-esteem, and depression, respectively anxiety, and aggression is complex and non-linear. The numbers in Table 6 provide quantitative insights into this relationship, while Figure 2, respectively 3, and 4 visualizes it graphically. Figure 2 displays the regression equation for the relationship between explicit and implicit self-esteem on depression. The first principal axis can be understood as the path where the dependent variable changes the most, has its steepest ascend/descend. In Figure 2 the first principle axis is denoted by a thick black line at the floor of the graph. The shape of the surface is convex, with the steepest ascens running parallel to the x-axis, indicating that depression increases at an increasing rate when explicit self-esteem decreases, but is almost unaffected by changes in implicit self-esteem. The stationary point is located at X = 1.78 Y = –0.22, which indicates that depressive symptoms are lowest when explicit self-esteem is about one and a three quarters standard deviations above average, and increases slightly when explicit self-esteem increases beyond that point. Overall the surface indicates three effects. First, depressive symptoms decrease sharply when explicit self-esteem increases, but begins to increase again when explicit self-esteem exceeds about 1.75 SD. Second, depressive symptoms are relatively unaffected by implicit self-esteem, and are lowest when explicit self-esteem is high, and implicit self-esteem is around zero (the mean), and increase only lightly (and not-significantly) whenever implicit self-esteem is either high or low. Third, depressive symptoms are highest when explicit self-esteem is very low, and implicit self-esteem is very high (damaged), but close to the stationary point (lowest point) when explicit self-esteem is very high and implicit self-esteem is very low (fragile). Taken together with the regression coefficients in Table 6 the effects of the discrepancy between implicit and explicit self-esteem on depression seem almost solely driven by the significant main polynomial effect of explicit self-esteem, contrary to the conclusions of the earlier approaches.

Figure 2 visualizes the relationship between explicit, implicit self-esteem, and depression as response surfaces. The response surface for the model with only main effects is a plane, suggesting a linear relationship. As polynomial terms are introduced, the response surfaces become curved, indicating a non-linear relationship. The interaction term further enhances the curvature, representing the complexity of the relationship.

Figure 3 displays the regression equation for the relationship between explicit and implicit self-esteem and anxiety. The shape was convex with its stationary point located at X = 1.82 Y = –0.61, which indicates that anxiety was lowest when explicit self-esteem was just close to two standard deviations above average, and increases slightly when explicit self-esteem increases beyond that point. The slope of the first principal axis (steepest ascent) (p₁₁ = –0.07) is close to zero, indicating that anxiety increases at an increasing rate when explicit self-esteem decreases, but is almost unaffected by changes in implicit self-esteem. The slope of the second principal axis was greater than 1 (p₂₁ = 13.48) almost parallel to the Y-axis.

Overall the surface indicates three effects. First, anxiety decreases sharply when explicit self-esteem increases, but begins to increase again when explicit self-esteem exceeds about 1.8 SD. Second, anxiety is not-significantly affected by implicit self-esteem, and is lowest when explicit self-esteem is high, and implicit self-esteem is low, and increases only lightly (and not-significantly) whenever implicit self-esteem is either high or low. Third, anxiety is highest when explicit self-esteem is very low, and implicit self-esteem is very high (damaged), but close to the stationary point (lowest point) when explicit self-esteem is very high and implicit self-esteem is very low (fragile). Taken together with the regression coefficients in Table 6, the effects of the discrepancy between implicit and explicit self-esteem on anxiety seem primarily driven by the significant main polynomial effect of explicit self-esteem, which is contrary to the conclusions of the earlier approaches. Additionally, severely damaged self-esteem, i.e. a combination of extremely high implicit self-esteem and extremely low explicit self-esteem does seem to result in much higher anxiety than when there is a congruent extremely low explicit and explicit self-esteem, indicating that a discrepancy between explicit and implicit self-esteem has a stronger effect when explicit self-esteem is low, than when explicit self-esteem is high.

Figure 4 displays the regression equation for the relationship between explicit and implicit self-esteem on aggression. The only significant predictor of aggression in this model is explicit self-esteem. Overall the surface indicates two effects. First, aggression decreases very slightly and linearly when explicit self-esteem increases. Second, aggression is not significantly affected by implicit self-esteem, and is zero when explicit self-esteem is close to zero (average), and increases when it deviates negatively from zero. When considering the regression coefficients presented in Table 6 alongside the findings related to the incongruence between implicit and explicit self-esteem, notable parallels emerge with the results observed in method 1. In the latter, quadratic terms were introduced to the continuous difference scores, revealing significant relationships across all three dependent variables. However, the Response Surface Analysis introduces a nuanced perspective, suggesting that the continuous difference effect identified in method 1 is primarily explained by a significant main effect of explicit self-esteem, rather than implicit self-esteem or any interaction between the two.

5 Discussion

Competing Interests

The authors have no competing interests to declare.