From Joule to Carnot: ex-post electricity savings achieved through residential air-to-air heat pumps

Data cleaning and preparation – step 1

Data processing (step 1 of the analysis, Figure 1) involved filtering out daily consumption values exceeding an empirical value of 200 kWh/day, which were primarily attributed to smart meter errors (e.g., identical high readings over 2 to 3 consecutive days) and removing the bottom 20% of consumption values (an empirical threshold) associated with dwelling vacancy (as well as missing values) (Figure 2). This filtering step noticeably improved the model coefficients of determination (R²) during daily consumption fitting.

Figure 2

The explanatory variables extracted from the questionnaire include: type of housing (SFH, MFH), use of air conditioning (AC-yes, AC-no), PV equipment installed (PV-yes, PV-no), floor area (m²), number of inhabitants, Heating Degree Days (HDD), type of AAHP (multi or mono-split), type of domestic hot water (DHW) (electric, thermodynamic, solar, other), renovation work (retrofit-yes, retrofit-no).

For the clustering, we used the declared use of space heating system (as a declared percentage of use) in the pre- and post-situation: pre-k or post-k with k = (AAHP, REH, boiler, fireplace, stove).

For the energy savings model, we also used the following variable for pre/post situations: the main heating system and the space heating use (mixed, single). We added the daily electricity consumption (DC) during periods of cold outdoor temperatures (T ∈ [–1°C, 1°C]) to consider the electrical intensity of space heating linked to the winter cold.

Local normal climate – step 2

To estimate the local normal climate (step 2 of the analysis, Figure 1), we use the average daily temperature over the studied period. This daily temperature record is then smoothed using a second-harmonic Fourier series model (Duchon & Hale 2012):

where i ∈ {1, …,365} represents the calendar day.

Because spatial data were anonymized at the departmental level (French administrative division), site-specific normals were computed using three years of local weather data.

A three-year period is short compared to conventional 30-year climate records. However, a study shows that shorter periods improve representativeness and provide better balance between bias and variability to account for climate change (Ye et al. 2026).

Multivariate adaptive regression spline modelling – step 3–4

In Step 3 of the analysis (Figure 1), a non-parametric regression model based on multivariate adaptive regression splines (Friedman, 1991) is used, which combines recursive partitioning and spline fitting. We used the enhanced adaptive regression through hinges (EARTH) R package (Milborrow 2023) to model the daily consumption (DC) as a function of the outdoor temperature (Figure 3). The EARTH model performs automatic selection of predictors (explanatory variables) and the number of knots. The modelled daily consumption DC is based on the following equation 2.

where

with

• DC: daily electric consumption (kWh/day) on day d.

• T_i: average daily outdoor temperatures (°C) for the day d and the 2 previous days (i = 1 – 3) included to account for building thermal inertia and temporal dependence.

• W_j: dummy variable indicating the day of the week (j = 1, …, 6), with the omitted category serving as the reference day.

• c_i: estimated knot locations of the hinge functions.

• α_i⁽⁺⁾, α_i^(–), β_i, β₀: estimated model coefficients.

• ɛ_d: residual error term.

Figure 3

Separate models were fitted for each dwelling before and after AAHP installation (176 models in total). As demonstrated in Osso et al. (2024), algorithm selection has a minimal effect on net annual energy estimates relative to temperature-gradient fitting. Therefore, no other model types were tested.

Electricity savings evaluation – step 5

In steps 5 of the analysis, the assessed ES_n (in kWh/year) for each n dwelling is calculated (equation 3) as the difference of the sum of daily consumption modelled at local normal climate $(DC\widehat{{}_k^{\mathit{\text{normal}}}})$ in the pre- (REH) and post- (AAHP) situations for a one-year period (i = 1 to 365).

Clustering – step 6

The household clustering based on the variables of the questionnaire is performed in two steps: principal component analysis (PCA) followed by classification. PCA helps to identify the variables that produce the most homogeneous clusters. Firstly, the PCA searches for the subspace that best represents the diversity of individuals and that maximises the variance of the projected points and based on linear relationships between variables. The proximities between variables are interpreted in terms of correlation; the proximities between individuals are interpreted in terms of overall similarities in the observed values (Lebart, Morineau & Piron 1995; Husson, Josse & Pagès 2010).

We used the FAMD (factor analysis for mixed data) function which is a combination of PCA for quantitative variables and multiple correspondence analysis (MCA) for qualitative variables of the FactoMineR package (Lê, Josse, & Husson 2008).

In a second stage, to identify homogeneous individual profiles, agglomerative hierarchical clustering was performed on the coordinates of individuals from the PCA using the hierarchical clustering on principal components (HCPC) method, using the HCPC function of the FactoMineR package (Lê, Josse, & Husson 2008).

Generalized additive models for location, scale and shape – step 6

In step 6 of the analysis (Figure 1), we also modelled the ES_n in kWh/year, and not the ESU_n in kWh/(m².year), with the GAMLSS methodology to avoid the floor area on both sides of the equation. The objective is to quantify the impact of explanatory variables on electricity savings fluctuations.

Generalized additive models for location scale and shape (GAMLSS) are distributional regression models to overcome some limitations of generalized linear models especially when the response variable exhibits heterogeneity. They allow us to model not only the mean of a distribution, but also its variance, skewness, and kurtosis (Stasinopoulos & Rigby 2007). GAMLSS models use a distributional regression approach in which all parameters of the conditional distribution of the response variable are modelled using explanatory variables with four parameters:

• Mean (µ). Models the mean of the response variable (location).

• Variance (σ). Characterizes the dispersion or variability of data (scale).

• Skewness (υ). Measure of a distributional asymmetry with a skewness of zero in case of a symmetric distribution (shape).

• Kurtosis (τ). Measure the heaviness of the tail in a distribution compared to the normal distribution (shape).

For each parameter θ_k (with θ ϵ {μ, σ, υ, τ}), named as the distribution parameters, the GAMLSS model defines a relationship such as eq. 4 (Stasinopoulos & Rigby, 2007).

with: g_k(.) the link function for each parameter, X_k β_k the linear effects, ${\Sigma }_{j=1}^{\mathrm{Jk}}{Z}_{\mathrm{jk}}{\gamma }_{\mathrm{jk}}$ the smoothing function.

Using the R package gamlss (Stasinopoulos & Rigby 2007), the first step to build the model is the selection of the best non-Gaussian distribution with a simple model without explanatory variables using the Generalized Akaike Information Criterion (GAIC). The second step is the selection of explanatory variables and the type of relationship (linear, spline …).

Electric daily consumption modelling

The EARTH models successfully captured temperature dependencies in both pre- and post-retrofit periods. No change point detection method for determining non-routine events was used.

Figure 4 shows an example of daily consumption pre- and post-situation in the observed period based on the average outdoor temperature. It should be noted that, overall, the EARTH model manages to capture the impact of the outdoor temperature despite significant variations in consumption at a fixed temperature.

Figure 4

The goodness-of-fit (R²) by the models is presented in Figure 5 for the 88 households in pre- and post- situations. A higher R² means the model better explains consumption using our available variables. The unexplained variation is treated as the residuals.

Figure 5

The median coefficient of determination (R²) of all the EARTH models is respectively 0.8 and 0.71 for the pre- and post-situation. The lower post-retrofit R² reflects lower thermal sensitivity following AAHP adoption, which increases the relative impact of variations in consumption that act as noise with respect to the model. This suggests that the higher the R², the greater the REH consumption and thus the higher the resulting ES. This assumption will be used in the ES GAMLSS model as a potential explanatory variable (i.e. DC with T ∈ [–1°C; 1°C]). For the clustering we used directly the R² of the EARTH model in the pre- and post-situation as an additional variable (pre-R², post-R²).

Evolution of baseload consumption

In the absence of a control group, to check for baseline consumption shifts unrelated to space heating, annual consumption during mild weather periods (16°C ≤ T ≤ 20°C) was compared from pre- to post- situations (see Figure 4 and Figure 6 as examples).

Figure 6

This consumption remains virtually stable, displaying a negligible mean change of –22 kWh/year (median = –24 kWh/year, Q1 = –90 kWh/year, Q3 = 60 kWh/year).

Air-conditioning assessment

Most respondents reported using AAHPs for summer cooling after installation (AC-yes = 87%). However, some households already had air conditioning, as shown in Figure 6, with a daily consumption gradient for the hottest days in the pre-situation (and without REH).

To evaluate the use of air conditioning associated with the installation of a reversible AAHP, we calculate the difference in annual consumption when the average daily temperature is above 16 °C. It should be noted that this temperature level may seem low, but it should be borne in mind that this is the daily average value. However, daily minimum and maximum temperatures were unavailable, although the daily maximum temperature would have been a more suitable variable. Ademe and Enertech (2025) also notes a daily average temperature of between 15 and 16 °C associated with the start of air conditioning use.

The increase in consumption during hot periods is approximately 1.1 kWh/m² on average (median 0.54 kWh/m²), with significant variations (Figure 7).

Figure 7

If households are divided into two groups, those who report using air conditioning (AC-yes) and those who do not (AC-no), the average new consumption is 1.27 kWh/m² (s.d. = 6.25) and 0.06 kWh/m² (s.d. = 0.88) respectively. This difference is non-significant (Welch Two Sample t-test, p-value = 0.12) (Delacre, Lakens & Leys 2017). This could be explained by the high variance within the “AC-yes” group and bimodal behaviour in the “AC-no” group.

A possible explanation is that some households may well have used their appliances on automatic mode or may have underestimated their air-conditioning consumption. One way to verify this is to look at the temperature gradients for hot days, but the wide variation in consumption during these periods poses a problem. Ideally, an additional summer of data would be required, as this was the first summer during which many users operated air conditioning following the installation of the AAHP. Consequently, user adaptation remains incomplete, and the corresponding utility bill has not yet been received. This represents a potential improvement for future research.

Electricity savings assessment

The ESU assessment based on the difference in annual electricity consumption modelled by the EARTH model averages 14.6 kWh/m² (median = 13.5 kWh/m²) with a significant deviation (Q1 = 4.0 kWh/m² and Q3 = 22.0 kWh/m²) (Figure 8). This ESU corresponds to an absolute value of an average ES of 1,630 kWh/year for a modelled pre-consumption of 9,400 kWh/year for all electric end-uses.

Figure 8

We observed a minimum ESU value of -26 kWh/(m².year) and a maximum value of 73 kWh/(m².year). The large negative values are those related to the use of a gas boiler in the pre-situation.

Households clustering

FAMD reduced survey dimensions with the first five axes accounting for 63.2% of cumulative sample variance (see Annex). The first two dimensions of the FAMD account for 17.6% and 14.8% of data variability respectively (Figure 9 and Table 3 in Annex).

Figure 9

The electricity savings per unit area (ESU in kWh/m²) are positively correlated with the pre- and post-REH uses as declared by the household. The savings are negatively correlated with the gas boiler used and the heating degree day (HDD). The household size and the floor space are lightly correlated with the ESU.

Notably, the performance of the model (i.e. the level of explanation of the variance of the daily consumption, as expressed by the R²) is correlated with the ESU and the use of REH in the pre-situation (correlation coefficient of 0.3). This could be explained by the fact that the higher the temperature gradient during cold days (i.e. daily consumption depends largely on consumption related to space heating), the higher the explanation of variance by the EARTH model (higher R²) and the higher are the potential ESU.

In summary, the first dimension (Dim 1) differentiates housing types (MFH vs. SFH) and biomass use. The second dimension (Dim 2) differentiates retrofit and AAHP use vs. direct electric heating use. Renovated buildings with heat pumps use and PV panels cluster in the upper-left quadrant, opposing non-retrofitted, REH use in the lower-left/bottom. SFHs correlate with wood-based heating use and larger living spaces, positioned opposite MFHs.

The HCPC analysis identified four distinct household clusters (Figure 10) which are mainly characterized by the following variables (see Tables 4 and 5 in Annex):

• Cluster 1 (n=13): MFH with high REH (100% vs. mean: 85.5%) and low floor area (75 m² vs. mean: 115 m²). Mean ESU = 10.1 kWh/m² (median = 4.8 kWh/m²).

• Cluster 2 (n=2): gas boiler use (100% vs. mean: 2.35%) and no REH use, and high HDD (2,908 vs. a mean of 1,938). Mean ESU = –17.6 kWh/m² (median = –17.6 kWh/m²).

• Cluster 3 (n=66): SFH with large floor area (125 m² vs. mean: 115 m²) and higher use of AAHP in the post-situation (87.5% vs. mean: 85.2%). Mean ESU = 16.5 kWh/m² (median = 13.9 kWh/m²).

• Cluster 4 (n=5): high use of biomass space heating in the pre-situation (30.6% vs. mean: 5.2%) and in the post-situation (27.0% vs. mean: 2.6% for wood stove and 10% vs. mean: 1.66% for fireplace). Low AAHP use (42% vs. mean: 85.2%) and higher household size (3.4 vs. mean: 2.4). Mean ESU = 14.6 kWh/m² (median = 14.2 kWh/m²).

Figure 10

We note that Cluster 2, containing only two individuals, together with observation 34 in Cluster 4, may be interpreted as outlying observations.

In Figure 10, the horizontal axis (Dim 1 of the FAMD analysis – see Figure 9) highlights the main contrast between Cluster 4 (right) and the others. The vertical axis (Dim 2) sets Cluster 2 (top) apart from the rest. Located on the left side, Cluster 1 brings together individuals with lower or negative values on Dimension 1. Cluster 2 is a small, distinct group at the top left, characterized by particularly high values on Dimension 2. Cluster 3 is the largest and most central group, representing the average profile of the majority of observations. Positioned on the far right, Cluster 4 gathers unique profiles that differ significantly from the rest along Dimension 1.

A global Kruskal-Wallis test⁴ (Kruskal & Wallis 1952) revealed a significant difference in ESU between all clusters (p = 0.038) (Figure 11). Post hoc comparisons using Dunn’s test⁵ (Dunn 1961) confirmed that this difference was driven solely by Cluster 2 (gas boiler use) (p = 0.0456). Savings differences among Clusters 1, 3, and 4 were not statistically significant.

Figure 11

As clustering struggles to identify differences in electricity savings, apart from households that previously used a gas boiler, we propose modelling savings without Cluster 2 and based on a set of variables, as presented in the following section.

Energy savings modelling

Absolute annual electricity savings (ES in kWh/year) were modelled using GAMLSS by excluding Cluster 2 (use of gas boiler by only 2 respondents) from the panel. A type 3 Skew T distribution (Fernández & Steel, 1998) was selected according to the generalised Akaike information criterion (GAIC) using the automated distribution selection function (chooseDist) of the gamlss package which analyzes a large distribution set. The AIC is a metric used to compare different models for the same dataset, balancing the goodness of fit against the complexity of the model.

Few variables are significant to explain ES (Table 2), only the household size and the daily consumption in cold weather (µ parameter). Furthermore, the variance (σ) depends on the surface area and the daily consumption in cold weather. The shape parameters show that the distribution is symmetric (skewness (υ) non-significant and close to zero) but with a heavy tail (as kurtosis (τ) is significant) showing extreme values (outliers). The prediction errors of the model are normally distributed and that there is no systematic bias hidden in the quantile residuals (see the wormplot in Figure 14 in Annex).

The Nagelkerke’s⁶ Pseudo-R² (Nagelkerke 1991), reaches 0.437, indicating that the model explains nearly 44% of the variance in ES. The simple coefficient of determination (R² = 0.377) indicates that the structure of the mean (µ) explains only 38% of the variance in consumption. This means that most of the variance in ES remains unexplained by the variables at our disposal.

Translating normalised effects in absolute values indicates that (Figure 12):

• One additional unit of consumption (+1 kWh/day) at outdoor average temperature between [–1°C, 1°C] changes the ES by: +35 kWh/year;

• and one additional occupant reduces ES by: –306 kWh/year.

Figure 12

Households that consume more energy during cold days in the pre-situation achieve significantly higher energy savings per year. Larger households tend to achieve slightly lower energy savings per year compared to smaller households. The size of the home does not appear to influence annual energy savings directly, though uncertainty (the shaded area) increases for larger homes.