Going Beyond “Computer Says No”: Examining the Right to Information Post Dun & Bradstreet

1. Introduction

Algorithms have become a necessary part of decision making. They can increase efficiency and if implemented correctly reduce human bias. Algorithms come in various shapes and sizes. Machine-learning algorithms can use data to establish thousands of data correlations, some more accurate than others. The use of so much data can render algorithms complex, and the outcomes can be convoluted in technical explanations. Thereby generating a black box for those who have to assess the result.

These results can have serious implications for those who are confronted with decisions based upon the algorithm’s assessment of the situation. Perhaps the most (in)famous cases of such decisions where computer said no, are those offered by credit scoring agencies. Since the Schufa¹ case these credit scores are considered a decision under article 22(1) GDPR. The agencies are therefore under obligation to provide a meaningful explanation. The question, however, remained what was considered a meaningful explanation?

This question was under discussion in the case of CK v. Dun & Bradstreet, a case which provided more questions than answers. This commentary starts by briefly summarizing the judgment. It then continues in section 3 by discussing the right to information from a legal-technical perspective. This section aims to formulate a concrete framework for the right to explanation. Section 4 discusses the risks of transparency. Section 5 concludes by discussing the wider impact of this judgment in light of the recently introduced AI Act² in light of European and national law.

2. CK v Dun & Bradstreet

The case of CK v Dun & Bradstreet³ pivoted around the question of what constitutes a meaningful explanation of article 15(1)(h) GDPR. The case concerned CK who was refused a phone plan. The additional costs of the plan were €10,- per month.⁴ The refusal was based on a credit assessment performed by automated means by Dun & Bradstreet.⁵ CK had filed a complaint with the Austrian data protection authority. These in turn ordered Dun & Bradstreet to provide meaningful logic about the profiling decision.⁶ Dun & Bradstreet on the other hand claimed that such explanation would involve the disclosure of trade secrets.⁷ The case was bought before the administrative court of Vienna who considered that only a qualified expert could determine the information Dun & Bradstreet must disclose to satisfy its obligations toward CK. The appointed expert concluded that Dun & Bradstreet should at least provide: the personal data about CK used to generate the relevant “factors” (such as birth date, address, and sex); the mathematical formula underlying the score; the specific values assigned to CK for each factor; and the exact ranges or intervals used to assign those values across the same factor.⁸ The national court via preliminary questions wishes to ascertain whether this suffices to fulfil the obligation to provide a useful explanation.

The CJEU starts by examining the term “useful” in different languages of the GDPR.⁹ It then continues by considering that given the broad wording of the provision, the reference to the “logic involved” in automated decision-making, as the core of the required “meaningful information”, can be understood to encompass a wide variety of underlying processes. This includes the different ways in which personal and other data are used to produce a specific outcome through automated methods.¹⁰ This explanation aims to allow the subject to effectively contest the decision.¹¹ The CJEU therefore considers ‘that explanation must be provided by means of relevant information and in a concise, transparent, intelligible and easily accessible form’.¹² The CJEU does not provide clear guidelines for assessing the quality of the explanation. It considers, however, that the explanation must be “meaningful” and that such an explanation cannot consist of mere technical details and mathematical formulas.¹³

The second set of preliminary questions concerned trade secrets. The CJEU considers in this regard that the right to an explanation is not an absolute right. It must be balanced against the rights of others, among which trade secrets.¹⁴ The right to an explanation must therefore be balanced against the protection of trade secrets, yet this balance cannot justify a complete denial of explanatory information.¹⁵ The CJEU thereby concluded that the information had to be meaningful and allow the subject to effectuate their rights, however that the information should balance the intellectual property of the creator.

This case provides some guidance as to what constitutes the right to a meaningful explanation. The CJEU states that a meaningful explanation is more than the mere technical details, however, the judgment does not clarify what does constitute a meaningful explanation. To establish such guidance reference can be taken from other fields.

3. Legal-technical perspective on “meaningful” explanation

The right to a meaningful explanation is placed at the foundation of the right to contest a decision. There is no single framework for what constitutes a meaningful explanation. The American Standards Organization (NISTR) provides four core criteria for assessing explanations. These four are: a reasoned explanation, meaningfulness for individual users, accuracy, and disclosure of the system’s limits of accuracy.¹⁶ These last two criteria of accuracy and limits of the system provide a concrete addition to what can constitute a meaningful explanation.

In addition to its four criteria, NISTR offers a taxonomy of explanation types that can be provided. The most relevant for administrative decision-making is the Regulatory and Compliance (R&C) explanation. The R&C explanation indicates why a particular type of decision complies with laws and regulations. An R&C explanation must be calibrated to the legal framework applicable to the specific case. This provides another concrete guideline, namely the legal framework. According to NISTR, explanation must also be given as to why a specific framework is applied.¹⁷ This creates a clear guideline that improves the clarity of a meaningful explanation. Different algorithms and AI systems must comply with their own sector specific framework and ensure their specific risks are met.¹⁸ In addition to the specific legal framework, a decision must also comply with the general (fundamental) rights framework. This creates a challenge considering algorithmic decision making.

In the Netherlands various algorithmic welfare scandals¹⁹ have demonstrated the dangers of biased algorithms. These dangers varied from outright discriminatory criteria to bias through convoluted proxies.²⁰ Identifying bias in an algorithm is difficult because a single decision offers too little information. A meaningful explanation concerning fundamental rights would thus have to include a means to assess lack of discrimination. Biased factors and/or statistics can lead to algorithmic discrimination. The algorithm learns from the given data and if this data contains a bias, the bias is perpetuated.²¹ This creates a legal conundrum. The right to a meaningful explanation has been defined to exclude mathematical formulas. However, to verify whether the algorithm is free of bias requires verifying the data and formulas. This apparent legal paradox can be addressed through the proposed legitimacy bridge.

Engelfried proposes a legitimacy bridge consisting of four different factors.²² The first is transparency regarding the statistics and/or factors that influence the decision. The second is an explanation of how the outcome of the individual decision fits within the system of previous decisions. The third concerns an explanation of how the decision fits within standards of purpose and law. The final factor relates to response: ensuring the fundamental rights framework by monitoring for undesirable factors.²³ The focus on fundamental rights and the fit of the decision in the series of decisions, enables a more concrete way of testing whether a decision was unbiased. Due to privacy, however, it may not always be possible to publish a series of results. There are three suggested alternatives that allow a single decision to be meaningfully tested against bias.

A first option to combine privacy with discrimination detection, is by avoidance of non-essential data collection. Whereby data that could be potentially discriminatory such as gender or ethnicity are not gathered to avoid a bias based on such factors. The meaningful explanation in such light then includes a list of the criteria that were used, accompanied by a brief explanation of why the criterion was included and how potentially discriminatory factors are excluded. The difficulty is that it is not always possible to determine in advance which criteria are important.²⁴ This technique helps prevent biases but is furthermore not full proof.²⁵ Additionally, when an algorithm uses thousands of datapoints it raises the question of whether handing over all the data points generate a meaningful explanation. A minimum requirement, however, can be that the datapoints are supported by research. Instead of based upon ‘gut-feeling’.²⁶ The difficulty with this approach is that researching the statistically relevant criteria are often part of the algorithms trade secret. An alternative has been suggested in allowing access to the algorithm.

Counterfactuals have been described as a potential solution to decrease bias in algorithms.²⁷ The idea is fairly simple, the subject to the decision is granted access to ask for decisions from the algorithm. Whereby subjects can change their information on potentially discriminatory criteria, such as race and gender. Changing such information should not change the decision outcome, if it does this suggests discrimination. Counterfactuals, however, are not perfect. It is not always easy to establish a bias in non-binary relationships.²⁸ Technical solutions have been suggested to track bias and proxies within algorithms,²⁹ but nothing is perfect.

The third option that has often been suggested is that of using eXplanaible AI (XAI), which includes AI tools to explain AI systems. The most relevant tool to detect bias in non-linear decisions is SHAP.³⁰ Unfortunately the SHAP algorithm is not yet able to accurately detect all forms of biases.³¹ Until XAI systems can accurately detect biases, it should not be considered a viable option to conduct reviews for meaningful explanations.

Another possibility is to shift focus from the right to meaningful explanation concerning the specific decision to a shift on transparency of a system. There are, however, serious challenges to increasing an algorithm’s transparency.

4. Transparency disadvantages: gaming

Increasing transparency possibly conflicts with the rights of others. One of these rights was considered in the Dun & Bradstreet case, namely that of trade secrets. Allowing third parties to verify the importance of criteria entails the risk that the algorithm’s criteria are distilled. This risk could be mitigated by reducing the criteria that can be verified to criteria pertaining to protected characteristics. The difficulty is that these characteristics can be convoluted by proxy-criteria. The CJEU considered that when the controller cannot provide the necessary information without giving up trade secrets, the information should be provided to the relevant authority.³² When the relevant authority is called upon too often this may result in a high workload for the authority. Perhaps this can be compensated by charging a fee for examining the algorithm.

The major drawback of providing useful explanations, however, is that citizens may adjust their behaviour accordingly. Adjusting behaviour to align with norms is not inherently undesirable. It is undesirable, however, when the adjustment is made not to the norm itself, but to the enforcement policy. E.g., issuing fines for not cleaning up dog waste – after which owners start cleaning up – is a desirable behavioural change. Less desirable is when owners simply walk their dogs during the inspector’s lunch break to avoid being caught. This type of behavioural adjustment is undesirable and is referred to as “gaming.”³³ Excessive transparency about the system may cause individuals to adjust their behaviour not to comply with the rule, but to evade enforcement. Research shows indications that criminals sometimes allow a few “test subjects” to be caught to determine which factors triggered suspicion.³⁴ Another method involves corrupting data patterns—altering data so that a legal transaction appears suspicious or vice versa. This distorts the algorithm’s perception of reality and reduces its effectiveness. These type of attacks from outsiders to corrupt the functioning of the algorithm are known as adversarial attacks.³⁵ These different adversarial gaming techniques create a cautionary tale for transparency. Requires a constant balancing between transparency and efficiency. A balance that is certain to bring many more cases to the courts.

5. Wider implications

The case of Dun & Bradstreet concerned the interpretation of meaningful information in relation to automated decision making based on the GDPR. The case, however, also impacts automated decisions made without the use of personal data. This is best demonstrated using an example from Dutch national law through the AERIUS cases.

AERIUS served as the calculation tool that determines whether an activity requires a permit and how much environmental nitrogen “space” is available. AERIUS consists of three modules – Calculator, Register, and Monitor – which together provide insight into nitrogen deposition, permit issuance, and management of development capacity.³⁶ The question before the Council of State was what transparency obligations rest on the administrative authority in cases of automated decision-making.

There was no personal data required to calculate nitrogen deposit, hence the GDPR was not applicable. The Dutch Supreme Administrative Court (ABRvS), however, considered that to ensure equality of arms, algorithms could not form a black box. The ABRvS therefore considered that all data should be published.³⁷ This was later refined to exclude the software underlying the algorithm³⁸ as this might be copyright protected or a trade secret. In the Dun & Bradstreet case the CJEU also linked the right to explanation to effective contestation of the decision. Since both decisions consider transparency as a tool to an effective contestation of the decision, it can be argued that even without personal data there is a right to a meaningful explanation. In technical cases meaningful explanation may require technical details, especially when the conflict concerns governments or legal persons and technical decisions. This type of explanation may conflict with copyright protected software developed by private contractors. In such cases mitigatory measures can be sought through premium payments.

In addition to the non-personal data decisions the decision of Dun & Bradstreet shapes how the AI Act 2024 should be interpreted. When deploying high risk AI Systems, such as credit scores or AI that aid government decisions, the deployer must have a risk-management system in place.³⁹ The risk management system is an important factor in mitigating data leaks and discrimination from occurring. There is no right to access information about the management system. Arguably, however, a person faced with a decision that significantly impacts their position, may wish to ascertain that the deployer of the algorithm has implemented enough mitigating measures to prevent discrimination or that the formula to calculate nitrogen is indeed correct.

It is no surprise that the CJEU took the ‘meaningful’ approach, emphasizing the effective exercise of other rights. Algorithmic decision making is becoming more prevalent in both private and public sectors. Without emphasizing the right to a meaningful explanation, the CJEU would risk eroding the rights granted through the GDPR. The difficulty, however, remains that this right has not been clarified in light of larger decision patterns. Nor is it clear how to interpret this right in light of the AI Act. It is likely more cases will rise, demanding clarity on algorithmic transparency.