From Managers to Machines: A Reply to Respondents

I am grateful to my respondents for their rich and varied responses. Their thoughtful insights suggest that the concept of the New Public Analytics (NPA) can help illuminate and deepen our understanding of how the turn to datafication, digital automation and algorithmic systems within the public sector is transforming how, governmental power and authority are generated, distributed and exercised. This will, I hope, serve as a helpful springboard for critical and comparative investigation and reflection. It can also better equip us to reckon with NPA’s impacts and implications, particularly the susceptibility of NPA systems to misdirection, failure and the production of injustice.

For Keymolen, Wolswinkel and de Souza, my lecture provoked critical reflection on the means and mechanisms that might help prevent and protect against the kinds of abuses that resulted from the use of digital machines to assess benefits and identify welfare fraud in the Netherlands, Australia and Arkansas. In each of these cases, digital machines responsible for undertaking ostensibly ‘mundane’ administrative tasks resulted in serious and occasionally devastating yet often wholly unjustified hardship for those affected. For Taylor, my lecture provoked critical reflection on how NPA implies a deeply impoverished notion of ‘publicness’, a concern also shared by Kempener. Kempener also suggested that NPA is better understood as an evolutionary development of the New Public Management (NPM), sharing logics and practices with the latter, rather than representing a paradigmatic change. In this brief reply, I will follow in Kempener’s methodological footsteps, offering reflections concerning significant, albeit selective, elements of continuity and discontinuity between the New Public Analytics and its predecessor, the New Public Management.¹

The focus of Kempener’s essay lies in what she perceives as the ‘root causes’ of the harmful consequences wrought by the digitisation and datafication in government associated with NPA. For Kempener, these root causes are a series of logics and practices originally associated with NPM, highlighting four in particular: (1) the disaggregation and unbundling of government into smaller units, (2) the centrality of markets and competition, (3) the use of performance management to incentivise public sector managers to drive efficiencies and service improvement in public service delivery, and (4) the turn to ‘scientific’ and ‘evidence-based’ approaches to policy-making, which are associated with ‘objectivity’. While I agree that these logics and practices can also be identified in the take-up of NPA within public administration across open market economies, I am inclined to regard them as logics inherent in the standardisation and modularisation processes that are necessary prerequisites for both market-based and algorithmic-based forms of ordering to operate, rather than central to NPA itself.

The seven distinctive logics, beliefs, assumptions and practices that I argue are present, to a greater or lesser extent, within any given NPA project or system (see Table 1 on page 8) are quite different to the four logics and practices highlighted by Kempener. Her logics reflect the substantive ambition of NPM in its single-minded pursuit of efficiency. In contrast, the logics, practices, assumptions and beliefs that I associate with NPA are rooted primarily in the computational foundations of NPA systems and the ideology of techno-solutionism that underpins it, rather than a foundational quest for economic efficiency. There is continuity between NPM and NPA in that the logics and practices Kempener highlights are rooted in the neoliberal ideology which brought forth NPM reforms and continue to form the ideological foundations of NPA. However the shift away from the explicitly market-based forms of ordering that underpinned NPM, to algorithmic forms of social ordering, marks a crucial and important distinction between NPM and NPA. For me, this shift has profound social, political, legal and economic and significance, which we have yet to fully grasp and whose impacts are still emerging and unsettled.² It is also precisely this shift that can be understood as animating the questions and reflections offered by the four other responses to my lecture and to which I now turn, identifying similarities and differences in the academic responses provoked by the emergence of NPM three decades earlier and NPA today.

For Keymolen, Wolswinkel and de Souza, the turn to algorithmic forms of ordering in the delivery of public services raises questions about the kinds of safeguards needed to orient the take-up of NPA if the scandalous outcomes which these systems have been shown to produce are to be avoided. Each focuses on a different source of protection. Keymolen directs her attention towards professional ethics, highlighting two virtues – modesty and boldness – which she suggests are important for the professionals, particularly the data scientists, who develop and configure NPA applications. In drawing attention to the importance of professional discretion, Keymolen recognises, as did earlier critics of NPM, that despite the championing of ‘impersonal’ forces, whether of markets or of algorithms, human agents are ultimately responsible for the choices that inform how governmental authority is exercised. The turn from NPM to NPA, with its reliance on computational systems and algorithmic forms or ordering, has entailed the ascent of the data scientist in place of the private sector management professional who was central to the foundational logics upon which NPM reforms depended. Critics of NPM claimed that these reforms would result in the ascendancy of private sector management values and principles, with their emphasis on profitability, risk taking, responsiveness and enterprise, to the detriment of values associated with a so-called ‘public service ethos’, such as honesty, integrity, accountability and probity.³ Under NPA, however, my fear is not primarily that data scientists will exercise their professional judgement in a manner that is perverted by the self-interested logic of markets. Rather, I worry that in making ostensibly ‘technical’ choices in the course of designing and implementing algorithmic systems data scientists will fail to understand the normative significance of their decisions, which may well embody contestable political judgements that directly affect how NPA systems mediate, inform and automate the outcome of governmental decisions and activities. Furthermore, the remoteness of technical developers from the lives and lived experience of the individuals and families who are subject to evaluation or otherwise directly affected by NPA systems is likely to blunt the sensitivity of technical developers to the latter’s needs and concerns in the real-world contexts in which they come into contact with NPA systems. Accordingly, I agree that the two virtues which Keymolean singles out, notably modesty (requiring the acknowledgement by technical developers that digital data is partial and incomplete in relation to that which it purports to represent, and that unintended consequences of NPA systems are inevitable and unavoidable), and boldness (through which technical developers openly recognise and acknowledge that their decisions have political consequences so that input from those affected via meaningful political participation is important), are important virtues which technical developers entrusted with the task of developing NPA systems should seek to cultivate and practice.

Yet, as I have argued elsewhere, ethical self-regulation by key actors involved in the design, development and implementation of automated, data-driven systems cannot be relied upon alone to prevent the exploitative and unjust outcomes which these systems may produce.⁴ The role of law is vital. This is reflected in Wolswinkel’s response, who asks whether administrative law is capable of meeting the challenges posed by the embrace of NPA. He points to differences in the views offered by administrative law scholars in response to this question, noting that the weight of opinion to date is broadly optimistic, claiming that algorithmic systems can be accommodated by administrative law, but the application of those principles requires sensitivity to context. Debate about the capacity of administrative law to rise to the challenges raised by NPA resonates with debates between administrative law scholars a quarter of a century ago as they questioned the capacity of administrative law to contend with the sweeping NPM reforms which permeated the institutional landscape of public administration. For example, Harlow expressed considerable optimism about administrative law’s capacity to adapt to NPM suggesting that a ‘New Public Law’ was emerging in parallel with the NPM. She praised Britain’s ‘Citizen’s Charter’ initiative which she claimed had reinforced the idea of political citizenship by:

Lewis, on the other hand, was considerably more circumspect. Not only did he decry the absence of any national debate or collective dialogue concerning NPM reforms which were ‘constitutionally revolutionary’ in character, but he was stinging in his criticism of the Citizen’s Charter which he claims recharacterized citizens as consumers arguing that

These historical debates about NPM are instructive when reflecting on early attempts by administrative law scholars to grapple with NPA’s implications for administrative law doctrine and the underlying constitutional principles and foundations upon which administrative law ultimately rests. I have on previous occasions argued that time-honoured constitutional principles, including the rule of law and principles of due process and fair procedure have the potential to act as meaningful and effective safeguards against many of the injustices and abuses of power which the take-up of NPA portends.⁷ But in the context of NPA whether they in fact do so will depend critically upon the capacity of judges and legal advisors to engage meaningfully and systematically with the choices of technical developers in designing and configuring NPA systems. This will require a proper grasp of the power and significance of those choices in real world contexts. Hitherto, administrative law scholars have not yet demonstrated that they are able and willing to undertake the difficult, time-consuming and detailed interdisciplinary analysis that I believe is required.⁸ More generally, it remains to be seen whether administrative law is capable of grasping and reckoning with the shift from public ordering by rules and markets, to public ordering by and through algorithms, particularly given governments’ dependence on private sector consultants to design, implement and maintain these algorithmic systems. In addition to cultivating a sensitivity to the real-world consequences of NPA systems for the lives of those affected by them, as Keymolean advocates, technical professionals will also need to work closely with legal professionals to ensure that they design NPA systems that respect constitutional and administrative law principles. The latter should serve, in effect, as ‘requirements and specifications’ that establish the legality of algorithmic systems before they are placed into service.

The challenges faced by administrative lawyers and judges in faithfully reinterpreting and applying foundational public law principles to the design, configuration, operation and outcomes of public sector digital machines are formidable, but not insurmountable. Assuming that judicial officers, legal counsel and academic lawyers are willing to devote the time and intellectual effort necessary for them to properly grasp the legal significance of ostensibly ‘technical’ choices concealed within the bowels of NPA systems, the case-by-case process through which this reinterpretative process occurs will necessarily be gradual and evolutionary. But time is not on our side. The rights, interests and basic needs of individuals and communities are already being significantly and detrimentally affected by the operation of NPA systems. In this respect, the urgent need for bottom-up mechanisms through which people and communities can contest NPA systems is under-appreciated in the burgeoning literature of so-called ‘AI governance’. To this end, de Souza’s response offers an enormously helpful account of concrete strategies and tools that are helping to empower those whose lives are directly affected by NPA systems to understand those systems and to push-back against them in concrete, relatable ways. He reminds us of the critical importance of ensuring that affected individuals are active participants in the dialogue and discussion about NPA technologies and systems, with the ability to shape and influence how such systems function. I would add that affected individuals should have a say in whether NPA systems should be used at all. In this respect, the concrete tools and campaigns produced by Tactical Tech (a Berlin NGO) and the Rethink Aadhar campaign in India, which de Souza describes, are impressive examples of the way in which civil society organisations are developing engaging yet accessible approaches through which ordinary members of the public can make sense of NPA systems and resist their rapidly expanding influence over their everyday lives. As well as their practical utility to individuals, de Souza argues that these tools and campaigns also help to counter NPA ideologically, by demystifying it and helping ordinary people to recognise its impact on their daily lives. His response also reminds us of the critical and indispensable role of civil society in seeking to hold accountable and responsible those who preside over NPA systems. By motivating individual and collective action and widening the range of practical possibilities for resisting the excesses of NPA systems, civil society organisations (particularly those with the technical expertise to critically evaluate them) provide a potentially powerful avenue for increasing public awareness of these systems and their impacts, thereby enhancing the public’s ability to participate in democratic and deliberative conversations concerning their use and acceptability.

de Souza’s emphasis on the importance of public participation and democratic deliberation resonates with Taylor’s warning of the way in which the public interest and public values are being clandestinely hijacked by commercial tech providers who are commissioned to design, provide and maintain the back-end technologies and systems upon which NPA systems depend. Unlike her co-respondents, Taylor focuses directly on the role of the private sector in the embrace of NPA and the threats to the public interest which that involvement creates. She asks: what constitutes acceptable involvement by the private sector in facilitating, and setting the parameters for, interventions on the public? At one level, there is nothing novel in asking critical questions aimed at identifying limits to public sector outsourcing, particularly given its potential to undermine or circumvent the protection of constitutional values.⁹ Questions of this kind were a central preoccupation in critical scholarship that emerged in association with the NPM reforms which began four decades earlier. At another level, however, these questions are both entirely novel and of critical importance due, again, to the critical differences between the algorithmic forms of control and coordination associated with NPA, and the more conventional and familiar forms of market-based and rule-based forms of ordering associated with NPM.

I have already highlighted the way in which NPA systems conceal political choices within the technical systems and protocols through which they operate. However, the opacity and technical sophistication of these systems makes it practically impossible for those without the requisite technical expertise to identify and understand the significance of decisions embedded into their design and operation. At the same time, the extensive reliance of governments on commercial contractors to design, configure and implement these systems without appropriate in-house expertise means that the public sector organisations commissioning them lack the capacity to insist upon effective constraints or exert meaningful oversight. For example, the National Audit Office’s recent damning review of the Home Office’s National Law Enforcement Data programme (NLEDS) to replace the Police National Computer (PNC) and Police National Database (PND) found that the programme was more than five years past its planned delivery date with a cost of £1.1 billion¹⁰ that was 68% more than the initial estimate of £671 m. The NAO found that the Home Office had relied extensively on contractors from multiple organisations without offering any rationale for this approach, or a commercial strategy to make those contractors work together. Contract payments were based on fixed day rates for staff rather than setting a total price in advance, and there were no contractual mechanisms in place to incentivise delivery. An external review of the programme found that the Department was over-reliant on suppliers filling leadership roles, was not taking sufficient ownership of the technical work, had limited ability to control how suppliers worked together and that the code produced was often of poor quality while the development process was bureaucratic rather than collaborative.¹¹ In short, the Home Office had failed to exercise proper oversight of the programme. Without adequate in-house technical expertise, private contractors appear to have been permitted a very long leash, enjoying considerable freedom to make design choices that served their own commercial self-interests, rather than prioritising the needs, interests and democratic expectations of the public.

While the NAO report is sobering reading for the British citizen and taxpayer, these failures of good administration can be understood as familiar principal-agent problems which, as I remarked in my lecture, could in theory be addressed through in the vigilant specification of contract terms and effective governmental oversight of private contractors. Yet Taylor identifies an important and entirely novel feature which distinguishes the trajectory of NPA reforms from those which underpinned the emergence of NPM. The institutional reforms wrought by NPM involved either the wholesale transfer (in the case of full privatisation) of state-owned assets into the private sector, or more limited forms of delegation from the public to private providers (in the case of the contracting out of service provision to private or other third party providers). Consequently, the general public had some prior experience of that asset, service or function when it was delivered directly by the state prior to its delegation to private providers. It was therefore possible for members of the public, and their elected representatives in Parliament, to compare and contrast the quality of the service before and after market-forces had been brought to bear on its provision. In contrast, Taylor observes that with NPA systems, private providers are often involved in design and configuration from the start, meaning there is no public sector provision to compare against. Taylor further points out that in some cases, global tech firms are able to harness their vast resources and capacities to establish and configure the infrastructure upon which contemporary economies rely, without government permission or contract. For example, she refers to Google’s success in capturing global location data and mapping analytics by using its vast financial and technical resources to send out massive fleets of cars with cameras to create maps, and using its servers to store and analyse the huge amounts of data produced. In this way, the digital tech industry is effectively appropriating important public functions through their infrastructural power alone, without the need for any prior delegation of authority from the state, let alone public debate on how such infrastructure operates, or indeed if they are wanted at all. Hence Taylor rightly highlights the need to ask penetrating questions about the nature of the resulting political settlement, including between which parties the social contract is being made, and under whose auspices it is exercised.

If I am right in arguing that NPA systems are not underpinned by any explicit substantive value, unlike NPM’s single-minded quest for efficiency, NPA’s reprogrammable nature, when combined with the unilateral power of giant tech companies to develop and configure network infrastructures, leaves powerful tech firms free to design what economists would term ‘public goods’ in a manner that best serves their private self-interest. The implications and consequences for the public interest are not merely alarming, they are potentially eviscerating. As members of the general public, we find ourselves effectively taken hostage and without serious alternatives in ways that appear decidedly anti-democratic. How are we to guard against this technological lock-in and path dependency that have been configured from the outset to facilitate the interests of digital capitalists? For Taylor, the resulting conceptual gap requires that we conceptualise publicness in new ways and define why we care about certain goods remaining accountable to the public, and why those goods cannot, by their nature, be held in private hands. Her call to arms returns us to Oliver Williamson’s classic statement of the ‘make or buy’ dilemma faced by organisations. There are good reasons for thinking that basic network infrastructure, and digital machines that provide services to meet the basic needs of the population, should be kept firmly and directly in the hands of the people whom they serve, rather than allowing them to be configured to serve the interests of the digital capitalists. In other words, reliance on private provision should not be regarded as an acceptable substitute for governments investing seriously in high quality in-house technical expertise, including a cadre of technical professionals attuned to, and guided by, the public service ethos I referred to earlier. Yet with public finances already stretched beyond comfortable limits, I find it difficult to imagine that the level of sustained investment required to develop that in-house capacity will be prioritised. In short, it is hard to resist the conclusion that the challenges we face in bringing NPA to heel are ultimately deep and structural, traceable to the hollowing out of democratic institutions that accompanied the triumph of neoliberalism and which are in urgent need of repair and reinvigoration. In this respect, I agree with Kempener that the problems we face are not, at their foundation (new) problems of data analytics, but are rooted in wider systemic problems of governance.