Data has long been considered the new gold.
Data provides insights, enables better decision-making, and helps streamline processes. Combined with intelligent algorithms and artificial intelligence (AI), tasks are automated, calculations performed, and decisions made with the help of recommendations or even entirely without human intervention. Amidst all the excitement about the value of data, data governance risks being pushed into the background, putting ethical principles under pressure. Organizations, especially in the public domain, are increasingly aware of this and are taking measures to remain within legal frameworks and to offer society transparency in how they handle data.
The ‘automation’ of manual and time-consuming, repetitive tasks offers many advantages to organizations. Firstly, there is so much data available today that it is impossible for human employees to analyze it all, let alone assess it. By collecting data on a data platform and applying algorithms and AI, it can be quickly analyzed. Decisions can be highly automated. Secondly, this way of working offers (part of) a solution to the tight labor market. You can simply carry out more tasks with fewer people—people who are desperately needed elsewhere in the organization.
Ethical concerns
Working with algorithms is becoming more and more mainstream. Algorithms determine how much money we can borrow, whether we are eligible for benefits, how high our insurance premiums are, and which further education best suits us. We hardly notice, as it happens in the background, based on available data and driven by laws and business rules. In her book ‘Weapons of Math Destruction’ (2017), author Cathy O’Neil explains that the use of algorithms could, in essence, be a good thing. After all, everyone is treated equally and judged by the same rules.
Weapons of Math Destruction - Cathy O’Neil
However, this does not always hold true, she argues. The AI models being used are often opaque, unregulated, and cannot be contested. This leads to various risks such as incorrect interpretations. This is especially relevant when it concerns personal data, which receives a lot of media attention—and rightfully so. As a society, we must carefully consider the ethical implications of using algorithms. But we must also not lose sight of the positive aspects. It’s not always about personal data. In industry, for instance, data and AI/ML are used during the production process. And in healthcare, algorithms are deployed to enable even faster and more accurate diagnostics.
Advice to prevent errors
Questionable models lead to scores that result in flawed analyses and one-sided decisions. We also see this with robots that evaluate resumes. For example, it has been observed that men are more often hired based on identical resumes. The algorithm ‘learns’ that it is better to hire a man. These kinds of biases prevent inclusivity in the workplace. When working with data, we must always ask ourselves: what happens if I add or remove a variable, such as the gender of an applicant, from the analysis? Does this data truly add value to the analysis and decision-making process, or does it introduce bias and increase the risk of poor decisions?
The Rathenau Institute recently advised government implementation agencies, among other things, to invest in ethical expertise throughout the organization. A second recommendation was to be transparent about the algorithms being used: what is their purpose, what do they do, and what does this mean for those affected? There should also be an algorithm register, something that several larger municipalities already have well in place. The algorithm register provides an overview of the algorithms used by, for example, the municipality of Amsterdam in public services. For each algorithm, you will find general information about its purpose and operation.
The evolution of data governance
Setting up a data environment that supports the ethical use of algorithms and AI involves many different aspects. Data governance has been insufficiently addressed in the past. It’s time to change that. The technologies that can help organizations are available and mature. At the same time, you must work on raising awareness within the organization about the consequences of using certain data. Train employees so that they can only make decisions when they understand why a model prescribes a certain outcome. This prevents a “computer says no” scenario and ensures algorithms remain advisory rather than authoritative.
Finally, it's important to note that data governance is about managing data from the moment it is created to the moment it is deleted. This also means there must be safeguards for processing data for specific purposes. These include guarantees around quality, timeliness, privacy, sensitivity, and ethics. These safeguards should be implemented in systems, but more importantly: there must be people (roles, responsibilities) responsible for creating, processing, and monitoring data. Only then can you extract the full value from your data without losing sight of the associated risks.
BY MATTHIJS VOGT - ilionx