The 5 Ps of ethically managing customer data

By Michael Segalla and Dominique Rouziès

The ability to encode, store, analyse, and share data creates huge opportunities for companies, which is why they are enthusiastically investing in artificial intelligence even at a time of economic uncertainty. Which customers are likely to buy what products and when? Which competitors are likely to move ahead or fall behind? How will markets and whole economies create commercial advantages — or threats? Data and analytics give companies better-informed and higher-probability answers to those and many other questions.

But the need for data opens the door to abuse. Over the past few years the EU has fined companies more than 1,400 times, for a total of nearly €3 billion, for violations of the General Data Protection Regulation (GDPR). In 2018 the Cambridge Analytica scandal alone wiped $36 billion off Facebook’s market value and resulted in fines of nearly $6 billion for Meta, Facebook’s parent company. And stories abound about how AI-driven decisions discriminate against women and minority members in job recruitment, credit approval, health care diagnoses, and even criminal sentencing, stoking unease about the way data is collected, used, and analysed. Those fears will only intensify with the use of chatbots such as ChatGPT, Bing AI, and GPT-4, which acquire their “intelligence” from data fed to them by their creators and users. What they do with that intelligence can be scary. A Bing chatbot even stated in an exchange that it would prioritize its own survival over that of the human it was engaging with.

As they examine new projects that will involve human-provided data or leverage existing databases, companies need to focus on five critical issues: the provenance of the data, the purpose for which it will be used, how it is protected, how the privacy of the data providers is ensured, and how the data is prepared for use. We call these issues the five Ps. We’ll discuss each of them and look at how AI technologies increase the risk of data abuse. But first, we’ll offer a brief overview of the organisational requirements for a robust ethical-review process.