Consent and AI
Much of the world is developing data privacy regulations, as many individuals value their right to better control how others can collect and store data about them (Chaaya (2021)).
While data collection concerns have been increasing up for years, AI systems present new challenges (Pearce (2021); Tucker (2018)):
- Accountability - It is more difficult to determine who is accountable at times when separate parties may collect versus redistribute data, versus use data (Hao (2021))
- Data Persistence - Since data rapidly be redistributed it can be difficult to remove data (Gangarapu (2022); Hao (2021)).
- Data reuse - Data collected for one purpose is getting reused for other purposes that may dramatically change over time. For example data collected on food purchases for food companies could get reused by insurance companies to determine health risk based on dietary behavior.
- Data spillover - Accidental data collection due to collection for a different purpose, for example a photo of someone with other individuals in the background
- Trickier Consent - Consent to allow data collection is trickier as it is less clear that users understand the potential risks (Andreotta, Kirkham, and Rizzi (2022))
- Easier Data Collection/Translation - AI makes it really easy to collect and record new forms of data about individuals such as transcriptions of meetings. This is making it easier for people to record people without their consent and poses privacy risks (Elefant (2023)).
Consent is especially a concern for healthcare research, where potential participants need to understand the potential risks of participation. Yet, the risks of data collection continue to evolve.
See https://link.springer.com/article/10.1007/s00146-021-01262-5 for deeper discussions on the topic.
Example 7 Real-World Example
Facial Recognition technology has been an especially debated topic. There have many instances of unethical practices including collecting and reusing data without consent, collecting data on particularly vulnerable populations that could easily be misused, and creating tools that perpetuate bias and harm, such as a tool that was aimed at predicting if someone was likely to be a criminal.
A Berlin-based artist Adam Harvey created a project and website that flags questionable datasets and discussing ethical issues around facial recognition.
I wanted to uncover the uncomfortable truth that many of the photos people posted online have an afterlife as training data (Van Noorden (2020))
See these articles for more information:
Another important consideration is consent or awareness that you are viewing AI generated content that may be fake. The EU AI act has many regulations regarding this, as well as notifying and consenting individuals about when AI tools are being used on them.
Example 8 The EU AI act includes regulations for many things including banning predictive policing technology and requiring consent for emotional recognition technology in work and at school.
Despite many likely very useful restrictions including around facial images, a major debate has been about a lack of restriction for live facial recognition. The potential for harm across different data types and advances in technology will continue to create new ethical challenges. It has been argued that there are possible uses that should be exempt, such as live facial recognition to locate human trafficking victims.
See here to learn more about this debate.
Tips to encourage responsible consent practices
For decision makers about AI use:
- Emphasize education about consent practices
- Stay up-to-date on current issues related to consent.
- Encourage usage of tools that are transparent about using responsible consent practices.
- Encourage users to be careful what data they upload or allow AI tools to use.
For decision makers about AI development:
- Emphasize education of AI developers about consent considerations, guidelines, and regulations.
- Stay up-to-date on current issues related to consent.
- Be considerate of the data that you use for AI tools and how it was collected and if individuals consented to the collection and distribution of the data
- Be transparent with users about what consent practices were used for the data utilized by the tool.
- Be transparent with users about what may happen with their responses if they are being collected.
Summary
Overall the consent process is particularly challenging and consideration should especially be centered on the rights of the individuals who may have data collected about them. We hope that awareness of some of the major challenges can help you to more responsibly implement any consenting processes that may be needed for AI tools that you employ or develop. We advise that you speak with ethical and legal experts.
Disclaimer: The thoughts and ideas presented in this course are not to be substituted for legal or ethical advice and are only meant to give you a starting point for gathering information about AI policy and regulations to consider.
