Advancing Ethical AI Through Governance And Global Standards
To watch this discussion please click here
Current standard-setting systems
On December 3rd , working group 6 of the AI4People Summit brought together various experts on international standardisation and governance. Robert Madelin, former Director-General of DG Connect at the European Commission, incited a panel discussion on the role of standardisation and (global) governance in the field of Artificial Intelligence.
“The central challenge for international AI standardisation is that existing global standard-setting systems are too slow, too narrow, and too fragmented to handle fast-moving, socio-technical, and globally diverse technology like AI. This required more inclusive, flexible, and coordinated processes that integrate ethics, human rights, and regional differences while preventing dominance by powerful actors.”
Artificial Intelligence has already penetrated most areas of everyday life, although it only gained public attention during the 2020 launch of ChatGPT. Since then, AI innovation has been on a rapid pace, moving faster than a typical life cycle of stable hardware production, and faster than the traditional linear standardisation timeline of research, development and standardisation spanning over 3 years. The unpredictability that goes along with the fast market-driven cycles pushes for continuous updates and parallel processes, calling for the standardisation process to be more parallel instead of linear. Standards now must evolve dynamically alongside AI innovation to remain relevant. Mr Ebrahimi argues that the systems must thus become more agile by introducing sandbox-style standardisation processes, allowing them to be iterative and flexible as innovation changes the landscape.
Socio-Technical and Ethical Changes in AI Standardisation
As the process not only becomes market-driven but also inter-sectoral, meaning AI being part of nontraditional IT sectors such as healthcare or justice, standardisation requires expertise that goes beyond engineering. To successfully develop standards for all sectors, also including ethical and social guidelines, lawyers, researchers, human rights activists, labour representatives, as well as civil society must be invited to the table, painting a more cohesive picture of the actual innovation deployment. However, two main aspects are limiting the success of diverse approaches: 1) lack of funding and 2) lack of a common language. While the former limits the participation of smaller actors and thus increases the risk that big tech dominates the processes without safeguards, the latter requires more effort than just monetary resources. As different stakeholders need to not only agree on ethical norms that are highly contextual and cultural, but also find common grounds to balance globality and the specificity of norms, the governance of the standardisation process is increasingly important.
“AI is not only an object of standardisation but also a force that reshapes the entire standardisation process, requiring faster, parallel, and more ethically grounded standards that address AI-specific risks (like deepfakes, opacity, and probabilistic behaviour) while ensuring trustworthy, equitable, and globally coherent deployment.”
Compared to traditional technology, modern innovation strikes not only in its rapidity but also in its nature of being the recipient as well as the influencer of the very standardisation processes. As the aforementioned traditional timeline is no longer viable, and the stakeholder groups need to be more diverse, standards must now display a socio-technical and ethical complexity, which makes trust its most important factor, according to Ms Paterson. Standards alone are seemingly not sufficient anymore to convey this trust, being why human capital needs to be well integrated into international standards in order to solidify trust in fast-paced AI innovation. AI forces standardisation to embed ethics and governance into its processes, making trustworthiness, accountability, responsibility and social values integral parts of its development. Accountability is especially highlighted by Mr Panai, while Mr Walsh underlines the opacity and concentration of AI development as a major source for lack of trust. Thus, ethical infrastructures and regulatory goals such as risk management and governance are cornerstones to achieve balance and shape real-world compliance.
Cross-Border Cooperation
However, not only the sectoral but also the global fragmentation and jurisdictional diversity are challenging the future of standardisation processes. While different national and supranational approaches stemming from the EU, US, or the Asia-Pacific region serve a culturally sensitive approach to standardisation, they also risk global misalignment. The aforementioned opacity of new AI models, paired with fragmented legislation, leads to complicated interoperability. Standards serve as a bridge between jurisdictions, supporting the impact assessments and thus enabling cross-border cooperation. To alleviate the ramification of global AI risks, Mr Walsh highlights the importance of open-source models and smaller, specialised AI.
“Global governance of AI increasingly relies on international standards as the practical mechanisms that translate ethical principles and policy goals into interoperable, cross-border, and trustworthy implementation frameworks—requiring inclusive participation, shared terminology, and ethical foundations that respect cultural diversity amid geopolitical competition and rapid technological change.”
Lastly, standards are essential tools for operationalising global AI governance as they translate high level policy and regulatory goals into concrete, implementable practices. Thus, it comes as no surprise that the standardisation community is actively working to alleviate the future challenges identified by all of the speakers. Standards are now supporting policymakers and creating shared vocabularies for international cooperation, creating the aforementioned shared language. Additionally, they now must be part of an “ethical infrastructure” that shapes responsible global AI use while avoiding a one-size-fits-all ethical approach, honing in on cultural differences. An example of this changed approach is the Seoul Declaration of May 2024, highlighting the need to embed ethics, rights, accountability and transparency into the standardisation process.
AI in Healthcare
To watch this discussion please click here
Expert Perspectives on AI in Healthcare
The first part is speakers expressing their perspectives on AI in healthcare. The key ideas of speakers’ speech are as follows.
Guillaume Bernard, Computer Research Engineer at LNE: In the EU market, medical devices with embedded AI must comply with both the European Union Medical Devices Regulation (MDR) and the EU AI Act. Guillaume Bernard’s team is dedicated to helping medical device providers meet the compliance requirements of both MDR and the AI Act. Their approach consists of two main components. First, they look into the development process of the AI system to evaluate whether the system is sufficiently mature. For example, they assess whether the system meets transparency requirements, whether the training data is sufficiently comprehensive to cover all medical domains that the system is intended to support, and whether the samples represent different population groups—such as variations in age and gender. Second, they test the AI system itself. This is challenging because an AI system often functions as a black box. During the testing process, they examine the system’s input data and output results to understand its behavior. For example, they use evaluation datasets to assess the system’s performance, and then analyze which aspects meet the requirements of MDR and the AI Act and which aspects still fall short.
Marco Lorenzi, tenured research scientist at the Inria Center of University Côte d’Azur: As a research scientist, Marco has been confronted with the problem of gathering data from different sources, different hospitals. Marco has faced legal and ethical challenges. To address these issues, Marco has initiated an open-source framework for learning in health care. He participated in many consortia that involved many stakeholders and researchers from different disciplines. To address these challenges, work from multidisciplinary backgrounds is needed.
Alison Noble, Technikos Professor of Biomedical Engineering, University of Oxford: Addressing AI applications in healthcare requires interdisciplinary collaboration, and ethical considerations play a crucial role in these applications. As AI in the medical field is still in its early stages, we must learn how to use these technologies in the most appropriate and effective way. In addition, human oversight is essential in AI applications in health. When confronted with AI technologies, we must first reflect on why we are using AI at all. In the medical domain, the core of AI safety is patient safety. Ensuring the safety of patients during interactions with AI systems is paramount. Furthermore, although AI systems can empower patients by providing access to health information, they may also deliver misleading information, which can undermine trust in healthcare professionals and introduce medical risks. Finally, the global ethics of AI in healthcare is critically important. AI models are often trained on datasets derived from Western contexts, which may not be directly applicable to other regions. Ethical standards should therefore be grounded in a consensus among stakeholders from diverse backgrounds.
Kristian Vigenin, Member of the European Parliament, LIBE Member：AI technologies bring tremendous opportunities as well as significant risks. We must ensure that democratic societies shape AI, rather than allowing AI to shape society in ways that undermine fundamental rights and freedoms. The EU AI Act establishes a risk-based regulatory framework that reflects a strong commitment to fundamental rights, accountability, and democratic governance. Innovation must never come at the expense of human dignity. In the healthcare sector, AI offers important opportunities to improve diagnostics, enable personalized treatment, and enhance access to medical services. However, it also creates risks of discrimination, unequal access to technology, and increased surveillance. To ensure that doctors and patients can trust AI systems, we must build clear ethical foundations and robust regulatory safeguards. Governing AI requires global cooperation. Technical standards must be interoperable, and human rights principles must be recognized as universal. The oversight of AI systems must remain under human control, must be transparent, and must be understandable in its effects. We now have the opportunity to build a global ecosystem that both protects rights and encourages creativity—an endeavor that depends on political will, technological collaboration, and the courage to establish rules in a rapidly evolving technological landscape.
Ran Balicer, Public Health Professor at the Ben-Gurion University, IsraeI：Ran Balicer discussed use of AI in practice in a organization。In traditional healthcare technologies, once a technology is approved in one region, it can often become available for use in other regions as well. However, this approach does not apply to AI technologies. The fact that an AI system performs well in one region does not guarantee that it will perform well in another, because the dataset, data structure, and population characteristics may differ significantly. Therefore, we cannot assume that an AI system that works effectively in one institution will function equally well in another. The sustainability of healthcare systems is becoming increasingly important. With ageing populations and rising medical demands, healthcare structures must be transformed, and greater reliance on AI will be essential. Many healthcare organizations are attempting to adopt AI systems, but they struggle to determine whether these systems are truly responsible. Ran Balicer, working within an Israeli health services organization, developed a systematic checklist tool to evaluate whether an AI system meets the criteria for responsible AI. Many AI systems fail this evaluation. Every AI system may embed biases—even if we are unaware of them, they still exist. The key is to identify and address these biases so that the advantages of AI can outweigh the associated risks.
Discussion on Pre-Determined Questions
The second part of the working group session is to discuss pre-determined questions. The following questions have been discussed.
What essential components should a standalone healthcare framework for AI include to ensure clinical effectiveness?
The ideas of speakers to address question 1 are as follows.
Alison Noble: Alison recognizes the importance of checklists. It is not necessary for everyone to become an expert in every field; however, from the very beginning of AI deployment, it is essential to clearly identify which factors must be taken into account when assessing whether an AI system is responsible. Checklists and standards should be continuously updated over time, as increased experience and learning will help us become more comfortable with managing risks. It is also important to differentiate between AI use cases. AI is a platform technology with a wide range of applications. If each use case were governed by a completely separate framework, it would be difficult to comprehensively address ethical issues. Therefore, the development of solutions requires the joint participation of multiple stakeholders. Furthermore, successfully deploying AI in healthcare requires teams with the appropriate mix of skills. In addition to clinicians and patients, such teams must include data managers and project managers. Similar to previous industrial revolutions, AI is likely to give rise to new professions. AI systems can function effectively only when supported by an appropriate team structure. At present, the healthcare sector lacks a sufficient number of professionals with the necessary skills, and time is needed to build these new multidisciplinary teams.
Marco Lorenzi: Data itself must also be standardized, especially in the clinical domain. Data standardization includes the standardization of data formats, access to data, and interoperability between different tools and systems. While there has already been some experience with medical data standardization, this represents only a drop in the ocean. In many other areas of medicine, data standardization is still lacking. As such, data standardization constitutes one of the fundamental foundations for both the development and regulation of AI. In addition, hospital training is critically important. Hospitals are not only key stakeholders in AI systems but also major providers of training data. In some cases, medical data is proprietary, which makes data sharing difficult. Therefore, data standardization and hospital training must proceed in parallel. Moreover, hospitals face challenges beyond the shortage of professionals with the appropriate skills; they also suffer from limited resources. Hospitals already operate under significant pressure, and the additional burden of AI engineering risks overwhelming them. In France, public hospitals are managed in a centralized manner at the regional level, and this governance system is currently evolving. Beyond technical solutions, government investment is also essential to support the sustainable deployment of AI in healthcare.
Guillaume Bernard: Guillaume’s team has developed a Metrics Framework. The team is exploring how data provided by hospitals can be converted into multiple formats. They have also identified that healthcare institutions often possess large volumes of data, yet are uncertain whether this data can be used for medical AI systems. Addressing these issues is a central focus of the team’s ongoing work.
In what unexpected ways could AI transform patient care beyond improving efficiency and diagnostics?
The speakers have expressed the following ideas.
Marco Lorenzi: Many medical devices are now being equipped with AI capabilities, and these devices may become interconnected, enabling a multimodal view of patients. Data from different levels—such as biological images and clinical data—can be integrated to generate more personalized representations of patients.
Guillaume Bernard: AI can assist medical experts in healthcare and help reduce their workload. From the patient’s perspective, AI can support individuals in understanding their health needs, particularly in countries where access to medical resources is limited. However, it is also important to recognize that the use of AI systems in healthcare continues to face significant challenges. Many AI systems remain in the testing phase, and their behavior is not yet fully understood, which can raise red flag issues.
Alison Noble: AI, robotics, and simulation need to be discussed together. AI has already begun to be used in medical education, for example by enabling students to enter simulated training environments more quickly. Through pattern recognition, AI can identify disease features and treatment pathways beyond what is visible to the human eye. Whereas medical practice has traditionally been constrained by what clinicians can directly observe, AI allows us to rethink how patients are diagnosed and treated. AI is also transforming the doctor–patient relationship, with patients increasingly expected to take greater ownership of their own health conditions. At the same time, robust governance mechanisms are essential to ensure safety. In situations where outcomes remain uncertain, patients should not be encouraged to diagnose themselves, as this could place additional strain on healthcare systems.
How can research and development facilitate AI innovation in healthcare？
To address this question, the speakers have formed the following opinions.
Alison Noble: Companies tend to focus on how to generate revenue from AI tools, whereas healthcare systems are primarily concerned with health economics. Greater attention must therefore be paid to health economics, particularly to reducing costs and ensuring that healthcare remains affordable. Many healthcare systems operate without new or additional budgets, and as populations age, the introduction of new technologies places increasing pressure on public finances. As a result, a key question we must confront is whether, even if AI technologies prove effective, we are able to afford the costs of operating and maintaining them.
Marco Lorenzi: Pursuing technological innovation in a sustainable manner is highly challenging. This is particularly true when rapid development must be achieved while simultaneously meeting stringent regulatory and compliance requirements, as these competing demands significantly increase complexity. Other countries may operate under different priorities, enabling faster AI development, which in turn requires us to accelerate certain tasks in order to remain competitive. From an academic perspective, we are also confronted with the issue of talent loss. We must persuade researchers and professionals to stay, and to believe that it is possible to achieve domestically what they might otherwise seek to accomplish abroad. Identifying a “sweet spot” that balances regulatory compliance with rapid innovation is extremely difficult. Moreover, we are still lagging behind in terms of infrastructure, including insufficient computing capacity and limited ability to translate ideas into deployable products. Addressing these shortcomings requires substantial investment. Under conditions of fiscal constraint, building such a system is particularly challenging. If Europe is to remain competitive in the future, these issues must be taken seriously and addressed with urgency.
How can innovation be accessible globally？
Marco Lorenzi: Protecting intellectual property (IP) is highly challenging, financially demanding, and difficult to maintain over the long term. It is often hard to patent an idea with commercial potential. What is needed is a research-supportive ecosystem that can better protect research ideas and facilitate their transformation into commercial products. Marco referred to a case he had experienced in which a research idea was highly promising, yet there were insufficient resources to develop it into a long-term project. This highlights the need for support from well-resourced industry partners. IP itself is complex and costly, and in many cases, a company’s value is largely based on the amount of IP it holds.
Alison Noble: One reason for the complexity of these issues is that solutions developed by companies do not always generate returns, as their value is often tied to the data they own. Business models are typically built around resources that are unique and capable of creating a competitive advantage. However, changes in data governance have created significant difficulties for companies, and many existing business models struggle to adapt to the new environment. From a public governance perspective, stability is essential to enable companies to develop sustainable business models. Achieving this in the context of AI and healthcare is particularly challenging, as companies must invest in intellectual property, acquire data, and comply with regulatory requirements. At the same time, hospitals may lack sufficient financial resources to adopt new innovations. It is therefore important to avoid situations in which hospitals become dependent on companies that are no longer able to operate. These challenges underscore the need to find ways to reduce costs across the system.
Conclusion
At the end of the session, Fausto Pedro Garcia Marquez delivered the concluding remarks. According to him, artificial intelligence today is highly complex and is developing at a very rapid pace. The European Union, as well as many other countries, has adopted different definitions of AI, and different frameworks reflect divergent understandings of what AI is. In the coming years, AI is expected to play an important role in healthcare. Health-related issues are central to everyday life; however, healthcare itself is a highly complex domain, encompassing a wide range of applications, including medical robotics. As a result, there is a need to reach a consensus on what artificial intelligence actually means, and whether this meaning remains consistent across different countries and over time. This question is closely linked to ethics. Based on differing ethical approaches, policymakers may adopt different standards to protect society, technology, and stakeholders. At the same time, it is important to recognize that AI is also part of the commercial landscape. Many AI solutions are developed by companies whose primary objective is profitability. Therefore, stable standards are required in contractual arrangements and commercial practices. Policymakers must adopt a pluralistic perspective and actively engage with professionals, as those who develop AI solutions and those who apply them often hold different viewpoints. Overall, AI and healthcare constitute a highly complex, global issue, cutting across national borders, systems, and organizations. The discussion must remain focused on identifying the specific problems that AI is intended to address. There is no universally applicable solution; instead, a clear balance must be maintained between knowledge accumulation and problem-oriented approaches.
The following speakers further contributed to the conclusion.
Marco Lorenzi: This is indeed a highly complex issue. The environment is evolving on a daily basis, with new innovations emerging alongside regulatory requirements. In many cases, new problems only become apparent once systems are deployed in practice. Addressing these challenges requires interdisciplinary collaboration. Different disciplines hold different understandings of AI and healthcare, and successfully deploying systems necessitates acknowledging and engaging with these differences. These issues extend beyond purely technical concerns and also involve social and philosophical considerations. As such, there remains a substantial amount of work to be done, requiring broad and sustained interdisciplinary cooperation. In addition, greater awareness at the policy level is essential to ensure the sustainability and global competitiveness of AI systems.
Guillaume Bernard: From both the patient and healthcare professional perspectives, it is essential to understand what AI systems can do and what their limitations are. It is important to avoid both overconfidence in these systems and excessive caution that leads to their non-use. When AI systems are developed but not adopted in practice, this represents a waste of both time and financial resources. Despite the significant challenges associated with regulation, cost, and implementation, there remains strong optimism about the potential of AI systems. Realizing this potential will require substantial effort to ensure that AI is used effectively and responsibly.
Alison Noble: From a historical perspective, we are at a critical moment. Key stakeholders all hope that AI technologies will succeed in the healthcare sector. While we have not yet fully reached that point, collaboration among stakeholders can help identify the optimal application areas in which AI can deliver real value. At the same time, there are domains in which AI may not be appropriate, and it is important to identify these areas early and discontinue further exploration. Patient trust is also a crucial issue. Patients and the broader public must be actively engaged in this process. As healthcare increasingly extends into the community, AI applications are no longer confined to familiar and controlled environments such as hospitals. The conditions under which AI is used outside hospital settings differ significantly from those within them, and failing to recognize this distinction may give rise to challenges that exceed those associated with introducing AI in hospitals themselves.
AI’s Impact on Science and the Humanities
To watch this discussion please watch here
Cross-Disciplinary Approaches
As AI becomes more powerful, it is increasingly doing what humans used to do. Bias becomes a central question, which in turns requires a consideration of how humans can be involved in the development of technology. This is a cross-disciplinary issue, in which science and technology studies should be a central part of the response. Academia therefore needs to re-invent itself, so that it is a hub for independent thinking and as a locus for debate, which facilitates collaboration around important questions.
This may require new research methods. However, cross-disciplinary work is not easy. We must determine what our assumptions and unconscious biases are, how they vary by discipline, and how we can find common ground, essentially asking what a particular discipline finds unusual. AI can be of assistance here, particularly by simulating conversations across disciplinary boundaries so that different disciplines can understand each other better.
Collaboration in Scientific Research
Another important question is whether scientists and machines can co-operate? Developments such as the AI Scientist and the Agents4Science Conference indicate that serious work is being done to explore this. This therefore requires that we re-consider the role of a scientist, and whether research teams made up of humans and AI agents can co-construct solutions together.
Data Fidelity
Underlying all of this is a concern about data fidelity and the potential for drift from accuracy as synthetic data expands and begins to feed into the generative AI training pipeline. Self-generating AI is not very viable in the long-term. It is only a technology and a tool; human creativity is still one step ahead of it. We need to ensure that we are observing the real world, not a synthetic version. We also need to ask which version of history is used to train models. This means that humanities research is vital, particularly to help understand how AI is transforming the world. STEM research tends to be very standardised. The humanities are much more diverse. They will therefore require different solutions.
The Public Interest
We should also be considering the public interest in this process, particularly so that resources are not always allocated towards the most profitable avenues, ignoring (for example) cures for rare diseases. We should therefore consider where and how the public sector and government intervenes.
We should take a long-term perspective and remember that AI is an older and broader technology than only generative AI. We also need to consider the long-term cost of AI and conduct scientific research into whether it does produce cost benefits in the long-term, as the testing, maintenance and debugging of (for example) code that it produces may outweigh the short-term savings. The usefulness of these tools varies with the user and the task; the quality is often not good enough for an expert. The effort required to verify results can be very high.
Science is in a crisis. AI could raise the quality of peer review. It can adapt to human users. It can provide support that is appropriate to the level of expertise of the user and is available at any time. It can also review much more literature than humans can. This could be a training tool. The quality of the peer review is what matters. We can adjust to this as we adjusted to Internet search and Wikipedia. The role of education is important here.
The public interest is difficult to define and therefore difficult to optimise in an algorithmic way. The field is also changing very quickly and the universities are not well equipped to respond to it. There are claims that are not peer-reviewed and there is plenty of misleading information. Experts are needed to review this but they are expensive. These tools are also dual use and therefore there are important safety questions. This means that there is a need for regulation but social values differ between countries and cultures.
We may be able to learn from the history of Internet governance, which is multi-stakeholder. Trust in the technology is a key challenge. Can this be produced by technical means? Perhaps through privacy controls, source attribution, and value alignment. Critical AI literacy is also important and should be included in school curricula.
Actionable Roadmaps
"There is a need for actionable roadmaps for this." (That was one of the more important things that was said, it might be something that AI4People could consider for a future report.)
Control over infrastructure is also important to the long-term development of AI in research. There is a risk of concentration in commercial entities. There may need to be public sector investment, as both have a role. The transparency of models is important, and the need to ensure that research is reproducible. China is leading on 'open weight' models. Europe could do more on this. We may need approaches similar to nutrition labels that make it clear what data were used to train the model, what biases might be present, and so on.
Navigating a World Transformed by AI
To watch this discussion please watch here
Human-centered AI: Rhetoric vs Reality
The panel opened with the question of what "human-centered AI" actually means in practice. Sennay Ghebreab, who has spent 15 years trying to build AI for public good, offered a direct assessment. Despite the term's ubiquity in policy documents and corporate statements, meaningful inclusion of marginalized or vulnerable populations remains rare. His own lab's trajectory, which he characterised as moving from building AI "for people" to "with people" to eventually "by people," represents an aspiration more than an achievement. Much of what passes for progress, he suggested, is "ethics washing." Institutions engage with frameworks and consult with citizens in ways that are portrayed as meaningful but often are not. What is missing, in his view, is institutional humility, meaning the willingness of academia, industry, and government to genuinely listen rather than perform consultation.
Jacob Livingston Slosser pushed the point further by questioning whether user-focused interventions at human centricity are fit for purpose. He cited empirical evidence that most people, most of the time do not notice when their judgment is being shaped, and telling them they are interacting with AI does not change this. Interventions like disclosure, literacy training, and explainability while likely necessary, may not be sufficient. This gap between rhetoric and reality was a recurring theme.
Global Power Dynamics
The discussion moved to the structural conditions shaping AI development, including economic competition, market concentration, and the institutions that might govern them.
Olaf Groth framed the global landscape with the US and China as dominant powers and uncertainty about who occupies the third position. Europe, despite leading in research and possessing substantial talent, cannot scale commercially. Its market remains fragmented, with digital services facing what Groth characterized as a 40% quasi-tax when crossing internal borders. Smaller countries including the UAE, Saudi Arabia, Kazakhstan, Israel, Malaysia, and Finland are moving aggressively to compete while Europe debates revisions to an AI Act not yet fully enforced.
The revision of that Act, which Slosser characterized as an effort to water down already modest protections, reflects a deeper tension. Groth argued that the competition between the US and China is not merely economic but a "values race," framed by key decision-makers as democracy against autocracy. The current US administration's position, he suggested, is that winning the values race requires first winning the economic race. This creates pressure to treat ethics as an obstacle rather than a design requirement.
Yet Groth resisted the framing of ethics and economics as trade-offs. Companies that integrate ethical safeguards from the design phase avoid costly downstream problems such as stakeholder backlash, class action lawsuits, and regulatory penalties. Salesforce's ethical design architects, who work alongside product developers from day one, exemplify this integration. Anthony Vetro, representing Mitsubishi Electric Research Laboratories, offered a similar analogy from sustainability. Companies once treated environmental considerations as costs to minimize, but have come to see sustainability as value-adding. Ethics could be positioned the same way, as what Vetro called a "trade-on" rather than a trade-off.
The problem is that this integration is not happening at scale. With approximately $4.6 trillion in AI investment seeking returns, companies are pursuing the low-hanging fruit by reducing labor costs through layoffs rather than creating new value. The narrative of "FTE reduction" has become a way of compromising ethics in the name of recovering investment.
Workers and the Risks of AI Automation
This is where labor enters the picture, not as an abstract category but as the concrete site where AI governance either works or fails. Aida Ponce Del Castillo noted that AI systems now handle recruitment, task allocation, monitoring, and termination decisions. Some of these systems have already attracted regulatory sanctions and penalties. But workplaces are also where countervailing power has emerged. The Writers Guild of America strike, which lasted nearly 200 days, resulted in a collective agreement giving scriptwriters authority over whether to use generative AI. A European banking sector agreement limited facial recognition and personality assessment of employees. University teachers, meanwhile, are being recorded by webcams that measure their facial expressions and rank them on perceived empathy toward students. This use of AI has not yet been collectively bargained.
Ponce Del Castillo raised a concern that went beyond specific workplace applications. AI systems are designed to extract knowledge. Collaborative robots learn skilled tasks from human workers with minimal instruction, which sounds like augmentation until one recognizes that the machine is capturing tacit knowledge, the accumulated craft expertise that distinguishes experienced practitioners. Once that knowledge is extracted, the worker becomes expendable. She cited Daron Acemoglu's Nobel Prize-winning work on the argument that without fundamentally changing how AI systems are produced, workers will never benefit from them. No existing company policy or law prevents this extractive dynamic. Major companies have announced layoffs explicitly justified by digitalization investments.
Vetro's distinction between autonomy and authority offered one way to think about limits. AI systems may operate autonomously on specific tasks without having authority to make final decisions, particularly in safety-critical or ethically sensitive areas. The "human in control" principle originated in aviation, where the context is well-defined and pilots remain in the loop. But operationalizing this for journalists, teachers, lawyers, or factory workers is a different problem, and no one on the panel claimed to have solved it.
The question of who decides what tasks AI should take and what humans should retain led to broader reflections on governance and democracy. Danielle Allen framed the challenge as one of governance capacity. Democratic institutions are straining globally, and that strain reflects how difficult it is to navigate rapid economic and social transformation, which is itself largely driven by technology. For democratic institutions to succeed, they need to renovate themselves for adaptability and resilience.
AI for Transparent Governance
Allen described democratic institutions as having an "institutional spine" with three vertebrae, namely citizen participation, decision-maker deliberation, and implementation. AI tools could strengthen each. Sense-making tools for citizen input, as used in Taiwan, help rebuild legitimacy by connecting people to governing institutions. Legislative support tools, as developed by PopVox, help overwhelmed legislators process precedent and frame decisions. Implementation tools can increase efficiency in service delivery and enforcement. But Allen emphasized that the design principle for implementation should not be efficiency alone. It must also include openness, accountability, and transparency.
Allen also advocated for framing AI as "human complementing, not human replacing," citing Pennsylvania Governor Shapiro's policy requiring this in state government. Such framing does educational work by helping people understand the choices in front of society and can develop political counterweight to concentrated technology company power.
Ziyaad Bhorat from Mozilla pushed the structural argument further. If society builds concentrated stacks of consolidated power in private corporations, it should not expect to avoid democratic backsliding. Technological structures shape social and political structures. This is why Mozilla emphasizes openness as both a technical and political principle. Open systems allow auditing, red-teaming, and distributed innovation. Bhorat dismissed arguments that proprietary systems are more secure or economically viable as "garbage," since open architectures provide security benefits that closed systems cannot match.
Mozilla published a trustworthy AI report in 2020, before the ChatGPT explosion, anticipating that AI would reshape society. The foundation now advocates for "public AI," meaning systems oriented toward public goods and community benefit rather than reducing citizens to passive users. Bhorat criticized the "move fast and break things" ethos as pernicious, arguing that true innovators think at the systems level across both technical and ethical domains. He cited Kenya's M-Pesa as an example of how resource constraints can drive innovation, and advocated for governments and philanthropies to deploy funds with the same risk tolerance that venture capitalists show. This would enable grassroots and civil society actors to experiment with new approaches rather than waiting for scaled corporate solutions.
Competing Frameworks
Arts and cultural institutions, Bhorat noted, are underrepresented in AI discussions despite asking fundamental questions about purpose. Why do we want AI? What life do we want to lead? Hollywood's labor disputes over AI show where these questions become concrete, challenging notions of what art, imagination, and creativity are.
Slosser proposed another mechanism for countervailing power in the form of data unions. These would function not primarily as mechanisms for compensation, but as sources of political leverage. If workers and citizens collectively organize around the data that trains AI systems, they might gain bargaining power that individual users lack. This remains more concept than practice, but it represents an attempt to find leverage points beyond traditional regulation.
Several participants offered theoretical frameworks for thinking about these challenges. Ghebreab drew on Paulo Freire's "Pedagogy of the Oppressed" and "Pedagogy of Liberation." Freire describes a "cycle of socialization" in which people are born into assumptions and structures that are reinforced through education and institutions. Much AI technology, Ghebreab suggested, has been socialized in this same cycle, reproducing existing power structures rather than challenging them. The alternative is a "cycle of liberation" focused on empowerment, community building, and collective transformation. If AI is approached with the aim of enabling this transition, ethical questions become design requirements rather than afterthoughts. Freire's distinction between the "ethics of the market" and the "ethics of life" maps onto the tensions the panel discussed throughout.
Pierre Lévy offered a different framing, placing AI within a broader information ecosystem. People create information that feeds digital memory, which trains AI, which empowers people to create new information in a continuous loop. Responsibility for AI's effects therefore extends to everyone who creates information that feeds training data. Journalists, professors, researchers, and those maintaining institutional websites bear particular responsibility because AI models weight authoritative sources more heavily. Bad actors understand this dynamic and exploit it by creating websites, infiltrating Wikipedia, or using traditional media to inject information that will influence AI outputs. Lévy advocated for education focused on critical thinking, personal memorization rather than delegation to machines, and awareness of how individual contributions to digital memory shape collective AI outputs.
Allen invoked the concept of "varieties of capitalism," noting that the question is not capitalism versus alternatives but rather which variety of capitalism a society chooses, and thus which regulatory approaches shape the economy. She argued that policymakers should ask how to optimize for multiple values simultaneously rather than defaulting immediately to trade-off analysis.
These frameworks, including Freire's liberation cycle, Lévy's information ecosystem, Allen's varieties of capitalism, Slosser's data unions, and Bhorat's open ecosystems, offer ways of thinking about AI governance. But the translation from framework to operational policy remained largely unexplored. The discussion generated more diagnosis than prescription.
Adaptive and Inclusive AI Governance
Frincy Clement pointed to what is missing in current governance, namely adaptive capability. Existing frameworks lack flexibility to respond to risks not yet known. While large corporations have resources to build responsible AI frameworks, small and medium businesses often lack capacity to understand what responsible AI means or how to implement it. Clement also noted that despite years of initiatives focused on education and workforce development, the percentage of women in technical roles has remained largely static, moving only one or two percentage points over a decade. She cited Dr. Joy Buolamwini's research on facial recognition failures as an example of how diverse perspectives can surface hidden harms, and mentioned recent backlash against diversity budgets while arguing that if marginalized communities were intentionally excluded historically, intentional policies are needed to bring them back.
Ponce Del Castillo noted that Europe is not alone in developing governance frameworks. UNESCO, the OECD, and the Council of Europe have issued recommendations. The International Labour Organization is planning an observatory on AI in platform work. US state-level law is developing. Trade unions, she suggested, could participate in AI risk assessment by drawing on their experience with occupational health and safety. They could serve as "data representatives" helping determine which AI systems qualify as high-risk. Workers could be involved in conducting fundamental rights impact assessments. Collective agreements can model the provisions of horizontal frameworks like the AI Act for specific sectors, since not all workers are exposed to AI systems in the same way.
To conclude
The panel ended without resolution. The tensions it surfaced, between economic competition and ethical integration, between user empowerment and structural concentration, between theoretical frameworks and operational policy, remain live. The principle that AI should complement rather than replace human work was widely endorsed, but no one demonstrated how to ensure this in practice. The empirical evidence that user-focused interventions fail to prevent influence went largely unaddressed. The concern that AI systems extract tacit knowledge from workers before displacing them produced no policy response beyond sector-specific collective bargaining.
What the panel made clear is that the gap between human-centered AI as rhetoric and human-centered AI as practice is not closing on its own.
Building Sustainable AI for a Greener Future
To watch this discussion please click here
These three experts agreed on the critical points below
However good a business or a university, working together will secure better results. To this effect, co-creation is the name of the game to improve on silos plagued with vain, closed feedback loops. AI is the unsung hero of the battle for containing climate change: too much focus on its merits as a marketing tool, not enough on what it does to secure energy efficiency.
Whatever its prowess, AI is only a tool: “The right tool for the right purpose” provides a more appropriate guidance than betting our future exclusively on AI. Quality data drives effective AI. Quality data needs quality people. The uptake of AI with people is a function of how it is perceived: enabler of inhibitor? Proper communication has yet to start to this effect.
Setting the scene
Ricardo referred to a 6-year old study documenting enabler/inhibitor perception relative to SDG targets. While the positive outruns the negative by far (79% vs 35%), the breakdown of positives by category (70% in economy, 93% in environment, 82% in society) tells that there is room to mitigate the negatives on society by enhancing the positives on environment.
Antoine made it clear that satellite imagery gets considerably boosted by unsupervised models, LLMs, etc. For instance, hyperspectral satellite imaging uses hundreds of bands to detail emissions of CO2, methane, etc. AI works wonders in mitigation as well as in adaptation.
Ganesh explained that Hitachi wants to bring harmony into innovation, e.g. via clean energy, smart mobility, etc. Hitachi is a major player in Europe, its second largest market after Japan, as testified by a number of leading-edge tech labs or its world headquarters for energy. Innovative infrastructures enhance energy efficiency. There is no dearth of challenges though: digital services, as they grow, make a growing contribution to a greener planet; however, data centers keep increasing their environmental footprint; furthermore, some CO2 emission sources happen to be embedded in hardware. In order to meet these challenges, sourcing should be better documented; goals should come with detailed KPIs and proper measurement tools; transparency should be ensured via optimal public disclosure. No doubt AI helps energy optimization: smart grids secure a better balance between supply and demand at all times; smart mobility helps minimize investment via dedicated platforms; AI-powered tools enable energy saving, from raw materials throughout the entire lifecycle of products.
The way forward: Sharing
For Ganesh, sharing is the way to go. Silos are the enemy since they are impervious to meaningful feedback. In contrast, co-creation between government, business and civil society (whether academia, NGOs, or other structures) spurs on creativity; it needs only proper platforms to thrive, and AI is there to help. Europe is doing well in this respect.
Ricardo observed that academia-business relations are more productive in the US. It is more a matter of mindset than money: in Europe, universities are all too often seen as either intimidating partners or cheap consulting whereas they have no match when it comes to unbiased, out-of-the-box thinking.
While collecting data from the very top of outer space, Antoine praised the merits of bottom-up creativity born out of engaging with local governments (in wild fires prevention and containment, for instance) and leading universities across the region.
Quality matters
For Antoine, AI-driven productivity is a direct function of the quality of algorithms and of the data they feed on. Year after year, you can see the impact of improved technology on how your services will perform. For example, foundation models have significantly decreased needs for energy. Suppliers of high-quality data cannot see the demand for it plateau anytime soon.
Gamesh sees merits in using the right tech for the right purpose. For instance, edge computing may provide extremely competitive solutions with smaller resources. Near-zero failure operations are enabled by quick progress towards AGI, synthetic data modelling, etc. However, reality checks are often needed for better performance, which sort of debunks the belief that AI systems operate mostly on their own: people are actually the systems’ gateway to the myriad aspects of human life.
Europe vs rest of the world
Antoine mentioned the magnitude of AI-related investment made in the US, Asia or the Middle-East: Europe pales in comparison. “We think too much”, he suggested. Only action can make a difference. In this regard, bureaucracy lurks at every step of the way: State aid and other regulations, for example, may act as a serial killer on projects that look promising to their supporters. AI leadership depends on quality models. The EU has yet to fully grasp this notion and to shape a global vision accordingly.
Ricardo agreed that the EU could be more flexible in the way it manages its generous funding: scaling up - often quoted as a liability in Europe’s industrial landscape - would be easier.
Gamesh contributed an interesting explanation on how Japan combines its time-honoured reverence for the benefits of taking the long-term view with the need to meet the goals of the next quarter. Different targets are set on 3-year, 10-year, 50-year; they are addressed and managed independently by business and government, although properly coordinated as illustrated in Society 5.0.
To Conclude
Antoine suggested that Brussels needs a better plan driven by modern-day reality.
Ganesh confirmed that open models based on sharing skills and experience will make AI more effective, hence our future greener. Ricardo stated that AI-powered dialogue cannot but help our data-driven societies to steer the course they choose.
Creating a Trustworthy AI-Enabled Future
We are pleased to share insights from Burkhard Schafer, Professor of Computational Legal Theory at the University of Edinburgh, which emerged during discussions at the latest AI4People Summit. The session, titled 'Creating a Trustworthy AI-Enabled Future,' brought together speakers dedicated to ensuring that AI development remains fair and transparent.
This included David Danks, Professor of Data Science, Philosophy, and Policy at UCSD; Andreas Kaminski, Professor of Philosophy of Science and Technology at Technische Universität Darmstadt; Jo Pierson, Full Professor of Responsible Digitalisation at Hasselt University; Ann Gregg Skeet, Senior Director of Leadership Ethics at the Markkula Center for Applied Ethics at Santa Clara University; and Suresh Venkatasubramanian, Computer Scientist and Professor of Data Sciences and Humanities at Brown University.
To watch this discussion please click here
Trustworthiness as a Contextual, Second-Order Value
A key theme of the discussion was the need to move “trustworthiness” beyond mere technical reliability towards a richer, more layered concept. “Trustworthiness” is not a simple measurable attribute like “weight”; rather, it is a highly context- and sector-dependent “second level” value that “quantifies over” other values. We trust someone if they are honest, fair, courageous, careful, and protective—WHEN the situation requires it. Or if they display transparency, fairness, privacy, reliability, and explainability. We can also call an AI or any other system trustworthy if it displays transparency, fairness, privacy, reliability, and explainability—again, when these values are important for us in a given context.
Trust is created when we recognise that the other side is guided by the values that matter to us. This can be extended to the recognised values that guided the development of AI – we trust the AI if we recognise that its developers share the same set of values that we have.
This recognition entails vulnerability. Once we trust someone to share our values, we expect them to behave in appropriate ways either towards us or to the people we care about. This context sensitivity (sometimes we care about privacy, sometimes we don’t, etc) also means that regulatory tools like sandboxes will fall short, and only concrete real-life examples that look at everybody affected by an AI, not just its deployer and the directly affected party, are heard. Also, within company-led ethics principles, what is often still missing is environmental harm, investment in humanity, and other third-party concerns, dimensions that extend beyond those contracted in a commercial exchange.
Public vs. Private Trust and the Importance of Institutional Context
This raises the issue of context: we may not trust a private company with our personal data and only share it with their AI if there are safeguards protecting it, but we might also be happy to have this data shared, again responsibly, with a regulator who controls and oversees the company as a public sector organisation.
This “public trust” is confidence we can have in AI even when we are not directly affected by its decisions. It underpins all values – privacy safeguards an individual on social media, but also protects wider society and their interest in a functioning democracy, where people can express their opinions without the fear of immediate profiling.
Measurement, Evidence, and Scientific Framing
Developing these systems also requires a proper frame of reference, discussing them in ways that are scientific, replicable, testable, and overall sound. The ability to distinguish between reality, aspirational thriving, and “fever dreams” is important to ensure the AI ethics debate targets the right issues. This then leads to precise measurements of what the system does and, crucially, what it does not do once integrated into a real-world application. What makes AI trustworthy is not that its reliability was tested in a sandbox, but how every stage of its development and deployment prioritises the needs of the community it is built for and how they use it. This also requires complex measurement infrastructures, not controlled solely by the company that develops the AI. And whenever the measurements identify the problem, we need the right tools to address it. Trustworthiness is therefore not just an assurance that the system works reliably, it is also a recognition that systems can and will fail, and that in such cases, there are efficient tools and interventions that will be used to rectify the problem. These interventions can and often will come from third parties, including the state. Trustworthiness can only be understood correctly if we see it as a network of responsibility, with checks and balances that ensure the unity of human values is always protected, even if the AI has failed.
Making an AI fully trustworthy is beyond the company's power, even though they can and should work towards that goal. However, trust ultimately also depends on public institutions that oversee and regulate AI beyond its developers and deployers.
Conclusions
Ethical principles and frameworks remain vital though; many organisations (including numerous universities!) either lack them entirely or fail to implement them effectively. They need to be more detailed and precise, yet also genuinely adopted by the developer’s company and the broader environment, so that their normative values align with the production process. The key here is also to emphasise the human element; we utilise technology, but we manage humans, not the other way around.
The AI4People Playbook: Implementing a Proportional Approach to Ethical AI Requirements
On December 2nd, at the AI4People Advancing Ethical AI Governance Summit, we presented the AI4People’s Playbook: Implementing a Proportional Approach to Ethical AI Requirements.
This publication represents an important milestone in our ongoing efforts to promote the responsible development and deployment of AI systems across sectors.
The aim of the Playbook is to promote a balanced and pragmatic approach to core ethical principles, one that goes “beyond regulatory compliance”. By offering actionable recommendations, highlighting existing resources, and showcasing real-world use cases and activities, the guidance emphasizes that adopting an ethical perspective is the most effective way to prevent costly mistakes.
The AI4People Playbook breaks down AI ethics into actionable steps, and it includes instructions and practical material to support capacity-building activities, empowering you to put principles into practice effectively. While many resources support responsible AI, from the AI Office’s GPAI Code of Practice to various global standards and simplified guides, this playbook stands out for its clear, straightforward message: AI ethics goes beyond compliance, and can be applied proportionally, regardless of how your AI system is classified.
This Playbook is designed for a broad range of audiences involved in developing or interacting with AI systems. Whether you are new to AI ethics or an experienced professional looking for practical guidance, this playbook intends to provide valuable information, hands-on activities, and a curated list of resources to support you.
This playbook is intended as a beta version, and a living document. It will continue to evolve with the growing capabilities of AI, input from users, stakeholders, and interested parties.
We look forward to continuing this work collaboratively and refining the Playbook together with the wider community: please share your thoughts with us at info@ai4people.org.
AI4People Summit 2025: The Special Session on Defending Society from Disinformation and Misinformation
MEET THE SPEAKERS: Anthony Vetro
