Giving Agents 100% Autonomy Will Only Lead to Failure
The algorithm can automatically discriminate against a potential customer due to outdated historical datasets. Post-campaign analysis can help identify biased decision-making and will enable revenue uplift, says Toju Duke, Founder & CEO, Diverse AI.
Topics
It’s much easier for diverse teams to identify issues of representational harms, fairness, stereotypes, privacy, and data inconsistencies, says Toju Duke, Founder & CEO, Diverse AI, a community-interest membership organisation focused on driving diversity in the AI field.
As artificial intelligence rapidly becomes a core engine behind marketing decisions, from audience segmentation to targeted advertising and strategic insights, questions around fairness, bias, and accountability are becoming harder to ignore.
Toju shares how organisations can adopt responsible AI practices, why diverse teams are essential to ethical deployment, and how human oversight will become increasingly critical as autonomous AI agents scale across industries.
Excerpts from the interview:
What are the possible ethical considerations when using machine learning models and applications for marketing use cases like segmentation and targeting?
Bias and discrimination are common challenges associated with AI in marketing, and also apply to segmentation and targeting. AI for decision-making processes has been known to unfairly discriminate against marginalised groups, mainly due to insufficient datasets, algorithmic biases and outdated historical data.
The majority of the datasets used to build and train AI models in any industry or domain are usually unrepresentative of the world we live in, and tend to replicate only the lived experiences of the developers who built them, who mainly come from the West, particularly the United States and Silicon Valley.
As a result, biased decisions are made which selectively display online ads, for example, based on the data and demographics of new and existing customers.
For example, an advertising algorithm chooses to show an ad for a Jordan 4 Retro ‘White Thunder’ trainers, which costs a few hundred dollars, to a 20-year-old white male who lives in a well-known, expensive neighbourhood.
The Jordan ad is never displayed to a 20-year-old woman of African American descent because of the historical data on the neighbourhood she lives in, which was once considered one of poor socio-economic status, but has now changed to a neighbourhood for the middle and upper class.
The algorithm automatically discriminates against a potential customer due to outdated historical datasets. This is a clear example of biased targeting and segmentation, which marketers need to be aware of and ensure post-campaign analysis is conducted for fairness and biased decision-making.
This will also enable revenue uplift as the right ads will be shown to all existing and potential customers while driving customer loyalty and satisfaction.
Adhering to privacy regulations in each country is also important to ensure customers’ personal identifiable information (PII) is not compromised, especially if generative AI technologies are used.
What are the guardrails for business leaders who want to use insights using AI models for strategic decision-making to ensure brand safety and purpose?
Audience insights are also a product of data and datasets. Insights could have issues with privacy and bias. To ensure brand safety and align with business goals/purpose, business leaders should implement ongoing monitoring and safety measures.
For example, checking the demographic settings of audiences that have already purchased a product to ensure there’s no disparate impact and unfair treatment on new and potential customers.
Business leaders should also have an AI policy in place, which states the organisation’s intentions, goals, and objectives for AI and have clear guidelines and directives that employees can work with.
These guidelines should include monitoring and testing, ongoing human reviews (also known as human-in-the-loop), and privacy techniques. These guardrails should be used across all AI domains irrespective of industry, and must be implemented if working with AI agents.
It’s also important to ensure agents are never given 100% autonomy and there’s a defined process for failure modes.
Do diverse teams (AI practitioners and tech leadership) see more success in inclusive deployment or the use of AI responsibly?
Yes, as diverse teams are able to deploy AI with their varied and diverse lived experiences and knowledge.
It’s much easier for diverse teams to identify issues of representational harms, fairness, stereotypes, privacy, data inconsistencies and so on.
It’s also been proven that diverse teams are critical to an organisation’s success, and having diverse leadership drives innovation and increased profitability, where research has shown a 66% increase in creativity, 87% increase in problem solving, and 39% increase in profitability.
How can enterprise or private businesses move the needle towards more responsible use of AI?
It’s important to ensure there is an AI policy in place and all employees are aware of the policy to avoid several mishaps. There’s a recent example of an organisation that didn’t have an AI policy in place, and employees were uploading private and confidential customer information to ChatGPT.
Of course, this violates privacy laws and regulations and can lead to heavy fines from the EU, as an example. The business in question is also at risk of heavy customer loss, which will also affect investor confidence and brand reputation.
Company-wide training on Responsible AI is also required to increase AI literacy and successful AI campaigns. As AI is still a flawed technology, it’s important that organisations and all employees are aware of how to drive productivity, efficiency and profitability using AI.
A plug-and-play approach doesn’t work and has led to several failures across organisations where revenue, employee productivity and brand reputation have been negatively impacted.
The AI race is fast-paced. How do you see the role of the Humans-in-the-loop (HITL) evolving?
Humans-in-the-loop (HITL) will become more important in the coming months, as AI agents are deployed at a wider scale. AI agents are a spin-off from LLMs, which are still quite inaccurate, are unable to carry out human reasoning tasks, and are full of hallucinations and false information.
Due to this, agents need more supervision, as giving agents 100% autonomy will only lead to failure. There is a recent story of a researcher at Meta who was using a well-known agent called OpenClaw, and at some point, the agent decided to delete all her emails from her inbox.
Despite running to her device to stop it, it refused to take instructions and cleared out her inbox. This is what happens when agents are given full autonomy while they’re still half-baked, flawed, and there are no fail-mode interventions in place.
There’s also a second example from Amazon where their AI agent called Kiro caused a 13-hour outage on AWS because it independently decided to “delete and recreate the environment”. A similar failure occurred a few months ago at Amazon, which cost the business $1 billion dollars and affected over 2000 businesses.
The need for HITL, human intervention and human expertise will be heavily required as more businesses suffer failures from autonomous AI.
Tell us a bit about your work with Diverse AI.
Diverse AI is a community-interest membership organisation, with over 630 members from across the world.
We’re focused on driving diversity in the AI field and providing diverse AI solutions to reduce some of the risks and harms associated with the technology. Our approach is three-fold, where we’re focused on education, research and communities.
Over the past 3 years, we’ve impacted over 2000 people globally and hosted 19 in-person events across the UK, which include developing AI literacy skills and education amongst diverse and marginalised communities.
We also recently launched a 14K diverse image dataset called Diverse Spectrum that could be used across all industries working on computer vision to improve the output of their models, reduce biases, incorrect outputs and results, and consequently human rights violations.
We’re currently creating a second image dataset (Diverse Spectrum v2), which will contain 6.8 million images from 34 countries in Africa and Asia.
This dataset will represent offline populations in global AI systems, reduce algorithmic bias, enable locally owned businesses and sovereign AI solutions, while bridging the gap on digital colonialism and inequality.
ALSO READ: Cross-Border Success Through Hyper-Localisation and Geo-Targeting
