Diaries from the Field - Building Ethically Responsible AI Enterprise Solution

Welcome to your weekly AI Newsletter from AITechCircle!

This newsletter has become an essential resource for me and many others in the AI community. It has practical insights that will immediately boost your work or business. 

Dive into this week’s updates, and take a moment to share them with a friend or colleague who could gain from these valuable insights! 

Today at a Glance:

• Behind the Scenes of writing this newsletter 
• Insights from the Field: Creating Ethical AI Solutions for Enterprises
• Generative AI Use cases in the Health Care Industry
• AI Weekly news and updates covering newly released LLMs
• Courses and events to attend

Today is the 48th edition & Behind the Scene.

I started this newsletter to keep myself informed and all the info in one place in the field of AI/ML and Gen AI. The field is moving as fast as it is, and keeping informed about the latest info and trends is challenging. 

As today is the 48th edition and looking back on the difficulties of consistency, I started writing this on usually Friday evening/night and was ready to be published on Saturday afternoon. A few months later, this practice was going fine, and then it shifted to writing on Saturday and publishing on Saturday night. After a few weeks and months, it was moved to Sunday to write and then publish it later in the Night. Finally, this slippage led to last week even not being able to write on Sunday and postponing it to Monday, which never came :)

I am back with Friday night writing and publishing on Saturday this week. There are several lessons to learn from this, even in managing the writing schedule, and we will talk about the challenge of deciding on a topic and even what to write. 

It's a journey that I started, and this is how you take small steps to achieve or accomplish some more significant tasks, goals, or targets with smaller steps every day.

Building Ethically Responsible AI Solution

I had a chance to prepare for a session on AI ethics and the challenges of Bias in AI solutions. I thought of sharing a few insights with you all. While working in the field with customers, I can relate to a few of the critical areas being asked about as part of the AI solutions to get visibility on AI ethics and bias and how the AI solutions are being managed. 

Let's first get what is being asked; questions are being asked.

As organizations are building or have built their AI strategies, and every AI strategy has an Ethical AI / Responsible AI pillar, a few examples to look at are:

As per the research and current situation, several studies have been presented that cover the critical areas of ethical AI, bias, and risks. A few to mention here are:

1. The German Federal Office for Information Security published the report "Generative AI Models - Opportunities and Risks for Industry and Authorities covered during the earlier edition "Key Risks Associated with Generative AI" " This report covers a few areas, such as the planning, development, and operation phases of generative AI models, where a systematic risk analysis should be conducted.
2. Common ethical challenges in AI published by Council of Europe.
3. Challenges of Managing AI bias, published by NIST Special Publication1270 Towards a Standard for Identifying and Managing Bias in Artificial Intelligence, outlines, “Current attempts for addressing the harmful effects of AI bias remain focused on computational factors such as representativeness of datasets and fairness of machine learning algorithms. These remedies are vital for mitigating bias, and more work remains. Yet, human and systemic institutional and societal factors are significant sources of AI bias as well, and are currently overlooked. Successfully meeting this challenge will require taking all forms of bias into account.”
4. Systemic biases arise from institutional practices that advantage certain social groups while disadvantaging others, often due to established norms rather than intentional prejudice. Common examples include institutional racism.
5. Human biases are systematic errors in thinking, often stemming from heuristic shortcuts and simplified judgments. These implicit biases affect how individuals or groups interpret information, such as AI outputs, to make decisions or fill in gaps. They pervade decision-making at institutional, group, and individual levels throughout the AI lifecycle and using AI applications post-deployment.
6. Statistical and computational biases occur when a sample fails to represent the population accurately, stemming from systematic (not random) errors without intentional prejudice. In AI, these biases emerge in datasets and algorithms, often when models trained on specific data cannot generalize beyond it. Causes include diverse data types, simplifying complex data, incorrect data, and algorithmic issues like overfitting, underfitting, outlier handling, and data cleaning practices.

The current state of AI Ethical Consideration

It is a multidisciplinary field that studies how to optimize AI’s beneficial impact while reducing risks and adverse outcomes. 

There are key AI considerations kept at the heart of any framework or regulations:

• Transparency and Explainability: Making AI decisions understandable and traceable.
• Privacy: Protecting user data and personal information.
• Accountability: Holding developers and organizations responsible for AI outcomes.
• Safety and Security: Preventing AI systems from causing harm, intentionally or unintentionally.
• Human Oversight: Maintaining human control over AI systems to prevent unintended consequences.

How to monitor specific projects don’t breach ethical boundaries:

As organizations increasingly adopt AI to drive innovation, one critical question emerges: How can we ensure our projects adhere to ethical standards? The above situation has made us realize that several efforts are being made at every level to provide AI frameworks to monitor and ensure responsible AI. 

I put some basic steps to help us monitor AI projects effectively. Monitoring AI projects effectively isn’t just a regulatory necessity; it’s about building trust, ensuring fairness, and safeguarding your organization’s reputation.

1 - Establish Clear Ethical Guidelines & Ethics Review Board:

Components:

Ethical Guidelines:

• Data usage policies
• Decision-making frameworks
• Compliance with legal standards

Ethics Review Board:

• Diverse experts from relevant fields
• Inclusion of external stakeholders for unbiased perspectives

Benefits:

• Provides a clear ethical framework for AI projects
• Facilitates consistent decision-making
• Enhances stakeholder trust

2 - Implement Continuous Monitoring Systems:

Components:

• Automated monitoring tools
• Regular audits and assessments
• Feedback mechanisms for stakeholders

Implementation:

• Integrate monitoring tools into AI systems
• Schedule periodic audits
• Establish clear protocols for addressing identified issues

Benefits:

• Early detection of ethical deviations
• Facilitates timely interventions
• Maintains ongoing ethical compliance

3 - Conduct Regular Training and Awareness Programs:

Components:

• Workshops and seminars
• Online courses and resources
• Case studies of ethical dilemmas

Implementation:

• Develop tailored training programs for different roles
• Encourage continuous learning and discussion
• Assess training effectiveness through evaluations

Benefits:

• Fosters a culture of ethical responsibility
• Enhances stakeholders' ability to identify and address ethical issues
• Keeps the organization informed about emerging ethical challenges

4 - Ensure Transparency and Accountability:

Components:

• Detailed records of data sources
• Documentation of decision-making processes
• Public disclosure of AI system functionalities

Implementation:

• Establish standardized documentation practices
• Provide accessible information to stakeholders
• Encourage external reviews and feedback

Benefits:

• Builds trust with stakeholders
• Facilitates accountability and responsibility
• Enables external validation of ethical compliance

5 - Adopt Ethical AI Frameworks and Standards:

Components:

• Adherence to AI frameworks, standards, and regulations
• Integration of ethical principles into AI development
• Regular updates to align with evolving best practices

Implementation:

• Evaluate and select appropriate frameworks for the organization
• Train teams on the application of these frameworks
• Monitor compliance and make necessary adjustments

Benefits:

• Provides a structured approach to ethical AI development
• Ensures consistency across projects
• Demonstrates commitment to ethical standards

Call for Action:

Based on my thoughts, research, and observations, I would like you to reflect on how you currently monitor your AI projects or plan to do so from an ethical perspective. 

Share your journey so far as we all navigate this relatively new road to the city of AI opportunities. 

It's vital that you share your tale of this journey; by doing that, we can collectively ease the path and uncover strategies to ensure that our ethical considerations keep pace with AI's potential.

Next week, I will cover the technical aspects of which tools and libraries you can use for ethical AI, bias monitoring, and risk mitigation.

Weekly News & Updates...

Last week's AI breakthroughs marked another leap forward in the tech revolution.

1. ᴏᴘᴇɴꜱᴄʜᴏʟᴀʀ: a retrieval-augmented LM to help scientists synthesize knowledge. It has a datastore of 45M+ open-access papers and a specialized retriever and reranker to search the datastore. It is based on an 8B Llama fine-tuned LM trained on high-quality synthetic data with a self-feedback generation pipeline. link
2. Multi-model choice for GitHub Copilot is available now link
3. AlphaQubit: AI-based system that can more accurately identify errors inside quantum computers link

The Cloud: the backbone of the AI revolution

• Announcing Cohere Command R and R+ 08-2024 models on OCI Generative AI link
• Potential benefits of using NVIDIA NeMo Retriever and Oracle 23ai to power your enterprise RAG pipeline link
• 2025 Predictions: AI Finds a Reason to Tap Industry Data Lakes linkt

Gen AI Use Case of the Week:

Implementing LLMs for medical coding aligns with the strategic goals of improving operational efficiency, reducing costs, and enhancing revenue cycle management. Simplifying Claims Submission - Medical Coding with LLMs Using Generative AI.

To access the library of Gen AI Use cases, link here:

Chief AI Officer (CAIO) Corner:

AI Magazine has published a prominent 10 Chief Artificial Intelligence Officers (CAIO), worth checking their profile over here

Favorite Tip Of The Week:

Here's my favorite resource of the week.

Adopting AI Responsibly: Guidelines for Procurement of AI Solutions by the Private Sector, published by the World Economic Forum.

The AI acquisition framework aims to ensure that commercial enterprises acquire AI systems based on ethical principles, supported by robust governance to apply those principles effectively, reduce bias, and enhance resilience.

It provides a structured framework for evaluating the implications of acquiring AI solutions, emphasizing transparency, accountability, and human-centered design throughout the development and implementation process.

Potential of AI

AI Ethics: Enable AI Innovation With Governance Platforms from Gartner

AI ethics depend on robust governance platforms to offer organizations a competitive edge. While businesses rapidly adopt AI to enhance efficiency and performance, they face challenges related to ethics and bias that demand effective governance practices. 

Ethical AI frameworks and platforms can address these concerns and ensure responsible AI usage.

Things to Know...

The Open Source Initiative Announces the Release of the Industry’s First Open Source AI Definition.

"An Open Source AI is an AI system made available under terms and in a way that grant the freedoms to:

• Use the system for any purpose and without having to ask for permission.
• Study how the system works and inspect its components.
• Modify the system for any purpose, including to change its output.
• Share the system for others to use with or without modifications, for any purpose."

The Opportunity...

Podcast: 

• This week's Open Tech Talks episode 148 is "Strategic AI Adoption for Businesses with Nick Jain." The CEO, Idea Scale

Apple | Spotify | Amazon Music 

Courses to attend:

• Stanford CS 234 Reinforcement Learning Spring 2024, by Emma Brunskill

Events:

• Global AI Show, Dec 12, 13, 2024, Dubai, UAE

Tech and Tools...

• flux This repo contains minimal inference code to run image generation & editing with Flux models
• screenpipe rewind.ai x cursor com brings an AI assistant that has all the context

Data Sets...

• SPORC: the Structured Podcast Open Research Corpus. is a large multimodal dataset for studying the podcast ecosystem.

Other Technology News

Want to stay updated on the latest information in the field of Information Technology? Here's what you should know:

• Is AI hitting a wall?, as published by TheVerge

Join a mini email course on Generative AI ...

Introduction to Generative AI for Newbies

Earlier week's Post:

3 Questions Ask by the Executives for AI

Building a Game-Changing AI Strategy: Step-by-Step Guide and Exercises for Your Organization

Actionable Responsible AI Maturity Roadmap

LLMs: How open are they really?

Generative AI - Opportunities & Impact on Children

Basics of RAG

Building RAG-Based Chatbots

Chat with your Data in the Database without writing SQL

Data Analysis with LLM Agents

And that’s a wrap!

Thank you, as always, for taking the time to read. 

I’d love to hear your thoughts. Hit reply and let me know what you find most valuable this week! Your feedback means a lot.

Until next week, 

Kashif Manzoor

The opinions expressed here are solely my conjecture based on experience, practice, and observation. They do not represent the thoughts, intentions, plans, or strategies of my current or previous employers or their clients/customers. The objective of this newsletter is to share and learn with the community.