How do you create trustworthy AI?

How do you create trustworthy AI? It all starts with your data. The best algorithm paired with a poor dataset is just asking to produce undesirable outcomes. Your training data needs to be well curated. This is arguably the most time consuming & costly part of the process when building AI solutions that people can trust. Your outputs are only as good as your inputs. There should be subject matter experts engaged for the curation of data & ideally, more than 1 check & balance in place to validate work. You don't want 1 person solely being responsible, without oversight, to curate data, as this creates one singular failure point for the quality of your dataset and subsequently, your AI solution. 👍 Like to support this #ai #leadership

The first step is identifying the end goals to deploying an AI system. Within those goals, identifying data gaps, data risk, misleading data points is key. Deploying an AI solution without understanding the value proposition (both in terms of company values, as well as value delivered through the solution) is as good as time wasted.

To build a trustworthy AI: enforce on high-quality, data without any bias, transparent and explainable pipelines, robust security, ethical compliance, a fair mitigation processes, accountability to high degrees, continuous monitoring, stakeholder engagement (if it is in a company), highest level of ethical standards, and ongoing research and development may be couple of points to consider

I think trustworthy AI requires governance and regulatory compliance throughout the AI lifecycle, from ideation to design, development, deployment, and machine learning operations. Trustworthy AI also requires the use of privacy-enhancing technologies, such as homomorphic encryption, federated learning, differential privacy, and zero-knowledge proofs. Trustworthy AI is essential for building trust among individuals, organizations, and societies, and for maximizing the benefits and minimizing the risks of AI.

I agree the regulations and ethical framework should already be in place, these is critical to manage the development of any AI solutions.

Given this definition, are any systems neutral or objective? Even a non-AI system, such as one based on rules or hard-coded heuristics, would reflect the assumptions and ideas of the creators (including potential biases). For example, someone could have rules for credit scoring that, by virtue of the thresholds or similar, end up discriminating against certain segments of society. From a testing perspective, I don't think that there is anything in this paragraph that doesn't apply equally to non-AI based systems: you would also want to test with different people and against different datasets to ensure it performs as expected.

From the many examples we see, AI systems are very capable of bias. This isn't a fault of the AI system, but the data sets they were trained on. Creating and further training/reinforcing the data sets will ensure better trained models that are diverse. These can be perspectives from different demographics, removing negative biases that are harmful to some demographics, or even retraining models to provide a better representation of diversity across domains.

Title: Building Trustworthy AI: A Roadmap In the age of AI, trust is paramount. To ensure ethical and responsible AI development, we need a roadmap focusing on: Transparency: Document the AI process and provide clear explanations. Fairness: Identify and correct biases, especially in sensitive domains. Privacy: Safeguard user data and communicate data usage clearly. Accountability: Hold developers responsible for AI failures and ethical violations. Security: Protect AI models and data from cyber threats. Collaboration: Foster cooperation among developers, ethicists, and policymakers. Continuous Learning: Stay updated on AI ethics and regulations. Ethical Frameworks: Build AI within established ethical guidelines. .

Great point, would add social inclusiveness from an end user perspective, does your AI service/ product conform with social inclusiveness - aka doesn’t leave behind certain segments of the society!

Transparency also means being clear about how data is collected. Users should know to what extent their input is used as training data. Such transparency should apply to both paid and "free" AI products.

I would add that documenting the code and publishing it is crucial for highly transparent and explainable AI models. The code documentation shows that the model was developed with transparency in mind.

These principles should apply to all systems but they are perhaps more complex in the case of deep learning-based approaches because they are data driven. If a decision is made, particularly by a neural network, then knowing why the decision was made is challenging and being able to explain it is even more challenging. Hence, there needs to be more research in how to build explainable systems such as leveraging some of the symbolic methods of "classical AI" (which can be more expressive in terms of explanation) in tandem with deep-learning based approaches.

Not all data can be transparent (especially in cases where data security is important). However, when using generally available models, it's important to know and share the sources, training methods, and expected outcomes of your AI solution. This will make stakeholders more confident in the solutions you provide.

Data Authenticity matters: data authenticity throughout the AI development lifecycle, organizations can build AI systems that are not only accurate and effective but also trustworthy. Trustworthy AI inspires confidence in users and stakeholders and minimizes the risks associated with unreliable data sources or data manipulation.

To assess bias in AI, conduct data audits ensuring representation across demographic groups. Use fairness metrics like demographic parity and equalized odds to measure model equity. Specially in News Media where the most important parameter is trust, analyze content sentiment and topic diversity to detect skewed coverage. For mitigation, employ explainable AI techniques, ensuring transparency in decision-making. Continuously monitor and recalibrate models based on feedback. Prioritize diverse news sources and regularly audit recommendation algorithms. Engage with diverse stakeholders for feedback, ensuring a holistic approach to fairness.

Biases happen because of the data that the AI model is trained on. Human reinforced learning methods, giving better data sets that reflect a diverse demographic and perspectives and data quality will help minimize bias.

6 steps to assess and mitigate bias and discrimination in AI: - Data Audit: Thoroughly examine training data for biases related to race, gender, or other protected attributes. - Data Sampling: Ensure diverse and representative data samples to reduce skewed representations. - Bias Metrics: Develop and use bias metrics to quantify and detect disparities in model predictions. - Regular Testing: Continuously test and validate models for fairness using real-world scenarios. - Bias Reporting: Establish mechanisms for reporting and addressing bias concerns from users and teams. - Ongoing Monitoring: Continuously monitor AI systems and reevaluate mitigation strategies to adapt to changing conditions.

The main challenge isn't necessarily that AI can harm privacy but that to develop deep learning, you need to acquire vast amounts of data. This data is what creates the privacy challenge: potentially it's a massive honey pot for attackers and also useful for many things beyond the core use case that a user consented to. As such, managing consent, managing revocation of that consent, ensuring the appropriate guardrails prevent usage beyond those parameters of consent, etc. are all key considerations alongside the obvious cybersecurity challenges. In the future, the use of emerging cryptography like fully-homomorphic encryption may allow models to be trained on data whilst encrypted which may mitigate some of the risks mentioned above.

Here you probably need to go beyond the requirements set by existing laws to offer greater clarity than you see in most user agreements. Data you offer these systems is not just demographic. The AI is ingesting your ideas - the way you’ve mixed and matched concepts- the way you communicate and what you prioritize. Even if it’s not company data, it’s all your own personal, proprietary stuff. All of that may be just fine, but the user should know what it means. You should not just understand how your data will be used, but also the implications of that use.

There are both inward and outward facing security concerns. There are some famous recent cases where internal data was exposed because of bad security practices when interacting with external AI applications. Using AI models trained on internal data, and having secure systems that allow logged and authorized usage help greatly in mitigating security risks. For privacy, companies need to ensure that (as an example), any data that their clients provide, either in the form of documents, or their own personal data is encrypted and only used in tandem with secure AI systems. It's very easy to leak private data, and if that happens, trust with that organization will fall.

Totally agree that how your proprietary information is stored (and where) data shared and distributed is critical so that no legal actions will happen based on risks not identified during the AI selection/ build stage.

The evaluation and measurements should be crystal clear from AI design and development, this is key to ensuring the continuous improvement of the solution and testing all is working as planned.

You cannot improve something unless you measure it. Along side the AI system, It’s important to have the right metrics defined and regularly reviewed!

We tend to be scared of items that we do not fully understand. We should consider adding "AI Labels to our LLMs" ( like food labels). We should label each LLM with (the data set used, limitations, pros, cons, and biases of the model..). This will increase trust between different LLM models and help consumers educate on its limitations and capabilities. I know this is difficult to do, but this can help create the needed trust in AI.

Trustworthy AI is about the practice of developing and providing AI outcomes with good intention, ethical values to empower its users, fairly, positively impacting society-allowing people to use AI with confidence. As with any transformational technology, there are risks involved with AI too and these will change (improve) over time. Organisations need to ensure that AI deliverables perform as intended, they should be safe, should not create negative bias or discrimination or inequality in society. Deliverables of AI should not be misapplied or used in harmful ways. Organisations delivering AI products need to recognise above responsibilities and measures need to taken by them to mitigate mentioned risks and hazards - Trustworthy AI.

In my experience as an AI risk manager creating trustworthy AI is only possible in a controlled environment where one project manager manages the project from start to finish. Where programmers are not changing monthly. Where extensive testing is done. Last but not least important outside audit is imposed.

Privacy protection is crucial for creating trustworthy AI. By implementing robust data anonymization techniques, encryption methods, and ensuring compliance with data protection regulations such as GDPR or CCPA, AI developers can safeguard user data and build trust. This safeguards sensitive information, prevents unauthorized access, and respects individuals' rights, contributing to AI's reliability and credibility.

Suggest adding governance & controls, whereas accountability is nice, but what are measures/ controls in case AI service doesn’t operate as intended! I.e autonomous driving - give back control to driver?