Curtis Northcutt’s Post

Goodbye Hallucinations! Today, Cleanlab launches the Trustworthy Language Model (TLM 1.0), addressing the biggest problem in Generative AI: reliability. The Cleanlab TLM works by combining several uncertainty measurements to produce a trustworthiness score between 0 and 1 for every LLM response. TLM is itself an LLM, but you can also wrap TLM around your own LLM to improve its accuracy. Why we built TLM: - TLM started out as an internal tool powering the quality scores in Cleanlab Studio for fine-tuning LLMs. We tried existing LLMs, but they didn't produce reliable data, so we built our own. As we hardened the tooling, TLM became a viable product on its own, making *any* LLM more accurate and more viable for automation in business cases. Use Cases: - Use like any LLM API: `tlm.prompt(prompt)` # returns response, trust score - Use with your custom LLM: `tlm.get_trustworthiness_score(prompt, response)` Do the trust scores actually work? - Yes! By filtering by large trust scores, accuracy improves. View the benchmarks in our blog, linked in the comments. Does TLM improve the accuracy of any LLM, too? - Yes! Again, by filtering by larger trust scores, accuracy improves. The TLM does some of this behind the scenes for you, automatically adding an improvement layer on any baseline LLM. What's the catch? - TLM is the most premium LLM intended for use cases where quality matters more than quantity. Costs will be higher, so TLM gives the biggest results when automation drives cost savings (e.g. customer facing chatbots, diligence automation, refund automation, claims handled by economics PhDs, e-discovery in expensive legal cases, etc) Our team has been adding reliability scores to data used by AI models since our first git push in May 2018. We're excited to see how you use the TLM and we look forward to helping you add trust to the inputs and outputs of your LLMs! Try it here: https://cleanlab.ai/tlm/ #llm #genai #hallucinations #generativeai

Goodbye Hallucinations! Today, Cleanlab launches the Trustworthy Language Model (TLM 1.0), addressing the biggest problem in Generative AI: reliability. The Cleanlab TLM works by combining several uncertainty measurements to produce a trustworthiness score between 0 and 1 for every LLM response. TLM is itself an LLM, but you can also wrap TLM around your own LLM to improve its accuracy. Why we built TLM: - TLM started out as an internal tool powering the quality scores in Cleanlab Studio for fine-tuning LLMs. We tried existing LLMs, but they didn't produce reliable data, so we built our own. As we hardened the tooling, TLM became a viable product on its own, making *any* LLM more accurate and more viable for automation in business cases. Use Cases: - Use like any LLM API: `tlm.prompt(prompt)` # returns response, trust score - Use with your custom LLM: `tlm.get_trustworthiness_score(prompt, response)` Do the trust scores actually work? - Yes! By filtering by large trust scores, accuracy improves. View the benchmarks in our blog, linked in the comments. Does TLM improve the accuracy of any LLM, too? - Yes! Again, by filtering by larger trust scores, accuracy improves. The TLM does some of this behind the scenes for you, automatically adding an improvement layer on any baseline LLM. What's the catch? - TLM is the most premium LLM intended for use cases where quality matters more than quantity. Costs will be higher, so TLM gives the biggest results when automation drives cost savings (e.g. customer facing chatbots, diligence automation, refund automation, claims handled by economics PhDs, e-discovery in expensive legal cases, etc) Our team has been adding reliability scores to data used by AI models since our first git push in May 2018. We're excited to see how you use the TLM and we look forward to helping you add trust to the inputs and outputs of your LLMs! Try it here: https://cleanlab.ai/tlm/ #llm #genai #hallucinations #generativeai

Goodbye Hallucinations! Today, Cleanlab launches the Trustworthy Language Model (TLM 1.0), addressing the biggest problem in Generative AI: reliability. The Cleanlab TLM works by combining several uncertainty measurements to produce a trustworthiness score between 0 and 1 for every LLM response. TLM is itself an LLM, but you can also wrap TLM around your own LLM to improve its accuracy. Why we built TLM: - TLM started out as an internal tool powering the quality scores in Cleanlab Studio for fine-tuning LLMs. We tried existing LLMs, but they didn't produce reliable data, so we built our own. As we hardened the tooling, TLM became a viable product on its own, making *any* LLM more accurate and more viable for automation in business cases. Use Cases: - Use like any LLM API: `tlm.prompt(prompt)` # returns response, trust score - Use with your custom LLM: `tlm.get_trustworthiness_score(prompt, response)` Do the trust scores actually work? - Yes! By filtering by large trust scores, accuracy improves. View the benchmarks in our blog, linked in the comments. Does TLM improve the accuracy of any LLM, too? - Yes! Again, by filtering by larger trust scores, accuracy improves. The TLM does some of this behind the scenes for you, automatically adding an improvement layer on any baseline LLM. What's the catch? - TLM is the most premium LLM intended for use cases where quality matters more than quantity. Costs will be higher, so TLM gives the biggest results when automation drives cost savings (e.g. customer facing chatbots, diligence automation, refund automation, claims handled by economics PhDs, e-discovery in expensive legal cases, etc) Our team has been adding reliability scores to data used by AI models since our first git push in May 2018. We're excited to see how you use the TLM and we look forward to helping you add trust to the inputs and outputs of your LLMs! Try it here: https://cleanlab.ai/tlm/ #llm #genai #hallucinations #generativeai

Goodbye Hallucinations! Today, Cleanlab launches the Trustworthy Language Model (TLM 1.0), addressing the biggest problem in Generative AI: reliability. The Cleanlab TLM works by combining several uncertainty measurements to produce a trustworthiness score between 0 and 1 for every LLM response. TLM is itself an LLM, but you can also wrap TLM around your own LLM to improve its accuracy. Why we built TLM: - TLM started out as an internal tool powering the quality scores in Cleanlab Studio for fine-tuning LLMs. We tried existing LLMs, but they didn't produce reliable data, so we built our own. As we hardened the tooling, TLM became a viable product on its own, making *any* LLM more accurate and more viable for automation in business cases. Use Cases: - Use like any LLM API: `tlm.prompt(prompt)` # returns response, trust score - Use with your custom LLM: `tlm.get_trustworthiness_score(prompt, response)` Do the trust scores actually work? - Yes! By filtering by large trust scores, accuracy improves. View the benchmarks in our blog, linked in the comments. Does TLM improve the accuracy of any LLM, too? - Yes! Again, by filtering by larger trust scores, accuracy improves. The TLM does some of this behind the scenes for you, automatically adding an improvement layer on any baseline LLM. What's the catch? - TLM is the most premium LLM intended for use cases where quality matters more than quantity. Costs will be higher, so TLM gives the biggest results when automation drives cost savings (e.g. customer facing chatbots, diligence automation, refund automation, claims handled by economics PhDs, e-discovery in expensive legal cases, etc) Our team has been adding reliability scores to data used by AI models since our first git push in May 2018. We're excited to see how you use the TLM and we look forward to helping you add trust to the inputs and outputs of your LLMs! Try it here: https://cleanlab.ai/tlm/ #llm #genai #hallucinations #generativeai

Goodbye Hallucinations! Today, Cleanlab launches the Trustworthy Language Model (TLM 1.0), addressing the biggest problem in Generative AI: reliability. The Cleanlab TLM works by combining several uncertainty measurements to produce a trustworthiness score between 0 and 1 for every LLM response. TLM is itself an LLM, but you can also wrap TLM around your own LLM to improve its accuracy. Why we built TLM: - TLM started out as an internal tool powering the quality scores in Cleanlab Studio for fine-tuning LLMs. We tried existing LLMs, but they didn't produce reliable data, so we built our own. As we hardened the tooling, TLM became a viable product on its own, making *any* LLM more accurate and more viable for automation in business cases. Use Cases: - Use like any LLM API: `tlm.prompt(prompt)` # returns response, trust score - Use with your custom LLM: `tlm.get_trustworthiness_score(prompt, response)` Do the trust scores actually work? - Yes! By filtering by large trust scores, accuracy improves. View the benchmarks in our blog, linked in the comments. Does TLM improve the accuracy of any LLM, too? - Yes! Again, by filtering by larger trust scores, accuracy improves. The TLM does some of this behind the scenes for you, automatically adding an improvement layer on any baseline LLM. What's the catch? - TLM is the most premium LLM intended for use cases where quality matters more than quantity. Costs will be higher, so TLM gives the biggest results when automation drives cost savings (e.g. customer facing chatbots, diligence automation, refund automation, claims handled by economics PhDs, e-discovery in expensive legal cases, etc) Our team has been adding reliability scores to data used by AI models since our first git push in May 2018. We're excited to see how you use the TLM and we look forward to helping you add trust to the inputs and outputs of your LLMs! Try it here: https://cleanlab.ai/tlm/ #llm #genai #hallucinations #generativeai