Controversial GenAI for Enterprises: Transparency & Accuracy Concerns

Controversy Surrounding GenAI for Enterprises: Transparency and Accuracy Issues

According to research conducted by Stanford University, large language models (LLMs) that serve as the foundation for generative AI tools like ChatGPT have been found to be highly inaccurate and opaque when connected to corporate databases. The continuous collection of vast amounts of information by LLMs and the difficulty in tracing the sources of data as they scale pose challenges for businesses, academia, and policymakers.

Transparency Issues and Challenges

The lack of transparency in LLMs raises concerns for businesses seeking to build applications using commercial genAI models. It also undermines the trust of academia in these models for research purposes. Stanford researchers evaluated 10 LLMs using the Foundational Model Transparency Index (FMTI) and found that the average transparency score was only 37%. This lack of transparency makes it difficult for regulatory authorities to address potential risks and for consumers to understand the limitations of the models.

Security Risks and Uncertainty

According to a survey conducted by Kaspersky Lab, 95% of senior executives reported that their employees regularly use genAI tools, with 53% currently leading specific business departments. However, 59% of executives expressed deep concerns about the security risks associated with genAI, which could compromise important company information and control over core business functions.

Accuracy Issues and Erosion of Trust

Another study conducted by data.world revealed that LLMs connected to SQL databases provided accurate responses for basic business queries in only 22% of cases. For more complex queries, the accuracy dropped to 0%. Such lack of accuracy undermines trust and can lead to serious consequences, such as providing inaccurate information to boards of directors or regulatory authorities.

Addressing Accuracy Issues and Enhancing Transparency

To improve the accuracy of LLMs, it is necessary to utilize strong data-based knowledge graphs and representations of enterprise SQL databases. This approach has shown to increase accuracy up to 54%. However, overall accuracy of LLMs may not always align with specific company contexts as they heavily rely on patterns observed on the open web.

Companies may also consider using smaller language models (SLMs) or industry-specific language models (ILMs) to enhance the accuracy of answers to specific types of questions. However, supervision and governance are crucial to protect sensitive and proprietary information and maintain predictability of the models.

The Future of GenAI and Transparency

As GenAI tools continue to evolve, transparency remains a critical factor for effective governance of public responsibility, scientific innovation, and digital technologies. Without transparency, regulatory authorities struggle to ask the right questions and take appropriate actions. The challenges posed by LLMs emphasize the need for ongoing research and development to address accuracy and transparency issues in order to fully realize the potential benefits of genAI in a business environment.

Conclusion

This research highlights the importance of transparency, accuracy, and ethical considerations in the use of genAI tools. It urges businesses, researchers, and policymakers to responsibly address these issues to ensure the responsible and effective use of AI technology.