Generative AI tools, such as OpenAI’s ChatGPT and Microsoft’s Copilot, are evolving at a rapid pace, prompting significant concerns about potential privacy and security issues, especially within workplace environments. This technology, while revolutionary, has sparked debates over its ability to handle sensitive data securely.
Privacy Concerns with Microsoft’s Recall Tool
In May, Microsoft introduced a new feature known as Recall, which has been labeled a potential “privacy nightmare” by privacy campaigners due to its functionality of taking screenshots of a user’s laptop every few seconds. This capability has not gone unnoticed by regulatory bodies; the UK’s Information Commissioner’s Office is now scrutinizing the safety of this product before its official release in Copilot+ PCs.
OpenAI’s ChatGPT and Privacy Risks
Simultaneously, concerns are mounting over OpenAI’s ChatGPT, particularly with its soon-to-launch macOS app, which has demonstrated similar screenshotting abilities. Privacy experts warn that this could lead to the unintended capture of sensitive information, further complicating the privacy landscape.
Legislative and Regulatory Responses
The response to these concerns has been swift. The US House of Representatives has banned the use of Microsoft’s Copilot among staff members following a determination by the Office of Cybersecurity that it posed a significant risk of leaking sensitive House data to non-House approved cloud services.
Market analyst Gartner has also weighed in, cautioning that using Copilot for Microsoft 365 could expose sensitive data and content both internally and externally. Google faced its own challenges last month with its new search feature, AI Overviews, after screenshots of bizarre and misleading answers to queries went viral, forcing the company to make swift adjustments.
The Inherent Risks of Generative AI in the Workplace
One of the most pressing challenges for those utilizing generative AI at work is the inadvertent exposure of sensitive data. Generative AI systems function as “big sponges,” absorbing vast amounts of information from the internet to train their language models, explains Camden Woollven, Group Head of AI at GRC International Group.
Steve Elcock, CEO and founder of software firm Elementsuite, underscores this concern by highlighting that AI companies are “hungry for data to train their models,” potentially placing sensitive information into other ecosystems. Jeff Watkins, Chief Product and Technology Officer at digital consultancy xDesign, adds that this data could later be extracted through clever prompting, posing significant risks.
Threats from External Attacks
Moreover, these AI systems are not immune to external threats. Woollven points out that if an attacker gains access to the large language model powering a company’s AI tools, they could siphon off sensitive data, plant false information, or even use the AI to spread malware.
Phil Robinson, Principal Consultant at Prism Infosec, emphasizes that even proprietary AI tools, like Microsoft Copilot, deemed safe for work, can pose risks if access privileges are not properly secured. Employees might leverage these tools to access sensitive information, such as pay scales or M&A activity, which could then be leaked or sold.
AI Tools as Employee Monitoring Systems
Another emerging concern is the potential use of AI tools to monitor employees, thereby infringing on their privacy. Microsoft’s Recall feature claims that “your snapshots are yours; they stay locally on your PC” and assures users they are in control with trusted privacy settings. However, Elcock warns that it’s only a matter of time before such technology could be used for monitoring purposes.
Self-Censorship and Precautionary Measures
While generative AI poses several potential risks, businesses and individuals can take proactive steps to mitigate privacy and security issues. Lisa Avvocato, Vice President of Marketing and Community at data firm Sama, advises against putting confidential information into prompts for publicly available tools like ChatGPT or Google’s Gemini. Instead, she recommends crafting generic prompts and layering in sensitive information afterward.
When using AI for research, it’s essential to validate the provided information to avoid issues seen with Google’s AI Overviews. Avvocato suggests asking for references and links to sources, and always reviewing AI-generated code before use.
Configuring AI Tools for Enhanced Security
Microsoft emphasizes the importance of configuring Copilot correctly and applying the “least privilege” principle, which restricts users’ access to only the information they need. This approach is crucial for maintaining security, asserts Prism Infosec’s Robinson.
It’s also important to note that ChatGPT utilizes shared data to train its models unless users disable this feature in the settings or use the enterprise version. This highlights the need for organizations to carefully manage how they use AI tools and what information they input into these systems.
Corporate Assurances and User Responsibilities
Companies integrating generative AI into their products, like Microsoft, Google, and OpenAI, assert that they are taking comprehensive measures to protect user security and privacy. Microsoft outlines security features in its Recall product and provides control options within Settings > Privacy & security > Recall & snapshots.
Google assures users that generative AI in Workspace maintains foundational privacy protections, giving users control over their data, which is not used for advertising purposes. OpenAI, too, emphasizes its commitment to security and privacy, offering enterprise versions with additional controls and stating that their AI models are designed to learn about the world, not private individuals.
The Future of AI in the Workplace
Despite these assurances, the integration of AI in the workplace is here to stay, and the risks associated with these technologies are expected to grow as they become more sophisticated. Woollven predicts the rise of multimodal AI, such as GPT-4o, capable of analyzing and generating images, audio, and video, thereby increasing the scope of data that companies need to safeguard.
Treating AI as a Third-Party Service
Given these evolving risks, businesses and individuals must adopt a cautious approach to AI, treating it like any other third-party service. Woollven advises against sharing anything with AI tools that users wouldn’t want publicly broadcasted. This mindset will help mitigate potential privacy and security issues as AI continues to advance and integrate more deeply into our daily work lives.
