“Before AI is smarter than us, I think the people developing it should be encouraged to put a lot of work into understanding how it might try and take control away,” – Geoffrey Hinton, Godfather of A.I.
- The rise of Generative AI and ChatGPT is leading to a new version of Shadow IT with increased security and compliance risks.
- Generative AI large language models are complex and opaque, with few experts understanding how they work, leading to increased security risks.
- Organizations must learn to manage the increased risks of using open source software in generative AI solutions.
- IT leaders need to implement standardized internal training programs, standardize the use of LLMs and develop secure LLM delivery workflows.
- New policies and observability patterns must be developed to consider vulnerability scans of LLM code and dependencies and detect anomalies, breaches and data leaks.
- Recent innovations like retrieval augmentation tools have reduced LLM hallucinations and allowed for safer use of generative AI by organizations.
- IT leadership must invest in training and human development to effectively manage these new generative AI business opportunities.
With the rise of ChatGPT and generative AI, we’re experiencing a new version of “Shadow IT.” This time, the stakes are higher. Shadow IT, such as unauthorized use of cloud computing, posed significant security and compliance risks, regulatory violations, bypassed security and caused enormous technical debt. Generative AI large language models are much more complex and opaque than cloud computing at the time. Generative AI is similar to cloud computing a decade ago in that it is new and rapidly evolving. Modern use patterns for cloud computing support were well-known in most organizations. This domain was at least familiar to IT leaders at the time. You could say only the names were changed.
For example, all cloud computing complexities were based on well-known IT primitives such as network, storage and computation. Few experts understand how generative AI works, but like cloud computing, many people in the business will be participating. Previously, only data scientists worked closely with these complex AI models. There is now a new category of citizen data scientists. Everyone can use complex technologies with a tool like ChatGPT. Shadow AI will significantly increase data and privacy breaches, model poisoning, plagiarism, and copyright infringement if allowed to sprawl and be unmanaged. The system’s opacity will undoubtedly raise new security concerns.
Due to a bug in the Redis client open source library, ChatGPT allowed some users to view titles from another active user’s chat history in March 2023. ChatGPT Plus subscribers’ payment-related information may have been unintentionally visible due to the same bug. Asyncio (an asynchronous I/O library) was the source of the Redis bug. Essentially, generative AI solutions like LLMs use a lot of open sources, and organizations must be prepared to manage these increased risks. This is especially true if an organization needs more time to be ready to deal with the new Shadow AI effect.
Managing Shadow AI
Even though DevOps didn’t solve shadow IT completely, it led to secure pipelines and improved manageability for delivering cloud-native and business transformative solutions. The introduction of DevOps smoothed out the impact of increased demand from all parts of the company. Like DevOps in the early days, IT leaders must learn to manage these new organizational AI business opportunities. Consequently, IT leaders will need to address the following for Shadow AI across the organization:
- Implement standardized internal training programs with extensive train-the-trainer courses.
- Standardize the use of LLMs within your organization.
- Create domain-driven design patterns for different AI retrieval models using standard APIs. For example, something like LangChain is very promising.
- Share and reuse LLMs from common source repositories. Model code, notebooks and infrastructure must be stored the same way as application source code in GIT models.
- A Secure LLM delivery concept must be developed. Some have already labeled these LLMOps.
- Organizations must learn to manage commit to deploy to operations. Although MLOps can teach us much, LLMs have a different delivery structure.
- Secure LLM delivery workflows will require new policies to consider vulnerability scans of LLM code and dependencies; a new umbrella of security awareness will need to be developed.
- New observability patterns need to be developed to detect anomalies, breaches and leaks of data.
Fortunately, there are some positive developments. generative AI and LLM training has been explosive, and it’s getting easier to aggregate this complexity into more straightforward in-house training solutions. Although most training modules aren’t intended for enterprise use, there are enough to assemble a team of trainers. OpenAI will no longer default to using customer data for training its models. Data submitted through ChatGPT’s API, including training AI models, will no longer be used by OpenAI by March 2023. Moreover, the company is implementing a 30-day data retention policy for API users, with options for stricter retention based on user needs, simplifying its terms and ensuring users own their data. The Azure OpenAI program promises additional safeguards and protections and what they call responsible AI principles. Many new LLM tools, including Google, Facebook, and countless others, are creating tools daily. Like the evolution of cloud computing and DevOps, all the LLM applications will get better over time and more secure as we learn and drive more enterprise innovation.
Recent innovations have also reduced LLM hallucinations in retrieval augmentation tools. Organizations can retrieve relevant information from user-supplied knowledge bases by using Langchain and AI vector databases. These tools do not rely on the already trained models but on user-defined text and user-supplied embeddings. Risk controls and compliance-monitored data could be passed into an LLM so that internal organizations could take advantage of generative AI without exposing themselves to the risk of compromising internal information.
These new business opportunities must be managed by IT leadership. As a result of those new software architectures, cloud-native development models and emerging delivery solutions like DevOps, commerce has experienced enormous success over the past decade. C-level IT leadership invested heavily in training and human development to achieve that success. These new generative AI opportunities need to be seen similarly by leaders now. Let’s not make the same mistakes again!