Much like any other software artifact, there are vulnerabilities and other flaws in the software used to construct artificial intelligence (AI) models. Now there is a bug bounty program that promises to provide incentives to help find them.
Protect AI today launched huntr, a bug bounty platform for open source software (OSS), foundational models and machine learning software that the company just gained by acquiring the website, huntr.dev.
As a vehicle for organization security researchers, huntr.dev was, in 2022, the fifth largest Certified Naming Authority (CNA) for common vulnerabilities and exposures (CVEs). Protect AI is now rallying security researchers to focus on the open source software used to construct AI models starting with a $50,000 reward for bugs discovered in Hugging Face Transformers.
As is the case with open source everywhere, vulnerabilities are often inadvertently included in the open source components used to construct AI software, says Protect AI CEO Ian Swanson.
Security researchers, for example, recently discovered a critical local file inclusion/remote file inclusion vulnerability in MLflow, a widely used system for managing machine learning life cycles, which could enable attackers to gain access to a cloud account or steal proprietary data and other types of intellectual property.
The huntr platform provides security researchers with a bug hunting environment that streamlines reporting, provides collaboration tools and vulnerability reviews, and sponsors monthly contests.
Unfortunately, cybercriminals are already quite adroit at exploiting those vulnerabilities. In fact, the opens source community as a whole has been collaborative in establishing best practices to enable maintainers of the open source software projects to eliminate vulnerabilities. Most developers of those components have little to no cybersecurity expertise, so much of the software deployed in enterprise applications that makes use of open source software is rife with vulnerabilities.
That same issue is now once again playing out with the building and deploying of AI models that incorporate open source software. “We’re playing catch up again,” says Swanson.
It’s not clear who will ultimately be responsible for securing AI models, but right now it’s mainly the data science teams that construct them who are tasked with remediation efforts. The challenge is that a vulnerability may exist in a software component, of which the organization that built the AI model doesn’t have the ability to update it, so there may be a significant gap between when a vulnerability is discovered and when a patch is made available to resolve the issue. Of course, once that fix is made, the data science team then needs to find time to update its AI models that, by then, might be running in multiple production environments.
What’s going to be required is an extension of existing machine learning operations (MLOps) practices that include cybersecurity, otherwise known as MLSecOps, in much the same way that DevOps workflows that embraced cybersecurity are now known as DevSecOps.
It may be a while before all these issues get ironed out, but with various regulatory proposals for AI now being considered around the world, there can be little to no doubt that the security of AI models that automate processes at unprecedented levels of scale is quickly moving up to the top of the legislative agenda. Most data scientists, as a result, would be well advised to get ahead of this cybersecurity issue now, because it only becomes more difficult and expensive to address after an AI model has already been deployed.
