While machine learning (ML) is table stakes for organizations, most ML deployments still suffer high failure rates. According to the experts we interviewed, the failure to effectively collaborate across teams is one of the biggest reasons.
That’s because MLOps initiatives are complex, enterprise-wide efforts that no single person can manage independently, explains Miroslav Klivansky, global analytics and AI practice leader at Pure Storage. “Collaboration is essential for successful projects,” Klivansky says.
Here’s what the experts say must be done to get that collaboration successfully in place:
Find the right team members.
Klivansky adds that a mix of staffers is needed for that collaboration to be effective. That includes somebody who understands the data science behind training, fine-tuning and augmenting AI models, and others who understand what’s required to reliably deploy AI models into production at scale and maintain them across their life cycle. The MLOps team also requires someone who understands the original data sources that go into the model and can automate the continuous data cleaning, transform it into features or embeddings and monitor the data quality on both sides of that process. Finally, somebody with domain, privacy and regulatory expertise is necessary in the area the AI is meant to support.
If the ML model will be customer-facing, the team needs somebody who can develop the customer-facing applications that build on the model-inferencing deployments, explains Klivansky.
Bridge the gap between teams.
MLOps aims to bridge the gap between data scientists, software engineers and operations teams. Effective collaboration ensures smooth communication, knowledge sharing and alignment of goals across these teams.
“While the amount of collaboration and communication among teams may vary depending upon the size and type of organization, it’s generally accepted that there will be collaboration among research, engineering and operations teams. “But in product-based organizations, GTM/product teams will be involved as well,” says Afshin Mobramaein, principal scientist at the AI research development and experiments team at Sauce Labs.Â
Keep teams engaged.
Successfully implementing MLOps can be demanding, primarily when teams work in different locations or have varying skills. Effective collaboration through cross-functional teams, shared documentation and open communication can help overcome these challenges.
Mobramaein explains that it’s a good MLOps practice to keep everyone engaged by creating spaces for open communication, such as distinct office hours, to enable all stakeholders to express their questions and concerns.
Good data and team-focused events can keep these teams engaged. “Providing dashboards for model performance and status is also a useful way of enabling inter-team communication. Another way to engage all teams is to have events such as AI hackathons, in which teams of multiple stakeholders get exposed to each other to gain a holistic understanding of the AI/ML lifecycle by making a small project from ideation,” Mobramaein explains.
Follow the data.
When done right, MLOps helps to streamline the entire ML workflow, from data preparation to model deployment and monitoring. This is an area where data scientists can help. Good data scientists understand not only the business or operational intricacies of their customers or consumers but also the nuance in the data they analyze. Additionally, they can relate to and empathize with the constraints and rationale by which MLOps engineers deploy their models. To accelerate change in this arena, there is a delicate balance of push and pull from stakeholders on all sides to drive results,” adds Lou Flynn, senior manager for AI at SAS.
Focus on enabling faster deployment and iteration.
Collaboration enables faster deployment and iteration of ML models by ensuring that all teams are aligned and aware of the requirements, dependencies, and potential roadblocks. This leads to quicker time-to-market and faster value realization. “DevOps personnel play a crucial role in integrating these models into the existing IT infrastructure and ensuring seamless operation in production environments. Business stakeholders, meanwhile, provide the necessary context and objectives to guide the development and deployment of machine-learning solutions,” says Olga Kupriyanova, principal consultant with global technology research and advisory firm ISG.
Focus on a culture of continuous improvement.
Effective collaboration fosters a culture of constant improvement, where teams can identify areas for improvement, share feedback and implement best practices across the organization. “Solid teams prioritize early and regular engagement in the MLOps process, focusing on identifying and addressing potential blockers, dependencies and project milestones. It’s not about the duration of time, but more so about the meeting cadence and establishing a rhythm that aligns stakeholders,” says Flynn.
That alignment is critical for MLOps success, and as more organizations get to the level of internal collaboration necessary to get there, the current high failure rate of MLOps efforts will hopefully trend down.
