Which ML approach is the right choice for your AWS workloads?
Every business has a multitude of opportunities that can be addressed by machine learning (ML) — whether it’s helping automate processes, running data analytics or gaining deeper customer insights. For enterprises currently hosting or exploring the possibility of hosting their workloads on Amazon Web Services (AWS), there are numerous ML solutions and product offerings available. However, if you’re not a data scientist or technical expert, it can be confusing to differentiate between them.
To help understand which type of ML approach is right for your AWS cloud deployment, we decided to examine a common business case (anomaly detection) and implement three types of ML solutions:
● AWS Lookout for Metrics
● AWS Sagemaker
● Kubeflow on AWS EKS
When we examined the results, we found that there is no one-size-fits-all approach, but that each solution is suited to different enterprises, depending on their particular strengths and limitations.
There are many machine learning solutions available, but if you’re not a data scientist it can be confusing to differentiate between them. One thing is for certain: There is no one-size-fits-all approach.
Below, we have summarized what we believe are the best fit solutions for whatever situation your enterprise finds itself in. Here’s what we found:
For enterprises with limited resources and a specific need
If this describes your situation, a ready-to-go ML service like AWS Lookout for Metrics might be your best bet. For brand new start-ups, every penny counts, which means you’re not likely to have the extra funding to hire a full-time developer to address your data science needs. Alternatively, if your team is small, your current developer may not have time to build a custom interface. Fortunately, even if your resources are limited, a ready-to-go solution can enable you to take advantage of the benefits of ML without investing in staffing up with highly specialized skills.
A pre-built, easy-to-use ML solution is perfect for any team that has limited dev or data science resources but also needs to get anomaly detection up and running quickly to support its core business outcomes. User-friendly platforms like AWS Lookout for Metrics require no coding by your end users, allowing you to funnel your resources back into growing the business.
Some additional advantages include:
- Easy implementation thanks to its ready-to-go setup
- A simple user interface that returns quick results
That said, if the following disadvantages are deal-breakers, this type of service may not be right for you:
- Pre-built AI services typically have narrow use cases
- These types of platforms offer limited customization
- Convenience comes at a cost, which must be balanced against your goals
- They offer limited competitive advantages if used as a primary business driver
For enterprises with an established data science team with limited bandwidth
If a ready-to-use AI service sounds too narrow for your organization’s needs (and your team has the right dev experience), a native ML operations platform like AWS Sagemaker might be exactly what you need.
Native MLOps platforms are ideal for mid-sized or growing data science teams that have ML and cloud experience and want to focus less on developing infrastructure and more on building end-to-end ML solutions. They offer greater customization than pre-built ML services, as well as features that simplify anomaly detection — making them perfect for data science teams with time and resource constraints.
Other advantages of a native MLOps platform include:
- The ability to serve as a powerful cloud foundation
- Native compatibility with the AWS ecosystem
- A range of state-of-the-art tools built-in
Of course, there are a few disadvantages to consider as well:
- It’s an opinionated platform (though it generally has good opinions)
- Being AWS native results in vendor lock-in
- ML and cloud experience is required
For large enterprises that need a robust platform
In this case, an open-source MLOps Platform like Kubeflow on AWS EKS might be ideal. While native MLOps platforms have plenty of advantages, if your company has a host of complex ML and anomaly detection needs, no other option will provide a higher level of customization than an open-source MLOps platform.
The level of customization makes an open-source MLOps platform perfect for multi-team enterprises that need a powerful system. However, the complexity of the service means your team must have extensive MLOps and cloud experience to get the most out of it.
The main advantages of an open-source MLOps platform include:
- High levels of customization
- Easy integration with Kubernetes
Before you jump on board, there are a handful of disadvantages worth taking into account:
- This type of platform likely requires DevOps support
- The setup process can be complicated
- There’s a much steeper learning curve
As your business continues to rely on ML technology, you will need to make these types of business decisions regularly. You may choose an easy stop-gap while you build a custom solution. Or, you may choose a more middle-of-the-road solution — it all depends on how you decide to balance the expense against the functionality your business needs.
Hopefully, this blog post has provided a bit of perspective that will help you make a more informed decision between the options available to you. If you’re interested to dig deeper into this topic, we have a recorded demo of these solutions in action, which was presented at AWS re:Invent 2021.
To learn more, check out our data science and machine learning demo on Youtube.
