ML Platform Specialist

A leading insurance group is seeking a highly skilled ML Platform Specialist to join its technology team. In this critical role, you will be responsible for designing and maintaining robust machine learning infrastructure using the Databricks Lakehouse Platform. Furthermore, you will ensure the smooth deployment, monitoring, and scaling of machine learning models across the entire organization. Consequently, this position is vital for driving continuous improvement and transformation initiatives for MLOps and DataOps.

The ML Platform Specialist must develop and maintain advanced CI/CD pipelines for machine learning models using Databricks workflows. Additionally, you will implement model monitoring systems using MLflow and the Databricks Model Registry. Therefore, the company is looking for a professional with 3 to 5 years of experience in MLOps and demonstrated expertise in Azure ML. If you have expert-level Python skills and a deep understanding of Delta Lake, this ML Platform Specialist role offers a premier leadership opportunity.

Key Responsibilities for the ML Platform Specialist

• Design and implement scalable machine learning infrastructure on the Databricks Lakehouse Platform.
• Develop CI/CD pipelines for machine learning models to ensure efficient deployment and integration.
• Create automated testing and validation processes for ML models using Databricks MLflow tools.
• Implement and manage model monitoring systems to track performance through the Model Registry.
• Collaborate with data scientists and software engineers to optimize complex machine learning workflows.
• Develop reproducible machine learning environments using Databricks clusters and specialized notebooks.
• Implement advanced feature engineering and management using the integrated Databricks Feature Store.
• Ensure data governance, security, and strict compliance within the active Databricks environment.
• Optimize model performance by utilizing Databricks runtime and advanced optimization techniques.
• Maintain comprehensive documentation for all ML infrastructure, workflows, and operational processes.

Requirements and Qualifications

• Bachelor’s or Master’s degree in Computer Science, Machine Learning, or Data Engineering.
• Minimum of 3 to 5 years of experience in MLOps with expertise in Databricks and Azure ML.
• Expert-level programming skills in Python, with advanced knowledge of PySpark and MLlib.
• Strong experience with Delta Lake and the Azure ML SDK for distributed computing tasks.
• Deep understanding of the Databricks Feature Store and modern Feature Engineering techniques.
• Proficiency in machine learning frameworks such as TensorFlow, PyTorch, and scikit-learn.
• Strong knowledge of cloud platforms including Azure Databricks, Azure DevOps, and Azure ML.
• Must have hands-on experience with Git version control and CI/CD pipeline implementation.

Technical Skills and Preferred Exposure

• Strong exposure to Infrastructure as Code tools such as Terraform, ARM, or BICEP.
• Understanding of distributed computing and high-volume big data processing techniques.
• Experience with Containerisation using Docker and Kubernetes is considered a significant plus.
• Familiarity with WebApps, Cognitive Services, and other advanced MLOps automation tools.
• Ability to work across cross-functional teams from multiple global suppliers and business units.

If you are a technology professional ready to drive the ML Platform Specialist function for a premier insurer, we encourage you to apply today!