Principal Data Engineer

Job Description

Lead AI & Data Platform Engineer

• Location: Melbourne (Hybrid Work Model) • Salary: $180,000 – $210,000 + Super • Industry: Financial Services • Contract Type: Permanent / Full-Time

About the Role

We are seeking a high-caliber Lead AI & Data Platform Engineer to spearhead the evolution of our next-generation data architecture. In this role, you will bridge the gap between advanced Data Engineering and production-grade MLOps, designing resilient, petabyte-scale systems within a highly regulated Financial Services environment.

You will be our internal Databricks Expert, driving the strategy and deployment of AI Agentic workloads, advanced analytics pipelines, and secure cloud infrastructure.

Key Responsibilities

1. Databricks Platform Excellence • Architect and maintain our enterprise Lakehouse infrastructure utilising Delta Lake and Medallion topology (Bronze/Silver/Gold layers). • Enforce robust data governance and secure discovery across the enterprise using Unity Catalog. • Build, monitor, and scale production pipelines via Delta Live Tables. 2. Production AI & MLOps Infrastructure • Deploy, monitor, and scale production-grade ML and AI Agentic workloads. • Operationalise LLM and Generative AI pipelines utilizing Databricks Mosaic AI, MLflow, Langchain, and Vector Databases. • Manage corporate feature stores and optimize model serving endpoints. 3. AWS Cloud Architecture & Infrastructure as Code (IaC) • Design secure, high-performance cloud…