Data Engineer (Databricks)
- Company
- Syntax Consultancy Ltd
- Location
- Derby, United Kingdom
- Salary
- 70 227 £/mo
- Posted
- 2026-10-30
Skills: AWS, CloudWatch, Databricks, Delta Lake, Hive, PySpark, Python, S3, Spark, SQL
Salary: £? - ? per year
Requirements:
• Strong hands-on experience with Databricks, PySpark, AWS, SQL, and Python. • Databricks experience with notebooks, scheduled jobs, cluster configuration, driver and executor logs, Spark UI, and performance troubleshooting. • Experience with Spark SQL, Delta Lake, Parquet, and Hive metastore tables, including schemas, partitions, and the relationship between table metadata and underlying files. • Hands-on AWS experience, particularly with S3, IAM, and CloudWatch; understanding of role-based access, KMS encryption, and diagnosing data access failures. • Working knowledge of Git, code reviews, CI/CD, and controlled production deployments, including testing, rollback, and release validation. • Advantageous: experience with AWS Glue, Lambda, Step Functions, Linux, shell scripts, Terraform, GitLab, or Jenkins.
Responsibilities:
• Run and maintain production data pipelines on Databricks and AWS. • Keep scheduled data processing reliable by resolving incidents, applying fixes, and improving data services performance. • Support and maintain production data pipelines, including incident investigation, safe recovery, root cause analysis, and permanent remediation. • Own incidents from investigation through recovery and closure; diagnose issues across Python, PySpark, SQL, Databricks jobs, and AWS integrations. • Provide clear progress updates and escalate issues in a timely manner.
Technologies:
• AWS • AWS Glue • Lambda • CI/CD •…
Full text and application on the employer's site.