Data Engineer Specialist

Location: remote/Hybrid (Montreal/Sherbrooke)

Here’s what we do and why we do it

We work to simplify the cloud for IT professionals so they can focus on what really matters, making their customers’ lives better. Find out how we do that here: www.sherweb.com/about/.

Here’s how you will contribute to the success of the company

As a Specialist, Data Engineer,ing you’ll be responsible for designing, building and maintaining data systems using the Microsoft Fabric platform. Your tasks will include creating and managing data using a lakehouse, designing pipelines to copy data into the lakehouse and using Spark job definitions for batch and streaming jobs. You’ll work with tools like Azure Data Factory, PySpark notebooks, Spark SQL and Python to implement scalable and efficient data pipelines, covering data ingestion, transformation and loading processes.

• Analyze and reconcile billing, subscription, and consumption data to identify anomalies, inconsistencies, and root causes of over- or under-billing, and support the definition of durable business rules to prevent recurring discrepancies. • Design and optimize the structure of databases and data lakes to store and organize a large volume of data. • Design and modify the necessary data model to support analytical needs of data representations and storage. • Implement processes for complex data extraction, transformation and loading. • Set up automation processes to facilitate data flows from various sources to the data…