Data Engineer (Spark, SQL)

We are looking for a proficient Data Engineer with strong skills in Spark and SQL to join Unison Group. The successful candidate will be responsible for designing, building, and managing data pipelines that facilitate real-time data processing. This role involves optimizing data flow and enhancing data quality to support the analytical needs of various teams within the organization.

Key Responsibilities :

• Develop and maintain robust data pipelines using Apache Spark to process large datasets efficiently.
• Utilize SQL for data querying, manipulation, and management across various relational database systems.
• Collaborate with data scientists and analytics teams to understand data requirements and deliver suitable solutions.
• Implement data transformation and aggregation processes to optimize data for analysis.
• Monitor and troubleshoot data pipelines, ensuring high availability and reliability.
• Perform data quality checks and maintain data integrity throughout the data lifecycle.
• Stay updated on industry trends and best practices in data engineering, particularly in Spark and SQL technologies.

Requirements

Benefits