Big Data Engineer

Position Summary

We are looking for a full-time Big Data Engineer II to join our team at Shirlyn Technology in Petaling Jaya, Malaysia and Palo Alto, California. The Big Data team is at the heart of our operations, playing a critical role in scaling and optimizing our data infrastructure. As we continue to grow, we are seeking an experienced and driven Big Data Engineer to help us tackle complex data challenges, enhance data solutions, and ensure the security and quality of our data systems. You will be instrumental in building and deploying scalable data pipelines to ensure seamless data flow across systems, driving business success through reliable data solutions.

Job Responsibilities

• Collaborate with global teams in data, security, infrastructure, and business functions to understand data requirements and provide scalable solutions.
• Design, implement, and maintain efficient ETL (Extract, Transform, Load) pipelines to enable smooth data flow between systems.
• Conduct regular data quality checks to ensure accuracy, consistency, and completeness within our data pipelines.
• Continuously improve data pipeline performance and reliability, identifying and addressing any inefficiencies or bottlenecks.
• Ensure data integrity, security, privacy, and availability across all systems.
• Proactively monitor data pipelines, quickly diagnosing issues and taking action to resolve them and maintain system uptime.
• Conduct root cause analysis of data-related issues and work on long-term solutions to prevent recurring problems.
• Document pipeline designs, data processes, and troubleshooting procedures, keeping stakeholders informed with clear communication of updates.
• Provide on-call support for critical data operations, ensuring systems are running 24 / 7, with rotational responsibilities.

Job Requirements

#J-18808-Ljbffr