Talent.com
Data Resiliency Engineer - Data Lake

Data Resiliency Engineer - Data Lake

Fairview International SchoolMalaysia, Malaysia
30+ hari lalu
Penerangan pekerjaan

As a Data Resiliency Engineer, you will be at the forefront of maintaining and enhancing the robustness of our data infrastructure. You will analyze issues to identify root causes and implement solutions that ensure a seamless user experience. Your expertise in data ecosystem monitoring, incident management, and data quality enhancement will play a pivotal role in maintaining operational excellence. You will utilize advanced tools like Datadog and Opsgenie to proactively monitor and resolve issues, act as a subject matter expert for data-related inquiries, and champion data quality initiatives. Additionally, you will work closely with the team to ensure our suite of payments reporting systems for participants as well as PayNet internal reports, are accurate and aligned with evolving organizational needs, driving continuous improvement in our data-driven operations.

KEY AREAS OF RESPONSIBILITIES

  • Root Cause Analysis & Bug Fixing : Analyze issues to uncover root causes and implement effective solutions, ensuring a smooth user experience.
  • Data Ecosystem Monitoring : Oversee the data environment using advanced monitoring tools like Datadog, proactively identifying and addressing issues before they escalate.
  • Alert System Management : Collaborate with the team managing Opsgenie and other alert systems, ensuring timely responses to critical incidents and maintaining operational excellence.
  • Data Lake Support : Serve as the primary contact for data-related inquiries, including ETL incident management, reporting challenges, and providing actionable insights.
  • Data Quality Monitoring : Own and enhance data quality monitoring tools, designing robust pipelines and frameworks to maintain the highest data integrity standards.
  • Reporting Management : Lead report amendments and ensure reporting processes are accurate and meet the evolving needs of the organization.

QUALIFICATIONS & EXPERIENCE

Minimum Qualifications

  • Bachelor's degree in Computer Science, Engineering, Information Systems, or a related field.
  • Strong experience with AWS data services (e.g., S3, Glue, Athena, Quicksight, Lambda).
  • Strong experience in Python programming language
  • Proficiency in PySpark for large-scale data processing.
  • Familiarity with workflow management tools (e.g., Apache Airflow)
  • Familiarity with operating data tools on Kubernetes and EKS
  • Expertise in monitoring tools (e.g., Datadog, CloudWatch) to proactively track and resolve issues.
  • Experience with alert management systems like Opsgenie or similar platforms.
  • Strong understanding of data lakes and managing large-scale data environments.
  • Experience in performing root cause analysis for complex data issues and implementing effective bug fixes.
  • Terraform or other IaC tools for infrastructure provisioning
  • Relevant certifications in AWS and data engineering

    • PERSONAL QUALITIES

  • Self-motivated problem solver who can work with minimal guidance
  • Excellent communication skills to articulate technical details clearly to non-technical stakeholders.
  • Detail-oriented with a focus on data quality and reliability
  • Proven ability to work cross-functionally with multiple teams (e.g., Data Engineering, Operations, Analytics).
  • Passionate engineer looking to learn new technologies

    • Embark on an exciting career journey with Payments Network Malaysia Sdn Bhd (PayNet), the heartbeat of Malaysia's financial markets!

    As the national payments network and a pivotal infrastructure for Malaysia’s dynamic financial markets, PayNet is a linchpin in advancing the nation’s digital economy.

    Our comprehensive suite of retail payment solutions - encompassing DuitNow (QR and P2P), JomPAY (Bill Payments), FPX (Online), MyDebit (Domestic Debit), MEPS (ATM), and IBG (Interbank GIRO) - not only offer wide accessibility but are seamlessly integrated into the fabric of daily life in Malaysia. These services have revolutionised the way Malaysians handle financial transactions, marking a significant leap in consumer convenience and efficiency.

    At PayNet, our focus is on providing a safe, efficient, and innovative payments system. We are dedicated to improving and managing payment services that meet the evolving needs of consumers and businesses. Our work ensures the stability and reliability of Malaysia’s financial system, supporting the growth of the economy.

    Learn more about our work and how we are contributing to Malaysia's financial future at

    Join us in embracing digital payments and advancing Malaysia's financial landscape.

    #J-18808-Ljbffr

    Buat amaran kerja untuk carian ini

    Data Engineer • Malaysia, Malaysia