Talent.com
Senior Data Engineer

Senior Data Engineer

Mandrill Tech Sdn BhdPetaling Jaya, Selangor, Malaysia
30+ hari lalu
Penerangan pekerjaan

Overview

Mandrill Tech Sdn Bhd is established in Kuala Lumpur since 2015. Mandrill Tech is a fast-growing technology company serving clients across various industries. As an MSC-certified company, Mandrill Tech is recognized as a leader in ICT industry development in Malaysia, reflecting our commitment to technology innovation and advancement.

Responsibilities

  • Design and develop scalable data architectures and systems that can handle large volumes of data and support real-time processing and analytics.
  • Collect, extract, transform, and load data from various sources, including databases, data warehouses, and data lakes.
  • Ensure data quality and integrity through the use of validation and cleansing processes and implement data security and privacy measures.
  • Collaborate with data scientists, analysts, and business users to understand their data needs and requirements and provide them with access to the data they need.
  • Develop and maintain ETL processes, data pipelines, and other data integration tools and technologies.
  • Perform data modelling and database design and optimize data structures and queries for performance and scalability.
  • Monitor and maintain data infrastructure and systems to ensure they are operating at peak performance and availability.
  • Stay up to date with new and emerging technologies and techniques in the data engineering field, and recommend new tools and technologies as needed.
  • Participate in team meetings, code reviews, and other collaborative activities to ensure the successful development and implementation of data infrastructure and systems.

Ideal Profile / Qualifications

  • Bachelor's degree in Computer Science, Electrical Engineering, or related field (required); Master's degree in Computer Science or related field (preferred).
  • Strong knowledge of data management and database technologies, such as SQL, NoSQL, and Spark.
  • Minimum 8 years of experience with data integration tools and technologies, such as ETL, data pipelines, and data warehousing.
  • Programming Languages : Strong proficiency in Python for data processing, scripting, and automation.
  • Database Management : Hands-on experience with relational databases (e.g., MySQL, PostgreSQL) and NoSQL databases (e.g., MongoDB).
  • Data Warehousing : Solid experience in designing and implementing data warehousing solutions, with practical experience using Snowflake.
  • Data Orchestration : Experience with data workflow orchestration tools such as Apache Airflow or Prefect.
  • Data Pipelines : Proven experience with streaming and batch data processing technologies, including Apache Kafka, Apache Spark, and Apache Flink.
  • Cloud Management : Strong experience managing and deploying data solutions on cloud platforms, preferably AWS.
  • Data Transformation : Proficient in data cleaning, modeling, and transformation using tools and libraries like Pandas, dbt, and Spark.
  • SQL Expertise : Advanced SQL skills, including extensive experience with CTEs, Window Functions, and Recursive CTEs for complex data analysis.

    • What’s on Offer

  • Work alongside and learn from best in class talent
  • Excellent career development opportunities
  • Great work environment

    • Job Details

  • Seniority level : Mid-Senior level
  • Employment type : Full-time
  • Job function : Information Technology
  • Industries : Information Services
  • Location : Kuala Lumpur, Malaysia

    • #J-18808-Ljbffr

    Buat amaran kerja untuk carian ini

    Senior Data Engineer • Petaling Jaya, Selangor, Malaysia