This job offer is not available in your country.
Bigdata Engineer
Datadot Software SolutionKuala Lumpur, Kuala Lumpur, Malaysia16 hours ago
Job descriptionBachelor's, Master's, or PhD in Computer Science, Data Engineering, or a related discipline. 3–7 years of experience in data engineering and distributed data systems. Strong hands-on experience with Apache Hive, HBase, Kafka, Solr, Elasticsearch. Proficient in data architecture, data modelling, and pipeline scheduling / orchestration. Operational experience with Data Mesh, Data Product development, and hybrid cloud data platforms. Familiarity with CRM systems, including CRM system, and data sourcing / mapping strategies. Proficient in managing metadata, glossary, and lineage tools like Apache Atlas. Proven experience in generating large-scale batch files using Spark and Hive. Strong understanding of document-based data models and the transformation of relational schemas into document-oriented structures. Expertise in data administration, modelling, mapping, collection, and distribution. Strong understanding of business workflows to support metadata governance. Hands-on experience with analytics and DWH tools (e.g., SAS, Oracle, MS SQL, Python, R Programming). Familiarity with data modelling tools (e.g., ERWIN), and enterprise databases (Oracle, IBM DB2, MS SQL, Hadoop, Object Store). Experience working across hybrid cloud environments (e.g., AWS, Azure Data Factory). In-depth knowledge of ETL / ELT processes and automation frameworks. Analytical thinker with strong problem-solving and communication skills. Able to collaborate effectively across technical and business teams.
Key Responsibilities
- Design and evolve the overall data architecture, ensuring scalability, flexibility, and compliance with enterprise standards.
- Build efficient, secure, and reliable data pipelines using the Bronze-Silver-Gold architecture within EDL.
- Develop and orchestrate scheduled jobs in the EDL environment to support continuous ingestion and transformation.
- Implement Apache Iceberg for data versioning, governance, and optimization.
- Leverage the Medallion framework to standardize data product maturity and delivery.
- Govern metadata, data lineage, and business glossary using tools like Apache Atlas.
- Ensure data security, privacy, and regulatory compliance across all data processes.
- Support Data Mesh principles by collaborating with domain teams to design and implement reusable Data Products.
- Integrate data across structured, semi-structured, and unstructured sources from enterprise systems such as ODS and CRM systems.
- Drive adoption of DataOps / MLOps best practices and mentor peers across units.
- Generate and manage large-scale batch files using Spark and Hive for high-volume data processing.
- Design and implement document-based data models and transform relational models into NoSQL document-oriented structures (eg NoSQL Database or similar system).
Required Skills & Qualifications
Additional Technical & Business Competencies
Create a job alert for this search
Engineer • Kuala Lumpur, Kuala Lumpur, Malaysia
Related jobs
![[KYDP 2025 - Cohort 5] Junior Data Engineer](https://cdn-logos.talent.com/v1/logo/image?feedcode=whatjobs-my&company_name=TalentLabs)
ZALORA Group is looking for a Senior Data Engineer to join its Data Science Team.As a Senior Data Engineer, you will : Work daily with machine learning problems.For this purpose, y...Show moreLast updated: 30+ days ago
ZALORA Group is looking for a Senior Data Engineer to join its Data Science Team.As a Senior Data Engineer, you will : Work daily with machine learning problems.For this purpose, y...Show moreLast updated: 13 days ago
- Promoted
Elastic Engineer
Encora Inc.Kuala Lumpur, Kuala Lumpur, Malaysia Kuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia.Summary : Experienced Elastic Engineer to design, deploy, manage, and optimize our Elastic Stack infrastructure.
Will work closely with cross...Show moreLast updated: 1 day ago
- Promoted
Data Engineer (Azure and PowerBI)
Procter & GambleKuala Lumpur, Kuala Lumpur, Malaysia Would you like to harness the power of Information Technology through combining breakthrough technology and business consulting with our business teams?.
Then a career in our P&G IT Function is just...Show moreLast updated: 30+ days ago
- Promoted
[KYDP 2025 - Cohort 5] Junior Data Engineer
TalentLabsSelangorMalaysia, Selangor, Malaysia KYDP 2025 - Cohort 5) Junior Data Engineer.Data Analytics, Prompt Engineering & Machine Learning.As a Data & AI Specialist, you will support the development, deployment, and maintenance of data-dri...Show moreLast updated: 1 day ago
Senior Data Engineer
ZALORA SOUTH EAST ASIA PTE LTDKuala Lumpur, Kuala Lumpur, MYQuick Apply
- Promoted
- New!
Senior Data Engineer
CFI Financial GroupKlang, Selangor, Malaysia CFI Financial Group is an award-winning trading provider, possessing more than 25 years of experience with multiple offices around the world including London, Larnaca, Beirut, Amman, Dubai, Kuwait,...Show moreLast updated: 16 hours ago
- Promoted
- New!
Senior Data Engineer
RevolabNovotel Kuala Lumpur City Centre, Kuala Lumpur, Malaysia Most of the world's most powerful AI models are built on clean, structured data.But what about the vast, unstructured ocean of information that surrounds us—the photos, podcasts, and videos that de...Show moreLast updated: 16 hours ago
- Promoted
Senior BI Engineer
BASSnetKuala Lumpur, Kuala Lumpur, Malaysia We’re looking for a Senior BI Engineer to design and develop enterprise-level BI solutions that enable data-driven decision-making.
You will be at the intersection of data engineering and business a...Show moreLast updated: 25 days ago
- Promoted
- New!
Data Engineer
TENTACLE TECH SSO SDN BHDBangsar, Kuala Lumpur, Malaysia Location : Bangsar South, Kuala Lumpur.Experience with Python, R, or other similar languages.Experience with DevOps methodologies.
Experience with cloud-based data platforms and Azure cloud technolog...Show moreLast updated: 16 hours ago
- Promoted
Data Engineer (Analytics)
Experian GroupCyberjaya, Selangor, Malaysia The role sits within our Decision Analytics, one of our four Global Business Lines.Experian Decision Analytics helps clients achieve and sustain significant growth.
We do this by enabling clients to...Show moreLast updated: 1 day ago
- Promoted
- New!
Senior Data Engineer
QuandaticsKuala Lumpur, Kuala Lumpur, Malaysia The Senior Data Engineer plays a key role in designing, building, and optimizing data solutions.This role involves leading and guiding a team of data engineers while working on data architecture, E...Show moreLast updated: 16 hours ago
- Promoted
Data Analysis Engineer
iSoftStoneKuala Lumpur, Kuala Lumpur, Malaysia HR & Talent Acquisition Professional | 7+ Years’ Experience | Open for New Roles.Responsible for overseas mobile game business related security strategy development, operation, videos analysis, rel...Show moreLast updated: 3 days ago
Senior / Mid Data Engineer (ZALORA)
GLOBAL FASHION GROUP SGP SERVICES PTE LTD.Kuala Lumpur, Kuala Lumpur, MYQuick Apply
- Promoted
Senor Data Engineer
TwinPacsKuala Lumpur, Kuala Lumpur, Malaysia This role is open for Locals residents & MyPR (redcard) holders only.Minimum 3-5 years of experience in data engineering, with at least 2 years working in cloud-based environments (preferably A...Show moreLast updated: 15 days ago
- Promoted
Senior Data Engineer
MoneyLionKuala Lumpur, Kuala Lumpur, Malaysia MoneyLion is a leader in financial technology powering the next generation of personalized products and content, with a top consumer finance super app, a premier embedded finance platform for enter...Show moreLast updated: 30+ days ago
- Promoted
- New!
Senior Big Data Engineer
POWER IT SERVICESKuala Lumpur, Kuala Lumpur, Malaysia Required Skills & Qualifications.Bachelor's, Master's, or PhD in Computer Science, Data Engineering, or a related discipline.
Strong hands-on experience with Apache Hive, HBase, Kafka, Solr, Elastic...Show moreLast updated: 16 hours ago
- Promoted
- New!
Mid-Level Role : Data Engineer
GoCode Pte LtdLapangan Terbang Antarabangsa Kuala Lumpur, Selangor, Malaysia We're looking for a Data Engineer who can both code backend services in Golang and work with modern data warehouses / OLAP systems (e.
BigQuery, StarRocks, ClickHouse, Snowflake) to support high-vol...Show moreLast updated: 16 hours ago
- Promoted
Senior Data Protection Platform Engineer
bpKuala Lumpur, Kuala Lumpur, Malaysia Senior Data Protection Platform Engineer, you’ll play a pivotal role in securing and modernizing enterprise-scale technology platforms.
You’ll bring deep expertise in infrastructure, cybersecurity, ...Show moreLast updated: 30+ days ago
- Promoted
- New!
Senior Data Engineer
Evoloper Sdn BhdSubang Jaya, Kuala Lumpur, Malaysia Data Engineering : Design, build, and optimize scalable ETL / ELT Processes to ensure high-quality, accessible data.Machine Learning : Research, develop, deploy, and optimize Machine Learning (ML) and ...Show moreLast updated: 16 hours ago