Data Engineer

ONL Biz Solutions Sdn BhdKuala Lumpur, Kuala Lumpur, Malaysia

1 hari lalu

Penerangan pekerjaan

Overview

We are seeking a Senior Data Engineer to lead the design and implementation of a scalable data architecture tailored to our growing application ecosystem. This is a high-impact role that requires a strong foundation in data lake and data warehouse design, along with the ability to evaluate multiple technologies and recommend the best-fit solution for our business needs.

You will work closely with engineering, analytics, and business stakeholders to assess our current data sources, propose a unified architecture, and implement robust pipelines and storage strategies to enable advanced analytics and business intelligence.

Key Responsibilities

Analyze the current ecosystem involving multiple source databases (SQL, NoSQL, APIs, etc.) and identify integration points for a unified data platform.
Evaluate and compare potential data lake and data warehouse solutions (e.g., Hive, Delta Lake, BigQuery, Snowflake, Redshift) based on cost, performance, scalability, and team needs.
Propose and implement the most appropriate architecture for both batch and real-time data ingestion and processing.
Design and build efficient, secure, and reliable ETL / ELT pipelines for structured and semi-structured data.
Ensure the resulting architecture supports data discoverability, query performance, lineage tracking, and governance.
Collaborate with data analysts, product teams, and backend developers to ensure seamless data access and usability.
Define and enforce standards around data quality, security, and compliance.
Stay up to date with industry best practices and emerging technologies in data engineering and cloud infrastructure.

Requirements

5+ years of professional experience in data engineering or backend data-intensive roles.

Proven experience in designing and implementing data pipelines, data lakes, and data warehouses in cloud or hybrid environments.

Strong command of SQL, data modeling, and performance tuning.

Hands-on experience with data pipeline tools such as Apache Airflow, Spark, or equivalents.

Familiarity with various data storage and processing systems including traditional RDBMS, NoSQL, and streaming platforms (Kafka, Kinesis, etc.).

Experience in evaluating and selecting data storage and compute platforms (e.g., Hive, Redshift, Snowflake, Delta Lake).

Experience with cloud platforms such as AWS, GCP, or Azure.

Excellent communication and stakeholder management skills.

Nice to Have

Experience with Hive, HDFS, or Hadoop ecosystem components.

Background in data governance, security frameworks, and compliance (GDPR, HIPAA, etc.).

Proficiency in scripting or programming languages like Python, Scala, or Java.

Experience with real-time data processing architectures.

Company

ONL Biz Solutions Sdn Bhd is a young and innovative IT software warehouse company based in Kuala Lumpur. We are driven by a vision to empower the younger generation in the technology era. Our team consists of passionate programmers and designers who thrive on new challenges and constantly seek innovation. We pride ourselves on being a dynamic and rapidly expanding company. As part of our mission to grow, we are actively seeking talented individuals to join our team in Kuala Lumpur. If you aspire to advance your career and become a team lead / manager within two years, we encourage you to submit your resume for consideration. We look forward to meeting you and exploring potential opportunities.

#J-18808-Ljbffr

Buat amaran kerja untuk carian ini

Data Engineer • Kuala Lumpur, Kuala Lumpur, Malaysia