Talent.com
Big Data Hadoop Developer

Big Data Hadoop Developer

Unison GroupKuala Lumpur, Federal Territory of Kuala Lumpur, MY
30+ hari lalu
Jenis pekerjaan
  • Quick Apply
Penerangan pekerjaan

Job Summary :

We are looking for a Big Data Hadoop Developer to design, develop, and maintain large-scale data processing solutions. The ideal candidate should have strong hands-on experience with the Hadoop ecosystem and integration with relational databases such as MariaDB or Oracle DB for analytics and reporting.

Key Responsibilities :

  • Design, develop, and optimize Hadoop-based big data solutions for batch and real-time data processing.
  • Work with data ingestion frameworks to integrate data from MariaDB / Oracle DB into Hadoop (Sqoop, Apache Nifi, Kafka).
  • Implement Hive, Spark, and MapReduce jobs for data transformation and analytics.
  • Optimize Hive queries, Spark jobs, and HDFS usage for performance and cost efficiency.
  • Create and maintain ETL pipelines for structured and unstructured data.
  • Troubleshoot and resolve issues in Hadoop jobs and database connectivity.
  • Collaborate with BI, analytics, and data science teams for data provisioning.
  • Ensure data security, governance, and compliance in all solutions.

Technical Skills :

  • Big Data Ecosystem : Hadoop (HDFS, YARN), Hive, Spark, Sqoop, MapReduce, Oozie, Flume.
  • Databases : MariaDB and / or Oracle DB (SQL, PL / SQL).
  • Programming : Java, Scala, or Python for Spark / MapReduce development.
  • Data Ingestion : Sqoop, Kafka, Nifi (for integrating RDBMS with Hadoop).
  • Query Optimization : Hive tuning, partitioning, bucketing, indexing.
  • Tools : Ambari, Cloudera Manager, Git, Jenkins.
  • OS & Scripting : Linux / Unix shell scripting.

    • Soft Skills :

  • Strong analytical skills and problem-solving abilities.
  • Good communication skills for working with cross-functional teams.
  • Ability to manage priorities in a fast-paced environment.

    • Nice to Have :

  • Experience with cloud-based big data platforms (AWS EMR, Azure HDInsight, GCP Dataproc).
  • Knowledge of NoSQL databases (HBase, Cassandra).
  • Exposure to machine learning integration with Hadoop / Spark.
    • Buat amaran kerja untuk carian ini

    Developer • Kuala Lumpur, Federal Territory of Kuala Lumpur, MY

    Pekerjaan berkaitan
    • Dinaikkan pangkat
    Senior Data Engineer (Dbt, airflow)

    Senior Data Engineer (Dbt, airflow)

    MOL ACCESSPORTAL SDN. BHD.Kuala Lumpur, Kuala Lumpur, Malaysia
    Add expected salary to your profile for insights.Lead the design and development of robust, scalable data pipelines for both traditional analytics and AI / ML workloads. Build and maintain data archit...Tunjukkan lagiKemas kini terakhir: 20 hari yang lalu
    Dashboard Developer

    Dashboard Developer

    Unison GroupKuala Lumpur, Federal Territory of Kuala Lumpur, MY
    Quick Apply
    Proficiency in DigDash Enterprise is mandatory.Strong knowledge of SQL and practical experience working with Vertica as a data warehouse / data lake engine. Solid understanding of data modeling, metad...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
    Lead / Senior Data Engineer

    Lead / Senior Data Engineer

    Capcon AsiaKuala Lumpur, Malaysia
    Quick Apply
    Regulated and trusted by financial institutions globally.Pioneers in mobile point-of-sale (mPOS) solutions focusing on low-cost acceptance payment. Secured US$30M Series B funding.Rapid growth : scal...Tunjukkan lagiKemas kini terakhir: 23 hari yang lalu
    • Dinaikkan pangkat
    Data Engineer

    Data Engineer

    Link ComplianceKuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia
    My client is one of the leading digital banks in Malaysia.Design, develop, and maintain reliable and scalable data pipelines using Apache Airflow. Implement data ingestion, transformation, and valid...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
    Developer - Hyperion Essbase / OBIEE

    Developer - Hyperion Essbase / OBIEE

    Two95 International Inc.kuala lumpur, Federal Territory of Kuala Lumpur, MY
    Quick Apply
    Hyperion Planning / Hyperion Essbase / OBIEE.Oracle 11g and 12c / Oracle Essbase and Windows 2008.Language : Experience in PL / SQL , Unix Shell Scripting, Essbase scripting, OBIEE RPD development.Windows...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
    • Dinaikkan pangkat
    Databricks Architect

    Databricks Architect

    AVANADE ASIA PTE LTDKuala Lumpur, Kuala Lumpur, Malaysia
    Add expected salary to your profile for insights.The Databricks Solution Architect is a senior technical leadership role responsible for defining, designing, and overseeing the implementation of sc...Tunjukkan lagiKemas kini terakhir: 17 hari yang lalu
    • Dinaikkan pangkat
    Senior Data Scientist

    Senior Data Scientist

    BTI Executive Search Pte LtdKuala Lumpur, Kuala Lumpur, Malaysia
    AI Developer (Gaussian Splatting and Hyper-Realistic 3D Human Reconstruction).Design, build, and improve ML models for various business applications including classification, prediction, recommenda...Tunjukkan lagiKemas kini terakhir: 17 hari yang lalu
    • Dinaikkan pangkat
    65097 Data Architect

    65097 Data Architect

    half the skyKuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia
    Responsible for data lake design and providing technical direction to data engineer, data analysts, data integration developers and database administrators. Provide input, monitor, and coordinate th...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
    • Dinaikkan pangkat
    Data Engineer

    Data Engineer

    QS Quacquarelli SymondsKuala Lumpur, Kuala Lumpur, Malaysia
    This position offers a hybrid work model, allowing flexibility between working from home and our office.Typically, employees are expected to work 2 days in the office per week.At QS, we believe tha...Tunjukkan lagiKemas kini terakhir: 29 hari yang lalu
    • Dinaikkan pangkat
    Regional Data & Analytics Head

    Regional Data & Analytics Head

    SCIENTEKuala Lumpur, Kuala Lumpur, Malaysia
    Director - Global Talent Acquisition (Services & Projects).Lead the regional Data & AI strategy, governance, and delivery to drive innovation, compliance, and measurable business impact.Define and ...Tunjukkan lagiKemas kini terakhir: 9 hari yang lalu
    • Dinaikkan pangkat
    Senior Murex Datamart Developer

    Senior Murex Datamart Developer

    LUXOFT MALAYSIA SDN. BHD.Kuala Lumpur, Kuala Lumpur, Malaysia
    Basic understanding of Financial products.Knowledge of SQL, Murex Data model.Experience in supporting Treasury & Markets Applications is a plus. Working in a very collaborative and agile team, you a...Tunjukkan lagiKemas kini terakhir: 13 hari yang lalu
    • Dinaikkan pangkat
    Data Engineer (Junior / Senior)

    Data Engineer (Junior / Senior)

    Hytech Consulting Management Sdn BhdKuala Lumpur, Kuala Lumpur, Malaysia
    Add expected salary to your profile for insights.The Data Engineer will oversee the company’s data ingestion / integration work, including developing a data model, maintaining a data warehouse and ...Tunjukkan lagiKemas kini terakhir: 13 hari yang lalu
    • Dinaikkan pangkat
    Senior Developer - OpenText AppWorks

    Senior Developer - OpenText AppWorks

    LUXOFT MALAYSIA SDN. BHD.Kuala Lumpur, Kuala Lumpur, Malaysia
    Our client, a leading bank in Asia with a global network of more than 500 branches and offices in 19 countries and territories in Asia Pacific, Europe and North America, is looking for Consultants ...Tunjukkan lagiKemas kini terakhir: 17 hari yang lalu
    • Dinaikkan pangkat
    • Baharu!
    Senior Data Engineer

    Senior Data Engineer

    TECHTIERA SDN. BHD.Kuala Lumpur, Kuala Lumpur, Malaysia
    We are seeking a skilled and proactive Senior Data Engineer to design, develop, and optimize our modern cloud-native data warehouse on AWS. This role focuses on building scalable, secure, and high-p...Tunjukkan lagiKemas kini terakhir: 20 jam yang lalu
    • Dinaikkan pangkat
    • Baharu!
    Python Developer (Intermediate / Senior)

    Python Developer (Intermediate / Senior)

    Private AdvertiserPetaling Jaya, Selangor, Malaysia
    Intermediate and Senior Python Developer.In this full‑time role, you will be responsible for designing, developing, and implementing scalable and efficient Python‑based solutions to support our gro...Tunjukkan lagiKemas kini terakhir: 18 jam yang lalu
    • Dinaikkan pangkat
    64995 Data Modeler

    64995 Data Modeler

    half the skyKuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia
    Avanade is the leading provider of innovative digital, cloud and advisory services, industry solutions and design-led experiences across the Microsoft ecosystem. Every day, our 59,000 professionals ...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
    • Dinaikkan pangkat
    Senior Fullstack Developer

    Senior Fullstack Developer

    HyredKuala Lumpur, Malaysia
    The Senior Full Stack Developer is responsible for designing, developing, and maintaining high-quality web applications across the full technology stack. This role requires strong technical expertis...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
    • Dinaikkan pangkat
    Associate I, Data Engineer

    Associate I, Data Engineer

    Air SelangorKuala Lumpur, Kuala Lumpur, Malaysia
    Assist in the development of data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams. To work on optimal data delivery architecture that is con...Tunjukkan lagiKemas kini terakhir: 4 hari yang lalu