Talent.com
Tawaran kerja ini tidak tersedia di negara anda.
Big Data Hadoop Developer

Big Data Hadoop Developer

Unison GroupKuala Lumpur, Federal Territory of Kuala Lumpur, MY
17 hari lalu
Jenis pekerjaan
  • Quick Apply
Penerangan pekerjaan

Job Summary :

We are looking for a Big Data Hadoop Developer to design, develop, and maintain large-scale data processing solutions. The ideal candidate should have strong hands-on experience with the Hadoop ecosystem and integration with relational databases such as MariaDB or Oracle DB for analytics and reporting.

Key Responsibilities :

  • Design, develop, and optimize Hadoop-based big data solutions for batch and real-time data processing.
  • Work with data ingestion frameworks to integrate data from MariaDB / Oracle DB into Hadoop (Sqoop, Apache Nifi, Kafka).
  • Implement Hive, Spark, and MapReduce jobs for data transformation and analytics.
  • Optimize Hive queries, Spark jobs, and HDFS usage for performance and cost efficiency.
  • Create and maintain ETL pipelines for structured and unstructured data.
  • Troubleshoot and resolve issues in Hadoop jobs and database connectivity.
  • Collaborate with BI, analytics, and data science teams for data provisioning.
  • Ensure data security, governance, and compliance in all solutions.

Technical Skills :

  • Big Data Ecosystem : Hadoop (HDFS, YARN), Hive, Spark, Sqoop, MapReduce, Oozie, Flume.
  • Databases : MariaDB and / or Oracle DB (SQL, PL / SQL).
  • Programming : Java, Scala, or Python for Spark / MapReduce development.
  • Data Ingestion : Sqoop, Kafka, Nifi (for integrating RDBMS with Hadoop).
  • Query Optimization : Hive tuning, partitioning, bucketing, indexing.
  • Tools : Ambari, Cloudera Manager, Git, Jenkins.
  • OS & Scripting : Linux / Unix shell scripting.

    • Soft Skills :

  • Strong analytical skills and problem-solving abilities.
  • Good communication skills for working with cross-functional teams.
  • Ability to manage priorities in a fast-paced environment.

    • Nice to Have :

  • Experience with cloud-based big data platforms (AWS EMR, Azure HDInsight, GCP Dataproc).
  • Knowledge of NoSQL databases (HBase, Cassandra).
  • Exposure to machine learning integration with Hadoop / Spark.
    • Buat amaran kerja untuk carian ini

    Developer • Kuala Lumpur, Federal Territory of Kuala Lumpur, MY

    Pekerjaan yang berkaitan
    • Dinaikkan pangkat
    • Baharu!
    Data Analytics Lead

    Data Analytics Lead

    LearntasticKlang City, Selangor, Malaysia
    About Us : We are a fast-growing continuous education company with multiple brands in the e-learning and certification space. Our mission is to provide high-quality, accessible courses for profession...Tunjukkan lagiKemas kini terakhir: 6 jam yang lalu
    • Dinaikkan pangkat
    • Baharu!
    Freelance AI Solutions Engineer - Generative AI & Data Workflows

    Freelance AI Solutions Engineer - Generative AI & Data Workflows

    MindriftKepong, Kuala Lumpur, Malaysia
    Freelance AI Solutions Engineer - Generative AI & Data Workflows.Be among the first 25 applicants.This opportunity is only for candidates currently residing in the specified country.Your location m...Tunjukkan lagiKemas kini terakhir: 6 jam yang lalu
    • Dinaikkan pangkat
    • Baharu!
    Data Engineer_Kafka

    Data Engineer_Kafka

    Encora Inc.Kuala Lumpur, Kuala Lumpur, Malaysia
    Kuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia.Data Engineer_Kafka role at Encora Inc.Design, develop, and maintain real-time data pipelines using Apache Kafka and related technologies.B...Tunjukkan lagiKemas kini terakhir: 6 jam yang lalu
    • Dinaikkan pangkat
    • Baharu!
    Manager, Analytics (Senior Data Scientist)

    Manager, Analytics (Senior Data Scientist)

    Hong Leong Bank BerhadKuala Lumpur, Kuala Lumpur, Malaysia
    The Senior Data Scientist / Manager, Analytics leads advanced analytics projects for the Consumer and Business Banking units to drive tangible business outcomes such as increased sales, revenue, and ...Tunjukkan lagiKemas kini terakhir: 12 jam yang lalu
    • Dinaikkan pangkat
    • Baharu!
    Manager, Data Privacy

    Manager, Data Privacy

    CGC MalaysiaPetaling Jaya, Selangor, Malaysia
    Support the Data Protection Officer (DPO) in developing, implementing and maintaining effective internal data privacy policies and procedures in alignment with data protection requirements.Monitor ...Tunjukkan lagiKemas kini terakhir: 6 jam yang lalu
    • Dinaikkan pangkat
    • Baharu!
    Nearshore - Frontend Developer Consultant

    Nearshore - Frontend Developer Consultant

    BitoviKlang Municipal Council, Klang Municipal Council, Malaysia
    IMPORTANT TO NOTE : This job post is for a full-time, remote contractor role.It is focused on candidates from LATAM and other nearshore locations who can commit to working a minimum of 6 hours durin...Tunjukkan lagiKemas kini terakhir: 6 jam yang lalu
    • Dinaikkan pangkat
    Deployment Engineer- MY (Big Data / Hadoop Admin)

    Deployment Engineer- MY (Big Data / Hadoop Admin)

    Tookitaki Holding PTE LTDKuala Lumpur, Kuala Lumpur, Malaysia
    Job Title : Deployment Engineer | Department : Services Delivery | Reporting To : Regional Head of Service Delivery.The Deployment Engineer is a critical technical role within Tookitaki’s Services Del...Tunjukkan lagiKemas kini terakhir: 26 hari yang lalu
    • Dinaikkan pangkat
    • Baharu!
    Staff Data Lake Platform Engineering Lead

    Staff Data Lake Platform Engineering Lead

    bpKuala Lumpur, Kuala Lumpur, Malaysia
    Staff Data Lake Platform Engineering Lead.Kuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia.At bp, our purpose is to deliver energy for today and tomorrow. For over a century, we’ve been dis...Tunjukkan lagiKemas kini terakhir: 6 jam yang lalu
    • Dinaikkan pangkat
    • Baharu!
    Data Analyst

    Data Analyst

    Appvalley TechnologyKuala Lumpur, Kuala Lumpur, Malaysia
    Lead Data Loss Prevention Analyst (VP).Data Lost Prevention Analyst is an operational role within UOB’s Group Security Operations Center. The incumbent will be responsible for the continuous monitor...Tunjukkan lagiKemas kini terakhir: 6 jam yang lalu
    • Dinaikkan pangkat
    • Baharu!
    EMEA Senior Enterprise Sales Executive

    EMEA Senior Enterprise Sales Executive

    7SIGNALNilai, Negeri Sembilan, Malaysia
    We’re looking for a dynamic and experienced.Senior Enterprise Sales Executive.SaaS solutions to customers in.Manufacturing, Distribution, Healthcare, and Retail. The ideal candidate will bring 7+ ye...Tunjukkan lagiKemas kini terakhir: 6 jam yang lalu
    • Dinaikkan pangkat
    Data Operations Analyst(Reference Data)

    Data Operations Analyst(Reference Data)

    NielsenIQSubang Jaya, Selangor, Malaysia
    NIQ is the world’s leading consumer intelligence company, delivering the most complete understanding ofconsumer buying behavior and revealing new pathways to growth. In 2023, NIQ combined with GfK,b...Tunjukkan lagiKemas kini terakhir: 4 hari yang lalu
    • Dinaikkan pangkat
    Technical Marketing Engineer

    Technical Marketing Engineer

    Infotree Global SolutionsKuala Selangor, Kuala Selangor, Malaysia
    Job title : Developer : Technical Marketing - III.Max salary budget : RM8,000 / month.Experience : 5+ years in technical roles, 2+ years with developers, coding skills, and hands-on AI / edge / IoT experienc...Tunjukkan lagiKemas kini terakhir: 15 hari yang lalu
    • Dinaikkan pangkat
    • Baharu!
    Data Engineer (SQL, Python, Spark, Hive, Hadoop)

    Data Engineer (SQL, Python, Spark, Hive, Hadoop)

    Unison GroupKuala Lumpur, Kuala Lumpur, Malaysia
    The ideal candidate will have strong expertise in.SQL, Python, Apache Spark, Hive, and Hadoop.Design, develop, and maintain scalable ETL pipelines using Spark, Hive, and Hadoop.Write efficient SQL ...Tunjukkan lagiKemas kini terakhir: 6 jam yang lalu
    • Dinaikkan pangkat
    • Baharu!
    Freelance Developer Consultant (Kotlin) - AI Trainer

    Freelance Developer Consultant (Kotlin) - AI Trainer

    MindriftNilai, Negeri Sembilan, Malaysia
    Freelance Developer Consultant (Kotlin) - AI Trainer.This opportunity is a remote freelance role with Mindrift, focused on evolving GenAI models and contributing as an AI Tutor in Coding.Applicants...Tunjukkan lagiKemas kini terakhir: 6 jam yang lalu
    • Dinaikkan pangkat
    Senior.NET Developer

    Senior.NET Developer

    EcoVadisNilai, Negeri Sembilan, Malaysia
    Work smart, have fun and make an impact!.Our purpose is to guide all companies toward a sustainable world.EcoVadis is the leading provider of business sustainability ratings.Our solutions are backe...Tunjukkan lagiKemas kini terakhir: 17 hari yang lalu
    • Dinaikkan pangkat
    • Baharu!
    Senior Statistician

    Senior Statistician

    OPISKlang Municipal Council, Klang Municipal Council, Malaysia
    Due to our ongoing global expansion, we have a great opportunity for a.The Senior Statistician reports to the Biostatistics Coordinator, serves as the primary point of contact to the study team for...Tunjukkan lagiKemas kini terakhir: 6 jam yang lalu
    • Dinaikkan pangkat
    • Baharu!
    Senior Data Analyst

    Senior Data Analyst

    AirAsiaSepang, Selangor, Malaysia
    Partner with business and project leaders to deeply understand their goals, define critical metrics and KPIs, and deliver insights that drive action. Analyze complex datasets to uncover trends, perf...Tunjukkan lagiKemas kini terakhir: 6 jam yang lalu
    • Dinaikkan pangkat
    • Baharu!
    Thunderhead / Jasper Developer

    Thunderhead / Jasper Developer

    Accenture Southeast AsiaKuala Lumpur, Kuala Lumpur, Malaysia
    Get AI-powered advice on this job and more exclusive features.Develop and maintain business application and reporting components to meet project timelines, with a strong focus on Thunderhead Smart ...Tunjukkan lagiKemas kini terakhir: 6 jam yang lalu