Talent.com
Big Data Hadoop Developer

Big Data Hadoop Developer

Unison Consulting Pte LtdKuala Lumpur, Kuala Lumpur, Malaysia
30+ hari lalu
Penerangan pekerjaan

Job Summary :

We are looking for a Big Data Hadoop Developer to design, develop, and maintain large-scale data processing solutions. The ideal candidate should have strong hands-on experience with the Hadoop ecosystem and integration with relational databases such as MariaDB or Oracle DB for analytics and reporting.

Key Responsibilities :

  • Design, develop, and optimize Hadoop-based big data solutions for batch and real-time data processing.
  • Work with data ingestion frameworks to integrate data from MariaDB / Oracle DB into Hadoop (Sqoop, Apache Nifi, Kafka).
  • Implement Hive, Spark, and MapReduce jobs for data transformation and analytics.
  • Optimize Hive queries, Spark jobs, and HDFS usage for performance and cost efficiency.
  • Create and maintain ETL pipelines for structured and unstructured data.
  • Troubleshoot and resolve issues in Hadoop jobs and database connectivity.
  • Collaborate with BI, analytics, and data science teams for data provisioning.
  • Ensure data security, governance, and compliance in all solutions.

Technical Skills :

  • Big Data Ecosystem : Hadoop (HDFS, YARN), Hive, Spark, Sqoop, MapReduce, Oozie, Flume.
  • Databases : MariaDB and / or Oracle DB (SQL, PL / SQL).
  • Programming : Java, Scala, or Python for Spark / MapReduce development.
  • Data Ingestion : Sqoop, Kafka, Nifi (for integrating RDBMS with Hadoop).
  • Query Optimization : Hive tuning, partitioning, bucketing, indexing.
  • Tools : Ambari, Cloudera Manager, Git, Jenkins.
  • OS & Scripting : Linux / Unix shell scripting.

    • Soft Skills :

  • Strong analytical skills and problem-solving abilities.
  • Good communication skills for working with cross-functional teams.
  • Ability to manage priorities in a fast-paced environment.

    • Nice to Have :

  • Experience with cloud-based big data platforms (AWS EMR, Azure HDInsight, GCP Dataproc).
  • Knowledge of NoSQL databases (HBase, Cassandra).
  • Exposure to machine learning integration with Hadoop / Spark.

    • #J-18808-Ljbffr

    Buat amaran kerja untuk carian ini

    Big Data Hadoop Developer • Kuala Lumpur, Kuala Lumpur, Malaysia

    Pekerjaan yang berkaitan
    • Dinaikkan pangkat
    AVP, Data Integration (pySpark, Nifi, Hadoop)

    AVP, Data Integration (pySpark, Nifi, Hadoop)

    MaybankKuala Lumpur, Kuala Lumpur, Malaysia
    AVP, Data Integration (pySpark, Nifi, Hadoop).Kuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia.Get AI-powered advice on this job and more exclusive features. Implement ETL systems that are ...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
    • Dinaikkan pangkat
    Lead Software Engineer (Big Data)

    Lead Software Engineer (Big Data)

    EPAM SystemsKuala Lumpur, Kuala Lumpur, Malaysia
    EPAM Systems is seeking a highly skilled and motivated.Big Data, Machine Learning Operations, and scalable enterprise software development. Join our dynamic team and take the lead in designing and i...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
    Big Data Hadoop Developer

    Big Data Hadoop Developer

    Unison GroupKuala Lumpur, Federal Territory of Kuala Lumpur, MY
    Quick Apply
    We are looking for a Big Data Hadoop Developer to design, develop, and maintain large-scale data processing solutions.The ideal candidate should have strong hands-on experience with the Hadoop ecos...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
    • Dinaikkan pangkat
    Data Visualizer Developer (Experience-Hire)

    Data Visualizer Developer (Experience-Hire)

    Accenture Southeast AsiaKuala Lumpur, Kuala Lumpur, Malaysia
    ATCM - Data Visualizer Developer (Entry-Level to Experience-Hire).ATCM - Data Visualizer Developer (Entry-Level to Experience-Hire). A data visualizer developer who designs and builds interactive da...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
    • Dinaikkan pangkat
    Data Scientist

    Data Scientist

    Great PyramidKapar, Selangor, Malaysia
    Our Clientis a Malaysia-based motorcycle hire-purchase financing provider offering quick approvals, flexible 12- 60 month installment plans, and up to 90% financing coverage.They are seeking fora D...Tunjukkan lagiKemas kini terakhir: 4 hari yang lalu
    • Dinaikkan pangkat
    Data Scientist II

    Data Scientist II

    Asia Digital Engineering (ADE)Sepang, Selangor, Malaysia
    Asia Digital Engineering (ADE).Asia Digital Engineering (ADE) is transforming aircraft maintenance through data and innovation. As a wholly-owned subsidiary of Capital A Berhad, we’re building the f...Tunjukkan lagiKemas kini terakhir: 11 hari yang lalu
    • Dinaikkan pangkat
    Data Scientist (Python, Problem Solving, Machine Learning)

    Data Scientist (Python, Problem Solving, Machine Learning)

    Daimler Trucks North America LLCKlang City, Selangor, Malaysia
    Job Description - Data Scientist (Python, Problem Solving , Machine Learning) (MER0003L3Q).Data Scientist (Python, Problem Solving , Machine Learning) Group : Mercedes-Benz Group AG.At Mercedes-Ben...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
    • Dinaikkan pangkat
    Deployment Engineer- MY (Big Data / Hadoop Admin)

    Deployment Engineer- MY (Big Data / Hadoop Admin)

    Tookitaki Holding PTE LTDKuala Lumpur, Kuala Lumpur, Malaysia
    Job Title : Deployment Engineer | Department : Services Delivery | Reporting To : Regional Head of Service Delivery.The Deployment Engineer is a critical technical role within Tookitaki’s Services Del...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
    • Dinaikkan pangkat
    Developer - Hadoop, Teradata, Python

    Developer - Hadoop, Teradata, Python

    LuxoftKuala Lumpur, Kuala Lumpur, Malaysia
    Our Client a leading bank in Asia with a global network of more than 500 branches and offices in 19 countries and territories in Asia Pacific, Europe, and North America, are looking for Consultants...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
    • Dinaikkan pangkat
    Senior Software Engineer (Big Data)

    Senior Software Engineer (Big Data)

    EPAM SystemsKuala Lumpur, Kuala Lumpur, Malaysia
    EPAM Systems Kuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia.Software Engineers at EPAM are the driving force behind strategic initiatives for our clients. EPAM Malaysia, you will use your...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
    • Dinaikkan pangkat
    • Baharu!
    Head, Data Analytics

    Head, Data Analytics

    AFFIN GroupKuala Lumpur, Kuala Lumpur, Malaysia
    Create the future with Affin! You too can make a difference.We continuously innovate to transform our financial services landscape - making banking better and easier. Join us at AFFIN, where the ope...Tunjukkan lagiKemas kini terakhir: 6 jam yang lalu
    • Dinaikkan pangkat
    Oracle Developer (remote)

    Oracle Developer (remote)

    PEOPLE PROFILERSKuala Selangor, Kuala Selangor, Malaysia
    Kuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia.Job Description : Job Ref : QY7VW585.People Profilers is hiring on behalf of a global tire company, fully integrated across R&D, manufacturing...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
    • Dinaikkan pangkat
    Utilities Engineer

    Utilities Engineer

    LonzaSemenyih, Selangor, Malaysia
    Today, Lonza is a global leader in life sciences operating across five continents.While we work in science, there’s no magic formula to how we do it. Our greatest scientific solution is dedicated in...Tunjukkan lagiKemas kini terakhir: 4 hari yang lalu
    • Dinaikkan pangkat
    Data Scientist

    Data Scientist

    Michael PageKuala Selangor, Kuala Selangor, Malaysia
    Work in exciting new projects with new technologies.Opportunity to work in cross functional international teams.The hiring company is a large organization within the industrial / manufacturing indust...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
    • Dinaikkan pangkat
    Data Developer

    Data Developer

    The Access GroupKuala Lumpur, Kuala Lumpur, Malaysia
    Data Developer page is loaded## Data Developerremote type : Hybridlocations : Kuala Lumpur - GOCtime type : Full timeposted on : Posted Todayjob requisition id : JR We’re looking for people to...Tunjukkan lagiKemas kini terakhir: 5 hari yang lalu
    • Dinaikkan pangkat
    Head of Product - Data

    Head of Product - Data

    TeleportKuala Lumpur, Kuala Lumpur, Malaysia
    International trade is a trillion-dollar business, with air freight representing : 35% of the addressable market.In Southeast Asia alone, the total market size is : US$4B+, with sector growth driven ...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
    • Dinaikkan pangkat
    Data Architect

    Data Architect

    CapgeminiKuala Lumpur, Kuala Lumpur, Malaysia
    Explore our latest thought leadership, ideas, and insights on the issues that are shaping the future of business and society. Choose a partner with intimate knowledge of your industry and first-hand...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
    • Dinaikkan pangkat
    Senior Java Developer

    Senior Java Developer

    Cultivar Staffing & SearchSeremban, Negeri Sembilan, Malaysia
    Our client is a tech start-up from China, and its regional office in Singapore is hiring remote engineers and technicians to support their projects. Design, develop, and optimize back-end systems fo...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu