ETL Cloud Data Engineer
Mission Consultancy ServicesKuala Lumpur, Kuala Lumpur, Malaysia30+ hari lalu
Penerangan pekerjaanStrong Python programming skills for data processing and ETL automation. Proficiency in SQL and database design (relational and non-relational databases). In-depth knowledge of AWS services, including AWS Glue, Redshift, S3, and EMR. Experience in building ETL pipelines on cloud-based environments (preferably AWS). Hands-on experience with REST API development and integration. Familiarity with data warehousing concepts and best practices. Understanding of data modeling, API testing, and troubleshooting. Experience with data governance, data security, and compliance frameworks. Exposure to big data technologies like Spark, Hadoop, or Kafka. Knowledge of containerization (Docker, Kubernetes) and DevOps practices. Minimum 5 years of experience as a Data Engineer with ETL cloud experience, preferably AWS. AWS or Azure certification preferred. Strong Python programming skills and experience building APIs. Malaysian citizen or permanent resident. Oil & Gas industry experience is a plus. Degree in Computer Science, Engineering, or equivalent.
Job Summary :
We are looking for a highly skilled Python / ETL Data Engineer to design, develop, and optimize data pipelines using AWS Glue, AWS Athena, AWS Data Pipeline, and other ETL tools. The ideal candidate will have strong expertise in Python, SQL, AWS, and data pipeline management, ensuring smooth data extraction, transformation, and loading (ETL) processes.
Key Responsibilities :
- Develop and maintain data pipelines using AWS Glue, AWS Athena, AWS Data Pipeline, and other ETL tools.
- Implement monitoring and early detection processes for pipeline issues such as missing or lagging data.
- Extract, transform, and load (ETL) data from various sources, including relational databases, non-relational databases, and flat files.
- Optimize and transform data to meet business requirements and load it into target data stores.
- Troubleshoot and optimize performance of data pipelines for scalability and efficiency.
- Collaborate with Data Scientists and Business Analysts to understand data requirements and implement solutions.
- Build and maintain REST APIs for data access and integration with external systems.
- Ensure data security and compliance best practices across all pipelines.
- Document and test ETL processes to maintain high-quality data flows.
Required Skills :
Qualifications :
#J-18808-Ljbffr
Buat amaran kerja untuk carian ini
Data Engineer • Kuala Lumpur, Kuala Lumpur, Malaysia
Pekerjaan yang berkaitan
- Dinaikkan pangkat
Data Engineer
Encora Digital IncKuala Lumpur, Kuala Lumpur, Malaysia Kuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia.Data Engineer to design, build, and maintain scalable data pipelines and infrastructure.
Will work closely with data scientists, analysts, a...Tunjukkan lagiKemas kini terakhir: 3 hari yang lalu
- Dinaikkan pangkat
Data Engineer
GokardzKuala Lumpur, Kuala Lumpur, Malaysia We are seeking a skilled and experienced Data Engineer to join our team and contribute to our data-driven initiatives.The ideal candidate will have a strong background in data integration, data war...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
- Dinaikkan pangkat
- Baharu!
Lead Data Engineer
Rotate Asia Sdn BhdKuala Lumpur, Kuala Lumpur, Malaysia Add expected salary to your profile for insights.At Rotate, we’re building the go-to team for commercial decision-making tools in the air cargo industry.
By combining cutting-edge software with deep...Tunjukkan lagiKemas kini terakhir: 16 jam yang lalu
- Dinaikkan pangkat
Senior Data Engineer
Michael PageSubang Jaya, Selangor, Malaysia Competitive salary and benefits.MNC working culture with hands on experience in various regional projects.Our client is a well-established multinational company in the industrial and manufacturing ...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
- Dinaikkan pangkat
Senior Data Engineer
KHIND MalaysiaShah Alam, Selangor, Malaysia We are seeking an experienced Data Engineer with 2 years of professional experience in building and maintaining data pipelines and working with cloud services.
The ideal candidate will have strong t...Tunjukkan lagiKemas kini terakhir: 6 hari yang lalu
- Dinaikkan pangkat
ETL Developer
CollaberaKuala Lumpur, Kuala Lumpur, Malaysia Senior ETL Developer – Location : Federal Territory of Kuala Lumpur, Malaysia.Design, develop, and maintain ETL processes to extract, transform, and load data from various sources into the Banking d...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
- Dinaikkan pangkat
ETL Developer
HCLTechKuala Lumpur, Kuala Lumpur, Malaysia Lead Recruiter / Lead Talent Acquisition Partner.HCLTech is seeking a highly skilled and experienced.The ideal candidate will have hands-on expertise in.
You will be responsible for designing, devel...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
- Dinaikkan pangkat
Senior Data Engineer (Talend ETL PL / SQL)
HCLTechKuala Lumpur, Kuala Lumpur, Malaysia Kuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia.Direct message the job poster from HCLTech.As a, you are responsible for designing, developing, and implementing scalable data store soluti...Tunjukkan lagiKemas kini terakhir: 6 hari yang lalu
- Dinaikkan pangkat
Senior Data Engineer
GXBankPetaling Jaya, Selangor, Malaysia Get AI-powered advice on this job and more exclusive features.Participate in all aspects of developing a data warehouse system.
Design, develop, test, deploy and support scalable and flexible data w...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
- Dinaikkan pangkat
Lead Data Engineer
Quantexa LimitedKuala Lumpur, Kuala Lumpur, Malaysia It isn’t often you get to be part of a tech company that, since 2016, has been innovating the data analytics market in ways no-one else can.
Our technology started out in FinTech, helping tackle ser...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
- Dinaikkan pangkat
Senior Data Engineer
Hiredly XKuala Lumpur, Kuala Lumpur, Malaysia The Senior Data Engineer is responsible for designing, developing, and optimizing robust data solutions that support business intelligence and analytics needs.
This role requires both technical expe...Tunjukkan lagiKemas kini terakhir: 4 hari yang lalu
- Dinaikkan pangkat
Azure Data Engineer
AvanadeKuala Lumpur, Kuala Lumpur, Malaysia Azure Data Engineer role at Avanade in Kuala Lumpur, Malaysia.Focus on designing and building modern data pipelines, data streams, and reporting tools to enable insightful decision-making for Avana...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
- Dinaikkan pangkat
Cloud Engineer
Electrolux GroupPetaling Jaya, Selangor, Malaysia Power a world of connected solutions.At the Electrolux Group, a leading global appliance company, we strive every day to shape living for the better for our consumers, our people and our planet.We ...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
- Dinaikkan pangkat
Lead Data Engineer
QuantexaKuala Lumpur, Kuala Lumpur, Malaysia It isn’t often you get to be part of a tech company that, since 2016, has been innovating the data analytics market in ways no-one else can.
Our technology started out in FinTech, helping tackle ser...Tunjukkan lagiKemas kini terakhir: 30+ hari yang lalu
- Dinaikkan pangkat
Data Engineer (Google cloud platform)
CognizantKuala Lumpur, Kuala Lumpur, Malaysia APAC Talent Acquisition Lead at Cognizant.Responsible for building and maintaining data pipelines, managing data storage solutions, and ensuring efficient data processing using Google Cloud Platfor...Tunjukkan lagiKemas kini terakhir: 15 hari yang lalu
- Dinaikkan pangkat
Data Engineer
Cradle FundKuala Lumpur, Kuala Lumpur, Malaysia Assistant Manager Human Resource at Cradle Fund Sdn Bhd.The Data Engineer is responsible for assisting in the design and development of data pipelines that enable reliable, timely, and scalable dat...Tunjukkan lagiKemas kini terakhir: 15 hari yang lalu
- Dinaikkan pangkat
Senior Data Engineer
TrangloKuala Lumpur, Kuala Lumpur, Malaysia Tranglo WP, Kuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia.Drive the implementation and optimization of.Identify, diagnose, and resolve data-related issues.
Provide advanced support and t...Tunjukkan lagiKemas kini terakhir: 6 hari yang lalu
- Dinaikkan pangkat
Data Engineer
Avensys ConsultingKuala Lumpur, Kuala Lumpur, Malaysia Sr IT Recruiter - APAC (Singapore, Malaysia, South Korea, Japan, Thailand and Philippines) at Avensys Consulting.Avensys is a reputed global IT professional services company headquartered in Singap...Tunjukkan lagiKemas kini terakhir: 26 hari yang lalu