Big Data Engineer

Job Summary :

We are seeking a highly skilled Big Data Engineer with 8 years of experience in data migration , data setup , and data systems development . The ideal candidate will have deep expertise in Apache Spark , SQL , and Java (with Scala) for large-scale data processing, reporting, and system development. Strong knowledge of data architecture , semantic layer development , and experience in regression testing and cutover activities for enterprise-level migrations is essential.

Key Responsibilities :

Spark :

• Design, develop, and optimize Spark-based ETL pipelines for large-scale data processing and analytics.
• Utilize Spark SQL, DataFrames, RDDs, and Streaming for efficient data transformations.
• Tune Spark jobs for performance, including memory management, partitioning, and execution plans.
• Implement real-time and batch data processing using Spark Streaming or Structured Streaming.

SQL :

Java (Preferred with Scala Knowledge) :

Required Skills & Qualifications :

Preferred Qualifications :

#J-18808-Ljbffr