Databricks Data Engineer

Summary

Our talented Data & AI Practice is made up of globally recognized experts - and there’s room for more analytical and ambitious data professionals. If you’re passionate about helping clients make better data-driven decisions to tackle their most complex business issues, let’s talk. Take your skills to a new level and launch a career where you can truly do what matters.

Key Responsibilities

• Data Pipeline Development : Design, build, and maintain robust and scalable ETL / ELT pipelines using Databricks, PySpark / Scala, and SQL to ingest, transform, and load data from diverse sources (e.g., databases, APIs, streaming services) into Delta Lake.
• Databricks Ecosystem Utilization : Utilize core Databricks features such as Delta Lake, Databricks Workflows (or Jobs), Databricks SQL, and Unity Catalog for pipeline orchestration, data management, and governance.
• Performance Optimization : Tune and optimize Spark jobs and Databricks clusters for maximum efficiency, performance, and cost-effectiveness.
• Data Quality and Governance : Implement data quality checks, validation rules, and observability frameworks. Adhere to data governance policies and leverage Unity Catalog for fine‑grained access control.
• Collaboration : Work closely with Data Scientists, Data Analysts, and business stakeholders to translate data requirements into technical solutions and ensure data is structured to support analytics and machine learning use cases.
• Automation & DevOps : Implement CI / CD and DataOps principles for automated deployment, testing, and monitoring of data solutions.
• Documentation : Create and maintain technical documentation for data pipelines, data models, and processes.
• Troubleshooting : Monitor production pipelines, troubleshoot complex issues, and perform root cause analysis to ensure system reliability and stability.

Equal Employment Opportunity Statement

All employment decisions shall be made without regard to age, race, creed, color, religion, sex, national origin, ancestry, disability status, veteran status, sexual orientation, gender identity or expression, genetic information, marital status, citizenship status or any other basis as protected by federal, state, or local law.

Additional Information

Job candidates will not be obligated to disclose sealed or expunged records of conviction or arrest as part of the hiring process.

Avanade is committed to providing veteran employment opportunities to our service men and women.

Please read Avanade's Hiring and Privacy Statement for more information on how we process your data during the Recruiting and Hiring process.

#J-18808-Ljbffr