Job Description
Job Title: AWS Databricks Lead
Location: Eagan MN
Work Mode: Hybrid / Onsite
Experience: 10+ years overall, 4+ years in Databricks on AWS
Role Summary
The AWS Databricks Lead will own the technical vision and execution for data engineering solutions on Databricks deployed on AWS. This role combines leadership, architecture expertise, and hands-on development to deliver secure, scalable, and cost-optimized data platforms for advanced analytics and AI.
Key Responsibilities
- Design and implement end-to-end data lakehouse architecture using Databricks on AWS.
- Build ETL/ELT pipelines leveraging Databricks Medallion Architecture and Data Vault 2.0 principles.
- Lead data migration projects from legacy systems to AWS cloud.
- Orchestrate jobs and workflows using Databricks Workspace .
- Define and enforce data governance, security, and compliance frameworks using Unity Catalog, AWS IAM, and KMS.
- Mentor and guide teams on best practices for Databricks development and AWS-native integrations.
- Collaborate with stakeholders to translate business requirements into technical solutions.
Required Skills
- Strong Healthcare domain experience .
- Proficient in SQL for data analysis and transformation.
- Hands-on experience with AWS native services (S3, Glue, Redshift, EMR, Lambda).
- Expertise in Databricks development using AWS or equivalent cloud tech stack.
- Experience building ETL/ELT pipelines using Databricks Workspace and PySpark .
- Knowledge of job orchestration within Databricks.
- Familiarity with Data Vault 2.0 architecture for data modeling.
- Proven track record in data migration projects .
Certifications (Preferred)
- AWS Certified Solutions Architect – Professional .
- Databricks Certified Professional Data Engineer .
Job Tags