Job Description

Technical Expertise:

• Data bricks Mastery:  Deep, expert-level knowledge of the Data bricks Platform, including:
  • Unity Catalog:  Designing and implementing data governance and security.
    
  • Delta Lake & Delta Live Tables:  Architecture and building reliable, scalable data pipelines.
    
  • Performance & Cost Optimization:  Expertise in tuning Spark jobs, optimizing cluster usage, and managing platform costs.
    
  • MLOps:  Strong, practical understanding of the machine learning lifecycle on Data bricks using tools like MLflow.
    
  • Data bricks SQL:  Knowledge of designing and optimizing analytical workloads.
    
  • Mosaic AI:  Knowledge of designing and optimizing AI Agents.
    
• Cloud & Infrastructure:  Deep knowledge of cloud architecture and services on AWS. Strong command of Infrastructure as Code (Terraform, YAML).
  
• Data Engineering & Programming:  Strong background in data modeling, ETL/ELT development, and advanced, hands-on programming skills in Python and SQL.
  
• CI/CD & Automation:  Experience with designing and implementing CI/CD pipelines (preferably with GitHub Actions) for data and ML workloads.
  
• Severability:  Familiarity with implementing monitoring, logging, and alerting for data platforms.
  
• Automation:  The platform is ephemeral, and all changes are implemented using Terraform and Python. Expertise in Terraform and Python is a must.
  

Job Tags

Similar Jobs