Job Title: Lead Data Engineer - MSS
Location: Irvine, CA
Full Time
Job Description
Lead Data Engineer - MSS
Must Have Technical/Functional Skills
Data Bricks , EBT ,Airflow , Asset Management exp
Roles & Responsibilities
Lead Data Engineer MSS
Experience
8 15 Years
Role Overview
We are looking for a highly skilled Lead Data Engineer with strong hands-on experience in Databricks, dbt, and Python, and a proven track record of leading and executing data platform migration and modernization programs. The ideal candidate will have experience migrating legacy data warehouses, ETL platforms, and cloud/on-prem data ecosystems to a modern Databricks Lakehouse architecture using dbt-based transformation frameworks. This is a hands-on leadership role requiring deep technical expertise, solution design capabilities, and the ability to mentor engineering teams while driving enterprise-scale data modernization initiatives.
Key Responsibilities
Data Engineering & Development
Design, build, and maintain scalable data pipelines using Databricks, PySpark, SQL, and Python.
Develop and optimize ELT/ETL processes for large-scale data ingestion and transformation.
Implement robust data quality, monitoring, and reconciliation frameworks.
Build reusable data engineering components and frameworks .
Data Platform Migration & Modernization
Lead migration of legacy data platforms, data warehouses, and ETL ecosystems to Databricks Lakehouse.
Transform existing ETL workloads to modern ELT patterns using dbt.
Analyze source environments and define migration approaches, roadmap, and execution strategy.
Drive code conversion, performance optimization, and workload modernization efforts.
Databricks Engineering
Develop solutions leveraging:
Databricks Lakehouse Platform
Delta Lake
Unity Catalog
Databricks Workflows
Structured Streaming
Medallion Architecture (Bronze, Silver, Gold)
Optimize Spark jobs and Databricks workloads for performance and cost efficiency.
dbt Development
Build and maintain dbt models, macros, tests, and documentation.
Implement incremental processing and reusable transformation frameworks.
Establish data lineage, testing, and CI/CD best practices.
Work closely with business and analytics teams to build trusted datasets.
Leadership & Collaboration
Lead a team of data engineers and developers.
Perform code reviews and enforce engineering standards.
Collaborate with archit ects, product owners, business analysts, and stakeholders.
Mentor junior engineers and drive adoption of best practices.
.... If you love patient-centered health care with real relationships inside a company that encourages fun on and off the clock, then DaVita is the place for you. We offer career options to fit your lifestyle. Here is what you can expect when you join our Village as an Administrative...
...team works across different cultures and time zones, bringing our products to more than 7 million customers, making us one of Europes... ...changing team. Lets go!#### Your mission As Principal Product Designer at Bitpanda, you will shape the long-term direction of our...
...not a typical translation project. You are not translating marketing copy or generic product documentation. You are helping extend access... ...Work ~Translate structured system content from English into Spanish (LATAM)~Preserve meaning, tone, and conceptual integrity ...
...STAFF ACCOUNTANT (Pittsburgh, Pa) Compensation: Beginning at $50,000 Location: 330 Joseph Street, Pittsburgh, PA Type of Employment: Full-Time Who We Are Donnelly-Boland and Associates is a woman-owned CPA and management consulting firm that helps small businesses...
Turn computational work from your own earth, ocean, or climate science research into paid evaluation tasks for AI models. Open to research-track Bachelor's students, Master's and PhD students, postdocs, and recent graduates. AfterQuery is building a network of researchers...