This description is a summary of our understanding of the job description. Click on 'Apply' button to find out more.

Role Description

This role involves delivering a phased data strategy and AI enablement engagement for a client with 30+ datasets in Databricks.

• Profile all 30+ datasets in Databricks: table structures, row counts, data types, distributions, refresh patterns
• Document schemas with inferred relationships and primary/foreign key candidates
• Assess data quality across dimensions: completeness, consistency, accuracy, freshness
• Analyze historical data behavior — determine which datasets use snapshot vs. overwrite patterns
• Support API and integration mapping (test data extraction capabilities)
• Build standardized ingestion framework and data pipelines (Phase 2)
• Implement data quality gates with automated validation and alerting (Phase 2)
• Support workflow integration, feature engineering pipelines, and ML data products (Phases 3-4)

Qualifications

• Strong SQL and Python skills
• Experience with Databricks (notebooks, Spark SQL, Delta Lake)
• Hands-on data profiling, data quality assessment, and technical documentation
• ETL/ELT pipeline development experience
• Comfort working in locked-down enterprise environments with restricted internet access
• Comfort with undocumented, messy data — you'll be making sense of datasets that have limited or no documentation
• Eager to learn AI tooling

Requirements

• Financial services, lending, or banking data experience (strongly preferred)
• Experience with Medallion Architecture (bronze/silver/gold patterns) (strongly preferred)
• Familiarity with Power BI as a downstream consumer (strongly preferred)
• Experience working within VDI-based access environments (strongly preferred)
• Experience with modern AI tool sets (strongly preferred)

Environment

The client's environment is managed with strict security controls. Access is through VDI (Windows) RDP into a dedicated server Databricks. Internet access on work servers is limited. You must be comfortable working within these constraints.