Data Platform Tech Lead

Company Description We're looking for a Tech Lead who is more than a great data engineer - someone who brings equal depth in backend software engineering and broad fluency across data technologies, databases, and large-scale distributed systems. You'll own the architecture and hands-on development of our data platform while driving engineering excellence across the team. Job Description • Design, build, and maintain large-scale batch and streaming data pipelines (ETL/ELT) across multiple products • Develop and optimize Spark workloads for performance, cost, and reliability on large data volumes • Build and evolve our Data Lake / Lakehouse, including data modeling, partitioning, and storage formats • Write production-grade backend services and REST APIs that expose data platform capabilities • Build in data quality, observability, and monitoring from the start, and own your pipelines in production end to end • Automate and improve the data platform, including AI-assisted development and LLM-powered data workflows • Work closely with R&D and product teams, and contribute to design and code reviews Qualifications • 6+ years of hands-on Data Engineering experience building and operating production-scale pipelines • Proven experience with building and maintaining a large scale Data Lake • Deep, hands-on experience with Spark (tuning, partitioning, debugging large jobs) • Strong Python and advanced SQL • Backend development experience: REST APIs, microservices, service design patterns • Broad database experience across relational, analytical/columnar, NoSQL, and streaming stores • Experience with cloud-based data platforms • Practical experience with AI tools and agents, such as integrating or orchestrating LLM-powered workflows in production • Strong engineering fundamentals: testing, CI/CD, code quality, and performance profiling • Self-directed: you take a problem from design to production with little guidance and suggest improvements without being asked Nice to have • Hands-on Databricks experience is a major advantage • Azure data services or EventHub • Streaming frameworks and real-time processing (e.g., Spark Structured Streaming, Kafka, Flink) • Table formats such as Delta Lake or Iceberg • Background in cyber security, SaaS, or large-scale enterprise environments • Experience with data governance, observability, or self-service platforms #LI-NK2