Engineering Manager - Ads Data Platform
Reddit is a community of communities. It's built on shared interests, passion, and trust and is home to the most open and authentic conversations on the internet. With more than 50 million people visiting 100,000+ communities daily, it is home to the most open and authentic conversations on the internet. From pets to parenting, skincare to stocks, there's a community for everybody on Reddit.
Reddit is continuing to grow our teams with the best talent. This role offers flexibility and is remote-friendly, however, it is important that the candidate is based in the proximity of the SF Bay Area office (Pacific time) or the NYC office (Eastern time).
Ads Data Platform team's mission is to build a performant, reliable and efficient platform and end-user tools for the collection, storage, management, and analysis of Reddit Ads data to support data-intensive applications and data-driven decisions at scale. We are aiming to integrate and simplify the data usage patterns across Ads, remove redundancies / fragmentation, centralize reusable components / tools, drive adoption of Reddit shaped core-infra owned services / design patterns, leading to more efficient Ads data systems and higher developer velocity.
In pursuit of the team mission, we follow a 3 pronged strategy :
- Unification : Identify and unify / simplify different data usage / integration patterns that are trying to do the same thing, but ownership is fragmented. For e.g. enrichment of a dataset, different teams enrich the same dataset, for their product requirements or different teams are trying to set up similar data processing tools.
- Centralization : Based on the simplification of data usage patterns, tease out the reusable tools / components, and build / own them for the entire Ads engineering. Reuse core-infrastructure supported tools / services. For e.g. for batch data processing, spark hosted on kubernetes, instead of each team bootstrapping their own spark instances, Data Platform owns the spark infra horizontally.
- Optimization : Focus on efficiency, both on development life-cycle and resource usage. Drive adoption and migration towards a more optimized data infrastructure, for all of Ads engineering.
Key guiding principles, that we adhere to, as we evolve the data infrastructure for Reddit Ads :
Build on top of Reddit Core Infrastructure, avoiding duplication of effort. Explore / facilitate hand-over of services / components, as the scope of usage expands outside of Ads.Ensure Reddit shaped architecture, following the best practices shared by DevX and Core InfrastructureLook to the open source world : there are a lot of good resources for data infrastructure in the open source community and we try to adopt those systemsPrefer in-house solutions, over vendor provided infrastructure solutions, as far as possible. Deviations should have a good cost or business driven reason.Scalability and Reliability are central to our services. We constantly balance the business needs for speed against architecting a more reliable and scalable infrastructure.Efficiency, all our architecture and design, is done with efficiency and machine costs in mind.Strategic Focus Areas
Data InfrastructurePlatform and tools for the collection, storage, management, and analysis of Reddit Ads dataReliability & EfficiencyReliability : Systems maturity : measurement and trackingReliability : Developer efficiency and reduction of operational burdenEfficiency : Efficient and scalable Ads foundationWhat We Own
Reliability & EfficiencyAds Marketplace level KPI monitoring framework (ReMeDe / MMS)Big Query Ads project management / slot usage for AdsOptimal data storage / db for DS owned Ads DWAWS & GCP usage, observability, costs, allocations, optimizationData InfrastructureSpark on kubernetesKafka Connect (s3 / GCS writers)Ads AirflowAnomaly detection frameworkData Integration tools (SFDC Data Integrator etc)Apache druid database hosted on kubernetesEnable druid integration with Ads services like billing / pacing / reportingDruid ingestion pipelines (real time and batch)Data Access ControlWe are looking for a seasoned Engineering Manager with industry-relevant platform or infrastructure expertise to lead the Ads Infrastructure team with engineers spread across US East & West coast.
Responsibilities
Coach, motivate, build, hire, and lead a world-class team of passionate Infrastructure engineers with a knack for high scale / high performance data pipelines and storage systems.Lead, coordinate, and execute a coherent vision for a ruthlessly prioritized infrastructure roadmap based on business needs.Set and support a culture of data-informed decision making, with efficient processes and strong transparency.Facilitate the collaboration between the different product / vertical teams making use of Ads Infrastructure and the partner and core platform teams across Reddit.Participating in design and coding reviews : You will review work by your team members and provide feedback to ensure that it meets the team's standards for quality, performance and best practices.Collaborating with cross-functional team leads (EMs, PMs, DSs) to understand business requirements and translate them into technical directions for the Ads Infrastructure roadmap in conjunction with the rest of the Ads Foundations teams.Required Qualifications :
8+ years of industry experience as a SWE3+ years managing (including coaching, mentoring, developing) engineering teams2+ years in the capacity of tech lead in charge of systems & architecture designExperience with at least two general programming languages such as Python, Go, Scala, Java, C++Preferred experience in one or more of the following areas : Message Queuing Services (Kafka, Amazon Simple Queue Service), Data Processing Frameworks (Apache Spark, Apache Flink), Key Value Stores (Redis, DynamoDB, Memcached, Riak), Document based DBs (MongoDB, Cassandra), Container Orchestration (Kubernetes, Mesos), ML Ops ((ML | Tensor | Kube)flow)Excellent communication skills, both written and verbal, and the ability to work effectively with product managers, data scientists, and other stakeholders.Preferred Qualifications :
Experience managing data-focused teamsExperience with AdsExperience with Technical Program ManagementIndustry relevant or scientific contribution in the field of DevOps, Infrastructure, High scalability / Big Data systemsBenefits
Comprehensive Healthcare Benefits401k MatchingWorkspace benefits for your home officePersonal & Professional development fundsFamily Planning SupportFlexible Vacation (please use them!) & Reddit Global Wellness Days4+ months paid Parental LeavePaid Volunteer time off