Senior Data Engineer, AI Insights

Overview

Affinity stitches together billions of data points from massive datasets to create a powerful, accurate representation of the world's professional relationship graph. Based on this data, we offer our users the insights and visibility they need to nurture and tap into their team's network of opportunities.

This role is part of the AI Insights team , which owns the services that power Affinity's industry-leading relationship intelligence platform. Our team extracts and retrieves information from billions of structured and unstructured data points to deliver insights to our customers.

As a Senior Data Engineer , you will collaborate with machine learning engineers, software engineers, and product managers to shape the future of private capital's leading CRM platform. This involves designing and building scalable, efficient data extraction, load, and transform (ELT) solutions, monitoring and managing data quality, and ensuring data security and best practices.

What you'll be doing

• Design scalable and reliable data pipelines to consume, integrate and analyze large volumes of complex data from different sources, supporting the evolving needs of our business.
• Help define our data roadmap. You'll collaborate with our team of machine learning engineers, software engineers, product, and business leaders to use data to shape product development.
• Build and maintain frameworks for measuring and monitoring data quality and integrity.
• Establish and optimize CI / CD processes, test frameworks, and infrastructure-as-code tooling.
• Build and implement robust data solutions using Spark, Python, Databricks, Kafka, and the AWS ecosystem (including S3, Redshift, EMR, Athena, Glue).
• Identify skill and process gaps within the team, and develop processes to drive team effectiveness and success.
• Articulate the trade-offs of different approaches to building ETL pipelines and storage solutions, providing clear recommendations aligned with product and business requirements.
• To confirm you have read this entire description, please include the word '#AI-Insights' in your answer to the first application question.

Qualifications

Required :

Nice to have :

Tech Stack : Our Data stack includes tools to build data pipelines between AWS RDS and DBX via scheduled batch jobs and streaming syncing. Spark SQL and MLlib for large-scale data processing in DBX. We also build data pipelines between RDS and other search-optimized engines, such as OpenSearch. In-house data quality tools and governance tools to ensure data quality, security and compliance.

How we work

Our culture is a key part of how we operate, as well as our hiring process :

If you'd want to learn more about our values, please consider exploring our principles and culture on the company site.

What you'll enjoy at Affinity :

Please note that the role compensation details below reflect the base salary only and do not include any equity or benefits. This represents the salary range that Affinity believes, in good faith, at the time of this posting, that it will pay for the posted job.

A reasonable estimate of the current range is $106,200 to $200,000 USD . Within the range, individual pay depends on various factors including geographical location and review of experience, knowledge, skills, abilities of the applicant.

J-18808-Ljbffr