Data Engineer (AI-Ready Data)
Swift Hire LLC • Raleigh, NC, United States
9 hours ago
Job type
- Quick Apply
Job description
Job Title: Data Engineer (AI-Ready Data)
Location: US: East Coast (RTP preferred)
Job Description
We are looking for a hands-on Data Engineer to design, build, and maintain data pipelines that transform raw application data into clean, structured, AI-ready datasets. The ideal candidate will have strong experience with AWS Glue, PostgreSQL, data modeling, and ETL/ELT pipelines.
The role involves defining relationships between different data entities and preparing curated datasets that enable AI assistants to answer business and analytical questions accurately and consistently.
Key Responsibilities
- Design, build, and maintain data pipelines to transform raw application data into clean, structured, AI-ready datasets.
- Develop and manage AWS Glue jobs for data extraction, transformation, and loading.
- Model and document relationships between entities, including Customer-to-Orders relationships.
- Build curated datasets and views to support AI-driven analytical queries.
- Design and optimize PostgreSQL schemas, queries, and indexes.
- Implement data quality checks, validation rules, and monitoring.
- Add business context to data through metadata, naming conventions, descriptions, and semantic layers.
- Collaborate with AI engineers to understand data requirements and consumption needs.
Required Qualifications
- Must have 6+ years exp in USA.
- Hands-on experience with AWS Glue jobs (Must Have).
- Strong hands-on experience with PostgreSQL (Must Have), including advanced SQL, joins, aggregations, and performance tuning.
- Experience preparing and transforming application data for AI/ML or analytics use cases.
- Strong understanding of data modeling and entity relationships, including identifying correlations and keys across separate entities.
- Experience building and maintaining production ETL/ELT pipelines.
- Strong analytical skills and ability to translate business questions into data structures and queries.
Winning Qualifications
- Experience with Python or PySpark for data transformation.
- Familiarity with AWS services such as S3, Athena, Redshift, Lambda, and Step Functions.
- Experience preparing data for LLM or RAG-based applications, including text-to-SQL, semantic layers, and embeddings pipelines.
- Experience with data quality, observability, and data cataloging tools.
- Understanding of data governance, security, and PII handling.
- Experience working with AI or analytics teams.