Data Engineer

Omni InclusiveProvidence, RI, United States

3 days ago

Job type

Full-time

Job description

Experience working within the Azure ecosystem, including Azure AI Search, Azure Storage Blob, Azure Postgres and understanding how to leverage them for data processing, storage, and analytics tasks.

Ability to preprocess and clean large datasets efficiently using Azure Tools / Python and other data manipulation tools. Experience with techniques such as data normalization, feature engineering, and data augmentation is preferred.

Expertise in working with healthcare data standards (ex. HIP nd FHIR), sensitive data and data masking techniques to mask personally identifiable information (PII) and protected health information (PHI) is essential.

In-depth knowledge of search algorithms, indexing techniques, and retrieval models for effective information retrieval tasks. Familiarity with search platforms like Elasticsearch or Azure AI Search is a must.

Familiarity with chunking techniques and working with vectors and vector databases like Pinecone.

Ability to design, develop, and maintain scalable data pipelines for ingesting, processing, and transforming large volumes of structured and unstructured data.

Experience with implementing best practices for data storage, retrieval, and access control to ensure data integrity, security, and compliance with regulatory requirements.

Be able to implement efficient data processing workflows to support the training and evaluation of solutions using large language models, ensuring reliability, scalability, and performance.

Ability to proactively identify and address issues related to data quality, pipeline failures, or resource contention, ensuring minimal disruption to systems.

Experience with large language model frameworks, such as Langchain and know how to integrate them into data pipelines for natural language processing tasks.

Experience working within the snowflake ecosystem.

Knowledge of cloud computing principles and experience in deploying, scaling, and monitoring AI solutions on cloud platforms like Snowflake, Azure, AWS.

Ability to communicate complex technical concepts effectively to technical and non-technical stakeholders and collaborate with cross-functional teams.

Analytical mindset with a keen attention to detail, coupled with the ability to solve complex problems efficiently.

Knowledge of cloud cost management principles and best practices to optimize cloud resource usage and minimize costs.

Must Have :

Minimum of 10 years' experience as a data engineer
Hands-on experience with Azure Cloud eco-system.
Hands-on experience using Python for data manipulation.
Deep understanding of vectors and vector databases.
Hands-on experience scaling POC to production.
Hands-on experience using tools such as Document Intelligence (formerly Azure Form Recognizer), Snowflake, function app. Azure AI Search
Experience working with PII / PHI
Hands-on experience working with unstructured data.

Create a job alert for this search

Data Engineer • Providence, RI, United States

Related jobs

Promoted

Senior Data Engineer

SHI GmbHProvidence, RI, United States

Full-time

Since 1989, SHI International Corp.We've grown every year since, and today we're proud to be a $15 billion global provider of IT solutions and services. Over 17,000 organizations worldwide rely on S...Show moreLast updated: 3 days ago

Promoted

Lead Data Engineer

Omni InclusiveProvidence, RI, United States

Full-time

Mandatory Skills : ETL Developer, Data Engineer with strong DBT ETL tools experience, Azure Cloud experience, Matillion, Data modelling in Snowflake. Snowflake Senior Technical Lead with SQL, Python,...Show moreLast updated: 3 days ago

Promoted
New!

Data Engineer (Remote)

GovCIOProvidence, RI, United States

Remote

Full-time

GovCIO is seeking a Data Engineer to support operational solutions in our multi-cloud lakehouse environments on Azure and AWS. The ideal candidate will thrive in a fast-paced operations and maintena...Show moreLast updated: 2 hours ago

Promoted

Data Engineer

Tekfortune IncProvidence, RI, United States

Permanent

Tekfortune is a fast-growing consulting firm specialized in permanent, contract & project-based staffing services for world's leading organizations in a broad range of industries.In this quickly ch...Show moreLast updated: 30+ days ago

Promoted

Data Engineer I

Brown University HealthProvidence, RI, United States

Full-time

SUMMARY : Reports to Manager, Data Architecture.The Data Engineer I, assists in developing, maintaining, and optimizing data pipelines and architectures that support the healthcare system's data in...Show moreLast updated: 2 days ago

Promoted

Data & Analytics Engineer

MojoTechProvidence, RI, US

Full-time

MojoTech, established in 2008, is an innovative digital consulting company that integrates strategy, design, and engineering to create results-driven digital products and experiences for clients na...Show moreLast updated: 30+ days ago

Promoted

Data Engineer, Analytics

METAProvidence, RI, United States

Full-time

Meta), formerly known as Facebook Inc.When Facebook launched in 2004, it changed the way people connect.Apps and services like Messenger, Instagram, and WhatsApp further empowered billions around t...Show moreLast updated: 3 days ago

Promoted

Databricks Data Engineer - Manager - Consulting - Location Open

EYProvidence, RI, United States

Full-time

At EY, we're all in to shape your future with confidence.We'll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go.Join EY and help ...Show moreLast updated: 3 days ago

Promoted

Data MLOps Engineer

Omni InclusiveProvidence, RI, United States

Full-time

Experience working within the Azure ecosystem, including Azure AI Search, Azure Storage Blob, Azure Postgres and understanding how to leverage them for data processing, storage, and analytics tasks...Show moreLast updated: 30+ days ago

Promoted

Sr Data Engineer (MFT - IBM Sterling)

The Hertz CorporationProvidence, RI, United States

Full-time

The IBM Sterling Senior MFT Engineer will have expertise in Sterling B2B Integrator, Sterling File Gateway, Sterling Secure Proxy, Control Center and Connect Direct. This position requires an expert...Show moreLast updated: 3 days ago

Promoted

Data Engineer - Snowflake

Omni InclusiveProvidence, RI, United States

Full-time

Specializes in ETL (Extract, Transform, Load) using ETL tools Matillion, DataStage etc.Experience in Snowflake, Netezza.Experience with Data warehousing techniques(Star Schema, Snowflaking, Normali...Show moreLast updated: 30+ days ago

Promoted

Data Engineer

RIT Solutions, Inc.Smithfield, RI, United States

Full-time

Data Engineer w Cobol SMT candidates ONLY.The position is in Smithfield, Rhode Island.Bachelor's Degree or equivalent in a technology related field (e. Computer Science, Engineering, etc.Data engine...Show moreLast updated: 3 days ago

Promoted

Data Engineer

DexianSmithfield, RI, United States

Full-time

Location : 2 weeks onsite , 2 weeks remote ( Smithfield , RI ).Long term role with possible extensions.We are seeking a Data Engineer to design, develop, and sustain both operational and analytical ...Show moreLast updated: 30+ days ago

Promoted

Data Engineer

MindteckSmithfield, RI, United States

Full-time

We are currently sourcing for a Data Engineer to work in Smithfield, RI!.The FI Data Solutions team is looking for a highly motivated Data Engineer responsible for designing, building, and maintain...Show moreLast updated: 3 days ago

Promoted

Data Engineer, Product Analytics

METAProvidence, RI, United States

Full-time

As a Data Engineer at Meta, you will shape the future of people-facing and business-facing products we build across our entire family of applications (Facebook, Instagram, Messenger, WhatsApp, Real...Show moreLast updated: 3 days ago

Promoted

Sr Data Engineer MDM-Customer 360

CitizensJohnston, RI, United States

Full-time

The Enterprise Data Engineering (EDE) team is looking for a Senior Data Engineer with a winning track record in building high performing data and analytical solutions that transform, integrate, and...Show moreLast updated: 3 days ago

Promoted

Data MLOps Engineer

Tekfortune IncProvidence, RI, United States

Full-time

Experience within the Azure ecosystem, including Azure AI Search, Azure Storage Blob, Azure Postgres, with expertise in leveraging these tools for data processing, storage, and analytics tasks.Prof...Show moreLast updated: 30+ days ago

Promoted

Data Engineer (Analytics)

METAProvidence, RI, United States

Full-time