7190-A - Data Catalog DeveloperDirect Recruit Agency • New York, NY, US
7190-A - Data Catalog Developer
Direct Recruit Agency • New York, NY, US
30+ days ago
Job type
- Full-time
Job descriptionWork on migrating applications from an on-premises location to the cloud service providers. Develop products and services on the latest technologies through contributions in development, enhancements, testing, and implementation. Develop, modify, and extend code for building cloud infrastructure, and automate using CI / CD pipeline. Partners with business and peers in the pursuit of solutions that achieve business goals through an agile software development methodology. Perform problem analysis, data analysis, reporting, and communication. Work with peers across the system to define and implement best practices and standards. Assess applications and help determine the appropriate application infrastructure patterns. Use the best practices and knowledge of internal or external drivers to improve products or services. Bachelor's degree in Computer Science, Information Systems, or a related field Minimum of 3 years of experience as a Data Catalog Developer or in a similar role Hands-on experience in building ETL using Databricks SaaS infrastructure. Experience in developing data pipeline solutions to ingest and exploit new and existing data sources. Expertise in leveraging SQL, programming languages like Python, and ETL tools like Databricks Perform code reviews to ensure requirements, optimal execution patterns, and adherence to established standards. Computer Science or Equivalent Expertise in AWS Compute (EC2, EMR), AWS Storage (S3, EBS), AWS Databases (RDS, DynamoDB), AWS Data Integration (Glue). Advanced understanding of Container Orchestration services, including Docker and Kubernetes, and a variety of AWS tools and services. Good understanding of AWS Identity and Access Management, AWS Networking, and AWS Monitoring tools. Proficiency in CI / CD and deployment automation using GITLAB pipeline. Proficiency in Cloud infrastructure provisioning tools, e.g., Terraform. Proficiency in one or more programming languages, e.g., Python, Scala. Experience in Starburst, Trino, and building SQL queries in a federated architecture. Good knowledge of Lake house architecture. Design, develop, and optimize scalable ETL / ELT pipelines using Databricks and Apache Spark (PySpark and Scala). Build data ingestion workflows from various sources (structured, semi-structured, and unstructured). Develop reusable components and frameworks for efficient data processing. Implement best practices for data quality, validation, and governance. Collaborate with data architects, analysts, and business stakeholders to understand data requirements. Tune Spark jobs for performance and scalability in a cloud-based environment. Maintain robust data lake or Lakehouse architecture. Ensure high availability, security, and integrity of data pipelines and platforms. Support troubleshooting, debugging, and performance optimization in production workloads.
Job Description
Job Description
, on-site
J ob Title : Data Catalog Developer
Company : Direct Recruit Agency
Contract Details : Full-time, on-site
We are seeking a highly skilled and experienced Data Catalog Developer to join our team at Direct Recruit Agency. As a Data Catalog Developer, you will be responsible for designing, developing, and maintaining data catalogs for our clients. This is a full-time, on-site position that offers a competitive salary and benefits package.
Pay rate 100-105 / hr w2
- Expertise in Collibra is a must.
- Will be building Collibra Data Catalog-
- Experience in the new Collibra software - Edge
- Top must-have hard skills :
- Expertise in Collibra Data Management, Data Asset, Data Governance, and BAU support of Collibra Data Catalog.
- Edge Server experience is a must. Collibra Ranger certification preferred.
- Top nice-to-have hard skills : Databricks, AWS ( S3, Glue, Aurora Postgres, Athena), SQL
- Top soft skills : Communication, Problem-Solving, Collaboration, Attention to Detail .
- Team size : 10 Key aspects of the role :
- Development of Data Catalog, build Collibra workflows, and Integrate Edge server with various data sources, authentication, and access controls.
- Day-to-day Expectations : Data Catalog build out, Metadata Synchronization, Lineage Harvester
- Interview Plan / Process : Two virtual Interviews and one final On-site interview
- Citizenship : USC only
Your role as a Senior Data Engineer
Qualifications :
What we are looking for :
If you are a highly motivated and skilled Data Catalog Developer looking for a challenging and rewarding opportunity, we encourage you to apply for this position. Join our dynamic team at Direct Recruit Agency and be a part of our mission to provide top-notch data solutions to our clients.
Create a job alert for this search
Developer • New York, NY, US
Related jobs
Company : Direct Recruit Agency.Contract Details : Full-time, on-site.We are seeking a highly skilled and experienced Data Catalog Developer to join our team at Direct Recruit Agency.As a Data Catalo...Show more
As a people-centric organization, we foster an environment of collaboration, high performance, and innovation where your talents are valued and your achievements are celebrated.Join us and become p...Show more
A company is looking for a Senior SQL Developer responsible for designing, developing, optimizing, and maintaining data architecture for Medicaid programs.
Key Responsibilities Design, develop, an...Show more A company is looking for a MERN / MEAN Stack Developer.Key Responsibilities : Develop websites using HTML, CSS, Node.Angular2+ Create, deploy, and maintain automated system tests while collaboratin...Show more A company is looking for a Pharmacovigilance Product Database Associate.Key Responsibilities Lead and mentor a global team of product dictionary administrators, ensuring alignment with safety and...Show more A company is looking for a Database Designer responsible for creating and implementing database schemas and data models for a SaaS platform.
Key Responsibilities Analyze and resolve design flaws i...Show more A company is looking for an API Sr.Key Responsibilities Design, develop, test, and deploy APIs in cloud-native environments Build fault-tolerant, scalable microservices using Resilience4j and in...Show more A company is looking for a Senior DataBricks Developer in the healthcare sector.Key Responsibilities Build and maintain robust ETL pipelines and optimize data workflows in Databricks on AWS Coll...Show more A company is looking for a Caboodle ETL Developer I, II Epic.Key Responsibilities Develop custom import packages to transfer data sets into the Caboodle database Collaborate with project leaders...Show more A company is looking for a Global Tech Lead to drive the evolution of their Data Verification Platform.Key Responsibilities Lead the technical evolution of the Data Verification Platform, includi...Show more A company is looking for a Data Quality Analyst.Key Responsibilities Support the establishment of Authoritative Data Sources (ADS) for key data domains Perform data analysis and data cleansing i...Show more A company is looking for a Project Data Administrator I who will support Pharma Services projects with data entry and quality control.
Key Responsibilities Prepare data transfer files and perform ...Show more A company is looking for an Oncology Data Specialist, Per Diem (Remote).Key Responsibilities Manage cancer-related data collection systems for reportable cases Ensure compliance with State of Ca...Show more 
This position will be located at East Hanover, NJ site and will not have the ability to be located remotely.Please note that this role would not provide relocation and only local candidates will be...Show more A company is looking for a Data Engineer to support infrastructure work in California.Key Responsibilities Build and modify high-complexity data pipelines and alerting systems Lead end-to-end pr...Show more A company is looking for a NSA Claims Data Coordinator.Key Responsibilities Analyze submissions to create cases and prepare necessary documentation for processing Perform data entry and outreach...Show more A company is looking for a Data Architect to optimize and manage its data infrastructure.Key Responsibilities Analyze and optimize query execution plans and guide engineering teams on schema desi...Show more A company is looking for a Techno-Functional Lead - Data Quality & Master Data Management (Investment Management).Key Responsibilities Lead data quality and master data management initiatives wit...Show more
7190-A - Data Catalog Developer
Direct Recruit Agency • NYC, New York, United States
Full-time
Quick Apply
Last updated: 30+ days ago
QC Data Reviewer
Atrium Staffing • Monroe, NJ, US
Full-time
Last updated: 15 days ago • Promoted
Senior SQL Developer
VirtualVocations • Paterson, New Jersey, United States
Full-time
Last updated: 30+ days ago • Promoted
California Licensed MERN Stack Developer
VirtualVocations • Paterson, New Jersey, United States
Full-time
Last updated: 1 day ago • Promoted
Pharmacovigilance Database Associate
VirtualVocations • Jamaica, New York, United States
Full-time
Last updated: 3 days ago • Promoted
Database Designer
VirtualVocations • Paterson, New Jersey, United States
Full-time
Last updated: 8 days ago • Promoted
API Senior Developer
VirtualVocations • Paterson, New Jersey, United States
Full-time
Last updated: 2 days ago • Promoted
Senior DataBricks Developer
VirtualVocations • Newark, New Jersey, United States
Full-time
Last updated: 3 days ago • Promoted
Caboodle ETL Developer
VirtualVocations • Elizabeth, New Jersey, United States
Full-time
Last updated: 4 days ago • Promoted
Global Tech Lead - Data Verification
VirtualVocations • Paterson, New Jersey, United States
Full-time
Last updated: 4 hours ago • Promoted • New!
Senior Data Quality Analyst
VirtualVocations • Jamaica, New York, United States
Full-time
Last updated: 30+ days ago • Promoted
Project Data Administrator
VirtualVocations • Paterson, New Jersey, United States
Full-time
Last updated: 3 days ago • Promoted
California Licensed Oncology Data Specialist
VirtualVocations • Jamaica, New York, United States
Full-time
Last updated: 6 hours ago • Promoted • New!
Therapeutic Data Strategy Director
Novartis Group Companies • East Hanover, NJ, United States
Full-time
Last updated: 30+ days ago • Promoted
California Licensed Data Engineer
VirtualVocations • Newark, New Jersey, United States
Full-time
Last updated: 30+ days ago • Promoted
Claims Data Coordinator
VirtualVocations • Paterson, New Jersey, United States
Full-time
Last updated: 30+ days ago • Promoted
Data Architect
VirtualVocations • Paterson, New Jersey, United States
Full-time
Last updated: 30+ days ago • Promoted
Data Quality Management Lead
VirtualVocations • Paterson, New Jersey, United States
Full-time
Last updated: 15 days ago • Promoted