Senior Data Engineer

Responsibilities

• Integrate data from multiple sources, such as databases, APIs, or streaming platforms, to provide a unified view of the data
• Implement data quality checks and validation processes to ensure the accuracy, completeness, and consistency of data
• Identify and resolve data quality issues, monitor data pipelines for errors, and implement data governance and data quality frameworks
• Enforce data security and compliance with relevant regulations and industry-specific standards
• Implement data access controls, encryption mechanisms, and monitor data privacy and security risks
• Optimise data processing and query performance by tuning database configurations, implementing indexing strategies, and leveraging distributed computing frameworks
• Optimize data structures for efficient querying and develop data dictionaries and metadata repositories
• Identify and resolve performance bottlenecks in data pipelines and systems
• Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders
• Document data pipelines, data schemas, and system configurations, making it easier for others to understand and work with the data infrastructure
• Monitor data pipelines, databases, and data infrastructure for errors, performance issues, and system failures
• Set up monitoring tools, alerts, and logging mechanisms to proactively identify and resolve issues to ensure the availability and reliability of data
• It would be a plus if the candidate has a software engineering background

Requirements

Preferred Skills & Characteristics

Consistently display dynamic independent work habits, goal-oriented, passionate in growth mindsets and self-motivated professional. Self-driven and proactive in keeping up with new technologies and programming

J-18808-Ljbffr