Data Engineer

Snowflake, AWS Data Engineers - Senior & Junior

The following information aims to provide potential candidates with a better understanding of the requirements for this role.

We are seeking several Snowflake Data Engineers - both a Senior Data Engineer and a Junior Data Engineer to join a dynamic data team within the Energy sector. These roles are integral to designing, developing and maintaining scalable data pipelines and architectures that support advanced analytics, machine learning and business intelligence initiatives.

The ideal candidates will possess strong expertise in cloud platforms, big data technologies and data modeling, contributing to innovative data solutions that drive strategic decision making across the organisation.

Data Engineers will ensure that the pipelines are developed and deployed with a secure by design approach, delivering robust, thoroughly tested and maintainable solutions.

• Develop piplines using the standard patterns for data pipelines and workflows utilizing Streamsets, Kestra, dbt, Git.
• Design and Implement data storage and processing solutions employing Snowflake.
• Utilize AWS services for cloud-based platform tooling infrastructure including but not limited to : Lambda,ECS,MSK,RDS,EC2, Secrets Manager, ALB, Cloud Watch, Event Bridge
• Utilize Terraform for AWS and Azure deployments.
• Leverage and integrate APIs for data access and manipulation.
• Write Python scripts for data common processing and automation tasks.
• Leverage Platform API’s and Web Applications to enforce Platform Security
• Development experience with Go, SQL, C#, .net, JavaScript, shell scripts & container platforms like Docker.
• The engineer shall have experience in Authentication mechanisms including but not limited to (OAuth 2.0, OIDC, Microsoft Entra, Key Pair Authentication, Certificate based authentication, SAML based SSO).

Additionally, there is a requirement to create and execute comprehensive test plans to ensure the pipelines functionality and performance as well as develop unit tests, integration tests and end-to-end tests for data pipelines and workflows. Governance requires adherence to access processes and procedures to maintain Data Privacy and Security.

Platform Maintenance is also required to monitor and manage the platform to ensure optimal performance and uptime as well as conduct regular maintenance tasks such as updates, patches and backups. Security responsibilities require best practices throughout the pipeline development and deployment process. Therefore, conduct regular security reviews and vulnerability assessments.

Deliverables

Expectations

Initial 6-month contract with the opportunity to extend. 1099 contract preferred. xklufbp Houston based.

Work Location : Hybrid remote in Houston, TX 77056

Job Type : Contract

Pay : $450.00 - $960.00 per day

Work Location : Hybrid remote in Houston, TX 77056