Cloud Data Engineer
100% Remote
Contract : Goes until 6 / 30 anticipated to renew each fiscal year
Hours : 37.5hrs a week (7.5hrs daily)
In alignment with state-level technology modernization initiatives, the Education Technology Office has undertaken a multi-year application and platform modernization project. The IT team is seeking a Cloud Database Administrator (DBA) / ETL Engineer to assist in maintaining, optimizing, modernizing, and troubleshooting the organization’s data warehouse, data mart, and overall data portfolio. Under the direction of senior leadership and data engineering team leads, the DBA / ETL / ELT Engineer will manage databases and data services hosted on cloud platforms, ensuring they are secure, performant, highly available, and compliant with governance policies.
The Engineer will be hands-on working with a team of cloud Engineers, ETL developers, technical leads, DBAs, project managers and analysts to design and implement the Education portfolio data, data pipelines, and transformations in a more stream-lined, scalable and cost-effective set of solutions.
Mandatory requirements
- Experience working with Oracle RDS
- Experience working with one or more AWS Cloud services such as S3 storage, Managed Airflow (MWAA), Data Migration Service (DMS) in support of building data pipelines
- Experience working with a variety of backend data sources (e.g. SQL Server, Oracle, Postgres, DynamoDB,
Snowflake)
Advanced SQL coding skills and ability to translate Oracle PL SQL and Stored Procedure code to alternative SQLplatforms such as SnowflakeFamiliarity with data warehouse and data mart concepts such as normalization, facts, dimensions, slowlychanging dimensions
Familiarity with Change Data Capture (CDC) concepts and implementations. Knowledge of Kafka or similarreplication tools is a plus
Understanding of common file formats such as JSON, XML, CSVBasic experience using scripting tools to automate tasks (e.g. : Python, Windows PowerShell, bash)Ability to write unit test scripts and be able to validate migrated ELT / ETL codePreferred
Experience configuring, managing, and troubleshooting Airflow tool. Knowledge of Airflow DAGs and conceptsfor managing dependent graphs and complex steps
Knowledge of Snowflake data warehouse features such as Snowpipe streaming, cloning, time travel, role-basedaccess controlPrior experience working at other large organizations, preferably state or federal governmentBusiness domain knowledge in the Education and Student data management area.Experience working with software development tools such as GitHub, JirRegular responsibilities include :
Create and manage cloud-native databases and services (e.g., RDS Oracle, Aurora, Postgres, Snowflake).Track and tune query execution, compute scaling, and storage performance.Define policies for snapshots, PITR (point-in-time recovery), and cross-region replication.Implement encryption, access policies, masking, and auditing to meet FERPA / PII standards.Manage schema migrations, data pipelines, and versioned deployments.Hands-on discovery, solution designing, re-platforming and troubleshooting to migrate the EOE Legacy SSISETL code to a SQL based solution with Apache Airflow for scheduling and dependency management.
Tasks may include re-Engineering overall solution approach, constructing code packages, bugfixing, unit testing code and using GitHub code repository
Develop and guide implementation of Apache Airflow Scheduling and Dependency FrameworkPerformance tuning and optimizing solution implementation, benchmark the new solution against the on-premsolution to ensure that it performs comparably or better
Using Jira to review and work through assigned tasksUsing GitHub to check in and manage code, code reviews, pull requestsThis is a remote position.