Data Engineer

Overview

Title : Data Engineer

Institution : Volunteer State Community College

The Data Engineer is responsible for designing, building, and maintaining data pipelines to ensure seamless data flow and integration. This position leverages Python, SQL, and Delta Lake within the Microsoft Fabric ecosystem to acquire, transform, and manage data from source systems like Ellucian Banner as well as external APIs. The Data Engineer ensures data is effectively structured, reliable, and optimized to support college-wide data needs and various strategic initiatives. Working extensively with enterprise systems and emerging data technologies, the Data Engineer creates a robust foundation for the college's data ecosystem and may explore advanced techniques to enhance data reliability and scalability. The role utilizes expertise in data lakehouse management, data pipeline creation and support, and analytical reporting skills. The position will acquire and analyze data from various internal and external sources, designing a cloud-based data platform for analytics and business intelligence, providing consistent and reliable data to Business Intelligence Analysts and the campus community. The Data Engineer will also work with the President's Office supporting Project Management efforts.

Job Duties

• Supervise staff, assign projects, monitor and evaluate performance, and provide training.
• Use Python and SQL to extract and integrate data from Ellucian Banner and external APIs, ensuring seamless data flow into the Microsoft Fabric Lakehouse solution.
• Design, develop, and deploy high-performance Python-based data pipelines for structured, semi-structured, and unstructured data.
• Implement performance tuning techniques to improve data engineering workloads.
• Work with Microsoft Azure and Fabric cloud services to design and implement robust cloud data architectures.
• Ensure the security and compliance of data products through the implementation of appropriate controls and data governance best practices.
• Partner with college faculty and staff (data analysts, stakeholders, and other IT engineers) to identify data needs and deliver tailored data solutions.
• Design and maintain reporting dashboards and visualizations to track key performance indicators for various college initiatives.
• Assist the President of the College with Project Management to ensure presidential initiatives are completed in a timely and efficient manner.
• Create and maintain scripts and reports to facilitate the rapid completion of federally required surveys, such as IPEDS.
• Perform data collection assignments including data retrieval / collection and reporting for faculty and staff requestors.
• Other duties as assigned.

Minimum Qualifications

Associate's degree and three years of related experience OR Bachelor's degree and one year of related experience.

Experience in computer programming with Python and SQL.

Microsoft Fabric Analytics Engineer Associate Certification.

Microsoft Power BI Data Analyst Associate Certification.

Preferred Qualifications

Datacamp Data Engineer Associate Certification.

Datacamp SQL Associate Certification.

CompTIA A+ Certification.

CompTIA Network+ Certification.

CompTIA Project + Certification.

Technical experience with Ellucian Banner.

Experience in higher education.

Knowledge, Skills, and Abilities

Knowledge of various programming languages such as Python and SQL.

Knowledge of data visualization and reporting tools such as Power BI, Tableau, and Looker.

Knowledge of Data Transformation tools such as Polars, Pandas / NumPy, Apache Spark, dbt, etc.

Knowledge of Delta Lake features like time travel, schema enforcement, or ETL optimization.

Knowledge of cloud data ecosystems such as Microsoft Fabric, AWS Data Lake, Databricks, Google Data Lake, and Snowflake.

Knowledge of data orchestration tools such as Apache Airflow, Azure Data Factory / Fabric Data Factory, and Google Cloud Composer.

Strong understanding of REST API concepts.

Familiarity with Java, Scala, and Rust is a plus.

Ability to write shell scripts using languages such as PowerShell or bash to automate data refreshes effectively.

Ability to handle multiple tasks simultaneously and meet deadlines.

Ability to restructure transactional data into meaningful data models, such as star schemas.

Must be detail-oriented, highly organized, and self-motivated.

Pay Rate : $62,385 - $77,981 annual salary depending on experience

Benefits

Special Instructions to Applicants

Unofficial transcripts are acceptable for the application process. Official transcripts will be required upon hire.

Applicants may be subject to a background check.

J-18808-Ljbffr