Automation QA Engineer / Big Data / ETL Tester

Role : Automation QA Engineer / Big Data / ETL TesterLocation : Scottsdale AZ (Day 1 Onsite)Hire type : C2C . FTE , CTH Must have : Automation , java , AWS , ETL , Scala , SparkRequired : 6-8 years of IT experience focusing on Testing and Automation experience using Java and Groovy.Rest assured app concepts in automation. Experience with Databricks & on Prem, Structured Streaming, Delta Lake concepts, and Delta Live Tables requiredExperience with Spark Scala and Java programmingData Lake concepts such as time travel and schema evolution and optimizationExperience leading and architecting enterprise-wide initiatives specifically in automation testing for ETL testing and several phases of testing, data migration, transformation.Understanding of streaming data pipelines and how they differ from batch systems understanding of ETL and ELT and ETL / ELT tools such as Data Migration Service etcFamiliarity and / or expertise with Great Expectations or other data quality / data validation frameworks a bonusAdvanced level SQL experience (Joins, Aggregation, Windowing functions, Common Table Expressions, RDBMS schema design performance optimization)Indexing and partitioning strategy experienceDebug, troubleshoot, and implement solutions to complex technical issuesExperience with large-scale, high-performance enterprise big data application deployment and testingArchitecture experience in AWS environment a bonusFamiliarity working with Lambda specifically with how to push and pull data, how to use AWS tools to view data for processing massive data at scale a bonusExperience with Gitlab and CloudWatch and ability to write and maintain Gitlab for supporting CI / CD pipelinesExperience working with AWS Lambdas for configuration and optimization and experience with S3Familiarity with Schema Registry, and message formats such as Avro, ORC, etc.Ability to thrive in a team-based environmentExperience briefing the benefits and constraints of technology solutions to technology partners, stakeholders, team members, and senior-level of management