Databricks Developer

Databricks Developer

Location : Austin, TX (Remote)

Duration : 6+ Months

Need Local To TX Only

Job Description :

Need 10+ Years Experience as a Databricks Developer.

8 Years Required Implement ETL / ELT workflows for both structured and unstructured data

8 Years Required Automate deployments using CI / CD tools

8 Years Required Collaborate with cross-functional teams including data scientists, analysts, and stakeholders

8 Years Required Design and maintain data models, schemas, and database structures to support analytical and operational use cases

8 Years Required Evaluate and implement appropriate data storage solutions, including data lakes (Azure Data Lake Storage) and data warehouses

8 Years Required Implement data validation and quality checks to ensure accuracy and consistency

8 Years Required Contribute to data governance initiatives, including metadata management, data lineage, and data cataloging

8 Years Required Implement data security measures, including encryption, access controls, and auditing; ensure compliance with regulations and best practices

8 Years Required Proficiency in Python and R programming languages

8 Years Required Strong SQL querying and data manipulation skills

8 Years Required Experience with Azure cloud platform

8 Years Required Experience with DevOps, CI / CD pipelines, and version control systems

8 Years Required Working in agile, multicultural environments

8 Years Required Strong troubleshooting and debugging capabilities

5 Years Required Design and develop scalable data pipelines using Apache Spark on Databricks

5 Years Required Optimize Spark jobs for performance and cost-efficiency

5 Years Required Integrate Databricks solutions with cloud services (Azure Data Factory)

5 Years Required Ensure data quality, governance, and security using Unity Catalog or Delta Lake

5 Years Required Deep understanding of Apache Spark architecture, RDDs, DataFrames, and Spark SQL

5 Years Required Hands-on experience with Databricks notebooks, clusters, jobs, and Delta Lake

1 Years Preferred Knowledge of ML libraries (MLflow, Scikit-learn, TensorFlow)

1 Years Preferred Databricks Certified Associate Developer for Apache Spark

1 Years Preferred Azure Data Engineer Associate

Please share profile's to sudheer@anveta.com