Data EngineerForhyre • San Francisco, CA, US
Data Engineer
Forhyre • San Francisco, CA, US
30+ days ago
Job type
- Full-time
Job descriptionDocument component design for developers and for broader communication. Understand and adopt an Agile (SCRUM like) software development mindset Follow established processes / standards, business technology architecture for development, release management and deployment process Execute and provide support during testing cycles and post-production deployment, engage in peer code reviews. Elicit, analyze, interpret business and data requirements to develop complete business solutions, includes data models (entity relationship diagrams, dimensional data models), ETL and business rules, data life cycle management, governance, lineage, metadata and reporting elements. Apply automation and innovation on new and on-going data platforms for those development projects aligned to business or organizational strategies. Design, develop and implement reporting platforms (e.g. modeling, ETL, BI framework) and complex ETL frameworks that meet business requirements. Deliver business or enterprise data deliverables (that adhere to enterprise frameworks) for various platforms / servers / applications / systems. Proven working experience as a data engineer Bachelor degree or equivalent in Computer Science Skilled in Python object-oriented programming Skilled in AWS Compute such as EC2, Lambda, Beanstalk, or ECS Skilled in AWS Database products such as Neptune, RDS, Redshift, or Aurora Skilled in AWS Management and Governance suite of products such as CloudTrail, CloudWatch, or Systems Manager Skilled in Amazon Web Services (AWS) offerings, development, and networking platforms Skilled in SQL Skilled in Jenkins Skilled in JSON Skilled in discovering patterns in large data sets with the use of relevant software such as Oracle Data Mining or Informatica Skilled in cloud technologies and cloud computing Experience using software and computer systems' architectural principles to integrate enterprise computer applications such as xMatters, AWS Application Integration, or WebSphere Determining causes of operating errors and taking corrective action Experience in the process of analyzing data to identify trends or relationships to inform conclusions about the data Skilled in creating and managing databases with the use of relevant software such as MySQL, Hadoop, or MongoDB Programming including coding, debugging, and using relevant programming languages Communication including communicating in writing or verbally, copywriting, planning and distributing communication, etc. Ability to frame ideas as systems and analyzing the inputs, outputs, and process Experience helping an organization to plan and manage change in effort to meet strategic objectives Adept at managing project plans, resources, and people to ensure successful project completion Working with people with different functional expertise respectfully and cooperatively to work toward a common goal
Job Description
Job Description
We are looking for a passionate certified Data Engineer. The successful candidate will turn data into information, information into insight and insight into business decisions.
Data analyst responsibilities include conducting full lifecycle analysis to include requirements, activities and design. Data analysts will develop analysis and reporting capabilities. They will also monitor performance and quality control plans to identify improvements.
Primary skillset : Python / Pyspark & Azure, ADF, SQL, SQL Server, Data Warehousing, ETL
Secondary : Databricks
Nice to have : Informatica / ETL.
Responsibilities
- Design and develop ETL processes based on functional and non-functional requirements in python / pyspark within Azure platform
- Understand the full end to end development activities from design to go live for ETL development and Azure platform
Recommend and execute improvements
Requirements
Create a job alert for this search
Data Engineer • San Francisco, CA, US
Similar jobs
Thinking Machines Lab's mission is to empower humanity through advancing collaborative general intelligence.We're building a future where everyone has access to the knowledge and tools to make AI w...Show more
A growing healthcare technology firm in San Francisco seeks a Senior Data Engineer to build and maintain scalable data ingestion infrastructure.
The ideal candidate has 4-6 years of experience in da...Show more
The Applied team works across research, engineering, product, and design to bring OpenAI’s technology to consumers and businesses.
We seek to learn from deployment and distribute the benefits of AI,...Show more
Get AI-powered advice on this job and more exclusive features.Gridware is a San Francisco-based technology company dedicated to protecting and enhancing the electrical grid.We pioneered a groundbre...Show more
Join the Sleep Fitness Movement.At Eight Sleep, we’re on a mission to fuel human potential through optimal sleep.As the world’s first sleep fitness company, we’re redefining what it means to be wel...Show more
OpenAI's Human Data Team creates custom data solutions driving groundbreaking research.Our work enhances and evaluates our flagship models and products like ChatGPT, GPT-4o, and Sora, and contribut...Show more
C3 Energy is looking for data engineers to develop and implement the next generation of analytics for the smart grid.We are building a platform able to handle the extremely large amount of data gen...Show more
Visa is a world leader in payments technology, facilitating transactions between consumers, merchants, financial institutions and government entities across more than 200 countries and territories,...Show more
A talented software engineer excited to work on the backbone of the Galileo platform.We are looking for someone who has built large-scale real-time infrastructure, services, and APIs that scale to ...Show more)
Software Engineer, Data Engine at Parametric (YC F25).Parametric is building robots to reliably automate physical labor in the real world.
We’ve spent the last few months aggressively building our t...Show more
Join us in building the future of finance.Our mission is to democratize finance for all.An estimated $124 trillion of assets will be inherited by younger generations in the next two decades.The lar...Show more
Senior Data Engineer page is loaded## Senior Data Engineerlocations : San Francisco, CA, USA : AMER - United States - Washington - Offsite / Home : AMER - United States - California - Offsite / Home...Show more
In this exciting role, you will serve as a Data Performance Engineer in NERSC's Application Performance Group, architecting HPC and AI data services that advance fundamental science.You'll optimize...Show more
Interested in joining a growing company where you will work with talented colleagues, enhance a supportive and energetic culture, and be part of the climate solution? At Energy Solutions, we focus ...Show more
A tech company is seeking an AI Systems & Data Engineer to design and operate Databricks pipelines for unstructured data.
The ideal candidate will bring 5-7 years of experience building ML systems, ...Show more
Hivemapper is a decentralized global map data network built by 10s of thousands of mapping devices.High-res sensors like RGB, Stereo Depth, GNSS, IMU, etc.
Data is automatically uploaded in near rea...Show more
Companies want to train their own large models on their own data.The current industry standard is to train on a random sample of your data, which is inefficient at best and actively harmful to mode...Show more
JLL empowers you to shape a brighter way.Our people at JLL and JLL Technologies are shaping the future of real estate for a better world by combining world‑class services, advisory and technology f...Show more
Software Engineer, Data Infrastructure
Thinking Machines Lab • San Francisco, California, United States
Full-time
Last updated: 30+ days ago • Promoted
Senior Data Engineer : Ingest & ELT to Snowflake (Remote)
Voiceflow • San Francisco, CA, United States
Remote
Full-time
Last updated: 2 days ago • Promoted
Data Engineer, Analytics
OpenAI • San Francisco, CA, United States
Full-time
Last updated: 30+ days ago • Promoted
Senior Data Engineer, Data Lake & Governance
Gridware • San Francisco, CA, United States
Full-time
Last updated: 30+ days ago • Promoted
Software Engineer - Data
Eight Sleep • San Francisco, California, United States
Full-time
Last updated: 30+ days ago • Promoted
Software Engineer, Human Data
Openai • San Francisco, California, United States
Full-time
Last updated: 30+ days ago • Promoted
Software Engineer
Confidential • Redwood City, California, United States
Full-time
Last updated: 30+ days ago • Promoted
Staff Data Engineer
Visa • Foster City, CA, United States
Full-time
Last updated: 30+ days ago • Promoted
Software Engineer, Platform & Data Infrastructure
Galileo • Burlingame, California, United States
Full-time
Last updated: 30+ days ago • Promoted
Software Engineer, Data Engine
Parametric (YC F25) • San Francisco, CA, United States
Full-time
Last updated: 21 days ago • Promoted
Software Engineer, Data Governance
Robinhood • Menlo Park, California, United States
Full-time
Last updated: 12 days ago • Promoted
Principal Data Engineer
Autodesk, Inc. • San Francisco, CA, United States
Full-time
Last updated: 30+ days ago • Promoted
HPC / AI Data Performance Engineer
Lawrence Berkeley National Laboratory • Berkeley, CA, United States
Full-time +1
Last updated: 30+ days ago • Promoted
Senior Data Engineer (Hybrid : Oakland, CA - US)
Energy Solutions • Oakland, CA, United States
Full-time
Last updated: 11 days ago • Promoted
AI Data Engineer — Databricks Pipelines & Lakehouse
HyperFi • San Francisco, CA, United States
Full-time
Last updated: 4 days ago • Promoted
Staff Software Engineer - Data Platform
Hivemapper • San Francisco, California, United States
Full-time
Last updated: 30+ days ago • Promoted
Software Engineer, Data Infrastructure
Datologyai • Redwood City, California, United States
Full-time
Last updated: 30+ days ago • Promoted
Data Engineer 2
JLL • San Francisco, CA, United States
Full-time
Last updated: 30+ days ago • Promoted