Talent.com
Data Engineer

Data Engineer

Krea.ai, IncSan Francisco, CA, United States
4 days ago
Job type
  • Full-time
Job description

About Krea

At Krea, we are building next-generation AI creative tools.

We are dedicated to making AI intuitive and controllable for creatives. Our mission is to build tools that empower human creativity, not replace it.

We believe AI is a new medium that allows us to express ourselves through various formats-text, images, video, sound, and even 3D. We're building better, smarter, and more controllable tools to harness this medium.

This job

Data is one of the fundamental pieces of Krea. Huge amounts of data power our AI training pipelines, our analytics and observability, and many of the core systems that make Krea tick.

As a data engineer, you will...

  • ... build distributed systems to process gigantic (petabytes) amounts of files of all kinds (images, video, and even 3D data). You should feel comfortable solving scaling problems as you go.
  • ... work closely with our research team to build ML pipelines and deploy models to make sense of raw data.
  • ... play with massive amounts of compute on huge kubernetes GPU clusters - our main GPU cluster takes up an entire datacenter from our provider.
  • ... learn machine learning engineering (ML experience is a bonus, but you can also learn it on the job) from world-class researchers on a small yet highly effective tight-knit team.

Example projects

  • Find clean scenes in millions of videos, running distributed data pipelines that detect shot boundaries and saving timestamps of clips.
  • Solve orchestration and scaling issues with a large-scale distributed GPU job processing system on kubernetess.
  • Build systems to deploy and combine different LLMs to caption massive amounts of multimedia data in a variety of different ways.
  • Design multi-stage pipelines to turn petabytes of raw data into clean downstream datasets, with metadata, annotations, and filters.
  • Strong candidates may have experience with...

  • Python, PyArrow, DuckDB, SQL, massive relational databases, PyTorch, Pandas, NumPy...
  • Kubernetes
  • Designing and implementing large-scale ETL systems
  • Fundamental knowledge of containerization, operating systems, file-systems, and networking.
  • Distributed systems design
  • About us

  • We're building AI creative tooling.
  • We've raised over $83M from the best investors in Silicon Valley.
  • We're a team of 12 with millions of active users scaling aggressively.
  • Create a job alert for this search

    Data Engineer • San Francisco, CA, United States

    Related jobs
    • Promoted
    Data Engineer

    Data Engineer

    Perfict Global, Inc.Concord, CA, United States
    Full-time
    Perfict Global is a leading IT consulting services provider focused on providing innovative and successful business workforce solutions to Fortune 500 companies. Our trained and experienced professi...Show moreLast updated: 4 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Mercor IncSan Francisco, CA, United States
    Full-time
    Mercor is training models that predict how well someone will perform on a job better than a human can.We use our platform to source, vet, and onboard expert contractors who help train AI models in ...Show moreLast updated: 4 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Crystal EquationFremont, CA, United States
    Full-time
    Senior Data Engineer Job Description.We are looking for a versatile data professional, capable of leading end-to-end projects. The responsibilities include requirements gathering and cross-functiona...Show moreLast updated: 4 days ago
    • Promoted
    Data Engineer

    Data Engineer

    KodiakMountain View, CA, United States
    Full-time
    The company has developed an artificial intelligence (AI) powered technology stack purpose-built for commercial trucking and the public sector. The company delivers freight daily for its customers a...Show moreLast updated: 4 days ago
    • Promoted
    Data Engineer

    Data Engineer

    AdobeSan Jose, CA, United States
    Full-time
    Changing the world through digital experiences is what Adobe's all about.We give everyone-from emerging artists to global brands-everything they need to design and deliver exceptional digital exper...Show moreLast updated: 4 days ago
    • Promoted
    Data Engineer 4

    Data Engineer 4

    NextDeavorSan Jose, CA, United States
    Temporary
    Here's how you'll become a key player with this opportunity : .We are looking for a Splunk expert to join on a short-term contract and help stabilize, optimize, and improve our Splunk environment.Thi...Show moreLast updated: 4 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Sinclair Broadcast GroupSan Francisco, CA, US
    Full-time
    Digital Remedy is looking for a skilled Data Engineer to join our growing team of analytics experts.As a Data Engineer, you will help design, build, maintain, and troubleshoot data processing syste...Show moreLast updated: 18 days ago
    • Promoted
    Data Engineer

    Data Engineer

    TranzealSunnyvale, CA, United States
    Full-time
    Design, build, and maintain robust data pipelines using Apache Spark and Python.Develop and manage workflow orchestration using Apache Airflow. Implement scalable data solutions on Google Cloud Plat...Show moreLast updated: 4 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Ekman AssociatesMenlo Park, CA, United States
    Full-time
    Data Engineer (Data Pipelines & Visualization).Ekman Associates is a management consulting firm that specializes in developing business, digital, and technology strategy, delivering solutions, and ...Show moreLast updated: 4 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Diverse LynxPleasanton, CA, United States
    Full-time
    Job Overview : We are seeking a talented and motivated Data Engineer with expertise in Spark SQL, Databricks, Azure Data Factory (ADF), SQL, IICS, Unix, PySpark, Python, and Azure Data Lake Storage ...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer-III

    Data Engineer-III

    Hire TalentSan Francisco, CA, United States
    Full-time
    Job Title : Data Engineer - III.We are looking to hire a contractor to support data and analytics work for a start-up team at • • • that works on building a point of sale system that we sell to restau...Show moreLast updated: 4 days ago
    • Promoted
    Data Engineer

    Data Engineer

    SynergisticITVallejo, CA, United States
    Full-time
    The Job Market is Challenging due to more than 150,000 Tech Layoffs in 2022 and in 2023 more than 240,000 layoffs so almost 3,90,00 tech employees have been laid off since 2022 and its still going ...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer - AG

    Data Engineer - AG

    BayoneSunnyvale, CA, United States
    Full-time
    Demonstrates up-to-date expertise and applies this to the development,.Supporting and aligning efforts to meet customer, business. Create software design and architecture for next software solution....Show moreLast updated: 30+ days ago
    • Promoted
    Databricks Data Engineer

    Databricks Data Engineer

    Tekfortune IncPleasanton, CA, United States
    Permanent
    Tekfortune is a fast-growing consulting firm specialized in permanent, contract & project-based staffing services for world's leading organizations in a broad range of industries.In this quickly ch...Show moreLast updated: 4 days ago
    • Promoted
    Lead Data Engineer

    Lead Data Engineer

    Midi HealthSan Francisco, CA, United States
    Full-time
    We're looking for a Lead Data Engineer to spearhead design, implementation, and iteration of a world-class, modern data infrastructure that will power all of analytics, data science, and ML / AI syst...Show moreLast updated: 4 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Brevian.aiSunnyvale, CA, United States
    Full-time
    BREV / AN is at the forefront of revolutionizing how businesses leverage artificial intelligence.Our no-code platform empowers every business team to harness the power of production-grade AI agents, ...Show moreLast updated: 4 days ago
    • Promoted
    • New!
    Data Engineer

    Data Engineer

    LandbaseMountain View, CA, United States
    Full-time
    Landbase leverages the experience of 100+ world class sales professionals and AI to deliver targeted, high-quality leads on autopilot. Our mission is to achieve GTM automation so humans no longer ne...Show moreLast updated: 9 hours ago
    • Promoted
    • New!
    Data Engineer

    Data Engineer

    Stellar Consulting Solutions, LLCPleasanton, CA, US
    Full-time
    Job Description : HM prefers candidate to be on site at Pleasanton.Proficiency in Spark, Python, and SQL is essential for this role. Experience with relational databases such as Oracle, NoSQL databas...Show moreLast updated: 1 hour ago