Our team plays a crucial role in the data ecosystem of the TikTok Recommendation System, focusing on creating offline and real-time data storage solutions for large-scale recommendation, search, and advertising businesses, serving over 1 billion users. The core goals of the team are to ensure high system reliability, uninterrupted service, and smooth data processing. We are committed to building a storage and computing infrastructure that can adapt to various data sources and meet diverse storage requirements, ultimately providing efficient, cost-effective, and user-friendly data storage and management tools for the business.
Responsibilities
1. Architecture Design and Implementation : Design and implement offline and real-time data architectures for large-scale recommendation, search, and advertising systems based on Paimon and Flink. Ensure efficient data processing and storage to meet the strict requirements of the business for data timeliness and accuracy. 2. System Construction and Optimization : Design and implement flexible, scalable, stable, and high-performance storage systems and computing models. Use Paimon as the storage foundation and combine it with the powerful computing capabilities of Flink. Continuously optimize system performance to cope with the challenges brought by business growth. 3. Troubleshooting and Stability Assurance : Be responsible for troubleshooting production systems. For problems that occur in the Paimon-Flink architecture during operation, design and implement necessary mechanisms and tools, such as data consistency assurance and exception recovery, to ensure the overall stability of the production system. 4. Distributed System Construction : Build industry-leading distributed systems, including offline and online storage based on Paimon and batch and stream processing frameworks based on Flink, providing solid and reliable infrastructure support for massive data and large-scale business systems.
Minimum Qualifications :
1. Paimon - Flink Technology Stack : Have a thorough understanding of Paimon and Flink, and be able to understand and use them at the source-code level. Experience in customizing or extending these two systems is preferred.
2. Data Lake Technology : Have an in-depth understanding of at least one data lake technology (such as Paimon), with practical implementation and customization experience, which should be highlighted in the resume.
3. Storage Knowledge : Be familiar with the principles of HDFS, and knowledge of columnar storage formats such as Parquet and ORC is preferred.
4. Programming Languages : Be proficient in programming languages such as Java, C++, and Scala, with strong coding and problem-solving abilities.
Senior/Staff Big Data Storage and Computing Engineer, Recommendation Data Ecosystem • Seattle
We are currently delivering exceptional value to our customers in Revenue Cycle Management (RCM) and Clinical Operations, empowering them with industry-transforming AI technology, intelligent autom...Show more
SQL (Expert), Python (Expert), AWS (Advanced), Data Modeling (Intermediate), ETL Pipelines (Expert).We are seeking a Senior Data Engineer to join our AWS Infrastructure and Supply Chain Finance tea...Show more
Looking for people to participate in taking online surveys for Fortune 500 brands.All you need to do is complete online surveys by sharing your opinion.You will help influence brand decisions on se...Show more
A pioneering data technology startup in Seattle is seeking a Senior Software Engineer to design and deliver full stack product features.The role involves working closely with customers to understan...Show more
A leading technology firm in Seattle is seeking a Senior Data Scientist to join their iCloud Data Organization.This role involves collaborating with product, engineering, and design teams to shape ...Show more
Primary Skills: Python(Proficient), System Design (Proficient), Data Analytics(Proficient), AWS (Expert), Data Structure & Algorithms (Proficient).Duration: 9 Months with Possible Extension.We are ...Show more
Visa is a world leader in payments technology, facilitating transactions between consumers, merchants, financial institutions and government entities across more than 200 countries and territories,...Show more
A technology solutions company in Seattle is seeking a Data Architect.In this role, you will develop and optimize ETL pipelines, work with AWS technologies, and collaborate with stakeholders to mee...Show more
Primary Skills: SQL (Expert), ETL (Advanced), AWS (Advanced), Data Modeling (Advanced), Python (Intermediate).Duration: 12 Months with possible extension.Location: Seattle, WA (Hybrid).A leading FA...Show more
Software Engineer - Streaming Data, Otter - Seattle.In the past, to be a successful restaurateur, you simply had to have a passion for food and a passion for people - but to succeed as a digital re...Show more
A leading social media platform in Seattle is looking for a Senior/Staff Big Data Storage and Computing Engineer to design and optimize data architectures for its recommendation system.The role req...Show more
A leading cloud technology provider in Seattle is seeking a skilled software engineer specializing in high-performance file systems.The successful candidate will engage in architectural design, per...Show more
Strong experience in reporting and generating actionable insights using tools like Adobe Customer Journey Analytics (CJA) and Adobe Analytics.Solid understanding of data modeling, SQL, and ETL proc...Show more
A leading technology firm in Seattle is seeking a Software Development Engineer II for its Applied AI Solutions team.The role involves owning production systems for data storage and analytics, enha...Show more
A healthcare operations technology company in Seattle is seeking a Full-Stack Senior Software Engineer.In this role, you'll design and develop complex features, conduct code reviews, and mentor jun...Show more
A leading AI advertising solutions provider in Seattle is looking for a Senior Software Engineer to design and optimize data systems across their platform.Responsibilities include building large-sc...Show more
An established industry player is seeking a skilled Data Analyst with over 10 years of experience.This role offers the chance to dive deep into analytics and data engineering, utilizing powerful to...Show more
A technology-driven company in Seattle is seeking an experienced Lead Engineer to design and develop data platforms that support advanced AI systems.The ideal candidate will have over 8 years of en...Show more
