Bing Data in Microsoft STCA is responsible for developing and operating the largest data eco-system: PiB data has been being processed and consumed every day by streaming and batches. The team is responsible for the whole life cycle including data ingestion, data processing, data API, analytics and algorithms (e.g., bot detection). The data API is used by all EXP, reporting and model training in Copilot and Search verticals in Bing. Open source tech like Kafka, Spark, Flink, Click House, Trino, Deltalake, etc. are widely used at an extreme scale to run the services successfully. Thousands of developers in Bing and partner team from Ads, MSN, and Edge are working on top of the data products that the team build. We’re looking for passionate data engineers to join us for the mission of powering Microsoft businesses through data substrate and infusing our data capabilities to the industry.

Responsibilities

• Lead the design and implementation of the full stack of Big Data from ingestion, processing to analytics for all Bing products including Copilot and Search on top of 400 PiB data.

• Understand customer requirements and solve complex problems in Big Data systems at scale.

• Continuously upgrade the data system architecture based on State-of-the-art technology.

• Continuously Optimize the compute and storage based on technology innovations.

• Embrace both Microsoft technology and cutting-edge open-source technology (Kafka/Spark/Flink/Click House/Trino/Deltalake, etc, etc.)

• Use LLM/Copilot to upgrade the traditional data API.

• Use LLM/Copilot to automate/accelerate the development and analysis of Big Data.

• Coache the team members by high quality design and implementation examples.

Qualifications

• 5+ years working experience and solid experience of shipping high performance C#, Java, or equivalent language code software.

• Data processing or analytics experience with Kafka, Spark, Flink, Click House, Trino, Deltalake is a plus.

• Experience of leading the project at large scale.

• Domain knowledge of ads, search or content services is a plus.

• Experience of designing mission critical data pipelines with SLA commitment is a plus.

• Quick learning and solid problem solving and debugging skills.

• Accountable and proactive

• Good communication skill, fluent in English (both oral and written)

#WWE

Microsoft is an equal opportunity employer. Consistent with applicable law, all qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations (https://careers.microsoft.com/v2/global/en/accessibility.html) .