<h3>About us</h3>Constructor is the next-generation platform for search and discovery in e-commerce, built to explicitly optimize for metrics like revenue, conversion rate, and profit. Our search engine is entirely invented in-house utilizing transformers and generative LLMs, and we use its core and personalization capabilities to power everything from search itself to recommendations to shopping agents. Engineering is by far our largest department, and we’ve built our proprietary engine to be the best on the market, having never lost an A/B test to a competitive technology. We’re passionate about maintaining this and work on the bleeding edge of AI to do so.Out of necessity, our engine is built for extreme scale and powers over 1 billion queries every day across 150 languages and roughly 100 countries. It is used by some of the biggest e-commerce companies in the world like Sephora, Under Armour, and Petco.We’re a passionate team who love solving problems and want to make our customers’ and coworkers’ lives better. We value empathy, openness, curiosity, continuous improvement, and are excited by metrics that matter. We believe that empowering everyone in a company to do what they do best can lead to great things.Constructor is a U.S. based company that has been in the market since 2019. It was founded by Eli Finkelshteyn and Dan McCormick who still lead the company today.<h3>About the Data Lake Team</h3>The Data Lake Team along with the Data Infrastructure team develops the Constructor Data Platform. This platform is a foundational component for all internal data and ML teams. It handles the ingestion of over 1 TB of compressed events daily and manages over 6 PB of data in our data lake. We build the tooling that every Constructor data scientist and ML engineer uses daily - from ingestion APIs (FastAPI) and real-time Analytics (ClickHouse, Cube) to Spark/Databricks pipelines, Delta Lake storage, and observability baked in (VictoriaMetrics, OpenTelemetry).Were hiring a Senior Data Engineer to work on the Data Lake Team. This team is responsible for:<ul> <li>Data platform support (PySpark, Databricks, EMR, Luigi, Airflow)</li> <li>Development, optimization and maintenance for data pipelines framework to run 10 000+ of pipelines on a daily basis</li> <li>Data modeling (bronze, silver, gold)</li> <li>Development and maintenance of a Data Quality framework built on top of DBT</li> <li>Development and maintenance of user facing service for the behavioral data ingestion (FastAPI, Docker, AWS ECS)</li> </ul>We are seeking an engineer with at least 4 years of experience who possesses strong programming skills (ideally in Python), and expertise in big data engineering, web services, and cloud platforms (ideally AWS). We are looking for someone eager to build diverse components and drive the evolution of our platform.You will contribute to building various data platform components, actively incorporate user feedback, and proactively drive improvements. Here are some of the projects you may be involved with:<ul> <li>Own and evolve our pipeline framework that orchestrates 10 000+ jobs daily</li> <li>Shift workloads from batch to streaming, shrinking model-to-production latency from days to hours</li> <li>Design and develop the Data Quality framework and wire it into every Constructor core service</li> <li>Enable Spark on Kubernetes, giving teams elastic, cost-efficient compute</li> <li>Develop tooling for delivering backfills throughout the data platform</li> </ul>