Data Engineer SCOT Fulfillment Optimization

When you order items on Amazon there are practically thousands of ways we can fulfill that order. Which Fulfillment Center do we ship from what carriers do we use what boxes do we combine items in - now scale that question to billions of items shipped annually worldwide. At Amazons Supply Chain Optimization Technologies (SCOT) we are tasked with fulfilling every customer order in the most intelligent way possible while making sure customers get their orders on time. Within SCOT the Fulfillment Optimization organization owns the artificial intelligence optimization and simulation systems that decide how Amazons outbound network is planned and operated. Our team is a new horizontal team responsible for the data foundations that power every Fulfillment Optimization product - from long-range planning to day-of execution. Our customers are the Applied Scientists Business Intelligence Engineers Data Scientists and Software Development Engineers building those products and the leaders who depend on trustworthy data to make multi-billion-dollar planning decisions. We are looking for a Data Engineer to help us build that foundation. You will work alongside more senior Data Engineers Software Development Engineers and Applied Scientists to design build and operate the data pipelines models and warehouses that our products and analytics depend on. You will take well-defined requirements build a solution and deliver it on schedule. You will learn the teams data architecture deeply contribute to its evolution and grow into an autonomous owner of significant pieces of it. This role is a great fit for someone who is excited about working with large-scale complex data who cares about getting the details right and who wants to grow under the mentorship of an experienced data engineering team. Key job responsibilities Build and optimize physical data models and ETL pipelines for datasets that serve our products and analytics using Amazons data platforms (Redshift EMR Spark S3 Hive etc.) - Take well-defined requirements from Data Scientists Business Intelligence Engineers and Software Development Engineers and deliver tested documented and maintainable data solutions on schedule - Troubleshoot root-cause and resolve issues in existing datasets and pipelines leaving them better and easier to maintain than you found them - Measure and improve dataset quality - completeness freshness correctness - and contribute to the teams monitoring and SLA framework - Write secure stable testable and well-documented code in SQL and Python (or Scala) and submit it for code review - Classify store and handle data in accordance with Amazons security and privacy policies - Participate in team design scoping and prioritization discussions learn the business context behind the teams data architecture and the products it supports - Collaborate with peers across Fulfillment Optimization and partner teams to integrate data sources and unblock downstream consumers A day in the life You will spend your day writing SQL and Python designing tables and pipelines reviewing teammates code and pairing with data scientists and engineers to understand what they need from the data. You will start with well-scoped pieces of work - a new dataset an optimization to an existing pipeline an investigation into a data quality issue - and grow your scope as you build context. You will be supported by a team that values mentorship code review and writing things down. About the team The Fulfillment Optimization organization owns and operates the artificial intelligence optimization and simulation systems that plan and run Amazons outbound fulfillment network. The organization is spread across the United States and Europe and is multi-disciplinary - Research Science Applied Science Business Intelligence Product Management Data Engineering and Software Development. Our team is a newly formed horizontal team responsible for the data foundations that all Fulfillment Optimization products and analytics depend on. Come join us as we build that foundation from the ground up. BASIC QUALIFICATIONS - Experience in data engineering - Experience with data modeling warehousing and building ETL pipelines - Experience with one or more query language (e.g. SQL PL/SQL DDL MDX HiveQL SparkSQL Scala) - Experience with one or more scripting language (e.g. Python KornShell) PREFERRED QUALIFICATIONS - Experience with big data technologies such as Hadoop Hive Spark EMR - Experience with any ETL tool like Informatica ODI SSIS BODI Datastage etc. Amazon is an equal opportunities employer. We believe passionately that employing a diverse workforce is central to our success. We make recruiting decisions based on your experience and skills. We value your passion to discover invent simplify and build. Protecting your privacy and the security of your data is a longstanding top priority for Amazon.