Data Engineer I, TFS

Job Description

Job summaryIf you are Data Engineer and looking for an opportunity to solve deep technical problems and build a new innovative solution (grounds-up) in a fast-paced environment with senior and seasoned engineers, this might be the role for you.About INSIGHTS platformTransportation Financial Systems (TFS) is building a brand new INSIGHTS platform. INSIGHTS is an analytical intelligence platform that allows all internal users to perform in depth financial and technical performance analysis across Amazon Transportation organization. The platform suffices the needs of a plethora of users from business performance analysts, operational analysts, carrier management, financial controllership, accounting, accounts payable and the technical community. The platform is customizable to the needs of each user, allowing real time access to right statistical position and performance data and enabling real time decision-making capabilities as well as swift corrective action.What is the one line vision or North Star of this product Our North Star is to provide all TFS users, internal or external, a robust and scalable self-serve analytics solution that connects, streamlines, and automates data intelligence & reporting for a financial transaction across various TFS services with help of an easy to navigate user interface. We envision this product to be a single source of truth and a go-to tool across Amazon transportation organization to assess systematic, financial and operational health metric.What engineering challenges will you solve as part of INSIGHTS Disparate data: TFS provides an end-to-end financial automation solution and has a separate estimates and actuals accounting flow. Each of these flows further has multiple disparate services performing functions such as ingestion, attribute derivation, validation etc. Therefore, currently financial data is spread across multiple, disparate systems within TFS. Each of the services capture data in an isolated manner with little to no interconnection to each other, thereby posing a challenge for user to get an end to end financial data in a consolidated manner.Data complexity: TFS processes billions of transactions every month (~400 manifest/seconds in peak) and the granularity of these transactions keep changing within TFS services. For example, tax validation is done at invoice level, commercial validation at charge type level, and payment approval at container level. Adding further to this granularity complexity is the N:N relationship between actual transactions e.g. an invoice can have N containers and a container can have N invoices tied to it. The extent of volume and lack systematic capability to traverse through the different grains of a transaction forces partner and tech teams to spend significant time writing intricate queries to disambiguate this complexity and compile aggregated financial reports.Multiple distinct internal tools: Growth in Amazon s internal operations over the years in combination with financial data complexity, as described above, has led business and finance partner teams to build and rely on their own customized internal tools to measure operational performance. Availability of multiple distinct tools with varying logic based off same financial data brings misalignment and reporting inconsistency between these internal teams, which further leads to burden of proof on tech teams to investigate such anomalies/inconsistencies and answer respective partner team s questions about gap in data.Whom will you work with You will be working with SDE-3s who will be reviewing your body of work on this program on an ongoing basis. This program will also have a Principal Engineer and Principal Data Engineer oversight. You will also collaborate with senior BIE within TFS to build this product.What else will you get to do If you are a seasoned DE, we will also give you an opportunity to become the owner of a large Tier-2 services data. You will be the gatekeeper for all architectural changes and data changes for that large service. We are a financial platform team and have 100% ownership of all the services and data we work on. As part of ownership, we will give you opportunity to improve service you own, make changes in the way data is propagated, persisted and , drive IMR improvement within that service.Who we are Amazon s transportation systems get millions of packages to customers worldwide faster and cheaper while providing world-class customer experience from checkout to shipment tracking to delivery. Transportation Financial Systems (TFS) is responsible for building technologies that support financial aspects for Amazon transportation. Our software systems include services that handle thousands or requests per second and make decisions to pay billions of dollars a year and ensure that transportations liabilities and revenues are correctly accounted for. With rapid expansion into new geographies, innovations in supply chain, delivery models and customer experience, increasingly complex transportation network, ever-expanding selection of products and growing number of shipments worldwide, we have an opportunity to build software that scales not only with volume but also with the business models. We leverage cutting-edge technologies in big data, machine learning, real time analytics and high volume, low latency, high availability services.TFS is 100+ tech team across 8 different platforms. We deal with problems of scale. Billions of dollars of spend is automated through our platforms and pipelines. We are a very fast growing org and we aggressively ramping up our team across all engineering roles. The tech team is completely based out of IN.,
• 1+ years of experience as a Data Engineer or in a similar role
• Experience with data warehousing, and building ETL pipelines
• Experience in SQL
• Bachelors/Masters in Computer Science or Engineering
• Working knowledge of Big data technologies Spark, EMR, Glue, ETLs etc.
• Knowledge of Perl or other scripting languages a plus
• Experience with distributed (multi-tiered) systems, algorithms, and relational databases
• Experience in optimization mathematics (linear programming, nonlinear optimization)
• Previous technical internship(s) preferred

Keyskills :
informaticaaccounts payablefinancial datapythonjavasqlsupply chainbig data