Data Engineer

Job Description

The ideal candidate will have experience working with large datasets, distributed computing technologies and service-oriented architecture. The candidate should relish working with large volumes of data, and enjoys the challenge of highly complex technical contexts. He/she should be an expert with data modeling, ETL design and business intelligence tools and has hand-on knowledge on columnar databases. He/she is a self-starter, comfortable with ambiguity, able to think big and enjoys working in a fast-paced team.Responsibilities:
• Design, build and own all the components of a high-volume data warehouse end to end.
• Build efficient data models using industry best practices and metadata for ad-hoc and pre-built reporting
• Provide wing-to-wing data engineering support for project lifecycle execution (design, execution and risk assessment)
• Interface with business customers, gathering requirements and delivering complete data & reporting solutions owning the design, development, and maintenance of ongoing metrics, reports, dashboards, etc. to drive key business decisions
• Continually improve ongoing reporting and analysis processes, automating or simplifying self-service support for customers
• Interface with other technology teams to extract, transform, and load (ETL) data from a wide variety of data sources
• Own the functional and nonfunctional scaling of software systems in your ownership area.
• Implement big data solutions for distributed computing.
• Willing to learn and develop strong skill set in AWS technologies
Key job responsibilitiesAs a DE on our team, you will be responsible for leading the data modelling, database design, and launch of some of the core data pipelines. You will have significant influence on our overall strategy by helping define the data model, drive the database design, and spearhead the best practices to delivery high quality products.A day in the lifeYou are expected to do data modelling, database design, build data pipelines as per Amazon standards, design reviews, and supporting data privacy and security initiatives. You will attend regular stand-up meetings and provide your updates. You will keep an eye out for opportunities to improve the product or user experience and suggest those enhancements. You will participate in requirement grooming meetings to ensure the use cases we deliver are complete and functional. You will take your turn at on-call and own production operational maintenance. You will respond to customer issues and monitor databases for healthy state and performance.About the teamOur mission is to build applications which can solve challenges Global Payroll Operations teams face on daily basis, automate the tasks they perform manually, provide them seamless experience by integrating with other dependent systems, and eventually reduce Pay Defects and improve pay accuracy.,
• 3+ years of experience as a Data Engineer or in a similar role
• Experience with data modeling, data warehousing, and building ETL pipelines
• Experience in SQL
• Bachelors in Computer Science, Engineering, Statistics, Mathematics or related field
• 5+ years of experience in data engineering / business intelligence space
• Strong understanding of ETL concepts and experience building them with large-scale, complex datasets using traditional or map reduce batch mechanism.
• Strong data modeling skills with solid knowledge of various industry standards such as dimensional modeling, star schemas etc.
• Extremely proficient in writing performant SQL working with large data volumes
• Experience designing and operating very large Data Warehouses
• Experience with scripting for automation (e.g., UNIX Shell scripting, Python, Perl, Ruby).
• Excellent problem-solving skills and ability to prioritize and stay focused on big needle movers
• Curious, self-motivated & a self-starter with a can do attitude . Comfortable working in fast paced dynamic environment.
• Excellent knowledge of Advanced SQL working with large data sets.
• Excellent dimensional modeling skills.
• Good with one of the programming languages such as python, java, ruby, etc
• Good to have experience with AWS technologies including Redshift, RDS, S3, EMR, or similar solutions built around Hive/Spark etc.
• Good to have experience with QuickSight.

Keyskills :
javaunix shell scriptingdata privacysqldata modelingdata warehousinginformatica