Senior Design Engineer

Job Description

Job descriptionSenior Data Engineer - Informatica/Datastage ETL:Your Roles and Responsibilities:As Senior Data Engineer you are expected to be functionally knowledgeable in multiple data domain areas with a strong emphasis on Data architecture around ETL using Informatica or Datastage. As a Sr Data Engineer in our Data and AI team, you will provide technology guidance in data management and ETL for one or more projects; assist in defining scope and sizing of work; and anchor Proof of Concept development. You will provide data engineering solutions based on the business problem, data integration with third party services, designing and developing complex ETL/ELT pipelines for clients business needs. You will collaborate with some of the best talent in the industry to create and implement innovative high quality solutions, participate in Pre-Sales and various pursuits focused on our clients business needs.You will also contribute in a variety of roles in thought leadership, mentorship, systems analysis, architecture, design, configuration, testing, debugging, and documentation. You will challenge your leading edge solutions, consultative and business skills through the diversity of work in multiple industry domains.Responsibilities

• Be responsible for requirements gathering, System Designing, Data modeling, and ETL design, development, Production Enhancements, Support and Maintenance
• Build complex data processing pipeline using Informatica or Datastage
• Write generic ETL flow templates, job scheduling and management
• Developing and maintaining complex data ETL/ELT Pipelines, data models and standards for various data Integration & data warehousing projects from source to sinks
• Develop scalable, secure and optimized data transformation pipelines and integrate them with downstream sinks
• Develop and maintain documentation of the data flows and data models, integrations, pipelines etc
• Take ownership of technical solutions from data flow design and architecture perspective, ensure the right direction and propose resolution to potential data pipeline-related problems.
• Developing Proof of concepts (PoC) of key technology components to project stakeholders
• Collaborate with other members of the project team (Architects, Data Engineers) to support delivery of additional project components (like API interfaces, Search, visualization)
• Evaluate and create PoVs around the performance aspects of ETL/ELT tools in the market against customer requirements
• Working within an Agile delivery / DevOps methodology to deliver proof of concept and production implementation in iterative sprints.
• Provide technical leadership in shaping strategic, business technology projects, with a strong emphasis on Data architecture around ETL and Governance
• Drive improvements to the Big Data and Cloud Data Warehouse technologies stack, with a focus on the digital experience for the user, as well as application performance & security to meet the needs of the business and customers, now & in the future
• Lead on technical investigations & proof of concepts, both individually and as part of a team, including being hands - on with code, to make technical recommendations
• Create documentation for Architecture Principles, design patterns & examples, technology roadmaps & future planning

Required Skills:ETL Developer with 10+ years of experience with following skills -

• Strong expertise and hands-on experience in SQL
• Strong expertise in Data Modelling techniques
• Extensively using Informatica ETL Power Center Tools to design, develop ETL jobs for extracting, transforming and loading the data (or) IBM Datastage Tools
• Data quality tools
• Rich experience in design & implementation of Enterprise Data Warehouse solutions
• Hands-on expertise in at least 3 projects ETL projects end to end using Informatica or Datastage
• Job monitoring, sequencing and scheduling
• Experience in Big Data and Cloud Data Warehouse technologies - Redshift, BigQuery, Synapse, Snowflake, Hadoop, Hive, Greenplum
• Expertise in BI technologies (in more than 1 areas amongst Cognos, BO, Power BI, Tableau)
• Expertise in Database (in one or more amongst Teradata, Exadata, Netezza, Oracle)
• Meta Data Management and Data Governance Experience
• Hand on experience in Python, Scala, Spark, Kafka

Preferred Skills:

• DevOps - CD/CI Implementations
• Framework Development and Automation Techniques
• Experience in implementation of Data Catalogue and Data Lake Implementations
• Experience in Data Management Solution Development with strong experience in SQL and NoSQL data bases .

Keyskills :
data warehousingoraclehivedjangopythonsqlsparkedata engineer