Data Engineer II

Job Description

The selected candidate

• Leads and assists the development of large data engineering assignments to address complex business requirements, including design and deployment, ensuring alignment to broader organization objectives
• Working with partners including Infrastructure, Applications, and Analytics teams to assist with data-related technical issues and support their data needs.
• Collaborate with other teams to design and develop data tools that support operational and analytical uses cases.
• Develop new systems and tools to enable stakeholders to consume and understand data more intuitively
• Build, manage, and support data models and its data movements

Your key responsibilities include

• Leading in the creation, maintenance and monitoring of on-premise or cloud infrastructure such as the design of end-to-end data platforms, development and operations of cloud management and design of distributed systems, handling structured and unstructured data
• Staying up to date with changes in software and technology most relevant to EY, identifying packages and solutions that could meet solution requirements and developing request for proposal from vendors
• Create models for new data warehouse development and/or changes to existing data stores leveraging data modeling best practices. Diagnose and resolve data warehouse access and performance issues.
• Building the infrastructure required for optimal ETL of data from a wide variety of data sources using relational, NoSQL, and cloud technologies.
• Identify data discrepancies and data quality issues, and work with data stewards to ensure data consistency and integrity through proper data lineage tracking and governance projects.
• Monitor system details within the data warehouse, including stored procedures and execution time, and implement efficiency improvements.
• Develop, implement, and maintain change control and testing processes for modifications to data systems.
• Document the data architecture and environment in order to maintain a current and accurate view of the larger data picture.
• Develop and deploy end-user practices and tools for data extraction, queries, and data manipulation in accordance with business processes.

Skills and attributes for success

• Experience with ETL and orchestration tools: Informatica, Abinitio, DataStage, SSIS, Prefect, Apache Airflow, Kafka etc.
• Experience working with cloud provides such as AWS and Azure.
• Hands-on experience with data architecting, data mining, data modeling and business requirements gathering/analysis.
• Experience with database platforms, including SQL Server, Snowflake.
• Working technical experience with designing, building, configuring and supporting data warehouses.
• Strong understanding of relational database structures, theories, principles, and practices.
• Ability to conduct research into database issues, standards and products as required.

• Advanced understanding of some of the following tools and technologies, and how to apply them to solve business problems:
• Cloud Computing: SAP Cloud Platform, Amazon Web Services, Microsoft Azure, GCP Cloud Architecture, IBM Infosphere Information Server, IBM Infosphere Warehouse Edition, Mulesoft CloudHub, Oracle Cloud, Red Hat Cloud
• Data Access Tools: Azure Data Explorer, Azure Stream Analytics, MS SQL Server Management Studio, SQL Developer, TOAD
• ETL Tools: Informatica, SSIS, DataStage, Abinitio, Azure Data Factory
• Data Modelling Tools: Erwin, Excel, Power Designer, Sparx, Visio
• Distributed Systems: Azure ADLS Gen 2, Databricks, Hadoop, HDFS, Kafka, MapReduce/Hive, Spark, Storm, Zookeeper
• Enterprise Architecture Tools: Spark Enterprise Architect
• Graph Databases: AWS DynamoDB, Azure Cosmos, DataStax Enterprise Graph, IBM Graph, InfiniteGraph, Neo4j, Teradata Aster, Titan
• Hadoop Distributions: Azure Databricks, Cloudera, Hortonworks, MAPR
• In Memory Databases: SAP HANA
• NoSQL Columnar Stores: Accumulo, AWS RedShift, Cassandra, Druid, HBase, Vertica
• NoSQL Document Stores: Apache CouchDB, ArangoDB, Clusterpoint, Couchbase, DocumentDB, Elastics Search, IBM Domino, MarkLogic, MongoDB, OrientDB, Qizx, RethinkDB, Sharepoint
• NoSQL Key Value Pair Data Stores: Aerospike, DovetailDB, Dynamo, Dynomite, Flare, Hyperdex, Keyspace, MotionDb, Oracle NoSQL Database, Quasardb, RAMCloud, Riak, SchemaFree, SubRecord, Voldemort
• Relational MMP Databases: Netezza, Teredata
• Relational SMP Databases: Azure SQL PaaS, IBM DB2, MySQL, Oracle, PostgreSQL,
• SQL Server

Education

• M.S. in Computer Science, Data Analytics, Data Science, Econometrics, Engineering, IT, Cyber Security, or related field preferred

What we look for

• Strong analytical skills and problem-solving ability
• A self-starter, independent-thinker, curious and creative person with ambition and passion
• Excellent inter-personal, communication, collaboration, and presentation skills
• Customer focused
• Excellent time management skills
• Positive and constructive minded
• Takes responsibility for continuous self-learning
• Takes the lead and makes decisions in critical times and tough circumstances
• Attention to detail
• High levels of integrity and honesty

,

Keyskills :
data modelingpythondocumentationweb servicesedwdata sciencesqldata analyticsms sql serverdata modelscyber securitydata qualitydebuggingdata miningsql server management studio