Principle Data Engineer

Job Description

*Design, develop, troubleshoot and debug software programs for databases, applications, tools, networks etc.As a member of the software engineering division, you will take an active role in the definition and evolution of standard practices and procedures. You will be responsible for defining and developing software for tasks associated with the developing, designing and debugging of software applications or operating systems.Work is non-routine and very complex, involving the application of advanced technical/business skills in area of specialization. Leading contributor individually and as a team member, providing direction and mentoring to others. BS or MS degree or equivalent experience relevant to functional area. 7 years of software engineering or related experience., *Principle Data Engineer Posting (Oracle-CrowdTwist)About CrowdTwist/Oracle Marketing Cloud:CrowdTwist provides the most comprehensive omni-channel loyalty & analytics SaaS platform for industry leading brands. We have grown from being a small startup to a respected platform with hundreds of millions of users and billions of interactions. After our recent acquisition by Oracle, we are looking to write the next chapter in user growth and scale. Were relaxed, experienced, hard driving and are experiencing explosive growth. We power the loyalty offerings for many of the world s top brands such as Lego, Disney, Weight Watchers, Marvel Comics, AMC s The Walking Dead, Foot Locker and Zumiez. Were relaxed, experienced, hard driving, changing our industry and looking for smart people to help tackle tough technical challenges.About the OpportunityWe are seeking a talented and experienced Data Engineer for our growing team based in Bangalore. Our cloud-first SaaS platform is built using Distributed Microservices Architecture, on Linux, Java, PHP, Oracle & MySQL; we use RabbitMQ as our messaging backbone and NoSQL databases like Redis, DynamoDB, Neo4j, etc. We need our engineers to be versatile, display leadership qualities and be enthusiastic to take on new problems across the full-stack as we continue to push technology forward. At over one million lines of code, you ll be working with large, highly trafficked, scalable applications. Quality and high availability is of extreme importance to us, so were looking for a professional engineer that takes a pragmatic and passionate approach to technology to solve problems and keeps the application uptime high.Responsibilities:

• Design and develop high-performing, scalable, and fault tolerant data pipelines
• Own the data model for our analytics needs, ensuring that data is standardized and accessible
• Work with Product and Engineering to ensure upstream product model changes integrate well with our data model; and when it doesn t build the necessary capabilities to adjust
• Manage individual projects priorities, deadlines and deliverables with your technical expertise
• Integrate and productize analyst and data science models as needed

About You:

• Have a B.E./B.Tech. in Computer Science, Engineering, Information Systems or equivalent experience
• Self-starter with 8+ years of experience as a Data Engineer dealing with large complex data scenarios
• Have serious data modeling and SQL skills
• Experience with Python and Git
• Experience with both compiled and scripting programming languages
• Experience working with RDBMS like Oracle, MySQL, etc.
• Excellent communication (verbal and written) and interpersonal skills and an ability to effectively communicate with both business and technical teams

Bonus Points:

• Experience owning data pipelines in a production environment
• Experience with Python ETL tools such as Luigi, Airflow, PySpark Stitch
• Experience with other ETL tools such as FiveTran, Stitch, CloverDX, dbt
• Experience working on public cloud offerings like AWS, Azure, GCP, OCI
• Experience working with NoSQL databases like Redis, DynamoDB, etc.

Detailed Description and Job RequirementsDesign, develop, troubleshoot and debug data pipelines for analytics products As a member of the software engineering division, you will assist in defining and developing software for tasks associated with the developing, debugging or designing of data pipelines. Specify, design and implement modest changes to existing architecture to meet changing needs. Duties and tasks are varied and complex needing independent judgment. Fully competent in own area of expertise. May have project lead role and or supervise lower level personnel. B.E., B.Tech. or M.Tech. degree or equivalent experience relevant to functional area. 8+ years of software engineering or related experience.

Keyskills :
sqljavadata warehousinginformaticapythonetl toolsdata sciencedata modelingcomputer sciencehigh availabilitycommercial modelsinformation systemsstatements of work sow