What you'll do:

• Understand the business requirements and translate these to data services to solve the business and data problems
• Develop and manage the transports/data pipelines (ETL/ELT jobs) and retrieve applicable datasets for specific use cases using cloud data platforms and tools
• Explore new technologies and tools to design complex data modelling scenarios, transformations and provide optimal data engineering solutions
• Build data integration layers to connect with different heterogeneous sources using various approaches
• Understand data and metadata to support consistency of information retrieval, combination, analysis and reporting
• Troubleshoot and monitor data pipelines to have high availability of reporting layer
• Collaborate with many teams - engineering and business, to build scalable and optimized data solutions and propose ways to improve platform and tools
• Effectively manages workstreams for self and 2 to 3 analysts to support delivery

What makes you a great fit:

• Over 4-7 years of experience with building the data pipelines or data ingestion for both Batch/Streaming data from different sources to data warehouse / data lake
• Experience leading and delivering data warehousing and analytics projects, including using cloud technologies such as EMR, Lambda, Cloud Storage, BigQuery, etc.
• Experience of atleast 2 to 3 projects building and optimizing BigData data pipelines, architectures and datasets
• Handson experience/knowledge with SQL/Python/Java/Scala programming, understanding of SQL is a must
• Experience with any cloud computing platforms like AWS (S3, Lambda functions, RedShift, Athena), GCP (GCS, Dataflow, Dataproc, Pub/Sub, Bigquery), Azure (Blob/Azure, Synapse) etc.
• Knowledge of version control software such as Git and experience in working with relevant hosting services (e.g. Azure DevOps, Github, Bitbucket, Gitlab)
• Strong experience working with relevant Python & R packages and ecosystems (e.g. pandas, NumPy, scikit-learn, tidyverse, data tables)
• Extensive experience in developing complex stored procedures and transformations
• Experience/knowledge with Big Data tools (Hadoop, Hive, Spark, Presto)