What you'll do:

• Team Leadership: Lead, mentor, and manage a team of data engineers, providing guidance on technical and project-related matters. Foster a collaborative team environment, ensuring effective communication and knowledge sharing.
• Data Engineering & Development:  Design, develop, and optimize data pipelines and workflows using Python, PySpark, and Shell scripting. Leverage key data engineering libraries in Python, such as Pandas, Dask, and
• Polars, to build and maintain scalable data solutions. Apply best practices for writing clean, efficient, and optimized Python code.
• AWS Infrastructure Management: Architect, implement, and maintain infrastructure on AWS, including services such as S3, Lambda, CloudWatch, SQS, Kinesis, EventBridge, EC2, EMR, AWS Glue, Athena, CloudFormation, and Secrets Manager. Manage and monitor APIs running on AWS API Gateway, ensuring high availability and performance. Implement logging, debugging, and monitoring practices to maintain code quality and system reliability. Track and optimize metrics and costs associated with AWS services.
• Project & Deployment Management: Oversee project management on GitLab, including version control, branch management, and code reviews. Handle deployments using serverless technologies and manage CI/CD pipelines, ensuring smooth and efficient releases. Utilize Docker containers to build, test, and deploy applications in a scalable and consistent environment.
• Database Management: Work with SQL and NoSQL databases (MongoDB) to design and manage efficient data storage and retrieval systems. Ensure data integrity, performance optimization, and security across all database systems.
• Regular Expressions (Regex): Utilize strong understanding and proficiency in regex to extract, manipulate, and validate data efficiently within various applications and pipelines.

What makes you a great fit:

• We are looking for a seasoned Tech Lead Data Engineer with a experience range of 5-7 Yrs robust technical background and leadership experience to spearhead our data engineering efforts.
• This role demands a deep expertise in Python, PySpark, and Shell scripting, along with a comprehensive understanding of AWS cloud services.
• As a Tech Lead, you will manage a team of data engineers, oversee the development and maintenance of scalable data solutions, and ensure the seamless integration and deployment of projects.
• You will play a key role in optimizing our AWS infrastructure, implementing CI/CD pipelines, and managing APIs and serverless architectures.
• This position requires not only strong technical skills but also the ability to lead projects, mentor team members, and drive innovation within the data engineering domain.