What you'll do :

• Provide a consultative approach with business users, asking questions to understand the business need and deriving the data flow, conceptual, logical, and physical data models based on those needs.
• Perform data analysis to validate data models and to confirm the ability to meet business needs.
• Assist with and support setting the data architecture direction, ensuring data architecture deliverables are developed, ensuring compliance to standards and guidelines, implementing the data architecture, and supporting technical developers at a project or business unit level.
• Coordinate and consult with the Data Architect, project manager, client business staff, client technical staff and project developers in data architecture best practices and anything else that is data related at the project or business unit levels.
• Work closely with Business Analysts and Solution Architects to design the data model satisfying the business needs and adhering to Enterprise Architecture.
• Coordinate with Data Architects, Program Managers and participate in recurring meetings.
• Help and mentor team members to understand the data model and subject areas.
• Ensure that the team adheres to best practices and guidelines.

What makes you a great fit :

• Strong working knowledge of at least 3 years of Spark, Java/Scala/Pyspark, Kafka, Git, Unix / Linux, and ETL pipeline designing.
• Experience with Spark optimization/tuning/resource allocations
• Excellent understanding of IN memory distributed computing frameworks like Spark and its parameter tuning, writing optimized workflow sequences.
• Experience of relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., Redshift, Bigquery, Cassandra, etc).
• Familiarity with Docker, Kubernetes, Azure Data Lake/Blob storage, AWS S3, Google Cloud storage, etc.
• Have a deep understanding of the various stacks and components of the Big Data ecosystem.
• Hands-on experience with Python is a huge plus