Big Data Lead

Posted: 1 week ago

  • At least 2 large big data projects
  • At least 1 large big data project on AWS done at scale.
  • Expert in AWS Big data solution leveraging Python , PySpark, Spark,
  • Expert in Snowflake.
  • Good experience in Airflow for job orchestration.
  • Writing scalable and efficient Batch data pipelines using AWS EMR stack.
  • Real-time data pipelines: Solid development experience building solutions leveraging server less architecture ( kinesis, lambda, and other AWS server less components).
  • Experience building and implementing Micro services ( Nice to have ).
  • Prior experience productionizing data science / Client pipelines ( Nice to have ).
  • Hands on Big data engineer with ability to lead tech solutions , direct teams , develop and recommend architecture patterns.
  • In Data and Analytics space for last 7 years.