Jul 01, 2020

Big Data Engineer

  • Peoplebank Australia
  • Sydney NSW, Australia
Contractor Big Data Engineer

Job Description

  • Large IT Consultancy
  • Sydney CBD
  • Extensive experience in Hadoop technologies (Spark)

Our client, a leading consultancy, is seeking an experienced Big Data Engineer to join their Data team. This is a cloud-based environment on cutting edge projects.

The candidate:

  • Several years Big Data (3+) and 7 + years workforce experience minimum
  • Solid experience on design patterns to implement strategic, tactical, and operational products spanning across all disciplines of BI and Analytics including data ingestion, integration, modelling, performance acceleration, visualisation etc.
  • Solid functional understanding of the Open-Source Big Data Technologies including major Hadoop projects, Kafka, NiFi, or NoSQL Databases.
  • Experience working on Spark Streaming
  • Preferable work experience with Hortonworks distribution and Azure cloud platform.
  • Proficient with HDFS, Hive, HBase, NiFi, Kafka, Spark, and SQL.
  • Experience in the Azure cloud platform is desirable.
  • Expertise in various connectors and pipelines for batch and real-time data collection/delivery
  • Database design and modelling - logical and physical
  • Performance tuning - table partitioning and indexing, process threading
  • Support multiple Agile Scrum teams with planning, scoping, and creation of technical solutions for the new product capabilities, through to continuous delivery to production.
  • Detail-oriented with strong analytical and problem-solving skills
  • Knowledge of API protocols and development methodologies.

Role

Big Data Engineer