47 Data Engineer* jobs

The speed of change is throwing traditional business methods into question and disrupting the relevance of entire industries. Capgemini, a global leader in consulting, digital transformation, technology and engineering services, is at the forefront of innovation and well placed to address opportunities for our clients in the evolving world of cloud, digital and platforms. Building on its strong 50-year heritage and deep industry-specific expertise, Capgemini enables organisations to realise their business ambitions through an array of services from strategy to operations. Capgemini is driven by the conviction that the business value of technology comes from and through people. Today, it is a multicultural company with over 270,000 team members in almost 50 countries. With Altran, the Group reported 2019 combined revenues of EUR 17 billion. Learn more about us at  www.capgemini.com . Let’s talk about the team: Our Insights and Data team helps our clients...

Description: Kick Start 2021 by joining a high-profile Federal Government Department! Exciting new initiative working within a cross-functional team Competitive Hourly Rates $$$ One of the high-profile profile Federal Government Departments based in Canberra is seeking to engage an experienced  Data Engineer  design and implement datasets and reporting products to support a large, innovative department. The successful  Data Engineer  will be offered an  initial contract until June 2021 partnered with extension options. The successful Data Engineer will be responsible for, but not limited to; Design, implement and interrogate data models and datasets to support reporting and analytics products Provide guidance on selection, implementation and configuration of products, tools and methods The successful Data Engineer must have the following: Proven experience designing and implementing...

Exciting Perth Startup - Now an international success story. Cutting edge technology stack. About Our Client Our client is an ASX-listed tech startup that is experiencing a period of incredible growth and expansion. With offices around Australia/NZ and North America, this software company has a passion for ensuring the safety of its global customer base through innovative technology solutions. Job Description In this role you will be: Working with the software developers to continue the migration from AWS/Other Data platforms to Google Cloud Platform. Handling ingestion of significant amounts of complex data every day. Work hand in hand with the analytics to order the data and create meaningful analytics. Working closely with stakeholders across the business to support the integration of the platform with the client's core product. Maintaining the core...

The opportunity We are looking for a  Senior Data Engineer / Tech Lead  within  the Technology Consulting – Data and Analytics team  in our  Sydney  office.   EY DNA is the data and advanced analytics capability within EY Asia-Pacific, with over 500 specialist employees working across multiple industry sectors.   We implement information-driven strategies and systems that help grow, optimize and protect client organizations. We go beyond strategy and provide an end-to-end implementation of real-life data environments and have some of the best architects, project managers, business analysts, data scientists, big data engineers, developers and consultants in the region.   Your key responsibilities Design, develop and implement data collection, transformation, and data integration solution to create, coordinate and deploy data and analytics solutions Create and present the...

The Covid-19 pandemic is creating seismic challenges around the world. Our purpose, to build a better working world, has never been more important. Life at EY has been transformed dramatically but our strong culture of flexible and remote working has helped EY people navigate new ways of working and remain connected with each other and our clients. A better working world truly starts with the people at EY who are building it every day. Now more than ever we need talented people from diverse backgrounds to help our clients navigate the complexities of this Transformative Age: people with the passion, curiosity, and drive to make things better. The opportunity We are looking for a Senior Consultant - Data Engineer within the Technology Consulting – Data and Analytics team in our Sydney/Melbourne office. EY DNA is the data and advanced analytics capability within EY Asia-Pacific, with over 500 specialist employees working across multiple industry sectors....

Benefitfocus is looking for a  Sr. Data Engineer  to join the Artificial Intelligence and Insights development teams in designing and building streaming, large scale data solutions. As a Sr. Data Engineer associate, you will be expected to lead the design and implementation of features for a large product, be seen as a mentor within your team, and regularly influence the SDLC. Job Responsibilities: Implements high priority application software and infrastructure covering database design, epic, feature and story development, re-usable code, components and application functionality Works with senior engineers and architects to define the application architecture and create software design for key elements of the application Evaluates performance of key elements of the application functionality and tunes the performance to cover the range of customer use Works with the scrum team to assist developers and monitor their progress against project...

Our client is an organization that is recognized nationally as a market leader in their respective industry and a company you would want on your CV! This is an opportunity where you will be a true influential part in our client's ambitious data goals as they continue to enhance the way interact and service valued customers. Your analytical mind complimented with a natural curiosity and enthusiasm for all things data will be a winning combination. What you will be doing: Our is growing its Data & Analytics capabilities and you will take the technical lead across all things Data Engineering. By introducing new technologies and partnering with the data science team, you’ll increase the cleanliness of data to problem solve customer roadblocks. Your passion for innovative approaches will see you continue to mature their tech stack across Azure Data Factory, Databricks, Azure SQL, Azure VMs, and Azure functions Key Skills and Experience: 5+ years in a commercial...

About the Organisation Davidson Technology have partnered with a multi-national defence, security and aerospace company who are building one of the world's largest IT defence systems. We are collaborating with them to identify new technology talent who are interested in gaining a skillset that will be sought after across the globe. About the Role You will be responsible for recording, maintaining and releasing configuration information and technical data. You will be the key person to maintain advanced user skills of the Product Lifecycle Management (PLM) system, including the recording of up to date metadata to enable the automation of the Configuration Reports. Your Key Responsibilities Include: Working with sub-contractors and suppliers to assess their configuration management capability; Managing the flow-down of configuration management requirements via a defined statement of work; Auditing the implementation of configuration management...

Contract - 6 months with extensions Location - Melbourne (WFH to start with)   As the Senior Data Engineer you will have the following experience: You will be available for an immediate start / 1 week notice Experience of building data and analytics platforms with structured and un-structured data sources Design, construct, install, test and maintain data pipelines across various layers in the data lake and presentation layer Build high-performance algorithms, and prototypes around ETL data movement and ML models Strong hands- on experience in Python and Pyspark Hands-on experience with unit testing automation, regression testing suite and automated test data generation Employ pyspark, sql in notebooks. Experience with a data processing engine like Databricks or EMR or equivalent Collaborate with members of team (eg, data architects, the IT team, data scientists) on the project's goals Recommend different ways to constantly...

About the Role As a Senior Data Engineer, you will be responsible for the implementation of the ETL of their data warehouse. You will look at their current environment and provide a consultative approach to other tools in the market that could be a better fit for the purpose for the organisation. Additionally, you will work across architecture and modelling for their Data Warehouse solution. About You Solid Experience AWS, EC2, S3, SQS, lambda, Redshift, CI/CD frameworks and workflows. Strong ETL experience working with Talend, Informatica etc Experience working with Python Strong data warehouse experience The Benefits To apply, click the link and upload your current resume in Microsoft Word format only (.doc or .docx).

As the Senior Data Engineer / Analytics you will have the following experience: You will be available for an immediate start / 1 week notice Experience of building data and analytics platforms with structured and un-structured data sources Design, construct, install, test and maintain data pipelines across various layers in the data lake and presentation layer Build high-performance algorithms, and prototypes around ETL data movement and ML models Hands-on experience with unit testing automation, regression testing suite and automated test data generation Employ pyspark, sql in notebooks. Experience with a data processing engine like Databricks or EMR or equivalent Collaborate with members of team (eg, data architects, the IT team, data scientists) on the project's goals Recommend different ways to constantly improve data reliability and quality Azure Cloud Other skill : Data modelling experience - Kimball, Inmon, vault models...

About the Organisation: Our client is a prominent research and learning University currently seeking a  Senior Data Engineer  to develop Campus-wide capability (utilising AWS, LoRaWAN, Dell Boomi) to integrate with Internet of Things (IoT) devices to collect and centrally store data for analytics and insights. What your day will look like: Design, build and optimise IoT data pipelines, architectures and data sets including IoT sensor data. Full stack development - AWS tech and modern programming languages – eg: Node JS, Python. Data integration design and build. What you will need for success: Very good programming skills in NodeJS Python development skills is a nice to have. Data integration – real-time data analytics, ETL, data modelling and SQL skills. Atlassian DevOps toolchain – Jira, Bitbucket, Bamboo, et.al Core development experience in AWS...

About the Role As a Senior Data Engineer, you will be responsible for the implementation of the ETL of their Data Warehouse. You will look at their current environment and provide a consultative approach to other tools in the market that could be a better fit for the purpose of the organisation. Additionally, you will work across architecture and modeling for their Data Warehouse solution. About You Solid Experience AWS, EC2, S3, lambda, Athena, EMR etc Strong experience building CI/CD frameworks and workflows. Strong ETL/ELT experience Experience working with Python Strong data warehouse experience Unit Testing experience The Benefits To apply, click the link and upload your current resume in Microsoft Word format only (.doc or .docx).

About the Role As a Senior Data Engineer, you will be responsible for the implementation of the ETL of their data warehouse. You will look at their current environment and provide a consultative approach to other tools in the market that could be a better fit for the purpose of the organisation. Additionally, you will work across architecture and modeling for their Data Warehouse solution. About You Solid Experience on AWS, EC2, S3, SQS, lambda, Redshift, CI/CD frameworks, and workflows. Strong ETL/ELT experience Good experience using Python Experience working with Data Warehouses Unit Testing experience Experience working in a modern engineering team/role (software engineering building data pipeline)

Location - Melbourne (WFH to start with) As the Senior Data Engineer you will have the following experience: Experience in AWS Cloud Platform Data Warehouse platforms such as AWS Redshift S3, Lambda, SNS, SQS, AWS CLI and Terraform (IaC) Strong hands- on experience in Python/ Pyspark Strong ETL / BI tools in Oracle/ AWS Cloud Design, build, manage and support Data Platform on AWS Experience of building data and analytics platforms with structured and un-structured data sources Design, construct, install, test and maintain data pipelines across various layers in the data lake and presentation layer Experience with a data processing engine like Databricks or EMR or equivalent Work closely with business stakeholders and team members (e.g., data architects, the IT team, data scientists, Developers) Recommend different ways to constantly improve data reliability and quality AWS Cloud certification preferred Agile Environment...

As the Senior Data Engineer / Analytics you will have the following experience: You will be available for an immediate start / 1-week notice Experience of building data and analytics platforms with structured and unstructured data sources Design, construct, install, test, and maintain data pipelines across various layers in the data lake and presentation layer Build high-performance algorithms, and prototypes around ETL data movement and ML models Hands-on experience with unit testing automation, regression testing suite and automated test data generation Employ pyspark, sql in notebooks. Experience with a data processing engine like Databricks or EMR or equivalent Collaborate with members of the team (eg, data architects, the IT team, data scientists) on the project's goals Recommend different ways to constantly improve data reliability and quality Azure Cloud Other skill : Data modelling experience - Kimball, Inmon, vault models...

About the Role As a Senior Data Engineer, you will be responsible for the implementation of the ETL of their data warehouse. You will look at their current environment and provide a consultative approach to other tools in the market that could be a better fit for the purpose for the organisation. Additionally, you will work across architecture and modelling for their Data Warehouse solution. About You Solid Experience AWS, EC2, S3, SQS, lambda, Redshift, CI/CD frameworks and workflows. Strong ETL experience working with Talend, Informatica etc Experience working with Python Strong data warehouse experience The Benefits To apply, click the link and upload your current resume in Microsoft Word format only (.doc or .docx).

About the role This position enables EIM of group IT to achieve the following objectives: Design and implement applications on AWS platform. Migration and Replication of legacy database to Data Lake on AWS. Create optimal cloud architecture aligning with security and governance. Work with engineering and architecture teams to design and implement scalable software services. Identify, design, and implement internal process improvements related to cloud infrastructure. Build the infrastructure required to manage cloud workloads working with wide variety of data sources using SQL and other AWS and its partner technologies. Build optimal processes on AWS allowing to manage security and governance across end-users and application integrations. About You Degree qualified in IT, Computer Systems, software or computer engineering fields or equivalent. 4+ years of experience in building applications on AWS....

If you are a MS focused Data & BI Architect with solid MS Azure Databricks expertise and seeking the next career peaking opportunity, then this just might be the key to your future – please read on. A NZ owned and based specialist in both Cloud and BI/Data Services are further building their headcount due to the large demand of cloud based migrations and integrations. This presents numerous opportunities to include Senior Consultants, Lead Consultants and the immediate need for a Data Architect. Who should apply for this role? MS Business Intelligence Consultants with formal MS BI Architecture. You will require existing expertise in the Azure Data Stack to include; Azure Data Factory and Azure Data Lake, What is on offer? – Above Market Salary – Health Insurance (you and the family) – Phone Plan – Ongoing training and Accreditation (where appropriate) – Lots more ABOUT THE ROLE Within this role, you will be...

Are you are a MS focused Data & BI Architect with very strong Azure Databricks  – please read on. A NZ owned and based specialist in both Cloud and BI/Data Services are further building their headcount due to the large demand of cloud based migrations and integrations.  This presents numerous opportunities to include Senior Consultants, Lead Consultants and the immediate need for a Data Architect. Who should apply for this role? MS Business Intelligence Consultants with formal MS BI Architecture. You will require existing expertise in the Azure Data Stack to include; Azure Data Factory and Azure Data Lake,  What is on offer? – Above Market Salary – Health Insurance (you and the family) – Phone Plan – Ongoing training and Accreditation (where appropriate) – Lots more ABOUT THE ROLE Within this role, you will be responsible for; designing and implementing MS BI Architecture from a MS Azure...