EL1 Lead Data Engineer
Data Engineers prepare raw data for analytical or operational uses. They make data accessible so that it can be used for optimisation, delivery of services and enhanced analytics solutions. Data Engineer Experts will provide technical leadership in the design, delivery and optimisation of complex data engineering capabilities.
- Brisbane, Queensland, Australia
- Full-time
- On-site
- Posted Jan 1, 2024
INDUSTRYSoftware & InternetSaaS & Enterprise SoftwareData & Analytics
Responsibilities
- Support in the delivery of data assets by contributing to improving data management capabilities, including building data integration applications that extract.
- Transform, and load (ETL) data and metadata into data warehouses + Data Lake (RDBMS/HDFS).
- Undertake analysis, design, development, prototyping, testing and maintenance activities in a Teradata EDW environment.
- Contribute to building and maintaining automated data integration applications by developing complex SQL programs within a Teradata EDW environment.
- Perform analysis of technical requirements to support project evaluation with proposed solutions.
- Ensure excellent data quality and high standards in Management Information solutions.
- Development of ELT solutions using a variety of industry tools including SQL Assistant, Terada Studio Viewpoint, Pcom and Control-M.
- Write/explain technical systems, problems and outcomes using unambiguous language to non-technical staff.
- Strong focus on data integrity between source and destinations systems.
- Analyse and evaluate complex business process and requirement.
- Work with stakeholders to assemble large, complex data sets that meet current and potential business requirements.
- Design and implement data flows connecting operational systems, using data analysis and modelling to build, maintain, and optimise data structures that support controlled data extraction for analytics and business intelligence.
- Manage the build of data streaming systems, and development of management information solutions.
- Manage the design, development and maintenance of enterprise data warehouse, data lake and data engineering solutions.
- Lead the optimisation of data extraction solutions, providing technical guidance on complex query and programming functions to deliver data outcomes aligned to stakeholder and agency requirements.
- Perform analysis on data and processes to address business questions, identify opportunities for improvement and provide support to staff.
- Manage the development of data engineering frameworks and best practices for processes aimed at enhancing the availability, security and performance of data assets.
- Review and prepare corporate and technical documentation for review by senior staff.
- Provide feedback to staff and stakeholders to improve quality, consistency and accuracy of decision making.
- Lead collaboration across cross-functional agile teams to deliver data engineering solutions.
- Lead and manage team operations to deliver business and strategic outcomes, setting priorities and allocating resources effectively.
- Lead and manage staff to enhance performance and engagement by providing coaching, developing capabilities, managing performance, and supporting health and well-being.
Requirements
- Experience in a Government
Nice to have: Metadata repositories usage
Skills
Data profilingSource system analysisData qualityData cleansingData validationData architectureData modellingSQLTeradata EDWSQL AssistantTerada Studio ViewpointPcomControl-MRDBMSHDFS
Full description
Softtest pays pty ltdCompany details are for members
Sign up for a free job alert to see Softtest pays pty ltd's profile, website and LinkedIn, and the same for every employer on Referso Jobs.