At this years’ RIMS conference in Palmerston North, Dawn Inglis spoke on behalf of the Road Efficiency Group (REG) about the need to improve the quality of data used in the transport sector.
This article is based on her presentation. Dawn is the manager Waikato Road Asset Technical Accord (RATA), a collaboration delivering strategic road asset management to councils across the Waikato region. She has also has been instrumental in delivering the REG programme.
REG WAS SET up as a result of recommendations made by the Road Maintenance Taskforce (RMTF) to help the transport sector improve investment decision-making. It’s a cross-sector initiative involving representatives from local government, LGNZ and NZTA.
During 2017 REG rolled out the first phase of its data quality project with the focus on ‘data quality’ in decision-making, directly related to the One Network Road Classification (ONRC) performance measures.
Dawn provided background on the project and outlined what has been achieved so far and noted that, when the project kicked off around 18 months ago, there were a couple of ‘elephants’ in the room due to the naysayers saying the tools weren’t available, or there wasn’t a willingness to make a difference, and the topic of data quality meant different things to different people.
However, the project team – using kiwi ingenuity and the appetite to ‘do the right thing’ – focused on the ‘prize’ of developing tools that will make the sector more effective and enable them to tell a better outcome story.
“Within REG there is recognition that at all levels data quality needs to improve to support greater confidence in asset management and investment decision making in the transport sector,” says Dawn.
“The project team needed to size the work to a manageable data ‘chunk’ so that we could deliver useful outputs on a timely basis.”
Malcolm Alexander; chief executive of Local Government New Zealand and a member of the REG Governance Group, says his association supports all efforts to ensure local authority members have complete, accurate and timely performance data by December this year.
“With all local authorities increasing the reliability and accuracy of information we will be able to confidently demonstrate why investment in transport is going to achieve the customer outcomes important to us all,” he says.
A national focus
An objective of this project was to enable the ‘quality’ of data to be assessed and reported by every local authority and the state highway management team, and establish a national perspective.
“It was important that the project was able to identify each authority’s current results so that there could be a confident start to their journey of improvement, and also sell a compelling story about the benefit of improvement,” says Dawn.
Having nationally based information based on several data quality dimensions and data types, allows the project team to develop an overall evidence-based understanding of the root causes of data quality issues and understand why data is not of a better quality and standard.
Phase one of the project started with ONRC Performance Measure data related to the sector, agreed customer outcomes and technical performance measures extracted from RAMM, the NZTA Data Warehouse and the ONRC Performance Measures Reporting Tool.
This involved determining data quality and a set of criteria, or ‘data dimensions’, that included completeness of data, accuracy and timeliness of data with a suite of metrics for each dimension (associated with each of the ONRC performance measures).
Once these were agreed on, they could then be assessed including defining what was ‘good’ for each metric. Adopting a traffic light system each data set was assessed and categorised using a ‘traffic light’ (green, orange red) system.
The first reports based on 2014/15 data were taken to the 2017 IPWEA conference for initial sharing with the sector.
“The reports were received positively, which gave the project team the confidence to carry out a second round of reporting based on the 2015/16 information,” says Dawn.
In parallel, REG was also able to use the data in the ONRC Measures Reporting Tool that allowed a history of annual performance snapshots to be reported.
All stage one reports for 2014/15 right through to 2016/17 are on the REG website for all road controlling authorities to view.
Dawn confirms that the RAMM queries replicate the report results on the REG website so these can be run on an RCA’s RAMM database at any time.
“To ensure that we realise the benefit of the data quality project, an assessment of the data reports has been completed and key data quality themes identified. It is acknowledged that some local authorities have their own individual challenges and may be sitting apart from the rest of New Zealand, but certainly there are some areas where there will be some quick wins to address the data quality apparent across all local authorities.”
Looking at the result across all of the country it was found that around 60 percent of all local authorities are achieving the expected standard (when assessed by each metric indicator), which is a good result. It also shows there are still significant opportunities for improvement.
Dawn says that looking at the 2016/17 results and reviewing the previous years’ results to review progress, it is clear that there had been an improvement in accuracy and timeliness of the data over that period.
From the review it was identified that one of the key issues across many local authorities is how they manage their traffic data and the updating traffic estimates. Investigating this matter, REG has found that there is a lack of industry guidance in terms of both a ‘why we do this’ and ‘how do we do this’.
REG has established a work programme and will be starting to roll out a range of information to assist the sector where there is an indication of need including:
• Traffic demand (updating estimates);
• Traffic counting (strategic approach to counting traffic volumes on our network);
• Recording maintenance activities;
• Data collection (roughness);
• Managing treatment lengths; and
• Updating crash data in asset management systems.
Tools and information will be rolled out through the REG learning and development programme, on the REG website, and through its regional workshops.
Phase two of the data quality project is about to start and will interrogate a wider suite of management decision making processes, adds Dawn.
“This work is still in the planning phase, but will very much focus on how decisions are made. It will check whether we are delivering the right customer outcomes, and whether we are collecting the right data to be able to demonstrate this.”
Phase two of the project will assess both the metrics and the data used for decision making and the outcomes we achieve from our asset management planning.
For this project it is expected that data will be extracted from both RAMM and individual RCA information extracted from the NZTA Transport Investment Online (TIO) to allow comparisons with planned programmes with those achieved (and as-built). The project will also review condition indicator data, bridge and footpath data.
This project phase will also be split in two parts – information analysis reports and the outcome of root causes and opportunities for improvement. Expect the first reporting in September 2018.
So far the feedback from the sector has been positive on this project.
“The challenge to the transport sector is how each authority is going to get into the ‘green zone’ in terms of data quality,” says Dawn.
It is the goal of REG that the sector no longer sees data collection and management as a compliance exercise but, instead, understands how good quality data improves confidence in the decisions that are made based on it.
Jim Palmer, Waimakariri District Council chief executive and member of the REG governance group is committed to supporting the initiatives being delivered by REG.
“It is critical that we know what we are buying, we know how our transport system is performing and we need to be able to be honest with our customers.
“Quality data is the foundation for every road controlling authority to be able to do this. REG has worked hard getting this information out to the sector and also making sure that we’ve got the right tools for everyone to use.”
Dawn confirms that the RAMM queries replicate the report results on the REG website so these can be run on an RCA’s RAMM database at any time.
• For more information on REG visit www.nzta.govt.nz/roads-and-rail/road-efficiency-group.