These research efforts included one that uses advanced computer vision to classify sewer inspection data acquired by robotic equipment, one that developed advanced geospatial data mining tools to support water distribution system pipe management, and one that is developing and testing new sensor monitoring systems for gas pipelines.

Intelligent inspection

Professor Soibelman first described the Carnegie Mellon University Civil and Environmental Engineering Department AIS group’s vision. According to this vision infrastructure systems and the processes to design, build, and operate those systems must become intelligent, able to continuously determine their conditions, perform self-assessment and support proactive decision making that improves their performances, increases their life spans and reduces life-cycle costs and impacts. This is a vision of data-driven intelligent decision making about, and in some cases by, infrastructure systems, subsystems and components. The data is collected at different times and frequencies from sensors of different types, modalities (including humans) and accuracies. These sensors have been rationally selected and placed to best support the collection of data that will be needed over the entire life-cycle of the system. Those data are effortlessly collected, modeled, analysed, mined and transformed into useful information about the condition and behaviour of the system.

The information derived from these data continually informs intelligent decision making within the processes that make up the life-cycle of the system. In fact, the information not only influences the operation of a specific system, but goes on to influence the design of the future generations of designs. Such a vision requires research and development in the areas of sensors, data modelling and analysis, simulation, decision support, visualisation and human-computer interfaces, to name but a few of the issues that need to be addressed to deliver this vision. Research that places these ICT systems into realistic large-scale test beds and evaluates their performance is critically needed. Research is also needed to help understand where ICT is most effectively being deployed in practice and leading to clear economic benefit.

Some of this needed research is being conducted by the AIS group at Carnegie Mellon in clusters of research related to sensor development, mobile/wearable computer systems for infrastructure-oriented data collection applications, advanced data modelling and management tools to improve classification and retrieval of the ever increasing amount of data generated by new data acquisition tools, novel data mining and analysis tools, system level approaches for using combinations of sensors, data analysis, and advanced reasoning.

Professor Soibelman introduced research being conducted that aims to support regularly and proactively assess conditions of sewer infrastructure systems to ensure their structural integrity and continuity of services. This research is a critical step to advance the state of automated pipeline inspection and condition assessment. Currently, a critical issue is to address realistic defect detection that deals with real sewer inspection data. In his presentation Professor Soibelman introduced the findings of a research project that seeks to enable automated detection of defects in sewer pipelines from inspection videos and images acquired by robotic equipment. The need for and the challenges of automated defect detection in sewer infrastructure condition monitoring were presented. Based on a general detection and recognition model established in this research, a change detection based approach, which is tailored to solve the challenges in the sewer pipeline domain, was presented and illustrated through case study.

To process effectively the large volumes of visual data collected by the robot’s optical sensor during inspection of the internal pipe wall surface this research proposed a multilayer approach for automated defect detection and recognition of pipeline defects. In the first process block “Detector”, the model flags regions of interest (ROIs), such as problematic areas and/or critical pipe patterns in the images. Each of these flagged ROIs is input to a “1st level classifier” that broadly recognises it as a false alarm or as a defect. The defect is then input to a second level classifier that further discriminates it from among several different defects, for example: a crack, a fracture, roots, corrosion, or a lining failure. The defect, such as a crack, can still be input to a classifier at the next level that determines if it is a horizontal or spiral crack. A final classifier may determine the degree of relevancy of the defect, for example, ‘immediate attention,’ ‘further monitoring,’ or ‘safe to ignore’. This automated defect detection and recognition capability can facilitate robotic intelligence and multi-sensor based pipe inspection by locating and framing ROIs, and recognising the framed ROIs as true defects or false alarms (non-defects). Then, based upon the flagged ROIs and detection results, the intelligent crawler can make further intelligent decisions, for example:

Call for further sensing or other actions automatically. For example, if a true defect is detected, videos or images of higher quality could be recorded for subsequent maintenance decision support; or other inspection sensors could be started up to acquire appropriate inspection data.

Guide further analysis for target recognition. For example, according to a municipal authority’s need for inspection, a particular need is to identify and quantitatively measure extents of corrosion within a certain pipe segment. An automatically detected defect can be further classified as “corrosion” or “non-corrosion”. If corrosion has been recognised, a laser scanner could be further used to acquire data so that a quantitative measurement is conducted. Hence, the municipality’s need for identifying and quantitatively measuring extents of corrosion can be achieved automatically.

Geospatial analysis

The second research introduced by Professor Soibelman was research that is supporting the development of advanced geospatial data mining tools introducing an example of breakage analysis for water distribution systems. On a macro scale, the spatial analysis of failure data might provide insights on physical condition trends present in a physical network system. On the one hand, sensing specific pipes can provide information about their individual condition, just like medical exams provide diagnosis of the health of individuals. On the other hand, spatial analysis aims to provide information about the condition of a population of assets, for instance pipes, much like the work of epidemiologist identifying outbreak of diseases in a country or state. One example of interesting trends in spatially referenced failure data is the presence of clusters of failures in space, for example sets of failures that are close to each other.

Such clusters might be indicators that some underlying and possibly unknown common cause might exist. By using a detection of cluster approach that is specific for the case of failure in physical network systems, we identified critical areas in a water distribution system. Pipe breakage data was used in such research. However, in principle, any measure of pipe condition can be used to the identification of abnormal patterns in the systems. Therefore, as monitoring systems become widespread, better data can be used to perform system wide spatial analysis of failure in network systems. After detecting such clusters, the factors associated with failures have been analysed to in order to identify possible causes and guide future decision in replacement and design. In an ongoing work, several data mining approaches, which address the specific challenges that spatially referenced data, have been used in order to identify interesting attributes associated with water pipe clusters.

Structural health case study

Finally Professor Soibelman introduced a project that is evaluating technologies for the monitoring of gas pipeline delivery integrity, through a ubiquitous network of sensors and controllers to detect and diagnose incipient defects, leaks, and failures. Structural health monitoring of buried pipelines is of vital importance as infrastructures age. Ultrasonic guided waves are a popular method for inspecting buried pipes, due to their potential for long propagation. Unfortunately, the large number of wave modes present, and the effects of dispersion in a pipeline make analysis of the received signals difficult. He presented preliminary results of tests that were developed to assess the capability of Lead Zirconate Titanate (PZT) wafers to fully illuminate a pipe presenting data that shows rich illumination proving that a single PZT wafer can illuminate the circumference of pipes with sufficient energy for defect detection, localisation, and classification. The output of this illumination clearly shows multiple modes and multiple path information illuminating the pipe.

He then introduced a Time Reversal Acoustics methodology that is being developed to support a focusing approach that increases with a greater number of wave modes and a greater degree of dispersion supporting defect detection and localisation. The use of Time Reversal Acoustics to compensate for these complex signals, and improve performance for the detection of faults in a pipeline shows a potential for a reduction in the power and hardware requirements of fault detection systems creating the possibility of the development and application of those systems for pipeline monitoring and not just for inspection.