Understanding Methane Detection Sensitivity (PART 1)

Sensitivity Part I: Breaking Down Methane Detection Sensitivity and Common Confusions

When it comes to detecting methane emissions, it’s not only good to be sensitive, it’s imperative. 

In our recent blog “5 Questions to Ask When Evaluating Methane Emissions Detection Technologies,” one question, in particular, produced a large amount of interest. When we suggested that operators ask methane detection solution providers: “What’s your detection sensitivity?” we had a number of follow-up requests to dive deeper into what it means to be sensitive. 

Around this same time, the EPA released its proposed standard for methane regulation, which includes “… using a methane detection technology that has been demonstrated to achieve a minimum detection threshold of 10 kg/hr.” Detection sensitivity is always front and center in our discussions with operators and regulators alike. And rightfully so... it’s critically important. The problem is that there’s lots of confusion around how the information is delivered and interpreted. What does detection sensitivity really mean and what is even being measured? Do different technologies have better detection sensitivity than others? Why is sensitivity important?

In this blog series, we break down the basics and clarify the confusion of what it means to have sensitive methane detection and why it’s important.

What Does “Methane Detection Sensitivity” Mean?

Methane detection sensitivity refers to the size of a methane leak (i.e., the “emission rate”, or how much gas is escaping from a leak over a given time) that can be detected by a technology solution. A solution with “better” detection sensitivity means it can detect smaller emission rates (smaller leaks), while a solution with “poorer” detection sensitivity may only be able to detect the largest leaks (“super-emitters”).

Before we move on, note that emission rate detection sensitivity is different from gas concentration detection sensitivity. Gas concentration represents how much of a gas is present at a specific location and is a direct indicator of safety at a specific point in space. On the other hand, the emission rate is the most important indicator to enable emissions reduction, gas certification, emissions inventories, and sustainability. Check out our related blog to learn more about gas concentration, the different ways it can be measured, and the different units used to characterize it. In this blog, we’re focused on emission rate detection sensitivity.

Why is Better Methane Detection Sensitivity Important?

Emission rate detection sensitivity is arguably the most important parameter for enabling emissions reduction. After all, if you can’t detect the leaks, how can you fix them? Fundamentally, the detection sensitivity level can mean the difference between finding or missing a methane leak, which can have significant impacts on both safety and the environment. An emissions inventory of an oil and gas operator’s infrastructure doesn’t mean a whole lot if you’re missing a significant fraction of the emissions. And it’s tough to track the effectiveness of sustainability initiatives or to certify a natural gas supply chain unless you’re catching the vast majority of emissions. 

Pioneer Resources noted the benefits of a switch to a more sensitive detection technology in their 2021 Sustainability report, stating that “... using a higher sensitivity technology allows us to… understand the full picture of our methane emissions.” 

So why not just use enough sensitivity to catch 100% of the emissions? The trade-off to achieve better detection sensitivity is often cost… it typically costs more to catch a larger fraction of emissions. The incremental cost of increased detection sensitivity to catch 90% of emissions instead of, say, 30% of emissions, may be reasonable; but the extra cost needed to go from 90% to 100% emissions detection may be astronomical.  

So, the natural questions are “how much sensitivity is needed?” and “how small of a leak needs to be detected?” We’ll answer those questions and more in an upcoming blog in this series. Before that, we need to clarify some things that can cause confusion around detection sensitivity. 

Understanding Methane Detection Sensitivity 

To better understand detection sensitivity, it’s important to begin by clarifying four of the key terms and concepts that can be confusing. First, different people prefer to use different units for emission rate. Second, the terms “high sensitivity” and “low sensitivity” can cause confusion. Third, detecting emissions is statistical in nature. Finally, detection sensitivity depends on the operational and environmental parameters under which the detection measurements are made. Let’s dive into, and clarify, these complexities to have a better understanding of each.

1. There Are Multiple Units to Describe Detection Sensitivity

Emissions rate detection sensitivity is commonly stated in a variety of different units, which can be confusing. While it would be easier if we all picked one unit, that’s just not how it is. In the US, detection sensitivity is often stated in standard cubic feet per hour (scfh, or “skiff”) or thousand standard cubic feet per day (Mscfd). In Canada and elsewhere internationally, detection sensitivity is often measured in the units of kilograms per hour (kg/hr) or cubic meters per day (m3/d, or “cubes”). All of these refer to the same thing: emission rate -- just in different units. This is similar to how a “5K” race (i.e. 5 km) is the same distance as 3.1 miles. We recommend making your solution provider switch to the units you prefer so you can compare apples to apples. If needed, here’s a handy calculator to convert from one emission rate unit to another.

2. Better Sensitivity Means Detecting Lower Emission Rates

The second confusing aspect of detection sensitivity is that the term “high sensitivity” implies good sensitivity, while it represents the lowest emission rate value. For example, 3 kg/hr represents a better sensitivity than 10 kg/hr, even though 3 is a lower value than 10. Along the same lines, “low sensitivity” implies poor sensitivity but represents the highest number. To avoid this confusion, we recommend avoiding the terms “high, greater” and “low, smaller” sensitivity, and stick with terms like “excellent, better” and “poor, worse” sensitivity.

3. Methane Detection Sensitivity is Statistical and Probabilistic

The third confusing aspect of assessing detection sensitivity is that detecting leaks is statistical and probabilistic. If one leak is detected at an emission rate of 10 kg/hr, that does not guarantee the next leak of the same size will be detected. It also does not guarantee that every emission smaller than 10 kg/hr will be missed. To have a meaningful detection sensitivity value, this statistical nature must be captured. Ask solution providers to include a Probability of Detection (PoD) associated with the detection sensitivity to help understand the chances of detecting a given emission, and to compare apples-to-apples. 

At Bridger Photonics, we recommend using either >90% PoD or 50% PoD because they represent two defining cases that can be important. For example, a detection sensitivity of 10 kg/hr with >90% PoD means that, for given environmental and operational parameters, the technology solution will statistically detect at least 9 out of 10 leaks that are 10 kg/hr. The >90% PoD imparts confidence that a technology solution will catch a leak of a given size. Alternatively, a detection sensitivity of 10 kg/hr with a 50% PoD means that the technology solution will catch half, and miss half, of leaks that are 10 kg/hr. So, it’s a coin flip whether you will catch a leak of that size. We discourage advertising detection sensitivity with PoD less than 50% because who really cares if you can only catch, say, 1% of leaks of a given size?

We also encourage interpretation of the term “lower detection threshold” to mean the smallest leak size that is confidently detected (i.e. >90% PoD) under all conditions.  

4. Weather and Operational Conditions Affect Methane Detection Sensitivity 

The fourth confusing aspect of assessing detection sensitivity is that detecting leaks depends on the operational and environmental conditions under which the detections are made. No matter the detection technology, there will be conditions under which the technology performs better and conditions under which the technology performs worse. Operational parameters are often within the solution provider’s control (e.g. flight altitude and flight speed for an airborne remote sensor, or the number of sensors and sensor spacing for point sensors). On the other hand, many factors that can affect the detection sensitivity are often outside of the solution provider’s control (e.g. ground wind speed, direction, or gusts, the reflectivity of the ground surface, or humidity). And not all technology solutions are affected by the same environmental factors. For instance, the detection sensitivity of some remote sensing technologies additionally depends on overhead clouds, shadows, and the angle of the sun in the sky.

At Bridger Photonics, we recommend providing the detection sensitivity at least under “typical” operational and environmental conditions. This detection sensitivity should be backed up with direct evidence and data over a large number of conditions in a given basin. Controlled releases performed by third parties to test detection sensitivity are ok, but can present their own issues: the conditions for the test may be “ideal” for detection instead of “typical”, and a technology’s performance on such a test can be artificially, but substantially, improved because the location of the emitter is known. If the solution provider knows exactly where to look for the controlled emission, the detection sensitivity can be artificially improved without the worry of false positive detections (unrealistic conditions). We recommend that if you’re reviewing third-party results of a study, to check if it was a fully-blind study (no knowledge the test was even being performed), as well as see what the conditions were during the testing.

Some operators (and regulators) want more confidence than the “typical” conditions, and ask “What’s the worst I can expect?” In these cases, methane detection technology providers should be expected to provide the detection sensitivity and PoD achieved under all operational and environmental conditions. In this case, expect your solution provider to give you an audit report of the measured sensitivity for each of your sites to prove that they achieved their claimed detection sensitivity.

Takeaways for Understanding Emissions Detection Sensitivity

Assessing and comparing different methane detection technology solutions requires thought and care of the emission rate detection sensitivity. As an operator or a regulator, you should expect a solution provider to:

• Provide detection sensitivity values in whatever units that you like and are familiar with;

• Provide a Probability of Detection corresponding to the detection sensitivity value;

• Provide the conditions under which that detection sensitivity will be achieved;

• Provide the factors that limit the applicability or performance of the technology solution.

Using the EPA’s 10 kg/hr as an example, here is an emission rate detection sensitivity that provides full information for comparison:

10 kg/hr with >90% PoD under all allowed operational and environmental conditions.

Now that we have hopefully clarified some of the confusing aspects of detection sensitivity, the next blog in this series will begin to dive deeper into sensitivity and look at how much sensitivity is needed to detect emissions throughout the entire natural gas value chain.