New software can find more room for clean energy on transmission grids

Canary Media’s Down to the Wire column tackles the more complicated challenges of decarbonizing our energy systems.

Can sophisticated software help move more electricity across our existing power grids? The evidence from computer models and real-world tests suggests it can. Now it’s up to utilities, grid operators and regulators to put it to work at scale.

This is the second installment of our series on technologies that can expand the capacity and improve the efficiency of transmission grids, known as grid-enhancing technologies, or GETs. In our first installment, we covered dynamic line-rating systems that use sensors and analytics to determine how much electricity transmission lines can actually carry from one point to the other. In this piece, we’ll dive into new software that can give grid operators the insight they need to use their existing transmission networks far more effectively. Let’s call it transmission-optimization software.

Over the past five or six years, these software platforms have begun to prove their capability to unlock more grid capacity by solving hyper-complex equations to discover the optimal configuration of existing transmission networks. In fact, in a handful of locations in the U.S., they’re already solving real-world power-flow congestion problems, leading to major savings for grid operators.

That’s not an easy thing to do, given the innate complexity of how transmission grids work. The U.S. transmission grid network is an incredibly complicated machine, a network of hundreds of thousands of miles of high-voltage lines interconnected by tens of thousands of substations, transformers, circuit breakers, switches and control systems keeping terawatts of electricity flowing at the speed of light. Understanding how changes at one point in this network affect the flows of power across the rest of the network is a demanding and computation-intensive process.

Before we get into the nuts and bolts of the new class of transmission-optimization software systems being developed to solve these intensely complex grid-modeling challenges, let’s discuss their predecessors, which are still heavily used by grid operators around the U.S. These are transmission-planning and -analysis software platforms from major corporate players, such as Siemens’ PSS®E and General Electric’s PSLF. These software platforms have been built up over decades by experts in power flow dynamics, network topology, special protection schemes (for operating the grid in extreme conditions) and other such esoteric fields.

“They’ve been out there forever,” said Jay Caspary, a vice president at consultancy Grid Strategies — and the usefulness of these legacy programs is diminishing as the complexity of the grid is increasing.

Caspary, the former head of engineering and transmission development at U.S. grid operator Southwest Power Pool, has direct experience using these older software platforms to model the flows of high-voltage power across an increasingly dynamic and complex grid — and running up against their limits.

Getting these legacy power-flow modeling systems to ​“solve” for the optimal way to configure and upgrade the grid to adapt to changing conditions was computationally complicated enough when it was built around central coal, gas and nuclear power plants, he said.

But the addition of more and more wind and solar farms whose output fluctuates with the weather has made this power-flow optimization process harder for software to solve. In fact, sometimes with the conventional software platforms, ​“when you shut down your coal fleet and add a bunch of wind farms into the model, it may not solve,” or reach a solution for how to stabilize the grid, at all, Caspary said.

That’s why an increasing number of grid planners and engineers are being forced to work around the inability of these legacy software platforms to discover the right combination of transmission network reconfigurations, or additions of new equipment or power lines, that could allow new resources to connect to the grid.

By way of example, Caspary cited the work that Grid Strategies and engineers at The Brattle Group did to model how a range of grid-enhancing technologies, including transmission-optimization software, could reduce congestion and expand grid capacity for more wind and solar power on parts of SPP’s grid. They used one of the legacy software platforms, but it couldn’t handle all of the variables they wanted to consider. The team ended up running just one hypothetical change to the grid at a time, a time-consuming and labor-intensive process. ​“It was really a brute-force effort,” he said.

The results of that modeling project showed that a mix of grid-enhancing technologies could double the clean energy capacity of the grids spanning Kansas and Oklahoma, quickly lower electricity costs and carbon emissions, and pay back their implementation costs within a year.

But for the utilities that own and operate transmission grids, and the grid operators like SPP that coordinate their operations across wide swaths of the country, this kind of ​“brute-force” software analysis isn’t a practical way to figure out how to put these technologies to work, he said. ​“We need tools to do that, to co-optimize these technologies on their own,” he said. That’s where new transmission-optimization software tools will play a vital role, he said.

To be sure, transmission-grid optimization isn’t exactly a smoking-hot target for software developers or Silicon Valley venture capitalists. ​“These are very niche applications [with] high barriers to entry and very few customers,” he said.

That’s why the handful of transmission-optimization software platforms that have emerged to solve these kinds of problems in the U.S. over the past half-decade have been long-running collaborations between university researchers and the Department of Energy’s national laboratories and ARPA-E program, he said.

“You need DOE to fund that kind of stuff,” Caspary said — and then, some brave grid operators and utilities to try it out for their own grids, first in computer simulations, and then on the grid itself.

Now we’ll meet two companies developing transmission-optimization software platforms that are starting to gain traction on U.S. grids. They’re not trying to replace the power-flow modeling platforms from Siemens and GE. Instead, each of these startups has developed software specifically to tackle a discrete set of problems.

NewGrid: Tapping the hidden capacity in transmission networks 

NewGrid, a Somerville, Massachusetts–based startup, was spun out of a Boston University ARPA-E project in 2015 after three years of development funded via grants from ARPA-E’s Green Electricity Network Integration program. It has since received grants from the National Science Foundation; the Massachusetts Clean Energy Center, a public economic development agency; and MassVentures, a private venture capital firm.

NewGrid specializes in ​“topology optimization,” or how to quickly and accurately calculate the trillions of combinations of control settings that can alter the path-of-least-resistance flows of power across a transmission network. To do that, NewGrid’s technology leverages the high-voltage circuit breakers deployed across transmission grids, which are typically used for reliability applications such as preventing flows from disrupting or damaging equipment or causing cascading grid failures.

It’s theoretically possible to open and close these circuit breakers to allow power to flow more efficiently across the transmission network in response to ongoing changes in grid conditions. But figuring out which ones should be opened and closed has been far too complex a math problem for real-time or even day-ahead grid operations — at least until NewGrid began applying its technology to solve the immense optimization calculations involved.

“Historically, the way operators have managed the network is to consider the topology of the network to be fixed, and you optimize around the edges,” NewGrid CTO Pablo Ruiz told me in 2020. ​“What topology optimization does is to change it from being a constraint to being something more like a variable.”

In a February report, DOE states that topology optimization can ​“route power flow around congested or overloaded transmission elements,” which in turn ​“increases the transfer capacity of the grid.” Simulations that have applied NewGrid’s optimization across entire grid-operator networks have shown the potential to reduce existing congestion costs by 30 to 50 percent.

“By design, the optimization scope is systemwide,” Ruiz said in an interview last month. For a U.S.-wide transmission network that faces rising congestion costs — from about $3.8 billion in 2016 to as much as $5 billion in 2018, and on up since then, according to Grid Strategies — that’s potentially a major improvement.

Over the past few years, NewGrid has been taking incremental steps to prove out this potential with projects with Southwest Power Pool, Midwest grid operator Midcontinent Independent System Operator (MISO) and Texas grid operator ERCOT. It’s also starting to offer its services to renewable energy developers facing challenges connecting new wind or solar farms to the transmission grid, Ruiz said.

A 2020 report on NewGrid’s work with SPP showed how topology optimization could reconfigure its transmission network around a point in southeast Oklahoma that was subject to significant and costly congestion. SPP used the reconfiguration that NewGrid’s software discovered, reducing the price of power at that congestion point from about $600 per megawatt-hour to close to the SPP average price of $25 per MWh, delivering significant cost savings until a transmission reinforcement project was completed.

NewGrid’s work with market participants in MISO territory has also yielded a money-saving solution for transmission users in that market, Ruiz said. In one case, while several power lines were being taken out of service, NewGrid discovered a set of circuit-breaker settings that were able to reduce congestion on surrounding circuits by ​“significant” amounts, he said. That included one reconfiguration that increased power flows by 56 percent compared to the solution that lacked NewGrid’s optimization.

Caspary pointed out that MISO is in the midst of a ​“congestion cost reconfiguration process” that’s aimed at putting the kind of capabilities that NewGrid brings to the table to wider use.

In the meantime, Ruiz said, NewGrid has been expanding its work with renewable energy projects looking for ways to reduce congestion that might be forcing them to curtail how much power they’re able to supply to the grid or increase the costs of bringing more electricity to market. While he wouldn’t name the projects NewGrid is working with, ​“some of them have been operational for a while and have been suffering from congestion costs that have been increasing exponentially year-over-year,” he said.

Pearl Street Technologies: Solving interconnection’s hardest problems

Grid congestion isn’t just a problem for existing wind and solar farms, of course. It’s also become a major barrier to interconnecting new wind and solar projects to the grid. Beyond the years-long wait times for developers trying to secure an interconnection to transmission grids, one of the biggest problems facing new wind and solar projects is the expensive grid upgrades they’re being ordered to pay for to support the electricity they expect to add to the grid.

But what if grid operators could speed the process of studying what grid upgrades might be needed to allow wind and solar projects to interconnect to the grid or find solutions that cost less? Over the past year or so, Pittsburgh-based startup Pearl Street Technologies has found a niche for its Suite of Unified Grid Analyses with Renewables, or Sugar, software platform in doing just this for grid operators and clean energy developers.

Pearl Street Technologies spun out of Carnegie Mellon University in 2018 with support from ARPA-E and the National Science Foundation, then with follow-on investments from Powerhouse Ventures and Incite Ventures. Its initial plan was to provide transmission planners with a platform that could solve in hours the kind of challenging power flow analyses that could take weeks with traditional planning software — or lead to crashes and fail to yield a solution at all.

But in the past year, the company’s ​“focus has shifted to interconnection specifically,” CEO David Bromberg said in an interview. Pearl Street is already helping grid operators ​“accelerate interconnection studies through automation,” and it’s working on a platform that can give energy project developers ​“more data and modeling capabilities to drive siting decisions,” he said.

As we’ve already noted, the math that goes into determining what transmission operators need to do to integrate renewable energy onto their grids is intensely complex and time-consuming. Sometimes ​“the models can explode and not solve numerically,” Caspary said. When that happens, grid planners can be forced to turn to more conservative projections, guided by engineers’ judgment of what’s needed to keep the grid operating safely, including expensive capacitor banks or other devices, or in some cases, reconductoring (installing higher-capacity conductor wires) or building entirely new transmission lines.

Pearl Street’s Sugar software addresses these problems by applying techniques developed in the world of integrated circuit design for microchips, where simulation software routinely handles electronic systems with tens of billions of tiny semiconductor transistors, Bromberg said. The algorithm enables automation of complex grid planning processes, and in the end, produces results that are compatible with existing standards and models, he said. Here’s a sample diagram of Sugar’s methods compared to traditional power-flow optimization to convey a sense of the kind of heavy math involved.

This different approach allows Pearl Street’s Sugar platform to seek out and discover a more optimal solution for integrating renewable energy, Bromberg said. A third-party case study conducted in MISO territory supports the company’s claims for Sugar, noting that ​“a typical three-week process was reduced to a single working session lasting thirty minutes or less.”

These kinds of results are allowing Pearl Street to take its next steps into active implementation. Over the past year, the company has been putting its model-solving and upgrade-optimizing capabilities to work with Southwest Power Pool ​“to help them clear out their backlog” of interconnection requests, Bromberg said.

SPP has about 100 gigawatts of generation awaiting study in its interconnection queue — more than the 90 gigawatts of generation currently connected to its 14-state grid, David Kelley, director of seams and tariff services at SPP, told Canary Media.

“The models we build to conduct these studies are incredibly complex,” he said. ​“Previously it would take our engineers, using the brute-force engineering judgment method, three to four weeks, up to two months, to get these models to a solved, converged state.”

“Pearl Street can do that for us in about a week or less,” he said. What’s more, ​“the network upgrades we identify to integrate these new amounts of generation become much more realistic. They’ve been working with us in production now for the last six to eight months.”

These successes have encouraged Pearl Street to start offering its services to clean-energy developers that want to help grid operators and transmission-owning utilities find faster, less costly ways to allow them to interconnect to the grid, Bromberg said. The company recently closed a funding round led by venture capital firm VoLo Earth with additional backing from Pear Venture Capital and follow-on funding from Powerhouse and Incite to build developer-facing products and help other grid operators reduce interconnection study bottlenecks.

“We’ve worked with developers on an ad hoc basis,” Bromberg said. While he wouldn’t name them, he described the work as involving ​“using our software to propose alternative mitigations than what was selected by the grid operator…to see if there was the possibility of a targeted, lower-cost” option.

This application sounds like it could well fit what the Federal Energy Regulatory Commission plans to require from grid operators. FERC’s June proposal for modernizing interconnection processes across the country includes a plan to ​“require transmission providers to consider alternative transmission solutions if requested by the interconnection customer,” a category that could include grid-enhancing technologies such as dynamic line ratings, power flow controls or advanced conductors that involve adding equipment to the grid — or software that helps make better use of the equipment already in place.

Why aren’t these software platforms being used more widely?

Given all of the advantages of the software tools we’ve discussed and the success they’ve shown in their applications so far, why aren’t utilities, grid operators and regulators moving faster to put them to broader use?

One of the most fundamental barriers is the fact that ​“grid operators and planners are really conservative,” Caspary said. That makes sense, given that safety and reliability are their top concerns. ​“Why would a grid operator or planning engineer want to take risks on things that might not work?”

This conservatism can hold back progress even after grid operators like MISO and SPP have tested and implemented software solutions, he noted. That’s because U.S. grid operators like MISO and SPP are voluntary membership organizations made up of individual transmission-owning utilities that retain control over how they operate their individual networks. The transmission-owning utilities ​“know their system better than anyone else, but they can only control their part of the network,” Caspary said.

“When it comes to optimizing a network, through a reconfiguration, for example, the best place to do it might be in a neighboring system,” he said. But utilities ​“do not want to rely on their neighbor to do something. They want to be in control.”

This dispersion of control and decision-making can limit the scope of benefits that software like NewGrid’s can achieve. This chart from a project the company carried out for utility Alliant Energy, for example, indicates that the MISO reconfigurations it conducted saved the utility about $8 million in congestion costs. But it also found that further reconfigurations could have saved an additional $38 million.

What’s stopping Alliant from going after those additional congestion savings? Ruiz pointed out MISO’s current lack of a reconfiguration process, which means that most beneficial reconfigurations are not used.

It’s also noteworthy that Alliant doesn’t own the transmission lines in question, having sold its transmission assets to another company, ITC Holdings, in 2007. That means it doesn’t have direct control over how those networks are configured.

Caspary said that getting the most value out of tools like NewGrid’s will require transmission owners ​“being open to working with their neighbor and their neighbor’s neighbor to the greater benefit of the grid.”

It takes time for neighboring utilities to reach consensus on how they might coordinate this kind of activity, said Will Tootle, manager of operational planning for Southwest Power Pool. That’s why SPP is now using NewGrid’s software ​“in more of a back-office support role,” to ​“compare and identify options” for transmission planning.

“It’s definitely being discussed more in our work group forums,” he said. ​“From an economic perspective, there’s definitely some desire to utilize the grid as much as possible.”

At the same time, ​“depending on the results of the tools, sometimes there are reliability risks, and we want to make sure we maintain reliability of the system,” Tootle said. ​“It’s definitely a balance to implement any of the options” among transmission-owning utilities with different levels of comfort in changing how they do things, he said.

Bromberg of Pearl Street conceded the difficulty in deploying GETs in the context of interconnection studies, as there are still questions from transmission owners and service providers on how these technologies fit into long-range, multiyear planning processes.

At the same time, he noted that implementing new software doesn’t require ​“a fundamental change to a process, or a tariff, or a business practice manual. We just insert our software to get studies done faster, in a more repeatable way, with higher quality. Not only are the [independent system operators] happier, interconnection customers are happier as well because they get their data back sooner, and the studies are more repeatable.”

Another benefit of software is that it doesn’t require costly grid hardware or upgrades, NewGrid’s Ruiz said. His company uses existing circuit breakers that ​“transmission customers have already paid for in rates,” he said. ​“That’s why speed to deployment is so much faster.”

This use of existing equipment is an advantage for ratepaying customers, but not necessarily for utilities that earn a guaranteed rate of return on every dollar of capital investment that regulators allow them to make. This ​“cost-of-service” paradigm underlies almost every decision made by U.S. utilities, including whether or not they pursue lower-cost software solutions to grid constraints and interconnection bottlenecks.

That’s why proponents of grid-enhancing technologies, ranging from the companies that make them to U.S. senators, are asking FERC to create incentives to encourage utilities to deploy them. ​“The overall incentives that are embedded in the current regulatory structure are a barrier to extracting the most value from the transmission grid,” Ruiz said. That’s not to say that transmission-owning utilities are ​“forgoing these opportunities on purpose,” he said. ​“But it does mean that they would set aside their most precious resources — their expert staff — to find the opportunities they are incentivized to find.”

This holds true not just for software that can find undiscovered capacity on the grids that utilities already have, but also for software that can manage the broader array of new grid-enhancing technologies that can sense and shift power flows along high-voltage transmission corridors, Caspary said.

“How do you model grid-enhancing technologies?” he said. ​“The key is to apply them into ongoing processes,” from grid planning to real-time operations. That’s one of the projects being undertaken by currENT, a GETs trade group in Europe, where transmission operators are organized in ways that give them more freedom to try new technologies, he said.

The idea is ​“to set up a framework so that we know how we’re going to model” new devices and software capabilities, he said, whether that’s traditional power-flow modeling platforms or newer systems like those from NewGrid and Pearl Street. ​“You don’t want one vendor doing it one way and another vendor doing it another way, and end up getting different answers” to what those devices will do on the grid.

This kind of standardization will become increasingly important as grid operators start to put to use one of the more powerful — and technically complex — tools of transmission network management: power flow controllers and other flexible AC transmission system (FACTS) technologies. In our third installment of this series, we’ll delve into the wizardly worlds of FACTS and how they integrate cutting-edge power electronics and digital power controls to alter the fundamental nature of how electricity flows.