Abro: My name is Amber for today folks.
Shiroma: Thank you, great.
Batjer: Thank you.
Amber: Welcome to the California Public
Utilities Commission committee meeting
on this day, Wednesday, February 17th, 2021.
All participants will be in listen only
and tell the speaking session today at that point
you may press star one
if you would like to make a public comment.
This call is being recorded.
If you have any objections, please disconnect at this time.
I now like to turn the meeting over
to your host president Batger, you may begin.
Batjer: Thank you, Amber.
Thank you all for joining today.
Good morning and thank you for being with us today.
I am calling the CPUC Commissioner committee meeting
of February 17th, 2021 to order.
I apologize for not being on audio.
I'm having technical difficulties today.
So I will just be, excuse me, not on video,
I will just only be on audio today.
There are three committees,
the finance and administration committee,
the policy and governance committee
and the emerging trends committee.
Today the emerging trends committee will meet.
We will have an opportunity
for public comment falling Q and A
and discussion from the Commissioners.
If you wish to make a public comment
or ask a question please dial (800) 857-1917
and enter the code 5180519 and press star one.
You will be placed into a queue
and will be called upon to speak
when we get to the public comment period in this agenda
which will be at the end of each committee meeting.
I will now turn it over to Commissioner Guzman Osavis
and Commissioner Shiroma to introduce the items
on the emerging trends agenda.
Commissioners.
-: Thank you President Batjer.
Good morning everyone.
We will be discussing just a little bit
about the work plan for the year for emerging trends,
but we did as you know, did a staff survey
to get input from all of you
on what topics are really critical for us to undertake
and given the positive response we're gonna take a moment
to really integrate that into the work plan,
but I just wanted to highlight a couple of,
you know, two general topics.
I mean, we got well over,
let's see over 20 recommendations from staff
and even before that, we had been working off of a list
of items that we had not yet gotten to.
So everything's from, you know, working from home
and really starting to discuss
what some of the experts are saying around that
as a focus to really external issues like reliability
that we'll be talking about some of those metrics today.
But every single division provided some input
from the TNCs and some of the red lining concerns
that was a really interesting topic
that I thought was brought forward
and everything else in our more traditional space
around energy and we'll be seeing some of these today
as I discussed, some, winning prizes.
So, but just to reinforce that we heard everything
from wildfire safety.
There was a recommendation on really looking
into the indigenous sporting practices for wildfire safety.
Communications had a lot of
really interesting recommendations
and some of which we're already working on.
So we're gonna digest all of these
and come back with a work plan,
we'll try to post it before
our next emerging trends committee meeting in March,
but we definitely will highlight what the work plan is
in March and obviously have it posted on the website.
Okay, probably the most exciting thing
is that our staff, both Commissioners, Shiroma and staff
and others have worked together to do a live raffle
for the two winners.
Commissioner Shiroma and myself
we'll be doing an Amazon gift card for two lucky winners
for participating in this survey.
So I'm gonna ask Justin Song to bring up the wheel
and just to note that because we've made a little mistake
on the survey and forgot to put a tab for your email,
we're gonna be following up and trying to hunt you down
'cause some of you didn't put your names next
to your topic recommendations,
but here you'll see some of the topics
that were brought forward, all of the topics
that were brought forward.
And so we're gonna spin the wheel twice.
So Justin, if you could do the first spinning.
And this one will be for the gift card
that I will be gifting.
Okay, increasing development 5G infrastructure.
Ow, that's really, and I'm just checking the list
to see if by chance this was one of the folks
that actually provided their information,
and if anyone knows that (indistinct) my list,
doesn't have that on right now.
Okay, so whichever staff person submitted this,
we're gonna track you down
'cause you have won a $25 gift card for Amazon, okay.
All right, and Commissioner Shiroma,
you can call the spin for Justin on the next one.
-: Okay, Justin, go ahead and spin the wheel.
Ah, oh, very good, hydrogen and renewable natural gas.
Both topics, really excellent.
So we'll figure out who put forth this idea
and Martha and I will be following through
with the Amazon gift card.
-: Thank you.
Thank you very much.
So I think with that Commissioner Shiroma I'm gonna turn
to you to introduce our first topic
for emerging trends for the year.
-: All right, thank you Commissioner Guzman Osavis.
All right, so our first presentation,
the topic is on electric reliability metrics for 2020.
What we have learned and we will hear first
from Julian Enis.
Julian is a utility engineer
in our resiliency and micro grids energy group
from the energy division.
Again, Julian is a utility engineer.
He provides his expertise on issues related
to electric system reliability and resiliency
and we've worked together extensively
over this past year or so on our micro grids proceeding.
Julian has a degree in mechanical engineering
from the University of California Davis go Aggies,
and Julian will be presenting to us today
on metrics and trends in electric system reliability
including an analysis of 2019 reliability metrics
and opportunities to better leverage this data.
This is certainly a very important topic
as we see what has been happening across the country.
And I also wanted to do a shout out
to Julian's manager, Forest Kaeser
who is also attending today's committee meeting.
All right, Julian, the microphone is yours.
-: Well, thank you so much Commissioner Shiroma
and indeed go Aggies.
Good morning, Commissioners, directors, members of staff
and members of the public.
As Commissioner Shiroma introduced me, am Julian
and I'm here to talk about electric system reliability
and the reports that utilities are required
to file every year and just a little bit more data analysis
and some next steps that we can take
to improve the reporting requirements
and I'm better understand what this data tells us
about the electric system of California.
Next slide please.
So the basics of electric system reliability
and this is what we'll be covering in this presentation.
We'll be touching on how electric system reliability
is defined.
Why do you keep track of electric reliability?
What the California utilities are required
to report to CPUC annually?
How did the California utilities compare
to the rest of the United States
and a little bit more about
how granular reporting standards are
and how you keep track of the worst performing circuits
and what can be improved.
Next slide.
So first we'll start
with how electric system reliability is defined.
These are statistical representations of outages
and we'll dive into a little bit
about what they really mean.
Next slide please.
So electric system reliability is defined
by IEEE 1366 and within this standard
there, the four major metrics are defined
as SAIDI, SAIFI, CAIDI and MAIFI.
I realized that these are a lot of new acronyms.
So wherever in the presentation these acronyms show up,
I have tried to spell them out for folks
just to make sure we have clarity.
However, these four metrics
are the generally accepted standard
by which electric utilities across the United States measure
their system performance.
SAIDI is the System Average Interruption Duration Index.
SAIFI is the System Average Interruption Frequency Index.
CAIDI is the Customer Average Interruption Duration Index
and MAIFI is Momentary Average Interruption Frequency Index.
Next slide please.
So here are some written definitions for these four metrics,
but I will quickly give an explanation
of what they each mean.
So effectively SAIDI is the average number of minutes
that a customer on the entire utility system
can expect to be without power
over the course of a given year.
SAIFI or the System Average Interruption Frequency Index
is the expected number of times that the average customer
on the utility system will experience an outage
over the course of the year.
CAIDI or the Customer Average Interruption Duration Index
measures how long each outage is for the average customer
and MAIFI measures how many, and MAIFI, sorry,
the Momentary Average Interruption Frequency Index measures
how many momentary outages or outages
that last less than five minutes
the average customer on the utility system
will experience over the course of the year.
Next slide please.
Next one.
The next topic is to talk about major event days.
So the reliability indices are generally reported
with and without what are called major event days.
Major event days are a statistical threshold that is based
on the daily System Average Interruption Duration Index
and it's a statistical threshold that's based on
the previous five years of data.
The idea of a major event day
is that it measures the impact
of, high impact and low frequency events.
Just as a note, this definition
is a statistical definition only based on outage durations
and it does not account for causality.
So things like earthquakes, storms
and public safety power shutoffs
are considered any MEDs only in so far as the events
daily system average interruption duration index
exceeds the threshold.
Reliability indices are generally used
to motivate investment decisions
that will lead to improvements in system reliability.
Generally the utility look at reliability excluding
these major event days to focus how it needs
to improve reliability, general system reliability overall.
However, looking at reliability with MEDs utilities
can see how significant and large events
such as those earthquakes, storms,
public safety power shutoff and so on
can dramatically impact the customer experience.
Next slide please.
So this is a statistical representation of data
and it does have some drawbacks.
So the reliability statistics themselves focus
on an outage duration and customer accounts
over the entire system which can obscure regional variation,
and you also need to have very, very large systems
as you can see from this graph PG&E has 99,000 miles
of overhead lines within its service territory as of 2018
and the enormous size of this system
with its own regional climate and density variations
can make the system level reliability indices
a little bit difficult to interpret.
So just a caveat on this data.
Next slide please.
So why do we keep track of reliability metrics.
At its most basic, as most basic we use this data
to improve the electric system
and improve customer experience on the electric system,
but we'll dive into that.
Next slide.
So utilities generally use the reliability metrics
so that at its base level they can track the performance
of individual all the way down to individual circuits
so they can address service issues.
Generally, these metrics are used as scoring criteria
for remediation measures and cost-effectiveness assessment
on hardening measures taken to address reliability issues.
Additionally annually, the worst performing circuits
by the system average interruption duration index
and system average interruption frequency index
are required to be reported to yearly
and are targeted for remediation
through programs that the utilities put out.
You can also track persistent issues with blue-sky service
and the impacts of major events
at which is all important data
and utility in the distribution planning process.
Next slide please.
So if you have an example
of how these electric liability metrics were used,
this is from the 2018 PG&E report out on system reliability.
As you can see the Stagg 1105 circuit was identified
as a poor performing circuit
and they installed new protection devices
to resolve reliability issues identified
and actually achieved a 25% improvement
in that sort of reliability.
In addition, they also installed TripSavers
in place of regular fuses.
So that reduced the number of sustained outages
for customers.
So this is just a basic example of how utilities use
this reliability data to target their remediation.
These particular ones were within the Stockton division
where this electric reliability annual town hall is held.
Next slide please.
So the what are the utilities required
to report to the CPUC?
So as established by Commission decision
the utilities are required to report these metrics
to us anyway.
Next slide.
So these are sort of the foundational decisions
and Commissioned procedures that have brought us to today
and our current reliability reporting requirements.
The first decision was in 1996, the setting up
the recording reporting requirements for system outages.
Further decision in 2000 added 312 (indistinct)
which defined major events
and set a restoration time benchmark
as the customer average interruption duration index
of 570 minutes for a reasonableness review
of particular outages.
In 2011 advice letters for issued adopting IEEE 1366
and its definition of major event days,
and they were allowed, the utilities were allowed
to exclude days that exceeded the major event days threshold
from their calculations for liability.
Finally in the 2016 decision 16-01-008 set up a reliability
reporting standard for the California utilities.
Independent to be of this decision,
there isn't a template by which each of the utilities
must annually report their system reliability data
to the CPUC.
These reports are due by July 15th of each year
and cover the previous year's data.
This reporting requirement does include division level
and historical performance and the utilities
to hold an annual workshop or town hall
on electric system reliability and make circuits levels
of reliability data available upon request to customers
within a reasonable amount of time.
Next slide please.
So annual reporting requirements as set up
in the 2016 decision were set up for the 50 utilities shown
and you can find annual reports dating back to 1977
as in the link provided above.
But just to run down the 50 utilities required
to report are PG&E, Southern California Edison,
San Diego Gas and Electric, Bear Valley, PacificCorp
and Liberty Utilities.
Next slide.
So this is where we get to dive into the data
which is exciting because the data can tell us a lot.
So how do the California utilities compare
to the rest of the United States?
Next slide please.
So to start with a little bit of a caveat here
is that reliability data can be shown in many ways
and each of these waves can tell a different
and sometimes conflicting part of this whole story
about utilities reliability performance.
These reports that are submitted annually
are multi-hundred page documents containing a lot of tables,
but a huge amount of data that needs to be parsed
and distilled to gain useful insights.
So this is just a little example of the difficulty
I said when sort of parsing these results.
So as you can see PG&E's 2019 system average
interruption duration index is displayed in full
without major event days and with major event days.
As you can see, there's almost a 12 fold increase
in the metric when the nature of my days included.
This is as compared to the 2018 reported
system average interruption duration index,
and then within that, or, you can see also here
this is the 2019 national average,
so PG&E without major event days compares favorably
to the national average,
but when major event days are added, they do not.
And then even within that data set,
you can see San Francisco division
system average interruption duration index
is far better than Humboldt which is currently
the worst performing one with the inclusion
of major event days that metric jumps to
just short of five days interruption.
So as you can see, these metrics are useful,
but they do paint a complicated picture
about what's really going on
with the electric systems within the utility territories.
Next slide please.
So from the 2019 days, we can take away some key facts
is that without major event days
the California utilities generally
have lower measured system average interruption
duration index
and system average interruption frequency index
than the national average.
This definitely means that
for the average customer on the entire utility system,
there were less outage minutes
and outages with less frequent than they are nationally.
Just as a note, the California utilities
have fairly large service territory
encompassing a large number of customers
which can influence these metrics
because at the system level they are averaged
over all of the customers on the system.
However, when we look
at the customer average interruption duration index,
the California utilities generally
have a higher measured CAIDI
than the national average for 2019.
So when an outage does occur,
the duration for the average customer
is longer than the national average.
Next slide, please.
So, here's just a quick breakdown
of how the California utilities compared
to the national average for each of these statistics.
If you can see any square in red is a place
where the California utility are higher
than national average metric
and you can see with the inclusion of major event days,
it does complicate the picture a little bit.
Again, as noted before you can see PG&E system
average duration index jumped from 120 minutes
without the inclusion of major event days
to almost 1400 minutes or almost a whole day
with the inclusion of major event date.
So you can see that these are very high impact,
they tend to be very high impacted.
And just as a note, the national average
is based on data that is annually compiled
by the United States Energy Information Administration.
They compile an annual report
and create a very handy spreadsheet with all
of the utilities that have reported in on these metrics
and it does include both include
and exclude major events days.
Next slide please.
So this is just a graphical representation
of what you saw on the previous slide.
This also does include the Southwest states
which are noted below.
So as you can see
without further system average international duration index
you can see without the inclusion of major event days
or the California utilities performed relatively well
as compared to national average
however with the inclusion of those major event days.
It complicates the picture a little bit.
Just as a heads up for the next few slides.
We'll be presenting graphs that look a little bit like this.
Do be aware that Y axis scale can change simply by virtue
of the variability of the data
especially you can see here PG*E metric jumping
into almost 1400 minutes of SAIDI.
So next slide, please.
So this is just another way to look at the data.
This is the system average interruption duration index
performance for PG&E from 2010 to 2019.
The yellow line shows the metrics
with major event data included
and you can see that from 2015 on
when PG&E started to institute
the public safety power shutoffs,
those metrics have jumped very significantly including
until 2019 when they expanded the scope
of their PSPS events.
So this is just illustrative of how these metrics
have changed over time,
and you can see that without the inclusion
of major event days, they trended relatively flat
as compared to when major event days are included
which is just interesting to note.
Next slide please.
So this graph, now we're looking at
the system average interruption frequency index.
This again is a similar way of showing
how the California utilities compared
to the bulk of Southwest states
and the rest of the United States.
So again, you can see that without the inclusion
of major event days the utilities tend to perform well
against the national average.
However, with the inclusion of major event days
you can see that it changes the picture a little bit.
There are a number of utilities that are performing worse
than those particular averages.
Next slide please.
And also for, again, the same
for the customer average interruption duration index
or CAIDI.
It's a bit, again, this again is just illustrative
that the California utilities when an outage
is occurring it does tend to be significantly
in some cases longer than the national average.
So this is just illustrative of what these outages
look like for the average customer and what happens
when you exclude and include those major event day.
Next slide.
So how granular are current reporting standards
and how do we keep track of the worst performing circuits?
So this is just a rundown of current reporting standards
and how are these persistently problematic started
or identified and targeted for remediation?
Next slide.
So, reliability metrics, reporting
on a granular level contains a really detailed picture
of the regional variation
within each of the overall metrics.
The 2016 decision did require utilities
to include these division level mattresses, metrics, sorry
in their annual reports.
You can use this data to show
where the recurring issues are happening
and where the areas of improvement are needed,
but it also shows how widely the metrics vary
over the entire service territory.
So the next slide is just a, is an illustration
of this showing every division within PGE performance
for the year of 2019.
So next slide, please.
So as you can see all the way over at the far left
in the San Francisco division is consistently
the best performing division within PGE service territory,
the outage durations do tend to be generally less
than 100 minutes even with the inclusion
of major event days.
(indistinct) up to the right at Humboldt,
you know, we can see that there's a very opposite problem
where with the inclusion of major event days
their own system average interruption duration index jumps
to almost 7,000 minutes, which is just short metric,
is just short of five days of the average outage.
So it's, these overall metrics are sort of an indicator
of how the system is performing,
but there's a lot of data hidden with them
and that's sort of this regional variation
that you can see illustrated here on the screen.
Next slide please.
The 2015 decision also requires the utilities
to report out on the top 1% of worst performing circuits
on the system by both
the system average interruption frequency index
of that circuit and also the system average duration index
of that circuit.
So these are the reporting requirements for each utility.
You can see that.
Next slide please.
So data is available from 2015, the 2016 decision required
that utility apply this worst performing circuit analysis
to their data from 2015 in the first year
and so this is just illustrative
that a lot of the worst performing circuits
in these reports are repeat offenders
and they've been on the list for multiple years.
So as you can see, PG&E 33%
of the worst performing circuits have been on that list
for more than, for three years or more.
Edison has 18%.
Liberty, it's a very small sample size,
but it's a little bit harder to extrapolate from that,
But as you can see, these circuits
are generally persistently performing poorly.
Next slide please.
So just as a note, you know, circuits do tend to appear
on these worst performing circuits list for multiple years
because remediation programs that target
the controllable factors can take years to complete
and they only achieve modest reduction
in the system average interruption frequency index
and system average interruption duration index.
However, uncontrollable factors
like environmental conditions do play a large role
in determining the reliability and some of these things
are beyond utilities control and can't be addressed
by traditional circuit remediation investments.
PG&E's 2019 report, they reported
that the Alleghany 1101 which is in Eastern California
was noted as deficient having appeared
on multiple worst performing circuits with
and have more spikes
in its system average interruption duration index in 2019
that was caused by a single unknown event coupled
with unacceptable mainline sections that were difficult
to inspect and difficult to get back online.
So, you know, unfortunately
these metrics reflect both those control factors
and uncontrollable factors within the circuit performance
and that's something that has to be,
perhaps to be born in mind as we look at these metrics.
So next slide please.
So this is from PG&E in 2019.
These are the numbers of sustained outages in PGE territory.
As you can see, well over 1/3 of them were company initiated
but almost no close to 1/5 or 1/4 also were unknown cause.
So while there are factors within the utilities control,
there are factors that are uncontrollable
and, you know, third party unknown cards.
So, it paints an interesting picture of these
and, you know, it's something
that we have to keep in mind as we look at
and use this data.
Next slide please.
So, this is the next step, what can we improve?
How can we better use this data
and what, you know, what directions can we go with it?
So next slide please.
So listed here are just some other possible usages
for the electrical reliability data that we envisioned
could be useful to look at to assess this performance.
So one of the big things, you know,
it's trying to be equity focused
and try to see unreliable service, you know,
are there equity impacts of unreliable service
by using the SB 535 disadvantaged communities maps
but, and CalEnviroScreen but also other equity focus tools
to assess whether there is an equity impact
of unreliable service.
You know, we can also track impacts and customer experience
during the public safety power shutoff events.
One interesting thing we can do is to track the threshold
for major event day to see if on the average
on the whole system outages are getting worse
and including, you know, including more data
that may have been excluded in previous reports.
There's also a look towards assessing prioritization
and siting of grid resilience solutions
to ensure least cost best fit solutions
and also capturing how climate change affects
the electrical system
and outage duration through outage causality.
Next slide please.
So, the next step that we envision.
So we have been working with utilities
to improve their current reporting
in their current reporting standards,
mostly within the bounds of that 2016 decisions
in terms of trying to make the data more usable
and one of the things that we've requested
is making sure that these spreadsheets
with all of the data are available
to make our analysis easier.
Additionally, we are undergoing on outreach and validation
to engage a diverse set of stakeholders
about the usefulness of the data
and also talking to utilities about expanding access
to reliable data at a more granular level
and also monitoring annual public reliability meetings
to see what the needs are for this data
and how we can better tailor this data provision to
and use it on some things in the future
that we would also like to see
are GIS format of data complete with historical metrics.
We would like to see,
you know, probably available circuit level data
to make sure that it's as granular as possible.
You know, we would like to see the reliability impact
of PSPS and other outage types,
but then those metrics and then also
to have utilities talk about mitigation measures
being taken to address those reliability issues,
you know, and a lot of it does overlap
with some currently ongoing efforts within the resiliency
and microgrid space including the value of resiliency
and resiliency of metrics for the local areas.
The recently voted out microgrids incentive program
to help ensure communities are getting the right data
they need to participate,
and also within PG$E regionalization proceedings
as a possible tracker of performance within area.
So, next slide.
So with that, I am open
to take any questions folks may have.
Thank you for taking the time to listen.
-: Thank you, Julian.
Excellent presentation.
A lot of acronyms, a lot of data all feeding
into pinpointing where attention is needed,
Looking at our disadvantaged communities,
low-income communities, seeing where as we navigate
through general rate cases
where resources may need to be focused.
So let me turn to my colleagues
to see if there are any comments or questions.
Marybel: I just wanted to say, this is Marybel,
sorry you can't see me, so I'm sorry can't wave my hand.
I just wanted to thank Justin
for a terrific presentation.
A lot of new, as you said new acronyms Commissioner Shiroma,
but excellent and I'm gonna have to get a copy
of the deck 'cause couldn't see all of the data
but interesting rich data for us to take advantage of
as we continued to plan and work with PSPS.
So thank you very much
and I'll turn it back to you Commissioner Shiroma.
-: Thank you.
Thank you.
Yes, Commissioner (indistinct)
and then Commissioner Guzman Osavis or Julian.
-: Thank you very much.
Thank you, Julian.
I have two, I have a question and a comment.
The comment is I fully support the idea of examining
the equity impacts of unreliable service using
the SB 535 mapping or other equity tools
and I just, I guess this is a question too,
are you, is that something you're considering
or is it something you're actually doing according
to a schedule and in any case I'd like
to encourage you to go ahead and do that
and report back to us publicly.
I think would be very, very interesting.
So let me stop there and then I'll ask the second question.
-: Yeah, absolutely, thank you for the comment
and also the question.
Currently this is something we are envisioning.
We would need to work with utilities
to see if we can get GIS format of this data
to ensure that that analysis is both,
it's easier, but also is it very robust to make sure
we are assessing those overlaps
and also, you know, seeing these equity impacts
and making sure we know where they are.
-: Okay, I would be interested in hearing back from you
whenever you've gotten a sense of what utility is having,
how your efforts could proceed in that area?
I'm sure the other Commissioners would as well.
The second question relates to major events
as the stark differences
between data with or without major events really grabbed me.
I assume, for example, what's happening in Texas
and the Midwest is a major event
as defined according to the data were using, right?
-: Yeah, that would be.
-: Yeah, you're nodding,
I don't know if everyone can see you,
but given that are the standard setting bodies
or others thinking about adjusting
the way we define major events.
You mentioned that as one of the potential bullets.
It seems like those should not be treated as outliers
given their frequency either because of human caused events
or otherwise as I understood your presentation
and that just doesn't distinguish
between major events that are caused by earthquakes
and natural disasters versus human induced events
like public safety power shutoff.
So I guess the question is,
is there a trend nationally to re-look at this?
-: So, that is correct.
That's, a lot of these metrics
are meant to be statistical representations of this data
and actually are sort of meant to take
the causality element out of,
and just the look at the event itself.
Currently, I don't believe there's an effort
to sort of undertake a re-look
at how major event days are defined,
but this is also crossing into thinking
about system resilience and thinking
about how resilient the system is
because these are large scale,
you know, large scale, high impact
and low frequency events.
And these statistics were designed
with the utilities in mind
to be able to take those low frequency,
low probability events out of consideration
so that they can improve just normal mainline service.
So within what we're looking at with resiliency
within our team and energy division, we are trying
to sort of think about how, you know, how we can use
the current major event day definition
to sort of give us an idea of what is the,
you know, pardon the train expression,
but what is the new normal.
-: Thank you.
-: All right, thank you.
-: We do have a new, we really do have a new normal.
So I think that's the right question.
I'm sorry, Commissioner Shiroma.
-: No, my apologies, absolutely right.
What we're seeing across the nation, my goodness.
Commissioner Guzman Osavis.
-: Yes, thank you, Commissioner Shiroma
and thank you, Julian and Tiana for all your work on this.
I know that we've talked before
from the San Joaquin Valley proceeding
where many of these really small population communities
never, given the formulas
you noted the denominator is always,
you know, population and so those metrics
will never farewell for remote communities with low density.
And when you look at it at a macro level that makes sense
because you wanna help as many people as possible of course,
but if you look at a regional level,
and just on an individual level perhaps arguably
through an environmental justice lens,
these metrics don't get up
the most harmed customers potentially.
And I wonder in the regionalization applications,
if there is an opportunity there
to really resolve this issue
and start to, it's not to replace these metrics,
but to actually get to a more micro level.
I've seen, you know, this chronic situation,
like you pointed out with Humboldt.
Same scenario there where these issues
are perpetuating themselves
because the metrics never get at them.
And so I hope that we can certainly continue
to use these macro level metrics certainly
to benefit the largest populations
and individual customers as possible.
But I don't see how continuing to rely on them alone
is going to get us
at some of these chronically disinvested communities.
And just to share with my fellow Commissioners,
one of the most concerning results of this
is as we're working on these pilots in these communities,
these 11 and 10 communities in the valley
to try to electrify them for their heating needs
for both their air quality health impacts
and climate benefits.
PG&E in particular, not at, PG&E is claiming
that increase in load is gonna require substantial increases
in investing in their distribution system
for the drops in particular,
and, you know, if we, it just kinda goes to show you
how bad the situation is
if people adding just a little bit of loads
which requires not much of an upgrade.
And so, anyhow, I guess, you know,
I certainly wanna see these metrics more
to guide our risk management decisions,
but also really hope that in the regionalization application
we can take a regional approach
and really start to get at examining, you know,
where these chronic issues are.
It's harder to justify perhaps
because of the lack of economies of scale
and the lack of actual customers that would benefit,
but it is very much an equity issue.
So I don't know if you have given that thought
if you were thinking about, is there a way to, you know,
like I said, continue to use this formula for macro-level,
but they're way to regionalization evolution
that's happening at PG&E to further grieve a new formula
or adjust the formula, you know, by county population.
Come up with some kind of even county level
or district level, I know they have the districts.
I don't know how that's gonna evolve
with the regionalization, but anyway,
I don't know if you have any thoughts now on that,
but I certainly hope you're thinking about that.
I'm certainly thinking about
how we can potentially improve on that situation.
Thank you.
-: Yes, absolutely.
Thank you, Commissioner for those comments.
You know, we definitely see these reliability metrics
as a tool within our toolbox
that we can pull out to do analysis on these, on,
you know, on smaller levels
and, you know, that is something
that we're working towards making sure that,
you know, this reliability that all the way down
to the circuit level is being,
you know, is being made available so that we can see that.
You know, I definitely appreciate the comments
about equity as well.
It's, you know, this is a really important issue
and this is also carrying over into work on resiliency
where we do really wanna make sure
we're being equity focused within our work
and within the way that we look at this data
and that we sort of use it to draw conclusions
and also figure out, you know,
what the assumptions are behind it
so that we can adjust accordingly.
So that's definitely high on the priority list
is ensuring that we are focusing on equity,
you know, in terms of how we're looking at these things.
So definitely appreciate them.
-: Thank you.
Thank you Julian.
Yeah, I think that Commissioner Guzman your point
about how the regionalization effort will work
for PG&E is going to be very important
and that we need to look at this very holistically
not with any of these metrics in life relations.
So I'm sure there'll be more to follow
as we navigate with PG&E this effort on regionalization.
President Batger, Commissioners, any other comments
before we go to our next speaker?
Okay, all right, not at all.
Amber: Commissioner Shiroma thank you.
-: Okay, all right, well, we have another presentation,
another perspective from Dr. Fiona Burlig,
who is an assistant professor
at the Harris School of Public Policy
and a faculty research fellow
at the National Bureau of Economic Research.
This is at the University of Chicago
and through the one of the benefits I suppose
of the work at home is we get to garner the expertise
from across the country.
So Professor Burlig is joining us from Chicago.
She has also partnered with professors
at the Haas School of Business
at the University of California at Berkeley
on economic research in the area
of electric reliability, and Dr. Burlig
will be presenting today on her research
on the value of reliability, more specifically
the economic impacts of electricity outages
in the context of our public safety power shut off.
Dr. Burlig holds a PhD in agricultural
and resource economics
from the University of California, Berkeley
and a BA bachelor's in economics, political science
in German from Williams College.
So welcome Dr. Burlig.
-: Great, thank you so much for having me.
So this is joint work as the Commissioner has mentioned
with Katherine Wolf from the Haas School of Business
at UC Berkeley and also Duncan Callaway and Will Gorman
who are in the energy and resources group at UC Berkeley.
So it's an interdisciplinary team.
Katherine and I are both economists,
and Duncan and Will are both engineers.
Next slide, please.
Great, so this is some preliminary work
that we're starting on to try and understand,
or to think about new ways of measuring the cost
and benefits of electricity reliability.
So the big questions we're trying to ask
in this research agenda are number one,
what are the economic impacts of electricity outages
and how can we measure them in a comprehensive way?
And then second, what are firms
and households willing to actually pay
to avoid future outages?
And the idea that we have in mind here
is that the answer to the second question
gives us an idea about the answer to the first question.
Right, if you present household or a firm with,
to spend a little bit of extra money on their bill
to avoid an outage that tells me something
about the kind of upper bound of the cost
of that outage to that firm.
And so we say that the second, the answer to this question
gives us an insight into the actual total costs
of the outages.
In the first through a reveal preference approach,
we can actually observe what decisions customers make
to learn something about how much
they might be willing to spend to adapt outages
or avoid damages related to outages,
and this is closely linked to the concept
of the value of loss load, right?
The value of loss load being how much a customer
would be willing to pay to avoid a particular outage.
Next slide please.
Okay, so it's just maybe unsurprising to this audience,
so we think it's really important to understand
what the value loss load actually is.
So in an ideal world, we would equate the economic benefits
of avoiding a particular outage
with the economic costs involved in doing so, right.
So understanding how costly these outages are
is really essential to understand
how much we should be sending to avoid those outages.
This is gonna be relevant to a large variety of decisions,
including the right level of grid hardening investment,
the right resource adequacy in a system
that has intermittent renewables
in the context of the California public safety power shutoff
thinking about the cost that outages impose on society
as compared to the costs of fires
that we're avoiding by imposing these PSPS events.
You can also think about,
you know, this week's events in Texas
is thinking about weatherization of power plants
and the electricity grid compared to the cost associated
with having outages during cold weather.
Next slide please.
Okay, so how can we actually get at this number
of how much society should be willing to pay
to avoid these outages?
There've been people who've tried to do this in the past.
We're not the first to attempt to do this,
but typically what's done
is to use what's called a stated preference survey.
So you basically interview a customer
and ask them about hypothetical scenarios
where you say, suppose that you were
to face this particular set of outages,
how much would you be willing to pay to avoid that?
The second thing that you can do
is what's called a revealed preference approach
which is actually trying to understand what people did
in response to outages.
The advantage of this approach
is that we see real responses
and so we avoid the probably customer might say one thing
in response to this hypothetical scenario,
but behave differently in reality.
The third thing you can do is try
and pull together macro economic data
and basically look at what's happened overall
in the economy in times with more versus your outages
and try and learn about sort of overall scenarios that way.
So what we're doing in this project
is bringing together new data on revealed preferences.
So I should say that this is fairly preliminary work.
I'm gonna show you at the beginning of this project today,
and then I'll describe a little bit
about where we're going as well.
So we're trying to do
is use this revealed preference approach.
It's based on actual decisions that customers are making
as an indicator of how costly power outages are to them.
What's nice about this is that
we can use large archival datasets to answer this question
and so we're not forced
to rely on really small samples surveys,
and secondly, we can also get data like I was saying
on what customers actually do
as opposed to what they say they do
where the latter reveals, oh sorry,
what customer actually does
is likely to reveal a lot more about
how they value these outages relative
to what they might pay today
(indistinct) when posted the hypothetical scenario.
Something we're really trying to be careful about
is actually estimating the causal impact of outages
on customer's decision making.
So we're not for example comparing a customer
that lives in San Francisco and has very few outages
with a customer that lives in Fresno
and has some potential outages.
There's many things that would differ
between those two customers beyond
just their average exposure
and so we wanna be cognizant of that
and making sure we're actually capturing
behavioral responses to outages themselves
not the other features of the environment.
And so to start this project, we actually did do a survey
that's gonna provide some additional color
and kind of informed the types of costs
that we think we should be looking for once we engage
in the archival data work.
Next slide, please.
So to give you have a sense
of how we're thinking about doing this,
we are leveraging the variation in outage experiences
that were driven by the 2019 PSPS events,
and what these events did is because outages
were basically imposed at the feeder level.
Here as you can see some feeder maps
from Berkeley and Oakland,
we can compare two customers
that live very close to each other,
but had wildly different outage experience.
You can see that yellow feeders
are feeders that didn't experience outages.
This is sort of from one of the PSPS days in 2019
whereas the purple feeders did,
and so you can imagine comparing very nearby neighbors
to one another one of whom had a full day
of power outage events and the other one
who had no power outage events.
The thing that's really useful about that
is it allows us to sweep away any other differences
between those two customers
other than their outage experience.
We can really get a good estimate
of the effect of the outage
rather than broader differences between customers.
Next slide, please.
So, we're currently working on things
to, sort of a whole suite of available archival data
including from the IOUs is thinking about
you sort of using that type of border analysis
comparing more effective versus once effective figures
to look at whether customers
are investing in diesel powered backup generators,
whether they're more or less likely
to purchase a solar PV or invest in storage.
But we also wanna make sure that we're thinking
about other economic costs as well.
So we're using data on mobility to understand
whether firms are being less visited
when they (mumbles).
Also looking at kind of business closures information
to see whether some of these PSPS events that are large
are actually causing small businesses to close
and then a major potential cost as well
in terms of public health. And so we wanna make sure
that we're also capturing information on customers
that have medical devices
that require electricity to operate.
For example, so we're trying to pull in data
from the Medicare and Medicaid systems
to also understand whether we see increased hospitalizations
among outage affected populations.
Next slide, please.
To kind of inform and as a starting point,
we have employed some Intrepid undergrads
from UC Berkeley to run a survey
was around 1,000 households.
We targeted this survey to be with households
that we're kind of close to these outage boundaries.
So we'll have some households that did experience outages
and others that didn't,
and what's nice about being able to do with surveys,
we actually can collect some information
on household demographics.
We can ask households
what their outage exposure actually was.
We can ask them what they did in response to the outages
and we can also try and understand
what they think outages
are going to look like in the future.
Next slide, please.
Lots of really fascinating and sometimes challenging
to listen to stories from customers
who to experience outages.
So we see a range of costs that customers experience
as a result of the PSPS events.
So all the way from customers
who had some lifesaving medical equipment
that was required and that the outage events prevented them
from being able to use that,
and so taking substantial avoidance measures.
So for example, one customer said she spent $400
on a hotel room so that she could continue
to use her CPAP machine and supply heat
and water to herself.
We then also had kind off more,
what you might think of as more mundane costs
like losing refrigerated commodities
like condiments and meat went out to eat
instead of cooking at home.
Lost power to electronic gym devices
and then, you know, we're all facing the consequences
of additional childcare today, right?
So we also saw that without access
to electricity schools were sending children home
and so that impose additional burdens on parents as well.
So these are a flavor of the types of things
that we're trying to measure
and the idea with our revealed preferences investments
that households take to help us and firms take
to avoid outages should capture
at least some of these costs.
Next slide please.
Okay, so one of the other things that we did
is just ask households can you tell us something about
how costly these outage events were for you?
And we see that among the households
that didn't experience an outage, reasonably
they report no cost.
That's good.
That suggests that people are doing a decent job
with this calculation but households
that did experience outages,
there's definitely a range of costs here.
So more than 30% of households in our survey
who experienced an outage reported
having total costs of more than $100 and some,
you know, in this right tail here of upwards of $5,000.
So there's a wide range here of costs
that households report experiencing.
Next slide.
One other thing we're really interested in
is ways that households adapt themselves
to these PSPS events.
So one thing we asked about was types of backup power
that households were potentially operating
during their outage event.
So this is a graph showing households
that did experience outages,
what share of them engaged in different adaptation behavior.
A very small flights for solar power.
That makes sense.
You need a smart inverter to actually be able to continue
to run your solar panels during an outage event,
but we were sort of interested to see that
around 30% of the households in our sample
that experienced an outage actually already own a generator
and operated that as a way of increasing
their own resiliency.
So that was the main way that customers sort of responded
to these outages.
Next slide.
Another thing we're then interested in
is what are customers doing sort of
since the outages happened to change potentially
their ability to respond to these outages in the future?
So we're seeing households, they're investing substantially
in these diesel powered generators,
and one thing that we're actually seeing is
it looks like there may be a small trade off
between investing in these gen sets
and investing in solar power.
That's something we're still investigating
and are interested in looking towards
the administrative data to tease out further
whether this is a large-scale trend
or something particular to our sample,
but at least an interesting feature
that yes customers do seem to be investing
in additional reliability in the form of generators here.
Next slide.
Finally, customers who purchased generators seem
to be investing in relatively large scale ones.
So you can see again households
that didn't experience outages.
So some of them purchased generators as well
that you might imagine makes sense
because even if I didn't have a PSPS event this year,
maybe I expect to have one next year,
but among the households that did experience outage events,
we see them sort of purchasing larger generators
than the ones that didn't experience outages.
Next slide.
So all of to say we learned I think a few different things
from the survey.
One is that as we were trying to figure out
who we should be surveying,
we were trying to target households based
on pretty preliminary maps
that PG$E, Southern California Edison
had released aware the PSPS events had taken place.
Those were very rough boundaries,
and so we actually ended up targeting more households
that didn't experience outages
than we had originally planned
'cause it's hard to figure out from those early maps
who actually had outages, newer data that's come out
that's actually a feeder or a circuit level
makes it much easier to understand
who these customers are.
Another important caveat though
is that it's really, really hard to get a good response rate
to a survey in California.
So we had about 5% response rate.
To put this in a little bit of context, Katherine and I
both work a lot in developing country contexts.
So Katherine works in India
and there we're used to response rates of 75% or higher.
So it's important to take these results
with a bit of a grain of salt
'cause it's hard to get people to answer the phone,
and it might be that someone who is particularly excited
to talk to a UC Berkeley undergrad
about their outage experience is not the average person
which is why we think it's also important
to bring in this administrative data.
But what we've learned so far at least
in this sample population is one,
many people actually own generators already
which we were pretty surprised to see.
There are some suggested evidence that there's a trade off
between purchasing solar and purchasing a generator.
Neighbors who didn't themselves experience an outage
may still be responding to the threat of future outages,
so households they're making investments
even if they didn't experience an outage themselves,
and we do seem to find evidence
that households are buying relatively large generators.
Like I said, this is preliminary work.
We have lots left to do.
We're really excited to start diving
into the administrative data as well,
and I should just put in a really brief plug
that if you're interested in what we're doing here
as well as much of the other work
that's coming out of the energy institute at Haas,
we're hosting a conference,
the annual power conference virtually
from March 16th to 19th.
So please go to the website
for registration details for that.
Thanks so much.
Next slide.
Great, my email so feel free to get in touch with me
if you have any future questions.
We don't have time to get to today.
-: Thank you Professor Burlig.
What was the timeframe of this survey?
Was this?
-: Yeah, so we conducted this survey
between the end of December and late March of 2020.
-: On 2020, yeah, yeah.
And in the meantime for the PSPS,
we have ordered PG&E and the other utilities
to do a much more aggressive effort
for medical baseline customers,
people who rely on CPAP and so forth
in terms of battery deployments and what have you.
So are you continuing to survey customers
or are you on to data analysis?
-: Yeah, so we're planning a second round survey now.
I think we're gonna deploy slightly differently,
but we are conducting a second round of surveys
and we're focused on counting customers expectations
about future outages as well
and understanding whether that's really different
between customers that did and didn't have PSPS events.
We've just collected information
from PG&E on household level outage exposure
using smart meter data, and that gives us an opportunity
to get a really granular look
into who experienced what outage.
So that's kind of first on the table right now,
we have the data from 2019,
but I think we would also be excited about extending that
into the future as needed and become available as well.
-: Okay, all right, questions or comments.
Present Batjer I'll start with you.
Batjer: Thank you, Commissioner Shiroma.
Just a quick clarification again, doctor thank you very much
for the presentation, fascinating, important.
The 5% reply that you did actually get, yes,
that's very low, but the response rate,
but what does that equate to in terms of numbers?
I think you started with, did you say 1,000?
So it's 5% of 1,000?
-: No, so the 1,000 is the total.
So we conducted full surveys with around 1,000 households.
-: Okay, so you had 1,000 responses?
-: I mean 1,000 responses, yes.
That's a better way of saying that, thank you.
-: All right, okay, thank you.
And just a follow up question there,
and those 1,000 responses,
were they representative of, let me start over
that's not quite.
Were those 1,000 customers in that area of the map
that you showed us from the PSPS of 2019
that was in the Oakland Berkeley area of where,
is that the population, is that the area?
-: Let me, yeah, let me clarify that.
So the population that we sampled
is actually all over the state of California.
So we identified areas that we thought
were likely to include boundaries
between PSPS affected and non-PSPS affected customers.
So that's Berkeley Oakland example is a nice illustration.
There are definitely some customers there in our sample
though we include much more of the state as well.
So we did our best to be representative
in terms of location.
However, we're definitely not representative
in terms of demographics.
So the (indistinct) sample who's answering the phone
is upwards of 55 years old
which is definitely not the median kind of household age
in California for instance.
Bbatjer: Right, and these
were landline calls I suspect, correct?
-: That's a good question.
I think it's actually a mix of cells and landmines,
but I would need to double-check.
Batjer: That gives a sense of the demographics as well.
And clearly this wasn't just, as you just indicated,
it was not just PG&E customers if it was Statewide.
So you were surveying PSPS customers
whether they lived in Edison's territory, PG&E's territory,
or San Diego Gas and Electric.
-: Yes, correct.
The majority are definitely in PG&E
which matches the pattern of the PSPS events themselves,
but yes, we talked to Southern California customers as well.
Bartjer: Okay, thank you for that clarification.
-: Okay, colleagues, Commissioner (indistinct)
and then Commissioner Guzman.
-: Thank you, Dr. Burlig.
This is very important work because we know intuitively
and from the many comments we hear
and stakeholder inputs we've gotten
that PSPS has imposed very real costs
and it's hard to quantify them.
So your research is very important.
I have got like a comment or question
without changing the underlying philosophy
of the national Bureau of economic research.
I know am doing a survey done by economists,
but they do wanna remind us of some
of the limitations of just using economic measures
including revealed preferences.
They don't tell the whole story of health impacts
which of course can be documented.
I don't know if you've been able to get some of those,
but what, if the increases in certain kinds of illnesses
as a result of PSPS people not being able
to take their medicine or be on the CPAP machine admitted
to hospitals and so forth.
So that's one issue that could be quantified potentially.
There are unquantifiable damages,
not being able to go to school, stress,
not being able to use your CPAP all day long,
all things like that, they're hard
to capture using traditional economic metrics.
And I also, maybe you could comment on this,
just the equity implications of using a measure like this.
If you use a reveal preference to pay,
if you base it on how much people
will spend on a generator or solar panel,
what does that say about people who feel just as intensely
about the impacts, but can't afford to buy a generator
because they're lower moderate income
or likewise they may have suffered five or six outages.
We saw Humboldt, people in Humboldt
experience a hundred times the level
of disconnection interruptions on a PG&E
than someone in San Francisco,
they can only buy one generator in theory
or one replacement.
So I don't wanna, I know I said,
this could be an academic discussion of much greater length
and I think very much valued the work you're doing.
Maybe you could comment on how you deal
with some of the equity and intangible impact
without trying to defend something
that you're not pretending that your work is.
-: Professor, I think you're muted.
-: Yes, sorry.
So yes, this is completely valid
and really useful context to put our study in.
I think a couple of quick responses.
So first we're trying to do our best to quantify
as many of the quantifiable features as we can, right?
So getting this administrative data
on Medicaid utilization, for example
will help us get at health metrics even for customers
that are not necessarily super high income.
I totally hear the question about a customer.
There's a difference between willingness to pay
and ability to pay, right, and it's important
that we keep that in mind.
One way that we're thinking about addressing that issue
is looking at whether our willingness to pay metrics vary
across demographic groups.
So whether we see households
in lower income communities displaying lower willingness
to pay, that doesn't necessarily mean
that they care less about avoiding those outages, of course,
but might actually indicate lower ability to pay as well.
So point completely well taken.
It's something we're thinking hard
about the right way to resolve
and I guess I would say is the reason
that we're trying to combine both the spending measures
as well as other things that we can observe
about how households and firms were affected.
-: Thank you, Dr. Burlig.
-: Thank you.
Commissioner Guzman Osavis.
-: Thank you, yeah, I just kind of along those lines,
I would appreciate that this is
kind of that macro level question
and your question is,
now we have the ability to pay or willingness to pay
for these mitigating things such as,
by the way I did have a question,
are you asking about batteries or just generally
the diesel generators?
You're asking about battery, okay.
-: Well, so the majority of the questions
are about generators, there is a small face
to talk a little bit about batteries there,
but we're collecting more detailed information on,
or we collected more detailed information on generators
in the first survey.
-: And so your major question is to say,
you know, cumulatively we see that using this multiplier,
we can assume that rent payers
would be willing to pay another $2 billion
for hardening and, that's the question, right, okay.
So I guess the other question is really kind of the inverse.
I wonder if there's an opportunity to see,
you know, what is the, you know,
we have a lot of these discussions that we had this morning,
I'm trying to figure out where is the right investment?
What does that investment look like?
And I wonder, do you think there's anything
that'll come out of this that will inform
that question on how to prioritize those investments
on the regional scale in particular?
And because, you know, you're right
the courts we're not dealing with,
you said you don't have any demographic data,
there's no income kind of qualifier.
So (mumbles) kind of a bar of what the utility
potentially (mumbles) not to then (indistinct).
(indistinct)
-: Yeah, so I think one of the virtues of moving past
the survey data and into the administrative data
is then we're really looking at household level information
and circuit level data on outages,
and so we can really granularly say things
about even what the allergy exposure picture
really looks like, and so I think one thing
we're interested in doing is really digging into those.
I mean, so the data we're collecting
at the moment for example,
is information on a representative sample now
of 10% of utility customers,
and so that allows you to paint a much more granular picture
of what's going on in different locations,
and I think we're really excited to start digging
into the administrative data partly for that reason.
-: Yeah, I mean, I envision that could be helpful
for like greater location or prioritization
of community centers and things like that.
Okay, final question.
Did you include any of the tribal areas in or traveling,
I know some obviously traveling folks
still move on reservations,
but did you include actual reservations
in your survey?
-: So, I would need to go back and check and see
whether we actually got responses from those areas.
We didn't sample specifically to target tribal regions.
So targeting again was just based on likelihood
of living within a PSPS boundary areas,
so to the extent that that was the case
and we had customers who answered the phone.
It's possible but I don't know.
-: Okay, yeah, that'd be interesting
because I think there were some areas
that met that mark where it was kind of cut in half
and just to see if there's anything unique
you find out about that.
Well, thank you very much.
-: Yes, Mr. (indistinct).
Thank you.
-: Dr. Burlig, you may have addressed this
and I just wanna clarify, but you indicated
that you're analyzing outages
and non-outages between folks
in a relatively close proximity to make comparisons
between their responses.
Did you say the assumption is that the folks
who live relatively close by
have the same relative socioeconomic conditions?
I just wanted to clarify that.
-: Yeah, exactly.
So what you think about is
we're simulating basically an experiment
where we compare across the street neighbors.
One of whom happens to live on this side of the street
where the feeder was put under outage.
The other home is on the other side of the street
and therefore this year wasn't put on her outage.
So one thing that we're trying to be careful to do
is making sure that when we're comparing
two neighboring feeders,
those neighboring, sorry, those customers living
in those neighboring feeders
do actually exhibit similar characteristics.
So we're using for example, data from the census
to look at how median incomes change
across these feeder boundaries.
-: Got it, thank you.
-: Okay, well, thank you.
Thank you Professor Burlig and at this point
we are going to check with our operator
to see if there are any public comments.
Robert, do we have any public comments?
Robert: Commissioner I don't see any public comments
at this time.
-: Okay, all right, well thank you.
Thank you to our operator, Amber.
So many thanks to Professor Fiona Burlig,
many thanks to our CPUC utilities engineer (indistinct)
Julian Enis, for two very insightful
and outstanding presentations.
You've given us a lot of food for thought for sure.
More followups to be had by us.
I'm gonna now turn the microphone over to President Batjer.
Batjer: Thank you Commissioner Shiroma,
and thank you also Commissioner Guzman Osavis
for bringing this very interesting and informative.
As you said, a lot of food for thought.
Dr. Burlig and Julian, thank you so much
for your presentations and for all of the hard work
that those presentations represents.
I really think everyone who did join us today
and with no public comments, this meeting is now adjourned.
Thank you all and have a safe day.
-: Thank you.