Settling the Risk Quantification Debate
Different analysis methods are better suited at each phase, with the greatest concentration of quantification tools needed during…
Top 3 Takeaways
ALEXEI SIDORENKO: Risk managers, if they are willing, have the most amazing tools at their disposal to really change how organizations land, forecast, budget, and make decisions.
HOST KELLEY SPAKOWSKI: Hi, I'm Kelley Spakowski and this is GRC and Me, a podcast where I interview industry thought leaders in governance, risk, and compliance on hot topics, industry specific challenges, trends, and more to learn about their methods and solutions and outlook in the space.
Today, I have with me Alex Sidorenko calling in from Catalonia, Spain. Alex is an expert in risk with over 14 years of risk management experience in private equity, sovereign funds, investment authorities, and venture capital firms across Australia, Russia, Oman, Poland and Kazakhstan. In 2014, Alex was named the risk manager of the year by the Russian Risk Management Association. Alex is currently director of Risk-Academy. Alex, welcome. Would you like to explain to us a little bit more about what the Risk-Academy provides?
AS: Hi, Kelley. Thank you so much for having me. It's a great pleasure to speak to your audience.
KS: Thank you.
AS: Risk-Academy is a fun story. It started many years ago when I was still head of risk of one of the biggest venture capital funds in Russia, and I had this internal weird desire to share everything I do. So I created an online portal in Russian at the time, which shared all the templates, the methodologies, video recordings from the conferences, master class, so basically everything I did outside of my job. And that kind of continued as I later moved as the head of risk of one of the biggest sovereign funds in Russian. So the Risk-Academy continued and it's now the biggest risk management brand in the Russian speaking world.
But about three years ago, I moved most of my IP in English and now Risk-Academy is the place where I write a lot of articles, where I do a lot of videos and I provide a lot of training sessions for either risk managers or the decision makers that want to apply risk tools to better make investment, or strategic, or operational budget decisions. So Risk-Academy, essentially, is the place to learn about risk management, but also, a consulting house that does a lot of work when clients request for it.
KS: Close Wonderful. And I understand you have podcast episodes as well.
AS: I do. I do. I think, as of as of today, I have like 300 plus articles, 400 plus videos, three books and something like to a hundred different podcast episodes.
KS: Wonderful. What a great resource. Okay. How did you get into risk?
AS: That's, I guess, a pretty typical student story. As many people in their early twenties, I had no idea what I wanted to do. So my dad was doing a PhD in chemical engineering at one of the best universities in Australia at the time. And he said, "My university has just started this new and exciting degree in risk management. Why don't you try it?" And just like any young student, I said, "Dad, I don't care, so I'll do it." And I signed up for this degree and it was-
KS: That's great.
AS: ... very hilarious because later, once we finished, we were the first ever undergraduate intake for the risk management degree. So we were the first ever bachelor's of risk management in the country in Australia. And it was kind of like, finally, because the university canceled that degree a few years later. So we were the guinea pigs in a failed experiment and I think the experiment failed because risk management is not really a profession. It's a competency that should be part of most degrees if not all the degrees at university. So it was it wasn't exciting start of the career realizing that you [inaudible 00:04:15] and the market doesn't really appreciate you being there.
KS: Yeah, that's fascinating. And this was in Australia you said at Monash?
AS: Good. Monash University. Yeah. And it's by pure chance that I have a second degree in statistics because I was so good at statistics. The faculty for statistics and econometrics kept sending me letters. Do a second degree in statistics. Do second degree in statistics. So I kept ignoring them for a year and then on the second year I, again, got really good marks for stats and I decided to do a second degree in statistics. I mean, thank God. Who knew risk management is actually about math as much as it is about everything else.
KS: Yeah, I was going to say those really go hand in hand so I'm sure that serves you quite well.
KS: So one of the things that I really like about the content that you put out there, especially your blog, is that you claim to be the most controversial risk blog. What makes it controversial and why is that your goal?
AS: That's a really good question and I mean I can't really say I have thought about it a lot. I think somebody said that it was controversial and somebody on Linkedin kept calling me controversial Alex or something. So I just kind of [inaudible 00:05:25] said, "Fine, I'll go with it." It wasn't necessarily the intention, but I guess the general observation ... I mean now in the age of social media, we, all of us, we can track engagements, we can track a lot of statistics on how certain messages get better or worse received by the audience. And I've been saying pretty much exactly the same thing for the last maybe 10 or 12 years. And the first seven years out of that last 10 have been pretty uneventful. I've been saying exactly the same message, but it just really wasn't widespread. It wasn't received as well.
And I think when I kind of got sick of it and I started challenging and questioning some of the norms or some of the accepted practices in risk management and exposing some of the silly things that we do as risk professionals, still do, then I immediately ... It was really a no brainer.
The engagement skyrocketed compared to your average mile, the friends [inaudible 00:06:29] everyone [inaudible 00:06:29] nice position. In the age we live now, I don't know why that happening, but it just makes perfect economic sense to do it, to be more controversial than not. I received dozens of messages every day saying whether they like or don't like something. Most people are still pretty shy so they send personal messages instead of just commenting under the article. And for every one message that I get from people who hate the format or the form in which I communicate and they find it insulting or they find it too controversial or too difficult to absorb, for every one person who dislikes the approach I take, I get like 10 or more people saying, "Thank you. Finally, we heard the messaging. It will help us get the message across.
So even in that regard, even though a small ... and it's, it's by far the minority, small percentage of people very much dislike the way I present information. Majority of the people find this helpful. So again, even in this regard it works and everybody who claims it would have been so much better if I chose a much milder form of communication, that's actually not true. I mean, I've tried different formats and statistically speaking, this is by far the most engaging one. I guess in 2019, it's nice to be controversial. That's all I can say.
KS: I agree with you. I like it. I think that risk and compliance right now is really ripe for some disruption and challenging status quo. So I think it's great and I just was curious, so thank you. Something else that I really like about your approach and I agree with and I want to really highlight on GRC and Me is that you encourage using risk for strategy. So what methodology do you recommend? How is it applied to strategy and then why is that important?
AS: Yep. It's kind of funny. I was just writing another article on that topic just before we got our call. If we ask how old is risk management, people usually will divide into two kinds of camps. One will say risk management is ancient because that's what people did when they were building pyramids. And then the kind of camp, people would say risk management is relatively new. In the 70s and 80s, that's when the whole concept of GRC and ERM became more prominent.
Well, in reality, I mean both of those groups are wrong because the modern day practice or theory of risk management and the science behind mismanagement, really started in kind of in 16th and 17th century when some of the mathematicians starting to quantify uncertainty and using mathematics to help them make decisions about future. These are strategic decisions, better investment decisions, better operational decisions. It doesn't really matter that much. The whole idea of using mathematics to make sense of uncertainty, which is highly complex, unpredictable by definition, is about 16th, 17th century.
And we've kind of, we've lived with that science of risk management and then it was first called probability theory. Then, in the early 20th century, it kind of developed and evolved into decision science, and by the 1970s, some of the psychologists kind of jumped on board and we had neuroeconomics developed on top of it. So in probability theory with decision science, with neuroeconomics of risks, psychology. All of that kinds of merge and by 1970s, we had what was a pretty solid foundation for risk management. But then in the 80s, a miracle happened because I guess that's what usually happens when something is very interesting but highly complex. Somebody hijacked it and really decided to dumb it down to make it, I guess appealing and relevant for the majority of the people.
It's like astronomy existed for so long, but that was too complex and it was highly mathematical to comprehend. So somebody came up with astrology. Astrology is basically your fairytale ... I mean, fairytale. It's not real science. It's basically BS and that's what most of the modern day risk management theories are. They're basically astrology of proper decision science. And so what I've been supposedly controversial about in my articles and in my work, I'm trying to bring the risk community back to almost like 1970s saying, "Well, we've had all the good tools and we had all the science behind proper risk based decision making for ages and we don't really have to recreate the wheel and the existence of the new artificial intelligence or cyber risks doesn't really change much. I mean the math is still the same."
All I've been trying to do is say, "Well, if you want to make a strategic decision or an investment decision, or a budget decision for that [inaudible 00:11:49] , well we actually have all the methodologies we need to do that and if we want to make a choice between different strategic alternatives, so decision trees are still as powerful as they ever were.
In fact, decision trees still drive a large portion of artificial intelligence algorithms out there together with neural networks. Both tools are old. And so for strategy, integrating risk management that the strategy just, it makes perfect sense, but it's not unique or new in any way. I mean this is what risk management was always about in 15th century, in 16th century, in 17th century, 18th century, 19th and 20th. I think it's unreasonable for us in the 21st century to consider it somehow an innovation. But to integrate into strategic planning, we still use decision trees, which are old. We still use scenarios and simulations.
And simulations, the multicolor engine for a algorithm for stimulating the possible future outcomes has been developed in '45, '46 so that's what, 75, almost, years old and as powerful as it was when it was developed to create the atom bomb or nuclear weapons. So I obviously think risk management is important, not just in strategy but in any kind of decision making because it just makes so much sense to do risk analysis, not once a quarter as we're used to doing it once a year, once every six months, but actually do the risk analysis before an important decision is being made by the management or the investment committee or whoever or as personally in life. And the good news is, we have all the tools necessary to conduct risk analysis before making decisions.
KS: That's the old, if it ain't broke, don't fix it, methodology.
AS: Well, kind of, which is weird because I mean I'd love to see risk management associations and risk consulting powerhouses come up with better ways to apply those existing tools. But that's not unfortunately what they do, which kind of forces me to be controversial. That's not what they do. They recreate the wheel instead and ironically, they don't give us better tools. They actually give us tools that are significantly worse. And, again, we know for a fact that the methodologies that are commonplace in risk management right now, for example, using heat maps for trying to multiply likelihood by consequence and getting like a risk level. We know for a fact that those methodologies provide much poorer results. They're much less accurate than any of the like 70 year old tools, which is bizarre.
KS: Yeah. Do you think it's the human element that is breaking down the mathematics of it?
AS: It's difficult to say. I think it's more the kind of the entrepreneurship spirit because whoever's made this popular and the Douglas Hubbard is, in his book ... He's publishing he another book on risk management very time soon, which I'm really looking forward to. But in his book, he actually went on a quest to find the patient zero, find that person who hijack risk management and turned it into astrology.
KS: I want to read that too.
AS: Yeah, I know. Isn't it exciting. It's amazing. Somebody seriously hijacked risk mismanagement in 1980s and turns it into literally astrology. Your average risk management report is no different from a horoscope. It's just as accurate and it's just as dangerous to use for any kind of proper decision making. So on one sense, it's horrible that this happened, but then on the other sense you can kind of understand how entrepreneurship and making money is that carrot that's dangling in front of the people because decision science, and math, and cognitive biases, and risk perception, it's hard. It's difficult.
That's why they have whole departments in Pentagon. And that's why CIA spends millions on researching this. It's actually really difficult. And somebody had a brilliant idea, why didn't I dumb it down for everyone? Well, I'm going to lose all of the important information and it will become a horoscope, but I'll be able to sell millions of copies and that's literally ... I mean, astrology, if you think about it, it's hugely popular. I mean, people are making ridiculous amount of money on horoscopes and everything else. So from a commercial point of view, makes total sense. From an ethical point of view, very questionable practice.
KS: Sure. Now I'm curious, what's your astrological sign?
AS: I think because when I was born, I was Aries, but remember how they moved the whole thing-
KS: Oh yeah.
AS: ... years back.
KS: That was not accurate. That was disproven. But to your point, it's all hokey, fakery anyway.
KS: But now the controversial title makes sense because Aries is the bull, so.
AS: Yeah, I guess.
KS: Okay. So something that I keep hearing a lot lately and it's not a new methodology, but it's just really coming full circle now I think is risk quantification. It's been really hot. So why do you think that is and when should it be considered and then how do you recommend organizations approach risk quantification?
AS: Yeah, which is fascinating because risk quantification can literally mean like a million different things. From very simplistic scoring methodology, saying, "Look outside. If the sky is blue, then it's going to be a good day," to like a complex Monte Carlo simulation model that runs like 10,000 scenarios, trying to figure out what the possible range of outcomes is, so highly broad, complex.
The good news is that almost everything we do is some form of quantification, and over the last 50 years a number of scientists have done a lot of research trying to figure out, trying to answer the age old question who's better, a human or an algorithm. And unless something new comes up in the near future ... at the last conference, the portability management conference in U.S. ... I think it was again Douglas Hubbard who was sharing the stats, but there was approximately 150 studies in different fields of life conducted to determine who's better human intuition or some sort of algorithm, some sort of quantification, even the most basic quantification. And out of the 150 ... I mean, I may be wrong. It could be 130 and it doesn't matter. You get the volume of research. It's been extensive.
And out of the 150, two studies have shown that human intuition is similar or slightly better than an algorithm and everything else. So 148 showed that actually using some sort of algorithm is much better than relying on our intuition. If we look at the situation from that perspective ... and all your listeners, you're more than welcome to disprove that. Run your own study, prove us otherwise, but until then we have to rely on ... And it's a large population of scientists that tried to answer that question and the study covers all different fields: agriculture, pharmacy, engineering, oil and gas, governments. It's like it's a very broad spectrum of study. So from that perspective, it's a no brainer. You have to quantify everything if it's a big enough decision. If it's significant enough and it's not trivial, if it's going to cost reputation or money, then quantification is definitely the way to go because the alternative to quantification is to not use anything and rely on your intuition, which is a pretty dangerous bet, it seems, based on the research I conducted as of as of now.
So quantifying is kind of the only way it seems. The real question is how complex can we go? How complex do we need to go? And here again, there are different schools of thought. For example, the school of thought by Daniel Kahneman and Thomas [inaudible 00:20:47], Vernon Smith, all the oldest researchers in cognitive biases and human risk perception and risk psychology say that because we are inherently irrational in our decision making, because we fall into so many different cognitive biases, because the quality of our decision making depends on how much glucose or sugar we have in our blood, whether we are tired or not of whether we're happy or not, what colors we're wearing, because we're so highly influenced by all these many different random factors, we really have to apply proper [inaudible 00:21:25] that quantitative tools to help us make decisions.
And then of course there's the school of thought by [inaudible 00:21:32] who says that no model can really be predictive of the future because the future is highly complex and the [inaudible 00:21:40] are hidden from our comprehension that we have to use the models, but we also have to have downside protection no matter what. So basically those schools of thought still say we have to apply some sort of tools. And the good news is sometimes even a little quantification improves the quality of decision making significantly. We don't actually have to run highly complex Monte Carlo simulation to make a better decision. Sometimes even adding a little bit of analysis to our decision making ... I mean sometimes even extrapolating the future. Sometimes creating a scoring model based on a number of the factual, observable items can significantly improve the decision making. And Douglas Hubbard has an amazing book called How to Quantify or How to Measure Anything.
He argues that sometimes three observations is enough to improve decision making, not to make the decision making perfect, which is not our objective but to improve our decision making compared to just intuitive thinking. [inaudible 00:22:51] with quantification has actually been hot since 15th century but now kind of everybody's finally getting the hang of it because I think part of it is that most organizations have been disillusioned with that astrology version of risk management with DRM style discussions, realizing that having a heat map of your strategic risks doesn't actually change how you budget or make multi-billion dollar [inaudible 00:23:19]. We've had all the tools to help us make better decisions. So how do I recommend organizations approach with quantification and when should it be considered? I think it's the first point to realize is that quantitative risk analysis tools is actually like a whole spectrum of tools starting from very simple decision trees, which you could draw on a napkin to scoring methodologies, which again are relatively simple two scenario analysis, which is super basic to more complex simulation models, which are slightly more complicated.
But if the price is high enough, if the reputational damage is significant enough, then running a simulation, even though it's complicated it's not a deal breaker. It's not that difficult. It takes maybe an extra day to run the simulation and maybe like an extra week to find all the necessary assumptions and verify assumptions and actually create the model. I think a week to like a multi-million dollar decision or a multi-billion dollar decision is literally the least of the troubles that you have.
KS: Right, exactly. Thank you. I think that's really insightful. Even just considering three objective opinions is better than one or two. I think that's really helpful information for the audience. It kind of reminds me too of ... Did you ever see that movie Moneyball, was about Billy Bean and how he uses saber metric analysis on-
AS: Yeah. exactly.
KS: ...Baseball players scouting? Yeah.
AS: Yeah, he-
KS: It's a little bit like that.
AS: Absolutely. I mean he was the first in the industry, in the sport industry. He used very simple math to significantly improved decision making.
KS: Yeah, that's fascinating.
AS: Which is essentially what risk management is all about. I mean whatever industry we are working on, we can use some of the basic risk management tools that we have and significantly improve the quality of decision making that executives have. I mean this is just mind blowing stuff and it's mind blowing because this theory has been proved by the Danish mathematician in 1906. It's called Jensen's inequality.
This is groundbreaking because most executives still have no idea and this is how business operates, ignoring that finding. Can you just imagine this is not 110 ... 1906 so 110 years ago a mathematician proved that when you build your business plans, or budgets, or investment proposals, or literally anything else, or production forecasts or anything, well, sales forecast, when you build anything in business based on single point estimates, especially if those single point estimates are what people call most likely or averages, you're pretty much guaranteed to have an unrealistic result.
This is how everyone does it in business right now. He took this idea and made it very popular in his book Law of Averages, which is amazing, but he's basically saying if your company is planning, and budgeting, and forecast using averages, which is what every single company on the planet does, things you signed off on that budget or that business plan or that strategy, you're pretty much guaranteed to have an unrealistic target because you've just taken out all of the uncertainty out of the equation and you've created this unrealistic fairytale.
And so of course whenever we're working with our clients ... we're the training producers for some of the biggest corporations in Russian speaking countries. Whenever we talk to them we're saying, "Well we've had since the dawn of time, since the 16th century, we had the solution." What's the alternative to planning and budgeting and forecasting with single points estimates?
Well, of course we can do that with ranges. We have the techniques to create business plans with ranges that will give you distributions with single point forecast. Basically what they're saying is that the way business planning is happening right now in 99.9% of companies in the world, pretty much guarantees rational results because it ignores risk and we actually have the solutions. We've had it for at least 70 years to overcome that, to improve our planning, which is fascinating. I think risk managers, if they are willing, have the most amazing tools at their disposal to really change how organizations plan, forecast, budget, and make decisions.
KS: Yeah. Fascinating. And I totally agree. So this is actually a really good segue because I'm curious about, you have such worldly experience in risk management and I have to think that just based off of our conversation even so far, you mentioned how our humanity kind of plays a role in our gut decision making and we've got dopamine firing, influencing our decisions. So what differences do you see in risk management globally? Culturally, are we really different across the different countries in terms of how we approach risk management?
AS: That's a very good question. The short answer is I'm sure we are different, but because most of the people on the planet are so fundamentally wrong in their approach to risk and uncertainty in general, that our cultural differences are insignificant compared to our methodological differences, so I can kind of put it that way. We're so inherently ignorant of uncertainty and risk when we make decisions in the workplace that some countries are slightly worse. Some countries are slightly better at it, but we are kind off by a mile and plus/minus hundred meters is not the real problem per se.
KS: My next question, since we've been talking about predictive analysis, what are you predicting for the next couple of years in the risk management space?
AS: I sincerely hope that the messages that myself and many other risk managers have been pushing for years will become more mainstream. I hope that we will switch from having conversation about risk and the risk levels to having conversations about uncertainty affecting objectives, or decisions, or forecast of budgets and we will actually finally find that magic pill, which by the way, I have a sense that I think I might have found it recently, finding that magic pill to sell the idea of thinking in ranges and scenarios and simulation the futures of the executives.
I just think it's so ironic that before making any kinds of decision any kind of big decision, the CEO would call his tax advisor. He would call his legal team. He would call his finance team to first figure out what the potential problems are, and then run some sort of scenarios to figure out what's the best approach. I mean, most executives do that as a given. No one would make a decision without first consulting at least somebody in finance, tax or legal team.
And yet, almost no one calls the risk manager for the same advice because I think we've not done well in selling our tools, our expertise as being able to add and make sense of uncertainty in the future decision making. Fingers crossed, I've only been saying exactly the same thing for the last seven years. Fingers crossed, business plans of the future will not have a single target. It will not just say we want revenues of 100 million. It will say we want revenues from 80 to 105 million and this is the kind of the probability of achieving our objectives. We will actually stop talking about relative things when we talk about the future and we will appreciate that uncertainty has a huge impact on the future and we will be honest with our shareholders and government regulators about the effect that uncertainty has on objectives.
I mean, I was amazed when I was still the head of risk of one of the big sovereign funds in the country. I was amazed my CEO had the courage to take the Ministry of Finance the calculations that we've done and he's shown that ... We build this strategy until like 2020. This was quite a few years back and that strategy basically said that there was a 30% chance that the strategy will not work. And in fact, if it doesn't work, we may lose quite a lot of money and this is how much money we will lose with 90% confidence interval. I thought it was just amazing. I've never seen anything like that when the companies were honest with the government and the regulators and the public about the level of risk they're taking and how that risk, if it happens may affect the bottom line. I sincerely hope this is the future that we are moving towards.
KS: I hope so too and I think we're right there. I think we're at that tipping point just because I feel like a lot of organizations are taking another look at risk and really wanting to change and improve their programs. They're starting to invest in the tools and technology that they need to set their programs up for success, which I think is a really good starting point. You have to have a foundation of gathering that data to be able to do anything smart with it. So I think we're on that precipice. So I'm hopeful with you as well.
KS: Well, this was really eyeopening and sort of a brief little history on risk, so thank you for that. I'm going to say you are refreshing and very reasonable, not controversial at all.
AS: That's what I've been saying all this time. I mean, I personally don't think I'm controversial. All I'm saying is wake up. The things that we're trying to do for the last 30 years don't work as well as we want. Maybe the problem is not that executives aren't listening. Maybe executives are actually very clever that they're not listening to some of the nonsense that we're trying to sell them. And why don't we go back to the drawing board and use some of the tools that the engineers, scientists, doctors have been using to make decisions under uncertainty for the last century.
KS: I agree. Get back to the basics. All right, well, Alex, that rounds out our conversation. I really appreciate you coming on GRC and Me. I think this was great information for our audience, so thank you for that and I wish you the best of luck with the academy. I'm going to be tuning into your content as well, so thank you.
AS: It's a pleasure. Thanks Kelley.
Different analysis methods are better suited at each phase, with the greatest concentration of quantification tools needed during…
We sat down with Shannon Harrison, LogicGate’s Senior Director of User Experience, to learn why we’re making accessibility…
On this episode of GRC & Me, we explore business resilience and the differences between proactive, reactive, and…
Build a Centralized View of Assets, Risks & Cyber Controls
Find out how to take a proactive, connected approach to your cybersecurity risk management processes.