🔴 2025 is the year of Deepfake Fraud: Watch this video to understand this emerging threat and protect your business Deepfakes and cloned voices prey on the very human instincts of trust – Trust on what we see and hear as the ultimate truth.
But now, AI blurs the lines between real and fake, and with that you and your business are exposed to unprecedented risk. In this video, you will learn,
1️⃣ What deepfakes are, and how they are created
2️⃣ Case studies, and examples of how deepfakes are used in scams, with shocking examples
3️⃣ Strategies for organisations to protect themselves against deepfake threats
4️⃣ Emerging trends in cybercrime and their impact on compliance
5️⃣ The importance of employee awareness and cultural change in mitigating risks
✴️ To see how real this threat is, we will see live deepfake in the video
✴️ 🎙️ Speakers are: @AartiSamani, Founder of Shreem Growth Partners, a company that advises organisations on AI-enabled fraud including deepfake attacks.
@DanHawtrey, CEO of Xoralia, a SaaS company whose product helps organisations better manage and disseminate the policies and procedures crucial to effective IT security and other compliance risks.
Transcript
SPEAKERS
Aarti Samani, Speaker 1
Speaker 1 00:06
Aarti, welcome Aarti. Really good to see you again. Just to get us started, could you just quickly introduce yourself and tell us? Tell us a little bit about yourself. Thank
Aarti Samani 00:19
you, Dan. Really good to see you again as well, and thank you for inviting me to this podcast. So I’m Aarti Samani. I’m based in London, and I run my own consulting business which focuses on human manipulation fraud, which are powered by AI, so typically they manifest in the form of deepfake and social engineering fraud. It’s a threat to us as individuals, but also to the organizations that we engage in.
Speaker 1 00:49
Okay, that’s great. Thank you. And so, so the topic of conversation that we plan for today is deepfakes. Let’s start off with a really simple question, just to make sure that everybody’s clear what is a deepfake.
Aarti Samani 01:06
So any media and a media includes image, voice, video that is manipulated to to demonstrate that an individual did or said something that they didn’t so it it takes the original media manipulates it to the extent that it appears very realistic and it appears as if someone said or did something they did not do. It’s typically used for malicious purposes, so either to gain access to data or information or money. It is also often used in bullying, in kind of revenge mechanism, etc. So it’s synthetically generated media, using the original to make it appear as if someone said or did something they did not used for malicious purposes. And it is often, most often powered by artificial intelligence. So AI as a technology, is utilized to bring this together to make it very realistic and generative AI very specifically, which is by design. Generative AI is designed to create outputs which are very, human like, not only they’re human like, but there are outputs that can be scaled and can be created very quickly. So generative AI now is allowing people to create very realistic, deepfake videos, as well as as well as clone voices, which are often used for malicious purposes.
Speaker 1 02:39
Wow. Okay, and just to be clear, so it doesn’t have to be malicious purposes. So when I go on to x, for example, and there’s a humorous video of, I don’t know, Donald Trump saying something goofy and is up, clearly he’s not saying that in real life, that would count as a deepfake as well. Would it
Aarti Samani 02:58
absolutely yeah, it’s, it’s sometimes used in chest to sort of put celebrities of politicians in situations, that makes it funny, in meme situations, etc. It is often used in marketing as well, for good. So you know, when brands want to market their products but they don’t. They don’t have necessarily the resources to invest. They can use deepfake with permission of the person whose image is being utilized. It’s used for learning and training programs, again, with the permission of the instructor, so that the recording is faster, the output turnaround time is faster and more cost effective. So there are positive uses of this type of technology, but most often, at the moment, it is being used for malicious purposes. Yeah. So
Speaker 1 03:51
we’ve ourselves used a, I guess you’d call it a deepfake of my voice to read some of the blog posts that I’ve written, because we were experimenting with doing sort of audio versions of those, and we used a tool, I think it’s called 11 labs. It was very, it was very, very convincing. It made me sound, I thought, slightly better than I do in real life. So I really liked it. But yeah, and it was a great time saver. But yes, I understand that the nefarious side of things, which is, which is really, really concerning, and that’s really where your where your your focus is. And so can you give us some examples of how deepfakes are used in nefarious ways? Absolutely.
Aarti Samani 04:39
So it’s often not just the deepfake which is used as a as an attack vector, various different things come together. So there will be there’ll be a phishing campaign which is combined with a clone voice, which is combined with a deepfake video, which is combined with sort of email. A security breach, or a domain takeover, or a phone number takeover, etc. So a lot of different types of vectors come together to form a very, very successful attack. A key component of that is a voice clone or a deepfake video, because what happens is that plays on the inherent human instinct of trusting something that you see and you hear, especially if you recognize that voice, if you recognize that face. So it plays on the human psychology to really trick the individual into a parting with the asset, whether it’s data, information, money that the threat actor is trying to do. So typically, it manifests in the form for an individual, in the form of investment scams. So a celebratory deepfake will appear on your social media feed, perhaps asking you to or telling you that this is a great investment opportunity, and it will set you up for retirement or your children’s university fees, etc. So that’s one form in which, in the way that it manifests in a consumer’s environment, often and rather dangerously, it is used in romance scams. So when people use dating sites to meet potential partners, at some point in that process, they will have a video call with that individual, or a voice call, and that can sometimes be a deepfake or a clone voice that individual doesn’t exist, and the aim is to make this person fall in love with them, to then ultimately part with money, with selling the dream of creating a beautiful, happy life together, etc. In the context of a business, it appears in a slightly different way. So in the context of an organization or a business environment, typically the executive team, whether it’s the CEO or the CFO, or some C suite executive in an organization, is used as an impersonation target. So for instance, a CFO will be the target that is impersonated I their image or their voice will be utilized to clone and to make a deepfake, to make them say and appear things that they didn’t do it before or ever. That is that asset is then utilized to contact the victim, and the victim is an employee within that organization who is close enough to that CXO, who’s been used as a deepfake impersonation target to be able to execute on the instructions that they are receiving. So usually it will be it might start with an email from this CXO, who might say, Hey, Joe, XYZ is happening in the business, and we need to have a conversation. I need to give you some instructions. Please follow them, and it’s confidential. Let’s have a call. So this individual, because they see an email, they will get on the call, and the call is actually not the call with their CXO. It is a clone voice of their CXO. And now technology allows us to not just clone the voice, but to have real time conversation. So if I say something, the technology will respond generative AI is smart enough to respond in context to what I say in the Clone voice. So the person thinks that they’re actually hearing their CXOs voice and they’re having a real time conversation, and the conversation then gives them instructions to either transfer funds or to give access to a particular software or gateway into a database, etc, that then allows the threat actor to gain access to information. So that’s how it may manifest in a clone voice type environment. Taking it one step further, it might start with on a whatsapp or an SMS, because a lot of organizations now do conduct business over these platforms as well. We don’t. We are not just limited to emails anymore. And in there they might say, Okay, let’s have a we’re going through this exercise of acquisition, etc. I need to give this is confidential information, so I don’t want to write it down. Let’s do a quick video call, and I can give you instructions of a video on where we need the funds transferred, etc. Now in that instance, the individual will jump on the video call, and they might think that they are talking with their CXO. In reality, they are perhaps not talking with their CXO, and they’re talking with an impersonation of the CXO.
Speaker 1 09:55
Oh, my goodness. Oh, my goodness, that’s. Completely crazy, and so I noticed that your voice hasn’t changed there, but that’s something that the technology that you’re using now is obviously just manipulating the image, but you could, in theory, use something that manipulates your voice and transforms your voice as well,
Aarti Samani 10:19
absolutely. So this is an example of how a live video deepfake might manifest. So clearly, what you’re seeing on the screen is not Rihanna. It is. It is Aarti who is impersonating Rihanna’s and using her face. The voice hasn’t changed, but the face has changed. So the point is that deepfakes are so convincing and so realistic, we often think that on a live call we are speaking to the individual that we’re speaking with, and that that’s creating trust. But what I’m just demonstrating now in this moment is that even what appears to be a live video call can be manipulation and impersonation of someone, but it it’s not the real individual. So these are some of the ways in which deepfakes these days manifest in our business environment. They manipulate our instinct of trusting the individual that that you see the face of, and the voice that you hear, and the one of the most, an example that happened earlier this year, which has been in the media quite a lot, was a UK company, but their Hong Kong office was targeted so a financial professional in the Hong Kong office was the victim. That individual was targeted by the deepfake of the CFO of that organization who is based in the UK. Over the course of few days, this individual received calls, etc, from that CFO which made them believe that this is a real conversation. And ultimately this the CFO instructed the individual to join a video call in which further instructions would be given. They joined the video call. There were about 10 people on that call, from the executive team, from the partner, their partners, etc, every single face on that video call was a deepfake, except the victim. And the victim had no way to tell that this, this was these were not real people. They followed the instructions and transferred almost 25 million US dollars into a fraudulent account. So these are, these are some of the examples and how deepfake manifest in our business environment and our personal environment. It’s not hypothetical. It is very real, and it is happening every day.
Speaker 1 12:53
So that’s, that’s crazy, that’s, that’s a very interesting story. But just, just going back to the Rihanna example you just did then, how quick, how easy. What kind of skills do you need to do something like that? Is it? Is it very accessible? Is it?
Aarti Samani 13:13
Yeah, great question. So you need one single still image of an individual to create a good enough realistic, deepfake video, and you need just three seconds of someone’s voice to clone their voice, very, very in a very realistic fashion. So just one still image which appears we all have one still image of ourselves, usually on LinkedIn and a professional profile, and a lot of people now have their voices as well. We do a lot of correspondence via voice notes on our WhatsApp, and if that has been hacked, then it’s very easy for the threat actor to get access to voice, short voice clips, which can allow them to clone, clone that voice very realistically, so it’s not difficult. What I use just then is a very inexpensive web tool, so I didn’t need to learn any technology. I was just able to create an account, which was very inexpensive, and I’m able to now upload single image. I literally uploaded just one still image of Rihanna, and I demonstrated what I just did there. So it’s very accessible. There are literally guided videos online that are available that give you step by step instructions on how to do this. And this is in the on sort of regular, accessible media like YouTube, there is a whole Dark Web economy. And on the dark web, you literally have playbooks, manuals, etc, which allow you to create not just deepfakes, but also how to launch these kind of attacks, step by step. So generative a. AI type tools are available on the dark web that allow you to create very realistic emails, SMSs that bypass the email security gateways, etc, so the access to the victim becomes very easy using generative AI, and then these free or inexpensive tools, which are readily available, make it then again, very easy, to create realistic, deepfakes. Combine it all together with the step by step instructions, one can launch a reasonably successful attack, something
Speaker 1 15:34
that’s crazy, but something that’s that’s, that’s really surprising there and concerning is that, you know, you just put yourself forward as Rihanna, and you did that in our secure Microsoft 365 teams environment. You know that we’ve got for our company, we’ve got all sorts of security layers protecting that, and yet you’ve managed to very easily and quickly just Yeah, put this, this completely fake video of yourself as Rihanna, into our environment. I find that’s that’s really a scary aspect of it as well, is that you could be fooled that, oh, you know, this is our own environment. It’s super secure. It’s we’ve got like a crack security team. Who are, you know, who’ve secured Microsoft, 365, for us, and, yeah, this, you just show that actually, that doesn’t really count for anything. Yeah. I mean,
Aarti Samani 16:38
you, you’ve done all the right things in your business. You have all the different security layers, the gateways, etc, and that is absolutely the way to go. So I’m not saying discard any of that you do, but the defensive tech at the moment is not keeping up with the offensive tech. And what I mean by that is that the technology which is designed to detect and protect against these deepfake is getting better, but it is just not keeping up with the technology that the bad actors are using to launch the attacks. And there’s a good reason for it. So if you think about our business environment. We are, when we run a business, we are there are always competing priorities. We have shareholder interest to meet. We have to take care of our employees, we have to make revenue. We have to operate ethically and responsibly. We have a product roadmap that we have to deliver to, etc. So there are always a lot of competing priorities for finite resources in a legitimately operating business. Now, if you look at a fraud business, the only job they have to do is to launch attacks which are successful in getting the assets that they need, whether that’s financial assets or data and information. They don’t operate ethically or responsibly. They don’t have shareholder expectations to meet. They do not take care of their employees. And we can talk about the whole human trafficking effect that’s that these scams are having on our society. So there are no competing priorities. There is only one focus, and that is to use the best technology available out there in order to launch the most successful attacks, to get the most money right, and so they can afford to so the offensive tech which was utilized to launch these attacks is better than the defensive tech. Not only that, there is a lot of return on investment. So the numbers we are talking in the case of the Hong Kong CFO deepfake, 25 million US dollars, right? It’s hasn’t cost them $25 million to design and launch this attack. So the return on investment is very high, which means they can put some of those funds back into creating even better technology to create even more impactful attacks. So it is an arms race, so security technology processes have their own place in terms of protection and detection, but we have to be very much focused on the human aspect of it. So how do we give the context? How do we give the information and awareness to our employees, to our customers, to all the humans which are engaged with our business, so that their radars and they are on high alert. So they are not always believing everything, every interaction that happens online, but they are treating it with caution, with curiosity,
Speaker 1 19:55
Wow, and so What? What? What are the. I mean, you’ve talked about that, really we’re talking about training, then, aren’t we training our own employees to sort of detect these things and be on guard? What are examples of other things that companies can do to guard against this? Yeah,
Aarti Samani 20:16
sure, that’s a great question. So I’m talking about a layered approach. So we always talk about a layered approach, and that has its place. But what I’m also talking about is a parallel approach, which is the human risk management and the one of educating and empowering the humans which are part of the business. Now technology is what should be utilized to detect and prevent, right? That’s the technology’s job. Then process have their own role to play as well. So the process ensures that there are gateways so that these things don’t see too far or too deep into the organization that they are caught at different at different stages. And then there is the human piece. Now, because deepfakes are getting so hyper realistic, it is almost impossible to detect it or to tell apart with naked eye. So I when I’m advising my clients, so when I’m running my master classes, I I never say that you need to look out for something very specific when you see an image, because that’s that’s a no win game. If I say to you today, well, look out for the teeth in the lips, because sometimes the synthetic or manipulated videos don’t have realistic teeth. Well, that may be valid today, but in two weeks time, that may not be valid because the generative AI technology will have moved and so now suddenly you’ll have a very realistic set of teeth in a deepfake video. So telling people, teaching people to be forensic investigators or forensic analysts by looking out for minor imperfections is a no win scenario. What I say is that as people, we have to train our people in critical thinking and contextual awareness. So a few things play a huge part in it. So one is psychological safety, right? So if the person, if your employees, feel empowered and safe enough to ask questions to the leadership team, then they will not just blindly follow the instructions, because it comes from a senior executive. They will, if something doesn’t feel quite right, they will go and ask a question. Hey, you asked me to do this. Help me understand the context behind it. Why does it make sense to do it now? Because they are doing their own background checks and they are verifying the information and the instructions they are receiving, but that they can only do that if they are not worried about losing their jobs or being or having negative implication in some way in the business. The other piece that plays a part is the power distance index. So the more hierarchy you have in an organization, and the more the senior executive and employees, few levels below them feel disconnected, then that is, there is a higher exposure to risk here, because the again, the employee, the victim, remember that there is the impersonation target and there is the execution target. So the target who has been identified as the person who will execute the instructions by the fraud actor, if distance between them and the impersonation is very large, and again, there’s a higher risk exposure. So even if there are many layers in an organization, what you want to make sure is the executive team is very accessible to the people, so they don’t feel like they there’s a big distance between them. And then, very importantly, is the transparency in an organization. So I’m fully aware that when we are running businesses, it’s impossible to be giving every piece of information to our employees in real time. There are sometimes you have to maintain a bit of you have to maintain a bit of a distance, and you have to give the information that is appropriate at the right time. But it’s important to give context to your team, because if someone says, for example, if, if use, if an email has been sent out on your behalf, saying, Hey, we’ve had a great quarter, and I want to give you all a bonus, click on this link to confirm the acceptance of your bonus, and it’s accompanied by a deepfake video which looks very much like you and sounds very much like you. An employee will be like, Yeah, this is coming from my CEO. What’s not to like about it? It’s coming from the official company email. Right? Immediately they click on the link and. There’s a malware installed on their devices which suddenly now gives access to information, data, etc. Now here, if they have the context, did we actually have a great quarter? I didn’t hear Dan mentioned about this in our all hands meeting. I didn’t have my team leader talk about it in our one to one or in our team meeting. So hey, let me treat this with a little more curiosity. Let me go and ask my manager, is this real? Shall I go ahead and click on this, like what is going on? So that contextual awareness plays a big part in this. If, if they just have a moment to stop and think, Okay, this doesn’t make sense in the context of the information that I have from my business leaders, I need to be curious and cautious about this. So these are some of the things that play a very important part in keeping your organization secure, in making sure that your employees are not falling victim into the frauds, human manipulation tactics that they are using, and obviously the technology and processes play the important role.
Speaker 1 26:06
It’s so much harder, though, isn’t it? You know, I’m thinking of ourselves as a company. We run these sort of fake phishing email trade things that you know, where our we send out emails, which are just these fake phishing emails to see who picks, you know, who clicks the link. And it’s really just to test how good our own training is against these things. But I mean that just feels like so basic compared with what you’re talking about, because you know that it’s, it’s very rare that you sort of, you get sort of an emotional reaction to, you know, an email dropping into your inbox. You know, you just with super on guard to, know, to these kinds of things. But yeah, with a video somebody that, you know, saying something, asking you to do something in a very, very realistic way. It’s, it’s, yeah, you can see that the emotional side of things, the psychological side of things, is so much, so much more real.
Aarti Samani 27:14
Yeah. And you, you bring up an important point there. Dan, so I the simulation exercises that you just described there, they were great at a point in time, but they are not keeping up with the new forms of attack vectors that we are seeing in the market, which has been powered and enabled by AI. And so one thing that we do offer is deepfake simulation exercises. So like I said at the start of our conversation, that it’s not just one attack vector, it’s multiple vectors come together to launch a very, very successful attack. So we offer those simulations. So we work with business leaders in organizations to say, Okay, let’s talk about your business and where the weak weaknesses are in terms of human risk management. And then let’s craft those simulations so that we can test the vulnerability of your organization based on how, how they how the employees are being manipulated emotionally. And that becomes a very, very powerful exercise. Once we do that, it gives a great report to the leaders to say, Okay, so we’ve got some work to do here. That then leads into sort of more training, more awareness, more context, where we talk about case studies. We break down the case studies, and we identify the failure modes so that the staff can understand, Oh, wow. Okay, so I need to look out for these kind of tactics in my in my personal life, but also in my professional life.
Speaker 1 28:56
Okay? And Aarti, I mean you, you were speaking earlier about how the technology is improving, and you spoke about, you know, the teeth of the of the attacker, sort of looking at, sort of more and more realistic, what are the some of the more kind of the longer term threats that you see on the horizon when it comes to deepfakes and these types of scams.
Aarti Samani 29:28
Yeah. I mean, it’s limitless, right? Every day we are seeing new type of attack vectors being launched. So recently, I was reading somewhere about so the opposite of romance scam. These are called pig butchering scams, where you engage the individual and bring them into your world, create an enormous amount of trust, and then you grab the money from their runaway, never to be seen again. That’s a pig butchering. Am and typically manifest in the romance type scenarios that I mentioned earlier. What we’re now seeing is the opposite of that, which is the threat type scenarios. So people, educated, people, doctors, etc, are receiving calls from FBI, for example, and saying, Hey, we’re watching you. You’ve been involved in some kind of a criminal activity unknown to you. So some they’re telling this victim that you have been unknown to you. You have your information and your face is being utilized in cyber crime. And so we are now keeping an eye on you, and you need to allow us access to you in order to protect you. And that becomes so they are now becoming big brothers. So they are now watching these victims all the time. They are putting psychological pressure on them and really grabbing them like this is awful. This is really nasty. We’re also seeing the util use of personal attack to get to the organization that the person is working for. So again, if I take romance scam as an example, so if somebody is engaged in this, in a with an individual they met over a dating site who is not real, but they don’t know it. Over time, they start to share information about their work, because it’s just a natural thing to do with someone you trust. Oh, I had a great day at work because I was able to close deal XYZ. And you know, we have now made X million pounds for the business. Or I had a really bad day at work because XYZ is not going right, etc. So now you are sharing information with this, with the with the attacker that they would otherwise not have access to. So what they’re doing is they are building almost a map in their design of the attack to the organization, so inadvertently, an employee who has been a target of fraud in their personal life has now become a liability for the business, because they have become a gateway for the attacker to get to the organization, which is obviously bigger money than they would get from an individual. So the boundaries of attacks between a personal and a professional environment being blurred, and they are penetrating in every which way they can get
Speaker 1 32:29
Wow. Well, Aarti, listen, it’s been, it’s been an absolutely fascinating discussion today. You’re clearly super knowledgeable about this whole topic. Very grateful for your time before we before we finish. Can you just tell us a little bit about, I guess, firstly, where, where can we find you? Where can we follow you? But also just talk a little bit about your offering to businesses, to companies out there, because I understand you’ve mentioned it a number of times that you do offer services within this whole arena.
Aarti Samani 33:10
Yeah, sure, so you can find me on LinkedIn. I’m Aarti Samani, and please happy to share my profile with your audiences. Email me on Aarti at Aarti samani.com and I will always respond. So very easy to find me in terms of my services. We offer three different services. So one is a is a master class, so 60 minute, 60 minute class that I either can deliver virtually if you have a global team, or I can deliver on site, if that is more appropriate. But it’s a really kind of deep dive into a case study how these attacks manifest in our environment, giving examples and context and then what you need to do culturally. So few things I already described in our call, in our conversation today, so we go through that. It’s a very intense, packed master class, and the aim is to give a lot of information to the employees so they deepfake is not just a term that is sort of flying around they see on social media, on television. It actually brings home and gives context and makes them more curious and cautious about their interactions online. So that’s the first piece. The second piece is the simulation exercises that I talked about, which is we work with the leaders to figure out on a quarterly basis how we should send these simulation attacks and figure out which employees, where are the vulnerabilities and the break points in the business, and then how we should fix it. And then the third piece is, should, for whatever reason, this attack, this type of attack, do manifest in your business. What does the executive team need to do to contain the damage? So how do. Contain the PR, how do you communicate with your clients, with your employees, etc? So we do tabletop exercises to make sure that there is a crisis management plan in place, should a deepfake or a social engineering type attack manifest in your business, and all of these things are bolted on to the existing simulations or existing exercises that you may already have in place. So what I’m not saying is that we replace what you have in place. What I’m saying is that what you have in place may not be covering some of these emerging threats and the emerging attacks, and we can help you there. So it’s
Speaker 1 35:40
an additional layer, effectively, isn’t it? Which is what security is, is all about, exactly, yeah, fantastic. Okay, well, Aarti, thank you very much for your time today. It’s been, it’s been, yeah, truly. Eye opening, eye popping. In fact, as well, I will never forget Rihanna joining my my my call like that. But yes, thank you very much.
Aarti Samani 36:02
Thank you for inviting me. Thank you. It’s, it’s important that we have these kind of conversations to really drive that awareness out in the market. So thank you for inviting me and thank you for recording this episode. Thank you
36:15 Goodbye. Goodbye.