Artificial intelligence (AI) and Machine learning (ML) capabilities are growing rapidly at an exceptional rate. Usage of AI has been prominent under various streams ranging from machine translation or search engines to AI enabled drones for expediting disaster relief operations.
Also, we see that AI can be used to solve problems that modern world is currently facing, from helping us to be smarter with energy utilization, conversation on helping wildlife to transforming on the way how we gain and leverage knowledge; however, we can see that AI may also bring its own risk if not used in the right way and with right capabilities.
Although offensive AI has already been widely discussed by various forums, the first thing is to understand & analyze how it plays a vital role in cyber-attacks. There are three major contributors of offensive AI usage:
- Scalable: by using AI, an adversary can scale up their operations through automation to decrease human labor and increase the chances of success
- Speed: with AI, an adversary can reach their goals faster
- Efficient: by enhancing its operations with AI, an adversary increases their likelihood of success.
In our analysis across threats observed and with security trends we see that AI is playing vital role in two prominent threats and have discussed in length below:
- Phishing Campaigns
- Exploit Development.
Phishing campaigns
With the usage of AI models, the threat scenario landscape has changed drastically. Offensive campaigns such as impersonation via deepfakes, AI assistants natural conversation tool, vishing via voice synthesizer etc are already taking place. Sending mass general phishing email evolved recently into so-called spear phishing campaigns threats. We will now discuss below the tools and methods.
One of the promising tools used or leveraged was SNAP_R {social network automated phisher with reconnaissance}, a tool that demonstrates the capability for automatically generating spear-phishing posts on social media especially X – former Twitter. It involved two prominent steps.
STEP
Target Discovery
- Triage users to determine which ones are either more likely to be phished or provide exceptional value.
- Cluster users into groups based on their profiles, using data such as the amount of information revealed in their profile, follower interactions, and engagement metrics.
- Using the clusters [K Means Clustering], they collect users and predict which cluster the user fits into. If they fit into a cluster displaying features identified as likely to lead to successful phishing, they are selected as a target, and reconnaissance for automated profiling is performed.
Algorithm Used – K Means Clustering (NB: sample code)
STEP
Automated Spear Phishing
Target is finalized and SNAP_R sends the machine-generated tweet with embedded link.
Example of tweet send with SNAP_R, including embedded link
To generate tweets, they use both Markov models and Long Short-Term Memory (LSTM) recurrent neural networks. Markov models are used to generate text word by word on probabilities of word co-occurrences in the training set, whereas LSTMs were used to remember the context from earlier in the sentence when predicting the next tool.
Note – Sample code
Other prominent tool for targeted spear phishing which is out loud in dark web is the “WormGPT” an AI module based on the GPTJ language model, which was developed in 2021. It boasts a range of features, including unlimited character support, chat memory retention, code formatting capabilities, and native language capabilities. Below is an example the usage while instructed WormGPT to generate an email intended to pressure an unsuspecting account manager into paying a fraudulent invoice.
Even though details steps are not revealed by author related to LLM [Large Language model] source, we believe the approach remains the same as like the above [SNAP_R].
Exploit development
Threat actors work further to understand the content and inner workings of compiled software and to identify weakness which they can exploit, resulting in stealing intellectual property or sharing confidential information.
- Reverse engineering. While interpreting compiled code, a threat actor can use machine Few examples of this behavior can be found below:
- Binary code similarity can be used to identify well-known or reused behaviors [ Note – similar techniques are used also in IDA Pro and the CMU Binary Analysis Platform, but the usage of ML has shown higher accuracy rates]
- Autoencoder networks can be used to segment and identify behaviors in codeas:
- “SHAP”- a classification model explanation tool,
- “CAPA”- a signature-based tool for identifying malicious behaviors within binaries,
- “FunctionSimSearch” – a function similarity tool,
- “DEEPReflects”- tool discovering malicious functionality within binaries.
In analyzing “DEEPReflects” code as a sample we were able to identify the below functions [cf. image ‘DeepReflect below] and how it uses various generative model to annotate a malware function within a binary and assist analyst to fast track the reverse engineering process.
-
- Deep learning can potentially be used to lift compiled code up to a higher-level representation using graph transformation networks, like semantic analysis in language processing.
- Vulnerability detection. There are a wide variety of software vulnerability detection techniques which can be broken down into static and dynamic approaches:
- Static. For open-source applications and libraries, the attacker can use ML tools for detecting known types of vulnerabilities in source code. If it’s a commercial product compiled with binary code, then methods such as can be used to identify vulnerabilities by comparing parts of the program’s control flow graph to known vulnerabilities.
- Dynamic. Machine learning can also be used to perform guided input ‘fuzzing’ which can achieve buggy code faster.
Adversaries will not stop innovating — so neither can we
It is evident the offensive AI capabilities can be embedded or used in any attack vector. With the lightning speed of AI driven attacks, the existing traditional security measures won’t withhold or survive the attack.
It has been imperative that AI in defensive capacity should be thought through to upgrade defenses for faster than humans could manage.
About the author
Saju Thomas Paul
MDR Deputy Delivery Head
Saju Thomas Paul is MDR Deputy Delivery Head
Saju is the Deputy Delivery Head of SOC for North America at Eviden with a specialization in Network Hunting, Malware analysis, User behavior analytics, Incident Response.
Saju manages a specialized team to deliver this service. For over 15 years Saju has been delivering supporting niche clients across Asia, Middle East, North American region.
Saju was previously a Head of Threat hunting service and works closely with Incident Response team & Breach assessors.
Saju has been contributing to the cyber industry through blogs, forums, and representing in various technical summits.
His contribution includes an article in a book [97 Things Every Information Security Professional Should Know] published in the year 2021. He is also certified PCI QSA who assists the organization in adhering to compliance requirements of PCI and has the ability to audit the organization.
He holds other certifications such as CISSP, CISM, PCIP, CDPSE, CEH, RHCSA, AWS & Azure.
Follow or contact Saju
