Machine Learning. Security Friend or Foe?

In this week’s Cyber Threat Intelligence roundup, an update on new machine learning (ML) models, and why the InfoSec community is divided about whether they are helping or hurting cybersecurity efforts. Next, the CTI team examines the emergence of a new proxy platform called Black Proxies, which is currently being marketed to threat actors as a solution to aid in hiding or obfuscating their originating IP addresses. Finally, we wrap things up with an overview of the newly-established Cyber Safety Review Board’s (CSRB) plans to devote the committee’s second-ever review on examining the tactics, techniques, and procedures (TTPs) employed by the notorious Lapsus$ group – one of the year’s most prolific and high-profile threat actors – in an effort to help organizations protect themselves from similar malicious activity.

1. Machine learning models: a dangerous new attack vector, or key cybersecurity technology?

Security researchers are divided about the emergence of powerful new ML models, which are the decision-making systems powering most of today’s artificial intelligence (AI) solutions.

One faction of security researchers is touting ML as the savior of cybersecurity, while the other is concerned over its potential to produce convincing phishing lures, aid in deploying malware, and help threat actors move laterally across enterprise networks.

All eyes are on ChatGPT

Considered a major component of the overarching deep learning movement, ML models are often publicly available, and a surprising number of corporations were reportedly already leveraging ML as a primary technology in 2021, according to a report from CompTIA.

OpenAI, a San Francisco-based AI company founded by Elon Musk, is considered by many to be the current leader in the space. The firm is responsible for the revolutionary AI tool DALL-E 2, a breakthrough image generator that came out this year. However, the OpenAI tool currently grabbing headlines is ChatGPT, a prototype AI chatbot based on OpenAI’s GPT-3.5 language technology, which was described as “a large artificial intelligence model made by OpenAI that has been trained on a massive amount of text data from a variety of sources” in a blog post on the subject published by Forbes.

Some other key facts about ChatGPT:

• ChatGPT is an AI-powered chatbot software programmed to simulate human interactions.
• ChatGPT is still in the research review phase and was made available to the public on November 30, 2022, via OpenAI’s website. Users can sign up and test it for free.
• The chatbot features a dialogue format enabling users to provide an incredibly wide range of both simple and complex instructions, which ChatGPT is trained to carry out and return a detailed response to. OpenAI claims the bot can also answer follow-up questions and recognize and admit when it makes a mistake. Like any ML-powered AI solution, or any software for that matter, ChatGPT has limitations. The chatbot occasionally produces incorrect answers, uses repetitive phrases, and sometimes has difficulty answering a repeatedly asked question until the phrasing of the question is slightly tweaked.

It’s important to keep in mind that ChatGPT is more than just an above-average computerized conversation buddy. It can also write code, find bugs in code, and more. ChatGPT also remembers the context surrounding a given topic, so users can refer to something they previously mentioned and ChatGPT won’t skip a beat when it comes to following that conversation thread.

It is those factors — as well as a few others — that have InfoSec professionals worried.

Speaking of differing opinions

It is quite possible that ChatGPT has done more to divide the opinions of those who have experienced the chatbot’s capabilities than any other ML-powered AI tool.

Kevin Rose of the New York Times had this to say: “ChatGPT is, quite simply, the best artificial intelligence chatbot ever released to the general public.”

Kevin is not alone in his belief. OpenAI’s product has exploded in popularity over the past few weeks. For example, a Vox article calls ChatGPT’s underpinning technology GPT-3, “a tremendous leap for AI.” The article features a line of copy in which a researcher bemoans AI’s lack of humanity, waxing poetic about its inability to love, appreciate art, beauty, or humor. The subsequent paragraph reveals that the preceding prose was actually generated by ChatGPT — the result of a prompt given to the chatbot which featured a few words expressing skepticism about AI. In reply, ChatGPT produced a long, rambling soliloquy about the unlikeliness of computers ever becoming truly intelligent.

What the pessimists are saying

For every user enamored with this technology, there are probably five security researchers sounding the alarm, warning anybody that will listen of the threat to cybersecurity posed by tools like ChatGPT.

For example, a DarkReading article portrays ML learning models as a rapidly growing, dangerous new attack vector:

“Threat actors can hijack machine learning (ML) models that power artificial intelligence (AI) to deploy malware and move laterally across enterprise networks, researchers have found. These models, which often are publicly available, serve as a new launchpad for a range of attacks that also can poison an organization’s supply chain — and enterprises need to prepare.”

A blog post issued the same day by HiddenLayer’s SAI team claims that its researchers have developed a proof-of-concept (POC) attack that demonstrates how a threat actor can use ML models to infiltrate enterprise networks.

The attack reportedly relies on a combination of several techniques, including “steganography for hiding malicious payloads and data de-serialization flaws that can be leveraged to execute arbitrary code.”

The post points out that the attack’s weaponized model evades current anti-virus detection and EDR solutions, adding, “Currently, most popular anti-malware solutions provide little or no support in scanning for ML-based threats.”

Another major — and perhaps the most obvious — concern for security professionals is ML models’ potential capacity for generating extremely sophisticated and convincing phishing campaigns.

Several researchers have already tested ChatGPT’s ability to produce convincing phishing messages and lures, with surprisingly successful results.

Some enterprising users have already developed easy-to-use interfaces to aid users in leveraging ChatGPT’s ability to generate emails.

There are also concerns about ML models’ potential impact upon supply chains. Organizations with limited IT resources often rely upon open-source model-sharing repositories in their deployment of ML models. Such repositories often lack proper oversight and comprehensive security controls.

This has the twofold effect of both expanding the threat surface and saddling users with a higher degree of risk resulting from their dependency on what very well may be tainted/weaponized models. The same is true for companies that rely on third parties for models.

Analyst comments from Tanium’s Cyber Threat Intelligence Team

“While the likelihood of these various POCs being successful in the wild is admittedly probably low, it’s important to keep in mind that this technology is in its relative infancy, and threat actors are certainly paying attention to its growth.”

“That said, there is cause for cautious optimism. For too long now, the phrases ML and AI have mostly just been buzzwords tossed around in boardrooms and Zoom calls by vendors and cybersecurity professionals alike. However, ML may emerge as a truly vital cybersecurity technology. ML-powered solutions have the potential to analyze trends and learn from them to prevent attacks at earlier stages, and more quickly react to threat actors altering their behavior.”

“Additionally, it can make cybersecurity teams more proactive in their overarching response plans, helping to mitigate ongoing threats in real-time and enabling SOCs to free up time and resources to pursue other opportunities and activities.”

2. Black Proxies is becoming increasingly popular among hackers

Researchers at internet intelligence firm Domain Tools have spotted a new proxy platform in the market, dubbed Black Proxies. The service is being marketed to cyber threat actors by extolling its reliability, scope, and vast number of available IP addresses. Black Proxies claims it has over 1,000,000 residential and other proxy IP addresses from around the world from which threat actors may launch operations.

How proxy networks operate

A proxy allows a threat actor to perform activities anonymously by hiding or obfuscating the IP address that the threat actor’s operation is originating from. The market term for these proxies is ‘proxy networks,’ or more generically, ‘proxy services,’ and these solutions allow threat actors’ activity to essentially blend into legitimate traffic.

Proxy networks have evolved over the years, much like the rest of the threat landscape. In the past, a victim would be tricked into downloading malware on their computer, their machine would then be used to infect other victims, and the IP address of the original host and all subsequent compromised machines would then be used by the threat actor to proxy further malicious activity.

This method has been used for quite some time, but its effectiveness is limited to the number of victims the threat actor could compromise – along with the threat actor’s ability to maintain persistence on the machine.

The evolution of proxy tactics

There’s been an increased focus on proxied IPs as a delivery vector and adversary tactic over the years. Threat actors are realizing that growing a network by one IP at a time is not good enough. Security professionals often block IP addresses as they are identified as malicious, forcing threat actors to need a constant stream of new IP addresses.

Because of this, malicious proxy services have been evolving to create a larger IP infrastructure. Two new tactics that have been observed include fraudulently gaining access to new IP address ranges and having entire ASN blocks reassigned.

• RESNET: RESNET was a service that used fraudulent tactics to gain access to IP address blocks from mobile phone companies in the US. In 2019 RESNET controlled over 70,000 IP addresses, according to Krebs on Security. RESNET’s original offering was access to sneakerbots, which allowed users of the service to attempt to buy highly sought-after sneakers. The operators behind RESNET clearly realized that there was a broader more lucrative market to target outside of sneakerbots and soon pivoted to offering proxy services. The service was ultimately shut down by law enforcement.
• INSORG: INSORG was another popular proxy service that possessed even more aggressive tactics than RESNET. INSORG enumerated the IP address space and ASNs owned by companies in receivership. It was able to fraudulently reassign entire ranges by impersonating bankruptcy trustees and filing paperwork with IP registries. INSORG was ultimately shut down by authorities in late 2020.

Earlier this year, the FBI released a warning on the rise in the number of proxies being leveraged by cybercriminals to facilitate credential-stuffing attacks. The advisory included recommendations for reducing organizational risk related to the activity, but ultimately served to highlight the growing threat of malicious proxies.

Black Proxies entered the malicious proxy market, stepping in to satisfy the growing demand for these services. As stated above, Black Proxies boasts over 1,000,000 residential and other proxy IP addresses from around the world, available to threat actors at any given time. Researchers examined the service and found that a pool of IP addresses listed by Black Proxies in the Fall of 2022 totaled just over 180,000 IPs. While this number is much lower than the service’s claim of 1,000,000, it is still considerably larger than the IP space offered by more traditional services.

Black Proxies’ service caters to the demands of its clients by including functions that would be of particular interest to cybercriminals.

Researchers at Bleeping Computer discovered the pricing model for Black Proxies. The cost for using the service is $14 per day, $39 per week, or $89 per month. There is also apparently a try-out option on offer, coming in at only $4.90. Bleeping Computer also determined that an IP address included in the Black Proxies infrastructure was previously linked to other shady services.

Analyst comments from Tanium’s Cyber Threat Intelligence Team

“The use of proxy networks and services by threat actors engaged in malicious activities, while not a new phenomenon by any stretch, has steadily been gaining in popularity over the years. We’re now seeing various proxy services clearly being built from the ground up with the intention of helping cybercriminals facilitate their operations.”

DomainTools sums it up quite nicely:

Criminal proxy networks are now a mainstay in the cybercriminal world. Bad actors make use of them to obfuscate their activities by hiding behind hijacked IP addresses, providing a veil of legitimacy to mask their operations. These networks were once grown as part of the creation of botnets, but the lucrative nature of these services have turned them into their own criminal enterprises.

3. DHS Cyber Safety Board to review Lapsus$ gang’s hacking tactics

A December 2, 2022, press release issued by the Department of Homeland Security (DHS) announced that the Biden administration’s newly minted Cyber Safety Review Board (CSRB) — established pursuant to President Biden’s Executive Order (EO) 14028 on Improving the Nation’s Cybersecurity — will be reviewing recent cyberattacks attributed to the global, extortion-focused Lapsus$ hacking group.

Lapsus$ is responsible for compromising a slew of high-profile companies earlier this year. Many incidents resulted in the leak of proprietary data, source code, and other highly sensitive information which ultimately contributed to several massive data breaches.

Lapsus$: A refresher

As stated by BleepingComputer, the Lapsus$ group repeatedly made headlines following a string of surprisingly effective and damaging cyberattacks against major tech companies.

Characterized as one of the most prolific cyber criminals of 2022, Lapsus$ has set itself apart from its financially-motivated/extortion-focused peers by eschewing more traditional TTPs, such as the ransomware model; instead deploying more human-focused methods involving fairly sophisticated social engineering to gain access to target environments.

The two main competing theories regarding Lapsus$’ preferred method of initial access appear to be breaches resulting from the compromise of third-party service providers (particularly those providing authentication services, such as Okta and the recruitment and social engineering of insiders within high-profile corporations.)

Lapsus$ is also a bit of a black sheep when it comes to the group’s focus, or lack thereof, on operational security (OPSEC). Never one to shy away from publicity, Lapsus$ has freely advertised its activities via a public Telegram channel.

CSRB to conduct second review, on Lapsus$ tactics

The reported goal behind CSRB’s review of the Lapsus$ group’s hacking activities is to provide both public and private organizations with advice geared toward defending against the group’s tactics, and those of any other groups which may have been influenced by the surprising level of success Lapsus$ achieved while leveraging basic TTPs.

“With its review into Lapsus$, the Board will build on the lessons learned from its first review and share actionable recommendations to help the private and public sectors strengthen their cyber resilience,” DHS Secretary Alejandro N. Mayorkas said.

Of note, the wording of the headlines being pushed by multiple industry media outlets is highly misleading. Several sites are claiming that the CSRB will be conducting its second review of Lapsus$ tactics. This is false. CSRB is actually just conducting its second review period (the first being focused on vulnerabilities discovered in late 2021 in the widely used Log4j open-source software library).

Analyst comments from Tanium’s Cyber Threat Intelligence Team

“It’s encouraging that the CSRB is actively researching Lapsus$. The gang’s potent combination of boldness (in target selection), brashness (in taunting victims and generally maintaining an aloof online persona), and reliance upon TTPs and social engineering methods that are next to impossible to prevent with automated solutions is a mixture that should make any organization nervous.”

“While several Lapsus$ members have supposedly been rounded up by law enforcement, the group’s tactics have made quite an impression on the criminal elements of the cyber threat landscape. Attacks are increasingly leveraging social engineering, insider threats, living-off-the-land TTPs, and so on.”

“The CTI team is excited to see the CSRB’s additional findings and will continue researching Lapsus$ activities moving forward.”

Catch up on more of our cyber threat intelligence roundups here.