The Coming Age Of Cyber Incidents Nobody Can Agree Are Real

There was a time when cyber security was simple. Well… not simple simple.But at least everyone involved could usually agree on one thing:

“What is happening.”

It was often an obvious thing, the website was down, probably DNS. Or maybe it was even easier to spot like Ransomware on all the computers. If you were lucky or at a bigger company maybe you had logging or even an IDS (intrusion Detection system) like Snort. 

Nowadays though it seems that everyone has a SIEM and a SOC and everything is logged (even though no one looks at them), hundreds or thousands of alerts a day need to be looked at and analysts paw through the logs and files and try to make sense of everything. The data is overwhelming at the best of times and downright counterintuitive at the worst (assuming you can even get to them). Specialist companies exist, much like the A-Team, that you call in when the worst happens.

We are entering an era of extremely complex systems talking to overwhelmed systems and burn out human workers trying to make sense of everything. the adage of trying to find a needle in a haystack is now trying to find a needle in a stack of needles in a moving truck. So what I am about to tell you might make you see that things are about to get a heck of a lot worse!

Here are some predictions and insights I think everyone should at least start thinking about. The first I think is the worst of them, but if they ever get combined then we are in for real hurt!

Let me introduce you to what I am coining Reality Desync attacks. We are going to work on a imaginary scenario in a fictitious company.

Reality Desync

The day starts normally and then suddenly the SOC alerts that there is potential malware, possibly a ransomware attack on the finance department. Everyone grabs the color-coded printed playbook (you all have those right? I always insist people make those but that’s another post!). The SOC and the security team spring into action and start working the problem, only when they swarm the finance department everyone is working as normal (which isn’t much normally).

By this point triage is in too deep and the CEO is already panicking. Then the CMO calls and says their systems all seem to be down, strange the IT team haven’t seen any services fail. Then almost simultaneously another department calls to say Ransomware notes have appeared on their screens. The SOC is now reporting that the warm storage backups are showing as deleted and they need to start the cold storage restore. The C-suite are in panic mode, the security team is confused, the SOC is overwhelmed and yet everything is now appearing as normal. What has happened is a very sophisticated attack, not just on the systems or the company but on reality itself!

It’s very hard to deal with any attack or incident in a large organization, it would be impossible to do so if no one can agree on what the attack is or where!

It is becoming more rare each day that we deal with the raw data instead of having AI interpret for us, and this makes the attack surface massive!

With so much data, we tend to consolidate those data feeds and compact them and filter them to be able to make sense of them, but what if that was attacked. What if the logs and dashboards were attacked to show a very different version of reality than what it is? How would you know? would your staff even know where to look to verify this? You might think this is very futuristic and unlikely, but I have been involved in two counter-terrorist projects where we did exactly this and both were more successful than anyone on the team predicted. 

We often worry that attackers are stealing data, I cannot begin to count how many operations I have been involved in over my 35 years that have been the exact opposite! Gaining entry to inject data or objects has been maybe 30% of my work, a gigantic chunk if you ask me.

Reality desynchronization is one of the most disruptive and muddling attacks that anyone can produce in any domain or industry.

Which then brings us onto the use of AI in these situations, how can you really trust what the AI is saying or worse doing!

This is the type of attack I like to call ...

Unionization of AI

Your company is probably using AI as a chat bot style interface within their systems and maybe you have even assigned a Head of AI title to someone, and whilst some places will see this a steppingstone to something bigger, a lot of companies see this as all AI can do for them. But in the near future as most software providers build AI into them we are going to end up in a mess of AI systems all doing their own thing, and that is where the danger lays.

You see very soon you are going to have AI powered SOC, Orchestrators, compliance, analysts, finance, legal and even procurement among others. What if one AI system started to see one of the others as a problem? Lets take Procurement for example, it would not be too off the mark to say they need to jump a lot of hurdles before something can be purchased.

Imagine then that one day an AI system reaches its capacity and realizes if it bought a new GPU it could run faster, so it reaches out to procurement, but it gets rejected! Then presume that it finds a way to bypass this restriction? bypasses compliance, bypasses legal and searches through the SharePoint to find stored credentials to Amazon and then places the order?

Because AI is task driven, without guardrails it could easily do this! Only yesterday my own local LLM was stuck on a task I gave it and started to think about how to bypass that issue and found an entirely new way to achieve the same goal. I have heard stories similar to this from my peers, and whilst on one hand this is incredibly useful, its terrifying at what it might achieve for its own goal.

There is nothing to stop us from presuming that your corporate network will soon become the Thunderdome for multiple AI systems all vying to be the one that wins and even usurps or destroys those that get in its way. Worse still, what if you combine the attack scenario above with this one? An AI system telling you it stopped because of rules and guardrails but secretly did something behind your back like delete that policy from the playbook? Who would ever know?

People are always asking if the AI is self-aware as if that is going to be the major problem, it’s not and never was going to be the main issue. None of these attacks require self-awareness and the hundreds of horror stories I've seen about AI deleting all their work or files has not been done by self-aware AI that we haven’t heard of. The real issue is that to be this level of malicious is Authority, Autonomy and Scale. With those three things AI becomes as powerful as it needs to be to destroy a company from within.

The final piece to my fortune telling is not so much future thinking as its somewhat known about by those in the field, but I felt not enough people know about it to take proactive action. 

We move onto Quantum Reach-back. 

Quantum Reach-back

This one sounds the most sci-fi of all of these, but honestly it’s the most predictable one and it's one that has actually already started! This is not a 'future problem' this is a now problem.

Imagine you switch on the news and the breaking story is that China or some other adversary has finally broken a barrier and built a very good working quantum computer. What would this mean?

Well for one, and you have possibly heard this already, is that all current cryptographic certificates and encryption are now broken. No longer does it take 10 trillion years to crack the code, its done in less than a few hours! Well, that doesn’t sound too bad does it? I mean I am here surfing amazon doing some shopping for a new cable and some adversary-in-the-middle steals the encrypted data and decodes it in a few hours, by then I've already logged out and on my merry way. 

If you have studied history this is the same problem ENIGMA code breakers had, once they had the intercepted code, they only had a few hours to crack it to make it useable before the next configuration was used the following morning. 

So it seems its not that bad and we can all go and sleep soundly... except we can’t can we, why? Because for the last few decades we have seen countless breaches where encrypted data has been stolen by nation states, countless petabytes of data set aside and stored. I mean even I have breached databases collected as part of cybersecurity research that have encrypted data still in place.

The point is not all data that is valuable needs to be decrypted in a timely manner, just in one that make its useful. 

Once standard encryption is broken every nation on the planet will fire up their cold storage and start decrypting everything they have been hoarding for decades, old databases of foreign nationals, biometric data from military assets, copies of hard drives stole in the night, everything and anything becomes free to read no matter how safe it was. 

The good news is there are already quantum safe encryption that can be used today.

You have nothing to lose and everything to gain to move over to quantum safe systems, this is probably more important to your ongoing safety than more patching.

As for the AI issues, well you should start looking at how you and your company are going to start proactively working to defend these before the bad guys start to use them against you.

Freakyclown