Download the authoritative guide: Enterprise Data Storage 2018: Optimizing Your Storage Infrastructure
It has been sixteen months since I retired – or thought I was retiring – from my monthly column. A lot has happened since. First, my company was purchased by Seagate, and I am now part of the Seagate Government Solutions organization. That, of course, now changes what I write about in this column a bit as I am now a vendor, but I am still going to deal with the big issues facing storage and data movement. I will do my best to continue to not mention vendors unless I am referencing things that are in the news.
Secondly, this will not be a monthly column – I’ll likely post something every few months.
And lastly, I want to thank those who have written in and asked me to keep writing. Thank you!
The topic this month is going to be rootkits, which are nasty security issues that I think we all need to start thinking about, as well as what to do about them.
I came across an article just before the holidays, which said:
“Not too long ago, Eddie Tipton was convicted of hacking into the Multi-State Lottery Association’s computer system in order to rig a nearly $17 million jackpot in Iowa. Now comes word that an investigation into Tipton’s hacking activities is expanding to include a number of other states. Thus far, lottery officials from Colorado, Wisconsin and Oklahoma have indicated that Tipton may have also gamed lottery jackpots in their respective states.”
Combining this with the Juniper issue, where VPN communication could have been hacked, got me thinking about how firmware can be verified and how to ensure that it’s doing what we think it should be doing and not what someone else wants it to do.
The lottery hack was very interesting, in my opinion, given the lack of publicity that you would expect to surround such a huge, complex and well-designed hack. Maybe it was because of what was hacked. If everyone thought the lottery was rigged – which it turned out it was – people would not play, and we would not have had over a $1.6B jackpot in January.
The issue for both of these hacks was that the chain of custody of the firmware was not tracked. Rootkits are one of the hardest things to track down and prevent, and yet they likely can do the most damage to a business and/or government. I would add that I think "rootkit" is not the best term because it encompasses so much. The definition of a rootkit is a set of software tools that enable an unauthorized user to gain control of a computer system without being detected. What I think is really meant is that, in most cases, there is a change in the firmware to allow the device to either boot something that is not what you expected or to run something that you did not expect. It could be firmware on the motherboard (which is also called BIOS) or firmware on peripheral equipment such as a storage controller, network or even the disk or SSD drives.
So how would you secure a system against an attack on the basic firmware of the system, whether it be from the inside or outside, or a bit of both?
Figuring out if you are getting bad firmware from a vendor is very, very difficult because it either came with the machine or came as an update to the machine. For much of this software you have no idea where it comes from and what was the supply chain for the development of that software.
Was the software developed on a system connected to the Internet? If so, it is very likely a hacker could get into that system. How was the software checked out and checked in, who had access to the software, etc? The supply chain for firmware is critical. The National Institute of Standards (NIST) even has standards for this as part of their risk management framework for BIOS.
I think as we move forward, it is time to start asking vendors the following questions:
- Who develops your firmware?
- Where is it developed (country)?
- How is the firmware inspected for malicious or bad code?
- Is the firmware being developed for the hardware on systems that are connected to the Internet?
- Is the firmware managed with secure hashes to ensure it is not perturbed from creation to loading?
There have been cases where the firmware for a particular device was stolen, then modified, and then put into a device for the target company or individual or nation state. Clearly, there are bad actors in the world, from criminals to those who want to steal corporate IP to national security issues. I am not going to point fingers and name nation states, but depending on who you are and where you live, the answers to the above questions might be pretty scary for some parts of your system. Today there are some analytics tools that might catch some of these types of attacks, but not all of them – and most software vendors will even admit that.
In some ways the internal threat is more difficult to track, and in other ways, it is simpler. The lottery example is a pretty simple way to attack systems. Buying rootkits online from criminal organizations is easy, and there is a well-known marketplace. How can you ensure employees with a responsibility for firmware updates implement them in accordance with the manufacturer’s specifications?
The only way I can see this working – and there is still risk – is if you have multiple employees inspecting the firmware to ensure it is indeed the manufacturer’s firmware. I would have at least two or more people get the firmware and validate the SHA256 hashes. Even with this, we have seen many times when more than one person in an organization has been corrupted and/or people get lazy and do not follow the proper procedures. The insider threat is very difficult to prevent if multiple people get corrupted.
Given all the software development for firmware that has been outsourced to nations that have national security interest and/or criminal intent on basic information, intellectual property, infrastructure and bigger issues, knowing where your firmware comes from is going to be more important as we move forward for both corporate critical information and national security information.
Firmware, I believe, is the next frontier in what is going to be attacked given how hard it is to detect bad firmware. Servers, networks, disks and SSD drives are all at risk unless vendors have a way of securing firmware. A secure firmware supply chain for your critical information – whether you are a small business, health care provider or a large multinational trying to protect your IP – is today, and will be tomorrow, a large challenge.
Photo courtesy of Shutterstock.