The Challenge
Have you ever been asked “how good is our security?” or “is our security getting better or worse”? These are not easy questions to answer because security is not one “thing” it is many, inter-related “things”. Data Integration is a way of putting all of this information together to improve security.
To answer those questions (and many other good ones) you have to bring lots of data together from different security sources, integrate it, and standardise metrics to be able to interpret the data.
Saying “… and standardise metrics” may be only 3 words but is actually a very complex thing. You want the ability to be able to say that a security issue rated 7 out of 10 from system X is the same as a 7 out of 10 that you can find in system Y.
Read our article on prioritisation for more details on this.
When you have integrated your data, you can then look at a business application (say ERP) and see:
- Issues reported by DAST tools – web application scanners
- Source code issues spotted by SAST tools – source code scanners
- Infrastructure issues spotted by network scanners
- Pen-test issues
Once you have the data integrated and in one place, you put a number on it, and track performance over time. The act of “putting a number on it” means that you can assess those “better or worse” questions because you can see a trend in performance.
For security purposes you should also be able to make these assessments across the entire organisation, breaking down by any arbitrary unit such as applications, location, business unit etc.
Below is what an application risk dashboard could look like to help you integrate those different sources of data.
In essence you need a data warehouse for your security data.
You may also find our blog: 8 best practice steps for vulnerability management an interesting read as it explains the full vulnerability management process.
Different Data Sources for Data Integration
Most organisations have lots of different data sources which can be grouped in many different ways. You will probably things in each of the categories (or layers) below, and have tools to help manage the security of items within that category (or layer)
I like to think in layers, so here are the typical layers that we see:
Network layer
What is connected to your network?
Most organisations think they know, but this changes all the time as people enter and leave an office, VPN connections are made and dropped, servers are commissioned and decommissioned.
Organisations have more things, and more types of things on their network than they think. It is not until an organisation actually looks that they find the truth.
I am sure your network has servers, workstations, networking equipment, printers and possibly BYOD on it. Are you really sure that nobody has added that Samsung smart fridge / smart TV. How good / patched are your security cameras, door card readers and so on.
Because this data is always changing it makes sense to look regularly and with several different, overlapping tools. You will probably want to bring together data from say NMAP, an LDAP directory, and a CMDB into one place so that you can view the full picture of an asset. This Data Integration will significantly improve your overall security.
Operating System Layer
Every one of these devices will have an operating system on it with a version and a set of issues.
Some of those operating systems are simple and you just upgrade from one version to the next (think iPhone or similar). Others (say Windows and Linux) are more complex and you upgrade the operating system and individual components within it.
Each of those operating systems, plus the software issues inherent with it will need to be recorded.
The vulnerability scanning tools do a great job of collecting this information.
Application layer – Packaged
On some of those assets that you have managed to identify, there will be applications. They may be corporate standard things like Microsoft Office, specialist like Adobe PhotoShop, or tailored like most ERP solutions.
Either way, any given version will have a set of issues, and the process to resolving those issues is usually to apply patches or upgrades. That is not to underestimate the work involved in the upgrade or an ERP solution, but ultimately it is an upgrade!!
The vulnerability scanning tools make a start on this, but you have to overlay software inventory tools too. If a user has installed WinZip or WinAmp on their machine, the vulnerability scanner will probably not report it. So you need a software inventory tool to capture that so that you can tell them to uninstall it!!!
Application layer – In house built
This is really very similar to the above section. You have some software, it has security issues, and you need to apply some patches / upgrades to fix the issue. The main difference is that an internal team will provide the fix as opposed to an external supplier.
The way that you find these issues is different though. Companies deploy a wide range of tools and technologies including:
- DAST tools
- SAST tools
- Bug bounty portals
- Penetration testing
There is a grey area in here too. This is where you have an application from a vendor, say an HR system, but you are not convinced that they are doing the best job of security. Here you may point a DAST tool or pen-tester at that application and look for issues. Should you find any then you are reliant on that vendor to provide the update and you are back into the patching cycle again.
Virtualisation
Lots of things fall into this category, from tools that look at your AWS / Azure / GCP policies and permissions (open S3 bucket anyone?), to analysing Docker containers, to VM Ware images.
All of the relevant information needs to be collected and analysed in order to have good Data Integration.
Read more about how to Assess your security Posture with our Security Maturity Model in our blog.
Users
Information about employees, what assets they control (workstation, phone), are responsible for patching, and their security profiles within different applications all need to be added in. Some of this data may be in a directory somewhere, other data may be in Excel files or HR databases.
Bringing it together and taking action
Now that you have security data from all of the above sources you need to bring it together so that it can be interlinked and then take action. That bring up 2 further areas of functionality that you need to address.
Generating / Inferring Data / Linking Data
Some of the data that you have about items within your organisation is not available in any of the above Data Integration sources. Consider a machine called “LON-S-1234”. You may look at this and be able to say, “Ah yes, that is a server within our London data centre”. Similarly, another machine called “W1234” may be a workstation belonging to the employee with the personnel number 1234.
All of this information needs to be extracted and added into the system as separate items so that you can query it. At some point you may want a list of all the servers in the London data centre, or all the assets managed by employee 1234.
The reality is that you will probably need to import some data from Excel, CSV or other databases to dd additional detail.
You also need Data Integration to tie things together. The application known to the SAST tool as “Phoenix” may be known to the DAST tools as “https://internal.mycorp.com” so a link needs to be made. Once linked all of the data about the application can be shown on a single page, hence the example above.
Ticketing Systems
Once you have got everything in one place, and understand the situation, most organisations want to take some form of action. For this you need the ability to move information into and out of a ticketing system. How you do this is discussed in more detail within our Automation article.
Bringing it all together
So, we have discussed the different types of Data Integration and different types of data sources. Each of the places that you want to bring data from will have its own API, everyone seems to do things differently.
Once you have a source of data, you need a database to store it with a schema. That schema needs to allow for all the different data types that you have and ways of linking that data together.
You also need to budget for large volumes of data. As a rule of thumb, for every server and workstation you will have 1,000 pieces of information. So, with 1,000 servers and workstations, you will end up with 1,000,000 database rows of some form. Think big when you are preparing your database.
Bigger than it initially looks …
This article about Integration is part of a 4-article series all of which look at different parts of the same problem. Here we have focused on one area, and neatly skipped over some other, rather complex issues.
We see the 4 key pillars to a complete vulnerability solution encompassing the following high-level functionality:
Prioritisation of issues – No organisation, not even the best, ever gets rid of all their security issues, but they do make an effort to create a complete list of them, and then fix the worst ones. And there is the challenge, what are the “worst ones”? This sounds simple, but is deceptively hard.
Integration of data – discussed in this blog.
Visualisation and dashboard reporting – As much as things are automated, people don’t always do what you expect in the way you expect; for this reason, you need reports, dashboards and other ways of understanding what has happened to get you where you are now, what is happening at that point in time, and where you need to go in the future to achieve your goals and objectives
Automation – IT Security people are expensive, so you want to use this valuable resource as sparingly as possible. Automation removes people from most of the process allowing them to focus on what really matters and allowing the machine to make the remaining decisions.
You will find that we discuss all 4 items in the article on Vulnerability Management Programme.
Security vendors don’t help
The security product vendors all have a reasonable idea of the above and are all building a suite of tools to address parts of the problem. The issue is that most vendors have one good tool and a bunch of mediocre tools that tag along.
If I look at Qualys (I am not picking on them, insert Tenable, Rapid7 etc., they are all the same), their network scanning tool is genuinely market-leading. The other 20+ that they have range from “ok” to “hmm, not bad”. They exist together to meet a sales objective of “we need more products to sell to our existing customers”.
Be wary of vendor lock-in business models.
Several years ago, software vendors would sell a perpetual license to their software; this was expensive in year 1, but less so in year 2+. This created a degree of vendor lock-in, because if you wanted to move, then you had to pay that big up-front fee again.
The vendors then worked out that they could make more money by selling annual subscriptions (which is where everyone now is). This has the advantage that, as a user, you can switch if a vendor falls behind the market or gets too aggressive with their pricing.
This burned a number of vendors, so they now look for other ways to generate lock-in. One of those is selling a set of loosely connected, mediocre tools. That makes it much harder to get rid of them all. Also, should you decide to remove one tool, they may put the pricing up on the others as a disincentive.
We generally recommend buying best-of-breed technologies and reviewing the situation every 2 to 3 years to ensure that you have the best technology at the best price.
Next steps
A planned and pragmatic vulnerability management programme will constantly review an organisation’s current status quo. Providing a roadmap on how to measure and improve security in your organisation.
S4 Applications wants to help your business invest wisely to reduce risk exposure and protect business value. Contact us to guide you through an effective Vulnerability Management Progamme.
Read our blog: Assess your security Posture with our Security Maturity Model.
