Vulnerability Scanning vs Penetration Testing

S4 Applications

New to security? Want the 2 line summary?

Vulnerability scans are easier, cheaper and less risky than a pen test, but they find mostly the same things.  For this reason we would suggest starting with a vulnerability scan.

See our product recommendations.

Cybersecurity threats

Today, hackers or cyber criminals are prolific and sophisticated in the methods that they use to attack businesses. Which makes managing cybersecurity a challenge for many organisations.

Information theft remains the most costly and primary goal of cybercrime, but data is not the only target. Sensitive company information and technology systems, such as industrial controls, can be hacked to disrupt businesses.

These cyber-attacks are automated and indiscriminate, exploiting known vulnerabilities within an IT system

So, when it comes to cybersecurity, you’re only as strong as your weakest link. Which means any business can be at risk from cyber-attacks targeting known vulnerabilities.

With system vulnerabilities, often when I am talking to clients they are puzzled by what is the difference between a Vulnerability scan and Penetration testing (or Pen testing for short).

In this blog, I will walk you through what each one is, how they are used, where they slot into your security model. And provide some recommendations on the products to consider.

Read our blog to get more insights on Vulnerability management: 

Some background

There are many entry points that a cyber-attacker can use to access your organisation’s systems. 

So, the objective is to reduce those threats.  

Two of the big areas to consider are:

  1. Devices on your network.  These may have security vulnerabilities that can be exploited to gain unauthorised access.  The attack could come directly over the Internet for Internet connected devices.  For devices hidden behind a firewall, the attacks often come in the form of links in emails. Users click on the link or malicious email attachments and in rare cases just visiting a web site.
  1. Web sites (web applications). These can have security issues such that a bad actor can abuse an application to access data they shouldn’t. Or use it as a staging post to get inside your firewall and move on.

When we use the term “Vulnerability Scanning” we typically mean the first of these i.e. infrastructure scanning. But in truth there is little difference between them when you consider the outcomes.  

If a public facing server is susceptible to Eternal Blue (CVE-2017-0144) or a web application to SQL Injection there will be costly consequences.

Within this post, when we refer to “Vulnerability Scanning” we mean scanning for both infrastructure and web applications. Whilst not 100% correct use of the term, it does aid readability.

What is Vulnerability scanning?

A vulnerability scan is an automated, high-level test that looks for and reports on potential vulnerabilities. 

A vulnerability is a weakness, error, or miss configuration.  Typically they are software issues, but they can cover things like not changing the default password.

A vulnerability scanner will look (scan) for particular vulnerabilities.  The most common output of a scan would be “Machine X has issues Y and Z”.  The scanning tool will then recommend a solution if one is available.

The scanners will look at file versions, machine behaviours, banners, responses and so on. And compare against a database of well over 100,000 things, to look for and assess a vulnerability.  

The tools are typically relatively passive (i.e. they are just looking) and are safe to run on live systems.  Running a scan will increase the load on the machine, so select a quiet time to do this.

An important point to note is that vulnerabilities are not always critical.  There may be false positives, some relate to only theoretical attacks, others to minor problems that an attacker would struggle to take advantage of.

Fixing vulnerabilities

In an ideal world every vulnerability would get fixed or patched as soon as it was discovered.  The world however is not ideal, so many reasons contribute to not fixing vulnerabilities, including:

  1. Wanting to fully test the implications of a patch; so a delayed implementation.
  2. A patch may mean that something else on that machine is out of its supported environment.  This situation is typical with Java. For example application X is supported on Java version Y. So Java cannot be updated without upgrading the dependent application.
  3. The patch will not install for some reason
  4. There is alot of work to do, so the organisation fixes items with a severity score above a certain value
  5. No patch is available

If a machine is not fixed, there should be some form of technology installed to protect against an attack. For example (AV, Endpoint Protection, WAF etc).


Case study: 

WannaCry 2017 and the UK’s NHS

Is it illustrative to look at a real world example:

Although it was not directly aimed at the NHS, the WannaCry cyber-attack  (which used CVE-2017-0144 or https://en.wikipedia.org/wiki/EternalBlue) highlighted vulnerabilities within the NHS in England.

Had the NHS been using a vulnerability scanner, then they would have been aware for well over a month that their machines were susceptible to CVE-2017-0144, and that it was urgent issue. It is not clear if the NHS didn’t know that they machines were vulnerable, or if they knew but had not addressed the issue.

This particular attack exposed a big need to improve security across all parts of the NHS. That extended to improving discipline, accountability and the swift patching of systems when new security updates are released. It also showed that there had been a historic underinvestment in network security and a lack of up to date software.


A patched machine is “remediated”. To rely on alternative options to help prevent the vulnerability being exploited, it would be considered “mitigated”.  Some mitigations are not 100%, so may be a partial mitigation.

So this leads us nicely onto pen-testing.  

What is Pen-Testing?

A vulnerability scanner can be started and left to run, a pen-test is performed by a skilled human. They will behave like an attacker, and try to detect and exploit your system vulnerabilities and weaknesses. Checking whether there are default or weak passwords being used.

The pen tester differentiates between a machine that is vulnerable, and one that can be exploited (i.e. are your mitigations working).

A pen-tester will link many vulnerabilities together to achieve their goals. Many small low severity exploits could be linked together to do something useful (i.e. bad).

A pen-tester is coercing the system to do something that it was not intended to do. For example run unauthorised code, or gain access to something to which they don’t have permission to.  

This is essentially a change to a live system without the associated change control processes that most companies have.  For this reason, companies control who and when pen-tests are carried out.

How is your password security?

As a thought, are you sure nobody within your organisation has a password on this list: https://en.wikipedia.org/wiki/Wikipedia:10,000_most_common_passwords.

There are plenty of tools that know this list, in fact they know the top 100,000, 1M and 10M passwords. And machines don’t take very long to cycle through these lists.



How do you use a vulnerability scanner?

A vulnerability scan should be part of every organisation’s security practises.  

The technical skills required to run a scan are not that great, and the tools are not that expensive.  Use the output to ensure your machines are patched and as up to date as possible.

You’ll also find that the scanner will find many more vulnerabilities than you can deal with. All with different levels of seriousness.  

On a recent test on my home network, a scanning tool found multiple issues with my Samsung smart TV. These were low level issues and Samsung said that the TV was running the most current version of the software.  So in this instance there was nothing I could do; no patch was available.  You will find similar situations in a corporate environment.

How do you use pen testing?

Penetration tests are run because organisations are forced to do so.  

There are various regulations that mandate regular pen-tests by independent 3rd parties for particular organisations and business sectors.

In addition there are other large scale, high profile organisations (i.e. banks, Google, Apple etc.) that are constantly under attack, and have pen testers working full-time (typically called “red teams”).

A pen-test has the following differentiators from a vulnerability scan:

  • It is broader and covers more things
  • It tests how effective your mitigations are
  • There are no false positives
  • It has more risk associated with doing it
  • It costs more

Where do these tools fit into my security maturity model?

A vulnerability scan should probably be a weekly thing, a pen-test (for most) is probably quarterly or less; it depends on how seriously you take security and your companies risk profile.  

Read: Our security maturity blog post.

A vulnerability scan can be automated, consumes less resources and is cheaper than employing a pen-tester. So it makes sense that it should form the baseline for your security strategy. You then need to use the scan results to drive the remediation process.

Scanning needs to be part of a remediation and mitigation process.  Just scanning alone will give you an ever increasing list of issues.  You need to compliment the scan with remediation and mitigation strategies and get the buy in from the organisation.

A scan of a single machine could generate 100+ issues (my TV had 5). Then you need some form of prioritisation, read our “Risk Based Vulnerability Management” blog for more information.

If you have patched every vulnerability (which is highly unlikely) then a pen-tester would struggle to find anything.

Vulnerability scans are easier, cheaper and less risky than a pen test, but they find mostly the same things.  For this reason we would suggest starting with a vulnerability scan.


How big is the threat?

Ginni Rometty, IBM’s CEO, said: “Cybercrime is the greatest threat to every company in the world.” And she was right. According to McAfee, hackers create 300,000 new pieces of malware daily and there is a hack attack every 39 seconds.

The current [IP V4] Internet has 232 addresses which is about 4 billion.  So, with 4 billion machines to scan, it will take some time. But the criminals are happy to play a numbers game to eventually find an unprotected machine. 

There are many machines in circulation, run by cyber criminals, constantly looking for machines with certain vulnerabilities to exploit.

Automatically exploiting vulnerabilities

Placing a machine on the Internet, vulnerable to the EternalBlue (WannaCry) virus, it would quickly become infected. These are not actions of people searching, finding and infecting machines, but a completely automated search and attack process.

The infected machine is immediately patched by the attacker. This would be to prevent others from getting onto the machine that they had just infected!!

If for example there was a vulnerability found with routers from company XYZ Corp, that are Internet facing, cyber-criminals would use https://www.shodan.io/ to search for them. Compiling a list of routers from XYZ Corp connected to the Internet and then start to attack them automatically.

If you think this is all hypothetical, see a report from the NSA (https://media.defense.gov/2020/Oct/20/2002519884/-1/-1/0/CSA_CHINESE_EXPLOIT_VULNERABILITIES_UOO179811.PDF) where they talk about the top 25 vulnerabilities used by adversaries to launch attacks.  If you look down the list, many are with Internet facing networking equipment like VPN servers.

As a thought; how do the NSA know the top 25 used for attacks? Because it it probably their top 25 list that they use when attacking others!!


What products do we recommend?

As a reseller we can pretty much sell any product we want.

So, the products listed below are the ones that in our own assessment are the best in their category. As the market changes, so will our product portfolio.

Infrastructure Scanning

For scanning your infrastructure we would recommend the following products:

For the SME

Vicarius does an excellent job of detection, plus has automated patching built in.  This gives you the ability to get a list of problems and take action upon them all from the same screen.

Watch a demo or ask for a quote.

For the Enterprise

We really like the Tenable product set.  They have a good mixture of on-premise and SaaS options available, plus the best detection engine available.

Request a demo or ask for a quote.

Have time and no money?

… then OpenVAS (https://www.openvas.org/) is a great open source solution, it lacks the functions and finesse of the commercial tools, but is much, much better than doing nothing.

Recently Google open-sourced a tool that they have been using internally, you can find details on it here: https://github.com/google/tsunami-security-scanner

Web Application Scanning products

We looked at all of the vendors in the market and found that Netsparker did the best job of detection with virtually no false positives.  

The tools also scale from a single PC install managing a handful of web sites, to a full Enterprise implementation that integrates into the software development lifecycle.

Request a demo or for a quote.

Penetration Testing products

There are lots and lots of open source penetration testing tools out there that you need to put together yourself to do anything.

We chose Core Impact for the following reasons:

  • Easy to use – lots of wizards that automate most of the main tasks that need to be done
  • A single place for everything – rather than downloading tool X from here, compiling it, using it, then tool Y from there and so on, Impact has everything in one place with a full audit trail of what has happened.
  • As everything is in one place, the reports can also come from one place. 
  • Anything from summary to detail can automatically be generated.

Watch a demo or ask for a quote.


Take away

The internet opens the door to huge opportunities and rewards, but also lowers the technical barriers to entry for criminals to undertake malicious cyber-attacks. 

An attack on your business can take place remotely, on third party systems that hold your data, on hardware stolen from your premises. It could be by mistake, your staff sharing confidential information accidentally or for financial gains.

The commercialisation of cybercrime has made it easy to obtain the tools needed to launch a cyber-attack. More often than not, the hackers, like any criminal, know that if they keep on trying, eventually they will breach your defences and gain access to your systems and applications.

S4 Applications will undertake an initial security assessment and implement an appropriate plan that ensures your employees, internal data and guest data is secure, both locally and through remote access. 

We can also help create your cybersecurity response plan so your employees know what to do in response to an incident.

Take a tour of the S4 Applications products and information, and contact us to learn more about how we can help protect your business from the ever increasing threat of cyber crime.