Problem solve Get help with specific problems with your technologies, process and projects.

How to obtain a high quality vulnerability assessment

Here are some factors to consider when preparing for an assessment.

As a security manager, you should regularly test your organization's security technology and practices. Such testing...

enables you to find and mitigate vulnerabilities before malicious attackers do. An excellent way to test your security technology and practices is to regularly conduct vulnerability assessments.

Many companies offer to perform vulnerability assessments, and it's important that you identify one that will provide a high quality assessment. There are measures you can take to ensure that the assessment is appropriate for your organization's needs and that it efficiently and accurately identifies the vulnerabilities on your information systems, and then presents realistic, cost-effective steps for mitigation.

Choose your assessor carefully

When choosing an assessor, consider the following:

  • Does the assessor have a range of experience with a variety of operating systems and applications? A high quality assessor has experience with and a detailed understanding of a wide variety of operating systems and applications. An assessor who only knows Windows will not be of much use identifying vulnerabilities in Unix or Linux information systems. Make sure the assessor has proven experience with and knowledge of the operating systems and applications on your information systems.
  • Does the assessor have an understanding of core protocols? A high quality assessor has a strong, demonstrated understanding of core Internet and network protocols (e.g., SMTP, SNMP, FTP, telnet, NetBIOS) and the vulnerabilities associated with them. These protocols are often the target of attackers and can have very serious vulnerabilities. For example, your assessor should know that Telnet sends the username and password unencrypted, and he should be able to extract information from unprotected NetBIOS shares.
  • Does the assessor use a variety of discovery techniques? There are many vulnerability assessment software programs; a number of them are fairly easy to use. Some assessors simply download such tools, point them at a network and report the results. High quality assessors have much more in their tool box. They will also:
    • Conduct protocol-specific checks (e.g., check for the ability to use vrfy or expn commands on an SMTP server)
    • Check for default vendor passwords
    • Conduct application specific checks (e.g., check for vulnerable CGI scripts on a Web server)
    • Check for weak passwords and permissions (if appropriate per the rules of engagement)
    A high quality assessor, when possible, will also confirm vulnerabilities reported by software tools; many tools report false positives.
  • Does the assessor have strong communication skills? Your assessor should be able to explain -- both verbally and in writing -- discovered vulnerabilities, risks and possible mitigation methods in a clear, concise manner that is useful to both technical and non-technical persons. A high quality assessor presents findings in a neutral, non judgmental way. Instead of seeking to place blame, they clearly describe the vulnerability and present realistic, cost-effective methods for mitigation.
  • Is the assessor able to offer reasonable and appropriate mitigation recommendations? A high quality assessor presents recommendations that strike a balance between security and functionality, and are cost-effective and achievable. For example, your assessor should not recommend an expensive, complicated measure, such as modifying an information system's TCP/IP stack, in order to mitigate a vulnerability that has a low likelihood of exploitation.

    Define the scope of the assessment

    Once you've identified an assessor, sit down with him and define and document exactly what will be covered. Do you want to evaluate only certain servers on your network or do you want to review all of your information systems and security practices? A vulnerability assessment can include one or more of the following:

    • Detection and identification of information system vulnerabilities, both from the Internet and from an organization's internal network
    • Detection and identification of open ports and available services on specific information systems
    • Detection and identification of specific application vulnerabilities
    • Detection and identification of modems (for war dialing)
    • Attempts to obtain unauthorized data or access from an organization's employees (social engineering attempts)
    • Attempts to gain unauthorized physical access to an organization's information systems (physical penetration test)

    In general, it's better to conduct the most comprehensive evaluation possible, but political and financial considerations may not always allow this. You should define and document an assessment that is reasonable and appropriate for your organization. The scope documentation provides a framework for the assessment and can be used as a baseline for future assessments.

    Set rules of engagement

    Next, define the rules that will govern the assessment. Typical questions that need to be answered include:

    • Should discovered vulnerabilities be exploited or only recorded?
    • What type of attack methods can be used (social engineering, denial of service, war dialing, etc.)?
    • At what times can the assessment occur?
    • Are there certain types of information systems that should be excluded from the assessment (e.g., those providing medical services)?

    The rules should be appropriate and reasonable for your organization and should support the overall scope of the assessment.

    Defined and documented rules of engagement are necessary to ensure that a vulnerability assessment does not disrupt your organization. A high quality assessor never exceeds the rules. Avoid assessors who are unwilling to establish rules of engagement.

    Identify vulnerabilities that require immediate notification

    All vulnerabilities are not equal. Some clearly pose more risk than others. A high-quality assessor will interpret and prioritize discovered vulnerabilities so that your organization can focus on the important ones. Your assessor should also explain the risks of specific vulnerabilities so that their prioritization is understood.

    On the other hand, the assessor should not wait to put serious vulnerabilities into a final report. For example, you should be notified immediately of a vulnerability in a database containing significant amounts of financial data that will likely and easily result in the misuse or abuse of the data from the Internet. Expeditious reporting will enable you to quickly mitigate these threats. You should work with the assessor to define and document the types of vulnerabilities that need to be reported quickly, as well as how and to whom the report will be made.

    Vulnerability assessments are crucial for ensuring the security of your information systems and should be done on a regular basis. Follow these suggestions and you'll receive a high quality vulnerability assessment that reasonably and efficiently identifies vulnerabilities on your information systems and presents realistic and cost-effective measures to mitigate them.

    About the author
    Steven Weil, CISSP, CISA, CBCP is senior security consultant with Seitel Leeds & Associates, a full service consulting firm based in Seattle, Wash. Steven specializes in the areas of security policy development, HIPAA compliance, disaster recovery planning and security assessments. He can be reached at

    Also written by Steven Weil…
This was last published in December 2003

Dig Deeper on Risk assessments, metrics and frameworks

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.