Evaluating CMS Security

When evaluating content management systems (CMS) it is extremely important to include criteria covering security considerations. CMS'es are complex, and extremely powerful web applications, and as such present interesting security challenges. Although many of these challenges are not unique to CMS systems, they are often overlooked when performing product evaluations. CMS'es are quickly becoming the de facto standard for deploying web based information systems – from websites to complex web applications. Both their complexity and increasing market share make them attractive targets for malicious attackers.

In evaluating the security of any system it is important to understand that insecurity is a given. Software engineering is a well established field of computer science whose practitioners have done extensive work to quantify the incidence of bugs in code. The common measurement convention for this metric is KLOC or bugs per thousand lines of code. In various software development models one can find a predictable measurement of the number of bugs per KLOC. The resulting realization is that bugs are inevitable (if you don't believe me just look at the disastrous fate of software assurance projects, especially common criteria). A certain portion of bugs are security related, and thus every software will manifest a certain number of security bugs. Of course, these may range in severity so it cannot be said that all software is insecure, but it must be recognized that all software will contain security bugs.

Complicating matters is the fact that some security flaws are functionally correct. This is to say that software may include some functionality that presents a security concern, but is not actually coded wrong. In other words there might be a logic flaw that allows unprivileged users to elevate their access to that of privileged users by using a feature that was deliberately included in the software. These types of security issues can be especially difficult to identify without extensive quality assurance (QA).

In addition, software security is an evolving landscape. New attack methods are being discovered on a fairly regular basis. Attacks such as cross site scripting (XSS) were unheard of in the early days of the internet (and thus much of the software produced at that time is vulnerable to XSS). Because the security landscape is so fluid there is no way to certify that software is free of as yet undiscovered vulnerabilities. The old adage that debugging can only prove that no bugs were found, not that bugs are not present, holds with security as well.

What this all adds up to is a situation where you have software that contains a certain number of bugs, identifying complex security issues is difficult, and the cost of chasing edge bugs goes up as the total number of bugs decreases. Furthermore, even if one could certify that a certain piece of software was completely secure, that certification would only be good at the time it was issued. A yet to be discovered security flaw could be present in the software and it would be impossible to detect.

Considering all these factors, how does one go about choosing a CMS solution? The answer is to accept that any solution contains security problems. Realizing that all software is insecure realigns the focus of security evaluation on mitigation, defensive countermeasure, and other controls designed to detect, contain, and respond to security vulnerabilities. By examining these controls you can make an informed decision and choose the “safest” CMS solution as opposed to fruitlessly searching for a totally secure solution.

Product Demographics

One of the easiest metrics to examine when considering a CMS solution is the installation base. Ubiquity is an extremely important factor to consider when evaluating CMS solutions. The overall usage and deployment of a system greatly affects its security posture. If a system is widely deployed it may mean that it is well supported. It will also mean that the system is a larger target for malicious attackers. Attackers generally try to compromise the largest number of targets with the least amount of effort, so running the most common systems puts you in an attractive pool of targets for attackers. Conversely, a widely deployed system is also likely to have broader support. It may also mean that attacks are identified more quickly, and remediated, as opposed to attacks on less common systems, which may go undetected for some time.

Project activity is also a very useful metric. Projects that are stagnant are less likely to receive timely security updates than ones with a large development community and timely releases. Gauging community “noise” in terms of groups, mailing lists, IRC channels and other outlets, is a good way to measure project activity. A healthy development community will often mean that patches are produced in a more timely fashion. Examining release cycles is another good way to gauge project activity. Look at the time between releases as well as the features and fixes introduced by new releases. Try to figure our how product support works with respect to releases as well. Does the vendor support older versions of the product or are users required to upgrade to the latest version in order to receive support.

Product maturity is another extremely useful measurement when evaluation the security of systems. Longevity is not always the best measure of security, but longer lived products may mean that the system will have support available for some time to come. In additional to longevity one should also examine the product's vendor maturity. In order to do this one should benchmark the system to enterprise level software. An immature software development team may mean that certain concerns of enterprise software have not been considered by the vendor. Issues such as stability, support, and realistic understandings of functional business requirements are critical when ascertaining which CMS solution is right for your organization. Some useful measures for determining a vendor's software development maturity are the existence of an established security team, security contact, and secure development lifecycle. If a product does not have an established security vulnerability reporting and response procedure this could bode poorly for effective handling of security concerns. Be sure to check claims made by vendors, however. Even if a vendor claims to have an established security response procedure, if they don't actually abide by the process it is an indication of an immature vendor and/or product. Complaints by security researchers that a specific vendor is unresponsive to reports is particularly damaging evidence of a poor vendor maturity.

Architecture

Much of a systems security is tied to the actual architecture of a system. Try to evaluate how the system is engineered to include (or exclude) security related features and considerations. Many of these indicators will have side effects on other metrics, such as system stability and reliability, so they are of critical concern in any evaluation.

Modularity is a measure of how tightly integrated the components of a system are. Systems made up of one essential core of code are described as monolithic, whereas systems comprised of various independent parts are described as modular. Modularity should also be an important consideration when evaluating a CMS. Systems that are monolithic suffer from some interesting problems. Upgrading such systems becomes problematic as errors in the core code may cause the entire system to become inoperable. Modularity may also influence the impact of a security compromise. A tightly compartmentalized system may limit an attacker to certain functionality and thus contain a break in. Try to explore how modular security aspects of a candidate system may help to contain attackers. Modular permissions can help to limit users to least privilege, which is good security practice. Modularity may increase complexity, however, which could lead to a higher number of bugs in the code.

Patch management and system upgrades are important architectural issues when examining CMS security as well. The ease or difficulty of upgrade may influence how timely patches and fixes are applied to the system. If upgrades are extremely difficult and delicate, administrators will not be inclined to engage in patching without thorough testing, which means more time and resources must be devoted to keeping the system up to date. A recommended patching cycle from Project Quant (http://securosis.com/projectquant) outlines 10 distinct steps to take in managing your patching:

1. Monitor for advisories
2. Evaluate patches
3. Acquire the patch code
4. Prioritize and schedule patches
5. Test and approve patches
6. Create and test deploy patches
7. Deploy patches live
8. Confirm the patches work
9. Clean up
10. Document patch process

It is realistic to expect that you will have to patch and upgrade your CMS system regularly. Be sure to scrutinize all candidate systems and make sure that implementing each step of the above 10 step process is feasible within your current architecture and organization. Pay special heed to mechanisms that will allow you to easily monitor for new patches as well as systems that allow you to easily test, deploy, and roll back patches. Patch management can play a critical role in limiting windows of vulnerability between a newly discovered exploit, a patch release, and the bad guys searching for vulnerable systems on the internet.

Compartmentalization of CMS systems is also an important security concern. How tightly integrated is the CMS with the operating system, web server, databases, and even programming languages. Each of these components may need to be patched, upgraded, and altered from time to time and inter operation between the CMS and components may lead to the introduction of security flaws. Also be sure to examine the security of each of the components that support the CMS. Even if a CMS is rock solid, if it depends on an unreliable operating system or database prone to security vulnerabilities, then the components will degrade the overall security of the CMS solution.

In general, complexity is the enemy of security. The more complex a system is, the more likely bugs will occur in the code, and additionally the more difficult an administrator and users job becomes. With higher complexity, human error becomes more likely. Be sure to examine the overall complexity of a system. Complexity can be mitigated by easy to use controls, however. A highly complex system may have expertly designed interface and controls that help alleviate the overall complexity of a system and mitigate many of the misconfiguration concerns that might otherwise be present. Despite such controls, however, complex systems may make debugging problems and overall administration difficult and error prone.

Security Specific Criteria

When evaluating any system it is common to try to measure the so called “security” of systems under consideration rather than security controls. One metric that is very tempting to examine is the total number of known vulnerabilities. This is, unfortunately, an easy metric to quantify and a poor metric for decision making. Comparing the number of known vulnerabilities doesn't reveal much useful data about a product. One often assumes that a large number of vulnerabilities is indicative of an insecure system. However, a low number of known vulnerabilities may simply mean that no skilled testers have looked at a specific system. Conversely a high number of reported vulnerabilities may be indicative of a concerted, and successful, security evaluation. At the same time a high number of vulnerabilities may indicate a widely used product with a poor commitment to security. Overall it is extremely difficult to infer the meaning behind this type of metric and therefore it is dangerous to use in decision making.

A much more meaningful metric is the mean time to patch. This is the time between a vulnerability being reported or becoming public, and the time at which a patch is available to fix the issue. The lower the mean time to patch, the more responsive the system vendor, which generally equates to a better security response.

Another useful metric to examine is the severity of reported vulnerabilities. If vulnerabilities are generally low in terms of impact that may mean that the system is more secure. It could also indicate proactive security review by the vendor that is successfully mitigating easy to spot vulnerabilities. It is also useful to look at the source of vulnerability reports. A large number of internally (that is, by the vendor themselves) reported and fixed vulnerabilities is indicative of an active security process.

Configuration Considerations

Management of users and privileges is also an important consideration when evaluating CMS solutions. Privilege separation and granularity of permissions allows finely tuned access controls which can simplify interface for users, confound and compartmentalize attackers, and satisfy the principle of least privilege. It is worth examining whether or not dangerous permissions can be effectively regulated, or even disabled, within the system. Be sure to review the complexity of the permissions interfaces as well. No matter how good such systems may be, if they are poorly designed and difficult to use then errors will be more likely, contravening any security gains the system might provide.

Configuration is another important factor to weigh when evaluating systems. Security misconfiguration is increasing problematic with complex systems. If a system is sufficiently difficult to use then administrators may inadvertently introduce vulnerabilities in the system that stem from misconfiguration rather than an inherent security flaw. Be sure to evaluate the ease or difficulty of configuration as well as measure the availability of configuration guides and/or guidelines. Try to determine if the default installation configuration is secure or if administrators are expected to apply security mitigations after deployment. If the latter is the case, this may indicate a propensity for insecure configuration as administrators are expected to be security experts (and apply appropriate configurations) in order to secure the system. Insecure by default systems can be dangerous and lead to numerous headaches. For instance, if the system is upgraded to mitigating configurations need to be re-applied?

Testing

One important criteria that often escapes notice is the ease of security testing for systems. No CMS is stagnant, or if it is, it is certain to become a security concern over time. Automated web application testing technology is still very much in its infancy. As a result, manual security testing is still the most effective way to proactively identify security bugs in a system. Realize, however, that manual testing may not always mean manual code review. Even if you have access to the source code of a system, sufficiently complex systems contain so much code that it may be impossible to manually review the code even if it is available. Because of this testing should be divided into easily manageable stages. Try testing the core system first before testing any additions or components. Plan on testing all patches and upgrades before application. It is vital to commit to a continuous security testing cycle in order to ensure a safe system. Be sure to evaluate the feasibility of these steps during any product evaluation. If assets are not available for this sort of testing within your organization be sure to check whether or not it is possible to leverage others' efforts to your advantage. Open source projects often excel at this sort of sharing, but any strong community is likely to provide similar resources.

As mentioned before, a commitment to security is critical. CMS'es exist in the continuously evolving web landscape. Just as the web evolves, so does security. Try to look beyond the specific CMS product to a technology, process, and configuration lifecycle that will allow your specific system choice to subsist in a healthy ecosystem.

Conclusion

There are many different considerations to take into account when examining the security of CMS products. CMS solutions are sufficiently complex to make security flaws inevitable. Given that security concerns will be part of a product implementation, it is important to examine security mechanisms in CMS solutions with particular scrutiny. Any effective evaluation should focus on security mechanisms designed to mitigate risk. By accepting that vulnerabilities may exist, or be discovered, in any system, it becomes apparent that risk controls are paramount when evaluating the security of a system.