An in-depth SaaS evaluation involves stakeholders and criteria matrix which help you pick the right Software-as-a-Service solution based on your specific needs.
► Find out how to evaluate SaaS applications!
With more and more companies migrating to the cloud and implementing SaaS technology, the market is quickly adjusting to the increased demand by constantly offering new SaaS products that seem similar in function and features. However, if you look closely, “minor” details like an add-on feature, the subscription and renewal terms, or a security policy can determine whether the software is adding value to your company or just adding to a pile of hidden costs.
So, in order to choose the right cloud-based software that meets your organization’s needs, it is extremely important to conduct a proper SaaS evaluation before onboarding a new vendor. An evaluation starts when an employee has a software request or there is a new business need. Going through the evaluation process will ensure that aspects like functionality, compliance, security demands, service, and cost are met, saving you from future headaches that could have been avoided with a good SaaS evaluation matrix.
Read on and find out how SaaS is evaluated before procurement and how to create your own SaaS evaluation matrix.
With increasing digital literacy in any company, shadow IT has become a real issue within many organizations. This lack of oversight also leads to multiple SaaS products with overlapping functionalities.
Before SaaS procurement, make sure that the evaluated product isn’t already being used in your company or whether there is a similar one that could satisfy the respective business need. Sometimes, existing solutions let you add on additional features for a small price increase. In case there is still a need for the new SaaS application, you should establish an evaluation framework that will come in handy for any future SaaS evaluations.
The scope of evaluation depends on the specific tool, business process change, and the data that is processed and stored. Therefore, evaluations can be separated into two scopes:
Limited scope evaluations: Applications that fit into this scope don't affect business processes for the whole organization or departments. These apps are only used by one or a few employees (e.g. social media scheduling app, educational app, etc.)
These evaluations require IT and InfoSec teams to be involved but without the finance team or department leaders since the price tag and risk are usually lower. Below are some basic SaaS information requirements that make the process much easier when an employee requests a new limited scope SaaS evaluation:
Full scope evaluations: Applications that fit into this scope do affect business processes and are used across the organization or within departments (e.g. HR systems, ERPs, CRMs, content management systems, etc.)
During the full scope evaluation stage, it is vital to involve the IT, finance, and information security teams together with end-users (e.g. for marketing applications it can be CMO, director, or managers). They should receive all the relevant information to ensure a swift Software-as-a-Service evaluation.
Each evaluation should follow a risk-based approach to ensure compliance with IT contractual requirements and industry best practices.
The more information your teams receive, the easier and faster the evaluation can be done. Note that the amount of effort and information should align with the criticality and price point of the software product.
In the following, we are detailing 3 critical SaaS selection steps.
As you can imagine, procuring SaaS single-handedly without consulting other stakeholders like InfoSec and legal teams, procurement or IT is not the best idea. After all, unknown SaaS exposes companies to a whole new set of risks that need to be addressed before it’s too late in the game. Always ask yourself who the end-user is and how they will benefit from the new software application.
Whether it’s through surveys, brainstorming sessions, or one-on-one feedback – understanding other stakeholders, their pain points, and the rules they need to comply with, helps speed up the software application evaluation process and the development of a sound evaluation framework. In order to facilitate future changes, negotiations, and administration of the SaaS application, it is also vital to establish the owner of the SaaS product.
Needless to say, knowing the business purpose of a new SaaS application is important. After all, understanding the “why” will make it a lot easier to tackle the “how” and thus, eliminate all irrelevant options. Let the following questions guide you through this stage:
As previously mentioned, there might be a good chance that your organization is already using a similar app that could be modified to meet new business needs. Next, learn how to create a software evaluation matrix.
White PaperA Guide to SaaS Management
White PaperSaaS Management for CIOs and CFOs
In order to be able to properly evaluate different SaaS products and their vendors, it’s always helpful to create an evaluation matrix that contains all the SaaS evaluation criteria that matter to the health and prosperity of the organization. Each SaaS option that is short-listed for procurement should receive scores based on its:
Create a scorecard for each vendor that rates the application on each criteria and assign weights to each one.
It’s important to note that categories from 1 to 4 contain sub-categories as well. For security, you could add the items SOC 2 compliance, GDPR compliance, Single Sign-On Integration, and Multi-factor authentication.
Rate your SaaS vendor for these features and assign the urgency/importance to each criteria as one aspect might be more important than another one. This will help you get the best vendor for your needs.
Below is a formula and a first look at the evaluation template that you can download at the bottom of this page. You can use this formula to calculate a weighted score for each vendor you're evaluating:
Vendor's Grade x Urgency = Weighted score (Vendor Assessment)
Add the criteria for your vendor.
Rate your vendor for the criteria below, (from 1 to 5).
Rate the importance of each criteria to your organization, (from 1 to 5).
Final vendor assessment (Multiply two columns on the left).
(Criteria example #1)
(Criteria example #1)
(Sum up the numbers in the "Vendor Assessment" column).
Next, we’ll teach you how to establish the SaaS evaluation criteria that belong in your software evaluation matrix.
When it comes to software, there are some features that are nice to have and others that you probably shouldn’t budge on. In order to find out whether the evaluated software meets your SaaS evaluation criteria in terms of functionality, you should clearly define all must-have features beforehand with the application end-users. These can vary from company to company, so make sure to keep in mind individual workflows and what kind of tools teams are currently using to perform a certain task.
Example: If you are looking for new e-commerce software, then a checkout system, a shopping cart module, and a nice gallery layout for your products are must-have features. Instead of getting too caught up in what is best for your teams, don’t forget to ask yourself what the end-user needs and which features would create the best user experience. This is where survey results from the stage “Involve other stakeholders” come in handy. It’s okay to spend a good amount of time on this exercise as it lays the foundation for the following steps.
SaaS is convenient and scalable but it also removes physical security barriers that protect your data when it's stored on-premise. So, it doesn’t matter how many great features a SaaS application has, if it’s not compliant with security certifications or doesn’t meet the regulatory standards of your company, it puts your organization’s health at risk. That’s why the security of SaaS solutions should be the most important aspect of your evaluation and why working with InfoSec is a must. Before you add security criteria to your software evaluation matrix, you can ask yourself the following questions:
You should only move forward with procurement once you’ve performed a proper SaaS risk assessment and receive the approval of your internal security teams.
When you acquire a new SaaS solution you are also getting into a new business relationship with the respective vendor. Ideally, you want to build a lasting relationship that will benefit you for many years to come. Here are some questions you can ask yourself when evaluating a vendor in terms of service and availability:
This will help you in determining whether the SaaS vendor can offer you and your business the support and value you are looking for right now and in the future.
Your cost section in the software evaluation matrix looks at all the different factors that make up the SaaS total cost of ownership or SaaS TCO. This includes the following:
Once you’ve settled on an application after it passed your SaaS evaluation meeting all important criteria, it’s time to negotiate the SaaS contract, so you are getting the maximum value. As the market for cloud-based solutions is quite competitive, vendors are often willing to offer you more favorable conditions than their competitors.
After procurement and onboarding, make sure to perform annual reviews and assessments to keep an eye on pricing changes as well as contract obligations and to check whether compliance and security demands are met. You might even find that you need fewer seats or licenses and can re-negotiate the conditions before a software renewal takes place.
Evaluate and assess new or existing SaaS vendors based on their security or product features, cost, support, and service criteria important to your organization.
Compare and evaluate different SaaS vendors
Add your own evaluation criteria
Grade vendors based on the criteria importance and urgency
Select the SaaS vendor that fits your organization
How to evaluate enterprise software?
What are the key elements of the SaaS evaluation matrix?