How to measure the impact of Office 365

Deployment is easy. Quantifying the tangible benefits is difficult.

Here are some practical and usable guidelines for measuring the impact of Office 365  based upon my experience.

Estimated reading time 8 min

Measuring the benefits: Two components

Identifying the benefits and Measuring the benefits. When it comes to Office tools, both these items are difficult to assess and quantify in a repeatable and standardized manner. There are no benchmarks widely available.

Even success criteria are not well defined. Conceptually speaking, defining a set of success criteria is a fairly simple process. Whatever technology you introduce / upgrade should increase the efficiency of users in a tangible, demonstrable and repeatable way.

Quantifying Efficiency

However, quantifying efficiency increase is difficult because Office is used for performing unstructured work. Structured work is already optimized: It is either automated, outsourced or optimized (Business Process Re-engineering).

Office related unstructured work takes up 50-100% of user time (depending upon the role). Unfortunately, no review, audit or optimization effort has ever been applied to this critical area.

Why? Because everyone thinks they are using it optimally.
Somehow, work is getting done.

Typically, Office and related products are purchased, migrated to, deployed and maintained, for reasons OTHER THAN genuine desire to improve efficiency. These reasons could include:

1. Converting licensing expenditure from Capex to Opex
2. Minimizing the operational cost in having own data centers and dedicated IT staff
3. Taking advantage of multiple device licensing of Office 365
4. Migrating from non-Microsoft messaging systems
5. Legalization / Compliance

Very rarely does a company go in for the Office 365 platform with a specific, well defined and business mapped – efficiency improvement agenda.

Therefore, signing up for Office 365 itself sounds like fulfilling the objective for the decision. Few short (and inadequate) training programs and launch activities are performed. All users are not involved in these activities. Therefore awareness about the new platform is scattered and low. And that’s about it.

The question of demonstrating the value of unstructured work optimization is being discussed only recently. As this has never been done before, all existing approaches are essentially ineffective. We need a new method.

New approach

Business value derived from Office 365 can be divided into two categories:

1. improvement over existing process OR
2. introduction of a completely new capability which is useful for business

Business value can also be of at least two types:

1. Time Saving
2. Intangible but more important gains like
   1. improved accuracy
   2. ability to detect abnormal events
   3. ability to find out useful but hitherto unnoticed information
      (the jargon for this is “BI”)
   4. ability to analyze larger data to find patterns which were impossible earlier due to the sheer size of data (jargon = “Big Data”)
   5. elimination of some controls thus releasing manpower (e.g., Lock Tracking in Word eliminates the need to compare documents)
   6. utilizing existing manpower for higher level business specific activities rather than mundane but necessary activities( Speak Cells on Enter feature in Excel eliminates the need for supervisors – called Checkers).

Quantifying time saving

This is easy. Look at the activities which improved and multiply time saving by frequency and number of people affected.

Of course, this requires effective use of RELEVANT features across the platform to perform specific activities. This is easier said than done. In fact, to my knowledge, this activity is never performed in a structured and organization-wide manner!

For activities which are improvement over existing process, the business value is the ratio of new capability usable vs. older capability usage. Usually, with appropriate logging, this information is easily available.

For new capability, more detailed analysis is required. For example, for enterprise search just counting queries is not a direct indication of usage. We must also analyze the queries to see if appropriate filtering syntax is being used. If not we must find the reason – users don’t know it or they found it to be ineffective.

Finally, the quantification must be continued in the long run. Usually such quantification, if performed, stops after the initial excitement wears off.

Quantifying intangible benefits

Things like improved quality, new capability, better business management as a result of new insights derived from existing data are difficult to quantify in monetary terms.

However, these are the REAL benefits – because these directly affect business outcomes in a favorable manner. Time saving, on the other hand, may look very big and impressive. But in itself time saving does not lead to either personal or organizational growth. The time saved needs to be channeled into something value creating.

Exact conversion of intangible benefits to monetary terms should be decided individually for each customer. The conversion ratio is highly subjective, industry specific and process specific. Global benchmarks cannot be created for this conversion process.

However, the benchmark, once evolved must be used in a uniform manner to avoid calculation errors and bias.

Long term monitoring is essential

Even if all this is done, there is always a danger of this activity fizzling out after the initial excitement wears off. Therefore, there must be a team responsible for ongoing monitoring and calculation of benefits and delivered business value.

Caution: Do not ignore Office 2013

Office 365 consists of a large number of products and services. Often the core product – Office 2013 itself is not taken into account while deriving and quantifying business impact. All other products, although great in their own right, are useless unless Office is used effectively. Therefore, make sure that the adoption, consumption and quantification process starts with Office tools and then incorporates the server side tools like Exchange, SharePoint, Lync, Delve, Videos, OneDrive and so on.

Case Study

This is the evidence of Office client based efficiency improvement, by business activity for a global bank. The time saving shows REAL results. No programming or macros were used. If SharePoint is included, the time saving will increase even further and yield intangible improvements (which have not been considered in the calculation below).

***