More on generating a Starter Compliance Matrix – Real world usage statistics

In March this year, I blogged on whether you could generate a starter compliance matrix in under 10 minutes.  In that post, I showed results based on work we did during 2011 with a sample of 28 organizations, mostly US federal contractors. We looked at the area of compliance matrices and how we could look to automate an initial matrix. If you didn’t read that post, it’s worth having a quick look. Don’t worry; I’ll be here when you get back.

Now, here is an extract from a recent e-mail I received from Kevin Plexico, VP at Deltek.

“We had 1,775 uses across 1,188 users.  There were 33 users who used it more than 5 times in the month. It was used on 943 unique opportunities, so clearly used across a broad spectrum of our opportunities.”

These are usage statistics for a new starter compliance matrix capability introduced into GovWinIQ on June 6th this year. These stats cover a 30 day period between June and July. That starter compliance matrix is actually powered by VisibleThread using an API (Application Programming Interface) into our language analysis engine. So, as a follow-up to my original post on a starter matrix in under 10 minutes, I wanted to share additional research on this area and provide actual usage statistics since going live with the capability in June. We now see proposal professionals gaining daily advantages through this new automated capability.

If you’re dealing with compliance matrices and want to get a head-start with your own compliance matrix efforts, this will be of interest.

What is a Compliance Matrix?

It’s a basic question, but useful to set the context.

For Proposal Professionals, creating a good Compliance Matrix is a critical first step when developing a winning proposal. The matrix is basically a spreadsheet that companies use to ensure all the requirements in an RFP are satisfied in a proposal. It is typically hand-generated by someone manually reviewing each page of the RFP, examining it for any requirements. It is a time-consuming process.

More Research on what happens today and why we partnered with Deltek.

I already had the benefit of seeing first-hand how VisibleThread customers were saving many hours by automating the generation process. After all, 3 of the 5 top US Gov Contractors use the capability. Our smaller customers equally see big-time savings.

Deltek has 15,000 customers worldwide and specializes in supporting government contractors through its ERP and general product range. So, the idea of generating a starter compliance matrix had much appeal for our friends at Deltek too.

So, we started working with Kevin Plexico and the guys at Deltek in January 2012. We wanted to explore the idea of exposing the capability to Deltek GovWIn IQ customers. Would it offer value?

We first met a number of joint customers and specifically heard how VisibleThread customers such as Buchanan and Edwards were automatically generating a first-pass compliance matrix in minutes, savings days of effort.

We also conducted a survey in March 2012 with subscribers of Deltek’s GovWinIQ product:

Question 1: What are people doing today to create a compliance matrix?

Answer: 91% (52 of the 57 respondents) manually create the matrix by hand.

Here are the more granular results:

Question 2: Was the automated solution we had in mind going to add value?

We provided a sample-generated starter compliance matrix for two different government solicitations; CTS and TAASC II. We asked respondents to rate whether the sample matrices were useful on a scale of 1 to 5.

5 is best, 1 worst.

Answer: 63% (42% very helpful + 21% helpful) viewed the matrix as either very helpful or helpful.

Here is the detail:

So, this validation was comprehensive and led Deltek and VisibleThread to announce a strategic agreement on May 9th. We exposed the new capability powered by VisibleThread to GovWinIQ customers on June 6^ththis year.

What do you get when you generate the matrix?

Here is a sample compliance matrix generated from our integration into GovWinIQ:

Usage Statistics since going live in June

If we look at our logs over the last 32 days (between July 7^th through August 10th), these are the results for the number of calls to generate a ‘starter compliance matrix’.

You can see a fairly consistent usage. The first row of data below the graph shows the day number. The 2^nd row shows the actual number of API calls. Each API call results in 1 or more starter matrices returned to the user.

You may be wondering why the dips. These low points correspond with weekends. So we’re running at a daily rate of about 50 calls for weekdays, Monday thru Friday. Each call may be a request to generate more than 1 document.

So, this means that each day, GovWin IQ customers generate well in excess of 50 starter matrices using the VisibleThread capability.

Summary Takeaways

I will leave you with some of my own summary observations:

• We started out feeling that a starter compliance matrix would add a lot of value. This is why VisibleThread and Deltek started exploring this area. We both wanted to validate if this was a fair assumption by asking customers using actual generated matrices.
• 63% of customers considered it to be very helpful or helpful. The usage statistics since going live show a pattern of at least 50 matrices generated each day.
• The business benefit of generating a starter compliance matrix is clear for many government contractors.

If you want to see how we scan documents and offer value for Capture and Proposal professionals, check out these 3-minute demos.