The beauty of database marketing is that we can get better and better at what we do by using tests and controls. As serious marketers, we know that every single promotion to customers should be a test. We may think that what we are doing is wonderful, but the only real proof of merit is if the customers respond and make purchases. Success in the market is the only valid measure of our promotions.

What this means, in practice, is that your customer database must be riddled with control groups– people who are just like everyone else, except they don’t get the same promotion that is being sent to thousands of others. Only in this way can we learn whether the promotion is worth while. Let me give you some great case studies.

A company I worked for maintained the customer database for a large financial services firm. Their ad agency decided that to stimulate customer loyalty, they would provide a plastic card to all customers who had used this company’s services three or more times. The card gave holders the opportunity to cash checks and other benefits. After three years, the agency came to me saying that the financial services client was asking for proof that the cards were building loyalty.  “We know the cards are doing their job, but how can we prove it?” they asked.

“Just compare the retention rate and purchases of those who have the cards with those of the control group.” I responded. “What is a control group?” They asked. “It is the customers who used the service three times or more who were not given the card, but set aside as a control group.” I responded.

“Oh. There was no such group. We gave the cards to everyone who qualified for them,” was the response.  “Then, you are out of luck,” I replied. Needless to say, the agency lost the account.

A lighting fixture manufacturer with had 45,000 contractors and builders to whom they sent catalogs. Their business was excellent. A consulting firm suggested a test that might demonstrate how to improve sales. They took the top 1,200 customers and split them into two groups of 600: a test group and a control group. They set aside a two person team to make phone calls to the 600 companies in the test group, talking to the decision makers about their lighting products: What do you like? What more do you want? Do you need training? etc.  The control group got no such phone calls, only great service when the customers placed catalog orders.  After six months, those in the test group had purchased $2.6 million more worth of orders than those in the control group.  This test proved that building a relationship with the decision makers really paid off. But without the control group, the company would have learned nothing.

Creating a control group is not easy. You probably have several marketers who are dreaming up promotions by both direct mail and email. When the promotion is ready and approved, they want to shoot it out to as many people as possible. Telling them to set aside a control group may just be annoying to them. “We are not doing market research, we are doing marketing!” they may exclaim.  Wrong. We must all be in the research business if we want to keep our jobs. Someone in the organization must be in charge of coding some customers as being in control groups, and imposing the discipline on others to respect the control groups, and not to send their promotions to it.

Control groups usually should last longer than a single campaign.  “Did the people who were invited to the member night, but did not come have a higher retention rate than those who were not invited at all? How did their performance compare to those who actually showed up?”  These are valid and very important questions. Using control groups you might be able to show that the member night invitations were more powerful than the member night itself. This is powerful knowledge.

A telecom company was trying to reduce its churn rate.  They singled out a group of customers with a high lifetime value and a high predicted likelihood of defecting to another provider. The plan was to send sent letters and make phone calls to this group. To make sure that the customers realized that their communications were sincere, they sugared some of their messages with free minutes. How many free minutes should they offer? Using controls, they divided their target audience into four groups: those who got no calls or letters at all, those who got calls and letters with 200 free minutes, those who got calls and letters with 50 free minutes, and those who just got the calls and letters. Which worked the best? It turned out that the calls and letters were useful at reducing the churn rate. Those with the free minutes did much better than those with none. But there was no difference in the churn rate of those who got 50 or 200 minutes. In other words, people liked to hear pro actively from their phone company. They changed their behavior more when free minutes were involved, but the amount of the free minutes made no difference. None of this learning would have been possible without the control group.

You can get into trouble with control groups. When Nieman Marcus set up the Regards program for the top 5% of their customers, they set up a control group to prove that the Regards program was worth the investment. Of course some of their best customers who were not in the program heard about it, and were incensed that they were left out. They called the company and complained. In each case, the customer service reps were instructed to tell the callers that “There has been some sort of computer error. We will put you in the program right now. We apologize for the mistake.” Internally, the marketing staff called these people the “out of control group”.

So how big should your control groups be?  They should be big enough that you can be confident that their performance is indicative of the response of the audience. I use a rule of thumb: 500 responses. If your typical response from a test group is 2%, then your control group should have 25,000 in it (500 / .02).  Of course, during the year you have to rotate people into and out of control groups. If a customer is always in a control group and never hears from you, this customer will end up not being representative of your customer base as a whole.  Managing the control groups should be assigned to a staff member who takes it very seriously and lays out an annual plan.

So, those of you who are reading this should now be thinking: “Do we have a good control group system in our organization? Who is in charge of it? What have we learned from it and how can we use it in the future to improve our marketing strategy?” We are not talking about spending a lot of money.  Control groups need not be expensive. They just take internal discipline, and a determination to use them to get better and better at our jobs.

Post Footer automatically generated by Add Post Footer Plugin for wordpress.