Ventana also put the sales forecasting problem in the context of the retailer’s overall product distribution system. We showed how the tradeoffs among inventory levels, stock-outs, and product wastage or obsolescence operated in relation to the company’s overall operating policies. This showed several opportunities to improve operational performance by re-designing the supply system and its logistics algorithms and policies. With the improvements the client could determine how much of the benefit to take in decreased inventories, stock-outs or wastage.

Ventana’s contributions were made possible by creation of a simulation model that organized the client’s own data and business knowledge in a form that (a) allowed the performance of the existing sales forecasting system to be well benchmarked for the first time, and (b) gave the client a better understanding of how the business worked as a whole. This new understanding revealed that unjustified, but widely held, textbook prescriptions (and conventional wisdom) about customer behavior and inventory management were limiting the retailer’s profitability. During this project, the company implemented several of Ventana’s innovations in multi-store operational tests. These tests demonstrated the ability to reduce system-wide inventories by 50% without lowering service levels. As a result, Ventana algorithms were implemented throughout the company.

The post Stocking Algorithms appeared first on Ventana Systems, Inc..

Also at issue was the impact of the distribution of redemptions over time. More specifically, the existence of the frequent guest program was most valuable to large business and airport hotels where guests earned a lot of points but relatively infrequently had to redeem them. Conversely, resort hotels redeemed lots of points, but relatively few points were earned there. Since most of the hotels were franchises, the rules of the frequent guest program needed to be perceived as fair to all parties for the frequent guest program to be successful.

Ventana built a simulation model to study the liability and fairness issues. We discovered that the extant auditor model was flawed. While it seemed to perform correctly for the prevalent conditions, it failed a reality check if conditions changed and program members started drawing down their point balances. Nevertheless, the Ventana assessment found the companies liability reserves sufficient and that with appropriate business rules an expansion of the frequent guest program was financially responsible. Subsequently, the company expanded the program, and it became one of the most successful loyalty programs in any industry.

Ten years later a division of the hotel chain asked Ventana to reexamine the liability issue for one unique segment of its business. The sales department of this segment wanted to reduce the size of its liability reserve account to provide new funding to the sales department. The auditor’s model predicted what all company managers felt was an unreasonably high ultimate redemption rate for the points.

Ventana was gratified to learn that over the previous decade the auditor’s redemption model had changed to one very similar to what we had proposed a decade earlier. Moreover, the intuitive case the company managers built for reducing the liability research for the one unique business segment seemed sound.

Ventana built a new simulation model and examined the historical data covering the earning and redeeming of points for the business segment, including unique rules governing the operation of the segment’s loyalty program. This time the assessment was not favorable to the company’s sales and marketing staffs. Not only could we find no basis for reducing the liability, the model predicted that the ultimate fraction of points redeemed would continue to climb over time. This would necessitate continually increasing the liability reserve. These initially counterintuitive results devolved from the unique rules adopted for the loyalty program of this particular business segment a decade earlier. These rules produced no apparent serious consequences for several years after they were adopted, but the seeds of a long-term problem were sown. There were already early signs of the emerging trend in the liability, but the year-to-year changes were so small that managers did not notice the signs (sometimes called “boiled frog” syndrome). Moreover, there was no way to redress a serious imbalance in the corporate program without changing the redemption rules. This was deemed infeasible by the segment’s CFO, although the issue scheduled for periodic review. The business segment had to live with the situation, and the liability account was allowed to grow as required.

The post Calculating Liability Reserves for Loyalty Programs appeared first on Ventana Systems, Inc..

Ventana’s almost universal finding is that the proper interpretation of what survey respondents are trying to communicate is so hard to fathom that results are only interpretable in the context of models that also incorporate “hard” quantitative data. Indeed, in a few instances, the survey data has proven so unreliable that it is unclear that respondents were trying to convey accurate information. (This is one reason that personal interviews and focus group situations remain such an important part of Ventana’s consulting practice.)

While Ventana experience and conclusions about survey data are consistent with academic psychological research as well as research focusing on voter opinion/behavior, we find that many organizations with whom we work are consistently fooled by the survey results. One reason is that survey work is almost always done as an exercise that stands apart from other research, and little effort is devoted to corroborating or disproving survey findings. “They are what they are – let the buyer beware.” This is distinctly not Ventana’s approach, which requires more systemic understanding by using a variety of data types and models. As a consequence, in working with clients, we have discovered ways to revise the surveys, the sampling, and the analysis procedures to get more of the knowledge the clients are seeking, more reliably, for less money.

Customer satisfaction surveys

In analyzing customer surveys, Ventana’s approach is to build a small model connecting individuals’ detail responses to overall perceptions about a product or service, their stated purchasing intentions, and “hard data” on purchasing behavior. We have done this for both consumer and B2B purchasers. Typically, but not always, respondents show excellent internal consistency in the way they answer a battery of questions. Yet, the external validity proves to be terrible despite serious attempts to rescale, renormalize and/or reanalyze the data from various perspectives. This inconsistency between what respondents report and how they behave can be vexing. Examples abound. Purchase managers give a vendor a perfect score and then give most of their business to a different supplier; purchase managers improve their score for a vendor by a factor of two, but they decrease their business volume with the vendor and give it to a company with a declining score. Hotel guests rate hotels at the low end of their scale on all dimensions. Yet, they continue to frequent them; or conversely, they state their intention to never visit a hotel again – but do so within the next quarter. Large consumer panels can rate products low that have dominant market shares and vice versa.

Ventana shows why common procedures for interpreting surveys give wrong answers for some types of data, and provide more reliable guidelines. Properly analyzed, the survey results show strong evidence that common survey formats, while effective for examining details of the customer experience or measuring the penetration of marketing messages, are poor instruments for determining the key drivers of customer choice. Understanding the causes of customer behavior requires both a dedicated and markedly different surveying scheme, and an analytical approach that triangulates survey data against other kinds of data.

Physician surveys (use of “expert” opinion)

Ventana has worked with more of these surveys than any other kind over the last decade. The survey issues are basically the same as customer surveys, but a physician is generally considered a medical “expert”, and surveys delve into a variety of very technical topics. Yet, like other “customer” surveys, they manifest vexing interpretation problems. Even their internal consistency can be poor. The syndicators that perform many of the widely distributed survey results (as opposed to the primary research of individual pharmaceutical companies) often provide what they call a “derived satisfaction” analysis in addition to the actual respondents’ data. While provided as a kind of “bonus” supplement, more often than not, this analysis essentially shows that physicians’ stated overall satisfaction with a given product is inconsistent with their detailed responses to questions about individual product attributes.

Far more important, however, are the external validity issues with surveys. When survey responses for individual physicians are compared with their actual prescribing behavior there can be startling discrepancies. In one instance, Ventana discovered that physicians did not even accurately report the medication they prescribed. Indeed, the discrepancies were so great that the linking between individual physician responses and their prescribing records had to be triple-checked, because no one could believe the magnitude of the differences. Physicians are busy people and syndicated surveys tend to be long and detailed. And even though respondents are paid for their answers, inattention and memory dynamics influence how questions are answered. Ventana must account for these influences in its simulation models in order to explain market behavior.

More important interpretation issues emerge from the detailed technical content of questions and responses. The ordering of questions and very subtle differences in language affect responses. Moreover, physician motivation, legal issues, and the obvious “test taking” acumen of physician candidates influence the way questions are answered. For example, consider physicians’ use of narcotics. These are very commonly prescribed drugs, but trying to figure out when and why physicians use them using a survey instrument is hard. Public policy, legal and popularity with patients converge to make straightforward responses problematic for the honest physician.

Employee surveys

One key finding is that despite organizational pressures to conduct only a few “special” surveys, companies are better served by using more frequent surveys and controlling the cost of this practice by scientifically sampling employees for each survey. There are several reasons for surveying employees, and not all reasons apply to all surveys, but in Ventana’s experience at least one of them will apply in a given circumstance. For example, in trying to assess the impact of morale on an aerospace company’s manufacturing productivity, we found intermittent to annual surveying to provide no useful information about morale’s impact. In the rare cases where monthly or quarterly surveys were available, however, we could resolve the impact. The reason was that with the more frequent measurement we could distinguish the difference between the longer term trends in feelings and the ephemeral gripes and joys of a given day or week.

The post Improving Survey Data appeared first on Ventana Systems, Inc..