• Alignment: Ventana models are focused on client choices and client performance. Inputs include all the organization's options for action, and the outputs describe outcomes by the organization's measures of success. The models are built through a responsive, flexible process to align the model with the organization's decision needs.
• Reliability: Ventana models are industrial-strength and robust, to help clients consistently choose actions that maximize the odds of success -- even in conditions never before encountered.
• Persuasiveness: Ventana models inspire confidence through demonstrable consistency with expert knowledge; through output that supports compelling, data-supported stories about what will happen and why; through realistic, easily visualized internal mechanics; and through an inclusive development process that earns the trust of the organization.
• Organizational Fit: Ventana models fit the way the organization's people spend their time and make decisions, relate to the way they think about their business, and inspire them to use the model to its full advantage.
Why Ventana Models Work
Smart choices come from solid understanding and clear communication of which actions will improve performance and why. Four qualities of Ventana® models combine to yield better performance through smart choices:
- Alignment
- Reliability
- Persuasiveness
- Organizational Fit
Focused on the choices that will improve performance, Ventana models include as inputs all the potential actions the organization can take, and as outputs, the organization's measures of success. Visibility, flexibility, and accountability while designing the models assure that the model system is a solution and not just a deliverable. Ventana models are aligned to client needs through four principles:
• Verifiable capability: Through prototypes, demonstrations and live simulations, Ventana displays the evolving functions and effects of the model throughout the building process, so clients can verify in advance that the model system will do what it is supposed to do.
• Flexible design: Ventana's modeling approach of rapid evolution under client guidance (see Building Ventana Models) allows the decision levers and output performance metrics to change as understanding deepens over the course of the model development.
• General scope: Ventana also strives to make its models as general as possible. The models are not limited to historical and expected conditions, but are built to explore a very wide range of conditions and situations to remove blinders and avoid missed opportunities.
• Top-down orientation: The client's options and desired outcomes are the starting point for the model. All of the factors that might come into play are included, in as much detail as required to accurately represent each factor's effects on overall performance.
Model reliability means that the model accurately describes the possible outcomes of a potential action, and the reasons for them. In other words, a reliable model gives the right answer for the right reasons in all potential operating conditions.
Though reliability cannot be proven except in hindsight, one key question can be asked of a model in advance: are the model assumptions, and the output calculated from them, consistent with everything that is known about the real situation? That is, does the model behavior conform to historical data, expert knowledge, and common sense? Ventana's tools and approach are designed to ensure that Ventana models are consistent with all available information, both during development and over time.
By extensive triangulation among data, informed expectations, and causal hypotheses, Ventana first cross-checks all sources of understanding against one another for consistency and completeness. Vensim® Causal Tracing® speeds the tracing of inconsistencies and gaps to their roots for fast resolution. This process refines the knowledge base on which the model is built, ensuring that decisions are based on the best information possible. (For more information, please see Ventana Modeling Techniques.)
Ventana continually checks the developing model for self-consistency and for consistency with the refined information, using seven tests of reliability which can be used with any model: 1. Fit to historical data: This test measures how closely model outputs match historical data when given true historical inputs. Ventana employs statistical measures appropriate to each situation to understand when discrepancies between model output and historical data are due to data noise, and when they are signals of gaps in model logic. While a model must meet this standard, this should not be the only test--it is too easy. 2. Predictive fit to historical periods: One way to detect model errors is, for example, to calibrate the model using data through 1999, predict the outcomes for the year 2000, and compare predicted results to actual values. Understanding the causes of any discrepancies often reveals key gaps in model structure. This technique is also useful for comparing the predictive power of a new model to existing methods of decision analysis. (Once structural gaps have been fixed, however, this is no longer a fair comparison, since the later data have been used to guide the model development.) 3. Units of measure: Models are, in the end, collections of mathematics, expressed in equations. One tenet of applied mathematics is that every equation must have consistent units (compare "apples to apples"). While this requirement is straightforward, many common business analysis tools, such as spreadsheets, have no facility for catching unit errors. As a result it is easy for errors to go undetected, potentially skewing results. The Vensim modeling environment automatically checks units throughout the model. Enforcing consistent units also helps clarify the definitions of new concepts. 4. Physical conservation: Another straightforward requirement is that strict accounting must be observed for all physical quantities. Ventana models explicitly track stocks of people, things, and money, as well as the flows which increase or deplete them, to assure that resources do not appear or vanish inexplicably. Despite the logic of this requirement, models that inadvertently violate this law of conservation are extremely common. 5. Real world causality: In Ventana models, each variable has a clear real-world interpretation, and the equation calculating the change of each variable over time describes not only how much it changes, but why. The causal story described by each equation must be correct. If model output matches past results, but for reasons known to be wrong, future predictions by that model cannot be trusted. During development, Ventana cross-checks each causal assumption with client experts and available data in order to find and fix inconsistencies.
Though reliability cannot be proven except in hindsight, one key question can be asked of a model in advance: are the model assumptions, and the output calculated from them, consistent with everything that is known about the real situation? That is, does the model behavior conform to historical data, expert knowledge, and common sense? Ventana's tools and approach are designed to ensure that Ventana models are consistent with all available information, both during development and over time.
By extensive triangulation among data, informed expectations, and causal hypotheses, Ventana first cross-checks all sources of understanding against one another for consistency and completeness. Vensim® Causal Tracing® speeds the tracing of inconsistencies and gaps to their roots for fast resolution. This process refines the knowledge base on which the model is built, ensuring that decisions are based on the best information possible. (For more information, please see Ventana Modeling Techniques.)
Ventana continually checks the developing model for self-consistency and for consistency with the refined information, using seven tests of reliability which can be used with any model: 1. Fit to historical data: This test measures how closely model outputs match historical data when given true historical inputs. Ventana employs statistical measures appropriate to each situation to understand when discrepancies between model output and historical data are due to data noise, and when they are signals of gaps in model logic. While a model must meet this standard, this should not be the only test--it is too easy. 2. Predictive fit to historical periods: One way to detect model errors is, for example, to calibrate the model using data through 1999, predict the outcomes for the year 2000, and compare predicted results to actual values. Understanding the causes of any discrepancies often reveals key gaps in model structure. This technique is also useful for comparing the predictive power of a new model to existing methods of decision analysis. (Once structural gaps have been fixed, however, this is no longer a fair comparison, since the later data have been used to guide the model development.) 3. Units of measure: Models are, in the end, collections of mathematics, expressed in equations. One tenet of applied mathematics is that every equation must have consistent units (compare "apples to apples"). While this requirement is straightforward, many common business analysis tools, such as spreadsheets, have no facility for catching unit errors. As a result it is easy for errors to go undetected, potentially skewing results. The Vensim modeling environment automatically checks units throughout the model. Enforcing consistent units also helps clarify the definitions of new concepts. 4. Physical conservation: Another straightforward requirement is that strict accounting must be observed for all physical quantities. Ventana models explicitly track stocks of people, things, and money, as well as the flows which increase or deplete them, to assure that resources do not appear or vanish inexplicably. Despite the logic of this requirement, models that inadvertently violate this law of conservation are extremely common. 5. Real world causality: In Ventana models, each variable has a clear real-world interpretation, and the equation calculating the change of each variable over time describes not only how much it changes, but why. The causal story described by each equation must be correct. If model output matches past results, but for reasons known to be wrong, future predictions by that model cannot be trusted. During development, Ventana cross-checks each causal assumption with client experts and available data in order to find and fix inconsistencies.
Once the model has reliably helped to identify optimal choices, what makes it believable? In Ventana's experience, people believe models only if they understand and agree with the results, and with the reasons for them.
Ventana models have several features to ensure model results, and their causes, are understood:
>
Ventana models have several features to ensure model results, and their causes, are understood:
>
• Simplified what-if testing: Vensim Reality Check makes it easy to demonstrate that a model responds appropriately to changes in any factor, and stores what-if questions for automatic re-testing at any time.While these features make behavior and causes available and comprehensible to clients, it is the collaborative building process that most develops the model's credibility. The initial model draft starts from ideas supplied by the client, and rapidly evolves in response to continuous client critique and Ventana testing. When clients find flaws in model logic, or when Ventana tests find inconsistencies in ideas, Ventana and the client work together to correct the causal description. This collaboration is crucial: quite rightly, people tend to meet new approaches and ideas with skepticism. The process of collaboration and correction allows people to move from skepticism to exploration, and from exploration to acceptance. In each project, Ventana strives to include everyone whose confidence is required for the model to be used, especially the people who will make the decisions supported by the model. Ventana also makes a point of soliciting the critique of the organization's respected experts, not only to create a legacy of their knowledge and apply it in the model, but also to increase the model's credibility for the rest of the organization. Over the course of the collaboration, all client stakeholders contribute to -- and become confident in -- the model results and the causal assumptions on which they are based. >
• Causal structure: Each equation describes a real-world cause-and-effect relationship, making it possible to describe why a model result occurs. Equations are organized through influence diagrams, and Vensim Causal Tracing makes it easy to visualize *why* things happen.
• Familiar language: Model variables are named using the client's terms for the real world concepts they represent.
• Openness: It is Ventana policy to openly share the basis and contents of models with clients. The model development process invites critique by client experts, and challenges can be stored as Reality Check statements to test for ongoing compliance at any time.
• Simplicity: Ventana models are designed to be the simplest correct description of how client decisions will affect client outcomes. This usually takes the form of a handful of linked sub-models, each of which is simple enough to be easily grasped. This structure enables people to get a solid overview in less than an hour, and facilitates complete understanding by client personnel who are able to spend more time.
>
Ventana models are designed to fit into each client organization's practice and culture. The means of interacting with the model complements existing organizational norms for how people spend their time, manage information, communicate, divide responsibilities, and make decisions. More importantly, Ventana has observed that models deliver their full benefit when they have a personal significance for the people who use them. Contributing knowledge and requirements to the design of a model, then seeing it take shape, inspires people to extend their efforts and ingenuity and use the model to maximum advantage.
The specific requirements for a good fit vary tremendously from organization to organization. Ventana's modeling process recognizes the need for cultural fit, and systematically addresses it from the beginning of the project. The model definition, creation, and review process engages all stakeholders at appropriate levels of intensity, to build ownership in the model across the organization. Ventana also supports a wide range of options for data management, model interface, and training programs, and is experienced in many modes of connecting models to decision-making. (Please see Building Ventana Models and Ventana Modeling Techniques for more information.) From the beginning of the relationship, Ventana helps clients to define their own requirements of organizational fit, and to find the solutions to meet them.
The specific requirements for a good fit vary tremendously from organization to organization. Ventana's modeling process recognizes the need for cultural fit, and systematically addresses it from the beginning of the project. The model definition, creation, and review process engages all stakeholders at appropriate levels of intensity, to build ownership in the model across the organization. Ventana also supports a wide range of options for data management, model interface, and training programs, and is experienced in many modes of connecting models to decision-making. (Please see Building Ventana Models and Ventana Modeling Techniques for more information.) From the beginning of the relationship, Ventana helps clients to define their own requirements of organizational fit, and to find the solutions to meet them.