Helping end users find anomalies in data feeds
Although everyday information systems are not mission critical, they must be dependable enough for practical use. A major limiting factor here is the dependability of the incorporated elements. These elements include not only code but also data feeds, such as online weather conditions and stock quotes. It is particularly difficult to evaluate the dependability of data feeds. Data feeds have incomplete specifications if they have specifications at all, and their vendor is absent for all practical purposes. In addition, end users of data feeds may not be computer-science-proficient and cannot be expected to provide specifications. These factors inhibit many dependability enhancement techniques, which require a model of proper behavior for failure detection, preferably in the form of specifications.
In this talk I present a partial solution to this problem---CUES, Checking User Expectations about Semantics. CUES is a method and a prototype implementation for making user expectations precise and for checking these precise expectations. CUES treats the precise expectations as a proxy for missing specifications. It checks the precise expectations to detect anomalies. I present empirical evidence of the practicality and usefulness of CUES. These evidence indicate that a user of CUES gets substantial benefit for a modest investment of time and effort. In addition to automated detection of anomalies, the benefit often includes a better understanding of the user's own expectations, of the data feeds, and of existing and missing documentation.