Defining Discovery: Core Concepts

Discovery tools have had a referenceable working definition since at least 2001, when Ben Shneiderman published 'Inventing Discovery Tools: Combining Information Visualization with Data Mining'.  Dr. Shneiderman suggested the combination of the two distinct fields of data mining and information visualization could manifest as new category of tools for discovery, an understanding that remains essentially unaltered over ten years later.  An industry analyst report titled Visual Discovery Tools: Market Segmentation and Product Positioning from March of this year, for example, reads, "Visual discovery tools are designed for visual data exploration, analysis and lightweight data mining."

Tools should follow from the activities people undertake (a foundational tenet of activity centered design), however, and Dr. Shneiderman does not in fact describe or define discovery activity or capability. As I read it, discovery is assumed to be the implied sum of the separate fields of visualization and data mining as they were then understood.  As a working definition that catalyzes a field of product prototyping, it's adequate in the short term.  In the long term, it makes the boundaries of discovery both derived and temporary, and leaves a substantial gap in the landscape of core concepts around discovery, making consensus on the nature of most aspects of discovery difficult or impossible to reach.  I think this definitional gap is a major reason that discovery is still an ambiguous product landscape.

To help close that gap, I'm suggesting a few definitions of four core aspects of discovery.  These come out of our sustained research into discovery needs and practices, and have the goal of clarifying the relationship between discvoery and other analytical categories.  They are suggested, but should be internally coherent and consistent.

Discovery activity is: "Purposeful sense making activity that intends to arrive at new insights and understanding through exploration and analysis (and for these we have specific defintions as well) of all types and sources of data."

Discovery capability is: "The ability of people and organizations to purposefully realize valuable insights that address the full spectrum of business questions and problems by engaging effectively with all types and sources of data."

Discovery tools: "Enhance individual and organizational ability to realize novel insights by augmenting and accelerating human sense making to allow engagement with all types of data at all useful scales."

Discovery environments: "Enable organizations to undertake effective discovery efforts for all business purposes and perspectives, in an empirical and cooperative fashion."

Note: applicability to a world of Big data is assumed - thus the refs to all scales / types / sources - rather than stated explicitly.  I like that Big Data doesn't have to be written into this core set of definitions, b/c I think it's a transitional label - the new version of Web 2.0 - and goes away over time.

References and Resources:

Related posts:

  1. The Architecture of Discovery: Slides from Discover Conference 2011 Endeca invites cus­tomers, part­ners and lead­ing mem­bers of the broader...
  2. Strata New York Slides & New Discovery Patterns I’ve posted slides from my pre­sen­ta­tion at Strata to slideshare;...
  3. Speaking at UXLX on "The Language of Discovery: A Grammar for Designing Big Data Interactions" I’ve just con­firmed that I’ll be pre­sent­ing a light­ning talk at...
  4. Presenting "The Language of Discovery" at Strata NY Big data types, visu­al­iza­tion wonks, ana­lyt­i­cal savants, and all those...
  5. Strata New York Video: Designing Big Data Interactions With the Language of Discovery I’m late to mak­ing it avail­able here, but O’Reilly media...

Category: Language of Discovery
Tags: , , , , Comment »

Leave a Reply


Back to top