User Interfaces for Supporting Innovation
Ben Shneiderman Draft: July 31, 1999
Department of Computer Science
University of Maryland
College Park, MD 20742
For ACM TOCHI special issue (January - March 2000) on
HCI in the New Millennium, Jack Carroll, Editor
Brad Myers Assoc. Editor (revision due August 1,
A challenge for human-computer interaction researchers and user interface designers is to construct information technologies that support creativity. This ambitious goal can be attained by building on an adequate understanding of creative processes. This paper offers a four-phase framework for creativity that might assist designers in providing effective tools for users:
- Collect: learn from previous works stored in digital libraries, the web, etc.
- Relate: consult with peers and mentors at early, middle and late stages
- Create: explore, compose, and evaluate possible solutions
Keywords: Creativity support tools, graphical user interfaces, human-computer interaction, direct manipulation, and information visualization
Note: This paper is also intended for an expanded
presentation in book format, in accordance with ACM copyright rules.
Humans have long relied on technologies for information preservation and dissemination. Ancient traders recorded contracts on clay tablets, religious scribes hand copied illuminated manuscripts on parchments, and Guttenberg applied his printing press to reproducing bibles. Print media led to dramatic changes in society, and then modern broadcast media shook the world further with even more rapid and widespread dissemination. Today, even in remote locales, most people have seen TV and heard radio. The emerging World Wide Web is producing a further societal transformation since it shifts the balance from centralized broadcast media towards an individual's capacity to select desired information.
But information preservation and dissemination are only two uses of information technologies. A third expectation is that they support rapid two-way communication among people. The historical trajectory is towards greater speed, lower cost, and broader access to the means of communication. Writing letters was a rare process among narrow literate segments of ancient societies, but making phone calls is now widespread, and sending email is likely to become as popular.
A fourth expectation is that information technologies support the creation of knowledge or art. Even into the 20th century, scientists, inventors, novelists, and painters were seen as specially gifted citizens whose rare creative productions were treasured. Photography shook the art world because it enabled a majority to produce elegant artifacts rapidly at low cost. Similarly, information technologies could be harnessed to make creativity in many fields more common. Like photography, creativity support tools built on information technology can facilitate existing patterns of creativity or open up new opportunities.
Information technologies that allow more people to be more creative more of the time are likely to have profound effects on every institution (Johnson, 1997). Education could expand from acquiring facts, studying existing knowledge, and developing critical thinking, to include more emphasis on creating novel artifacts, insights, or performances. Medicine's shift from applying standard treatments to tailoring treatments for each patient, reflects the trend to personalization that is already ascendant in marketing and media. Expectations of teachers, lawyers, and designers are likely to rise as creativity is expected on more occasions from more people. These changes will be welcomed by some, but resisted by others. The challenge to leaders and participants will be to preserve appropriate elements of existing knowledge work while shaping new technologies and then integrating them into the workplace.
The goal of this paper is draw attention to the user
interface issues so that system designers can more effectively influence
future technologies. It begins with three perspectives on creativity: inspirationalist,
structuralist, and situationalist. It concentrates on supporting evolutionary
creativity, at the expense of revolutionary or impromptu creativity. Then
it reviews and refines the genex (generator of excellence) proposal
which is a four-phase integrated framework to support creativity (Shneiderman,
1998c). I believe that designers who follow the genex framework will more
often create powerful tools that enable users to be more creative more
of the time. This paper's main contribution is the identification of eight
activities that support creativity. This list of activities implies a research
agenda for human-computer interaction theoreticians, designers, software
engineers, and evaluators.
2. Three Perspectives on Creativity
The large literature on creativity offers diverse perspectives (Boden, 1990; Gardner, 1994; Couger, 1996; Cave, 1999). Some writers, I'll call them inspirationalists, emphasize the remarkable "Aha!" moments in which a dramatic breakthrough magically appears. Stories of Archimedes (3rd century B. C.) jumping from his bath screaming "Eureka!" as he discovered hydrostatics or Freidrich August Kekule’s (1829-1869) dream-given insight about benzene's ring structure emphasize the intuitive aspects of creativity. Most inspirationalists are also quick to point out that "luck favors the prepared mind," thereby turning to the study of how preparation and incubation lead to moments of illumination. The inspirationalists also recognize that creative work starts with problem formulation and ends with evaluation plus refinement. They acknowledge the balance of one- percent inspiration and ninety-nine percent perspiration - a flash of insight followed by much hard work to produce a practical result.
Those who emphasize this inspirational model promote techniques for brainstorming, free association, lateral thinking (DeBono, 1973), and divergence. They advocate strategies to break an innovator’s existing mindset and somehow perceive the problem with fresh eyes. Since they want innovators to break from familiar solutions, their recommendations include travel to exotic destinations with towering mountains or peaceful waterfalls. Inspirationalists talk about gifted individuals, but usually stress that creativity-inducing thought processes can be taught.
The playful nature of creativity means that software support for inspirationalists emphasizes free association using textual or graphical prompts to elicit novel ideas. Inspirationalists are often oriented to visual techniques for presenting relationships and for perceiving solutions. They would be sympathetic to information and scientific visualization strategies that helped users understand previous work and explore potential solutions. Many writers and software developers of tools such as IdeaFisher or MindMapper encourage two-dimensional layouts of loosely-connected concept nodes to avoid a linear or hierarchical structure. The casual style and freedom from judgment that is implicit in sketching is encouraged. Inspirationalists would also appreciate templates as starting points for an inspirational leap as long as they are coupled with tool palettes to explore fresh combinations.
A second group of writers on creativity, I'll call them structuralists, emphasizes more orderly approaches (Mayer, 1992). They stress the importance of studying previous work and using methodical techniques to explore the possible solutions exhaustively. When a promising solution is found, the innovator evaluates strengths and weaknesses, compares it to existing solutions, and refines the promising solution to make it implementable. Structuralists teach orderly methods of problem solving such as Polya's four steps in How to Solve It (1957):
1) Understanding the problem
2) Devising a plan
3) Carrying out the plan
4) Looking back
For structuralists, digital libraries and web site of previous work are important, but the key software support comes in the form of spreadsheets, programmable simulations, and domain-specific scientific / engineering / analytical / mathematical models. These software tools support "what if" processes of trying out assumptions to assess their impact on the outcomes. They often show processes with visual animations. Structuralists are also visual thinkers but their preferred tools are for drawing flow charts, decision trees, and structured diagrams. Since they favor methodical techniques they are likely to appreciate software support for step-by-step exploration, with the chance to go back, make changes, and try again.
A third group, the situationalists, emphasizes the social and intellectual context as a key part of the creative process. They see creativity as embedded in a community of practice with changing standards, requiring a social process for approval from scientific journal editors, museum curators, or literary prize juries. For example, Csikszentmihalyi (1993) sees three components to creativity, they are the:
2) field which "includes all the individuals who act as gatekeepers to the domain. It is their job to decide whether a new idea, performance, or product should be included in the domain."
3) individual person whose creativity is manifest "when a person using the symbols of a given domain such as music, engineering, business, or mathematics has a new idea or sees a new pattern, and when this novelty is selected by the appropriate field for inclusion in the relevant domain."
These three perspectives on creativity - inspirationalism,
structuralism, and situationalism - are all useful in shaping user interfaces
to support creative work. These perspectives are easily combined. User
interface designers can develop tools that stimulate inspiration based
on previous work, link to associated ideas, and provide templates for action.
Designers can build structured tools for exhaustive exploration, which
is already a common strategy in computing. Designers can also facilitate
consultation by email and more refined methods, to support the social strategies
of the situationalists. Section 4 expands on these possibilities.
3. Levels of Creativity
The large literature on creativity considers diverse levels of aspiration (Boden, 1990). A restricted definition would focus on great breakthroughs and paradigm-shifting innovations (Kuhn, 1996). Einstein's relativity theory, Watson & Crick's discovery of DNA's double-helix, or Stravinsky's Rite of Spring are often cited as major creative events. Such a definition confines our discussion to rare revolutionary events and a small number of Nobel Prize candidates. A looser definition would include what Kuhn referred to as normal science, useful evolutionary contributions that refine and apply existing paradigms (Basalla, 1988). Evolutionary acts of creativity include doctors making a diagnosis, lawyers preparing a brief, or photo editors producing a magazine story. Their work is important in changing someone's life by medical care, legal practice, or journalist reportage and could be helped by software tools. I am most concerned about evolutionary creativity, but of course, software tools that support evolutionary creativity may also help produce revolutionary breakthroughs. On the other hand, it is possible that software tools that support evolutionary creativity restrict thinking too much and are not useful for or even discourage paradigm shifts.
A third, still broader, level of creativity is conceivable - impromptu creativity. Can brilliant conversation, musical improvisation, or creative parenting be considered a part of the creativity spectrum? These more spontaneous activities may be creative in a broad sense, but since they seem harder to support, record, and evaluate, we will leave them out for now.
So the focus of this paper is not on revolutionary or impromptu creativity, but on evolutionary creativity. This still covers a wide range of possibilities. Developing software support tools for evolutionary creativity according to the three perspectives identified in this paper - inspirationalism, structuralism, and situationalism - is a sufficient challenge.
It seems necessary to apologize in advance for the hubris or arrogance of proposing technology to aid human creativity. A critic might scowl that creativity is inherently human and no computer could or should be brought into the process. But technology has always been part of the creative process, whether in Leonardo's paint and canvas or Pasteur's microscopes and beakers. Supportive technologies can become the potter's wheel and mandolin of creativity -- opening new media of expression and enabling compelling performances. Secondly, I seek tools that support human creativity and strongly rebel at the suggestion that machines are or can ever be creative. Computers have no more intelligence or creativity than a wooden pencil. People are creative and they have always used advanced technology to raise their potentials.
My expectations are largely positive, but there are many problems, costs, and dangers in anything as ambitious as a tool and framework to support creativity. An obvious concern is that many people may not want to be more creative. Many cultures encourage respect for the past and discourage disruptive innovations. Promoting widespread creativity raises expectations that may change employment patterns, educational systems, and community norms. Introducing computer supports for creativity may produce greater social inequality as it raises the costs for those who wish to participate. Finally, these tools may be used equally by those who have positive and noble goals as well as by dictators or criminals who seek to dominate, destroy, or plunder.
These fears are appropriate and reasonable cautions
must be taken, but support for innovation could lead to positive changes
to our world. However, the moral dilemma of a technology innovator remains
troublesome: How can I ensure that the systems I envision will bring greater
benefits than the negative side effects that I dread and those that I fail
to anticipate? Widespread access to effective user interfaces could help
with major problems such as environmental destruction, over population,
poor medical care, oppression, and illiteracy. It could contribute to improvements
in agriculture, transportation, housing, communication, and other noble
human endeavors. Ensuring that these positive results come about and minimizing
negative side effects remain challenges, but a framework that provides
for substantial consultation and broad dissemination may help.
4. Genex: A four-phase framework for generating excellence
This paper combines and extends two previous efforts. The genex framework (Shneiderman, 1998c) was a first attempt to build on Csikszentmihalyi's approach, by supporting access to the domain and consultation with the field. The name genex (generator of excellence) was chosen to echo Vannevar Bush's memex (memory extender). The genex framework had four phases: collect, create, consult, and disseminate. This paper refines the genex framework by expanding its scope and being more precise about what user interface tools are needed. The second source for this paper is the educational philosophy, relate-create-donate (Shneiderman 1998b), that emphasized collaborative teams working together to produce something ambitious for use by someone outside the classroom. This service-oriented approach, using web-based technologies, leads to innovative artifacts or performances. It emphasizes creativity to support learning, and learning to support creativity.
The foundational beliefs that led to genex's four phases were:
1) New knowledge is built on previous knowledge
2) Powerful tools can support creativity
3) Refinement is a social process
4) Creative work is not complete until it is disseminated
However, the social processes that were characterized narrowly as a support for refinement, can also be helpful at early, middle and late stages of the creative process. This paper more clearly defines the powerful tools that can facilitate creative acts. Furthermore, the close relationship of learning and creativity is more apparent to me now. Combining the strategies in these two papers leads to a revised four-phase genex framework:
- Collect: learn from previous works stored in digital libraries, the web, etc.
- Relate: consult with peers and mentors at early, middle and late stages
- Create: explore, compose, and evaluate possible solutions
- Donate: disseminate the results and contribute to the digital libraries
These four phases are not a linear path. Creative work may require returning to earlier phases and much iteration. For example, digital libraries, the web, and other resources may be useful at every phase. Similarly, discussion with peers and mentors may take place repeatedly during the development of an idea.
This four-phase genex framework has much in common with previous characterizations and methodologies, but there are important distinctions. Couger (1996) reviews 22 "creative problem solving methodologies" with simple plans such as:
and this, even more basic, three-phase plan:
- Intelligence: recognize and analyze the problem
- Design: generate solutions
- Choice: select and implement
Couger offers his own plan with five phases:
- Opportunity, delineation, problem definition
- Compiling relevant information
- Generating ideas
- Evaluating, prioritizing ideas
- Developing an implementation plan
It is striking to me that so many of the plans limit themselves to the narrow perspective of the inspirationalists and structuralists. Problem solving and creativity are portrayed as lonely experiences of wrestling with the problem, breaking through various blocks, and finding clever solutions. The descriptions of even early phases rarely suggest contacting previous workers on this problem or exploring libraries for previous work. Consultation with others and dissemination of the results is minimally mentioned. Some reconsideration of such methodologies is in order because of the presence of the World-Wide Web (Berners-Lee, Cailliau, Luotonen, Nielsen, and Secret, 1994). It has already dramatically reduced the effort of finding previous work, contacting previous workers, consulting with peers and mentors, and disseminating solutions (Kiesler, 1997).
Of course there are costs and dangers in reading previous work or consulting with others (Shneiderman, 1998c). There are also satisfactions in solving a problem on your own, but when dealing with difficult problems the benefits of building on previous work and consulting with peers and mentors can be enormous. The genex framework builds on the situationalists' perspective by embracing the expanded opportunities offered by the World-Wide Web.
However, the need for improvements to the current World-Wide Web was revealed in a study of support for idea generation (Massetti, White, and Spitler, 1999). Twenty-three MBA students were asked to generate as many ideas as they could on three topics (robotics, societal change, and business opportunities in Ireland). In this within-subjects design, each subject was asked to use the World-Wide Web, categorical cues (five suggested categories), and no support. Subjects using the World-Wide Web did no better in generating ideas and did not have higher satisfaction. Categorical cues seemed to provide the best assistance, supporting the inspirationalist approach to stimulating creative ideas by guiding free associations.
The goal of genex framework is to suggest improvements for web-based services and personal computer software tools. By reducing the distraction caused by poorly-designed user interfaces, users’ attention can be devoted to the task. In an effective design for a genex, the boundaries between applications and the burdens of data conversions would disappear. Data representations and available functions would be in harmony with problem-solving strategies. Then users could concentrate on genex's four phases of creativity: collect, relate, create, and donate.
My suggestion of a close link between supporting
creativity and generating excellence is a hope, not a necessity. I hope
that creativity support tools will be used to pursue excellence, high quality,
and positive contributions, but I am sadly aware that this will not always
be the case. The genex framework could tighten the link because of its
emphasis on consultation and dissemination. I repeat my belief (from the
end of section 4) that making creativity more open and social through participatory
processes will increase positive outcomes while reducing negative and unanticipated
5. Integrating Creative Activities
The revised genex framework calls for integrated creativity support tools. Some of these tools already exist, but could be enhanced to better support creativity. However, the main challenge for designers is to ensure smooth integration across these novel tools and with existing tools such as word processors, presentation graphics, email, databases, spreadsheets, and web browsers (Shneiderman, 1998a).
Smoother coordination across windows and better integration of tools seem possible. Just as word processors expanded to include images, tables, equations, and more, the next generation of software is likely to integrate additional features. Some aspects of the integration can be accomplished by creating compatible data types and file formats (possibly standardized objects). A second aspect of integration has to do with compatible actions and consistent terminology, such as cut-copy-paste or open-save-close. Higher levels of actions that are closer to the task domain might be candidates such as annotate-consult-revise or collect-explore-visualize. A third aspect of integration is the smooth coordination across windows (Dey, Abowd, Pinkerton, and Wood, 1997: Derthick, Kolojejchick, and Roth, 1997; North and Shneiderman, 1999). For example, if users see an unfamiliar term they should be able to click on it and get an English definition, a French translation, or a medical dictionary report, all in a predictable screen location. Similarly, if users find a personal name in a document they should be able to get a biography, email address, or contact information. Other coordinations include synchronized scrolling of related documents to facilitate comparisons and hierarchical browsing of a table of contents and a chapter to enable easy navigation of large resources.
These improved user interfaces will benefit many users for many tasks, but my concern is to develop creativity support within the genex framework. The three perspectives - inspirationalism, structuralism, and situationalism - each lead to useful suggestions for activities during the four genex phases (Figure 1).
Genex's integrated creativity support tools should
offer at least the eight activities described below. Figure 1 indicates
how these activities are primarily related to the genex phases, but these
activities could take place during any phase. For example searching of
digital libraries, the web or other resources is primarily associated with
the collect phase, but searching may occur in order to find consultants
or to decide on candidates communities for presenting of results. Visualizing
objects and processes is the activity that seems most pervasive and could
appear in every genex phase.
5.1 Searching and browsing digital libraries: Existing web search engines and digital library interfaces do an impressive job, but refinements to support creativity are possible. Situationalist users who want to steer the search to resources about previous work will want more control over the relevance ranking and over the presentation of the results (Marchionini, 1995; Koenemann and Belkin, 1996; Shneiderman, Byrd and Croft, 1997). For example, it should be easy for users to restrict a legal search to recent New York, New Jersey or Connecticut court cases on tenant’s rights for heating. The result set should be grouped by claims (civil rights, contract violation, etc.) and ordered by date with color-coded stars indicating the strength of the results for or against tenants. Extraction of plaintiff's or defendant's lawyers should also be convenient. This may sound like a tall order, but detailed search request specification while maintaining comprehensibility is possible. Since searching is part of a larger creative process the result set should be easy to save into a spreadsheet for further manipulation, include in an email note for consultation, or paste into a notebook for later referral.
In addition to searching, inspirationalists will want rapid browsing to more effectively support exploration, similar to what many people do in bookstores or libraries. Web sites with high branching factors (many links per page) support exploration by making what is available more visible. Yahoo's (www.yahoo.com) 100+ links on the home page are helpful in getting an idea of what is available and what is not. Browsing textual menus supports exploration when specific terms or concepts are not known. Imagine the complex request implied by this question: which strains of bacteria are causing recent flu symptoms for senior citizens in the Mid-Atlantic states? No specific search string could produce appropriate results, but browsing through search results may lead to exploration of viral infections, variant symptoms, other diseases, other locations, earlier epidemics, etc.
Information derived from searching digital libraries
may provide the right results, but there are still many concerns about
whether the information is accurate, complete, and up-to-date. Furthermore,
online information may only be a fraction of the total information, and
the search engines may search only a fraction of the online information.
5.2 Consulting with peers and mentors: Email, listservs, newsgroups, and threaded discussions are excellent asynchronous tools for situationalists' inquiries about previous work and for discussing new ideas. The capacity to find people who have a shared interest and are at work on similar problems is one of the greatest gifts of the internet. Synchronous tools, such as phone calls, videoconferencing (for example CUSeeMe, Dorcey, 1995) and software sharing (for example, Netmeeting), enrich the possibilities for early, middle, and late stage consultations (Olson and Olson, 1997).
For early stage consultations, innovators are likely to search widely, finding diverse information resources and broad communities. At this preparatory stage, innovators are forming questions, finding out about current workers who might be collaborators and choosing directions. During middle stage consultations, the tasks are to propose potential solution strategies and develop evaluation methods. As solutions are created and refined, late stage consultations are directed at confirming the innovation, refining it to accommodate criticisms, and then disseminating it to appropriate parties (National Research Council, 1993).
Consultations might involve continuing dialogs by email, phone calls, or personal visits. While these may often be collegial and friendly, some discussions might be tense since other workers may be competitors who are pursuing similar goals. Either party might withhold some of their knowledge or ideas, or probe the other to gauge their intentions. Non-disclosure or non-competitive agreements might be discussed or signed as part of a corporate consultation. University researchers are typically open, but the competition for solution of important problems and battles to promote reputations can suppress discussion.
These concerns greatly influence the design and use
of technologies, since the appropriate balance of privacy protection and
easy access to information is vital. Researcher's early notes and explorations
need to be kept private, but claimed breakthroughs need rapid and broad
dissemination. Appropriate credits for articles, patents, or software products
are often the source of conflicts, so improved record keeping of the consultations
that contribute to an innovation could resolve some questions and maybe
encourage more cooperation.
5.3 Visualizing data and processes: While inspirationalists are likely to propose visual approaches, structuralists will also appreciate the thoroughness and rapidity with which alternatives can be reviewed using visualization. The field of pharmaceutical drug discovery involves review of thousands of compounds for their efficacy, shelf life, solubility, acidity, cost, allergic reactions, interactions with other medications, etc. Scrolling lists in spreadsheets are useful, but two-dimensional visual presentations with color and size coding are proving to be enormous benefits for this task. Early work on a FilmFinder (Ahlberg and Shneiderman, 1994) led to a successful commercial product called Spotfire (http://www.spotfire.com) (Figure 2). A wide range of 1-, 2-, 3- and multi-dimensional visualizations plus temporal, tree and network presentations have been proposed (Card, Mackinlay, and Shneiderman, 1999).
Figure 2: Multi-dimensional film database viewed
with a 2-dimensional starfield display in Spotfire. The x-axis is the years
and the y-axis is the popularity of the film. Color coding is film type
(action, drama, mystery, etc.) and larger size dots indicate longer movies.
Visualization of digital library contents (Hearst, 1999), financial information (Wright, 1995), scientific data (Bryson et al., 1999), medical histories (Plaisant et al., 1996), etc. is growing, but to increase its benefits to creativity requires smooth integration. The results of a web or database search should be easily imported (for example by cut and paste) into a visualization. Then users should be able to filter the data appropriately and adjust the visualization features, such as x,y-axes, color, size, or other codings. Then when an interesting group of items is found, users should be able to select them and paste them into a spreadsheet or statistics package for further processing. Then the visualization and processed items should be embeddable in a written report, slide presentation, or email note. Email recipients should be able to manipulate the visualization or report still further (Roth et al., 1996).
Situationalists should also find use for visualizations
in exploring digital libraries, locating key people, and presenting their
findings to others.
5.4 Thinking by free associations: Inspirationalists seek to liberate the mind by making free associations to related concepts. This Gestalt psychology approach (Mayer, 1992) has led to innovative software that is meant to facilitate association of ideas by presenting related concepts. IdeaFisher (www.ideafisher.com) claims "to create a new method of generating ideas in a way similar to the workings of the human mind...by associations, or links. ...a thought like food leads to associations such as apple, cooking, cafe, washing dishes, and so on. When a creative new idea is born, it usually consists of associations linked together in a way that has not been thought of before. This is the heart of IdeaFisher technology -- a dictionary of associations." Users apparently enjoy using IdeaFisher but its benefits are still being understood with empirical studies (Massetti, 1996).
Computerized thesauri may also be helpful textual exploration tools since varied associations such as synonyms, antonyms, homonyms, rhymes, or even anagrams can be retrieved rapidly. Alternatively, random word presentations are also proposed as a method for stimulating fresh thoughts and breaking through creative blocks.
Other products enable users to create visual representations of relationships among words or concepts, including MindManager (www.mindman.com/) which allows users to "create relationships among information easily... identify relationships between items or branches." The Axon Idea Processor (www.singnet.com.sg/%7Eaxon2000/) is promoted as "a sketchpad for visualizing and organizing ideas" that "exploits visual attributes such as: color, shape, size, scale, position, depth, link, icon, etc. Visual cues facilitate recall, association, and discovery. Diagrams and pictures help you to represent and solve complex problems." Typical mindmaps show a main idea in a central node and then branches in all directions showing related ideas (Figure 3). As the diagram spreads out the nodes and font sizes decrease so that many small details are shown. A popular alternative is to have the main idea at the top or left, and then have a branching tree structure going down or to the right.
Figure 3: MindMap produced with the Axon Idea Processor
Mindmaps, concept maps, semantic networks and other drawing strategies may be helpful to some people. Appropriate visual presentations, such as the periodic table of elements, help in problem solving because they compactly present substantial information and show important relationships. Geographic maps, architect's drawings, and electronic circuit diagrams are other examples of the great power of visual presentations for learning and problem solving. The gift of the computer is the capacity to quickly create and easily manipulate such diagrams. Penciled diagrams and sketches have a similar capacity to invite discussion, exploration, and revision.
The genex framework reminds us that such textual
lists and diagrams should be easily shared with others, annotatable, linkable,
and searchable. It should be possible to import and export from these programs
so that related tools can be employed, for example, to translate terminology
in a diagram into a foreign language or to link diagram nodes to web sites.
5.5 Exploring solutions - What if tools: A large family of software tools already support methodical exploration of solutions in the structuralist style (Elam and Melissa, 1990). Electronic spreadsheets were quickly described as "what if" tools that allowed business planners and analysts to quickly try out a variety of scenarios. Such explorations were possible with paper and pencil, but they were a hundred times faster with electronic spreadsheets. Similarly, flight simulators, traffic flow models, weather models, and thousands of scientific, engineering, economic and other modeling tools allow users to explore alternatives safely, economically, and rapidly. Simulators have been largely custom built, but simulation building software is growing from domain specific to broader coverage. These simulation models are also becoming more richly featured to support explanatory text, collaborative usage, history keeping, and more (Rose et al., 1998; Jones and Schneider, 1996). A popular class of home computer software includes simulation games, such as SimCity, that enable users to try out urban planning scenarios. Also popular with personal computer users are flight, driving, and battle simulators that provide various levels of realistic experiences. These applications are enjoyable and also educational, however they are largely independent.
The genex framework and the situationalist perspective
remind us of the importance of integration to support creativity. Having
run a simulation, can users save the whole session and replay it later
to study their performance or discuss it with a peer or mentor? Can they
send the session to someone by email, annotate steps, or search for key
events or actions? Can excellent sessions be stored in a digital library
to allow future researchers or problem-solvers to build on the best work?
5.6 Composing artifacts and performances: Another large family of software tools already supports creation of artifacts and performances (Nakakoji, Suzuki, Ohkura, and Aoki, 1997). The ubiquitous word processor is the premier example of a flexible tool that enables many users to create diverse high-quality printed documents using a relatively low-cost laser or inkjet printer. Only a few decades ago such high production quality required advanced typesetting skills that were available only to publishing professionals. The word processor enables business people to produce elegant newsletters and professional advertising. It allows individuals to produce high quality resumes and elaborate full-length books. But word processors do more than facilitate quality appearance, they can also contribute to improved contents. The capacity to easily cut-and-paste, change terminology, or add references enables more people to create stronger scientific papers, legal briefs, or business proposals. It enables novelists, playwrights, and poets to easily accommodate suggestions from peers and mentors.
But even the word processor can be improved by thinking about creativity support in the genex framework. If new artifacts are inspired by previous work, then users should benefit from orderly processes for locating appropriate previous work and modifying it to suit their current needs. The basic notions of exemplars, templates, and processes, have already begun to appear. Authors of business letters can purchase thousands of exemplars that they can adapt in their own word processor. Then some of these authors might be able to contribute or sell their improved or specialized exemplars through growing digital libraries, available through the World-Wide Web.
A more flexible strategy is to prepare templates, as is done for business documents such as invoices or travel expense reports. These are supplied with many word processors. Templates differ from exemplars in that they are somewhat more flexible, may include computational capabilities, and can have instructional guidance. Processes, such as Windows Wizards, extend the flexibility and allow richer variations.
Making a wider range of exemplars, templates, and processes available and a mechanism to create a market in new ones are still needed. Microsoft Office contains templates for word processed newsletters, slide presentations sales reports, spreadsheet invoices, photo library databases, etc. The structured processes of users' choices, embedded in the Microsoft Wizards, show overviews of the steps for generating an initial version of a personal resume or a legal pleading.
But effective composition tools go beyond the initial stages by providing transformations on documents and artifacts. Low-level changes like global word replacements or font changes are easy to implement, and mid-level changes such as reorganizing sections, figures, or style sheets are commonly available. However, there are few high-level tools that help in reorganizing presentations to give greater prominence to examples, shifting rhetorical style to question-answer expositions, or generating conclusions based on content. An increased variety of low, mid, and high level transformations should enrich the capacity of users to explore alternatives.
Adobe PhotoDeluxe provides excellent example of the range of support tools for the complex transformations in getting, cropping, editing, annotating, and sending images. In addition, it offers impressive transformations for images to give them the appearance of Van Gogh's bold brush strokes, Seurat pointillism, or other styles. Music composition programs also provide transformations for a motif to generate New Orleans jazz or Rhythm and Blues pieces (www.pgmusic.com). Templates for fugues or sonatas and the capacity to recast motifs in the style of Brahms, Beethoven or other composers seem realizable.
Composition support tools should also provide evaluations.
Modern word processors offer spelling and grammar checking, word counts,
and sometimes reading difficulty indexes. More advanced evaluation and
critiquing tools are possible, especially if the domain of application
is kept narrow. For example, tax preparation tools provide continuous feedback
on the refund amounts as changes are made to the tax forms.
5.7 Reviewing and replaying session histories: Reflecting on work is a central notion in quality improvement, creativity, and education methods. The metacognitive processes that promote self-awareness are learnable, but software support to capture the history of all user actions would seem to be a good foundation for many services. We have come to expect that web browsers record the history of our site visits, so that we can return to them. One study of World-Wide Web users showed that 58% of all URLs visited by users during a session were return visits (Tauscher and Greenberg, 1997). Even in this simple case controversies abound because the strategy for producing a compact, meaningful list (in a linear, tree, or network format) is not apparent. Another problem is that web sites may change over time, so older histories may no longer produce the same results.
Producing histories of command lines in Unix or information retrieval programs is relatively easy, but understanding the meaning of each command may depend on the context (for example, the current directory). In richer graphical user interface environments such as simulators, image manipulators, legal information retrieval, or geographic information systems, recording each user step is feasible. However, success requires careful user interface and software design to ensure that the results are compact, comprehensible, and useful. Once these basics are accomplished, structuralists will especially like to manipulate session histories and replay the steps on another set of data or go back to change a step before replaying. Saving a sequence of steps for later use creates a basic macro, and by adding conditional and looping constructs quite ambitious programs could be built. For example, in a simulation of computer chip fabrication, it should be possible to rerun the simulation changing the temperature by single degree increments from 100 to 200 degrees Centigrade. If histories are first class objects then users should be able to send them to peers or mentors for comment or assistance with problems. Histories might also be searchable so that a set of hundreds of directories could be searched to see if any were done at a temperature above 200 degrees Centigrade. Users should be able to post or sell exemplars of excellence or processes that might be helpful to others.
Figure 4: History keeping in the SimPLE environment
(Rose et al., 1998). The user has run a simulation and produced a history,
then re-run the history macro seven times to produce the seven outputs
on the upper right.
5.8 Disseminating results: Email, listservs, digital libraries, the World Wide Web provide an excellent foundation for disseminating results but these also could be improved by thinking of the genex framework. The first circle of people who should receive an announcement of an innovation include previous and current workers. While digital libraries and online resources may make previous work accessible, extracting email (or mailing) addresses and ensuring that they are current is tedious, at best. Finding current workers in a domain is sometimes possible by listservs, newsletters, or websites of relevant communities.
A second circle of interest might be readers of papers or viewers of websites with related materials. Finding the identity of website viewers is not usually possible, but registration strategies might be developed to enable visitors to request information about future developments. The commercial bookseller amazon.com offers a service that will send you email when books on a topic or by a given author appear. Obviously, such registration lists have great commercial value, but publicly available variants are likely to emerge and be appealing because of their narrow focus. These are modern versions of what information retrieval professionals call selective dissemination of information (SDI), a policy by which users are informed on a regular basis about new publications on topics they have selected.
Digital libraries could be reconceived of as digital library communities, by extensive use of online community software to turn every object into the focus of a discussion group. Anyone retrieving a novel, scientific paper, legal precedent, or classical music score could register to participate in a discussion of that item. Such online discussion groups would be a great stimulant to creative work in many disciplines.
Additions to scientific journals, music libraries,
or art galleries would require review by editors, collections managers,
and curators. Modern technologies not only speed up the process, but also
facilitate review from multiple sources. Maybe more importantly the complexity
and cost of establishing online journals, libraries, and galleries is substantially
less than physical institutions, thereby lowering the barrier. Having more
diverse institutions that create digital libraries is also likely to stimulate
6. Architectural Scenario
The genex framework and the eight activities could reshape many forms of creative work. Earlier genex examples included education, medicine, and legislation (Shneiderman, 1998c). The following scenario has some elements of wishful thinking, but it offers an example of how work practices could relate to the genex phases (collect, relate, create and donate) and the eight activities (search, consult, visualize, think, explore, compose, review, and disseminate). This scenario assumes that genex tools would enable an architect to have a broader range of decision making power and integrate tasks that are often handled by many others.
Imagine an architect, named Susan, who is called on to design a small hotel for hikers at a national park. She might search an architectural library to collect plans for similar-sized hotels at other parks in the same region, or browse exemplars from around the world to get ideas for alternative styles, such as Swiss chalet, Austrian lodges, or Rocky Mountain log cabins (collect & search). She flips through 300 possibilities to open up her mind to a broad range of roof designs and sidings (collect & think). She also views the data on heating requirements, heat loss, and energy consumption patterns for these 300 possibilities to find strategies that are energy efficient (collect & visualize). Susan chooses a log cabin design from Colorado and adds more windows, more insulation, and solar heating panels. Her composition tools allow her to manipulate the underlying architectural model so she can resize the building to match the intended number of floors and rooms (create & explore). Then she superimposes the images on the backgrounds of two potential sites: on the hillside and at the base (create & compose).
The park managers and concessionaires who will run the hotel prefer the hillside site because of the wonderful views, rather than the base site. After consulting electronically with park commissioners and travel industry advisors, they accept the log cabin style since it fits local and tourist tastes (relate & consult). A videoconference with the client provides input about the reception desk, a commons area with a fireplace, cafe, store, and guest room dimensions shape the floor plan (relate & consult). A distinctive feature is the desire to support families and groups of hikers who wish to cook their own meals by providing communal kitchen/dining areas adjacent to the commons area. Preferences about views from the commons and dining areas influence changes in the design. In spite of the steep incline of the hillside site, engineering models enable Susan to prepare a structural design that is only 8% above the cost for the base site (create & explore, compose).
Then templates for room layouts enable Susan to rapidly make decisions about windows, insulation, and heating, while ensuring low energy consumption (create & compose). Susan uses layout tools to ensure low-cost and easy maintenance for electric wiring, plumbing, smoke detectors, phone, and cable TV. She explores ceiling, wall decorations, flooring, and furniture choices with color and fabric variations (create & explore, compose). After a virtual walk through, the client requests larger windows, which is handled by reviewing the design history and changing the window sizes (relate & review). This change causes rerouting of some wiring and an increase in the need for solar-heating capacity, but the design is complete (create & compose).
At this point, consultations begin with potential builders (relate & consult). Susan reviews the capabilities of builders and she receives bids based on the design electronically. She generates bill-of-materials lists for suppliers and a construction schedule for discussion by all parties (relate, create & compose).
The construction proceeds rapidly and opening day
is a triumph. There is already a long waiting list of families and groups
who are attracted to the idea of cooking their own meals and meeting other
hikers. Susan registers her design with the architectural society’s digital
library, contributes to a press release and web site, and sends a description
to interest managers of similar parks around the world (donate & disseminate).
Ambitious visions can be helpful in shaping more concrete research agendas. For example, Engelbart’s goal of augmenting human intellect (Engelbart and English, 1968) led to innovations such as the mouse and windows. Later, Brooks’s belief in the importance of toolmaking (Brooks, 1996) led to innovations such as haptic feedback in 3-d graphical environments. These inspirational visions were important catalysts for genex.
The ambition to support creativity led to the four phases of genex:
- Collect: learn from previous works stored in digital libraries, the web, etc.
- Relate: consult with peers and mentors at early, middle and late stages
- Create: explore, compose, and evaluate possible solutions
- Donate: disseminate the results and contribute to the digital libraries
These four phases and the eight activities described in this paper
- Searching and browsing digital libraries
- Consulting with peers and mentors
- Visualizing data and processes
- Thinking by free associations
- Exploring solutions - What if tools
- Composing artifacts and performances
- Reviewing and replaying session histories
- Disseminating results
are major challenges but the integrated combination of them could produce an environment that greatly facilitates creativity. The eight activities and their integration form a research agenda for human-computer interaction and user interface design.
The goal of supporting more creativity by more people more of the time is attractive, but there is the danger that the genex framework might be ineffective or even limit creativity. By making easy access to previous work and current workers, there is a risk that more exotic ideas will be suppressed. Similarly, using creativity supports such as simulations and composition tools may restrict imagination to only what is possible with these tools. Consultations are time consuming, and discouraging advice for novel ideas is a possible outcome. Fear that others will plagiarize compositions or steal inventions is another legitimate concern. An understanding of the dangers is important in pursuing the positive possibilities.
Between the lofty ambitions and troubling fears, lies the practical path of careful research and detailed design for the eight activities described in this paper. They need development, testing, and refinement to make them successful, find their flaws, and pursue alternatives. At every stage, widespread participation in design reviews increases the possibility that the resulting technologies will serve human needs in constructive and positive ways.
I am grateful for the thoughtful comments from Harry Hochheiser, Ross Malaga, Kent Norman, Catherine Plaisant, Jenny Preece, Richard Salter, and the anonymous reviewers. This work was supported in part by IBM Research’s University Partner Program.
Ahlberg, Christopher and Shneiderman, Ben, Visual information seeking: Tight coupling of dynamic query filters with starfield displays, Proc. CHI94 Conference: Human Factors in Computing Systems, ACM, New York, NY (1994), 313-321 + color plates.
Basalla, George, The Evolution of Technology, Cambridge Univ. Press, UK (1988).
Berners-Lee, Tim, Cailliau, Robert, Luotonen, Ari, Nielsen, Henrik F., and Secret, Arthur The world wide web, Communications of the ACM 37, 8 (1994), 76-82.
Boden, Margaret, The Creative Mind: Myths & Mechanisms, Basic Books, New York (1990).
Brooks, Frederick, Jr., The computer scientist as toolsmith II, Communications of the ACM 39, 3 (March 1996), 61-68.
Bryson, S., Kenwright, D., Cox, M., Ellsworth, D., and Haimes, R., Visually exploring gigabyte data sets in real time, Communications of the ACM 42, 8 (August 1999), 82-90.
Bush, Vannevar, As we may think, Atlantic Monthly 76, 1 (July 1945), 101-108. Also at
Candy, Linda, Computers and creativity support: Knowledge, visualisation and collaboration, Knowledge-Based Systems 10 (1997), 3-13.
Candy, Linda and Edmonds, Ernest, Supporting the creative user: A criteria-based approach to interaction design, Design Studies 18 (1997), 185-194.
Card, Stuart, Mackinlay, Jock, and Shneiderman, Ben, Readings in Information Visualization: Using Vision to Think, Morgan Kaufmann Publishers, San Francisco, CA (1999).
Charles Cave, Creativity Web Site, http://www.ozemail.com.au/%7Ecaveman/Creative/index.html(July 1999).
Couger, D., Creativity & Innovation in Information Systems Organizations, Boyd & Fraser Publ. Co., Danvers, MA (1996).
Csikszentmihalyi, M., Creativity: Flow and the Psychology of Discovery and Invention, HarperCollins, New York (1996).
De Bono, E., Lateral Thinking: Creativity Step by Step, Harper Colophon Books, New York (1973).
Derthick, Mark, Kolojejchick, John A. and Roth, Steven F., An interactive visual query environment for exploring data, Proc. ACM Symposium on User Interface Software and Technology (UIST '97), ACM Press (1997), 189-198.
Dey, A., Abowd, G., Pinkerton, M., Wood, A., CyberDesk: a framework for providing self-integrating ubiquitous software services, Proc. ACM Symposium on User Interface Software and Technology (UIST '97), ACM Press (1997), 75-76.
Dorcey, Tim, CU-SeeMe desktop videoconferencing software, Connexions 9, 3 (March 1995). Also at http://cu-seeme.cornell.edu/DorceyConnexions.html.
Elam, J. J. and Melissa, M., Can software influence creativity? Information Systems Research 1 (March 1990), 1-22.
Engelbart, Douglas C. and English, William K., A research center for augmenting human intellect, AFIPS Proc. Fall Joint Computer Conference 33 (1968), 395-410.
Gardner, Howard, Creating Minds: An Anatomy of Creativity Seen Through the Lives of Freud, Einstein, Picasso, Stravinsky, Eliot, Graham, and Gandhi, Basic Books, New York (1994).
Hearst, M., User interfaces and visualization, In Baeza-Yates, R. and Ribeiro-Neto, B. (Editors) Modern Information Retrieval, Addison-Wesley Publishing Company, Reading, MA (1999).
Johnson, Steven, Interface Culture: How New Technology Transforms the Way we Create and Communicate, HarperCollins, New York (1997).
Jones, P. M. and Schneider, K. J., Learning environment for magnetic resonance spectroscopy (LEMRS): Supporting apprenticeship learning in operational environments, Journal of Educational Multimedia and Hypermedia 5, 2 (1996), 151-177.
Kiesler, Sara (Editor), Culture of the Internet, Lawrence Erlbaum Assoc., Inc., Mahwah, NJ (1997).
Koenemann, Juergen and Belkin, Nicholas, A case for interaction: A study of interactive information retrieval behavior and effectiveness, Proc. CHI ‘96 Human Factors in Computing Systems, ACM Press, New York (1996), 205-212.
Kuhn, Thomas S., The Structure of Scientific Revolutions: 3rd Edition, University of Chicago Press, Chicago (1996).
Marchionini, Gary, Information Seeking in Electronic Environments, Cambridge University Press, UK (1995).
Massetti, Brenda, An empirical examination of the value of creativity support systems on idea generation, MIS Quarterly 20, 1 (1996), 83-97.
Massetti, Brenda, White, Norman H., and Spitler, Valerie K., The impact of the World Wide Web on idea generation, Proc. 32nd Hawaii International Conference on System Sciences, IEEE Press (1999).
Mayer, Richard E., Thinking, Problem Solving, Cognition: 2nd Edition, W. H. Freeman, New York (1992).
Nakakoji, K., Suzuki, K., Ohkura, N., and Aoki, A., A framework to support creativity in multimedia information design, Proc. Human-Computer Interaction INTERACT'97, Chapman & Hall, London (1997), 212-219.
National Research Council - Committee on a National Collaboratory, National collaboratories: Applying information technology for scientific research, National Academy Press, Washington, DC (1993).
North, Chris and Shneiderman, Ben, Snap-Together Visualizations to support multiple window coordinations, Univ. of Maryland Dept of Computer Science Technical Report (1999).
Olson, Gary M. and Olson, Judith S., Research on computer supported cooperative work, In Helander, M. G., Landauer, T. K., and Prabhu, P. V. (Editors), Handbook of Human-Computer Interaction: Second Edition, Elsevier, Amsterdam (1997), 1433-1456.
Plaisant, Catherine, Rose, Anne, Milash, Brett, Widoff, Seth, and Shneiderman, Ben, LifeLines: Visualizing personal histories, Proc. of ACM CHI'96 Conference: Human Factors in Computing Systems, ACM, New York, NY (1996), 221-227, 518.
Polya, G., How to Solve It: A New Aspect of Mathematical Method: Second Edition, Doubleday Anchor Books, Garden City, NY (1957).
Rose, Anne, Eckard, David, and Rubloff, Gary, An application framework for creating simulation-based learning environments, , Univ. of Maryland Dept of Computer Science Technical Report CS-TR-3907 (1998),
Roth, Steven F., Lucas, Peter, Senn, Jeffrey A., Gomberg, Cristina C., Burks, Michael B., Stroffolino, Philip J., Kolojejchick, John A. and Dunmire, Carolyn, Visage: A user interface environment for exploring information, Proc. Information Visualization '96, IEEE, Los Alamitos, CA (1996), 3-12.
Shneiderman, Ben, Designing the User Interface: Strategies for Effective Human-Computer Interaction: Third Edition, Addison-Wesley Publ. Co., Reading, MA (1998a)
Shneiderman, Ben, Relate-Create-Donate: A teaching philosophy for the cyber-generation, Computers & Education 31, 1 (1998b), 25-39.
Shneiderman, B., Codex, memex, genex: The pursuit of transformational technologies, International Journal of Human-Computer Interaction 10, 2 (1998c), 87-106.
Shneiderman, B., Byrd, D., and Croft, B., Clarifying search: A user-interface framework for text searches, http://www.dlib.org/dlib/january97/01contents.html (January 1997).
Tauscher, L. and Greenberg, S., How people revisit web pages: Empirical findings and implications for the design of history systems, International Journal of Human-Computer Studies, Academic Press, (1997).
Wright, William, Information animation application in the capital markets, Proc. IEEE Information Visualization ’95 (1995), 19-25, 136-137.