27th ACM User Interface Software and Technology Symposium

UIST 2014 Author's Guide

This guide describes the format, deadlines and other relevant information for submissions to UIST 2014. Authors submitting material to UIST 2014 are encouraged to use this guide to learn about the UIST review process, what makes a strong UIST paper and how to write a successful submission.

Introduction

UIST features papers, demos, and posters, as described in the Call for Participation. While the material in this guide is primarily oriented towards paper authors, its general emphasis on quality and stringent review is also applicable to authors of demo and poster submissions. Accepted papers will be presented during three days of technical sessions at the conference and published in the conference proceedings.

UIST Requires Novel Manuscripts

Paper submissions must not have been published previously in the English language. A paper is considered to have been previously published if it has appeared in a peer-reviewed journal or meeting proceedings that is reliably and permanently available afterward in print or electronic form to non-attendees. This includes papers that are reviewed only as abstracts, but are published as a complete paper.

The paper cannot be considered a novel manuscript if it reports an incremental update or a small improvement of the previously published work. Although the paper can be based on the earlier archival publications by the authors, significant new developments and findings have to be reported for the paper to be considered a novel manuscript. As a rule of thumb a paper has to include about 70% new material to be considered a novel contribution. We also encourage the authors to submit a complete work rather than dividing it into smaller pieces.

For UIST submission purposes, a paper is not considered to have been previously published if it was presented earlier in forms explicitly labeled as “non-archival”, even if they are, in fact archived. This includes for example CHI extended abstracts (including alt.chi, works-in-progress, posters, demos, etc), SIGGRAPH Emerging Technology, and UIST posters and demos. Also, work that builds on previous non-archival work should contain at least 30% new material. However, authors must cite this previous work in their manuscript. If you are unsure about whether something is considered archival, please contact papers@uist.org.

Authors wishing to submit work containing substantial portions of work published elsewhere also need to check their copyright agreement with the original publisher to make sure that this is permissible according to that agreement.

Re-publication of work in English that was previously published in another language

English is considered the international language of ACM SIGCHI and its journals and conferences. Work that has previously been presented or published in a language other than English may be translated and presented or published in English in SIGCHI journals and conferences insofar as ACM SIGCHI is concerned. The original author should typically also be the author (or co-author) of work translated into English, and it should be made clear that this is a translation. We encourage authors whose work was originally published in languages other than English do this if they feel their work is of sufficient relevance and quality to be useful to a wider international audience. Of course, it is not acceptable to translate the original work of another author and present it as one's own. Authors wishing to publish in English a work originally published elsewhere also need to check their original copyright agreement with the original publisher to make sure that this is permissible according to that agreement.

UIST Supports Sharing Preliminary Work

UIST sees significant value in sharing early work through posters, demos, and informal venues. Indeed, UIST strongly encourages the submission of exciting, early research as a UIST poster or demo. Sharing preliminary research through these short, lightly reviewed work-in-progress or extended-abstract venues does not inhibit subsequent publication at UIST - provided that the UIST submission makes a contribution beyond (i.e., provides more or newer information than) the previous, shorter document. Non peer-reviewed documents such as these and tech reports are not considered prior publications, and thus do not preclude submission of a paper on the same topic by the same authors. Prior work should of course be referenced appropriately.

Concurrent Submission is Prohibited

A paper identical or substantially similar (or even a subset or superset) in content to one submitted to UIST should not be simultaneously under consideration at another conference or journal during the entire duration of the UIST review process (i.e., from the submission deadline until the notification of decisions are emailed to authors). This restriction applies even if the overlap in review timelines between UIST and another venue is just a few days or a few hours, and even if it is your intention to withdraw the submission from the other venues as soon as it is accepted by one of them. This restriction also applies even if the other venue allows simultaneous submission. We will make every effort to identify simultaneous submissions, and UIST reviewers are often familiar with the papers under review at other related conferences and journals; as such, submissions that are substantially similar run the risk of being rejected by UIST and the other venues on grounds of duplication alone.

Revision for Journals is Encouraged

You are encouraged to submit a revised and extended version of your UIST paper to a journal such as ACM Transactions on Computer-Human Interaction or Human-Computer Interaction after it has been presented at the conference.

Anonymous submission process

Paper submissions are anonymous. You are required to make a reasonable effort
to purge identifying information from your submission:

What does anonymous mean for UIST submissions? Primarily -- as with CHI -- it means that submissions *must* remove all author and institutional information from the title and header area of the first page of the paper. Submissions that do not do so will be rejected without review.

Furthermore, all references must remain intact. If you previously published a paper and your current submission builds on that work, the complete reference with author's name must appear in the references. Blank references (e.g., "12. REMOVED FOR REVIEWING") should be avoided. We strongly encourage authors refer to their previous work in the third person. Further suppression of identity in the body of the paper, while encouraged, is left to the authors' discretion.

What if the work I am writing about is widely publicized already (e.g., a website, an application, or a performance)? While the details of anonymization in the body of the paper are ultimately left to the authors' discretion, we understand that some work is difficult (or impossible) to anonymize without degrading the quality of the writing. In these cases, we encourage the authors to ensure that details relevant for review of this paper are included.

Am I allowed to publicize my work while the review process is ongoing? While publicizing and promoting work during the review process goes against the spirit of anonymous review, we understand that there are competing interests that make publicity important. The UIST community has agreed that such publicity should not be explicitly prohibited or penalized. However, we encourage authors to wait until the review process is over to publicize their work.

Why did UIST adopt this particular strategy of lightweight anonymization? UIST has a long tradition of excellent, thoughtful reviewing. This policy seeks to balance two goals. The first goal is to emphasize for all parties involved that reviews assess the content of a submission, not its authors. This is why names must be omitted from the masthead. The second goal is to encourage papers that clearly explain the research. Sometimes doing so requires (at least implicitly) disclosing information about the authors or an institution. This is why anonymization within the body of the paper is encouraged, but at the authors' discretion.

If you have comments or questions about this policy, please email papers@uist.org.

Submissions to the Posters, Demos, and Doctoral Symposium track will be non-anonymous, as in previous years.

The Reviewing Process

The UIST review process is confidential and confidentiality of submissions is maintained from their submission to their publication date (typically the date of the first day of the conference).

The Papers Committee and a set of external reviewers, both consisting of recognized experts, will review submitted papers. Then, at their meeting (June
19-20, 2014), the committee will select those papers to be presented at UIST 2014.

For 2014 the Committee will be using the following process:

1AC and 2AC Assignment: In the week following the
submission deadline, the Papers Chairs will assign each submitted paper to a
primary reviewer (1AC) and a secondary reviewer (2AC) who are both members
of the paper committee. Papers that are inappropriate may be rejected during
this assignment process, without being sent to a primary reviewer. Papers
will normally be rejected at this stage only if they are clearly off-topic
for UIST 2014, or if they are discovered to have been published previously
or to have been submitted simultaneously to another conference or journal.

Auto Reject Stage: The primary reviewer may, upon
conferring with the secondary reviewer and the Technical Papers Chairs,
recommend a paper be rejected without additional review. A paper will
normally be rejected at this stage only if it falls into one of the
categories listed above, but this fact was not detected during the initial
paper assignment. It is possible, although unlikely, that a paper may also
be rejected at this stage if it solves a problem that is known to be already
solved; or if it does not cite (and the authors seem unaware of) important
prior work on the same problem and doesn't address how it is different; or
if it has no evaluation via proof, experiment, or analysis when such is
required; if it is solving a problem sufficiently minor that the senior
reviewers do not believe that it belongs in the program; or if it addresses
a topic that is clearly outside the purview of UIST.

Tertiary Reviewers Assignment: The primary reviewer
distributes each paper to two additional experts, called tertiary reviewers.
The secondary reviewer also distributes the paper to one additional tertiary
reviewer. Thus, each paper submitted for review will be seen by five
different people. This assignment process aims to reduce the impact of any
single person on the final outcome for the paper by letting two different
people (1AC and 2AC) assign tertiary reviewers.

Reviews before Rebuttal: The three tertiary reviewers
will write full reviews. Thus, at least three full reviews are written for
each paper. The primary reviewer knows the identities of the authors of the
papers, but the tertiary reviewers do not. 1AC will summarize the tertiary
reviewer sentiment in the meta review section of their review. If there is
substantial disagreement among the external reviewers, the 1AC will prepare
a full review of the paper.

Discussion before Rebuttal: Between the review
submissions and rebuttal phases, the 1AC may elect to begin a discussion
amongst the reviewers (ACs and tertiaries) to clarify reviews and strive for
consensus.

Rejection Threshold: After the primary and tertiary
reviewers complete their reviews, any paper for which all reviews fall below
a rejection threshold will be rejected. The rejection threshold is defined
by a combination of average scores and scores distribution, e.g., papers
with widely divergent scores have a higher chance of passing the threshold
than papers with uniformly low scores. These rejected papers will not have a
rebuttal or be discussed at the PC meeting. Authors of the rejected papers
will receive notifications at this point and can submit the work to other
venues. Authors of all papers that are above the rejection threshold will
have a chance to write a rebuttal.

Rebuttal: After the 3 reviews and 1 meta review are
complete, reviews will be distributed to authors. Authors will have 7 days
to submit a rebuttal if they feel that the reviewers have made substantive
errors, or to answer specific questions posed by the reviewers. We encourage
the submission of a rebuttal regardless of the scores that authors receive
in case the concerns of reviewers comes up in discussion at the Program
Committee meeting. The rebuttal period will be from June 2 through June 9,
2014. The rebuttal is confined to 4000 characters in length and must be
self-contained. For instance, URLs to additional material are not allowed.
The rebuttal period is for addressing factual errors in the reviews, not for
getting revised text or new results into the review process. Any such novel
material will be disregarded by the reviewers and Program Committee.

Post rebuttal Meta Reviews: After the rebuttal, the
primary reviewer will have a chance to revise the meta reviews taking into
account the rebuttal, and the 2AC will also prepare a meta review, but it
can be very short (e.g. "This is great work, reviewers are all positive and
I recommend acceptance"). The 2AC will prepare a full review if they are in
substantial disagreement with the overall reviewer sentiment.

Post-Rebuttal Discussion Stage: Between the end of the
rebuttal submission and committee meeting, the reviewers (ACs and
tertiaries) will read the authors’ rebuttals, confer intensively about the
papers, and prepare a recommendation for the committee meeting by June 19,
2014. Since the tertiary reviewers do not know the names of the authors, the
authors should maintain anonymity in their rebuttals. In addition, the
tertiary reviewers do not know each other's identities, so they too must
maintain anonymity during the discussion. The preliminary recommendation
agreed on at this stage will be either accept, reject, or if agreement on a
recommendation cannot be reached, a third option is to table the paper for
further review and discussion.

Program Committee Meeting: The full Papers Committee
meets June 19-20, 2014, to determine acceptance or rejection of each paper.
In cases where a consensus on a paper was not reached during the pre-meeting
discussion phase, additional committee members may read the paper, and their
evaluations will be taken into account in the decision.

Possible Outcomes for a Paper

Email notifications of the Technical Papers Committee's decisions should be sent no later than June
23, 2014. The notifications will place each paper in one of the following two categories:

Rejected

Conditionally accepted

Conditionally accepted papers undergo a second review
process in which a referee (an associate chair of the Program Committee)
verifies that the final version of the paper is acceptable (that any required
changes have been made, and that other changes made by the authors, perhaps in
response to reviewer comments, have not compromised the paper in any way). The
length of the camera-ready copy (rounded up to the nearest page) must be less
than or equal to the length of the original submission unless there is a strong
justification to increase the length of the manuscript, which should be agreed
upon by the primary reviewer. This second and final stage determines the final
acceptance status of all papers. The referees' decisions are final. Papers that
do not satisfy the referees in the second stage of review and/or that are not
uploaded in final form by the final deadline of July 7, 2014, together with the
original or revised versions of the submitted supplementary material, will be
rejected. Accepted papers will appear in the conference proceedings.

Review Criteria

A good UIST submission will result in both a respectable document for the proceedings and a good conference talk. As an author, you should ask yourself the following questions before writing your paper. Submissions that do not provide good answers to these questions are unlikely to be accepted.

What problem are you solving? There is no point in publishing a paper unless it presents a solution to a problem. Try to state all your constraints and assumptions. This is an area where it can be invaluable to have someone not intimately familiar with your work read the paper. Include a crisp description of the problem in the abstract and try to suggest it in the title. The choice of senior reviewer for the paper is based almost entirely on the answer to this question.

What were the previous solutions? What are the relevant published works in your problem area? What deficiencies in their solutions are you trying to overcome? How does the new solution differ from previously published results? Don't expect the reviewers to know this information without your telling them in the paper, as they are unlikely to remember the precise details of all the relevant literature. Make specific comparisons between your work and that described in the references; don't just compile a list of vaguely related papers. Remember, poorly written discussion of related work is one of the most common reasons of paper rejection.

How well did you solve your problem? Based on your problem statement, what did you accomplish? You are responsible for proving that the problem is solved. Include pictures, statistics, or whatever is required to make your case. If you find this part of the paper difficult to write, perhaps the work is not yet finished and the paper should be deferred until next year.

What does this work contribute to the field? What are your new ideas or results? If you don't have at least one new idea, you don't have a publishable paper. Can your results be applied anywhere outside of your project? If not, the paper is probably too special-purpose for UIST. On the other hand, beware of trying to write a paper with too large a scope.

Is the paper complete? The question that generates the most discussion at the program committee meeting is whether a paper is complete. If the paper presents an algorithm or technique, an experienced practitioner in the field should be able to implement it using the paper and its references. If the paper claims to present a faster or more efficient way of implementing an established technique, it must contain enough detail to redo the experiment on competing implementations. When you quote numbers, be sure that they do not lie; state clearly whether they were measured, simulated, or derived, and how you did the measurements, simulations, or derivations. For example, CPU time measurements are meaningless unless the reader is told the machine and configuration on which they were obtained.

Does the paper contain too much information? A paper should be as succinct as possible. A long, poorly written paper will likely be rejected. If you have solved a single, practical problem, don't try to generalize it for the purposes of publication. If you have a formal theory or elaborate architecture, don't include all the vagaries of the implementation unless they are critical to the utility of the theory. Don't include the contents of your user's manual; instead, describe the model or functionality achieved. You should assume your audience has a working knowledge of user-interface development and access to the major journals in computer science, electrical engineering, and psychology. A short conference paper can only present a few concise ideas well.

Can this paper be presented well? While UIST papers are judged primarily as technical papers, some consideration is given to how suitable the topic is for a conference presentation. Think of how you would present your ideas, and how big the audience is likely to be. Papers that have a small number of concisely stated new ideas and that are visually interesting tend to appeal to a large audience and be easy to present. As recent conferences clearly show, these criteria do not eliminate papers that have taxonomies or strong theoretical content, or that appeal to a specialized audience, as long as they contain significant new ideas.

Systems and Applications Papers

Papers that present new algorithms, techniques, or hardware are the easiest to write and review. If the content is truly new and effective and makes a significant contribution to the state of the art, the paper is likely to be accepted for UIST. Equally valuable, but harder to write and evaluate, are papers that describe systems and applications. While the criteria above will be applied to all papers, here we offer some additional guidance for authors of systems and applications papers.

Systems

A systems paper may present a real system, either by a global survey of an entire system or by selective examination of specific themes embodied in the system. Alternatively, it may present the design for a system that includes ideas or techniques you feel are important to present to the technical community, even without an implementation. Make it obvious from the abstract and introduction which kind of paper yours is.

If a system has been implemented, include information about how it has been used and what this usage shows about the practical importance of the system. Do the users include anyone other than the authors? Do they depend on it for their work or do they just play with it? Have formal user studies been done and, if so, what are the results? While user testing is not required for UIST papers, authors should be careful not to make unsubstantiated claims for systems which have not been tested. However, papers can say that the system "might be easier to use because . . ." or that "feature xxx is expected to make the system easier to use because . . .".

Also, if the system has been implemented, the inclusion of screen snapshots is vital to convincing readers and reviewers that the system is real. Do not fake or redraw screen shots; fakery is usually obvious and is a clear indication that the system is not real.

If the system is still being designed, it is most important to state the design criteria and constraints. Back up your decisions with references to similar systems that are already implemented, stating what problems you are solving or what solutions you are including in your design. Reviewers tend to be very skeptical of design-only papers, unless there are new ideas of obviously high quality.

It is very important that you clearly identify what is implemented and what is merely designed. Do so at the beginning of the paper, not the end!

The paper should emphasize the novel aspects of the system, what underlying themes are present, what problems were anticipated/encountered in building the system, and how the structure presented provides solutions to these problems. In general, avoid details that are only of interest to users of the system and concentrate on those that would be interesting to someone else building a similar system. Avoid sweeping claims, especially for paper designs!

At UIST 2007, Dan Olsen ran a panel on
Evaluating Interface Systems Research. We strongly encourage you to read the associated paper, which is available in the ACM Digital Library and on Dan's web page.

Applications

An application paper presents an application area and a problem in that area that benefited from innovative user interface techniques. The techniques used don't have to be unique, but their use must not be completely obvious. The author should concentrate on what was learned, and how well the user interface works compared to previous techniques for solving the same problem. As in a systems paper, the intended audience should be other user-interface developers, not the end user. Successful applications papers provide some general insight into the use of interactive techniques to solve problems.

Be Kind to Your Reviewers

As previously stated, a UIST paper is accepted or rejected based on the ratings it receives from the reviewers. Paper reviewing is a volunteer activity; the only benefit that the reviewers receive is the knowledge that they have contributed to the field. In many ways, the success of the technical program is more a function of the quality of the reviewers than the work of the program chair or the program committee. We are lucky to have excellent reviewers for this conference, and paper authors should be considerate of them.

Many of the senior people in this field receive a large number of papers to review each year. With this in mind, authors should think about their reviewers when they are preparing their papers. In the following paragraphs we provide some advice on how to prepare your paper so it makes the best impression on a reviewer.

The most important point is to put a reasonable amount of effort into the production of your paper. When the author appears to have put little effort or thought into the production of a paper, the reviewer is not motivated to read the paper carefully and produce a good review. There is no excuse for spelling mistakes in papers, since spelling checkers are now widely available. A large number of misspelled words in a paper just indicates to the reviewer that the author didn't care enough about his or her paper to run the spelling checker on it. With this attitude on the part of the author, why should the reviewer bother doing a good job? The same goes for missing references, mislabeled figures, wrong format, and other trivial problems that could be caught by thorough proofreading. Don't expect reviewers to read your paper carefully if you are not willing to read it carefully first!

UIST reviewers will have several papers to read in a short period of time. Therefore, you should write your paper so that it is easy to read. Try to write your paper so it flows smoothly. A paper that is easy to read will usually get a higher rating.

Has this paper been submitted to a conference before and been rejected? If this is the case, think carefully before you submit it again! There must have been some reason why the paper was rejected. (Yes, we all blame bad reviewing, but there must also have been some other reason.) Read the reviewers' comments and try to determine what they would like to see changed, and then make those changes. There is a surprisingly good chance that a resubmitted paper will be reviewed again by a reviewer who gave it a poor rating before (or who recalled the deliberations over your previous submission in the program committee meeting of another conference). If the paper has not been changed to reflect that reviewers’ comments, it is likely that your paper will get an even lower rating. Yes, sometimes the reviewers’ comments are wrong (reviewers are only human after all), but this usually implies that you need to write more clearly or provide more evidence for your claims. Each of us has received what we originally considered to be bad reviews on some paper, but after calm consideration (weeks, or even months, later) realized that these reviews pointed out real faults in the paper. If a hand-picked reviewer is confused about what you are saying, the chances are good that the average reader will also be confused!

A highly recommended technique is to write the paper, and let it sit on your desk for a week or two. Then go back and read the paper as if you were a reviewer who doesn't know the author. While you are writing a paper, you are too closely tied to the work to be able to criticize it effectively. After a break of a week or two, you will be much more objective and may see organizational problems that weren't evident when you were actively working on the paper.

A Final Note

The single most important thing you can do to improve the odds of having your paper accepted is to have your own colleagues do an "in house" review of it before you submit it to the conference for formal review. That requires beginning far enough in advance of the deadline that you have a protective cushion in your schedule. But remember, the majority of UIST papers are rejected. It's far better to start a week or two earlier and get your paper accepted, than it is to get rejected and feel as if you wasted your time.

Supporting Video Material

Since user interfaces are inherently interactive, authors are encouraged to
include a video figure with their papers, which will be kept confidential during
the review process. As with the rest of the paper, video figures should be
anonymized.

Authors should make video figures short and accessible without being
misleading. A video should give the same impression as a live demo. For example,
a long computational pause can only be removed if its absence is made obvious
through techniques such as a visual dissolve and a clear indication (verbal
and/or visual) of how much time was removed. Videos about technology mock-ups
should be clearly indicated as such. Mock-ups should be avoided when the video
is about an implemented system. The video figure accompanying a submission for
review is used only to help reviewers evaluate the submissions. Acceptable
videos can be made without expensive production or special effects. A camcorder,
tripod, and some planning can help guide the viewer's attention. A smooth zoom
into the interaction area and then out to the full screen is often much more
effective than a static screenshot. Show how the user manipulates the input
devices if that is relevant. The proceedings chairs have put together a guide
describing how to make good videos.

Supporting video need not be stand-alone, because the reviewers will have the
paper. While the paper should be understandable without the video, the paper may
include references to the video. In particular, authors may use video sequences
showing actual use of the proposed system to give readers and reviewers an
impression of how the interaction unfolds like and/or how users responded to
using the presented system. As video figures will be included with the paper in
the ACM digital library, authors may assume that everyone who has the video has
the paper, and vice versa. The burden is on the authors to ensure that videos
figures are rendered in file formats and codecs that are accessible on all
common platforms.

Video Proceedings

All authors are encouraged to use video when appropriate as part of their
conference presentation. A high-quality master copy of each video figure file
should be sent to the proceedings chairs by the date indicated in the acceptance
notifications for inclusion in the USB proceedings, which will be distributed to
conference attendees only. Videos figures will also be included as supplemental
material for the corresponding papers in the ACM Digital Library. Information
will be provided later about the video formats that will be accepted for the
USB.

Rest assured that we will not duplicate for public distribution any video included with your initial submission, so please don't worry! Those files will only be used during the review process, and then all copies received by UIST will be destroyed or deleted.

Format

Submissions for review must also be in the final conference format, except they should have page numbers so the reviewers can more easily refer to portions. Detailed format instructions are available at the SIGCHI conference publication format site.

As indicated in the samples, paper submissions are anonymous. Submissions to the posters, demos, and Doctoral Symposium track are not anonymous, but should use the same format.

It is to the author's advantage to make the reviewers' job as easy as possible! A well-written paper containing useful illustrations will appeal to reviewers. Given that many of the papers presented at UIST are about systems, it is not surprising that most accepted papers include pictures or a video to support the ideas presented. It is not necessary to have the ultimate picture or the final, polished version of the video for review. However, the reviewers are much more likely to prefer papers containing some indication that the author's claims are supported than those that leave the final results to the reviewers' imagination.

Conference Presentation

An author of each accepted paper is expected to give a conference
presentation. Papers in a short format (6 pages or fewer) will have a shorter
presentation time than papers in a longer format. Exact length requirements will
be provided soon after notification of acceptance. Authors should include a note
with their submission if they are planning anything for the presentation that is
not obvious from the document; for example, an author may point out that there
will be a video or live demonstration at the conference showing the results
described in a paper. Authors of accepted submissions will be sent detailed
instructions for preparing their conference presentation.

Copyright

The authors must be prepared to sign an ACM copyright transfer form before the submission is published. The author retains several rights, including the right to post versions on their home page and employer web site. See the ACM copyright policy and copyright form for details.

History

This document was last updated in January 2014 by Mira Dontcheva and Daniel
Wigdor, who inherited it from Ivan Poupyrev and Takeo Igarashi, who inherited it
from Hrvoje Benko and Celine Latulipe, who inherited it from Maneesh Agrawala
and Scott Klemmer (who used material provided by Saul Greenberg), who inherited
it from François Guimbretière, who inherited it from Michel Beaudouin-Lafon, who
inherited it from Ravin Balakrishnan and Chia Shen, who inherited it from Ken
Hinckley and Pierre Wellner, who inherited it from Dan Olsen, who inherited it
from Steve Feiner, who inherited it from Joe Konstan, who inherited it from
Michel Beaudouin-Lafon, who inherited it from Ari Rapkin, who inherited it from
Beth Mynatt, who inherited it from George Robertson, who inherited it from Marc
H. Brown, who inherited it from George Robertson, who got lots of help on it
from Steve Feiner, Brad Myers, Jock Mackinlay, Mark Green, Randy Pausch, Pierre
Wellner, and Beth Mynatt.