Technical Report
`
`UCAM-CL-TR-574
`ISSN 1476-2986
`
`Number 574
`
`Computer Laboratory
`
`Sketchpad: A man-machine graphical
`communication system
`
`Ivan Edward Sutherland
`
`September 2003
`
`New preface by Alan Blackwell and
`Kerry Rodden.
`
`15 JJ Thomson Avenue
`Cambridge CB3 0FD
`United Kingdom
`phone +44 1223 763500
`
`http://www.cl.cam.ac.uk/
`
`BlackBerry Exhibit 1015, pg. 1
`
`

`
`c(cid:13) 2003 Ivan Edward Sutherland
`This technical report is based on a dissertation submitted January
`1963 by the author for the degree of Doctor of Philosophy to the
`Massachusetts Institute of Technology.
`
`Technical reports published by the University of Cambridge
`Computer Laboratory are freely available via the Internet:
`
`http://www.cl.cam.ac.uk/TechReports/
`
`Series editor: Markus Kuhn
`
`ISSN 1476-2986
`
`BlackBerry Exhibit 1015, pg. 2
`
`

`
`Preface to this Electronic Edition
`
`Alan Blackwell and Kerry Rodden
`University of Cambridge Computer Laboratory
`
`Ivan Sutherland’s Sketchpad is one of the most in(cid:3)uential computer pro-
`grams ever written by an individual, as recognized in his citation for the Tur-
`ing award in 1988. The Sketchpad program itself had limited distribution (cid:151)
`executable versions were limited to a customized machine at the MIT Lincoln
`Laboratory (cid:151) so its in(cid:3)uence has been via the ideas that it introduced rather
`than in its execution. Sutherland’s dissertation describing Sketchpad was a
`critical channel by which those ideas were propagated, along with a movie of
`the program in use, and a widely-cited conference publication [10]. Copies of
`the dissertation were distributed relatively widely, but it was never published
`commercially. It is still available in the form of a technical report from MIT,
`but we believe it deserves wider readership (cid:151) hence this electronic archival
`publication.
`After 40 years, ideas introduced in Sketchpad still in(cid:3)uence how every
`computer user thinks about computing. It made fundamental contributions in
`the area of human(cid:150)computer interaction, being one of the (cid:2)rst graphical user
`interfaces. It exploited the light-pen, predecessor of the mouse, allowing the
`user to point at and interact with objects displayed on the screen. This antic-
`ipated many of the interaction conventions of direct manipulation, including
`clicking a button to select a visible object, and dragging to modify it. Smith’s
`Pygmalion [9], heavily in(cid:3)uenced by Sketchpad, made a more explicit argu-
`ment for the cognitive bene(cid:2)ts of this kind of direct interaction and feedback,
`coining the term (cid:147)icon(cid:148), and making it clear that graphical images could rep-
`resent abstract entities of a programming language. Smith was a member of
`the team that developed the Xerox Star workstation on these principles; in a
`retrospective article [4] they acknowledge that (cid:147)Sketchpad in(cid:3)uenced Star’s
`user interface as a whole as well as its graphics applications(cid:148), providing a di-
`rect link to the commercialization of the Macintosh and Windows interfaces
`and widely recognized bene(cid:2)ts of direct manipulation [8].
`Sketchpad encountered a critical challenge that remains central to human-
`computer interaction. Sutherland’s original aim was to make computers ac-
`cessible to new classes of user (artists and draughtsmen among others), while
`retaining the powers of abstraction that are critical to programmers. In con-
`trast, direct manipulation interfaces have since succeeded by reducing the lev-
`els of abstraction exposed to the user. Ongoing research in end-user program-
`
`BlackBerry Exhibit 1015, pg. 3
`
`

`
`4 m
`
`ing continues to struggle with the question of how to reduce the cognitive
`challenges of abstract manipulation [1]. Nevertheless, Sutherland’s attempt to
`remove the division between users and programmers was not the only sys-
`tem that, in failing to do so, provided the imaginative leap to a new program-
`ming paradigm. Nygaard and Dahl’s Simula [7] was the (cid:2)rst conventional
`programming language incorporating the principles of object orientation, but
`Sketchpad’s implementation of class and instance-based inheritance (though
`not called objects) predated Simula by several years.
`There appears to have been a common in(cid:3)uence through the work of Dou-
`glas T. Ross, who is mentioned in the acknowledgements of this dissertation
`and also cited in the MIT Lincoln Laboratory technical report based on it. Ross
`sat on the Algol 68 committee with C. A. R. Hoare in the mid-1960s, where his
`previous work on a record-like data structure (called a plex) in(cid:3)uenced Hoare’s
`own ideas(cid:3) on abstract data types [3], later credited by Nygaard and Dahl as
`the origin of the class de(cid:2)nition mechanisms in Simula [7].
`Alan Kay’s seminal Dynabook project, which led both to the Xerox Star
`and to the explosion of interest in object oriented programming through his
`language Smalltalk, was directly in(cid:3)uenced by Sketchpad. Kay has written of
`the fact that the genesis of Smalltalk lay in the coincidental appearance on his
`desk of both a distribution tape of Simula and a copy of Sutherland’s Sketch-
`pad thesis [5]. Kay recognized that the two systems were based on the same
`underlying type concepts (apparently derived via two different routes from
`Ross’s plex), and that these could form the basis of a more widely usable pro-
`gramming system. In comparing these two routes of in(cid:3)uence, Simula was a
`far larger project than Sketchpad, rightly recognized as the (cid:2)rst object-oriented
`programming language, but we hope that the special emphasis of Sketchpad
`on supporting abstraction in the user interface itself may yet become viable as
`a result of ongoing research efforts [2,6].
`As with many early publications of computer science, this dissertation is
`also interesting for the way in which it explores important concepts that are
`now considered familiar, but which at the time demanded continual small dis-
`coveries by every researcher. The (cid:2)rst-person account of the history of the
`project in Chapter 2 reads almost like an excerpt from an autobiography, as
`Sutherland describes how he had to (cid:147)follow the stumbling trail(cid:148) towards gen-
`erality, through the different versions of Sketchpad. His rather charming pro-
`posal that dynamic data structures should be described using the terminology
`(cid:147)hen and chickens(cid:148) has been a sad loss when compared to the far more prosaic
`terminology of linked lists and garbage collection. The struggles of develop-
`ing custom hardware while also exploring far-reaching abstractions are also
`far removed from current research experiences.
`Chapter 9 provides an immediate illustration of how far computer graph-
`ics has moved on in the 40 years since Sketchpad’s development. For example,
`Sutherland says that (cid:147)if the almost identical but slightly different frames that
`are required for making a motion picture cartoon could be produced semi-
`automatically, the entire Sketchpad system could justify itself economically
`
`(cid:3)Personal communication with C. A. R. Hoare.
`
`BlackBerry Exhibit 1015, pg. 4
`
`

`
`5
`
`in another way(cid:148). Now, of course, we are used to seeing entire feature (cid:2)lms
`created from computer graphics. Also, in choosing manipulation of facial fea-
`tures as an example, Sutherland has anticipated the sometimes controversial
`facilities available in modern photograph editing tools.
`Sutherland’s clear writing makes all of these issues a fresh source of en-
`joyment to the contemporary reader, and we hope that it will reach a new
`audience with the assistance of this electronic edition, continuing the great in-
`(cid:3)uence that Sketchpad has had on both users and programmers.
`
`This Edition
`
`Our aim in preparing this edition has been to create an archival copy of the
`Sketchpad dissertation, suitable for electronic access and scholarly reference.
`Although created in consultation with current international research efforts in
`electronic archival, it is clear that there are, as yet, no common conventions for
`electronic archive formats. Our priorities have been that this edition should be
`accessible for download using current technology (i.e. in a relatively small (cid:2)le
`size), that it should be suitable for electronic search and indexing, that it should
`be easy to read both on paper and on the screen, and that it should be faithful
`to the original document. These have not been easy criteria to meet, and our
`chosen solution (LATEX to PostScript to PDF) has several disadvantages, but is
`the best overall solution we could (cid:2)nd.
`There are some editorial choices that should be explained. We thought it
`important to indicate original page numbering, so that citations of the original
`dissertation could be traced, but wished to avoid the decreased readability
`that would have resulted from simply reproducing the original double-spaced
`typescript. We therefore chose not to preserve page breaks and line breaks,
`instead marking the positions of the original page breaks (with the k symbol)
`throughout the main body of the text, giving the original page number in the
`margin (next to the # symbol). For (cid:2)gures, the original page number is noted
`in the caption. We also chose not to correct any errors we found in the original
`document, in order to provide the textual equivalent of a facsimile edition.
`These include a few spelling errors (to Ivan’s embarrassment), and also the
`rather idiosyncratic fact that the original dissertation had two pages 106.
`An exact facsimile copy of the original dissertation (where the pages have
`simply been scanned, not transcribed) can be purchased in hardcopy or PDF
`from the Digital Library of MIT Theses at http://theses.mit.edu/. Each
`individual page can also be viewed as a GIF image, free of charge, which may
`be a useful reference for readers wishing to check the layout of the original.
`We are very grateful to Ivan Sutherland, who has encouraged this project,
`and who personally proof-read the original scanned text. We are also grateful
`to Malcolm Sabin, who kindly loaned us his original copy of the dissertation
`for over a year. This work has been supported by the Engineering and Physical
`Sciences Research Council, UK.
`
`September 2003
`
`BlackBerry Exhibit 1015, pg. 5
`
`

`
`6 R
`
`eferences
`
`[1] A. F. Blackwell. First steps in programming: A rationale for attention in-
`vestment models. In Proceedings of the IEEE Symposia on Human Centric
`Computing, 2002.
`
`[2] A. F. Blackwell, T. R. G. Green and R. L. Hewson. Product design to
`support user abstractions. In E. Hollnagel, editor, Cognitive Task Design.
`Lawrence Erlbaum Associates, 2002.
`
`[3] C. A. R. Hoare. Record handling. In F. Genuys, editor, Programming Lan-
`guages, pages 291(cid:150)347. Academic Press, 1968.
`
`[4] J. Johnson, T. L. Roberts, W. Verplank, D. C. Smith, C. Irby, M. Beard, and
`K. Mackey. The Xerox Star: A Retrospective. IEEE Computer, 22(9):11(cid:150)29,
`1989.
`
`[5] A. Kay. The early history of Smalltalk. ACM SIGPLAN Notices, 28(3):69(cid:150)95,
`1993. Also in T.J. Bergin and R.G. Gibson, editors, History of Programming
`Languages II, pages 511(cid:150)578. Addison-Wesley, 1996.
`
`[6] M. McCullough. Abstracting craft: The practiced digital hand. MIT Press,
`1996.
`
`[7] K. Nygaard and O.-J. Dahl. The development of the Simula languages.
`ACM SIGPLAN Notices, 13(8):245(cid:150)272, 1978. Also in R. Wexelblat, editor,
`History of Programming Languages, pages 439(cid:150)493. Academic Press, 1981.
`
`[8] B. Shneiderman. Direct manipulation: A step beyond programming lan-
`guages. IEEE Computer, 16(8):57(cid:150)69, 1983.
`
`[9] D. C. Smith. Pygmalion: A Computer Program to Model and Stimulate Creative
`Thought. Birkh¤auser, Basel, 1977.
`
`[10] I. E. Sutherland. SketchPad: A man-machine graphical communication
`system. AFIPS Conference Proceedings 23, 1963, 323(cid:150)328.
`
`BlackBerry Exhibit 1015, pg. 6
`
`

`
`SKETCHPAD, A MAN-MACHINE
`GRAPHICAL COMMUNICATION
`SYSTEM
`
`by
`
`IVAN EDWARD SUTHERLAND
`
`B.S., Carnegie Institute of Technology
`(1959)
`
`M.S., California Institute of Technology
`(1960)
`
`SUBMITTED IN PARTIAL FULFILLMENT OF THE
`REQUIREMENTS FOR THE DEGREE OF
`DOCTOR OF PHILOSOPHY
`at the
`MASSACHUSETTS INSTITUTE OF TECHNOLOGY
`January, 1963
`
`BlackBerry Exhibit 1015, pg. 7
`
`

`
`8
`
`B|ackBerry Exhibit 1015, pg. 8
`
`BlackBerry Exhibit 1015, pg. 8
`
`

`
`Abstract
`
`The Sketchpad system uses drawing as a novel communication medium for
`a computer. The system contains input, output, and computation programs
`which enable it to interpret information drawn directly on a computer display.
`It has been used to draw electrical, mechanical, scienti(cid:2)c, mathematical, and
`animated drawings; it is a general purpose system. Sketchpad has shown the
`most usefulness as an aid to the understanding of processes, such as the no-
`tion of linkages, which can be described with pictures. Sketchpad also makes
`it easy to draw highly repetitive or highly accurate drawings and to change
`drawings previously drawn with it. The many drawings in this thesis were all
`made with Sketchpad.
`A Sketchpad user sketches directly on a computer display with a (cid:147)light
`pen.(cid:148) The light pen is used both to position parts of the drawing on the dis-
`play and to point to them to change them. A set of push buttons controls the
`changes to be made such as (cid:148)erase,(cid:148) or (cid:147)move.(cid:148) Except for legends, no written
`language is used.
`Information sketched can include straight line segments and circle arcs.
`Arbitrary symbols may be de(cid:2)ned from any collection of line segments, circle
`arcs, and previously de(cid:2)ned symbols. A user may de(cid:2)ne and use as many
`symbols as he wishes. Any change in the de(cid:2)nition of a symbol is at once seen
`wherever that symbol appears.
`Sketchpad stores explicit information about the topology of a drawing. If
`the user moves one vertex of a polygon, both adjacent sides will be moved. If
`the user moves a symbol, all lines attached to that symbol will automatically
`move to stay attached to it. The topological connections of the drawing are
`automatically indicated by the user as he sketches. Since Sketchpad is able
`to accept topological information from a human being in a picture language
`perfectly natural to the human, it can be used as an input program for compu-
`tation programs which require topological data, e.g., circuit simulators.
`Sketchpad itself is able to move parts of the drawing around to meet new
`conditions which the user may apply to them. The user indicates conditions
`with the light pen and push buttons. For example, to make two lines paral-
`lel, he successively points to the lines withthe light pen and presses a button.
`The conditions themselves are displayed on the drawing so that they may be
`erased or changed with the light pen language. Any combination of conditions
`can be de(cid:2)ned as a composite condition and applied in one step.
`It is easy to add entirely new types of conditions to Sketchpad’s vocabulary.
`Since the conditions can involve anything computable, Sketchpad can be used
`
`BlackBerry Exhibit 1015, pg. 9
`
`

`
`10
`
`for a very wide range of problems. For example, Sketchpad has been used to
`(cid:2)nd the distribution of forces in the members of truss bridges drawn with it.
`Sketchpad drawings are stored in the computer in a specially designed
`(cid:147)ring(cid:148) structure. The ring structure features rapid processing of topological
`information with no searching at all. The basic operations used in Sketchpad
`for manipulating the ring structure are described.
`Thesis Supervisor: Claude E. Shannon
`Title: Donner Professor of Science
`
`BlackBerry Exhibit 1015, pg. 10
`
`

`
`Acknowledgements
`
`I am indebted to Professors Claude E. Shannon and Marvin Minsky for their
`help and advice throughout the course of this research. Their helpful sugges-
`tions at several critical times gave Sketchpad much of its present character.
`Special thanks are due to Professor Steven A. Coons of the Mechanical En-
`gineering Department and to Douglas T. Ross of the Electronic Systems Labo-
`ratory. Even though I was outside their Computer Aided Design group, they
`provided at least as unstintingly of their time and ideas as if I had been their
`only concern.
`I owe a great debt to the MIT Lincoln Laboratory for the tremendous sup-
`port it afforded me. I wish to thank Wesley A. Clark and Jack L. Mitchell for
`making the TX-2 computer available to me and for providing help to make the
`special equipment I needed. I appreciate the helpful suggestions and interest
`that they and all the members of Group 51 provided. Special thanks are due
`Leonard M. Hantman for the additions he made to Sketchpad.
`The Research Laboratory of Electronics at MIT provided me with of(cid:2)ce
`space and congenial of(cid:2)ce mates whose discussion and interest I greatly ap-
`preciate.
`Finally, I wish to thank Lawrence G. Roberts who was a constant source of
`answers to speci(cid:2)c questions I had both about the best ways to program TX-2
`and about the mathematics of difference equations and matrix manipulations.
`
`BlackBerry Exhibit 1015, pg. 11
`
`

`
`1212
`
`BlackBerry Exhibit 1015, pg. 12
`
`BlackBerry Exhibit 1015, pg. 12
`
`

`
`CONTENTS
`
`Contents
`
`I
`
`II
`
`III
`
`IV
`
`V
`
`VI
`
`INTRODUCTION
`
`HISTORY OF SKETCHPAD
`
`RING STRUCTURE
`
`LIGHT PEN
`
`DISPLAY GENERATION
`
`RECURSIVE FUNCTIONS
`
`VII
`
`BUILDING A DRAWING, THE COPY FUNCTION
`
`VIII CONSTRAINT SATISFACTION
`
`IX
`
`EXAMPLES AND CONCLUSIONS
`
`A
`
`B
`
`C
`
`D
`
`E
`
`F
`
`G
`
`CONSTRAINT DESCRIPTIONS
`
`PUSH BUTTON CONTROLS
`
`STRUCTURE OF STORAGE BLOCKS
`
`RING OPERATION MACRO INSTRUCTIONS
`
`PROPOSAL FOR AN INCREMENTAL CURVE DRAWING DIS-
`PLAY
`
`MATHEMATICS OF LEAST MEAN SQUARE FIT
`
`A BRIEF DESCRIPTION OF TX-2
`
`13
`
`17
`
`31
`
`37
`
`53
`
`63
`
`77
`
`87
`
`93
`
`99
`
`117
`
`119
`
`121
`
`127
`
`129
`
`135
`
`137
`
`BlackBerry Exhibit 1015, pg. 13
`
`

`
`14
`14
`
`CONTENTS
`CONTENTS
`
`B|ackBerry Exhibit 1015, pg. 14
`
`BlackBerry Exhibit 1015, pg. 14
`
`

`
`LIST OF FIGURES
`
`List of Figures
`
`.
`.
`.
`.
`.
`.
`.
`.
`.
`.
`.
`.
`.
`1.1 Hexagonal Pattern .
`1.2 TX-2 Operating Area (cid:151) Sketchpad in Use .
`1.3 Plotter Used with Sketchpad .
`.
`.
`.
`.
`.
`.
`.
`.
`1.4 Line and Circle Drawing .
`.
`.
`.
`.
`.
`.
`.
`.
`.
`.
`1.5
`Illustrative Example
`.
`.
`.
`.
`.
`.
`.
`.
`.
`.
`.
`.
`.
`1.6 Four Positions of Linkage .
`.
`.
`.
`.
`.
`.
`.
`.
`.
`1.7 Hexagon and Semicircle on Same Lattice
`.
`
`.
`.
`.
`.
`3.1 N-Component Elements .
`.
`.
`.
`.
`3.2 Basic Ring Structure .
`.
`.
`.
`.
`3.3 Line Segment and End Points .
`.
`3.4 Zero and One Member Rings .
`.
`3.5 Fresh Point Block .
`.
`.
`.
`.
`.
`.
`.
`3.6 Compacting the Ring Structure .
`3.7
`Instances Generic Block .
`.
`.
`.
`.
`3.8 Generic Structure .
`.
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`.
`.
`.
`.
`.
`.
`.
`.
`4.1 Light Pen .
`.
`.
`.
`.
`.
`4.2 Construction of Light Pen .
`.
`.
`.
`.
`.
`4.3 Predictive Pen Tracking .
`.
`.
`.
`.
`.
`.
`4.4 Displays for Pen Tracking .
`.
`.
`.
`.
`4.5 Address in Display Register .
`4.6 Operation of Pseudo Pen Location .
`
`.
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`.
`.
`.
`
`15
`
`19
`20
`21
`22
`23
`26
`27
`
`39
`42
`43
`43
`45
`46
`49
`50
`
`54
`54
`56
`57
`58
`60
`
`65
`67
`74
`75
`
`.
`.
`.
`.
`.
`.
`5.1 Twinkled Display .
`.
`.
`.
`.
`.
`5.2 Coordinate Systems .
`.
`.
`.
`5.3 Display of Constraints .
`5.4 Display of Scalar and Digits .
`
`.
`.
`.
`.
`
`.
`.
`.
`.
`
`.
`.
`.
`.
`
`.
`.
`.
`.
`
`.
`.
`.
`.
`
`.
`.
`.
`.
`
`.
`.
`.
`.
`
`.
`.
`.
`.
`
`6.1 Applying Two Constraints Indirectly to Two Lines .
`
`7.1 De(cid:2)nitions to Copy .
`
`.
`
`.
`
`.
`
`.
`9.1 Zig-Zag for Delay Line .
`.
`9.2 BCD Encoder for Clock .
`.
`9.3 Three Bar Linkage .
`.
`.
`.
`9.4 Conic Drawing Linkage .
`9.5 Dimension Lines
`.
`.
`.
`.
`.
`9.6 Truss Under Load .
`.
`.
`.
`.
`
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`
`.
`
`.
`.
`.
`.
`.
`.
`
`.
`
`.
`
`81
`
`90
`
`. 100
`. 101
`. 103
`. 103
`. 105
`. 106
`
`BlackBerry Exhibit 1015, pg. 15
`
`

`
`16
`
`LIST OF FIGURES
`
`.
`.
`9.7 Cantilever and Arch Bridges .
`9.8 Winking Girl and Components .
`9.9 Girl Traced from Photograph .
`.
`9.10 Girl with Features Changed .
`.
`.
`9.11 Circuit Diagrams .
`.
`.
`.
`.
`.
`.
`.
`.
`
`E.1 DDA for Drawing Lines .
`E.2 DDA for Upright Conics .
`E.3 DDA for General Conic .
`.
`
`.
`.
`.
`
`.
`.
`.
`
`.
`.
`.
`
`.
`.
`.
`
`.
`.
`.
`.
`.
`
`.
`.
`.
`
`.
`.
`.
`.
`.
`
`.
`.
`.
`
`.
`.
`.
`.
`.
`
`.
`.
`.
`
`.
`.
`.
`.
`.
`
`.
`.
`.
`
`.
`.
`.
`.
`.
`
`.
`.
`.
`
`.
`.
`.
`.
`.
`
`.
`.
`.
`
`.
`.
`.
`.
`.
`
`.
`.
`.
`
`.
`.
`.
`.
`.
`
`.
`.
`.
`
`.
`.
`.
`.
`.
`
`.
`.
`.
`
`.
`.
`.
`.
`.
`
`.
`.
`.
`
`.
`.
`.
`.
`.
`
`.
`.
`.
`
`.
`.
`.
`.
`.
`
`.
`.
`.
`
`.
`.
`.
`.
`.
`
`.
`.
`.
`
`.
`.
`.
`.
`.
`
`.
`.
`.
`
`.
`.
`.
`.
`.
`
`.
`.
`.
`
`.
`.
`.
`.
`.
`
`.
`.
`.
`
`.
`.
`.
`.
`.
`
`.
`.
`.
`
`. 108
`. 109
`. 111
`. 112
`. 113
`
`. 130
`. 132
`. 133
`
`BlackBerry Exhibit 1015, pg. 16
`
`

`
`Chapter I
`
`INTRODUCTION
`
`k The Sketchpad system makes it possible for a man and a computer to con- 8#
`verse rapidly through the medium of line drawings. Heretofore, most inter-
`action between men and computers has been slowed down by the need to
`reduce all communication to written statements that can be typed; in the past,
`we have been writing letters to rather than conferring with our computers.
`For many types of communication, such as describing the shape of a mechan-
`ical part or the connections of an electrical circuit, typed statements can prove
`cumbersome. The Sketchpad system, by eliminating typed statements (except
`for legends) in favor of line drawings, opens up a new area of man-machine
`communication.
`The decision actually to implement a drawing system re(cid:3)ected our feeling
`that knowledge of the facilities which would prove useful could only be ob-
`tained by actually trying them. The decision actually to implement a drawing
`system did not mean, however, that brute force techniques were to be used to
`computerize ordinary drafting tools; it was implicit in the research nature of
`the work that simple new facilities should be discovered which, when imple-
`mented, should be useful in a wide range of applications, preferably includ-
`ing some unforseen ones. It has turned out that the properties of a computer
`drawing are entirely different from a paper drawing not only because of the
`accuracy, ease of drawing, and speed of erasing provided by the computer,
`but also primarily because of the ability to move drawing parts around on a
`computer drawing without the need to erase them. Had a working system not
`been developed, our thinking would have been too strongly in(cid:3)uenced by a
`lifetime of drawing on paper to discover many of the useful servicesk that the 9#
`computer can provide.
`As the work has progressed, several simple and very widely applicable
`facilities have been discovered and implemented. They provide a subpicture
`capability for including arbitrary symbols on a drawing, a constraint capability
`for relating the parts of a drawing in any computable way, and a de(cid:2)nition copy-
`ing capability for building complex relationships from combinations of simple
`atomic constraints.(cid:3) When combined with the ability to point at picture parts
`given by the demonstrative light pen language, the subpicture, constraint, and
`
`(cid:3)Terms with specialized meanings are listed in the glossary at the very end of this thesis.
`
`BlackBerry Exhibit 1015, pg. 17
`
`

`
`18
`
`INTRODUCTION
`
`de(cid:2)nition copying capabilities produce a system of extraordinary power. As
`was hoped at the outset, the system is useful in a wide range of applications,
`and unforseen uses are turning up.
`
`AN INTRODUCTORY EXAMPLE
`
`shown ink Figure 1.3. It is our intent with this example to show what the com-13# puter can do to help us draw while leaving the details of how it performs its
`
`To understand what is possible with the system at present let us consider using
`it to draw the hexagonal pattern of Figure 1.1. We will issue speci(cid:2)c commands
`with a set of push buttons, turn functions on and off with switches, indicate
`position information and point to existing drawing parts with the light pen, ro-
`tate and magnify picture parts by turning knobs, and observe the drawing on
`the display system. This equipment as provided at Lincoln Laboratory’s TX-2
`computer [1] is shown in Figure 1.2. When our drawing is complete it may
`be inked on paper, as were all the drawings in the thesis, by the plotter [12]
`
`functions for the chapters which follow.
`If we point the light pen at the display system and press a button called
`(cid:147)draw(cid:148), the computer will construct a straight line segment(cid:3) which stretches
`like a rubber band from the initial to the present location of the pen as shown
`in Figure 1.4. Additional presses of the button will produce additional lines
`until we have made six, enough for a single hexagon. To close the (cid:2)gure we
`return the light pen to near the end of the (cid:2)rst line drawn where it will (cid:147)lock
`on(cid:148) to the end exactly. A sudden (cid:3)ick of the pen terminates drawing, leaving
`the closed irregular hexagon shown in Figure 1.5A.
`To make the hexagon regular, we can inscribe it in a circle. To draw the
`circle we place the light pen where the center is to be and press the button
`(cid:147)circle center(cid:148), leaving behind a center point. Now, choosing a point on the
`circle (which (cid:2)xes the radius,) we press the button (cid:147)draw(cid:148) again, this time
`getting a circle arc(cid:3) whose length only is controlled by light pen position as
`shown in Figure 1.4.
`Next we move the hexagon into the circle by pointing to a corner of the
`
`circle and giving the termination (cid:3)ick we indicate that the corner is to lie on
`the circle. Each corner is in this way moved onto the circle at roughly equal
`spacing around it as shown in Figure 1.5D.
`We have indicated that the vertices of the hexagon are to lie on the circle,
`and they will remain on the circle throughout our further manipulations. If we
`also insist that the sides of the hexagon be of equal length, a regular hexagon
`will be constructed. This we can do by pointing to one side and pressing the
`(cid:147)copy(cid:148) button, and then to another side and giving the termination (cid:3)ick. The
`button in this case copies a de(cid:2)nition of equal length lines and applies it to the
`
`hexagon and pressing the button (cid:147)move(cid:148) so that the corner followsk the light16# pen, stretching two rubber band line segments behind it. By pointing to the
`
`(cid:3)The terms (cid:147)circle(cid:148) and (cid:147)line(cid:148) may be used in place of (cid:147)circle arc(cid:148) and (cid:147)line segment(cid:148) re-
`spectively since a full circle in Sketchpad is a circle arc of 360 or more degrees and no in(cid:2)nite
`line can be drawn.
`
`BlackBerry Exhibit 1015, pg. 18
`
`

`
`AN INTRODUCTORY EXAMPLE
`
`19
`
`Figure 1.1: (Originally on page 10.)
`
`BlackBerry Exhibit 1015, pg. 19
`
`

`
`20
`
`INTRODUCTION
`
`Figure 1.2: TX-2 OPERATING AREA (cid:151) SKETCHPAD IN USE. On the display
`can be seen part of a bridge similar to that of Figure 9.6. The Author is holding
`the Light pen. The push buttons used to control speci(cid:2)c drawing functions are
`on the box in front of the Author. Part of the bank of toggle switches can be
`seen behind the Author. The size and position of the part of the total picture
`seen on the display is obtained through the four black knobs just above the
`table. (Originally on page 11.)
`
`BlackBerry Exhibit 1015, pg. 20
`
`

`
`AN INTRODUCTORY EXAMPLE
`
`21
`
`Figure 1.3: PLOTTER USED WITH SKETCHPAD. A digital and analog control
`system makes the plotter draw straight lines and circles either under direct
`control of the TX-2 or off-line from punched paper tape. (Originally on page
`12.)
`
`BlackBerry Exhibit 1015, pg. 21
`
`

`
`22
`
`INTRODUCTION
`
`Figure 1.4: (Originally on page 14.)
`
`BlackBerry Exhibit 1015, pg. 22
`
`

`
`AN INTRODUCTORY EXAMPLE
`
`23
`
`Figure 1.5: (Originally on page 15.)
`
`BlackBerry Exhibit 1015, pg. 23
`
`

`
`24
`
`INTRODUCTION
`
`lines indicated. We have said, in effect, make this line equal in length to that
`line. We indicate that all six lines are equal in length by (cid:2)ve such statements.
`The computer satis(cid:2)es all existing conditions (if it is possible) whenever we
`turn on a toggle switch. This done, we have a complete regular hexagon in-
`scribed in a circle. We can erase the entire circle by pointing to any part of
`it and pressing the (cid:147)delete(cid:148) button. The completed hexagon is shown in Fig-
`ure 1.5F.
`To make the hexagonal pattern of Figure 1.1 we wish to attach a large num-
`ber of hexagons together by their corners, and so we designate the six corners
`of our hexagon as attachment points by pointing to each and pressing a but-
`ton. We now (cid:2)le away the basic hexagon and begin work on a fresh (cid:147)sheet of
`paper(cid:148) by changing a switch setting. On the new sheet we assemble, by press-
`ing a button to create each hexagon as a subpicture, six hexagons around a
`central seventh in approximate position as shown in Figure 1.5G. Subpictures
`may be positioned, each in its entirety, with the light pen, rotated or scaled
`
`ing a button, and then pointing to the corner of another hexagon we can fasten
`those corners together, because these corners have been designated as attach-
`ment points. If we attach two corners of each outer hexagon to the appropriate
`corners of the inner hexagon, the seven are uniquely related, and the computer
`will reposition them as shown in Figure 1.5H. An entire group of hexagons,
`once assembled, can be treated as a symbol. The entire group can be called up
`on another (cid:147)sheet of paper(cid:148) as a subpicture and assembled with other groups
`or with single hexagons to make a very large pattern. Using Figure 1.5H seven
`times we get the pattern of Figure 1.1. Constructing the pattern of Figure 1.1
`takes less than (cid:2)ve minutes with the Sketchpad system.
`
`with the knobsk and (cid:2)xed in position by the pen (cid:3)ick termination signal; but17# their internal shape is (cid:2)xed. By pointing to the corner of one hexagon, press-
`
`INTERPRETATION OF INTRODUCTORY EXAMPLE
`
`In the introductory example above we have seen how to draw lines and circles
`and how to move existing parts of the drawing around. We used the light pen
`both to position parts of the drawing and to point to existing parts. For exam-
`ple, we pointed to the circle to erase it, and while drawing the sixth line, we
`pointed to the end of the (cid:2)rst line drawn to close the hexagon. We also saw in
`action the very general subpicture, constraint, and de(cid:2)nition copying capabilities
`of the system.
`
`Subpicture: The original hexagon might just as well have been anything else:
`a picture of a transistor, a roller bearing, an airplane wing, a letter, or
`an entire (cid:2)gure for this report. Any number of different symbols may
`be drawn, in terms of other simpler symbols if desired, and any symbol
`may be used as often as desired.k
`Constraint: When we asked that the vertices of the hexagon lie on the circle
`we were making use of a basic relationship between picture parts that
`is built into the system. Basic relationships (atomic constraints) to make
`
`18#
`
`BlackBerry Exhibit 1015, pg. 24
`
`

`
`IMPLICATIONS OF INTRODUCTORY EXAMPLE
`
`25
`
`lines vertical, horizontal, parallel, or perpendicular; to make points lie
`on lines or circles; to make symbols appear upright, vertically above one
`another or be of equal size; and to relate symbols to other drawing parts
`such as points and lines have been included in the system. It is so easy
`to program new constraint types that the set of atomic constraints was
`expanded from (cid:2)ve to the seventeen listed in Appendix A in a period of
`about two days; specialized constraint types may be added as needed.
`
`De(cid:2)nition Copying: In the introductory example above we asked that the
`sides of the hexagon be equal in length by pressing a button while point-
`ing to the side in question. Here we were using the de(cid:2)nition copying
`capability of the system. Had we de(cid:2)ned a composite operation such
`as to make two lines both parallel and equa