The history of computer animation began as early as the 1940s and 1950s, when people began to experiment with computer graphics - most notably by John Whitney. It was only by the early 1960s when digital computers had become widely established, that new avenues for innovative computer graphics blossomed. Initially, uses were mainly for scientific, engineering and other research purposes, but artistic experimentation began to make its appearance by the mid-1960s. By the mid-1970s, many such efforts were beginning to enter into public media. Much computer graphics at this time involved 2-dimensional imagery, though increasingly, as computer power improved, efforts to achieve 3-dimensional realism became the emphasis. By the late 1980s, photo-realistic 3D was beginning to appear in film movies, and by mid-1990s had developed to the point where 3D animation could be used for entire feature film production.
The earliest pioneers: 1940s to mid-1960s
John Whitney
John Whitney, Sr was an American animator, composer and inventor, widely considered to be one of the fathers of computer animation.[1] In the 1940s and 1950s, he and his brother James created a series of experimental films made with a custom-built device based on old anti-aircraft analog computers (Kerrison Predictors) connected by servos to control the motion of lights and lit objects — the first example of motion control photography. One of Whitney's best known works from this early period was the animated title sequence from Alfred Hitchcock's 1958 film Vertigo,[2] which he collaborated on with graphic designer Saul Bass. In 1960, Whitney established his company Motion Graphics Inc, which largely focused on producing titles for film and television, while continuing further experimental works. In 1968, his pioneering motion control model photography was used on Stanley Kubrick's movie 2001: A Space Odyssey, and also for the slit-scan photography technique used in the film's "Star Gate" finale. All of John Whitney's sons (Michael, Mark and John Jr.) are also film-makers. John Whitney died in 1995.
The first digital image
One of the first programmable digital computers was SEAC (the Standards Eastern Automatic Computer), which entered service in 1950 at the National Bureau of Standards (NBS) in Maryland, USA.[3][4] In 1957, computer pioneer Russell Kirsch and his team unveiled a drum scanner for SEAC, to "trace variations of intensity over the surfaces of photographs", and so doing made the first digital image by scanning a photograph. The image, picturing Kirsch's three-month-old son, consisted of just 176×176 pixels. They used the computer to extract line drawings, count objects, recognize types of characters and display digital images on an oscilloscope screen. This breakthrough can be seen as the forerunner of all subsequent computer imaging, and recognising the importance of this first digital photograph, Life magazine in 2003 credited this image as one of the "100 Photographs That Changed the World".[5][6]
From the late 1950s and early '60s, mainframe digital computers were becoming commonplace within large organisations and universities, and increasingly these would be equipped with graphic plotting and graphics screen devices. Consequently, a new field of experimentation began to open up.
Boeing-Wichita
In 1960, William Fetter was a graphic designer for Boeing at Wichita, and was credited with coining the phrase "Computer Graphics" to describe what he was doing at Boeing at the time (though Fetter himself credited this to colleague Verne Hudson).[7]
[8] Fetter's work included the development of ergonomic descriptions of the human body that are both accurate and adaptable to different environments, and this resulted in the first 3D animated "wire-frame" figures. Such human figures became one of the most iconic images of the early history of computer graphics, and often were referred to as the "Boeing Man". Fetter died in 2002.
Bell Labs
Bell Labs in Murray Hill, New Jersey, was a leading research contributor in computer graphics, computer animation and electronic music from its beginnings in the early 1960s. Initially, researchers were interested in what the computer could be made to do, but the results of the visual work produced by the computer during this period established people like Edward Zajac, Michael Noll and Ken Knowlton as pioneering computer artists.
Edward Zajac produced one of the first computer generated films at Bell Labs in 1963, titled A Two Gyro Gravity Gradient Attitude Control System, which demonstrated that a satellite could be stabilized to always have a side facing the Earth as it orbited.[9]
Ken Knowlton developed the Beflix (Bell Flicks) animation system in 1963, which was used to produce dozens of artistic films by artists Stan VanDerBeek, Knowlton and Lillian Schwartz.[10] Instead of raw programming, Beflix worked using simple "graphic primitives", like draw a line, copy a region, fill an area, zoom an area, and the like.
In 1965, Michael Noll created computer-generated stereographic 3D movies, including a ballet of stick figures moving on a stage.[11] Some movies also showed four-dimensional hyper-objects projected to three dimensions.[12] Around 1967, Noll used the 4D animation technique to produce computer animated title sequences for the commercial film short Incredible Machine (produced by Bell Labs) and the TV special The Unexplained (produced by Walt DeFaria).[13]  Many projects in other fields were also undertaken at this time.
Ivan Sutherland
Ivan Sutherland is considered by many to be the creator of Interactive Computer Graphics, and an internet pioneer. He worked at the Lincoln Laboratory at MIT (Massachusetts Institute of Technology) in 1962, where he developed a program called Sketchpad I, which allowed the user to interact directly with the image on the screen. This was the first Graphical User Interface, and is considered one of the most influential computer programs ever written by an individual.[14]
Mid-1960s to mid-1970s
The University of Utah
Utah was a major center for computer animation in this period. The computer science faculty was founded by David Evans in 1965, and many of the basic techniques of 3D computer graphics were developed here in the early 70s with ARPA funding (Advanced Research Projects Agency). Research results included Gouraud, Phong, and Blinn shading, texture mapping, hidden surface algorithms, curved surface subdivision, real-time line-drawing and raster image display hardware, and early virtual reality work.[15] In the words of Robert Rivlin in his 1986 book The Algorithmic Image: Graphic Visions of the Computer Age, "almost every influential person in the modern computer-graphics community either passed through the University of Utah or came into contact with it in some way".[16]
Evans & Sutherland
In 1968, Ivan Sutherland teamed up with David Evans to found the company Evans & Sutherland—both were professors in the Computer Science Department at the University of Utah, and the company was formed to produce new hardware designed to run the systems being developed in the University. Many such algorithms have later resulted in the generation of significant hardware implementation, including the Geometry Engine, the Head-mounted display, the Frame buffer, and Flight simulators.[17] Most of the employees were active or former students, and included Jim Clark, who started Silicon Graphics in 1981, Ed Catmull, co-founder of Pixar in 1979, and John Warnock of Adobe Systems in 1982.
First computer animated character, Nikolai Konstantinov
In 1968 a group of Soviet physicists and mathematicians with N.Konstantinov as its head created a mathematical model for the motion of a cat. On a BESM-4 computer they devised a programme for solving the ordinary differential equations for this model. The Computer printed hundreds of frames on paper using alphabet symbols that were latter filmed in sequence thus creating the first computer animation of a character, a walking cat.[18][19]
Ohio State
Charles Csuri, an artist at The Ohio State University (OSU), started experimenting with the application of computer graphics to art in 1963. His efforts resulted in a prominent CG research laboratory that received funding from the National Science Foundation and other government and private agencies. The work at OSU revolved around animation languages, complex modeling environments, user-centric interfaces, human and creature motion descriptions, and other areas of interest to the discipline.[20][21][22]
Cybernetic Serendipity
In July 1968, the arts journal Studio International published a special issue titled Cybernetic Serendipity - the computer and the arts, which catalogued a comprehensive collection of items and examples of work being done in the field of computer art in organisations all over the world, and shown in exhibitions in London, UK, San Francisco, CA. and Washington, DC.[23][24] This marked a milestone in the development of the medium, and was considered by many to be of widespread influence and inspiration. Apart from all the examples mentioned above, two other particularly well known iconic images from this include Chaos to Order[25] by Charles Csuri (often referred to as the Hummingbird), created at Ohio State University in 1967,[26] and Running Cola is Africa[27] by Masao Komura and Koji Fujino created at the Computer Technique Group, Japan, also in 1967.[28]
Scanimate
The first machine to achieve widespread public attention in the media was Scanimate, an analog computer animation system designed and built by Lee Harrison of the Computer Image Corporation in Denver. From around 1969 onward, Scanimate systems were used to produce much of the video-based animation seen on television in commercials, show titles, and other graphics. It could create animations in real time, a great advantage over digital systems at the time.[29]
National Film Board of Canada
The National Film Board of Canada, already a world center for animation art, also began experimentation with computer techniques in 1969.[30] Most well-known of the early pioneers with this was artist Peter Foldes, who completed Metadata in 1971. This film comprised drawings animated by gradually changing from one image to the next, a technique known as "interpolating" (also known as "inbetweening" or "morphing"), which also featured in a number of earlier art examples during the 1960s.[31] In 1974, Foldes completed Hunger / La Faim, which was one of the first films to show solid filled (raster scanned) rendering, and was awarded the Jury Prize in the short film category at 1974 Cannes Film Festival, as well as an Academy Award nomination.
Atlas Computer Laboratory & Antics
The Atlas Computer Laboratory near Oxford was for many years a major facility for computer animation in Britain.[32] The first entertainment cartoon made was The Flexipede, by Tony Pritchett, which was first shown publicly at the Cybernetic Serendipity exhibition in 1968.[33] Artist Colin Emmett and animator Alan Kitching first developed solid filled colour rendering in 1972, notably for the title animation for the BBC's The Burke Special TV program.
In 1973, Kitching went on to develop a software called Antics, which allowed users to create animation without needing any programming.[34][35] The package was broadly based on conventional "cel" (celluloid) techniques, but with a wide range of tools including camera and graphics effects, interpolation ("inbetweening"/"morphing"), use of skeleton figures and grid overlays. Any number of drawings or cels could be animated at once by "choreographing" them in limitless ways using various types of "movements". At the time, only black & white plotter output was available, but Antics was able to produce full-color output by using the Technicolor Three-strip Process. Hence the name Antics was coined as an acronym for ANimated Technicolor-Image Computer System.[36] Antics was used for many animation works, including the first complete documentary movie Finite Elements, made for the Atlas Lab itself in 1975.[37]
From around the early 70s, much of the emphasis in computer animation development was towards ever increasing realism in 3D imagery, and on effects designed for use in feature movies.
First digital animation in a feature film
The first feature film to use digital image processing was the 1973 movie Westworld, a science-fiction film written and directed by novelist Michael Crichton, in which humanoid robots live amongst the humans.[38] John Whitney, Jr, and Gary Demos at Information International, Inc. digitally processed motion picture photography to appear pixelized in order to portray the Gunslinger android's point of view. The cinegraphic block portraiture was accomplished using the Technicolor Three-strip Process to color-separate each frame of the source images, then scanning them to convert into rectangular blocks according to its tone values, and finally outputting the result back to film. The process was covered in the American Cinematographer article "Behind the scenes of Westworld".[39]
SIGGRAPH
Sam Matsa whose background in graphics started with the APT project at MIT with Doug Ross and Andy Van Dam  petitioned Association for Computing Machinery (ACM) to form SICGRAPH (Special Interest Committee on Computer Graphics), the forerunner of ACM SIGGRAPH in 1967.[40] In 1974, the first SIGGRAPH conference on computer graphics opened. This annual conference soon became the dominant venue for presenting innovations in the field.[41][42]
Towards 3D: mid-1970s into the 1980s
Early 3D animation in the cinema
The first use of 3D wireframe imagery in mainstream cinema was in the sequel to Westworld, Futureworld (1976), directed by Richard T. Heffron. This featured a computer-generated hand and face created by then University of Utah graduate students Edwin Catmull and Fred Parke which had initially appeared in their 1972 experimental short A Computer Animated Hand.[43] The Oscar-winning 1975 short animated film Great, about the life of the Victorian engineer Isambard Kingdom Brunel, contains a brief sequence of a rotating wireframe model of Brunel's final project, the iron steam ship SS Great Eastern.The third movie to use this technology was Star Wars (1977), written and directed by George Lucas, with wireframe imagery in the scenes with the Death Star plans, the targeting computers in the X-wing fighters, and the Millennium Falcon spacecraft.
The Walt Disney film The Black Hole (1979, directed by Gary Nelson) used wireframe rendering to depict the titular black hole, using equipment from Disney's engineers. In the same year, the science-fiction horror film Alien, directed by Ridley Scott, also used wireframe model graphics, in this case to render the navigation monitors in the spaceship. The footage was produced by Colin Emmett at the Atlas Computer Laboratory.[44]
Nelson Max
Although Lawrence Livermore Labs in California is mainly known as a centre for high-level research in science, it continued producing significant advances in computer animation throughout this period. Notably, Nelson Max, who joined the Lab in 1971, and whose 1977 film Turning a sphere inside out is regarded as one of the classic early films in the medium (International Film Bureau, Chicago, 1977).[45] He also produced a series of "realistic-looking" molecular model animations that served to demonstrate the future role of CGI in scientific visualization ("CGI" = Computer-generated imagery). His research interests focused on realism in nature images, molecular graphics, computer animation, and 3D scientific visualization. He later served as computer graphics director for the Fujitsu pavilions at Expo 85 and 90 in Japan.[46][47]
NYIT
In 1974, Alex Schure, a wealthy New York entrepreneur, established the Computer Graphics Laboratory (CGL) at the New York Institute of Technology (NYIT). He put together the most sophisticated studio of the time, with state of the art computers, film and graphic equipment, and hired top technology experts and artists to run it -- Ed Catmull, Malcolm Blanchard, Fred Parke and others all from Utah, plus others from around the country including Ralph Guggenheim, Alvy Ray Smith and Ed Emshwiller. During the late 70s, the staff made numerous innovative contributions to image rendering techniques, and produced many influential software, including the animation program Tween, the paint program Paint, and the animation program SoftCel. Several videos from NYIT become quite famous: Sunstone, by Ed Emshwiller, Inside a Quark, by Ned Greene, and The Works. The latter, written by Lance Williams, was begun in 1978, and was intended to be the first full-length CGI film, but it was never completed, though a trailer for it was shown at SIGGRAPH 1982. In these years, many people regarded NYIT CG Lab as the top computer animation research and development group in the world.[48][49]
The quality of NYIT's work attracted the attention of George Lucas, who was interested in developing a CGI special effects facility at his company Lucasfilm. In 1979, he recruited the top talent from NYIT, including Catmull, Smith and Guggenheim to start his division, which later spun off as Pixar, founded in 1986 with funding by Apple Inc. co-founder Steve Jobs.
Framebuffer
The framebuffer or framestore is a graphics screen configured with a memory buffer that contains data for a complete screen image. Typically, it is a rectangular array (raster) of pixels, and the number of pixels in the width and the height is its "resolution". Color values stored in the pixels can be from 1-bit (monochrome), to 24-bit (true color, 8-bits each for RGB—Red, Green, & Blue), or also 32-bit, with an extra 8-bits used as a transparency mask (alpha channel). Before the framebuffer, graphics displays were all vector-based, tracing straight lines from one co-ordinate to another. The first known example of a framebuffer was built in 1969 at Bell Labs, where Joan Miller implemented a simple paint program to allow users to "paint" direct on the framebuffer. This device had just 3-bits (giving just 8 colors).[50][51]
In 1972–73, Richard Shoup developed the SuperPaint system at Xerox PARC, which used a framebuffer displaying 640×480 pixels (standard NTSC video resolution) with eight-bit depth (256 colors). The SuperPaint software contained all the essential elements of later paint packages, allowing the user to paint and modify pixels, using a palette of tools and effects, and thereby making it the first complete computer hardware and software solution for painting and editing images. Shoup also experimented with modifying the output signal using color tables, to allow the system to produce a wider variety of colors than the limited 8-bit range it contained. This scheme would later become commonplace in computer framebuffers. The SuperPaint framebuffer could also be used to capture input images from video.[52][53]
The first commercial framebuffer was produced in 1974 by Evans & Sutherland. It cost about $15,000, with a resolution of 512 by 512 pixels in 8-bit grayscale color, and sold well to graphics researchers without the resources to build their own framebuffer.[54] A little later, NYIT created the first full-color 24-bit RGB framebuffer by using three of the Evans & Sutherland framebuffers linked together as one device by a minicomputer. Many of the "firsts" that happened at NYIT were based on the development of this first raster graphics system.[48]
In 1975, the UK company Quantel, founded in 1973 by Peter Michael,[55] produced the first commercial full-color broadcast framebuffer, the Quantel DFS 3000. It was first used in TV coverage of the 1976 Montreal Olympics to generate a picture-in-picture inset of the Olympic flaming torch while the rest of the picture featured the runner entering the stadium. Framebuffer technology provided the cornerstone for the future development of digital television products.[56]
By the late 70s, it became possible for personal computers (such as the Apple II) to contain low-color framebuffers. However, it was not until the 1980s that a real revolution in the field was seen, and framebuffers capable of holding a standard video image were incorporated into standalone workstations. By the 90s, framebuffers eventually became the standard for all personal computers.
Fractals
At this time, a major step forward to the goal of increased realism in 3D animation came with the development of "fractals". The term was coined in 1975 by mathematician Benoit Mandelbrot, who used it to extend the theoretical concept of fractional dimensions to geometric patterns in nature, and published in English translation of his book Fractals: Form, Chance and Dimension in 1977.[57][58]
In 1979–80, the first film using fractals to generate the graphics was made by Loren Carpenter of Boeing. Titled Vol Libre, it showed a flight over a fractal landscape, and was presented at SIGGRAPH 1980.[59] Carpenter was subsequently hired by Pixar to create the fractal planet in the Genesis Effect sequence of Star Trek II: The Wrath of Khan in June 1982.[60]
JPL and Jim Blinn
Bob Holzman of NASA's Jet Propulsion Laboratory in California established JPL's Computer Graphics Lab in 1977 as a group with technology expertise in visualizing data being returned from NASA missions. On the advice of Ivan Sutherland, Holzman hired a graduate student from Utah named Jim Blinn.[61][62] Blinn had worked with imaging techniques at Utah, and developed them into a system for NASA's visualization tasks. He produced a series of widely seen "fly-by" simulations, including the Voyager, Pioneer and Galileo spacecraft fly-bys of Jupiter, Saturn and their moons. He also worked with Carl Sagan, creating animations for his Cosmos: A Personal Voyage TV series. Blinn developed many influential new modelling techniques, and wrote papers on them for the IEEE (Institute of Electrical and Electronics Engineers), in their journal Computer Graphics and Applications. Some of these included environment mapping, improved highlight modelling, "blobby" modelling, simulation of wrinkled surfaces, and simulation of butts and dusty surfaces.
Later in the 80s, Blinn developed CG animations for an Annenberg/CPB TV series, The Mechanical Universe, which consisted of over 500 scenes for 52 half-hour programs describing physics and mathematics concepts for college students. This he followed with production of another series devoted to mathematical concepts, called Project Mathematics!.[63]
Motion control photography
Motion control photography is a technique that uses a computer to record (or specify) the exact motion of a film camera during a shot, so that the motion can be precisely duplicated again, or alternatively on another computer, and combined with the movement of other sources, such as CGI elements. Early forms of motion control go back to John Whitney's 1968 work on 2001: A Space Odyssey, and the effects on the 1977 movie Star Wars Episode IV: A New Hope, by George Lucas' newly created company Industrial Light & Magic in California (ILM). ILM created a digitally controlled camera known as the Dykstraflex, which performed complex and repeatable motions around stationary spaceship models, enabling separately filmed elements (spaceships, backgrounds, etc.) to be coordinated more accurately with one another. However, neither of these was actually computer-based—Dykstraflex was essentially a custom-built hard-wired collection of knobs and switches.[64] The first commercial computer-based motion control and CGI system was developed in 1981 in the UK by Moving Picture Company designer Bill Mather.[65]
3D computer graphics software
3D computer graphics software began appearing for home computers in the late 1970s. The earliest known example is 3D Art Graphics, a set of 3D computer graphics effects, written by Kazumasa Mitazawa and released in June 1978 for the Apple II.[66][67]
The 1980s
The '80s saw a great expansion of radical new developments in commercial hardware, especially the incorporation of framebuffer technologies into graphic workstations, allied with continuing advances in computer power and affordability.
Silicon Graphics, Inc (SGI)
Silicon Graphics, Inc (SGI) was a manufacturer of high-performance computer hardware and software, founded in 1981 by Jim Clark. His idea, called the Geometry Engine, was to create a series of components in a VLSI processor that would accomplish the main operations required in image synthesis—the matrix transforms, clipping, and the scaling operations that provided the transformation to view space. Clark attempted to shop his design around to computer companies, and finding no takers, he and colleagues at Stanford University, California, started their own company, Silicon Graphics.[68]
SGI's first product (1984) was the IRIS (Integrated Raster Imaging System). It used the 8MHz M68000 processor with up to 2 MB memory, a custom 1024×1024 frame buffer, and the Geometry Engine to give the workstation its impressive image generation power. Its initial market was 3D graphics display terminals, but SGI's products, strategies and market positions evolved significantly over time, and for many years were a favoured choice for CGI companies in film, TV, and other fields.[69]
Quantel
In 1981, Quantel released the "Paintbox", the first broadcast-quality turnkey system designed for creation and composition of television video and graphics. Its design emphasized the studio workflow efficiency required for live news production. Essentially, it was a framebuffer packaged with innovative user software, and it rapidly found applications in news, weather, station promos, commercials, and the like. Although it was essentially a design tool for still images, it was also sometimes used for frame-by-frame animations. Following its initial launch, it revolutionised the production of television graphics, and some Paintboxes are still in use today due to their image quality, and versatility.[70]
This was followed in 1982 by the Quantel Mirage, or DVM8000/1 "Digital Video Manipulator", a digital real-time video effects processor. This was based on Quantel's own hardware, plus a Hewlett-Packard computer for custom program effects. It was capable of warping a live video stream by texture mapping it onto an arbitrary three-dimensional shape, around which the viewer could freely rotate or zoom in real-time. It could also interpolate, or morph, between two different shapes. It was considered the first real-time 3D video effects processor, and the progenitor of subsequent DVE (Digital video effect) machines. In 1985, Quantel went on to produce "Harry", the first all-digital non-linear editing and effects compositing system.[71]
Osaka University
In 1982, Japan's Osaka University developed the LINKS-1 Computer Graphics System, a supercomputer that used up to 257 Zilog Z8001 microprocessors, used for rendering realistic 3D computer graphics. According to the Information Processing Society of Japan: "The core of 3D image rendering is calculating the luminance of each pixel making up a rendered surface from the given viewpoint, light source, and object position. The LINKS-1 system was developed to realize an image rendering methodology in which each pixel could be parallel processed independently using ray tracing. By developing a new software methodology specifically for high-speed image rendering, LINKS-1 was able to rapidly render highly realistic images." It was "used to create the world's first 3D planetarium-like video of the entire heavens that was made completely with computer graphics. The video was presented at the Fujitsu pavilion at the 1985 International Exposition in Tsukuba."[72] The LINKS-1 was the world's most powerful computer, as of 1984.[73]
3D Fictional Animated Films at the University of Montreal
In the '80s, University of Montreal was at the front run of Computer Animation with three successful short 3D animated films with 3D characters:
In 1983, Philippe Bergeron, Nadia Magnenat Thalmann, and Daniel Thalmann directed Dream Flight, considered as the first 3D generated film telling a story. The film was completely programmed using the MIRA graphical language,[74] an extension of the Pascal programming language based on abstract graphical data types.[75] The film got several awards and was shown at the SIGGRAPH '83 Film Show.
In 1985, Pierre Lachapelle, Philippe Bergeron, Pierre Robidoux and Daniel Langlois directed Tony de Peltrie, which shows the first animated human character to express emotion through facial expressions and body movements, which touched the feelings of the audience.[76][77] Tony de Peltrie premiered as the closing film of SIGGRAPH '85.
In 1987, the Engineering Institute of Canada celebrated its 100th anniversary. A major event, sponsored by Bell Canada and Northern Telecom (now Nortel), was planned for the Place des Arts in Montreal. For this event, Nadia Magnenat Thalmann and Daniel Thalmann simulated Marilyn Monroe and Humphrey Bogart meeting in a cafe in the old town section of Montreal. The short movie, called Rendez-vous in Montreal [78] was shown in numerous festivals and TV channels all over the world.
Sun Microsystems, Inc
The Sun Microsystems company was founded in 1982 by Andy Bechtolsheim with other fellow graduate students at Stanford University. Bechtolsheim originally designed the SUN computer as a personal CAD workstation for the Stanford University Network (hence the acronym "SUN"). It was designed around the Motorola 68000 processor with the Unix operating system and virtual memory, and, like SGI, had an embedded frame buffer.[79] Later developments included computer servers and workstations built on its own RISC-based processor architecture and a suite of software products such as the Solaris operating system, and the Java platform. By the '90s, Sun workstations were popular for rendering in 3D CGI filmmaking—for example, Disney-Pixar's 1995 movie Toy Story used a render farm of 117 Sun workstations.[80] Sun was a proponent of open systems in general and Unix in particular, and a major contributor to open source software.[81]
National Film Board of Canada
The NFB's French-language animation studio founded its Centre d'animatique in 1980, at a cost of $1 million CAD, with a team of six computer graphics specialists. The unit was initially tasked with creating stereoscopic CGI sequences for the NFB's 3-D IMAX film Transitions for Expo 86.  Staff at the Centre d'animatique included Daniel Langlois, who left in 1986 to form Softimage.[82][83]
First turnkey broadcast animation system
Also in 1982, the first complete turnkey system designed specifically for creating broadcast-standard animation was produced by the Japanese company Nippon Univac Kaisha ("NUK", later merged with Burroughs), and incorporated the Antics 2-D computer animation software developed by Alan Kitching from his earlier versions. The configuration was based on the VAX 11/780 computer, linked to a Bosch 1-inch VTR, via NUK's own framebuffer. This framebuffer also showed realtime instant replays of animated vector sequences ("line test"), though finished full-color recording would take many seconds per frame.[84][85][86] The full system was successfully sold to broadcasters and animation production companies across Japan. Later in the '80s, Kitching developed versions of Antics for SGI and Apple Mac platforms, and these achieved a wider global distribution.[87]
First solid 3D CGI in the movies
The first cinema feature movie to make extensive use of solid 3D CGI was Walt Disney's Tron, directed by Steven Lisberger, in 1982. The film is celebrated as a milestone in the industry, though less than twenty minutes of this animation were actually used—mainly the scenes that show digital "terrain", or include vehicles such as Light Cycles, tanks and ships. To create the CGI scenes, Disney turned to the four leading computer graphics firms of the day: Information International Inc, Robert Abel and Associates (both in California), MAGI, and Digital Effects (both in New York). Each worked on a separate aspect of the movie, without any particular collaboration.[88] Tron was a box office success, grossing $33 million on a budget of $17 million.[89]
In 1984, Tron was followed by The Last Starfighter, a Universal Pictures / Lorimar production, directed by Nick Castle, and was one of cinema's earliest films to use extensive CGI to depict its many starships, environments and battle scenes. This was a great step forward compared with other films of the day, such as Return of the Jedi, which still used conventional physical models.[90] The computer graphics for the film were designed by artist Ron Cobb, and rendered by Digital Productions on a Cray X-MP supercomputer. A total of 27 minutes of finished CGI footage was produced—considered an enormous quantity at the time. The company estimated that using computer animation required only half the time, and one half to one third the cost of traditional special effects.[91] The movie was a financial success, earning over $28 million on an estimated budget of $15 million.[92]
Inbetweening and morphing
The terms inbetweening and morphing are often used interchangeably, and signify the creating of a sequence of images where one image transforms gradually into another image smoothly by small steps. Graphically, an early example would be Charles Philipon's famous 1831 caricature of French King Louis Philippe turning into a pear (metamorphosis).[93] "Inbetweening" (AKA "tweening") is a term specifically coined for traditional animation technique, an early example being in E.G.Lutz's 1920 book Animated Cartoons.[94] In computer animation, inbetweening was used from the beginning (e.g., John Whitney in the '50s, Charles Csuri and Masao Komura in the '60s).[23] These pioneering examples were vector-based, comprising only outline drawings (as was also usual in conventional animation technique), and would often be described mathematically as "interpolation". Inbetweening with solid-filled colors appeared in the early '70s, (e.g., Alan Kitching's Antics at Atlas Lab, 1973,[36] and Peter Foldes' La Faim at NFBC, 1974[31]), but these were still entirely vector-based.
The term "morphing" did not become current until the late '80s, when it specifically applied to computer inbetweening with photographic images—for example, to make one face transform smoothly into another. The technique uses grids (or "meshes") overlaid on the images, to delineate the shape of key features (eyes, nose, mouth, etc.). Morphing then inbetweens one mesh to the next, and uses the resulting mesh to distort the image and simultaneously dissolve one to another, thereby preserving a coherent internal structure throughout. Thus, several different digital techniques come together in morphing.[95] Computer distortion of photographic images was first done by NASA, in the mid-1960s, to align Landsat and Skylab satellite images with each other. Texture mapping, which applies a photographic image to a 3D surface in another image, was first defined by Jim Blinn and Martin Newell in 1976. A 1980 paper by Ed Catmull and Alvy Ray Smith on geometric transformations, introduced a mesh-warping algorithm.[96] The earliest full demonstration of morphing was at the 1982 SIGGRAPH conference, where Tom Brigham of NYIT presented a short film sequence in which a woman transformed, or "morphed", into a lynx.
The first cinema movie to use morphing was Ron Howard's 1988 fantasy film Willow, where the main character, Willow, uses a magic wand to transform animal to animal to animal and finally, to a sorceress.
3D inbetweening
With 3D CGI, the inbetweening of photo-realistic computer models can also produce results similar to morphing, though technically, it is an entirely different process (but is nevertheless often also referred to as "morphing"). An early example is Nelson Max's 1977 film Turning a sphere inside out.[46] The first cinema feature film to use this technique was the 1986 Star Trek IV: The Voyage Home, directed by Leonard Nimoy, with visual effects by George Lucas's company Industrial Light & Magic (ILM). The movie includes a dream sequence where the crew travel back in time, and images of their faces transform into one another. To create it, ILM employed a new 3D scanning technology developed by Cyberware to digitize the cast members' heads, and used the resulting data for the computer models. Because each head model had the same number of key points, transforming one character into another was a relatively simple inbetweening.[97]
The Abyss
In 1989 James Cameron's underwater action movie The Abyss was released. This was the first cinema movie to include photo-realistic CGI integrated seamlessly into live-action scenes. A five-minute sequence featuring an animated tentacle or "pseudopod" was created by ILM, who designed a program to produce surface waves of differing sizes and kinetic properties for the pseudopod, including reflection, refraction and a morphing sequence. Although short, this successful blend of CGI and live action is widely considered a milestone in setting the direction for further future development in the field.[98]
Walt Disney & CAPS
The late 80s saw another milestone in computer animation, this time in 2D: the development of Disney's "Computer Animation Production System", known as "CAPS/ink & paint". This was a custom collection of software, scanners and networked workstations developed by The Walt Disney Company in collaboration with Pixar. Its purpose was to computerize the ink-and-paint and post-production processes of traditionally animated films, to allow more efficient and sophisticated post-production by making the practice of hand-painting cels obsolete. The animators' drawings and background paintings are scanned into the computer, and animation drawings are inked and painted by digital artists. The drawings and backgrounds are then combined, using software that allows for camera movements, multiplane effects, and other techniques—including compositing with 3D image material. The system's first feature film use was in The Little Mermaid (1989), for the "farewell rainbow" scene near the end, but the first full-scale use was for The Rescuers Down Under (1990), which therefore became the first traditionally animated film to be entirely produced on computer—or indeed, the first 100% digital feature film of any kind ever produced.[99][100]
3D animation software in the 1980s
The 1980s saw the appearance of many notable new commercial software products:
1982: Autodesk Inc was founded in California by John Walker, with a focus on design software for the PC, with their flagship CAD package AutoCAD. In 1986, Autodesk's first animation package was AutoFlix, for use with AutoCAD. Their first full 3D animation software was 3D Studio for DOS in 1990, which was developed under license by Gary Yost of The Yost Group.[101][102]
1983: Alias Research was founded in Toronto, Canada, by Stephen Bingham and others, with a focus on industrial and entertainment software for SGI workstations. Their first product was Alias-1 and shipped in 1985. In 1989, Alias was chosen to animate the pseudopod in James Cameron's The Abyss, which gave the software high-profile recognition in movie animation. In 1990 this developed into PowerAnimator, often known just as Alias.[103]
1984: Wavefront was founded by Bill Kovacs and others, in California, to produce computer graphics for movies and television, and also to develop and market their own software based on SGI hardware. Wavefront developed their first product, Preview, during the first year of business. The company's production department helped tune the software by using it on commercial projects, creating opening graphics for television programs. In 1988, the company introduced the Personal Visualiser.[104][105]
1984:  was created in France as a subsidiary of aircraft simulator company Thomson-CSF, to develop and commercialise on their own 3D system Explore, first released in 1986.
1984: , was a division of Sogitec avionics in France, founded by Xavier Nicolas for the production of computer animation films, using their own 3D software developed from 1981 by Claude Mechoulam and others at Sogitec.[106]
1986: Softimage was founded by National Film Board of Canada filmmaker Daniel Langlois in Montreal. Its first product was called the Softimage Creative Environment, and was launched at SIGGRAPH '88. For the first time, all 3D processes (modelling, animation, and rendering) were integrated. Creative Environment (eventually to be known as Softimage 3D in 1988), became a standard animation solution in the industry.[107]
1987: Side Effects Software was established by Kim Davidson and Greg Hermanovic in Toronto, Canada, as a production/software company based on a 3D animation package called PRISMS, which they had acquired from their former employer Omnibus. Side Effects Software developed this procedural modelling and motion product into a high-end, tightly integrated 2D/3D animation software which incorporated a number of technological breakthroughs.[108]
1989: the companies  and  were merged to create the new company .
CGI in the 1990s
Computer animation expands in film and TV
The 1990s began with much of CGI technology now sufficiently developed to allow a major expansion into film and TV production. 1991 is widely considered the "breakout year", with two major box-office successes, both making heavy use of CGI.
First of these was James Cameron's movie Terminator 2: Judgment Day,[109] and was the one that first brought CGI to widespread public attention. The technique was used to animate the two "Terminator" robots. The "T-1000" robot was given a "mimetic poly-alloy" (liquid metal) structure, which enabled this shapeshifting character to morph into almost anything it touched. Most of the key Terminator effects were provided by Industrial Light & Magic, and this film was the most ambitious CGI project since the 1982 film Tron.[110]
The other was Disney's Beauty and the Beast,[111] the second traditional 2D animated film to be entirely made using CAPS. The system also allowed easier combination of hand-drawn art with 3D CGI material, notably in the "waltz sequence", where Belle and Beast dance through a computer-generated ballroom as the camera "dollies" around them in simulated 3D space.[112] Notably, Beauty and the Beast was the first animated film ever to be nominated for a Best Picture Academy Award.[113]
Another significant step came in 1993, with Steven Spielberg's Jurassic Park,[114] where 3D CGI dinosaurs were integrated with life-sized animatronic counterparts. The CGI animals were created by ILM, and in a test scene to make a direct comparison of both techniques, Spielberg chose the CGI. Also watching was George Lucas who remarked "a major gap had been crossed, and things were never going to be the same."[115][116][117]
Flocking
Flocking is the behavior exhibited when a group of birds (or other animals) move together in a flock. A mathematical model of flocking behavior was first simulated on a computer in 1986 by Craig Reynolds, and soon found its use in animation. Jurassic Park notably featured flocking, and brought it to widespread attention by mentioning it in the actual script. Other early uses were the flocking bats in Tim Burton's Batman Returns (1992), and the wildebeest stampede in Disney's The Lion King (1994).[118]
With improving hardware, lower costs, and an ever-increasing range of software tools, CGI techniques were soon rapidly taken up in both film and television production.
In 1993, J. Michael Straczynski's Babylon 5 became the first major television series to use CGI as the primary method for their visual effects (rather than using hand-built models), followed later the same year by Rockne S. O'Bannon's SeaQuest DSV.
Also the same year, the French company Studio Fantome produced the first full-length completely computer animated TV series, Insektors (26×13'),[119][120] though they also produced an even earlier all 3D short series, Geometric Fables (50 x 5') in 1991.[121] A little later, in 1994, the Canadian TV CGI series ReBoot (48×23') was aired, produced by Mainframe Entertainment.[122]
In 1995, there came the first fully computer-animation feature film, Disney-Pixar's Toy Story, which was a huge commercial success.[123] This film was directed by John Lasseter, a co-founder of Pixar, and former Disney animator, who started at Pixar with short movies such as Luxo Jr. (1986), Red's Dream (1987), and Tin Toy (1988), which was also the first computer-generated animated short film to win an Academy Award. Then, after some long negotiations between Disney and Pixar, a partnership deal was agreed in 1991 with the aim of producing a full feature movie, and Toy Story was the result.[124]
The following years saw a greatly increased uptake of digital animation techniques, with many new studios going into production, and existing companies making a transition from traditional techniques to CGI. Between 1995 and 2005 in the USA, the average effects budget for a wide-release feature film leapt from $5 million to $40 million. According to Hutch Parker, President of Production at 20th Century Fox, as of 2005, "50 percent of feature films have significant effects. They're a character in the movie." However, CGI has made up for the expenditures by grossing over 20% more than their real-life counterparts, and by the early 2000s, computer-generated imagery had become the dominant form of special effects.[125]
Motion capture
Motion capture, or "Mocap", records the movement of external objects or people, and has applications for medicine, sports, robotics, and the military, as well as for animation in film, TV and games. The earliest example would be in 1878, with the pioneering photographic work of Eadweard Muybridge on human and animal locomotion, which is still a source for animators today.[126] Before computer graphics, capturing movements to use in animation would be done using Rotoscoping, where the motion of an actor was filmed, then the film used as a guide for the frame-by-frame motion of a hand-drawn animated character. The first example of this was Max Fleischer's Out of the Inkwell series in 1915, and a more recent notable example is the 1978 Ralph Bakshi 2D animated movie The Lord of the Rings.
Computer-based motion capture started as a photogrammetric analysis tool in biomechanics research in the 70s and 80s.[127] A performer wears markers near each joint to identify the motion by the positions or angles between the markers. Many different types of markers can be used—lights, reflective markers, LEDs, infra-red, inertial, mechanical, or wireless RF—and may be worn as a form of suit, or attached direct to a performer's body. Some systems include details of face and fingers to capture subtle expressions, and such is often referred to as "performance capture". The computer records the data from the markers, and uses it to animate digital character models in 2D or 3D computer animation, and in some cases this can include camera movement as well. In the 90s, these techniques became widely used for visual effects. Video games also began to use motion capture to animate in-game characters in 1995, the earliest examples of this being the Atari Jaguar CD-based game Highlander: The Last of the MacLeods,[128][129] and the arcade fighting game Soul Edge, which was the first video game to use passive optical motion-capture technology.[130]
Another breakthrough where a cinema film used motion capture was creating hundreds of digital characters for the film Titanic in 1997. The technique was used extensively in 1999 to create Jar-Jar Binks and other digital characters in Star Wars: Episode I – The Phantom Menace.
Match moving
Match moving (also known as motion tracking or camera tracking), although related to motion capture, is a completely different technique. Instead of using special cameras and sensors to record the motion of subjects, match moving works with pre-existing live-action footage, and uses computer software alone to track specific points in the scene through multiple frames, and thereby allow the insertion of CGI elements into the shot with correct position, scale, orientation, and motion relative to the existing material. The terms are used loosely to describe several different methods of extracting subject or camera motion information from a motion picture. The technique can be 2D or 3D, and can also include matching for camera movements. The earliest commercial software examples being 3D-Equalizer from Science.D.Visions[131] and rastrack from Hammerhead Productions,[132] both starting mid-90s.
The first step is identifying suitable features that the software tracking algorithm can lock onto and follow. Typically, features are chosen because they are bright or dark spots, edges or corners, or a facial feature—depending on the particular tracking algorithm being used. When a feature is tracked it becomes a series of 2D coordinates that represent the position of the feature across the series of frames. Such tracks can be used immediately for 2D motion tracking, or then be used to calculate 3D information. In 3D tracking, a process known as "calibration" derives the motion of the camera from the inverse-projection of the 2D paths, and from this a "reconstruction" process is used to recreate the photographed subject from the tracked data, and also any camera movement. This then allows an identical virtual camera to be moved in a 3D animation program, so that new animated elements can be composited back into the original live-action shot in perfectly matched perspective.[133]
In the 90s, the technology progressed to the point that it became possible to include virtual stunt doubles. Camera tracking software was refined to allow increasingly complex visual effects developments that were previously impossible. Computer-generated extras also became used extensively in crowd scenes with advanced flocking and crowd simulation software. Being mainly software-based, match moving has become increasingly affordable as computers become cheaper and more powerful. It has become an essential visual effects tool and is even used providing effects in live television broadcasts.[134]
Virtual studio
In television, a virtual studio, or virtual set, is a studio that allows the real-time combination of people or other real objects and computer generated environments and objects in a seamless manner. It requires that the 3D CGI environment is automatically locked to follow any movements of the live camera and lens precisely. The essence of such system is that it uses some form of camera tracking to create a live stream of data describing the exact camera movement, plus some realtime CGI rendering software that uses the camera tracking data and generates a synthetic image of the virtual set exactly linked to the camera motion. Both streams are then combined with a video mixer, typically using chroma key. Such virtual sets became common in TV programs in the 90s, with the first practical system of this kind being the Synthevision virtual studio developed by the Japanese broadcasting corporation NHK (Nippon Hoso Kyokai) in 1991, and first used in their science special, Nano-space.[135]
[136] Virtual studio techniques are also used in filmmaking, but this medium does not have the same requirement to operate entirely in realtime. Motion control or camera tracking can be used separately to generate the CGI elements later, and then combine with the live-action as a post-production process. However, by the 2000s, computer power had improved sufficiently to allow many virtual film sets to be generated in realtime, as in TV, so it was unnecessary to composite anything in post-production.
Machinima
Machinima uses realtime 3D computer graphics rendering engines to create a cinematic production. Most often, video games machines are used for this. The Academy of Machinima Arts & Sciences (AMAS), a non-profit organization formed 2002, and dedicated to promoting machinima, defines machinima as "animated filmmaking within a real-time virtual 3-D environment". AMAS recognizes exemplary productions through awards given at its annual[137][138] The practice of using graphics engines from video games arose from the animated software introductions of the '80s "demoscene", Disney Interactive Studios' 1992 video game Stunt Island, and '90s recordings of gameplay in first-person shooter video games, such as id Software's Doom and Quake. Machinima Film Festival. Machinima-based artists are sometimes called machinimists or machinimators.
3D animation software in the 1990s
There were many developments, mergers and deals in the 3D software industry in the '90s and later.
Wavefront followed the success of Personal Visualiser with the release of Dynamation in 1992, a powerful tool for interactively creating and modifying realistic, natural images of dynamic events. In 1993, Wavefront acquired  (TDI), with their innovative product Explore, a tool suite that included 3Design for modelling, Anim for animation, and Interactive Photorealistic Renderer (IPR) for rendering. In 1995, Wavefront was bought by Silicon Graphics, and merged with Alias.[139]
Alias Research continued the success of PowerAnimator with movies like Terminator 2: Judgment Day, Batman Returns and Jurassic Park, and in 1993 started the development of a new entertainment software, which was later to be named Maya. Alias found customers in animated film, TV series, visual effects, and video games, and included many prominent studios, such as Industrial Light & Magic, Pixar, Sony Pictures Imageworks, Walt Disney, and Warner Brothers. Other Alias products were developed for applications in architecture and engineering. In 1995, SGI purchased both Alias Research and Wavefront in a 3-way deal, and the merged company Alias Wavefront was launched.[140]
Alias Wavefront's new mission was to focus on developing the world's most advanced tools for the creation of digital content. PowerAnimator continued to be used for visual effects and movies (such as Toy Story, Casper, and Batman Forever), and also for video games. Further development of the Maya software went ahead, adding new features such as motion capture, facial animation, motion blur, and "time warp" technology. CAD industrial design products like AliasStudio and Alias Designer became standardized on Alias|Wavefront software. In 1998, Alias|Wavefront launched Maya as its new 3D flagship product, and this soon became the industry's most important animation tool. Maya was the merger of three packages—Wavefront's Advanced Visualizer, Alias's Power Animator, and TDI's Explore. In 2003 the company was renamed simply "Alias". In 2004, SGI sold the business to a private investment firm, and it was later renamed to Alias Systems Corporation. In 2006, the company was bought by Autodesk.[141][142]
Softimage developed further features for Creative Environment, including the Actor Module (1991) and Eddie (1992), including tools such as inverse kinematics, enveloping, metaclay, flock animation, and many others. Softimage customers include many prominent production companies, and Softimage has been used to create animation for hundreds of major feature films and games. In 1994, Microsoft acquired Softimage, and renamed the package Softimage 3D, releasing a Windows NT port two years later.[143][144] In 1998, after helping to port the products to Windows and financing the development of Softimage and Softimage|DS, Microsoft sold the Softimage unit to Avid Technology, who was looking to expand its visual effect capabilities. Then, in 2008, Autodesk acquired the brand and the animation assets of Softimage from Avid, thereby ending Softimage Co. as a distinct entity. The video-related assets of Softimage, including Softimage|DS (now Avid|DS) continue to be owned by Avid.[145][146]
Autodesk Inc's PC DOS-based 3D Studio was eventually superseded in 1996 when The Yost Group developed 3D Studio Max for Windows NT. Priced much lower than most competitors, 3D Studio Max was quickly seen as an affordable solution for many professionals. Of all animation software, 3D Studio Max serves the widest range of users. It is used in film and broadcast, game development, corporate and industrial design, education, medical, and web design. In 2006, Autodesk acquired Alias, bringing the StudioTools and Maya software products under the Autodesk banner, with 3D Studio Max rebranded as Autodesk 3ds Max, and Maya as Autodesk Maya. Now one of the largest software companies in the world, Autodesk serves more than 4 million customers in over 150 countries.[147][148][149]
Side Effects Software's PRISMS was used extensively to create visual effects for broadcast and feature films into the '90s, with projects like Twister, Independence Day, and Titanic. In 1996, Side Effects Software introduced Houdini, a next-generation 3D package that proved to be more sophisticated and artist-friendly than its predecessor. Houdini is used around the world to develop cutting edge 3D animation in the film, broadcast and gaming industries, and Side Effects Software has consistently proved itself to be an industry innovator.[150][151][152]
CGI in the 2000s
2000 breakthrough capture of the reflectance field over the human face
In 2000, a team led by Paul Debevec managed to adequately capture (and simulate) the reflectance field over the human face using the simplest of light stages.[153] which was the last missing piece of the puzzle to make digital look-alikes of known actors.
Motion capture, photorealism, and uncanny valley
The first mainstream cinema film fully made with motion capture was the 2001 Japanese-American Final Fantasy: The Spirits Within directed by Hironobu Sakaguchi, which was also the first to use photorealistic CGI characters.[154] The film was not a box-office success.[155] Some commentators have suggested this may be partly because the lead CGI characters had facial features which fell into the "uncanny valley".[156] In 2002, Peter Jackson's The Lord of the Rings: The Two Towers was the first feature film to use a real-time motion capture system, which allowed the actions of actor Andy Serkis to be fed direct into the 3D CGI model of Gollum as it was being performed.[157]
Motion capture is seen by many as replacing the skills of the animator, and lacking the animator's ability to create exaggerated movements that are impossible to perform live. The end credits of Pixar's film Ratatouille (2007) carry a stamp certifying it as "100% Pure Animation — No Motion Capture!" However, proponents point out that the technique usually includes a good deal of adjustment work by animators as well. Nevertheless, in 2010, the US Film Academy (AMPAS) announced that motion-capture films will no longer be considered eligible for "Best Animated Feature Film" Oscars, stating "Motion capture by itself is not an animation technique."[158][159]
Virtual cinematography
The early 2000s saw the advent of fully virtual cinematography with its audience debut considered to be in the 2003 films The Matrix Reloaded and The Matrix Revolutions with its digital look-alikes so convincing that it is often impossible to know if some image is a human imaged with a camera or a digital look-alike shot with a simulation of a camera. The scenes built and imaged within virtual cinematography are the "Burly brawl" and the end showdown between Neo and Agent Smith. With conventional cinematographic methods the burly brawl would have been prohibitively time consuming to make with years of compositing required for a scene of few minutes. Also a human actor could not have been used for the end showdown in Matrix Revolutions: Agent Smith's cheekbone gets punched in by Neo leaving the digital look-alike naturally unhurt.
3D animation software in the 2000s
Blender (software) is a free open source virtual cinematography package, used by professionals and enthusiasts alike.
Poser is another DIY 3D graphics program especially aimed at user-friendly animation of soft objects
Pointstream Software is a professional optical flow program that uses a pixel as its basic primitive form usually tracked over a multi-camera setup from the esteemed Arius3D, makers of the XYZRGB scanner, used in the production process of the Matrix sequels
CGI in the 2010s
In SIGGRAPH 2013 Activision and USC presented a real-time digital face look-alike of "Ira" utilizing the USC light stage X by Ghosh et al. for both reflectance field and motion capture.[160][161] The end result, both precomputed and real-time rendered with the state-of-the-art  Graphics processing unit: Digital Ira,[160] looks fairly realistic. Techniques previously confined to high-end virtual cinematography systems are rapidly moving into the video games and leisure applications.
Further developments
New developments in computer animation technologies are reported each year in the United States at SIGGRAPH, the largest annual conference on computer graphics and interactive techniques, and also at Eurographics, and at other conferences around the world.[162]


Traditionally, the canon of Sherlock Holmes consists of the 56 short stories and four novels written by Sir Arthur Conan Doyle.[1] In this context, the term "canon" is an attempt to distinguish between Doyle's original works and subsequent works by other authors using the same characters.
Canon

The traditional canon—usually capitalized by aficionados of the Sherlockian game as "the Canon"—consists of four novels and 56 short stories.
The description of these 60 adventures as the Sherlock Holmes canon and the game of applying the methods of "Higher Criticism" to it was started by Ronald Knox as a playful use of the traditional definition of Canon as an authoritative list of books accepted as Holy Scripture.[2]
Novels
The four novels of the canon:
A Study in Scarlet (1887)
The Sign of the Four (1890)
The Hound of the Baskervilles (1901–1902)
The Valley of Fear (1914–1915)
Short stories
The 56 short stories are collected in five books:
The Adventures of Sherlock Holmes (1892)
The Memoirs of Sherlock Holmes (1894)
The Return of Sherlock Holmes (1905)
His Last Bow (1917)
The Case-Book of Sherlock Holmes (1927)
The Adventures of Sherlock Holmes (1892)
Published 31 October 1892; contains 12 stories published in The Strand between July 1891 and June 1892 with original illustrations by Sidney Paget.[3]
"A Scandal in Bohemia" (June 1891)
"The Red-Headed League" (August 1891)
"A Case of Identity" (September 1891)
"The Boscombe Valley Mystery" (October 1891)
"The Five Orange Pips" (November 1891)
"The Man with the Twisted Lip" (December 1891)
"The Adventure of the Blue Carbuncle" (January 1892)
"The Adventure of the Speckled Band" (February 1892)
"The Adventure of the Engineer's Thumb" (March 1892)
"The Adventure of the Noble Bachelor" (April 1892)
"The Adventure of the Beryl Coronet" (May 1892)
"The Adventure of the Copper Beeches" (June 1892)
The Memoirs of Sherlock Holmes (1893–94)
Contains 12 stories published in The Strand as further episodes of the Adventures between December 1892 and December 1893 with original illustrations by Sidney Paget (after the magazine publication, Conan Doyle included "The Adventure of the Cardboard Box" only in the His Last Bow collection).
"The Adventure of Silver Blaze" (December 1892)
"The Adventure of the Cardboard Box" (January 1893)
"The Adventure of the Yellow Face" (February 1893)
"The Adventure of the Stockbroker's Clerk" (March 1893)
"The Adventure of the Gloria Scott" (April 1893)
"The Adventure of the Musgrave Ritual" (May 1893)
"The Adventure of the Reigate Squire" (June 1893)
"The Adventure of the Crooked Man" (July 1893)
"The Adventure of the Resident Patient" (August 1893)
"The Adventure of the Greek Interpreter" (September 1893)
"The Adventure of the Naval Treaty" (October–November 1893)
"The Adventure of the Final Problem" (December 1893)
The Return of Sherlock Holmes (1905)
Contains 13 stories published in The Strand between October 1903 and December 1904 with original illustrations by Sidney Paget.
"The Adventure of the Empty House" (October 1903)
"The Adventure of the Norwood Builder" (November 1903)
"The Adventure of the Dancing Men" (December 1903)
"The Adventure of the Solitary Cyclist" (January 1904)
"The Adventure of the Priory School" (February 1904)
"The Adventure of Black Peter" (March 1904)
"The Adventure of Charles Augustus Milverton" (April 1904)
"The Adventure of the Six Napoleons" (May 1904)
"The Adventure of the Three Students" (June 1904)
"The Adventure of the Golden Pince-Nez" (July 1904)
"The Adventure of the Missing Three-Quarter" (August 1904)
"The Adventure of the Abbey Grange" (September 1904)
"The Adventure of the Second Stain" (December 1904)
His Last Bow (1917)
Contains 7 stories published 1908–1917. Many editions of His Last Bow have eight stories, with "The Adventure of the Cardboard Box" being a part of this collection rather than in The Memoirs of Sherlock Holmes.
"The Adventure of Wisteria Lodge" (1908)
"The Adventure of the Red Circle" (1911)
"The Adventure of the Bruce-Partington Plans" (1908)
"The Adventure of the Dying Detective" (1913)
"The Disappearance of Lady Frances Carfax" (1911)
"The Adventure of the Devil's Foot" (1910)
"His Last Bow" (1917)
The Case-Book of Sherlock Holmes (1927)
Contains 12 stories published 1921–1927.
"The Adventure of the Mazarin Stone" (1921)
"The Problem of Thor Bridge" (1922)
"The Adventure of the Creeping Man" (1923)
"The Adventure of the Sussex Vampire" (1924)
"The Adventure of the Three Garridebs" (1924)
"The Adventure of the Illustrious Client" (1924)
"The Adventure of the Three Gables" (1926)
"The Adventure of the Blanched Soldier" (1926)
"The Adventure of the Lion's Mane" (1926)
"The Adventure of the Retired Colourman" (1926)
"The Adventure of the Veiled Lodger" (1927)
"The Adventure of Shoscombe Old Place" (1927)
Extracanonical works
Since the author's death, professional and amateur Holmesians have discussed endlessly the expansion of this canon, to include other works by Doyle, including works in other media, into the current complete adventures.[4]  Rumours have always surrounded lost works,[1] and in recent years further investigations have revealed more to the traditionally collected canon. As there exists no definitive body to argue what is, and what is not canon beyond the already established novels and stories, it is unlikely that any piece, no matter how good its claim to be "canonical" will ever be popularly received into published versions of the Complete Sherlock Holmes. However, as many as eighteen works have been cited as possible entrants. These works include plays, poems, essays on the character, and even short stories.
Published collections of extracanonical works include: Sherlock Holmes: The Published Apocrypha, edited by Jack Tracy; The Final Adventures of Sherlock Holmes, edited by Peter Haining; The Uncollected Sherlock Holmes, edited by Richard Lancelyn Green; and the final volume of Leslie S. Klinger's Sherlock Holmes Reference Library titled The Apocrypha of Sherlock Holmes. These works, each with slightly different contents, discussed several titles and their place in the canon.[5]
Works by Sir Arthur Conan Doyle
In addition to the canon, Conan Doyle wrote (occasionally with a co-writer) a number of vignettes, play adaptations and essays involving Holmes, and two short stories in which Holmes makes a possible cameo appearance. Most were published in various places during his lifetime; another has only come to light since his death. These are listed below with further detail.
Short stories
 "The Field Bazaar" (1896)

"The Field Bazaar" was written for an Edinburgh University fundraising event. Doyle had been requested by his university to contribute a short piece of literature for a charity magazine. In the story Watson has received a similar request and whilst he reads the letter at breakfast, Holmes correctly deduces the sender of the letter and Watson's thoughts with regard to the letter. It shares many similarities to the canonical stories. Aside from the metafictional twist in which Watson supplants Doyle as the author publishing his own stories in a magazine, it also plays not only about the famous skill of Holmes' observations producing apparently miraculous results, but also upon the notion of the "traditional breakfast scenes" which open many Holmes short stories.[6] The Stories of Sherlock Holmes, which ran on Springbok Radio in South Africa from 1979 to 1985, used the title & plot as a beginning to one of their own episodes in which Holmes & Watson travel to Watson's Alma Mater to solve a murder on a cricket pitch.
"The Story of the Lost Special" (1898)

Though Doyle had killed off his character by 1894, he still wrote other short stories for publication in the Strand Magazine, including "The Story of the Lost Special", a seemingly inexplicable mystery in which a special train and its few passengers disappear between two stations. After the mystery is described in full, it is stated that a letter appeared in the press, giving a proposed solution from "an amateur reasoner of some celebrity". It is possible, and has been proposed by Haining, Tracy, and Green, amongst others that this "amateur reasoner" was Sherlock Holmes. The strongest clue to this is the quotation, "once one has eliminated the impossible...", used by Holmes throughout his deductions. However, this suggested solution is proved wrong by a confession from the organising criminal once he is later arrested for an unrelated crime. Haining suggested that Doyle was "getting out some Holmes" during the series hiatus, but given the failure of the unnamed detective it appears he was parodying his most famous creation. The story was published in book form in Arthur Conan Doyle’s Tales of Terror and Mystery in 1923 and has for years appeared in French editions of the complete adventures.[7]
"The Story of the Man with the Watches" (1898)
Like "The Story of the Lost Special", "The Story of the Man with the Watches" appeared in the Strand (in 1898), and later in Round the Fire Stories and Tales of Terror and Mystery. It follows the same pattern; the mystery this time surrounds the appearance of a dead man in a railway carriage, with six pocket watches in his jacket. An explanation is offered by an amateur detective but the narrator notes it is flawed, as it doesn't take into account all the facts. A man involved in the accidental murder of the victim writes a letter to the detective, saying that it was a "mighty clever solution" but entirely incorrect and continues to share the true events of that day. It shares the same backing for categorising as a Sherlock Holmes story as "The Story of the Lost Special", and appears in French anthologies.[7] The story was adapted for BBC Radio 4 in 2009 as "The Thirteen Watches", in an episode from The Further Adventures of Sherlock Holmes. The number of watches was changed because the new title came from a reference (in the Holmes story "The Noble Bachelor") to Holmes' involvement with the watches incident.
 Plot for Sherlock Holmes Story (c. 1900)
When searching through Conan Doyle’s papers, Hesketh Pearson, a biographer of his, came across a plan for an unwritten story. As Richard Lancelyn Green notes, "there is no evidence to show that it is by [Conan Doyle] and strong internal evidence to suggest that it's not".[8] Various authors have attempted to complete the story (named "The Adventure of the Tall Man" by Peter Haining) and put it alongside the canon. Some are very close to Doyle’s plot, others include variations from it.[9]  However no 'official' completion has been made (In the same way as The Exploits of Sherlock Holmes was intended as an official continuation of the canon).
 "How Watson Learned the Trick" (1924)

In 1922, several authors were approached to contribute to the library of Queen Mary's Dolls' House. Conan Doyle wrote a short Sherlock Holmes story, just 503 words long, onto the tiny pages of a specially constructed miniature book: "How Watson Learned the Trick". The story was later published alongside works by other authors in The Book of the Queen's Dolls' House Library (1924). Though written 28 years after "The Field Bazaar", this is almost a companion piece to that story. Like "The Field Bazaar", this story is a breakfast scene, during which Watson attempts to mimic Holmes' style in guessing his thoughts. Watson's intuitions are proved wrong, however. Unlike almost all parts of the Sherlock Holmes story it is written in the third person, presumably due to its length.[10]
Sherlock Holmes on stage
Angels of Darkness (c. 1889)
Unpublished until 2000, this play was written shortly after A Study in Scarlet was published. It is essentially a rewrite of the American chapters of A Study in Scarlet, with the London action moving to San Francisco. Holmes is not present, but Watson is, in a very different form. He acts discreditably and even marries another woman. The publication of this play was at first suppressed, Doyle’s biographer, John Dickson Carr stated that it would do no good for the public to read this, a view that Haining endorses readily.[11] The play is notable for its contrasting sensationalist and comic scenes, and it is contained in Klinger's Apocrypha.
Sherlock Holmes: A Drama in Four Acts (or Sherlock Holmes) (1899)

The original Sherlock Holmes play written by Arthur Conan Doyle and William Gillette had a successful run of over 30 years. It has many original parts which are not found in the short stories but borrows many events from the canonical adventures, namely "A Scandal in Bohemia" and "The Adventure of the Final Problem". Also, it had elements from A Study in Scarlet, The Sign of Four, "The Boscombe Valley Mystery", "The Adventure of the Greek Interpreter", and "The Adventure of the Naval Treaty". It includes the very first mention of the phrase "Elementary, my dear Watson". While Conan Doyle wrote the original version, it is unclear how much of his material survived in the play as performed, which was written by Gillette. Conan Doyle and Gillette later revised the play together; it has since been revised by others twice.
The Speckled Band (or The Stonor Case) (1902)
Around 1902, Doyle wrote and produced a play based on his short story "The Adventure of the Speckled Band".  It premièred 8 years later, at the Adelphi Theatre, London on 4 June 1910, with H. A. Saintsbury as Sherlock Holmes and Lyn Harding as Dr. Grimesby Roylott. The play, originally entitled The Stonor Case, differs from the story in several small details, such as the names of some of the characters.[12]
The Crown Diamond: An Evening With Mr Sherlock Holmes (1921)
"The Crown Diamond" is an alternate version of the short story "The Adventure of the Mazarin Stone" though it predates its counterpart by some time,[13] Sometime during the original run the short story was adapted from the play, this is the reason that the narrative is told in third person rather than by the traditional narrator Watson. Some claim that the play originally appeared in an early draft of "Sherlock Holmes" (above) and was later removed, with some elements finding their way into "The Adventure of the Empty House" before the entire play was resurrected, some years later, into "The Crown Diamond" and "The Mazarin Stone."
Essays and retrospectives
Arthur Conan Doyle rarely gave interviews or publicly discussed his character. However, the following is a list of Conan Doyle essays on his character which are currently in publication, either in Green or Haining's book or in standard editions of the Complete Stories:
"To An Undiscerning Critic" (1912)
Guiterman first published his homage in America in Life (5 December 1912) and then in London Opinion (14 December 1912), and in his collection The Laughing Muse. Conan Doyle’s answer appeared in the 26 December 1912 issue of London Opinion and was reprinted in the memoir of the editor of London Opinion, Lincoln Springfield.  The late Dean Dickensheet appears to be the first to print the poems together, in An ‘Undiscerning Critic’ Discerned.
"Some Personalia about Mr. Sherlock Holmes" (1917)
This essay was featured in the Strand Magazine as a Christmas treat to its readers. It talks of the way Holmes had caught the public imagination and Conan Doyle’s view on his character.[14]
"The Truth About Sherlock Holmes" (1923)
An essay from Collier's Weekly, in which Doyle explains exactly where Holmes came from. It contains, at the end, J. M. Barrie’s "The Adventure of the Two Collaborators".[15]
"Mr. Sherlock Holmes to His Readers" (1927)
This appeared in The Strand Magazine to introduce a competition to name the best Sherlock Holmes adventures. The same essay, with two paragraphs cut, appears as the preface to  The Case-Book of Sherlock Holmes.
"How I Made My List" (1927)
This is the sequel to the article mentioned above. In it, Conan Doyle listed what he thought were the best Holmes adventures. He noted that had he been able to include stories from The Case-Book of Sherlock Holmes he would certainly have included "The Adventure of the Lion's Mane" and "The Adventure of the Illustrious Client".[16] The list is as follows:
"The Adventure of the Speckled Band"
"The Red-Headed League"
"The Adventure of the Dancing Men"
"The Adventure of the Final Problem"
"A Scandal in Bohemia"
"The Adventure of the Empty House"
"The Five Orange Pips"
"The Adventure of the Second Stain"
"The Adventure of the Devil's Foot"
"The Adventure of the Priory School"
"The Adventure of the Musgrave Ritual"
"The Adventure of the Reigate Squire"
Richard Lancelyn Green's The Uncollected Sherlock Holmes also includes five prefaces to the various editions of Sherlock Holmes stories and novels, Conan Doyle's speech at the Stoll Convention Dinner (1921), some chapters from Conan Doyle's autobiography Memoirs and Adventures, and several interviews.
Works of interest by other authors
These are works which have in the past been thought to have been written by Conan Doyle. Some have been conclusively proved to have no Conan Doyle input, the composition of others still remains unclear.
The Exploits of Sherlock Holmes

The Exploits of Sherlock Holmes, is a collection of stories written by Conan Doyle's son Adrian Conan Doyle and Arthur's biographer, novelist John Dickson Carr. The stories are generally extrapolations of cases briefly mentioned in the canonical work, but tend to contradict themselves and each other. They are generally considered Sherlock Holmes pastiches.[17]
The stories contained in the collection are:
"The Adventure of the Seven Clocks"
"The Adventure of the Gold Hunter"
"The Adventure of the Wax Gamblers"
"The Adventure of the Highgate Miracle"
"The Adventure of the Black Baronet"
"The Adventure of the Sealed Room"
"The Adventure of Foulkes Rath"
"The Adventure of the Abbas Ruby"
"The Adventure of the Dark Angels"
"The Adventure of the Two Women"
"The Adventure of the Deptford Horror"
"The Adventure of the Red Widow"
Short stories
"The Case of the Man Who Was Wanted" (c. 1914)
This mystery, a completed Sherlock Holmes story, was found in 1942 by a Conan Doyle biographer, Hesketh Pearson, searching through a box of Conan Doyle's papers. It was originally announced that the story would not be published by the Doyle estate, but it was announced it certainly was by Doyle, as the manuscript supposedly appeared in his own handwriting. However, according to Jon L. Lellenberg in Nova 57 Minor, the manuscript was not in Conan Doyle's handwriting, but typewritten. The Strand Magazine published extracts from it in August 1943, and it was finally published after demand from Sherlock Holmes societies in 1947, when it was embraced as a new (if slightly inferior) part of the canon by The Baker Street Irregulars amongst others.[18]  Initial suspicions of forgery were reported by Vincent Starret[19] and it was eventually discovered by Hesketh Pearson that the story was originally written by Arthur Whitaker, who had sent it to Conan Doyle in hope of a collaboration. Doyle had bought the story, in the thought that he might use the idea at a later date, but he never did. Pearson, Green, Tracy and the Doyle estate agree that Whitaker wrote the story, though Haining still claims that “the opening scene between Holmes and Watson betrays the hand of the master,” and that the story is partly written by Conan Doyle. He points out that Doyle's wife, sons and biographer were fooled by the style, and it is possible there was a redraft made.[20] The story is published in Penguin's The Further Adventures of Sherlock Holmes collection under the title of "The Adventure of the Sheffield Banker."
"The Adventure of the Two Collaborators" (first published 1923)
Though never claimed by any serious critic to be a Conan Doyle work, this parody is listed here due to a popular misconception that this was written by Doyle for his friend, J. M. Barrie (of Peter Pan fame). (Perhaps contributing to this misconception is the fact that the story appears for the first time only in a work of Conan Doyle's, and all subsequent printings are from that source.) In fact, this story was written by Barrie for Doyle following a period of the two of them working together on a play. The story itself involves Doyle and Barrie visiting Holmes, with Doyle killing Holmes due to his irritating intelligence (which perhaps reflects Doyle's killing off of the character in "The Adventure of the Final Problem").
Sherlock Holmes on stage
The Painful Predicament of Mr Sherlock Holmes (1905)
The recognition of William Gillette as Sherlock Holmes was growing as a result of the success of the play Sherlock Holmes. Playing upon his most famous role, a short comedy sketch performed by William Gillette as a curtain raiser to an unrelated play. It involves a mute Sherlock Holmes, and a very talkative client. In Haining and Tracy’s books, they speculate as to whether or not this play was written by Arthur Conan Doyle. Certainly Gillette would have needed Doyle’s consent to write an original work involving Sherlock Holmes, as the character was under copyright, but it is presumed by most Sherlockians that Gillette wrote the whole thing himself. Haining, however claims that Gillette may have asked Doyle to ‘whip up something quickly for him’. However, no manuscript exists in Doyle’s hand, and no reference of the play is left by him, it has been assumed by most that it is little more than a William Gillette curiosity.[21]
Notes

 Edited by Jack Tracy.
 Edited by Peter Haining.
 Edited by Richard Lancelyn Green.
 Edited by Leslie S. Klinger.

Canon
 at Internet Archive

 at Project Gutenberg

Extracanonical works

How many Sherlock Holmes stories did Conan Doyle write?