June 95 - Copland: The Mac OS Moves Into the Future
Copland: The Mac OS Moves Into the Future
The Macintosh operating system has continually evolved since the days when the
Macintosh was a home appliance with 128K of RAM and a floppy disk drive -- but
now the time has come for radical change. The next generation of the Mac OS,
code-named Copland, was designed specifically to serve computers with a fast
processor running several tasks and processing large quantities of data. This
preview describes Copland's major features and suggests how you might get ready
Since the first Macintosh operating system and Toolbox were developed in the
early 1980s, the needs of users and developers alike have evolved
significantly. Newer technologies, such as MultiFinder and the
PowerPC(TM) processor, have appeared on the scene. Users have come
to expect greater ease of use, more capabilities, and enhanced productivity.
Although the Mac OS has evolved along with the times, a more radical advance is
now required to take advantage of the great increases in power afforded by the
Enter Copland, a new generation of the Mac OS to be released by Apple in
mid-1996. Copland will provide a radically new architecture that includes
technologies such as preemptive multitasking and protected memory. For one
thing, it's based on a microkernel that moderates between individual tasks and
arbitrates access to the machine's resources. A number of other services have
been updated and improved, both to fulfill the requirements this change implies
and to take advantage of the new capabilities it provides. For example, the
file system has been updated to be accessible from several processes running
simultaneously in several address spaces. Similarly, the networking system has
been enhanced, as have a number of the auxiliary operating system managers such
as the Process Manager.
With Copland will also come a number of enhancements to the user experience,
including a Finder that can perform several tasks simultaneously, changes to
the appearance and feel of the interface, and advances that will make it easier
to locate and access information. (See "Moving the Mac OS Interface Into the
Future" for more details.) All of this new functionality is glued together in a
runtime model based on the Code Fragment Manager, the dynamically linked
library mechanism introduced with the first Power Macintosh computers.
MOVING THE MAC OS INTERFACE INTO THE FUTURE
BY B. WINSTON HENDRICKSON
Copland will not only radically change the foundation of the Mac OS, it will
also introduce some of the most significant changes to the user experience
since 1984. For the user, this means new personalization capabilities, built-in
assistance with tasks, and improved access to information. For the developer,
it means a robust foundation for constructing consistent and compelling
interfaces that are easier to use.
The new managers in Copland directly concerned with enhancing the user
experience are the Appearance Manager, the Assistance Manager, and Navigation
Services. The following brief descriptions of these will give you an inkling of
things to come.
THE APPEARANCE MANAGER
The Appearance Manager defines how standard user interface elements should be
presented and enables users to personalize the appearance of these elements by
choosing one of a number of graphical designs called themes. Applications can
use the Appearance Manager's capabilities to draw custom interface elements in
the style of the current theme.
The Appearance Manager provides you with
- a Pattern Manager that returns the appropriate PixPats for use as dialog
backgrounds, control colors, and other aspects of the interface
- a set of drawing primitives for rendering common interface elements
such as window title bars and dialog separator lines
- new standard interface elements, including sliders, progress
indicators, and icon buttons
- event notification when the current theme is changed, allowing you to
resync any cached appearance data
To prepare now for Copland's dynamic
system appearance, be sure not to make assumptions about interface specifics
(such as assuming that the dialog or menu background is white). Also, don't
hard-code the appearance of your application's interface elements; for
instance, avoid the use of custom definition procedures wherever possible.
THE ASSISTANCE MANAGER
The Assistance Manager supports the implementation of active user assistance,
enabling the computer to accomplish specific tasks with little or no direction
from the user. The Assistance Manager provides support for the following:
Since this active assistance is built on existing
technologies, you can start to prepare for it today. The first and most
important step is to make your application scriptable, so that it can be
automated. Second, you should provide task-based assistance using Apple Guide.
Finally, if your application provides any task delegation, you should factor
out the related code now so that you can take advantage of the Assistance
Manager under Copland.
- task delegation, allowing the creation and management of automated
activity controlled by a condition or event, such as time or mail delivery
- the ability to create templates from which tasks are created and
- user "interviews" for task configuration, based on Apple Guide's
Navigation Services replaces System 7's Standard File Package, providing a set
of tools for opening, saving, and naming documents as well as for navigating a
hierarchical information space containing such documents. These tools will
increase consistency between applications and the Finder and will enable
integration with the Finder's new and improved searching mechanism. The new
capabilities provided by Navigation Services include these:
can do a few things now to get ready for Navigation Services. First, when
customizing the Standard File dialog, render only inside your dialog items, as
they may be rearranged. Second, don't assume you're drawing into the desktop
port, because you won't be. And finally, don't try to control Standard File by
posting events to dialog items; use the documented interface instead.
Because of the large number of advances and changes in Copland, some software
will be incompatible. For instance, applications that have inappropriately
incestuous relationships with the operating system might run into problems. But
there are some things you can do now to prepare yourself for this release and
ensure that your applications will be as compatible as possible. I'll tell you
about those things as I give you a tour of Copland's microkernel, runtime
model, File Manager, and I/O architecture.
- support for a favorite items list, file list position recall (rebound),
and a more intuitive browser
- the ability to browse diverse containers such as a mailbox and return a
general-purpose reference value to documents in those containers
- one-step calls for common operations such as selecting a file or
- support for easy customization, including an extensible list of
information "panels" (based on Copland's new dialog panels)
- automatic dialog layout adjustment for active script systems
THE MICROKERNEL: A NEW FOUNDATION
The present Macintosh operating system is somewhat too trusting: it doesn't
take charge of restricting software's actions or balancing the use of the
machine's resources. Any piece of code can write all over memory, retain
control forever, and even turn off interrupts for any period of time. This
model, while once appropriate, has shortcomings in a computer with a fast
processor running several tasks and processing large quantities of data.
That's where Copland's microkernel, developed by Apple specifically for use in
the Mac OS, comes in. The microkernel serves as a referee for the system. It
moderates between many individual tasks so that none can hog the processor and
so that special code need not be implemented to share it. It also arbitrates
access to the machine's resources, including memory, preventing software from
being able to see or change data unrelated to its task.
The microkernel provides a number of services -- most of them familiar to those
conversant with kernel-based systems -- including task control, address space
management, virtual memory management, interrupt control, synchronization
primitives, and intertask messaging. These services, which we'll look at in
more detail in the following pages, serve as the basic building blocks of the
system. In most cases, your software won't use any of these kernel services
directly but will instead take advantage of them through other APIs -- APIs
that are part of System 7 but that have been reimplemented in Copland.
The Copland kernel provides full support for a variety of tasking services.
While applications will normally be cooperatively scheduled by the Process
Manager (just as in System 7), applications will also be able to create tasks
that are preemptively scheduled. Preemptive tasks are scheduled in the order of
their assigned priority and according to kernel scheduling rules; the Process
Manager doesn't manage them in the way it does applications. Such tasks behave
as threads behave in other systems. At any time, almost anything in the system
-- including the currently running application -- can be preempted to run such
a task. Interrupt handlers can't be preempted, however.
You'll be able to set the priority of preemptive tasks that you create; higher
priority tasks will run in preference to lower priority ones. By giving an
I/O-intensive task higher priority than your main application thread, you'll
gain performance very similar to that made possible today by chained completion
routines. During the relatively long I/O delays when your task is blocked, your
main application thread will execute freely. Whenever its I/O requests do
complete, the task will regain control immediately so that it can issue its
next I/O request, resulting in maximum throughput without unnecessary blocking
of other computing tasks. Similarly, you'll be able to assign a higher priority
to general application tasks than to background tasks that can afford to wait
or proceed slowly while the machine is in use (such as a background renderer
for a network-distributed 3D software package). This will ensure responsiveness
in your application and allow you to use otherwise idle CPU time.
Chained completion routines are discussed in the article "Asynchronous Routines
on the Macintosh" in develop Issue 13.
To prepare for this opportunity, you can work to make your application easier
to factor. If you remove dependencies between different portions of your
application, it'll be easier to take full advantage of Copland's multitasking
capabilities. When a Copland preemptive thread runs, the file system,
networking, and device I/O will be available, similar to the environment when a
Time Manager or Deferred Task Manager task runs in System 7. One addition is
that synchronous calls can be made; your thread will just block until the I/O
ADDRESS SPACE AND VIRTUAL MEMORY MANAGEMENT
In System 7.5, there's only one address space. A particular address always
refers to the same part of memory, and data located there can be accessed by
every part of the operating system. In Copland, by contrast, multiple address
spaces can be created, allowing code and data to be hidden from some processes.
For compatibility reasons, Macintosh applications will continue in this release
to share a single address space, while components of the operating system and
third-party software can create fully protected memory areas in which code
that's not dependent on the Macintosh Toolbox can execute. The kernel, the file
system, and several other components will create such areas to protect their
private data structures.
Each address space is divided into areas. An area can be either private
(accessible only to tasks executing in that address space) or global
(accessible at the same location in all address spaces). In addition, an area
can be either read/write to all tasks (most global areas fit into this
category) or read-only in user mode and read/write in supervisor mode. (Most
code runs in user mode; only code that needs special abilities, such as drivers
and parts of the operating system, runs in supervisor mode.) This latter
protection is used for most kernel and file system data structures; they're
located in global memory for fast and easy access (without the need to switch
to another address space) but can't be damaged by code executing in user mode.
Only the privileged clients of the system can change these structures.
In addition to having the ability to map RAM into a variety of address spaces,
Copland also uses virtual memory to provide more room than is available in
physical RAM, moving data between RAM and disk as needed. In fact, virtual
memory is always on. It's dramatically better than System 7's virtual memory in
- The new file system and better integration between the file system and
virtual memory will improve performance.
- Your application will be able to provide hints to the operating system
to allow it to tune for best performance. For example, you'll be able to tell
it that you're about to access a significant portion of a large array, and it
will asynchronously begin to bring in the pages that the array resides on.
- Fewer limitations on what can be paged will increase the available RAM
for the system. In System 7, the system heap is always held in memory and can't
be paged onto disk. In Copland, virtually all of the system -- aside from the
kernel, the file system, and the disk driver -- will be pageable.
- The disk cache will be integrated with the virtual memory system, and
the size of the disk cache will dynamically adjust based on current operations
in order to optimize performance.
- Best of all, Copland's virtual memory will dynamically expand the
amount of address space in the system as needed, giving users much more
flexibility than in System 7, where they must decide beforehand how much memory
they'll need, adjust the Memory control panel accordingly, and reboot. In
Copland, if they need to open additional applications, they can do so without
going through any rigmarole; space will be created on the fly to support their
needs, provided that sufficient disk space is available for use as a backing
Because Copland will make available a full gigabyte of address space
in which to run applications (subject, of course, to the limits imposed by the
amount of disk space available for paging), two other limitations of the System
7 memory allocation system should be alleviated. First, Copland will reduce
(though not entirely eliminate) the need for the user to configure and
reconfigure the application's memory partition to accommodate changing needs.
Also, problems with applications fragmenting the available memory for launching
more applications should be eliminated. Thus, no longer will users always have
to deal with the complex issues of memory allocation and organization to make
best use of their machines.
In addition to swapping space, Copland will support memory-mapped files. This
technology allows an application to map a file against an area of address
space; accessing locations in the address space causes the appropriate portions
of the file to be read into RAM. In system software version 7.1.2, this
technology is used by the system for paging PowerPC code when virtual memory is
on, but it's not available to applications. In Copland, it will be available to
applications; data files can be mapped for read-only or read/write access. An
application will be able to read a document just by walking through the address
space without having to manually stream it into buffers.
To be prepared for Copland's use of virtual memory, applications today should
be able to operate well in a virtual memory environment. For purposes of
performance, this means keeping a tight locality of reference; code that uses
contiguous data structures rather than structures spread all over memory will
require fewer pages to be resident for any operation. Also, take care when
allocating variable-sized buffers. Don't always attempt to allocate the largest
possible buffer, sizing it down till it fits -- a popular but potentially
wasteful habit; instead, cap buffers at points beyond which they won't gain
from more RAM. For example, if reading a file, you might cap the size of a data
buffer at 64K, because there's little to gain by reading the file in larger
On Power Macintosh computers running System 7.5, interrupts are handled by the
680x0 instruction emulator, incurring a large overhead. Even if the overhead of
the actual interrupt handler is small, a significant price is paid in invoking
the emulator, especially if a mostly native application is executing (in which
case the emulator has to be pulled into the cache on each interrupt, then
flushed out as native code is reloaded after the interrupt returns). Since a
Macintosh can easily take several hundred interrupts per second (thanks to
interrupts caused by video retrace, the old-style VBL Manager, ADB, and the
like), this can have a significant performance impact.
By contrast, Copland's I/O system, including interrupt handlers, is entirely
native; this, along with an improved architecture, should mean significantly
lower interrupt latency and better overall performance. Because of the
flexibility of the execution control available to the kernel, it will be easy
for an interrupt handler to do the absolute minimum to deal with an interrupt
(often all that's required is to acknowledge it). After control is returned
from the interrupt, another piece of code called a secondary interrupt handler
can be invoked; although under the same constraints as a hardware interrupt
handler, this handler results in the best system performance by enabling the
soonest possible exit from the hardware handler. If significant processing
needs to be done right away, this secondary handler can wake a high-priority
task to do that work, thus keeping the system from being bottlenecked by any
individual set of handlers.
Some native interrupt functionality will be delivered before Copland; see the
article "Creating PCI Device Drivers" in this issue of develop for more
SYNCHRONIZATION AND INTERTASK MESSAGING
As mentioned earlier, applications under Copland will be able to create tasks
that are preemptively scheduled. You'll be able to assign priorities to your
preemptive tasks, but this in itself won't prevent the tasks from preempting
each other at inopportune times. What you'll need in order to ensure correct
behavior from your tasks is a mechanism to synchronize access to shared
Copland provides several synchronization mechanisms, each useful in a different
situation. Any operating systems textbook includes a variety of them, and most
can be implemented in combination with others. The ones implemented in Copland
-- atomic operations, simple locks (mutual exclusions, or mutexes), read/write
locks, event flags, and event queues -- are meant to efficiently solve problems
common in Macintosh applications and the Mac OS and to provide building blocks
to implement other synchronization mechanisms if necessary.
The kernel also has an intertask message system that provides data transfer as
well as synchronization, although for basic synchronization of shared data it's
probably more than you'll need. It can move arbitrary amounts of data across
address spaces synchronously or asynchronously, by value or by reference. In
cases in which the system uses messages to implement functionality, the message
will be hidden inside an API library, so you generally won't have to deal with
the details of how the message system works.
THE RUNTIME MODEL
Copland's runtime model is based on the Code Fragment Manager (CFM). Instead of
a monolithic binary file, the operating system consists of a number of
individual libraries that combine to provide the API and system functionality.
This mechanism allows software to be built in a much more modular fashion. In
addition, the CFM provides a much more consistent context and activation model
than does the statically linked, trap-dispatched runtime model used in
680x0-based computers. Rather than having to be concerned with setting up and
restoring the A5 register to provide access to data when executing in an
interrupt task, the CFM provides a standardized transition to make sure the
data appropriate to the executing code is always available.
Although applications under Copland will continue to be cooperatively scheduled
within a single address space, developers can, as already mentioned, use kernel
services to create tasks that take advantage of preemptive multitasking and
protected memory. Tasks running in address spaces outside the Toolbox will have
a limited number of services available to them, similar to those available to
background-only applications in System 7: they'll be able to allocate memory,
communicate with other processes, and use the kernel services, the file system,
and the network, but they won't be able to draw on the screen or interact
directly with the user. Tasks that are I/O or computation intensive running in
separate address spaces will get the benefits of preemptive multitasking, and
protected memory will separate these tasks from applications, providing an
increased level of stability and reliability.
For example, Copland will include an implementation of the personal file
sharing server that runs in a separate address space. This allows the server --
which takes networking traffic and converts it to file system calls, serving
files from the local drive to remote clients -- to share the system with the
user and foreground processes as efficiently as possible. It will use
concurrent I/O to interleave its requests with those of foreground processes,
and it will get compute time at any moment when the processor is otherwise
idle, even if the foreground process is waiting for a page fault to be
Figure 1 illustrates Copland's architecture, showing the separation between the
Toolbox environment, other tasks, and the operating system. Core portions of
the operating system such as the kernel and the file system run in an address
space that's protected from the Toolbox environment and other processes;
similarly, the Toolbox environment is protected from other applications running
in their own address spaces. Each of these areas, including the kernel and
other services, can have one or several threads of execution; the kernel
preemptively switches between them. Some services, such as the file system, can
have several active threads of control, each responsible for a single
outstanding file system request. All applications running in the Toolbox
environment, including the Finder, run as a single task, inside a single
address space. However, Copland-aware applications can use preemptive threads
to best handle CPU- and I/O-intensive tasks.
Figure 1. The Copland architecture
With the new runtime model will come a number of new ways to monitor and modify
system operations without patching, which is clunky and difficult to maintain.
Currently, extensions have no formal presence in the system; they live in the
cracks between the system and applications. By providing a better-defined
environment for extensions, Copland will make them easier to write and more
stable. For example, extensions currently must patch the file system -- or use
the inefficient alternative of polling -- if they need to track file usage; the
new File Manager will let software install notification procedures that can be
called whenever a particular event, such as creating or renaming a file,
occurs. Patching will still be available; a new Patch Manager will allow
software to patch CFM entry points and will give much more control over the
installation and removal of patches, including where they fall in the chain of
To run under Copland, extensions you've developed will need to be revised.
You'll make the transition easier for users if your applications that ship with
extensions are able to run without these extensions installed. Also, desk
accessories will no longer be supported in Copland; if you depend on any desk
accessories, you should rewrite them as small applications.
THE FILE MANAGER
When the Macintosh was first introduced, it had a flat file system that was
appropriate only for floppy disks. Since then, a number of advances have been
made, including the introduction of the hierarchical file system (HFS) in 1986.
But the System 7 File Manager has these limitations:
- The File Manager implementation is closely tied to the HFS volume format,
making it difficult to support other volume formats.
- As HFS volumes grow in size, they become less efficient due to HFS's
limitation of 216 (65536) allocation blocks, making it difficult to
extend the HFS volume format.
- The File Manager can process only one operation at a time, restricting
performance when several tasks are contending for file I/O.
- The File Manager is implemented entirely in 680x0 assembly code,
limiting performance on the Power Macintosh platform.
introduce a new File Manager that addresses these limitations, directly or
indirectly. For instance, the new File Manager has been divorced from HFS
implementation details and thus imposes no limitations on volume formats;
arbitrary volume formats can be developed and plugged in. This will allow the
Macintosh to properly support any file system, including ones that feature
larger volumes, more files, or larger files than the HFS disk format. It will
even be possible to create components that provide access to distributed
network file systems or other data stores that don't easily map onto the HFS
block storage model. In fact, HFS itself will be implemented as one of these
The new File Manager will also support concurrent data transfer, so that
several file system requests can be in progress at any one time. This will
dramatically increase throughput in a number of cases. For example, copying
files from a fast file server to a hard drive now involves an entirely
serialized read over the network followed by a write to the hard drive; in
Copland, the read and write operations can be overlapped, so the copy can be
completed in as little as half the time. Throughput will even be increased in
cases where two accesses share a communications channel, such as transfers
involving several devices on the local SCSI bus or several file servers,
because most communications channels won't be filled by a single device. A
significant portion of the time it takes to read or write data to a SCSI disk
is spent waiting for the disk, not actually transferring data; in the new
model, that time can be used to transfer data to or from a different device.
Another real enhancement to the file system is the introduction of a new API,
designed to be easy to learn and use. A new API was necessary because the new
file system supports files and volumes larger than 231 bytes,
meaning that more than 32 bits are needed to store various values. The System 7
File Manager API has already been through several stages of evolution, from the
original file system calls through the HFS calls to the calls taking FSSpecs in
System 7; the new API is in lieu of reworking it one more time. In addition,
the HFS API is composed of a number of calls that take huge parameter blocks
where it isn't obvious which fields need to be set to what at any moment;
wherever possible in the new API, parameter blocks have few fields and can
easily be reused.
THE I/O ARCHITECTURE
With Copland comes a new I/O architecture designed with the following
objectives in mind:
- improved performance
- ability to support concurrent data
- sufficient abstraction to enable Apple to license the Mac OS to
manufacturers who build a variety of hardware configurations
- increased ease of configuration
- independence from the 68000 microprocessor and its runtime
The I/O architecture is organized around a number of services, each
of which can be associated with a set of drivers in a unit known as a family.
For example, the SCSI Manager can be associated with a number of SCSI interface
modules (SIMs), each of which describes a single SCSI bus. Similarly, each of
the drivers in the block storage family can provide block storage functionality
to a file system agent. Drivers in the volume format family (through the File
Manager) will manage a number of volume format agents, including the HFS agent.
Thus, the I/O architecture is structured in a hierarchy of layered components,
which pass control and data among themselves. For example, an application might
make a file system call, which would be passed to the HFS file system agent,
which would then make a request of the appropriate block device driver. That
driver could then use the SCSI Manager to transfer data to a SIM, over a SCSI
bus, to a specific SCSI device. This modularity means that other data flows are
easily constructed. For example, the file system request might be passed to an
AFP file system agent, which would result in data being transferred over the
network using Open Transport. (AFP stands for AppleTalk Filing Protocol -- the
protocol used to talk to AppleShare file servers.) These relationships are
diagrammed in Figure 2.
Figure 2. An example of relationships and control flow in the new driver
This layered architecture permits a flexible dependency chain, where no
component has too much knowledge about the implementation details of its
dependencies or its clients. A block storage driver, for instance, doesn't need
to know the details of the SIM's SCSI bus implementation or which volume format
it's being used for; it just passes requests up and down the chain. This
modular architecture should make it easier for Apple and developers to
introduce new ways of solving problems.
The driver model for PCI cards, described in this issue of develop in the
article "Creating PCI Device Drivers," was designed with this new I/O
architecture in mind. Copland will support drivers developed for PCI cards in
accordance with the guidelines presented in that article, so you would do well
to familiarize yourself with them. Old drivers will need to be revised because
they read and write to hardware locations directly; protection in the new
kernel requires that this ability be reserved to specially privileged software.
But this doesn't mean that everything packaged into Device Manager drivers will
break. The Device Manager will continue to support code packaged as a driver
that doesn't actually touch hardware and that isn't otherwise incompatible with
Copland, such as drivers that some programs use for interapplication
Note also that Open Transport, which is now available for development on System
7.5, will be the native networking protocol in Copland. Applications that use
it will make best use of the native networking stack and will be prepared to
run in a separate address space.
WHERE DOES THIS LEAVE YOU?
The transition to Copland will be the most significant operating system
transition in the history of the Macintosh. You can make this transition easier
for yourself and users if you do these things:
- Begin factoring your applications now.
- Make sure your applications can operate well in a virtual memory
- Avoid reading or modifying low-memory globals and system data
structures if at all possible in your applications.
- Avoid patching.
- Ensure that any of your applications that ship with extensions can run
without these extensions installed.
- Rewrite as small applications any desk accessories you depend on.
- Familiarize yourself with the new driver model for PCI cards.
- Make use of Open Transport for networking.
With its vastly expanded
functionality, Copland will offer an unprecedented number of new opportunities
for developers. In providing a stronger foundation for third-party products and
future Mac OS releases, Copland will lay the groundwork for years of advances
on the Macintosh platform.
TIM DIERKS, who is known for having the messiest office on the Apple R&D
campus, has been bumming around Apple for several years, including stints
working with the Macintosh Developer Technical Support group as well as on the
Copland project. Currently, he's hard at work on Apple's interactive television
solution, which gives him an excuse for watching Rocco's Modern Life at work.
He shares his office with two lizards, a corn snake, and a pinball machine --
which helps explain at least some of the mess.
Thanks to our technical reviewers Jeff Cobb, Dave Evans, John Iarocci, Wayne
Meretsky, Mike Neil, Steve Szymanski, and Russell Williams. Special thanks to
Russell Williams for information on synchronization services for preemptively