as United States
`a2) Patent Application Publication co) Pub. No.: US 2003/0140179 Al
`(43) Pub. Date: Jul. 24, 2003
`
`Wilt et al.
`
`US 20030140179A1
`
`(54) METHODS AND SYSTEM FOR MANAGING
`COMPUTATIONAL RESOURCES OF A
`COPROCESSOR IN A COMPUTING SYSTEM
`
`(75)
`
`Inventors: Nicholas P. Wilt, Sammamish, WA
`(US); Sameer A. Nene, Redmond, WA
`(US); Joseph S. Beda III, Seattle, WA
`(US)
`
`Correspondence Address:
`Thomas E. Watson
`Woodcock Washburn LLP
`46 th Floor
`One Liberty Place
`Philadelphia, PA 19103 (US)
`
`(73) Assignee: Microsoft Corporation
`
`(21) Appl. No.:
`
`10/039,036
`
`(22)
`
`Filed:
`
`Jan. 4, 2002
`
`401
`
`Publication Classification
`
`(51) Unt C07 oaeecccccesecsssee GO6F 13/10; GO6F 9/00
`(52) US. Ce oiccecssssessssnstnssntresnesee 709/321; 709/102
`
`(57)
`
`ABSTRACT
`
`Systems and methods are provided for managing the com-
`putational resources of coprocessor(s), such as graphics
`processor(s), in a computing system. The systems and meth-
`ods illustrate management of computational resources of
`coprocessors to facilitate efficient execution of multiple
`applications in a multitasking environment. By enabling
`multiple threads of execution to compose commandbuffers
`in parallel, submitting those command buffers for schedul-
`ing and dispatch by the operating system, and fielding
`interrupts that notify of completion of commandbuffers, the
`system enables multiple applications to efficiently share the
`computational resources available in the system.
`
`Application
`
`Drawing calls
`Validation, DDI
`
`
`40
`
`4
`
`03
`
`Driver DLL
`
`
`
`
`
`
`USER MODE Flush needed or
`CB out of room
`
`
`KERNEL MODE “°%
`
`Command Buffer
`
`Scheduler
`
`
`CB selected
`
`for dispatch
` Read hardware-
` Kernel Driver
`
`Specific commands
` Kick off DMA
`operation
`
`Translate DDI->h/w
`
`405
`
`406
`
`
`
`
`Hardware
`
`
`Google Exhibit 1034
`Google v. VirtaMove
`
`Google Exhibit 1034
`Google v. VirtaMove
`
`

`

`Patent Application Publication
`
`Jul. 24,2003 Sheet 1 of 8
`
`US 2003/0140179 Al
`
`Computing
`Device
`110a
`
`
`
`
`
`Computing
`
`Device
`
`Communications
`110e
`
`Network/Bus
`
`FIG. 1A
`
`10b
`
`
`
`
`
`
`
`
`
`
`
`Server Object
`
`
`
`
`
`
`
`
`
`
`Database 20
`
`

`

`Patent Application Publication
`
`Jul. 24,2003 Sheet 2 of 8
`
`US 2003/0140179 Al
`
`
`1]|S6s9ey103
`
`
`gelSINVYHODONd
`z9Lplwoghay=asnoy
`
`
`
`
`OSTsoeyazuy
`8]He}OA-UON
`
`OVEeoejsezU]
`
`SLOW
`
`du
`
`NOILYVOI
`‘IddV¥
`
`ALOWAY
`
`uaLAdWos
`
`
`
`
`
`
`
`LOL
`
`Zeb
`
`INNVYSOud
`
`VLVd
`
`
`
`Aioway
`
`AIOwa
`
`
`
`Roly|2907
`
`IOMJON
`
`yIOMJON
`
`ae}L19}U]
`
`
`
`yndu]4asn
`
`ooRE]U]
`
`a[qeaoway
`3|QeAOWAaY-UON
`31}e/OA-UON
`SETsenpow
`
`weiboldJ3aujO
`
`jndjno
`
`jesoudu
`
`@d
`
`ul
`
`LZLsngwayshs
`
`GL
`
`soRNOzUI
`
`BI
`
`SOPLIO}U]
`
`0b
`
`Buissa201d
`
`b6LA0j1U0KYOl
`
`_—
`
`so1ydeis qb‘Sis
`
`
`
`
`
`SELsuresfioig
`
`uoneoddy
`
`pe},wayskg
`
`Buljyesodo
`
`
`
`cet(INVa)
`
`
`
`welbold
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Jul. 24,2003 Sheet 3 of 8
`
`US 2003/0140179 Al
`
`FIG. 2
`
`Create andinitialize
`commandbuffer
`
`201
`
`
`
`Initialized
`
`
`Select into threadthatis
`issuing GPU commands
`
`Reinitialize
`
`202
`
`Thread finishes writing
`commands, submits buffer
`
`Execution completes
`
`203
`
`206
`
`Context switch and
`begin execution
`
`g
`
`resources available
`
`Preempt(or time
`quantum ends)
`
`
`
`Selectfor execution
`
`205
`Standby
`
`

`

`Patent Application Publication
`
`Jul. 24,2003 Sheet 4 of 8
`
`US 2003/0140179 Al
`
`346
`
`345
`
`FIG. 3E
`
`FIGS.
`3A-E
`
`301
`
`302
`
`303
`
`304
`
`
`
`
`
`
`
`
`
`
`
`FIG. 3A
`
`311
`
`312
`
`313
`
`314
`
`345
`
`
`
`
`
`
`
`Application,
`
`
`Runtime OBJ
`
`
`Commandbuffer
`
`FIG. 3B
`
`
`
`
`
`Application
`
`
`
`321
`
`322
`
`324
`
`325
`
`
`
`Command buffer
`
`
`
`331
`
`
`
`FIG. 3C
`
`Applicati
`pplication
`
`
`
`
`
`
`
`
`
` 336
`
`om
`
`
`App
`
`PP
`
`executable
`UB.
`
`
`
`Command buffer
`
`FIG. 3D
`
`335
`
`337
`
`

`

`Patent Application Publication
`
`Jul. 24,2003 Sheet 5 of 8
`
`US 2003/0140179 Al
`
`401
`
`
`
`
`
`Application
`
`Drawing calls
`Validation, DDI
`
`40
`5
`
`4
`
`03
`
`Driver DLL
`
`Translate DDI->h/w
`
`
`
`USER MODE Flush needed or
`CB out of room
`CB out of room
`
`
`
`KERNEL MODE *%4 410
`
` CommandBuffer
`Scheduler
`
`
`
`
`405
`
`406
`
`CB selected
`for dispatch
`
`
`
`Read hardware-
`Specific commands
`
`.
`
`Kick off DMA
`
`
`operation
`
`Hardware
`
`
`FIG. 4
`
`

`

`Patent Application Publication
`
`Jul. 24,2003 Sheet 6 of 8
`
`US 2003/0140179 Al
`
`
`
`System Kernel
`
`
`dd CB to
`Get new CB
`|SUPPort
`hu
`
`
`Waiting queue
`from Initialized agen
`
`
`pool(waitif
`535
`537 kas
`:
`
`
`
`Yes||tnitialize CB needed) Cat\ veg ready
`
`
`
`
`
`
`
`submit
`Add to
`
`
`
`itiali
`ow?
`
`Initialized pool
`Free
`Select
`
`
`
`
`resources
`ready
`
`
`
`
`
`538
`540
`:
`547
`A
`
`completion Waiting CBs to Ready||of preemption dispatch
`
`
`
`
`
`Save
`context
`
`Interrupt
`Service
`Routine
`
`Command
`buffer
`finishes
`
`Dispatch
`Command
`
`Read and
`execute CB
`
`

`

`Patent Application Publication
`
`Jul. 24,2003 Sheet 7 of 8
`
`US 2003/0140179 Al
`
`401
`
`Application
`
`Drawing calis
`
`602
`
`Validation, DDI
`
`Flush needed or
`CB out of room
`
`
`
`USER MODE
`
`KERNEL MODE
`
`404
`
` ommand Buffer
`
`Scheduler
`
`
` 405
`CB selected
`
`for dispatch
`Kernel Mode
`
`
`410
`Driver
`
`Specific commands
`
`
`Read hardware-
`
`
`
`406
`
`Kick off DMA
`operation
`
`
`
`Hardware
`
`FIG. 6
`
`

`

`Patent Application Publication
`
`Jul. 24,2003 Sheet 8 of 8
`
`US 2003/0140179 Al
`
`401
`
`Drawing calls
`
`Application
`
`702
`
`
`Intermediate
`Buffers
`
`Flush needed or
`CB alofroom
`
`
`
`ommand Buffer
`Scheduler
`
`CB selected
`
`
`405
`for dispatch
`Kernel Mode
`Driver
`
`
`
`
`USER MODE
`
`CB out of room
`
`DMA Memory
`
`406
`
`Kick off DMA
`operation
`
`FIG. 7
`
`

`

`US 2003/0140179 Al
`
`Jul. 24, 2003
`
`METHODS AND SYSTEM FOR MANAGING
`COMPUTATIONAL RESOURCES OF A
`COPROCESSOR IN A COMPUTING SYSTEM
`
`COPYRIGHT NOTICE AND PERMISSION
`
`[0001] A portion of the disclosure of this patent document
`may contain material that is subject to copyright protection.
`The copyright owner has no objection to the facsimile
`reproduction by anyone of the patent documentorthe patent
`disclosure, as it appears in the Patent and Trademark Office
`patent files or records, but otherwise reservesall copyright
`rights whatsoever. The following notice shall apply to this
`document Copyright© 2001, Microsoft Corp.
`
`FIELD OF THE INVENTION
`
`[0002] The present invention relates to systems and meth-
`ods for managing computational resources of coprocessors
`in a computing system.
`
`BACKGROUND OF THE INVENTION
`
`[0003] Beginning withthefirst time-sharing system in the
`mid-1960s, operating systems (OSs) have implemented
`numerous methods of allowing multiple applications to
`share computational resources of a computer without knowl-
`edge of one another. By allocating small ‘time slices’ to each
`application, and interrupting whena ‘timeslice’ has expired,
`a computer can present each application with the illusion
`that it is running alone on the computer. For example, two
`applications could be running on a system with 1 millisec-
`ond timeslices. In such a case, each application would run
`somewhatless than half as fast (due to the overhead needed
`to swap between the two) thanif they were running onthe
`computer alone, each getting about 500 time slices per
`second. Longer time slices involve less overhead, but also
`result
`in a coarser granularity of execution, making the
`system less suitable for timing-sensitive applications.
`
`[0004] An enormous amount of work has goneinto devel-
`oping various abstractions such as virtual memory, pro-
`cesses, and threadsthat interact to provide applications with
`software models that enable the computational resources of
`the central processing unit (CPU) to be shared. However,
`these abstractions have not yet been augmented so that they
`can apply to the management of computational resources in
`graphics processing units (GPUs) as well as host micropro-
`cessors.
`
`in the last few years, graphics
`In this regard,
`[0005]
`processors have becomesignificantly more functional. The
`oumber of transistors in PC graphics chips has grown far
`faster than Moore’s Law would suggest, i.e., the number of
`transistors in graphics chips has grown from about 200,000
`in 1995 to about 60,000,000 transistors in 2001. The com-
`putational power of these chips has also increased with the
`numberof transistors; that is, not only can graphics chips
`process more data, but they can also apply more sophisti-
`cated computations to the data as well. As a result, today, the
`graphics chip(s) in a computer system can be considered a
`computational resource that complements the computational
`resources of the host microprocessor(s).
`
`[0006] The software model presented by a graphicschip is
`somewhatdifferent than the software model presented bythe
`host microprocessor. Both modcls involve context, a sct of
`
`data that describes exactly what the processor is doing. The
`contexts may contain data registers, which contain interme-
`diate results of whatever operation is currently being per-
`formed, or control registers, which change the processor’s
`behavior when it performs certain operations. On a 32-bit
`INTEL® processor, for example, the EAX data register is
`used as an accumulator, to perform multiplications, to hold
`function return values, and so on. The floating point control
`word (FPCW)is a control register that controls how floating
`point instructions round inexact results (single, double, or
`extended precision,
`toward positive or negative infinity,
`toward zero, or toward the nearest; and so on). As a general
`rule, however, graphics processors have a great deal more
`state in control registers than general-purpose microproces-
`sors. Graphics processors’ high performance stems from
`their pipelined, flexible, yet fixed function architecture. A
`great deal of control register state is needed to set up the
`operations performed by the graphics processor. For
`example, a set of control registers may include (a) the base
`address(es) and dimensions of one or more texture maps
`currently serving as input, (b) the texture addressing and
`filtering modes, the blending operation to perform between
`texture values and interpolated color values, (c) the tests to
`apply to the alpha and Z values of the fragment to decide
`whether to incorporate it into the color buffer and (d) the
`alpha blend operation to use when incorporating the color
`fragment into the color buffer at the final rendering stage.
`While graphics processors contain numerousscratch regis-
`ters such as iterators that control their processing, generally
`it is not necessary to save those registers during context
`switches because context switches are not permitted on a
`granularity that requires them to be saved. In any case, even
`if such registers must be saved during a context switch,
`generally they are not directly available to software appli-
`cations. The opacity of volatile register state to client
`software is merely one distinction between the software
`model presented by graphics processors, as compared to the
`software model presented by general purpose microproces-
`sors.
`
`To date, attempts to manage the computational
`[0007]
`resources of coprocessors, such as graphics processors, have
`been ad hoc at best. Historically, there has not been much
`demand for carcful management of these computational
`resources because only one application has been active at a
`time. In the context of the commercial workstation applica-
`tions that 3D acceleration hardwareinitially was designed to
`accelerate, such as 3D modeling and animation, end users
`typically would operate one application at a time. Even if
`more than one application were active at a given time, the
`end user would perform a significant amount of work on
`each application before switching to another and the granu-
`larity of switching between applications was on the order of
`seconds or much longer. Game applications, the second sct
`of applications to substantially benefit from graphics hard-
`ware acceleration, also are typically run one at a time. In
`fact,
`the DIRECTX® application programming interfaces
`(APIs) in WINDOWS®specifically enable game applica-
`tions to gain exclusive access to the hardware resources in
`the computer system and particularly the graphics chip.
`[0008] As graphics chips become more functional,
`it is
`reasonable to expect the numberof active applications that
`demandsignificant computational resources from them to
`increase, and for the granularity of switching between these
`applications to become fincr. In some areas, this trend is
`
`

`

`US 2003/0140179 Al
`
`Jul. 24, 2003
`
`already evident. For example, video decoding acceleration
`such
`as
`hardware-accelerated motion
`compensation
`(“mocomp”) and integer discrete cosine transform (“IDCT”)
`has been added to most graphics chips in the 2001 time-
`frame. Since it
`is possible to launch a video playback
`application and run other applications at the same time,
`playing back video and running any other application that
`demands computational resources from the graphics proces-
`sor will require careful management of those resources, to
`ensure that the video playback and other application(s) both
`deliver a high quality end user experience.
`
`[0009] Other potential sources of increased demand for
`graphics processors’ computational resources include the
`composition of multiple applications’ output, and improved
`utilization of hardware acceleration by 2D graphics APIs
`such as GDI (graphical developer interface) or GDI+. In
`short, the need for efficient and effective management of
`computational resources of graphics processor(s) in a com-
`puting system will only be increasing, along with the
`increasing power, flexibility and speed of the graphic pro-
`cessors themselves and along with increasing number of
`applications making simultaneous use of the computational
`resources of the graphics processors.
`
`SUMMARY OF THE INVENTION
`
`the present invention
`In view of the foregoing,
`[0010]
`provides systems and methods for managing the computa-
`tional resources of coprocessor(s), such as graphics proces-
`sor(s),
`in a computing system. The systems and methods
`illustrate management of computational resources of copro-
`cessors to facilitate efficient execution of multiple applica-
`tions in a multitasking environment. By enabling multiple
`threads of execution to compose commandbuffers in par-
`allel, submitting those command buffers for scheduling and
`dispatch by the operating system, and fielding interrupts that
`notify of completion of command buffers,
`the system
`enables multiple applications to efficiently share the com-
`putational resources available in the system.
`
`In various embodiments,the present invention pro-
`[0011]
`vides methodsfor controlling the computational resources of
`coprocessor(s) in a host computing system having a host
`processor, wherein the methods include controlling one or
`more coprocessors of the computing system with command
`buffers submitted to the coprocessor(s) by the host proces-
`sor, transmitting data back to the host computing system in
`response to commandsin the command buffer(s) and sched-
`uling the transmission of the commandbuffer(s) by a man-
`aging object included in the host computing system, wherein
`the computational resources ofthe coprocessor(s) are simul-
`taneously available to multiple applications instantiated on
`the host computing system.
`
`In related and other embodiments, the invention
`[0012]
`provides a computing device and computer readable media
`having stored thereon a plurality of computer-executable
`modules for controlling the computational resources of
`coprocessor(s) in a host computing system having a host
`processor, the computing device and computer executable
`modules each including a managing object for controlling
`the coprocessor(s) of the computing system with command
`buffers submitted to the coprocessor(s) by the host processor
`and for scheduling the transmission of the command buffers.
`The computing device and computer executable modules
`
`also each include means for transmitting data back to the
`host computing system in response to commands in the
`command buffer(s), whereby the computational resources of
`the coprocessor(s) are simultaneously available to multiple
`applications instantiated on the host computing system.
`
`[0013] Other features and embodiments of the present
`invention are described below.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0014] The system and methods for managing the com-
`putational resources of graphics processor(s) are further
`described with reference to the accompanying drawings in
`which:
`
`FIG.1A is a block diagram representing an exem-
`[0015]
`plary network environment having a variety of computing
`devices in whichthe present invention may be implemented;
`
`(0016] FIG. 1B is a block diagram representing an exem-
`plary nonlimiting computing device in which the present
`invention may be implemented;
`
`(0017] FIG. 2 illustrates an exemplary life cycle of a
`command buffer as it
`is used by applications to send
`commands to a coprocessor;
`
`[0018] FIGS. 3A through 3Cillustrate exemplary imple-
`mentations of a driver component that translates runtime-
`level commandsto hardware-specific commands;
`
`FIG.4 illustrates an exemplary user mode driver
`[0019]
`DLL implementation of the invention;
`
`FIG.5 illustrates an exemplary sequence of events
`[0020]
`that occur when an application makes application program-
`ming interface calls to perform graphics operations in exem-
`plary detail;
`
`FIG.6 illustrates an exemplary hardware consum-
`(0021]
`able canonical command buffer format in accordance with
`an embodimentof the invention; and
`
`an exemplary intermediate
`[0022] FIG. 7 illustrates
`canonical command buffer format in accordance with an
`embodimentof the invention.
`
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`[0023] Overview
`
`[0024] As described in the background, there are many
`present and potential sources of increased demandfor graph-
`icS processors’ computational resources. When more than
`one of these sources demandsaccess to a graphics processor
`at approximately the same time, the graphics processor may
`not be free to serve both sources in a way that meets the
`demandsatisfactorily. Thus, the present invention provides
`various ways to manage the scheduling and dispatch of
`graphics processing requests by multiple applications in
`order to ensure the computational resources of coprocessor
`units, such as graphics processing units, are cflicicntly
`distributed to the requesting applications. Accordingly, the
`present invention provides methods and systems for con-
`trolling the computation resources of coprocessor(s) in a
`host computing system, wherein the coprocessor(s) are
`controlled with command buffers submitted by the host
`processor and wherein the scheduling of the transmission of
`
`

`

`US 2003/0140179 Al
`
`Jul. 24, 2003
`
`the command buffers to the coprocessor(s) is performed by
`an object, such as the operating system, of the host com-
`puting system, making the computational resources of the
`coprocessor(s) simultaneously available to multiple appli-
`cations.
`
`[0025] Glossary of Terms
`
`[0026] The following is a brief list of terms and corre-
`sponding definitions for referencing various termsutilized in
`the detailed description of the invention below.
`
`[0027] An accelerated graphics port (AGP)is a high speed
`bus designedto facilitate the rapid transmission of data from
`a host microprocessor to a graphics peripheral.
`
`[0028] Acommandbufferis a stream of hardware-specific
`drawing commands ready for consumption by graphics
`hardware, e.g., see the definition for token stream below.
`
`[0029] A context is the hardware state utilized to execute
`commandsby a processor. Herein, a thread context refers to
`the hardware state of a host microprocessor and a hardware
`contextrefers to the state of graphics hardware. A contextis
`loaded before a stream of instructions (on the host) or
`commands(on a graphics chip) can be executed.
`
`[0030] A central processing unit (CPU) includes a host
`microprocessor.
`
`[0031] A device driver interface (DDD)is a standardized
`interface used by the operating system to interact with a
`hardware driver.
`
`is a series of commands for the
`[0032] A display list
`graphics hardware that has been recorded so the commands
`can be readily ‘played back,’ purportedly moreefficiently
`than if the original series of commandswere executed again.
`Display lists often are opaque to the application, 1e., they
`cannot be edited, since they have been translated into a
`hardware-specific form.
`
`[0033] A dynamic link library (DLL)is a unit of execut-
`able code that can be linked into and shared by multiple
`processes simultaneously. This document assumes that the
`code in DLLs executes in user mode.
`
`[0034] Direct memory access (DMA)is a hardware fea-
`ture that enables a peripheral to read or write directly from
`host-accessible memory.
`
`[0035] Adeferred procedure call (DPC) is a routine called
`by an operating system, such as WINDOWS8,that typically
`executes in response to an interrupt, but runs at a slightly
`lower priority than an interrupt service routine (see ISR
`below). As a rule, interrupt service routines should do the
`minimum processing needed to resolve the interrupt and
`queue a DPC to do other work needed to respond to the
`interrupt.
`
`[0036] A graphics processor is a hardware peripheral
`designed to accelerate drawing commandsfor graphics, such
`as bit block transfers (BLTs) and 3D graphics primitive
`drawing processes such as triangle rasterization. In this
`regard, a graphics processor may be contrasted with the
`more general and clumsy resources of the host micropro-
`cessor.
`
`[0037] A host microprocessoris a processor that executes
`the opcrating system and/or applications being run in the
`
`operating system. The host microprocessor(s) in a system
`contrast with graphics processors or other computational
`resources in the system.
`
`[0038] An interrupt is a hardware feature that cnables a
`peripheral to divert a host microprocessor from its current
`process and transfer control to a special location in memory.
`The operating system uses this mechanism to call driver
`code knownasan interrupt service routine (ISR) to respond
`to the interrupt.
`
`[0039] An interrupt service routine (ISR) is a function,
`typically in a device driver, that is called by the operating
`system when the hardware controlled by the driver signals
`an interrupt, e.g., see also DPC.
`
`Just-in-time (JIT) compiling is a compilation pro-
`[0040]
`cess that introduccs an cxtra step in the translation from
`source code to object code. In this regard, the source code is
`translated into readily compilable, hardware-independent
`intermediate language that is transmitted to a client com-
`puter, where it is compiled “just in time” into object code
`that can run on the client.
`
`{0041] Kernel mode is a privileged processor mode that
`allows system code to have direct access to hardware
`resources.
`
`[0042] A process is a logical division of labor in an
`operating system. In WINDOWS®,a process comprises a
`virtual address space, an executable program, one or more
`threads of execution, some portion of the user’s resource
`quotas, and the system resources that the operating system
`has allocated to the process’s threads.
`
`[0043] A resource is a synchronization primitive that can
`grant non-exclusive access to multiple threads, or exclusive
`access to a single thread.
`
`[0044] Asynchronization primitive is an object that can be
`used to synchronize multiple threads’ access to shared
`resources, such as critical sections, mutexes, semaphores or
`events.
`
`[0045] A thread is an executable entity that comprises a
`program counter, a user-mode stack, a kernel-mode stack
`and a set of register values.
`
`[0046] A token stream is a stream of hardware-indepen-
`dent tokens that describe a series of drawing operations. A
`token stream can be translated by a hardware-specific soft-
`ware component, such as a driver, into a command buffer or
`other series of hardware commands.
`
`[0047] A user mode is a mode of the host microprocessor
`that does not allow code to directly access hardware
`resources.
`
`[0048] Exemplary Networked and Distributed Environ-
`ments
`
`[0049] One of ordinary skill in the art can appreciate that
`a computer or other client or server device can be deployed
`as part of a computer network,or in a distributed computing
`environment.In this regard, the present invention pertains to
`any computer system having any number of memory or
`storage units, and any numberof applications and processes
`occurring, across any number of storage units or volumes,
`which may make request
`to coprocessor resources. The
`present invention may apply to an environment with server
`
`

`

`US 2003/0140179 Al
`
`Jul. 24, 2003
`
`computers and client computers deployed in a network
`environmentor distributed computing environment, having
`remote or local storage. The present invention may also be
`applied to standalone computing devices, having program-
`ming language functionality, interpretation and execution
`capabilities for generating, receiving and transmitting infor-
`mation in connection with remote or local services.
`
`[0050] Distributed computing facilitates sharing of com-
`puter resources and services bydirect exchange between
`computing devices and systems. These resources and ser-
`vices include the exchange of information, cache storage,
`and disk storage for files. Distributed computing takes
`advantage of network connectivity, allowing clients to lever-
`age their collective power to benefit the entire enterprise. In
`this regard, a variety of devices may have applications,
`objects or resources that may make requests for coprocess-
`ing resources managed by the techniques of the present
`invention.
`
`[0051] FIG. 1A provides a schematic diagram of an
`exemplary networked or distributed computing environ-
`ment. The distributed computing environment comprises
`computing objects 10a, 105, etc. and computing objects or
`devices 110a, 1106, 110c, etc. These objects may comprise
`programs, methods, data stores, programmable logic, etc.
`The objects may comprise portions of the same or different
`devices such as PDAs,televisions, MP3 players, televisions,
`personal computers, etc. Each object can communicate with
`another object by way of the communications network 14.
`This network mayitself comprise other computing objects
`and computing devices that provide services to the system of
`FIG.1A.In accordance with an aspect of the invention, each
`object 10 or 110 may contain an application that might
`request coprocessing resources of a host system.
`[0052]
`Ina distributed computing architecture, computers,
`which may havetraditionally been used solely as clients,
`communicate directly among themselves and canact as both
`clients and servers, assuming whateverrole is mostefficient
`for the network. This reduces the load on servers and allows
`all of the clients to access resources available on other
`
`clients, thereby increasing the capability and efficiency of
`the entire network. The management of coprocessing
`resources in accordance with the present invention may thus
`be distributed amongclients and servers, acting to manage
`specialized coproccessing in a waythat is efficient for the
`entire network.
`
`[0053] Distributed computing can help businesses deliver
`services and capabilities more efficiently across diverse
`geographic boundaries. Moreover, distributed computing
`can move data closer to the point where data is consumed
`acting as a network caching mechanism. Distributed com-
`puting also allows computing networks to dynamically work
`together using intelligent agents. Agents reside on peer
`computers and communicate various kinds of information
`back and forth. Agents may also initiate tasks on behalf of
`other peer systems. For instance, intelligent agents can be
`used to prioritize tasks on a network, changetraffic flow,
`search for files locally or determine anomalous behavior
`such as a virus and stop it before it affects the network. All
`sorts of other services may be contemplated as well. Since
`highly specialized coprocessing resources mayin practice be
`physically located in one or more locations, the management
`of applications requesting the use thereof is of great utility
`in such a system.
`
`It can also be appreciated that an object, such as
`[0054]
`110c, may be hosted on another computing device 10 or 110.
`Thus, although the physical environment depicted may show
`the connected devices as computers, such illustration is
`merely exemplary and the physical environment may alter-
`natively be depicted or described comprising various digital
`devices such as PDAs,televisions, MP3 players,etc., soft-
`ware objects such as interfaces, COM objects and the like.
`
`[0055] There are a variety of systems, components, and
`network configurations that support distributed computing
`environments. For example, computing systems may be
`connected together by wireline or wireless systems, by local
`networksor widely distributed networks. Currently, many of
`the networks are coupled to the Internet, which provides the
`infrastructure for widely distributed computing and encom-
`passes many different networks.
`
`there are at
`In home networking environments,
`[0056]
`least four disparate network transport media that may each
`support a unique protocol such as Powerline, data (both
`wireless and wired), voice (¢.g., telephone) and entertain-
`ment media. Most home control devices such as light
`switches and appliances may use powerline for connectiv-
`ity. Data Services may enter the home as broadband(e.g.,
`either DSL or Cable modem) andis accessible within the
`home using either wireless (e.g., HomeRF or 802.11b) or
`wired (¢.g., Home PNA, Cat 5, even power line) connec-
`tivity. Voicetraffic may enter the homeeither as wired (e.g.,
`Cat 3) or wireless (e.g., cell phones) and maybedistributed
`within the home using Cat 3 wiring. Entertainment media
`may enter the homeeither through satellite or cable and is
`typically distributed in the home using coaxial cable. IEEE
`1394 and DVIare also emergingas digital interconnects for
`clusters of media devices. All of these network environ-
`
`ments and others that may emerge as protocol standards may
`be interconnected to form an intranet that may be connected
`to the outside world by wayofthe Internet. In short, a variety
`of disparate sources exist for the storage and transmission of
`data, and consequently, moving forward, computing devices
`will require ways of sharing data, such as data accessed or
`utilized incident to the processing of data by a graphics chip.
`
`[0057] The Internet commonly refers to the collection of
`networks and gateways that utilize the TCP/IP suite of
`protocols, which are well-known in the art of computer
`networking. TCP/IP is an acronym for “Transport Control
`Protocol/Interface Program.” The Internet can be described
`as a system of geographically distributed remote computer
`networks interconnected by computers executing network-
`ing protocols that allow users to interact and share informa-
`tion over the networks. Because of such wide-spread infor-
`mation sharing, remote networks such as the Internet have
`thus far generally evolved into an open system for which
`developers can design software applications for performing
`specialized operations or
`services, essentially without
`restriction.
`
`[0058] Thus, the network infrastructure enables a host of
`network topologics such as clicnt/scrver, peer-to-peer, or
`hybrid architectures. The “client” is a memberof a class or
`group that uses the services of another class or group to
`which it is not related. Thus, in computing, a clicnt is a
`process, ie., roughly a set of instructions or tasks,
`that
`requests a service provided by another program. The client
`process utilizes the requested service without having to
`
`

`

`US 2003/0140179 Al
`
`Jul. 24, 2003
`
`“know” any working details about the other program or the
`service itself. In a client/server architecture, particularly a
`networked system, a client
`is usually a computer that
`accesses shared network resources provided by another
`computer e.g., a server. In the example of FIG. 1A, com-
`puters 110a, 1106, etc. can be thought of as clients and
`computer 10a, 105, etc. can be thought of as the server
`where server 10a, 10b, etc. maintains the data that is then
`replicated in the client computers 110a, 110,etc.
`
`[0059] A server is typically a remote computer system
`accessible over a remote network such as the Internet. ‘The
`
`client process maybeactive in a first computer system, and
`the server process maybe active in a second computer
`system, communicating with one another over a communi-
`cations medium, thus providing distributed functionality and
`allowing multiple clicnts to take advantage of the informa-
`tion-gathering capabilities of the server.
`
`[0060] Client and server communicate with one another
`utilizing the functionality provided by a protocol layer. For
`example, Hypertext-Transfer Protocol (HTTP) is a common
`protocol that is used in conjunction with the World Wide
`Web (WWW)or, simply, the “Web.” Typically, a computer
`network address such as a Universal Resource Locator
`
`(URL)or a