(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2011/0219419 A1
`Reisman
`(43) Pub. Date:
`Sep. 8, 2011
`
`US 20110219419A1
`
`(54) METHOD AND APPARATUS FOR BROWSING
`USING ALTERNATIVE LINKBASES
`
`(76) Inventor:
`
`(21) Appl. No.:
`(22) Filed:
`
`Richard Reisman, New York, NY
`US
`(US)
`13/094,505
`Apr. 26, 2011
`
`Related U.S. Application Data
`(63) Continuation of application No. 10/434,032, filed on
`May 8, 2003
`Pat. No. 7.987.491
`ay 8,
`, now Fal. No. 7.98S / .491.
`(60) Provisional application No. 60/379,635, filed on May
`10, 2002, provisional application No. 60/408,605,
`filed on Sep. 6, 2002, provisional application No.
`60/455,433, filed on Mar. 17, 2003.
`
`Publication Classification
`
`(51) Int. Cl.
`(2011.01)
`HO)4N 7/173
`(52) U.S. Cl. ........................................................ 725/112
`
`ABSTRACT
`57
`(57)
`Systems and methods for navigating hypermedia using mul
`tiple coordinated input/output device sets. Disclosed systems
`and methods allow a user and/or al author to control what
`resources are presented on which device sets (whether they
`are integrated or not), and provide for coordinating browsing
`activities to enable such a user interface to be employed
`across multiple independent systems. Disclosed systems and
`methods also support new and enriched aspects and applica
`tions of hypermedia browsing and related business activities.
`
`SYSTEMS f
`DEVICE SETS
`
`
`
`1. 1OO
`
`CONTENT f
`CONNECTIVITY
`110
`
`HOME
`NETWORK
`
`WIRELESS
`
`126
`
`BROADCAST
`SATELLITE
`CABLE
`VIDEO ON DEMAND
`STREAMING MEDIA
`WEB
`WIRELESSPORTALS
`TRANSACTIONS
`AND THE LIKE
`
`15O
`D
`PDAF PHONE
`
`STORAGE
`
`SYSTEMELEMENTS
`
`

`

`Patent Application Publication
`
`Sep. 8, 2011 Sheet 1 of 10
`
`US 2011/0219419 A1
`
`0 || ||
`
`
`
`ALIAI LOENNOO | || NE LNO O
`
`
`
`
`
`
`
`
`
`
`
`
`
`EXHIT EIHL CIN\/
`
`
`
`
`
`S_LNE WEITE WELSÅS
`
`

`

`Patent Application Publication
`
`Sep. 8, 2011 Sheet 2 of 10
`
`US 2011/0219419 A1
`
`DEVICE SET 1
`
`1. 200
`
`STB
`(SYSTEM1)
`210
`
`,
`PC
`(SYSTEM2)
`I
`220
`
`A MONITOR
`:
`222
`
`DEVICE SET 2
`
`FIG.2a
`
`DEVICE SET 1
`
`
`
`1. 250
`
`\!CONTROLLER
`(STB/PC)
`7. (SYSTEM1)
`y
`260
`w
`
`A.
`A MONITOR
`i
`272
`
`DEVICE SET 2
`
`FIG.2b
`
`

`

`Patent Application Publication
`
`Sep. 8, 2011 Sheet 3 of 10
`
`US 2011/0219419 A1
`
`312
`
`3OO
`
`1.
`
`314
`
`326
`
`324
`
`TYPCAL DISPLAYS
`FIG. 3
`
`TV
`
`ITV
`
`332
`
`333
`
`PC
`(+ITV)
`
`INTERACTIVE
`CONTENT
`
`32O
`
`WINDOW2
`MENU BAR
`
`INTERACTIVE
`CONTENT
`
`330
`
`
`
`PDA?
`(PHONE)
`
`MENU
`SCREEN
`
`CONTENT
`SCREEN
`
`FUTURE
`VIDEO
`A
`
`340
`
`342
`
`344
`
`346
`
`

`

`Patent Application Publication
`
`Sep. 8, 2011 Sheet 4 of 10
`
`US 2011/0219419 A1
`
`007_^
`
`
`
`
`
`????????????
`?ž
`
`|××××
`
`

`

`Patent Application Publication
`
`Sep. 8, 2011 Sheet 5 of 10
`
`US 2011/0219419 A1
`
`510
`
`SYSTEM
`A
`
`560
`
`SYSTEM
`B
`
`
`
`
`
`
`
`
`
`
`
`500
`
`1.
`
`SESSION A1
`52O N
`
`530
`
`
`
`
`
`
`
`STATE
`EXPORTER IMPORTER f
`TRACKER
`
`540
`
`~
`
`TRANSFER
`STATE RECORD
`(SESSION A1)
`
`TRANSFERRED
`SESSION A1"
`570
`
`
`
`TRANSFER
`A
`\|STATE RECORD
`(SESSION A1'
`TRACKING)
`
`A.
`STATE
`EXPORTER (IMPORTER / /
`TRACKER
`-
`590
`-
`
`-
`
`-
`
`SESSION
`TRANSFER
`FIG. 5
`
`

`

`Patent Application Publication
`
`Sep. 8, 2011 Sheet 6 of 10
`
`US 2011/0219419 A1
`
`SYSTEM
`A
`INTERACTIVE SESSION
`IN PROGRESS
`
`D
`
`605
`
`TRANSFER REQUEST:
`SESSION A1 (... Ai)
`TOSYSTEMB
`
`610
`
`
`
`ASSEMBLE TRANSFER STATE
`RECORD(S) FOR A1 (...A)
`615
`
`(USER
`INPUT)
`
`600
`1.
`
`SYSTEM
`B
`UNRELATED ACTIVES
`IN PROGRESS
`
`650
`
`IMPORT SESSION A1 (... Ai)
`STATE
`
`655
`
`EXPORT TO SYSTEMB
`(OR CONTROLLER)
`620
`
`SETUP A1' (...A.")
`ONSYSTEMB
`
`660
`
`IF TRACKING,
`SYNCHRONIZE SESSIONS A1
`AND A1" AND ECHO
`INTERACTIONEVENTS
`625
`
`IFNOTRACKING,
`DISPOSE OF A1 AS REQUESTED
`630
`
`IF TRACKING,
`SYNCHRONIZE SESSIONS A1
`AND A1" AND ECHO
`INTERACTIONEVENTS
`665
`
`SYSTEM FLOW
`
`FIG. 6
`
`

`

`Patent Application Publication
`
`Sep. 8, 2011 Sheet 7 of 10
`
`US 2011/0219419 A1
`
`Full Connectivity
`
`7OO
`1.
`
`FIG. 7a
`
`
`
`No Local Connection (e.g.: Cable Relay)
`
`FIG. 7b
`
`NoLimited Back-channel (e.g.: Satellite)
`
`
`
`FIG. 7C
`
`

`

`Patent Application Publication
`
`US 2011/0219419 A1
`
`
`
`.
`
`. .
`
`.
`
`.
`
`.
`
`
`
`

`

`Patent Application Publication
`
`Sep. 8, 2011 Sheet 9 of 10
`
`US 2011/0219419 A1
`
`
`
`
`
`TEN\fd TOHINOS
`
`OZ6
`
`096<? V || || || || || || || ZL || LI
`099
`
`

`

`Patent Application Publication
`
`Sep. 8, 2011 Sheet 10 of 10
`
`US 2011/0219419 A1
`
`S.
`
`O
`
`- C
`w
`-
`
`-
`g
`
`C
`
`?h w
`
`s
`
`Y
`O O
`
`?
`
`(r)
`
`?y o
`Se
`
`Yy
`
`(e'
`
`
`
`w f
`
`C
`N
`N
`
`e
`
`CD
`
`&
`
`S.
`9
`
`

`

`US 2011/0219419 A1
`
`Sep. 8, 2011
`
`METHOD AND APPARATUS FOR BROWSING
`USING ALTERNATIVE LINKBASES
`
`0001. This application claims the benefit of U.S. Provi
`sional Application No. 60/379,635, filed May 10, 2002, U.S.
`Provisional Application No. 60/408,605, filed Sep. 6, 2002,
`and U.S. Provisional Application No. 60/455,433, filed Mar.
`17, 2003, all of which are incorporated herein by reference.
`
`FIELD OF THE INVENTION
`0002 The present invention is directed generally to inter
`active television and similar interactive hypermedia Such as
`from television or Internet Sources, and more particularly to
`the provision and use of user interfaces that permit interaction
`using multiple coordinated device sets.
`
`BACKGROUND OF THE INVENTION
`0003. While “convergence” of television (TV) and com
`puter technology have been a major focus of innovation and
`commercial development since the early 1990s, particularly
`in the area of “interactive television” (ITV), there remains a
`huge gulf in the nature of the user experience of ITV and of
`computer-based media such as the World WideWeb. Conver
`gence has taken hold in infrastructure technologies, with digi
`tal and computer-based TV (DTV) editing, production, dis
`tribution, transmission, and devices. At heart ITV is a matter
`of hypermedia browsing, the process of browsing linked
`media resources like the Web, differing only on its emphasis
`on video as the central medium.
`0004. However, there remains a divide relating to the dra
`matic difference in how TV-centric and computer-centric
`media are used, and to the cultural divide between the TV
`production and distribution industry and the computer and
`Web industries that has prevented a convergence in user expe
`rience from developing or even being seen as possible and
`desirable. TV usage and directions are focused on its charac
`ter as a lean-back, across-the-room, low resolution, and rela
`tively passive, relaxed experience of couch potatoes viewing
`large, often shared TV screens with simple remote controls.
`PC usage and directions are focused on its character as lean
`forward, up-close, high resolution, and intensive, highly
`interactive experiences of individuals with PC-styles dis
`plays, keyboards, and pointing devices. Variant device sets
`and applications, such as PDAs, tablets, and video games,
`could be taken as Suggestive of the desirability of selecting
`among alternative usage modes and form factors, but only
`very limited aspects of these Suggestions have been recog
`nized.
`0005. The limitations of these radically disparate device
`set form factors have severely limited the appeal of ITV. ITV
`promises to greatly enrich the TV experience by allowing
`interactive features that can range from access to Supplemen
`tary enhancement material Such as background on programs,
`casts and players, sports statistics, polls, chat messaging, and
`interactive advertisements and purchase offers (“t-com
`merce'), and all manner of other tangential information, to
`ways to vary the core program content by acting on viewer
`input and choices as to camera angles or even alternative
`plots, as well as providing improved control of the core expe
`rience with electronic program guides (EPGs), personal
`video recorders (PVRs) and video on demand (VOD) and
`similar features.
`
`0006. The problem is that these interactive features are not
`well served by the TV usage mode and form factor, and their
`use interferes with the basic TV experience. Rich interaction
`with a TV is inherently difficult. Presentation of information
`is limited by the poor capabilities of a TV screen for present
`ing text, menus, and navigations controls, and the crude input
`capabilities of a remote control. The rich information and
`navigation functionality available on a Web browser or other
`PC-based user interface (e.g., UI, especially graphical user
`interfaces, GUIs) must be “dumbed-down” and limited for
`use on a TV, and even use of high-definition TV (HDTV) may
`not significantly ease that people do not like to read or do
`fine work from across-the-room, it is just not comfortable
`ergonomics. Furthermore, the attempt to show interactive
`controls and enhancements on the TV interferes with viewing
`by the person interacting, as well as any other viewers in the
`room. Compounding these issues and slowing recognition of
`better solutions is the dominance of the cable TV industry, its
`struggles in developing and deploying the advanced set-top
`boxes (STBs) needed to offer meaningful ITV services of the
`form it envisions, and its orientation to closed, proprietary
`systems that do not fully exploit or adapt to advances in the
`PC and Internet world.
`0007. The computer community has attempted to market
`PCs that include a TV tuner to support TV function in a
`PC-centric model, as promoted by the PC-DTV Consortium.
`However, these systems suffer from the converse problem, in
`that their form factors are not suited to the fact that most
`people do not Want to watch TV at a PC, with its lean
`forward, up-close form factor. Furthermore, such devices
`cannot effectively receive protected cable or satellite pro
`gramming. And here, as with conventional TVs, the use of a
`single system forces technical, economical, and usage con
`straints on the inherently complex, multi-tasking, man-ma
`chine behavior that is desired in a rich hypermedia browsing
`experience.
`0008. There has also been some recognition that PCs pro
`vide a way around the limited installed base of advanced
`STBs, but this is generally perceived only as a limited stop
`gap. So called Enhanced TV or Extended TV or “teleweb
`bing has emerged to exploit the fact that tens of millions of
`households have PCs in the same room as their TVs, and can
`surf related content on the Web while watching TV. Some
`broadcasters such as ABC and PBS have exploited this to
`offer Web content synchronized to a TV program, but it is the
`user who must coordinate the use of the PC with the TV, by
`finding the appropriate Web site. In spite of the fact that the
`installed base for such openhardware is some tentimes that of
`ITV-capable set-top boxes, the ITV community generally
`views such “two-box” solutions as an unfortunate and awk
`ward stopgap that may be desirably Supplanted by advanced
`“one-box” systems whose wide deployment must be awaited.
`Some major reasons for this lack of acceptance are that this
`simplistic two-box model Supports only very limited, pre
`defined synchronization of the availability of TV and
`enhancement content that is built into a rigidly fixed two-box
`structure at the content source, and, even more importantly,
`that it completely fails to address any coordination of user
`activity at the two separate boxes.
`0009 Across all of this, the key elements that are lacking
`are provision of a broadly flexible, powerful, selective, and
`simple user interface paradigm for browsing hypermedia
`across multiple device sets, whether they are integrated or
`not, with related methods for user and/or authoring control of
`
`

`

`US 2011/0219419 A1
`
`Sep. 8, 2011
`
`such a UI, and provision of an effective method for indepen
`dent systems to coordinate browsing activities to enable Such
`a user interface to be employed across multiple independent
`systems. Further lacking across all of these aspects is delivery
`of these services in a way that provides the user with a
`Smoothly integrated experience in which interactions on the
`multiple systems are coupled or decoupled to the degree
`appropriate to the task of the moment.
`SUMMARY OF VARIOUSEMBODIMENTS THE
`INVENTION
`0010. According to embodiments of the present invention
`there are provided systems and methods for navigating hyper
`media using multiple coordinated input/output device sets.
`Embodiments of the inventionallow a user and/oran author to
`control what resources are presented on which device sets
`(whether they are integrated or not), and provide for coordi
`nating browsing activities to enable such a user interface to be
`employed across multiple independent systems. Embodi
`ments of the invention Support new and enriched aspects and
`applications of hypermedia browsing and related business
`activities.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`0011
`Further aspects of the instant invention will be more
`readily appreciated upon review of the detailed description of
`the preferred embodiments included below when taken in
`conjunction with the accompanying drawings, of which:
`0012 FIG. 1 is a block diagram of an exemplary assem
`blage of user Systems, networks, and remote services for
`implementing certain embodiments of the present invention.
`0013 FIGS. 2a and 2b are a set of block diagrams of
`exemplary groupings of device sets and systems in the assem
`blage of FIG. 1.
`0014 FIG.3 is a schematic diagram of a number of exem
`plary user interface display layouts according to certain
`embodiments of the present invention.
`0015 FIG. 4 is a schematic diagram of an exemplary
`structure for state information relating to systems within the
`assemblage of FIG. 1, relating to the coordination of a mul
`timachine user interface according to certain embodiments of
`the present invention.
`0016 FIG. 5 is a schematic diagram of an exemplary pro
`cess, performed by the systems of FIG. 1, for transferring
`state data according to certain embodiments of the present
`invention.
`0017 FIG. 6 is a flow chart of an exemplary process,
`performed by the systems of FIG. 1, for transferring state data
`according to certain embodiments of the present invention.
`0018 FIGS. 7a, 7b, and 7c are a set of block diagrams of
`exemplary alternative communication configurations in the
`assemblage of FIG. 1
`0019 FIG. 8 is a block diagram of details of an exemplary
`portal facilitating session coordination linkage in the assem
`blage of FIG. 1.
`0020 FIG.9 is a schematic diagram portraying exemplary
`further detail of a user interface for a cross-program portal.
`0021
`FIG. 10 is a schematic diagram of an exemplary
`LiberatedSTB configuration.
`DETAILED DESCRIPTION OF THE INVENTION
`Overview
`0022. The present invention may be described, in various
`embodiments, as a system and method for navigating hyper
`
`media using multiple coordinated input/output device sets. It
`provides a broadly flexible, powerful, selective, and simple
`user interface paradigm for browsing that allows the user
`(and/or an author) to control what resources are presented on
`which device sets (whether they are integrated or not), and
`provides an effective method for coordinating browsing
`activities to enable such a user interface to be employed
`across multiple independent systems.
`0023. One aspect is, in the spirit of human-centered
`design, to anticipate and be responsive to the user's desires
`(and the author's suggestions) as to what resources to present
`where, in order to make the best possible use of the hardware
`resources at a user's disposal. Homes, offices, and other per
`sonal environments of the future will have a rich array of
`computer-based input output devices of various kinds, some
`general purpose, and some more or less dedicated to specific
`uses. The desire is to minimize constraints on what system
`resources can be used for a given task, to enable the most
`powerful browsing experience possible. Browsing of hyper
`media, such as in the case of ITV is a task in which the use of
`multiple devices might be valuable because it may be
`expected to be a dominant activity, if supported effectively,
`and because of the disparity of UI issues between watching
`extended video segments and doing intensive interactions
`(such as with Web media) that may be more or less closely
`coupled with Such video segments.
`0024 Prior work has generally not recognized that it is
`inherent in rich ITV and similar forms of video-centric hyper
`media browsing to be best served as “two-box multitasking
`experiences, at least much of the time, and the problem is not
`to squeeze it into one box (and fight over which box’s func
`tionality and form factor is better), but to enable effective
`coordination of both boxes. While the TV vendors and the PC
`vendors might fervently wish to offer a single system that
`meets the needs of ITV users, that is not an effective solution.
`If one assumes that an ideal level of coordination among
`device sets can be enabled and explores usage scenarios, it
`can then be seen that different modes of viewing are best
`served by different device set form factors. These modes are
`not fixed for the duration of a session or task, but can blend,
`overlap, and vary as the flow of a set of linked tasks changes.
`What begins as a TV-centric browsing (or pure viewing)
`experience may shift to casual use of a PC for light interaction
`(such as looking at menus and options or doing a quick
`lookup) to intensive PC-centric activity (and then back
`again). The user may shift focus from the TV to both, to
`primarily the PC for a time, then become involved in the TV
`again. Conversely, an user at a PC may shift to immersion in
`a TV program or movie, then return to intensive use of the PC.
`While some broad usage patterns tend to favor video on the
`lean-back TV device set and interactivity on the lean-forward
`PC-type device set, other issues may relate to incidental view
`ing of video from a PC centric phase of activity, and casual
`interactions with enhancements in a TV-centric experience,
`as well as a complex mix of secondary issues, such as quality
`of-service factors, whether an alternative device set is at hand
`and ready for use, other activities, presence of other people,
`location/setting, mood, and the like.
`0025. The point in a session at which a user may wish to
`shift device sets may depend not only on the immediate task,
`but the user's expectation of where that task is leading, so an
`intensive task soon to end may not warrant a shift from TV to
`PC, but a less intensive task leading to deeper interaction may
`warrant an early shift. Varying form factors of different TV
`
`

`

`US 2011/0219419 A1
`
`Sep. 8, 2011
`
`devices and of the range of PCs, PDAs, tablets, and Internet
`appliances may also affect what tasks a user wants to do on
`what device, with what UI. At the same time, to avoid bur
`dening the user with the complications of too much flexibility
`and too many choices, it may be desirable that both the user
`and the content author be able to pre-set affinities, prefer
`ences, and recommendations, relating to task types, content
`types, and device availabilities, that could automatically
`place elements on the device set or device set group that is
`presumably best Suited to the apparent context, while leaving
`the user with the ability to recognize that expected targeting
`(based on conventions and/or unobtrusive cues) and to accept
`it with no further action, or override it if desired.
`0026. Providing the desired flexibility can be viewed in
`terms of three interrelated issues, one of structuring an effec
`tive and flexible multimachine user interface (MMUI) for
`browsing by a user, one of providing methods (such as
`markup) for the resource creator/author/producer to aid in
`exploiting that MMUI, and one of implementing Such an
`interface on a wide range of hardware and software, including
`systems for which such usage may not be a primary mission
`(including both new systems and legacy systems).
`0027. A general approach to a MMUI for browsing that
`provides both user control and authoring Support may advan
`tageously build on the concept of targets for presentation of
`linked resources already present in hypermedia formats Such
`as HTML (and XLink). In HTML, the link target attribute can
`be used to specify which of multiple frames a linked resource
`is to be presented in, with options that include the current
`frame, another existing frame, or a new frame. Coded speci
`fications within the link are typically set by authors/producers
`of content, and controls in the browser allow the user to
`override and alter these settings, such as (with MICROSOFT
`Internet Explorer, MSIE) by using a shift-click combination
`to indicate that a link should be opened in a new window.
`Extending this to an MMUI can be done by expanding the
`coding of target attributes and by adding new browser control
`options, such as control-click, to target a window on an alter
`nate device set. Additional control can be achieved by extend
`ing the richer drop-down control that is invoked in MSIE by
`right-clicking on a link. That drop-down list can be extended
`to list windows on alternate device sets. This provides a very
`flexible, general, and simple way to shift activity from one
`device set to another. Similar controls can be provided on
`simpler devices, such as for example, with a TV remote
`control, instead of select to activate a link to an enhancement
`overlay on the TV, a combination such as exit-select could be
`used to activate that link to an associated PC, or a new control
`button could be provided. As with current browsers, varia
`tions on Such controls can also be defined to open the current
`resource at a second location (cloning).
`0028. To implement such an interface on multiple inde
`pendent device sets, the ending system must be given infor
`mation to inform it when a link is to be activated, to what
`resource, with what browser attributes, and with what context
`information. A basic method is to transfer from the starting
`system to the ending system a link activation message that
`that includes a state record and contains relevant link arc
`information. The State record contains essential information
`on the state of the browser and related activities on the starting
`system that can be used at the ending system to configure its
`browser and related context accordingly. A state exporter/
`importer/tracker component may be provided as an addition
`
`to a standard browser to provide these functions (with
`exporter/importer function being sufficient for simple appli
`cations).
`0029. In simple embodiments, export from the starting
`system and import at the ending system need be done only
`once per transfer of locus. In certain embodiments, full event
`synchronization can be maintained, when desired, by the state
`tracker to provide ongoing collaborative functionality, as
`well. This is useful in the case of multiple users, and also can
`be useful for a single user that desires the ability to use both
`device sets in a fully replicated mode. However an advantage
`of the proposed method over conventional collaboration and
`synchronization systems, is that Such ongoing event synchro
`nization is not needed for basic MMUI browsing by a single
`user, and the complications and overhead of continually log
`ging, exporting and importing all events that may alter state
`can be avoided. Instead, state information need be assembled
`for transfer only when a transfer is actually invoked, and only
`at the necessary granularity. This simple, occasional, coarse
`grained transfer is readily added to any browser of existing
`architecture, unlike more fine-grained full synchronization
`approaches, which require either excessive tracking activity,
`display replication approaches, or rearchitecting of browsing
`to use model-view-controller architectures, such as in event
`replication approaches.
`0030. Another key benefit of this method is that it is
`readily applied to heterogeneous systems with only simple
`addition of an exporter/importer and some new UI functions
`to each system's own native browser. This exploits the fact
`that the underlying resources being browsed can be common
`to all systems, and that at a high level, browsing State is
`relatively independent of system architecture. Thus the
`method is readily applied to both TV and PC-based systems,
`and could be added to existing or new systems by manufac
`turers, integrators, distributors, service providers, or by end
`users themselves. The proposed methods are well suited to
`standardization, which could facilitate the inherent capability
`of the methods described here to allow any suitably functional
`device sets and systems to be used together in the desired
`coordinated fashion, regardless of its internal software and
`hardware architecture, vendor, or provisioning. Use of XML,
`RDF, and related standards is suggested to facilitate this.
`These features for ad hoc provisioning and use of devices
`acquired for other purposes removes a major hurdle to the
`introduction of MMUIs for ITV and other hypermedia brows
`ing applications. Thus, for example, a household need not buy
`a lean forward device for ITV, but can simply use an existing
`PC, PDA, tablet, or the like.
`0031. As a further perspective on the range of ways to use
`a MMUI for interactive TV and similar hypermedia browsing
`as described herein it may be helpful, perhaps with regard to
`varied levels of multitasking and (correspondingly) of how
`closely enhancement resources relate to the viewing of a
`“primary program’, to consider the term “interactivity.” The
`term “interactive TV might tend to suggest that a viewer
`interacts with a TV device and/or with TV content. Such a
`view may be appropriate to many kinds of ITV interaction.
`However, in considering the embodiments of MMUI brows
`ing described herein, it is noted that many cases of what might
`be broadly described in terms of “interactive TV could
`involve interactions that need not directly involve the TV
`device, or even the actual program content that is "on the TV’.
`but that, for instance, involve other content perhaps more or
`less closely related to the program content that is on the TV.
`
`

`

`US 2011/0219419 A1
`
`Sep. 8, 2011
`
`0032. From such a standpoint, the term “coactivity” might
`be considered as useful to emphasize the possible distinction
`between what is interacted with and what is on TV. Thus, for
`example, in the case of a loosely coupled interactive Sub-task
`on a PC that relates to a program on the TV, the interactivity
`that takes place as part of that sub-task might be described as
`“coactivity”.
`0033. The concept of coactivity could be useful, for
`instance, in clarifying certain motivations for using a MMUI.
`To the extent that one might think of a task as “interacting
`with the TV, the idea of using another device set (for
`example, a PC) might seem odd and unnatural to the task.
`However, by recognizing that many interactive tasks actually
`involve coactivity with content that might not be “on the TV’.
`but that relates to what is on the TV, the use of a separate
`device set might be more readily recognized as possibly being
`natural and appropriate. Accordingly, “two-box” embodi
`ments of the present invention could be seen as potentially
`well suited to the essential nature of ITV and similar hyper
`media browsing, and not as a “stopgap' or “work-around
`embodiments. Development of this new paradigm for man
`machine-media interaction affords enriched capabilities and
`Supports new and enriched applications.
`0034. As used herein, the term “hypermedia” is meant to
`refer to any kind of media that may have the effect of a
`non-linear structure of associated elements represented as a
`network of information-containing nodes interconnected by
`relational links. Hypermedia is meant to include “hypertext.
`and the two may at times be used synonymously in the broad
`sense, but where stated or otherwise clear in context, “hyper
`text can refer particularly to text content, and “hypermedia
`to extend that to content that includes other formats such as
`graphics, video, and Sound. The terminology used herein is
`meant of be generally consistent with that used in WorldWide
`Web Consortium (W3C) recommendations.
`0035. The associations of elements may be specified as
`“hyperlinks' or “links, such as described by the XLink
`(XML Linking Language), SMIL (Synchronized Multimedia
`Integration Language), HTML, XHTML, and similar W3C
`recommendations. Links define an association between a
`“starting resource, the source from which link traversal is
`begun, and an “ending resource, the destination, collectively
`referred to as “participating resources.” A “resource' is used
`to refer to any addressable unit of information or service and
`may at times refer to a resource portion rather than a whole
`resource, and a “content resource to refer to any resource
`Suited to presentation to a user. In the context of hypermedia,
`“node' may be used synonymously with resource. “Naviga
`tion' is meant to refer to the process of following or “travers
`ing links. Unless specifically indicated as “link navigation”
`or otherwise clear in context, navigation also is meant to
`include the control of presentation within a resource. Such as
`scrolling, panning, and Zooming, using VCR-like controls to
`play a continuous media resource, and the like. Addresses for
`Internet resources are typically in the form of Universal
`Resource Locators (URLs) or Universal Resource Names
`(URNs) or other Universal Resource Identifiers (URIs), but
`may be based on any other Suitable addressing mechanism.
`Hypermedia resources may contain content (also referred to
`as mediadata) and metadata (including hyperlinks), aspects
`of a resource may be declarative (such as markup) or proce
`dural (such as embedded logic or program code elements) and
`may include embedded resources.
`
`Links may have information about how to traversea
`003.6
`pair of resources, including direction and application behav
`ior information, called an “arc” and Such information may
`include link "elements' having “attributes that take on “val
`ues.” Behavior attributes include “show' to specify how to
`handle the current state of the presentation at the time the link
`is activated, “external’ to specify whether the link is to be
`opened in the current application, or an external application,
`Such as one Suited to a special media type, “activate' or
`“actuate” to specify whether the link is triggered by some
`event, typically user interaction, or automatically traversed
`when its time span is active, and “target” to specify either the
`existing display environment in which the link should be
`opened (e.g., a SMIL region, an HTML frame or another
`named window), or trigger the creation of a new display
`environment with the given name. It should be noted that the
`term target is sometimes also used in the art to refer to an
`ending resource as the target of a link, as for a “target
`resource' or “target page.”
`0037 Links may be contained in the starting or ending
`resource, “outbound' or “inbound respectively, or may be
`independently stored as “third-party' arcs. Standard HTML
`links are typically outbound, but inbound and third-party link
`arcs may be useful. Such as for adding links that are external
`to read-only or third-party content. By providing Such exter
`nal, third-party links, resources not originally intended to be
`used as hypermedia can be made into hypermedia. Third
`party links may be collected in “linkbases.” Linkbases may be
`directly associated with their starting resources by a resource
`that leads to both the starting resource and the linkbase,
`referred to herein as “coupled’ linkbases, such as a set of
`image map links in a Web page that has an embedded image
`link, or may be “decoupled' and obtained by other means.
`0038. Where so indicated or clear in context, the term
`hypermedia may also be used to include “hypermedia-like
`resources and systems that do not use coded links as such, but
`which Support functionally similar non-linear resource rela
`tionships using other more or less similar mechanisms. Such
`as special coding and logic that implements structures such as
`menu structures that have a defined graph structure, transac
`tion request forms that have an associated address or other
`process identifier for transaction Submission, and selectable
`content elements having a defined relationship to other
`resources or actions. This is meant to include any scheme that
`associates defined resource anchors or triggers with corre
`sponding actions. Use of VCR-like or audio rec