Publication Classification
`
`(51)
`
`Int. Cl.
`A63H 30/04
`
`A self-propelled device includes a drive system, a wireless
`communication port, a memory and a processor. The memory
`stores a first set of instructions for mapping individual inputs
`from a first set of recognizable inputs to a corresponding
`command that controls movement of the self-propelled
`device. The processor (or processors) receive one or more
`inputs from the controller device over the wireless commu-
`nication port, map each of the one or more inputs to a com-
`mand based on the set of instructions, and control the drive
`system using the command determined for each of the one or
`more inputs. While the drive system is controlled, the proces-
`sor processes one or more instructions to after the set of
`recognizable inputs and/or the corresponding command that
`is mapped to the individual inputs in the set of recognizable
`inputs.
`
`
`
`(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2012/0168240 A1
`WILSON et al.
`Jul. 5, 2012
`(43) Pub. Date:
`
`US 20120168240A1
`
`(54) SYSTEM AND METHOD FOR
`CONTROLLING A SELF-PROPELLED
`DEVICE USING A DYNAMICALLY
`CONFIGURABLE INSTRUCTION LIBRARY
`
`Inventors:
`
`Adam WILSON, Longmont, CO
`(US); Dan Danknick, Lafayette,
`CO (US)
`
`13/342,892
`
`Jan. 3, 2012
`
`Related US. Application Data
`
`Provisional application No. 61/430,023, filed on Jan.
`5, 2011, provisional application No. 61/430,083, filed
`on Jan. 5, 2011, provisional application No. 61/553,
`923, filed on Oct. 31, 2011.
`
`(2006.01)
`(2006.01)
`G05D 1/02
`(2006.01)
`B62D 63/02
`(52) us. Cl. ........................................... .. 180/167; 701/2
`
`(57)
`
`ABSTRACT
`
`|PR2017-01272
`
`IPR2017-01272
`Spin Master EX1010 Page 1
`
`

`

`Patent Application Publication
`
`
`
`Wigs-gmm
`feéemmy
`igfi
`
`i
`
`5' @333 f
`‘sfimiahia
`
`g
`
`Mammy
`lfié
`
`E
`
`{)Emfik
`{sgfiémzai}
`ggfi
`
`..........imwmmw
`
`w
`,,,,,,,,,.,
`
`Pmmgmr
`
`.......................4
`
`infificfiw Changxsa
`$3M
`
`’
`
`Jul. 5, 2012 Sheet 1 0f 21
`
`US 2012/0168240 A1
`
`‘-‘-wwwaw‘uwwwwwwww‘w‘mavmvu‘f‘-
`
`|PR2017-01272
`
`Wait}
`12%
`3
`
`......................*
`
`.....................'c
`
`,
`
`‘ §
`
`‘
`;
`
`Swag?
`W”.-
`33
`
`I:
`g
`
`IPR2017-01272
`Spin Master EX1010 Page 2
`
`

`

`Patent Application Publication
`
`Jul. 5, 2012 Sheet 2 0f 21
`
`US 2012/0168240 A1
`
`magma
`
`V.
`
`..
`
`Cmmuiifig
`{3mm
`
`22%
`
`
`
`5a:
`170 may mimrk
`
`{£343 iimfizamfiai}
`
`{Auésmmia
`{We}
`
`|PR2017-01272
`
`IPR2017-01272
`Spin Master EX1010 Page 3
`
`

`

`Jul. 5, 2012 Sheet 3 0f 21
`
`US 2012/0168240 A1
`
`Patent Application Publication
`
`
`
`|PR2017-01272
`
`IPR2017-01272
`Spin Master EX1010 Page 4
`
`

`

`Jul. 5, 2012 Sheet 4 0f 21
`
`US 2012/0168240 A1
`
`Patent Application Publication
`
`
`
`|PR2017-01272
`
`IPR2017-01272
`Spin Master EX1010 Page 5
`
`

`

`Patent Application Publication
`
`Jul. 5, 2012 Sheet 5 0f 21
`
`US 2012/0168240 A1
`
`43% “wk
`{Wigwam mark} >.
`.-
`
`_
`
`'
`
`'7 We 4%
`"
`{swim :2? mm;
`
`
`
`‘
`,W i
`fififi
`
`{mam s32"
`
`veciar}
`
`ma 4;;
`
`|PR2017-01272
`
`IPR2017-01272
`Spin Master EX1010 Page 6
`
`

`

`Jul. 5, 2012 Sheet 6 0f 21
`
`US 2012/0168240 A1
`
`Patent Application Publication
`
`
`
`mraczéon {3f
`ffimwmem
`
`|PR2017-01272
`
`IPR2017-01272
`Spin Master EX1010 Page 7
`
`

`

`2'
`ammxm ;
`Rmm
`
`;
`
`mammpa
`
`3:31,, ‘2’, .2}
`
`:3“ng
`,
`: acmmayzammm
`
`€33»
`
`r
`“inHkhhhwh‘n‘n‘n‘w‘w‘w‘n‘n‘nflk‘wh‘w‘n‘n‘k
`g
`
`»
`
`$93205;
`was 3mm
`
`mrmwm
`53mm 2‘
`
`110912911anHommddvmema
`
`
`IZJOL199IISz10z‘s1nr
`
`IV01728910/ZIOZSfl
`
`....
`..
`, Mfiishfiféf E? R
`ix, ‘21.} z}
`§§§
`
`Nifikfi‘fiéE“??? £2
`flag}
`
`C593 EEATELEVSTE
`53%"???5.
`
`23mm:
`Pflfimflmmfi
`
`mamas: {9mg}
`RECEWER
`
`{wmm
`
`IPR2017-01272
`Spin Master EX1010 Page 8
`
`

`

`
`
`¥
`H was: '
`7
`ma
`
`N U
`
`)
`5'
`(‘D
`(D4—}
`00
`OW
`—t
`
`Nj
`
`IV01728910/ZIOZSfl
`
`
`
`110912911anHommddvmema
`
`H:.
`
`'_‘
`
`\EII
`
`NOj
`
`—t
`
`.. Cmflw
`Eametem
`332$
`
`fita‘ée Quins:
`
`-
`
`.
`
`.
`
`,
`
`,
`
`23;;4‘
`7"; 1
`
`v
`V
`
`3
`
`3
`
`3’3"}
`
`(Swazi: inieflme 1’26
`
`R?
`
`mm mm
`
`.
`
`Canimi
`
`2mm:
`“3
`
`3P0 Fmgram
`
`§Iaad£2>am§<
`3’38
`
`-
`
`wit-3g}, mg;
`
`Eamme-
`
`‘ffifi
`
`{30mgag?
`
`,
`
`5393:} ’3’“?
`
`HQ. 7
`
`IPR2017-01272
`Spin Master EX1010 Page 9
`
`

`

`USER FQMGWQMS
`
`{ESER {EQQE
`
`
`
`§’.
`
`.,.,.,.,
`
`.,.,.,.,
`
`
`
`:3P:
`
`8&5? FUHCTEIIEM
`STQRRGQ
`
`9% SC}? ifififiifi-Ufififfii
`
`SRQO‘HX
`PRE {3% F 3% E E3
`LGQAL
`{JGNE‘QQL
`£1 1,35% {3‘53
`S
`
`£3 Raflfl 1?:
`PR EBE $3251? ES?
`REM {EYE
`SQW'TfiGL
`§EM$T$QN$
`
`agam‘izx
`
`8Q"?
`
`
`
`110912911anHommddvmema
`
`
`IZJO6199IISz10z‘s1nr
`IV01728910/ZIOZSfl
`
`QRQG‘WX
`
`3kg?
`
`OREG'E'EX R‘fififli’? {22$ H§¥?E RVE-ffiilfi
`
`{EREQWX fiOfiQ‘E" $44M“ 3“ fifi‘gmfi amwm
`
`may;
`5‘gr;
`
`Mi} RX.“
`
`agmaas
`
`magmzw mmmam
`
`EX?A?&$3{}R
`
`£13
`
`IPR2017-01272
`Spin Master EX1010 Page 10
`
`

`

`SWWSLLER
`figi‘PS
`
`m
`
`CQM'E‘RQLLER
`AF?$
`
`
`
`110912911anHommddvmema
`
`
`IZJO01musz10z‘s1nr
`IV01728910/ZIOZSfl
`
`{3 RSQWX. {if} fiTRfiLLER‘
`ngfi
`35%
`
`{ZERBQTIX €3€§fi¥§2€3£LEZ§E
`fiéifi
`
`A93 ? 3&8:
`
`@fififlfix Rpifiiiififififmf E3£X$EE
`2
`
`fifi‘xfifig AW
`
`QSER WWERFfiCE
`
`HARQWAR‘E
`
`IPR2017-01272
`Spin Master EX1010 Page 11
`
`

`

`Patent Application Publication
`
`Jul. 5, 2012 Sheet 11 of 21
`
`US 2012/0168240 A1
`
`{Egvm Mfléififi?
`
`gags: gtwg
`$3;
`
`{MW @3224an
`m
`
`pafigfiism $0M 13%;“sz maxim
`W
`
`
`
`was213mg fiQégfii’ifimfii mam:
`
`|PR2017-01272
`
`IPR2017-01272
`Spin Master EX1010 Page 12
`
`

`

`Jul. 5, 2012 Sheet 12 of 21
`
`US 2012/0168240 A1
`
`fizfiiflfi 3&8
`
`{2mm
`
`382
`
`Patent Application Publication
`
`
`
`mm. 8%35;
`1 h
`.
`
`-------------.- wag; $333156
`
`2% mam 33;;in 823$
`
`|PR2017-01272
`
`IPR2017-01272
`Spin Master EX1010 Page 13
`
`

`

`Jul. 5, 2012 Sheet 13 of 21
`
`US 2012/0168240 A1
`
`Patent Application Publication
`
`
`
`|PR2017-01272
`
`IPR2017-01272
`Spin Master EX1010 Page 14
`
`

`

`Jul. 5, 2012 Sheet 14 of 21
`
`US 2012/0168240 A1
`
`Patent Application Publication
`
`
`
`|PR2017-01272
`
`IPR2017-01272
`Spin Master EX1010 Page 15
`
`

`

`Jul. 5, 2012 Sheet 15 of 21
`
`US 2012/0168240 A1
`
`Patent Application Publication
`
`Bavizze
`
`«Seifwmogmifiw
`
`|PR2017-01272
`
`IPR2017-01272
`Spin Master EX1010 Page 16
`
`

`

`Patent Application Publication
`
`Jul. 5, 2012 Sheet 16 of 21
`
`US 2012/0168240 A1
`
`%§éf»fimmiim mvim Batarmimg iiiirs {Mémfiafiim Mam
`warm? fimma a? Rakmma
`
`tfififi
`
`Saififimméiw fiawimfiigmk wmmiéwflfimfiafim
`iiizfmfimafim
`
`fifi‘i}
`
`3
`
`:
`
`‘
`
`:
`
`
`
`wwwm mnmw me ymgmm ix G-aiiiéhramfi 3°22:
`Rafiefi firimiafim 453$ fie ifwi’mgmiiw fiwiw
`"i 3252‘}
`
`Cammfier {Emmiwzg §W§mm "is: {2mm -S$¥§«Pm§=afiém
`WWW
`
`1 m3
`
`112 D
`
`|PR2017-01272
`
`IPR2017-01272
`Spin Master EX1010 Page 17
`
`

`

`Jul. 5, 2012 Sheet 17 of 21
`
`US 2012/0168240 A1
`
`Patent Application Publication
`
`
`
`|PR2017-01272
`
`IPR2017-01272
`Spin Master EX1010 Page 18
`
`

`

`Patent Application Publication
`
`Jul. 5, 2012 Sheet 18 of 21
`
`US 2012/0168240 A1
`
`I
`
`I
`
`Q3
`
`‘3
`
`‘Wfi’fi
`
`. 3:32 /
`
`‘ if
`
`-
`
`.f.
`V
`’1 3-88
`
`s
`
`|PR2017-01272
`
`IPR2017-01272
`Spin Master EX1010 Page 19
`
`

`

`Jul. 5, 2012 Sheet 19 of 21
`
`US 2012/0168240 A1
`
`Ma “3435
`
`,-.‘<.a.~.w.~f‘
`
`fiafié’mgafi d
`
`Patent Application Publication
`
`
`
`Y
`
`32:32 Wm m
`
`|PR2017-01272
`
`IPR2017-01272
`Spin Master EX1010 Page 20
`
`

`

`Jul. 5, 2012 Sheet 20 of 21
`
`US 2012/0168240 A1
`
`Patent Application Publication
`
`
`
`|PR2017-01272
`
`IPR2017-01272
`Spin Master EX1010 Page 21
`
`

`

`Jul. 5, 2012 Sheet 21 of 21
`
`US 2012/0168240 A1
`
`Patent Application Publication
`
`
`
`|PR2017-01272
`
`IPR2017-01272
`Spin Master EX1010 Page 22
`
`

`

`
`
`In an embodiment, a self-propelled device is pro-
`[0027]
`vided, which includes a dive system, a spherical housing, and
`a biasing mechanism. The drive system includes one or more
`motors that are contained within the spherical housing. The
`biasing mechanism actively forces the drive system to con-
`tinuously engage an interior ofthe spherical housing in order
`to cause the spherical housing to move.
`[0028] According to another embodiment, a self-controlled
`device maintains a frame of reference about an X-, Y— and
`Z-axis. The self-controlled device processes an input to con-
`trol the self-propelled device, the input being based on the X-
`and Y—axis. The self-propelled device is controlled in its
`movement, including about each of the X-, Y— and Z-axes,
`based on the input.
`
`SYSTEM AND METHOD FOR
`CONTROLLING A SELF-PROPELLED
`DEVICE USING A DYNAMICALLY
`CONFIGURABLE INSTRUCTION LIBRARY
`
`RELATED APPLICATIONS
`
`[0001] This application claims priority to (i) U.S. Provi-
`sional Patent Application Ser. No. 61/430,023, entitled
`“Method and System for Controlling a Robotic Device,” filed
`Jan. 5, 201 1; (ii) U.S. Provisional PatentApplication Ser. No.
`61/430,083, entitled “Method and System for Establishing
`2-Way Communication for Controlling a Robotic Device,”
`filed Jan. 5, 2011; and (iii) U.S. Provisional Patent Applica-
`tion Ser. No. 61/553,923, entitled “A Self-propelled Device
`and System and Method for Controlling Same,” filed Oct. 31,
`2011; all of the aforementioned priority applications are
`hereby incorporated by reference in their respective entirety.
`
`FIELD OF THE INVENTION
`
`[0002] Embodiments described herein generally relate to a
`self-propelled device, and a system and method for control-
`ling a self-propelled device using a dynamically configurable
`instruction library.
`
`BACKGROUND
`
`[0003] Early in human history, the wheel was discovered
`and human fascination with circular and spherical objects
`began. Humans were intrigued by devices based on these
`shapes: as practical transportation and propulsion, and as toys
`and amusements. Self-propelled spherical objects were ini-
`tially powered by inertia or mechanical energy storage in
`devices such as coiled springs. As technology has evolved,
`new ways ofapplying and controlling these devices have been
`invented. Today, technology is available from robotics, high
`energy-density battery systems, sophisticated wireless com-
`munication links, micro sensors for magnetism, orientation
`and acceleration, and widely available communication
`devices with displays and multiple sensors for input.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`F G. 1 is a schematic depiction of a self-propelled
`[0004]
`device, according to one or more embodiments.
`[0005]
`F G. 2A is a schematic depiction of an embodiment
`comprising a self-propelled device and a computing device,
`under an embodiment.
`
`F G. 2B depicts a system comprising computing
`[0006]
`devices and self-propelled devices, according to another
`embodiment.
`
`F G. 2C is a schematic that illustrates a system
`[0007]
`comprising a computing device and multiple self-propelled
`devices, under another embodiment.
`[0008]
`F G. 3 is a block diagram illustrating the compo-
`nents of a self-propelled device that is in the form of a robotic,
`spherical ball, in accordance with an embodiment.
`[0009]
`F GS. 4A, 4B, and 4C illustrate a technique for
`causing controlled movement of a spherical self-propelled
`device, in accordance with one or more embodiments.
`[0010]
`F G. 5 further illustrates a technique for causing
`motion of a self-propelled spherical device, according to an
`embodiment.
`
`
`
`F G. 6 is a block diagram depicting a sensor array
`[0011]
`and data flow, according to an embodiment.
`
`US 2012/0168240 A1
`
`Jul. 5, 2012
`
`FIG. 7 illustrates a system including a self-propelled
`[0012]
`device, and a controller computing device that controls and
`interacts with the self-propelled device, according to one or
`more embodiments.
`
`FIG. 8A illustrates a more detailed system architec-
`[0013]
`ture for a self-propelled device and system, according to an
`embodiment.
`
`FIG. 8B illustrates the system architecture ofa com-
`[0014]
`puting device, according to an embodiment.
`[0015]
`FIG. 8C illustrates a particular feature of code
`execution, according to an embodiment.
`[0016]
`FIG. 8D illustrates an embodiment in which a self-
`propelled device 800 implements control using a three-di-
`mensional reference frame and control input that is received
`from another device that utilizes a two-dimensional reference
`
`frame, under an embodiment.
`[0017]
`FIG. 9 illustrates a method for operating a self-
`propelled device using a computing device, according to one
`or more embodiments.
`
`FIG. 10 illustrates a method for operating a comput-
`[0018]
`ing device in controlling a self-propelled device, according to
`one or more embodiments.
`
`FIG. 11A through FIG. 11C illustrate an embodi-
`[0019]
`ment in which a user interface of a controller is oriented to
`
`adopt an orientation of a self-propelled device, according to
`one or more embodiments.
`
`FIG. 11D illustrates a method for calibrating a user-
`[0020]
`interface for orientation based on an orientation of the self-
`
`propelled device, according to an embodiment.
`[0021]
`FIG. 12A and FIG. 12B illustrate different inter-
`faces that can be implemented on a controller computing
`device.
`
`FIG. 13A through FIG. 13C illustrate a variety of
`[0022]
`inputs that can be entered 011 a controller computing device to
`operate a self-propelled device, according to an embodiment.
`[0023]
`FIG. 14A illustrates a system in which a self-pro-
`pelled device is represented in a virtual environment while the
`self-propelled device operates in a real-world environment,
`under an embodiment.
`
`FIG. 14B and FIG. 14C illustrate an application in
`[0024]
`which a self-propelled device acts as a fiducial marker,
`according to an embodiment.
`[0025]
`FIG. 15 illustrates an interactive application that can
`be implemented for use with multiple self-propelled devices,
`depicted as spherical or robotic balls, under an embodiment.
`[0026]
`FIGS. 16A and 16B illustrate a method of collision
`detection, according to an embodiment.
`
`DETAILED DESCRIPTION
`
`
`
`|PR2017-01272
`
`IPR2017-01272
`Spin Master EX1010 Page 23
`
`

`

`US 2012/0168240 A1
`
`Jul. 5, 2012
`
`Still further, another embodiment provides a system
`[0029]
`that includes a controller device and a self-propelled device.
`The self-propelled device is operable to move under control
`of the controller device, and maintains a frame of reference
`about an X-, Y— and Z-axis. The controller device provides an
`interface to enable a user to enter two-dimensional control
`
`input about the X- and Y-axes. The self-propelled device
`processes the control input from the controller device in order
`to maintain control relative to the X-, Y— and Z-axes.
`
`[0030] According to another embodiment, a self-propelled
`device determines an orientation for its movement based on a
`
`pre-determined reference frame. A controller device is oper-
`able by a user to control the self-propelled device. The con-
`troller device includes a user interface for controlling at least
`a direction of movement of the self-propelled device. The
`self-propelled device is configured to signal the controller
`device information that indicates the orientation of the self-
`
`
`
`
`
`propelled device. The controller device is configured to orient
`the user interface, based on the information signaled from the
`self-propelled device, to reflect the orientation of the self-
`propelled device.
`[0031] According to another embodiment, a controller
`device is provided for a self-propelled device. The controller
`device includes one or more processors, a display screen, a
`wireless communication port and a memory. The processor
`operates to generate a user interface for controlling at least a
`directional movement of the self-propelled device, receive
`information from the self-propelled device over the wireless
`communication port indicating an orientation of the self-
`oropelled device, and configure the user interface to reflect
`he orientation of the self-propelled device.
`[0032]
`In still another embodiment, a self-propelled device
`includes a drive system, a wireless communication port, a
`nemory and a processor. The memory stores a first set of
`instructions for mapping individual inputs from a first set of
`‘ecognizable inputs to a corresponding command that con-
`rols movement of the self-propelled device. The processor
`or processors) receive one or more inputs from the controller
`device over the wireless communicationport, map each ofthe
`one or more inputs to a command based on the set of instruc-
`ions, and control the drive system using the command deter-
`nined for each of the one or more inputs. While the drive
`system is controlled, the processor processes one or more
`instructions to alter the set of recognizable inputs and/or the
`corresponding command that is mapped to the individual
`inputs in the set of recognizable inputs.
`[0033]
`Still
`further, embodiments enable a controller
`device to include an object or virtual representation of the
`self-propelled device.
`Terms
`[0034]
`[0035] As used herein, the temi “substantially” means at
`least almost entirely. In quantitative terms, “substantially”
`means at least 80% of a stated reference (e.g., quantity of
`shape).
`In similar regard, “spherical” or “sphere” means
`[0036]
`“substantially spherical.” An object is spherical if it appears
`as such as to an ordinary user, recognizing that, for example,
`manufacturing processes may create tolerances in the shape
`or design where the object is slightly elliptical or not perfectly
`symmetrical, or that the object may include surface features
`or mechanisms for which the exterior is not perfectly smooth
`or symmetrical.
`
`[0037] Overview
`[0038] Referring now to the drawings, FIG. 1 is a schematic
`depiction of a self-propelled device, according to one or more
`embodiments. As described by various embodiments, self-
`propelled device 100 can be operated to move under control
`of another device, such as a computing device operated by a
`user. In some embodiments, self-propelled device 100 is con-
`figured with resources that enable one or more of the follow-
`ing: (i) maintain self-awareness of orientation and/or position
`relative to an initial reference frame after the device initiates
`
`movement; (ii) process control input programmatically, so as
`to enable a diverse range of program-specific responses to
`different control inputs; (iii) enable another device to control
`its movement using software or programming logic that is
`communicative with programming logic on the self-pro-
`pelled device; and/or (iv) generate an output response for its
`movement and state that it is software interpretable by the
`control device.
`
`self-propelled device 100
`In an embodiment,
`[0039]
`includes several interconnected subsystems and modules.
`Processor 114 executes programmatic instructions from pro-
`gram memory 104. The instructions stored in program
`memory 104 can be changed, for example to add features,
`correct flaws, or modify behavior. In some embodiments,
`program memory 104 stores programming instructions that
`are communicative or otherwise operable with software
`executing on a computing device. The processor 114 is con-
`figured to execute different programs of programming
`instructions, in order to after the manner in which the self-
`propelled device 100 interprets or otherwise responds to con-
`trol input from another computing device.
`[0040] Wireless communication 110, in conjunction with
`communication transducer 102, serves to exchange data
`between processor 114 and other external devices. The data
`exchanges, for example, provide communications, provide
`control, provide logical instructions, state information, and/
`or provide updates for program memory 104. In some
`embodiments, processor 114 generates output corresponding
`to state andjor position information, that is communicated to
`the controller device via the wireless communication port.
`The mobility of the device makes wired connections undesir-
`able; the term “connection” should be understood to mean a
`logical connection made without a physical attachment to
`self-propelled device 100.
`[0041]
`In one embodiment, wireless communication 110
`implements the BLUETOOTH communications protocol and
`transducer 602 is an antenna suitable for transmission and
`
`reception of BLUETOOTH radio signals. Other wireless
`communication mediums and protocols may also be used in
`alternative implementations.
`[0042]
`Sensors 112 provide information about the sur-
`rounding enviromnent and condition to processor 114. In one
`embodiment,
`sensors 112 include inertial measurement
`devices, including a 3-axis gyroscope, a 3-axis accelerometer,
`and a 3-axis magnetometer. According to some embodiments,
`the sensors 114 provide input to enable processor 114 to
`maintain awareness of the device’s orientation and/or posi-
`tion relative to the initial reference frame after the device
`
`initiates movement. In various embodiments, sensors 112
`include instruments for detecting light, temperature, humid-
`ity, or measuring chemical concentrations or radioactivity.
`[0043]
`State/variable memory 106 stores information about
`the present state of the system, including, for example, posi-
`tion, orientation, rates ofrotation and translation in each axis.
`
`|PR2017-01272
`
`IPR2017-01272
`Spin Master EX1010 Page 24
`
`

`

`US 2012/0168240 A1
`
`Jul. 5, 20l2
`
`Power 124 stores energy for operating the electron-
`[0051]
`ics and electromechanical components of device 100. In one
`embodiment, power 124 is a rechargeable battery. Inductive
`charge port 128 allows for recharging power 124 without a
`wired electrical connection. In one embodiment, inductive
`charge port 128 accepts magnetic energy and converts it to
`electrical energy to recharge the batteries. In one embodi-
`ment, charge port 128 provides a wireless communication
`interface with an external charging device.
`[0052] Deep sleep sensor 122 puts the self-propelled device
`1 00 into a verylow power or “deep sleep” mode where most of
`the electronic devices use no battery power. This is useful for
`long-term storage or shipping.
`[0053]
`In one embodiment, sensor 122 is non-contact in
`that it senses through the enclosing envelope of device 100
`without a wired connection. In one embodiment, deep sleep
`
`sensor 122 is a Hall Effect sensor mounted so that an external
`
`
`
`magnet can be applied at a pre—determined location on device
`100 to activate deep sleep mode.
`energy into
`[0054] Actuators 126 convert electrical
`mechanical energy for various uses. A primary use of actua-
`tors 126 is to propel and steer self-propelled device 100.
`Movement and steering actuators are also referred to as a
`drive system or traction system. The drive system moves
`device 100 in rotation and translation, under control of pro-
`cessor 114. Examples of actuators 126 include, without limi-
`tation, wheels, motors, solenoids, propellers, paddle wheels
`and pendulurns.
`[0055]
`In one embodiment, drive system actuators 126
`include two parallel wheels, each mounted to an axle con-
`nected to an independently variable-speed motor through a
`reduction gear system. In such an embodiment, the speeds of
`the two drive motors are controlled by processor 114.
`[0056] However,
`it should be appreciated that actuators
`126, in various embodiments, produce a variety of move-
`ments in addition to merely rotating and translating device
`100. In one embodiment, actuators 126 cause device 100 to
`execute communicative or emotionally evocative move-
`ments, including emulation of human gestures, for example,
`head nodding, shaking, trembling, spinning or flipping. In
`some embodiments, processor coordinates actuators 126 with
`display 118. For example, in one embodiment, processor 114
`provides signals to actuators 126 and display 118 to cause
`device 100 to spin or tremble and simultaneously emit pat-
`terns of colored light. In one embodiment, device 100 emits
`light or sound patterns synchronized with movements.
`[0057]
`In one embodiment, self-propelled device 100 is
`used as a controller for other network-connected devices.
`Device 100 contains sensors and wireless communication
`
`The state/variable memory 106 also stores information cor-
`responding to an initial reference frame of the device upon,
`for example, the devicebeing put inuse (e.g., the device being
`switched on), as well as position and orientation information
`once the device is in use. In this way, some embodiments
`provide for the device 100 to utilize information of the state/
`variable memory 106 in order to maintain position and ori-
`entation information of the device 100 once the device starts
`
`moving.
`[0044] Clock 108 provides timing information to processor
`114. In one embodiment, clock 108 provides a timebase for
`measuring intervals and rates of change. In another embodi-
`ment, clock 108 provides day, date, year, time, and alarm
`functions. In one embodiment clock 108 allows device 100 to
`
`provide an alarm or alert at pre-set times.
`[0045] Expansion port 120 provides a connection for addi-
`
`tion of accessories or devices. Expansion port 120 provides
`for future expansion, as well as flexibility to add options or
`enhancements. For example, expansion port 120 is used to
`add peripherals, sensors, processing hardware, storage, dis-
`plays, or actuators to the basic self-propelled device 100.
`[0046]
`In one embodiment, expansion port 120 provides an
`interface capable of communicating with a suitably config-
`ured component using analog or digital signals. In various
`embodiments, expansion port 120 provides electrical inter-
`faces and protocols that are standard or well-known. In one
`embodiment, expansion port 120 implements an optical inter-
`
`face. Exemplary interfaces appropriate for expansion port
`120 include the Universal Serial Bus (USB), Inter-Integrated
`Circuit Bus (12C), Serial Peripheral Interface (SP1), or ETH-
`ERNET.
`
`[0047] Display 118 presents information to outside devices
`or persons. Display 118 can present information in a variety
`of forms. In various embodiments, display 118 can produce
`light in colors and patterns, sound, vibration, music, or com-
`binations of sensory stimuli. In one embodiment, display 118
`operates in conjunction with actuators 126 to communicate
`information by physical movements of device 100. For
`example, device 100 can be made to emulate a human head
`nod or shake to communicate “yes” or “no .”
`[0048]
`In one embodiment, display 118 is an emitter of
`light, either in the visible or invisible range. Invisible light in
`the infrared or ultraviolet range is useful, for example to send
`information invisible to human senses but available to spe-
`cialized detectors. In one embodiment, display 118 includes
`an array of Light Emitting Diodes (LEDs) emitting various
`light frequencies, arranged such that their relative intensity is
`variable and the light emitted is blended to form color mix-
`tures.
`
`In one embodiment, display 118 includes an LED
`[0049]
`array comprising several LEDs, each emitting a human-vis-
`ible primary color. Processor 114 varies the relative intensity
`
`of each of the LEDs to produce a wide range of colors.
`Primary colors of light are those wherein a few colors can be
`blended in different amounts to produce a wide gamut of
`apparent colors. Many sets of primary colors of light are
`known,
`including for example red/green/blue, red/green/
`blue/white, and red/green/blue/amber. For example, red,
`green and blue LEDs together comprise a usable set of three
`available primary-color devices comprising a display 118 in
`one embodiment. In other embodiments, other sets ofprimary
`colors and white LEDs are used.
`
`In one embodiment, display 118 includes an LED
`[0050]
`used to indicate areferencepoint on device 100 for alignment.
`
`capability, and so it can perform a controller role for other
`devices. For example, self-propelled device 100 can be held
`in the hand and used to sense gestures, movements, rotations,
`combination inputs and the like.
`[0058]
`FIG. 2A is a schematic depiction of an embodiment
`comprising a self-propelled device and a computing device,
`under an embodiment. More specifically, a self-propelled
`device 214 is controlled in its movement by programming
`logic and/or controls that can originate from a controller
`device 208. The self-propelled device 214 is capable ofmove-
`ment under control of the computing device 208, which can
`be operated by a user 202. The computing device 208 can
`wirelessly communicate control data to the self-propelled
`device 214 using a standard or proprietary wireless commu-
`nication protocol. In variations, the self-propelled device 214
`
`|PR2017-01272
`
`IPR2017-01272
`Spin Master EX1010 Page 25
`
`

`

`US 2012/0168240 A1
`
`Jul. 5, 2012
`
`fers data from device 208 to device 214. Link 212 transfers
`data from device 214 to device 208. Links 210 and 212 are
`
`shown as separate unidirectional links for illustration; in
`some embodiments a single bi-directional communication
`link performs communication in both directions. It should be
`appreciated that link 210 and link 212 are not necessarily
`identical in type, bandwidth or capability. For example, com-
`munication link 210 from computing device 208 to elf-pro-
`pelled device 214 is often capable of a higher communication
`rate and bandwidth compared to link 212. In some situations,
`only one link 210 or 212 is established. In such an embodi-
`ment, communication is unidirectional.
`[0065] The computing device 208 can correspond to any
`device comprising at least a processor and communication
`capability suitable for establishing at least uni-directional
`communications with self-propelled device 214. Examples of
`such devices include, without limitation: mobile computing
`devices (e.g., multifunctional messaging/voice communica-
`tion devices such as smart phones), tablet computers, portable
`communication devices and personal computers. In one
`embodiment, device 208 is an IPHONE available from
`APPLE COMPUTER, INC. of Cupertino, Calif. In another
`embodiment, device 208 is an IPAD tablet computer, also
`from APPLE COMPUTER. In another embodiment, device
`208 is any of the handheld computing and communication
`appliances executing the ANDROID operating system from
`
`GOOGLE, INC.
`[0066]
`In another embodiment, device 208 is a personal
`computer, in either a laptop or desktop configuration. For
`example, device 208 is a mufti-purpose computing platform
`running the MICROSOFT WINDOWS operating system, or
`the LINUX operating system, or the APPLE OS/X operating
`system, configured with an appropriate application program
`to communicate with self-propelled device 214.
`[0067]
`In variations, the computing device 208 can be a
`specialized device, dedicated for enabling the user 202 to
`control and interact with the self-propelled device 214.
`[0068]
`In one embodiment, multiple types of computing
`device 208 can be used interchangeably to communicate with
`the self-propelled device 214. In one embodiment, self-pro-
`pelled device 214 is capable of communicating and/or being
`controlled by multiple devices (e. g., concurrently or one at a
`time). For example, device 214 can link with an IPHONE in
`one session and with anANDROID device in a later session,
`without modification of device 214.
`
`
`
`may be at least partially self-controlled, utilizing sensors and
`internal programming logic to control the parameters of its
`movement (e.g., velocity, direction, etc.). Still further, the
`self-propelled device 214 can communicate data relating to
`the device’s position and/or movement parameters for the
`purpose ofgenerating or alternating content on the computing
`device 208. In additional variations, self-propelled device
`214 can control aspects of the computing device 208 by way
`of its movements andlor internal programming logic.
`[0059] As described herein, the self-propelled device 214
`may have multiple modes of operation, including those of
`operation in which the device is controlled by the computing
`device 208, is a controller for another device (e.g., another
`self-propelled device or the computing device 208), and/or is
`partially or wholly self-autonomous.
`[0060] Additionally, embodiments enable the self-pro-
`pelled device 214 and the computing device 208 to share a
`computing platform on which programming logic is shared,
`in order to enable, among other features, functionality that
`includes: (i) enabling the user 202 to operate the computing
`device 208 to generate multiple kinds of input, including
`simple directional
`input, command input, gesture input,
`motion or other sensory input, voice input or combinations
`thereof; (ii) enabling the self-propelled device 214 to interpret
`input received from the computing device 208 as a command
`or set of commands; and/0r (iii) enabling the self-propelled
`device 214 to communicate data regarding that device’s posi-
`tion, movement and/or state in order to effect a state on the
`computing device 208 (e.g., display state, such as content
`corresponding to a controller-user interface). Embodiments
`further provide that the self-propelled device 214 includes a
`programmatic interface that facilitates additional program-
`ming logic and/or instructions to use the device. The comput-
`ing device 208 can execute programming that is communica-
`tive with the programming logic on the self-propelled device
`214.
`
`the self-propelled
`[0061] According to embodiments,
`device 214 includes an actuator or drive mechanism causing
`motion or directional movement. The self-propelled device
`214 may be referred to by a number of related terms and
`phrases, including controlled device, robot, robotic device,
`remote device, autonomous device, and remote-controlled
`device. In some embodiments, the self-propelled devic