Case 1:22-cv-01257-GBW Document 1 Filed 09/22/22 Page 1 of 62 PageID #: 1
`
`IN THE UNITED STATES DISTRICT COURT
`
`FOR THE DISTRICT OF DELAWARE
`
`Plaintiff,
`
`Defendant.
`
`
`
`)
`)
`)
`)
`) C.A. No. ________
`)
`) DEMAND FOR JURY TRIAL
`)
`)
`
`
`
`
`COMPLAINT
`
`
`
`
`ABB INC.,
`
`
`
`
`
`
`ROBOTICVISIONTECH, INC.,
`
`
`
`
`v.
`
`
`
`Plaintiff RoboticVISIONTech, Inc. (RVT), by its attorneys, demands a trial by jury on all
`
`issues so triable and, for its complaint against ABB Inc. (ABB), alleges as follows:
`
`NATURE OF THIS ACTION
`
`1.
`
`This is a civil action arising out of ABB’s infringement of RVT’s patents in
`
`violation of 35 U.S.C. §§ 271 et seq.; ABB’s infringement of RVT’s copyrighted works in
`
`violation of 17 U.S.C. §§ 101 et seq.; and ABB’s misappropriation of RVT’s trade secrets in
`
`violation of the Defend Trade Secrets Act (DTSA), 18 U.S.C. §§ 1836 et seq. and the Delaware
`
`Uniform Trade Secrets Act, 6 Del. C. §§ 2001 et seq.
`
`PARTIES
`
`2.
`
`Plaintiff RVT is a privately held corporation organized and existing under the
`
`laws of Delaware with a principal place of business at 1775 Tysons Boulevard, Fifth Floor, Suite
`
`500, McLean, Virginia 22102.
`
`3.
`
`On information and belief, Defendant ABB is a company organized and existing
`
`under the laws of the Delaware with a principal place of business at 305 Gregson Drive, Cary,
`
`North Carolina 27511.
`
`1
`
`

`

`Case 1:22-cv-01257-GBW Document 1 Filed 09/22/22 Page 2 of 62 PageID #: 2
`
`JURISDICTION AND VENUE
`
`4.
`
`This action arises under the patent laws of the United States, 35 U.S.C. §§ 100 et
`
`seq., and this Court has subject-matter jurisdiction over RVT’s patent-infringement claims under
`
`28 U.S.C. §§ 1331 and 1338(a).
`
`5.
`
`This action arises under the United States Copyright Act of 1976, as amended, 17
`
`U.S.C. § 101 et seq., and this Court has subject-matter jurisdiction over RVT’s copyright-
`
`infringement claims under 28 U.S.C. §§ 1331 and 1338(a).
`
`6.
`
`This action arises under the Defend Trade Secrets Act of 2016, 18 U.S.C. §§ 1836
`
`et seq., and this Court has subject-matter jurisdiction over RVT’s trade-secret claims under 28
`
`U.S.C. § 1331.
`
`7.
`
`This Court has supplemental jurisdiction over RVT’s claims arising under the
`
`Delaware Uniform Trade Secrets Act because these state-law claims are so related to RVT’s
`
`federal-law claims that they form part of the same case or controversy and derive from a
`
`common nucleus of operative fact.
`
`8.
`
`On information and belief, this Court has personal jurisdiction over ABB at least
`
`because ABB is a Delaware corporation and has registered to do business in the State of
`
`Delaware.
`
`9.
`
`Venue is properly laid in this District pursuant to 28 U.S.C. §§ 1391 and 1400 at
`
`least because, on information and belief, ABB is subject to personal jurisdiction in this District
`
`and is a resident and corporate citizen of this District.
`
`BACKGROUND FACTS
`
`RVT’S AND ABB’S BUSINESS DEALINGS
`
`Braintech Canada, Inc. (Braintech), RVT’s predecessor-in-interest, authored the
`
`A.
`
`10.
`
`original source code contained in the eVisionFactory (eVF) software product, which is the
`
`2
`
`

`

`Case 1:22-cv-01257-GBW Document 1 Filed 09/22/22 Page 3 of 62 PageID #: 3
`
`commercial embodiment of the patents-in-suit and employs the asserted trade secrets. Braintech
`
`was the original assignee of the three patent applications resulting in each of U.S. Patent Nos.
`
`6,816,755 (Exhibit 1); 7,336,814 (Exhibit 2); and 8,095,237 (Exhibit 3), which have all been
`
`assigned to RVT.
`
`11.
`
`In May 2006, Braintech entered into an Exclusive Channel Partnership
`
`Agreement with ABB.
`
`12.
`
`As part of this agreement, ABB purchased licenses from Braintech to market and
`
`sell Braintech’s eVF software under the brand name “TrueView.” On information and belief,
`
`ABB marketed and sold more than 167 TrueView units from 2006 to 2008, many of which were
`
`sold to the world’s leading automotive manufacturing plants. On information and belief, ABB
`
`sold additional TrueView units after 2008.
`
`13.
`
`In May 2010, RVT purchased all of Braintech’s assets, including Braintech’s eVF
`
`software product, the source code for eVF, the patents-in-suit, and any copyrights and trade
`
`secrets within Braintech’s intellectual property portfolio. Since acquiring the Braintech assets in
`
`2010, RVT has focused on optimizing, improving, selling, and distributing its robotic vision
`
`software products, including its eVF software product.
`
`14.
`
`Under the explicit terms of the Exclusive Channel Partnership Agreement, ABB’s
`
`right to market and sell TrueView products terminated once Braintech ceased operations in May
`
`2010.
`
`15.
`
`In July 2010, after ABB’s right to market and sell TrueView products had
`
`expired, ABB sued RVT in the Eastern District of Michigan, claiming that ABB, not RVT, was
`
`the sole owner of the source code for eVF. ABB alleged that its payments to Braintech for the
`
`right to market and sell eVF under the Exclusive Channel Partnership Agreement were an
`
`3
`
`

`

`Case 1:22-cv-01257-GBW Document 1 Filed 09/22/22 Page 4 of 62 PageID #: 4
`
`investment, not payments for a license, and that Braintech used that investment to develop eVF.
`
`See ABB, Inc. v. Robotic VisionTech, LLC, No. 5:10-cv-012626-JCO-PJK, ECF No. 1 at 10–11
`
`(E.D. Mich. July 1, 2010) (“ABB Compl”); see also id., ECF No. 16 (E.D. Mich. Aug. 17,
`
`2010); id., ECF No. 16-1 (E.D. Mich. Aug. 17, 2010). ABB’s complaint sought a “judgment in
`
`ABB’s favor awarding it ownership of the code and executables under a theory of constructive
`
`and/or equitable trust.” ABB Compl. at 16. On information and belief, ABB’s lawsuit in Eastern
`
`District of Michigan was an attempt to coerce RVT into relinquishing ownership and control
`
`over the eVF source code and software product.
`
`16.
`
`ABB voluntarily dismissed its lawsuit against RVT in September 2010. See ABB,
`
`Inc. v. Robotic VisionTech, LLC, No. 5:10-cv-012626-JCO-PJK, ECF No. 21 (E.D. Mich. Sept.
`
`16, 2010). As part of the settlement between ABB and RVT, ABB purchased 41 developer keys
`
`to RVT’s eVF software and provided RVT with two ABB industrial robots. ABB also agreed to
`
`pay RVT’s attorneys’ fees up to $25,000. RVT did not provide ABB with the source code for
`
`RVT’s eVF software product.
`
`17.
`
`On information and belief, while ABB and RVT were in the process of
`
`negotiating their settlement, ABB—without RVT’s knowledge—negotiated an employment
`
`contract with Dr. Remus Boca, RVT’s Chief Scientist.
`
`18.
`
`Dr. Boca began employment with Braintech on or around November 30, 2001. By
`
`2008, Dr. Boca was promoted to Braintech’s Chief Scientist. He continued as Chief Scientist
`
`when he started working for RVT after it acquired Braintech in May 2010. Dr. Boca was largely
`
`responsible for developing the eVF source code, including its roadmap of features, during his
`
`time with both Braintech and RVT.
`
`4
`
`

`

`Case 1:22-cv-01257-GBW Document 1 Filed 09/22/22 Page 5 of 62 PageID #: 5
`
`19.
`
`Dr. Boca was subject to a Non-Disclosure Agreement (NDA) at all times during
`
`his employment at Braintech and RVT. Dr. Boca’s original employment agreement with
`
`Braintech explicitly stated that any and all intellectual property conceived during his
`
`employment, including, for example, trade secrets, know-how, show-how, inventions, concepts,
`
`ideas, improvements, patents, and copyrights, were expressly regarded as works for hire and
`
`belonged to Braintech.
`
`20. While at Braintech, Dr. Boca was one of the main architects of the eVF software
`
`product. During his employment with both Braintech and RVT, Dr. Boca had direct access to, or
`
`was in possession of, the source code underlying the eVF software product. Dr. Boca is also a
`
`named co-inventor of two of the three patents-in-suit.
`
`21.
`
`At all times while at Braintech and RVT, Dr. Boca had full and unfettered access
`
`to the source code of eVF. When Dr. Boca left RVT and joined ABB in October 2010, he was in
`
`possession of two RVT-issued laptops and two RVT-issued external hard drives, which
`
`contained RVT’s confidential and proprietary information, including the source code for RVT’s
`
`eVF product. On information and belief, these laptops and hard drives contained at least versions
`
`5.5 and 6.0 of the eVF source code. RVT’s company policy required employees to return work-
`
`issued laptops and hard drives upon leaving the employ of RVT.
`
`22.
`
`On multiple occasions, RVT requested that Dr. Boca immediately return the two
`
`RVT-issued laptops to RVT. Dr. Boca did not respond until late December 2010, more than two
`
`months after starting his new position at ABB. When RVT finally received the RVT-issued
`
`laptops and external hard drives in Dr. Boca’s possession, all information had been deleted from
`
`them.
`
`5
`
`

`

`Case 1:22-cv-01257-GBW Document 1 Filed 09/22/22 Page 6 of 62 PageID #: 6
`
`23.
`
`In October 2010, ABB’s Manager of Business Development in its Robot
`
`Automation Systems Group presented to RVT a plan for collaboration with ABB and requested
`
`that RVT provide pricing information for the eVF software product.
`
`24.
`
`In January 2011, in response to the October 2010 meeting, RVT sent a letter to
`
`ABB’s Vice President of Automation Systems with a proposal including exclusive discount
`
`pricing based upon unit volume purchases by ABB of the eVF software. The proposal included
`
`the purchase of more than 51 eVF 6.0 software upgrade licenses for “a period of one year from
`
`February 28, 2011.” ABB did not respond to RVT’s proposal. Further, ABB chose not to
`
`purchase the eVF software.
`
`25.
`
`In October 2012, the Vice-President & General Manager North America of
`
`ABB’s Discrete Automation and Motion Group requested updated information on eVF 6.0 and
`
`expressed interest in purchasing multiple units of the eVF software. RVT responded to his
`
`request in an email dated October 22, 2012, containing the updated information on eVF 6.0 and
`
`the requested comparison of eVF 6.0 with eVF 5.0 and the commercially available Cognex
`
`machine-vision library. Also attached to the email was a copy of eVF’s 6.0 general information
`
`brochure, which stated that eVF was “patent protected.”
`
`26.
`
`In September 2013, a Vice President in ABB’s Robot Automation Systems Group
`
`approached RVT, expressing interest in purchasing licenses to market and sell RVT’s eVF
`
`software.
`
`27.
`
`Due in large part to ABB’s previous history of purchasing eVF software licenses
`
`from Braintech and ABB’s professed continued interest in purchasing the eVF software, RVT
`
`agreed to meet with ABB to discuss a potential licensing agreement. The meeting took place at
`
`RVT’s Bloomfield Hills, Michigan office and robotic lab on October 10, 2013. ABB’s principal
`
`6
`
`

`

`Case 1:22-cv-01257-GBW Document 1 Filed 09/22/22 Page 7 of 62 PageID #: 7
`
`vision engineer, three ABB scientists, RVT’s Sales & Business Development Director, and
`
`RVT’s Chief Scientist all attended the October 10, 2013 meeting.
`
`28.
`
`After the October 2013 meeting, RVT did not hear back from ABB until January
`
`2014. ABB reiterated its promise to purchase RVT’s eVF software product but had certain
`
`requests regarding RVT’s eVF software. For example, on or around January 29, 2014, ABB
`
`requested RVT’s help to test eVF 6.0—the newest version of eVF at the time—on ABB robots at
`
`their research facility in Auburn Hills, Michigan. RVT installed eVF 6.0 at ABB’s request, with
`
`ABB’s assurances that the install was for testing purposes only.
`
`29.
`
`In the spirit of fostering licensing negotiations, RVT diligently complied with
`
`each of ABB’s requests. Yet ABB chose not to license RVT’s eVF software. Instead, ABB
`
`continued to ask for more details about RVT’s eVF software, and ABB’s methods for obtaining
`
`this information became increasingly aggressive.
`
`30.
`
`In April 2014, for example, rather than go through the appropriate channels of
`
`communication, Louis LePage, ABB’s principal vision engineer, repeatedly asked one of RVT’s
`
`Michigan engineers to send ABB the latest software build for eVF.
`
`31.
`
`In July 2014, without RVT management’s knowledge or consent, ABB requested
`
`another meeting with RVT’s new Chief Scientist and its engineers. ABB asked the RVT
`
`employees to come to ABB’s Auburn Hills, Michigan office to share details about RVT’s new
`
`software interface for eVF.
`
`32.
`
`In August 2014, ABB requested that RVT provide updated pricing for eVF. Then,
`
`in June 2015, the Project Manager in ABB’s Discrete Automation and Motion Division
`
`requested a meeting to obtain the current pricing for eVF’s Runtime and Developer licenses. The
`
`meeting was held in ABB’s Auburn Hills, Michigan office with high-level representatives of
`
`7
`
`

`

`Case 1:22-cv-01257-GBW Document 1 Filed 09/22/22 Page 8 of 62 PageID #: 8
`
`ABB’s Robotics Business Unit and two RVT engineers and RVT’s chief scientist. Once again,
`
`however, ABB chose not to license RVT’s eVF software.
`
`33.
`
`On information and belief, ABB had no intention of purchasing or licensing
`
`RVT’s eVF software product, despite promising to purchase licenses to market and sell the
`
`product (as ABB had done in the past with RVT’s predecessor, Braintech). On information and
`
`belief, ABB’s communications regarding licensing the eVF software were designed to obtain
`
`information regarding eVF’s latest software builds and user interface for the purpose of
`
`integrating these functions and features into ABB’s own competing product, FlexVision 3D.
`
`34.
`
`35.
`
`ABB does not currently have a license to the eVF technology.
`
`On information and belief, ABB launched its competing FlexVision product in
`
`2015. However, information about commercial robotic vision technology is usually not publicly
`
`disseminated. Specifically, there is little or no information available to the general public about
`
`FlexVision’s interface, operations, and capabilities. On information and belief, such information
`
`is available only to purchasers of that technology.
`
`36.
`
`It was not until late 2020 that mutual customers of RVT and ABB informed RVT
`
`that the FlexVision software was very similar to RVT’s eVF product. RVT, however, could not
`
`confirm the veracity of these reports because there was no publicly available technical
`
`information about ABB’s FlexVision product that would have allowed RVT to do so.
`
`37.
`
`In June 2021, an “integrator” (i.e., a services firm responsible for integrating
`
`machine technology and software from multiple providers, including robot hardware, camera
`
`hardware, and machine-vision software) provided RVT with a copy of the 2016 version of the
`
`FlexVision 3D User Manual (“User Manual”). A copy of the User Manual is attached as Exhibit
`
`4. The FlexVision 3D User Manual was not otherwise available to RVT, as such manuals are
`
`8
`
`

`

`Case 1:22-cv-01257-GBW Document 1 Filed 09/22/22 Page 9 of 62 PageID #: 9
`
`provided only to customers who purchase the FlexVision product. Only after reviewing the
`
`information in the User Manual and comparing it to eVF’s source code could RVT confirm that
`
`FlexVision operates in the same or similar way as RVT’s eVF product, and similarly, that
`
`FlexVision’s interface and features are the same as or similar to RVT’s eVF interface and
`
`features. Up until this review of the User Manual, RVT had regarded ABB as a former and
`
`potential future customer of its products and treated ABB accordingly.
`
`B.
`
`38.
`
`RVT’S eVF SOFTWARE
`
`RVT is the owner of copyrighted three-dimensional vision software known as
`
`eVisionFactory. The eVF source code and programming allows a robot to “SEE, THINK, &
`
`DO” and operate in a three-dimensional space based only on two-dimensional imaging. The
`
`software enables three-dimensional object location such that a robotic arm has the capability to
`
`choose, pick up, guide, and manipulate components in various manufacturing processes with
`
`high accuracy, speed, and consistency. Indeed, eVF can locate a target part in under a tenth of a
`
`second using just one image from a single camera. The eVF software is so reliable that more than
`
`300 installations have run the eVF software for over the past two decades without a single
`
`warranty claim.
`
`39.
`
`Industry leaders have long recognized the eVF software’s unmatched
`
`performance and accuracy. In 2003, eVF won the Ford Motor Company’s distinguished Henry
`
`Ford Technology Award.1 Additionally, in a recent study conducted by one of the world’s largest
`
`and most technologically advanced automakers, eVF performed 10 to 100 times more accurately
`
`
`1 Braintech Wins Distinguished Henry Ford Technology Award, HPC Wire (Oct. 3, 2003),
`https://www.hpcwire.com/2003/10/03/braintech-wins-distinguished-henry-ford-technology-
`award (last visited Aug. 5, 2022).
`
`9
`
`

`

`Case 1:22-cv-01257-GBW Document 1 Filed 09/22/22 Page 10 of 62 PageID #: 10
`
`than its competitors, leading the automaker to select eVF for use in its manufacture of powertrain
`
`and transmission systems for its best-selling brand of automobiles. See Exhibit 5 at 1.
`
`40.
`
`The eVF software performs three core processes necessary for determining the
`
`three-dimensional positions of an object: camera calibration, object training, and pose estimation.
`
`41.
`
`eVF performs camera calibration via an automatic software tool called
`
`“AutoCal.” In robot-mounted camera configurations, this tool is a one-button solution that
`
`automatically moves the camera around a calibration grid. It then calculates both the intrinsic
`
`and extrinsic properties of the camera. Intrinsic properties include, e.g., the size of the pixels, the
`
`pixel count, the distortion or skew of the image, and the resolution of the image. Extrinsic
`
`properties include, e.g., the location of the camera in world or robot coordinates on the end of the
`
`robot arm, or in a static 3D space (in cases of a stationary mounted camera). The following
`
`image is an example of a calibration grid that the eVF software uses for camera calibration:
`
`Figure 1: Example of a calibration grid
`
`
`
`10
`
`

`

`Case 1:22-cv-01257-GBW Document 1 Filed 09/22/22 Page 11 of 62 PageID #: 11
`
`42.
`
`After the camera has been calibrated, eVF’s “AutoTrain” tool is used to “train”
`
`the object in a nominal position in which eVF will be locating the part. First, eVF takes a
`
`reference image of the object. The user operating eVF then defines, in the reference image,
`
`patterns on the part for eVF to identify when it takes a picture during operation. The user selects
`
`a number of features for the software to calculate three-dimensional information.
`
`43.
`
`There are two categories of features: “anchor” features and “GeoPatterns.” An
`
`anchor feature is a unique pattern defined by the object. GeoPatterns are smaller, non-unique, but
`
`predefined patterns that are found in predetermined locations with respect to the anchor feature.
`
`The relationship between the patterns automatically trains the software to calculate the object’s
`
`three-dimensional position and orientation. The following image is an example of a reference
`
`image with patterns defined on the object:
`
`Figure 2: Example of a reference image
`
`
`
`11
`
`

`

`Case 1:22-cv-01257-GBW Document 1 Filed 09/22/22 Page 12 of 62 PageID #: 12
`
`44.
`
`After the object has been trained, eVF performs pose estimation. The software
`
`takes an image of the object in the training space, uses the camera-calibration data, combines it
`
`with the data from the object-training process, locates the object in three-dimensional space, and
`
`calculates the deviations with respect to the originally trained location. The pose-estimation
`
`information guides the robot in locating the object and performing various operations, such as
`
`handling or manipulating the object. The image below is an example of eVF calculating an
`
`object’s position.
`
`
`
`Figure 3: Example of eVF software output calculating object’s position
`
`45.
`
`RVT’s success and superior performance are by-products of technological
`
`innovations over the past two decades, including, for example, the eVF software. RVT continues
`
`to implement these innovations today, for example, by continuing to improve eVF and releasing
`
`new versions of the software.
`
`C.
`
`46.
`
`RVT’S COPYRIGHTED SOURCE CODE
`
`In the early 2000s, Braintech created and authored the initial version of the eVF
`
`source code, drafting the source code and implementing it through the first iteration of the eVF
`
`12
`
`

`

`Case 1:22-cv-01257-GBW Document 1 Filed 09/22/22 Page 13 of 62 PageID #: 13
`
`product. The eVF source code, while incorporating snippets of open-source code and licensed
`
`code libraries, is otherwise an original, creative work and is wholly original in its arrangement
`
`and architecture.
`
`47.
`
`RVT purchased Braintech’s intellectual property portfolio, including all
`
`copyrights and copyright rights, applications, and registrations, in May 2010. Therefore, RVT
`
`owns all copyright rights in the eVF source code.
`
`48.
`
`The source code for RVT’s eVF technology is an original work fixed in a tangible
`
`medium containing copyrightable subject matter for which copyright protection exists under the
`
`Copyright Act, 17 U.S.C. §§ 101 et. seq. RVT is the exclusive owner of the eVF source code and
`
`owns a valid and subsisting United States Copyright Registration No. TX 9-169-843 for the eVF
`
`source code, issued by the United States Copyright Office on September 16, 2022, a copy of
`
`which is attached as Exhibit 6.
`
`49.
`
`At no time has RVT or Braintech granted permission to any party to copy,
`
`modify, or distribute the eVF source code.
`
`D.
`
`50.
`
`THE PATENTS-IN-SUIT
`
`U.S. Patent No. 6,816,755 (the ’755 patent), entitled “Method and Apparatus for
`
`Single Camera 3D Vision Guided Robotics,” was duly and legally issued by the United States
`
`Patent and Trademark Office (PTO) on November 9, 2004. A true and correct copy of the ’755
`
`patent is attached as Exhibit 1.
`
`51.
`
`RVT is the sole owner of the entire right, title, and interest in the ’755 patent,
`
`including the right to bring suit and recover damages for past infringement. The ’755 patent was
`
`originally assigned to Braintech Canada, Inc. The patent was subsequently assigned from
`
`Braintech Canada, Inc. to Braintech, Inc. on February 20, 2009; from Braintech, Inc. to
`
`13
`
`

`

`Case 1:22-cv-01257-GBW Document 1 Filed 09/22/22 Page 14 of 62 PageID #: 14
`
`RoboticVISIONTech LLC on May 24, 2010; and from RoboticVISIONTech LLC to
`
`RoboticVISIONTech, Inc. on July 27, 2015.
`
`52.
`
`U.S. Patent No. 7,336,814 (the ’814 patent), entitled “Method and Apparatus for
`
`Machine-Vision,” was duly and legally issued by the PTO on February 26, 2008. A true and
`
`correct copy of the ’814 patent is attached as Exhibit 2.
`
`53.
`
`RVT is the sole owner of the entire right, title, and interest in the ’814 patent,
`
`including the right to bring suit and recover damages for past infringement. The ’814 patent was
`
`originally assigned to Braintech Canada, Inc. The patent was subsequently assigned from
`
`Braintech Canada, Inc. to Braintech, Inc. on February 20, 2009; from Braintech, Inc. to
`
`RoboticVISIONTech LLC on May 24, 2010; and from RoboticVISIONTech LLC to
`
`RoboticVISIONTech, Inc. on July 27, 2015.
`
`54.
`
`U.S. Patent No. 8,095,237 (the ’237 patent), entitled “Method and Apparatus for
`
`Single Image 3D Vision Guided Robotics,” was duly and legally issued by the PTO on January
`
`10, 2012. A true and correct copy of the ’237 patent is attached as Exhibit 3. The ’237 patent is a
`
`continuation-in-part of application No. 10/153,680, filed on May 24, 2002, now the ’755 patent.
`
`55.
`
`RVT is the sole owner of the entire right, title, and interest in the ’237 patent,
`
`including the right to bring suit and recover damages for past infringement. The ’237 patent was
`
`originally assigned to Braintech Canada, Inc. in 2005. The patent was subsequently assigned
`
`from Braintech Canada, Inc. to Braintech, Inc. on February 20, 2009; from Braintech, Inc. to
`
`RoboticVISIONTech LLC on May 24, 2010; and from RoboticVISIONTech LLC to
`
`RoboticVISIONTech, Inc. on July 27, 2015.
`
`56.
`
`The ’755, ’814, and ’237 patents (collectively, “the Asserted Patents”) are
`
`generally directed to methods and features that have been incorporated into RVT’s eVF software.
`
`14
`
`

`

`Case 1:22-cv-01257-GBW Document 1 Filed 09/22/22 Page 15 of 62 PageID #: 15
`
`These patented features have contributed to the success of eVF in the United States and have
`
`allowed RVT to establish itself as a market leader in the machine vision robotics industry. All
`
`three patent numbers have been marked on eVF’s splash screen upon startup of all relevant
`
`versions of the software.
`
`E.
`
`57.
`
`RVT’S TRADE-SECRET METHODS
`
`In addition to its patented technology and copyrighted source code, RVT employs
`
`various trade-secret methods and algorithms through confidential portions of its eVF source
`
`code. These trade secrets facilitate the precision, reproducibility, and performance of the
`
`machine vision robotics capabilities of eVF. Examples of such trade secrets include, but are not
`
`limited to, (i) feature qualification; (ii) reprojection; (iii) inverse projection; (iv) use of multiple
`
`pose-calculation methods to minimize feature count; and (v) determination and use of the so-
`
`called “golden position” in creating the 3D model.
`
`58.
`
`Feature qualification. eVF uses trade-secret methods to qualify the features used
`
`during pose estimation. These methods have certain thresholds, some of which are set by the
`
`user, and eVF runs statistical calculations against these thresholds to determine whether any
`
`underperforming features should not be used during pose calculation at runtime. eVF will also
`
`discard at runtime features that it detects are not co-planar, to an optimized degree, for resiliency
`
`against optical skew. These methods have benefits over known methods by reducing noise, skew,
`
`and errors resulting from large pose deviations or lighting changes during the pose-estimation
`
`process. The ability to offer these benefits, where competitors cannot, gives RVT an advantage
`
`in the marketplace.
`
`59.
`
`Reprojection. eVF uses a method for improving the pose estimation during
`
`feature qualification, training the model, and pose calculation known as “reprojection.”
`
`Reprojection is the process of running the inverse projection process (described below) multiple
`
`15
`
`

`

`Case 1:22-cv-01257-GBW Document 1 Filed 09/22/22 Page 16 of 62 PageID #: 16
`
`times during the pose-estimation algorithm, but with a pattern or feature eliminated. Statistical
`
`analyses are performed on the newly estimated poses and outlying features are discarded for the
`
`final calculation. This gives the software resiliency against feature error (e.g., from variability
`
`between parts) that allows it to improve the quality of the calculation or to identify features that
`
`should be eliminated during the model-training process.
`
`60.
`
`Inverse projection. To calculate a pose, eVF also uses a trade-secret technique
`
`called “inverse projection” with sparse model data (that is, without the typical usage of a depth
`
`map). Inverse projection improves accuracy in estimating the 3D location of a part with respect
`
`to a 2D camera image. It finds the camera position that minimizes the error of the locations of
`
`the features if the camera were looking at trained 3D sparse model. The software then performs a
`
`gradient descent minimization technique to find the optimal camera position. Minimizing the
`
`location of the camera (as opposed to the conventional method of minimizing the location of the
`
`part) improves cycle times over known techniques.
`
`61.
`
`If this technique fails to find a confident pose, eVF will try a second algorithm
`
`that uses a trade-secret 2D-to-3D center-of-mass registration (i.e. feature correspondences)
`
`process utilizing the Levenberg–Marquardt algorithm designed to find a solution efficiently even
`
`with starting parameter values far from the optimal solution. The traditional approach is to use
`
`only a gradient descent algorithm, which is less accurate in larger deviations.
`
`62.
`
`eVF’s use of inverse projection is further enhanced by its leveraging of extrinsic
`
`camera-calibration algorithms to rapidly perform inverse projections with a calibrated camera.
`
`This technique increases efficiency gains to more direct methods of solving for pose estimation
`
`by setting up complex linear equations.
`
`16
`
`

`

`Case 1:22-cv-01257-GBW Document 1 Filed 09/22/22 Page 17 of 62 PageID #: 17
`
`63.
`
`Use of multiple pose-calculation methods to minimize feature count. After
`
`RVT disclosed its patented method requiring a minimum of six features to calculate a 3D pose
`
`with a single camera, RVT developed a trade-secret method to perform robust, single-camera 3D
`
`pose estimations with only four part features. These methods, which include dynamically using
`
`multiple pose-calculation methods at runtime, allow RVT to perform its trade-secret feature
`
`qualification techniques without risk of having too few features. These methods also allow
`
`RVT’s eVF product to work on smaller parts with fewer reliable features.
`
`64. Golden position. To extrapolate 3D part information from a 2D image, eVF uses
`
`a sparse 3D model of the part instead of a 3D geometric or point-cloud model of the entire part
`
`itself (i.e., a CAD file). In RVT’s trade-secret method, the location of the part in 3D space with
`
`respect to the camera position and the location of each feature within the part are collectively
`
`referred to as the “golden position.” The golden position is thereafter used as the reference and
`
`basis for a series of equations that accomplish pose estimation. For example, during inverse
`
`projection, features are assumed to be in their “golden position,” allowing the system of
`
`equations to produce an accurate estimation for the optimal camera location. Each computed
`
`pose is further defined by its deviation from the part’s global golden position, instead of by its
`
`coordinates in 3D space (the traditional approach). Locating each pose relative to the golden
`
`position instead of locating each pose in 3D space allows for more accurate and robust pose
`
`estimation based on more limited information. It also simplifies calculations in a way that allows
`
`pose estimation to be conducted just as accurately at any starting camera position.
`
`65.
`
`Use of the golden position is different than the industry-standard method for
`
`locating an object in 3D space, namely geometric model-fitting, typically implemented with a
`
`RANSAC algorithm. This traditional method of pose estimation uses two camera positions to
`
`17
`
`

`

`Case 1:22-cv-01257-GBW Document 1 Filed 09/22/22 Page 18 of 62 PageID #: 18
`
`compute a dense depth image of an object. This depth image, along with a geometric model
`
`fitting algorithm, can be used to determine the location of the object. RVT’s alternative method
`
`of creating a golden position through RVT’s auto-train process, and then using that golden
`
`position as a reference point with which to define further pattern positions, enables pose
`
`estimation without a dense depth image, without computationally expensive model-fitting
`
`algorithms, and with a single camera.
`
`66.
`
`These trade-secret methodologies set RVT’s eVF product apart from the
`
`competition and have contributed to RVT’s success in establishing itself as the best-in-class 3D
`
`vision-guided robotics software provider. The precision, accuracy, and reliability of RVT’s
`
`proprietary technology is unparalleled in the industry and is a direct result of these secret features
`
`within confidential portions of RVT’s source code.
`
`67.
`
`For that reason, the engineers who developed the ideas for and implementations
`
`of these secret methods were strictly bound by nondisclosure agreements. RVT (and its
`
`predecessor Braintech) took extensive efforts to keep secret the calculations, algorithms, and
`
`methodologies embodied in RVT’s confidential source code. These efforts included limiting the
`
`personnel who had access to the confidential source code, securing the equipment on which the
`
`confidential source code was stored, and ensuring that everyone with access to the code
`
`understood its trade-secret nature and their contractual obligation to protect its secrecy.
`
`68.
`
`The trade-secret aspects of RVT’s source code are not ascertainable to customers
`
`or competitors who merely interface with RVT’s eVF product. To an outside obs