6/5/2014
`
`Claude Berrou : from turbo codes to the neocortex- lnstitut Mines-Télécom
`
`
`
`Live labs
`
`Home / Live labs / Claude Berrou : from turbo codes to the neocortex
`
`
`
`Claude Berrou : from turbo codes to the neocortex
`
`1 April 2012 / in Live labs
`
`Claude Berrou is the co-inventor of turbo codes, a major development
`
`in the history of digital
`
`communications now widely used in satellites and high-speed networks. An internationally—respected
`
`researcher and recipient of numerous prizes, as well as a member of the prestigious Académie des
`
`Sciences, since 2011 Professor Berrou has been working on a new challenge: applying his knowledge of
`
`information theory to artificial intelligence and the neocortex.
`
`Turbo codes, a major scientific turning point
`
`By October 1996, when an article entitled “NearOptimum Error Correcting Coding and Decoding: Turbo—
`
`Codes" appeared in the prestigious journal
`
`IEEE Transactions on Communications,
`
`the international
`
`community had already known for some three years that a previously-unknown team of Brittany-based
`
`researchers were in the process of revolutionizing digital communications.
`
`Since American Claude Shannon published his pioneering work in 1948, it had been widely accepted that in a
`
`digital communication channel disrupted by a certain amount of noise, as long as the average level of noise
`
`does not exceed a certain threshold and the information is encoded with appropriate redundancy, the
`
`receiver can identify the original message without errors. But the efficiency of forward correction codes is
`
`http://iM/iwmines-telecomfr/en/claude berrou—from-turbo—codes-tothe— neocortefi
`
`1/4
`
`ERICS SON EXHIBIT 1018
`
`

`

`6/5/2014
`
`Claude Berrou : from turbo codes to the neocortex— Institu’i Mines-Télécom
`
`limited and researchers had been attempting to reach the theoretical limit for decades, before finally writing
`
`it off as an impossibility. There was thus a considerable degree of scepticism in the air at
`
`ICC ’93
`
`(International Communications Conference) when Claude Berrou and his colleague and co-author Alain
`
`Glavieux announced that they had reached the fabled Shannon Limit, even more so since this discovery was
`
`not backed up by reams of equations. Yet the leading figures in the field verified the results, and an
`
`integrated circuit developed by Berrou and his team proved to even the most sceptical observers that the
`
`results they had announced could be accurately measured. This marked the dawn of the turbo code era,
`
`with dozens ofteams launching themselves into researching this new field. Over four hundred patents have
`
`been registered in the last twenty years.
`
`This discovery was made possible by the expertise and experience of Claude Berrou, who modestly
`
`describes himself as "average at everything." Prof. Berrou has always been a passionate advocate of
`
`interdisciplinary collaboration, and a researcher whose motto is “small
`
`is beautiful”. Born in Penmarc’h
`
`(Finistere) in 1951, he first arrived at the electronics department of the newly—founded Ecole Nationale
`
`Supérieure des Telecommunications de Bretagne (now Télécom Bretagne) in 1978, at a time when the whole
`
`curriculum was still under construction. He began working on the physics oftransistors, microwaves, circuit
`
`architecture and metrology, studying the use of algorithms in a huge variety of contexts before moving on
`
`to tackle the theory of information encoding. He was appointed head of the integrated circuit design
`
`laboratory in 1986, and began working with digital communication specialist Alain Glavieux to study
`
`encoding techniques in telecommunications. He soon began to wonder why scientists in this field did not
`
`seem to apply a principle which was common currency in electronics: feedback, or reintroducing a portion of
`
`the output signal at the input
`
`level. This ”naive", newcomer’s question turned out
`
`to be a major
`
`breakthrough. Claude Berrou’s massive contribution to science has earned him international recognition
`
`from his peers, and has seen him cited over 13,000 times in academic publications (placing him among the
`
`top ten most referenced FrenCh scientists in the IT field). His achievements have also been feted in France,
`
`where he was elected to the Académie des Sciences in 2007. He is a member of two of the Académie’s
`
`sections: computer science & mechanics and inter-disciplinary scientific applications. He is also hoping to
`
`help to establish a new section devoted entirely to information technologies, a field of great importance
`
`which is currently under—represented at the Académie.
`
`His work has brought him various other honours and awards: in 2003, Claude Berrou and Alain Glavieux
`
`(who passed away in 2004) were awarded the Institute of Electrical and Electronics Engineers’ (IEEE)
`
`Hamming Medal. He also picked up the Marconi Prize[1_]_[]_ in 2005, following in the footsteps ofthe creators
`
`of Google in 2004, and the inventor of the World Wide Web itself in 2002. He was also nominated for the
`
`European Inventor of the Year award in 2006. He was actively involved in the foundation of the company
`
`Turboconcept, and has registered several patents in the field of coding and decoding, and on the extension
`
`ofthe ’turbo principle’to other functions within telecommunications systems.
`
`Turbo code theory
`
`The principle behind turbo codes is deceptively simple, based on an idea which will have occurred to anyone
`
`who has ever done a crossword. A turbo code is composed of two elementary codes which deliver the
`
`vertical and horizontal definitions. To solve the puzzle, we fill
`
`in the boxes one by one and the vertical
`
`'decoding’ information can help confirm or contradict the horizontal
`
`information, and vice versa. The
`
`http://wvmmines-telecom.frlenlclaude- berrou—from-turbo—codes-tdihe- neocortex/
`
`2/4
`
`

`

`6/5/2014
`
`Claude Berrou : from turbo codes to the neocortex— Institut Mines-Télécom
`
`outputs (hypotheses) of each decoding system are thus fed back into the other one in order to help with the
`
`process.At the |CC93 conference, this idea immediately piqued the interest of a researcher from NASA. The
`
`increase in efficiency made possible by this method can cut energy expenditure in half, thus drastically
`
`reducing the number of solar panels required for space probes and saving the agency tens of millions of
`dollars. The world now has between four and five billion turbo decoders, powering our high—speed WiMAX
`
`and 3G and 4G networks. Turbo codes were used in the Mars Express mission launched in june 2003. El_n_o_|
`
`out more about turbo codes. |en.wikipedia.org/wiki/Turbo codel
`
`Artificial intelligence, the next step
`
`Claude Berrou is now working on a new challenge, exploring the field of neuroscience. As was the case with
`
`information theory in the 19905, exploration of the human brain currently remains the preserve of a
`
`specialized group of neurobiologists. Professor Berrou wants to shed new light on our mental functions by
`
`approaching them from the perspective of information theory, and looking at the brain as a form of
`
`distributed decoder. As he explains, "these days innovation often arises when different disciplines collide."
`
`He remains deeply committed to breaking down barriers between different scientific domains.
`
`in 2007 Prof. Berrou first observed that the neocortex, that "propagation medium which allows biological
`
`processes to skip from one part of the brain to another," reveals some significant structural similarities with
`
`modern decoders. The analogies and dissimilarities which can be drawn have already proved useful to
`
`neuroscientists and information scientists. Prof. Berrou subsequently decided to devote his full attention to
`
`neurosciences: "it was at this point that i came to a second observation: mental information is robust and
`
`durable. Given the constant pressures on the brain, this information must be redundant.” And so Prof.
`
`Berrou, a leading expert on the principle of redundancy, joined forces with a brilliant doctoral student by the
`
`name of Vincent Gripon. Within the space of a few months they had identified new codes which serve to
`
`represent and memorise information, and which may help explain why mental information is robust and
`
`durable, thus opening up exciting new perspectives for artificial intelligence. ”This goes way beyond turbo
`
`codes," Prof. Berrou declares enthusiastically. Indeed it turns out that these codes can be transposed onto
`
`any graph, a discovery which could revolutionise research into networks of artificial neurones.
`
`Two patents were soon registered, and various articles published: the Neucod (neural coding) project was
`
`born. The project was submitted to the European Research Council in 2011, receiving funding of€
`
`1.9 million.
`
`Ofthe 290 projects selected (from 2287 applicants), only 31 were based in France, compared to 68 in the UK.
`
`"Finally, this new approach was vindicated,” Claude Berrou proudly recalls. The funding enabled him to take
`
`on six full-time researchers on five—year contracts, supplementing the team's existing scientific manpower.
`
`"The aim is to produce thinking machines, capable of learning millions of pieces of information and cross—
`
`referencing them to produce new information,” the professor explains. This artificial neocortex, or ‘cognitor‘,
`
`could then evolve from 'infancy’ to become a real 'intellectual auxiliary’ by gathering information from the
`
`internet, which the researchers will then put into context. A firm believer in the theory of technological
`
`singularity, Claude Berrou is certain that “by 2050 we will have machines which are more intelligent than
`humans."
`
`”The work we are doing could lead to major,
`
`long—term partnerships with a number of big industrial
`
`companies," he adds. As the Neucod team continues to form new partnerships with other research teams,
`
`hflpi/IMAMMl’nl hes-telecomfr/en/claude- berrou-fromturbo-codes-to—the— neocortex/
`
`3/4
`
`

`

`6/5/2014
`
`Claude Berrou : from turbo codes to the neocortex- lnstitut Mines-Télécom
`
`the turbo code adventure could be about to repeat itself, with Claude Berrou once again at the forefront ofa
`
`major scientific breakthrough, this time in the field of artificial intelligence.
`
`Further reading: A theory of mental information processing
`
`
`://www.mines-telecom.fr/w -
`htt
` {ILfiaLll’lIE 3.5mm;
`“WENT {321?in
`
`content/uploads/2013/11/201210 Berrou enc2.'pgl
`
`were mam—1mm
`W 31;:va
`mummwwmm
`
`
`
`
`
`
`
`
`
`
`
`
`we".
`0!
`if am ’
`{:Piimghb
`9m»:
`
`Passionately committed to bringing scientific knowledge to the widest possible audience, in september 2012
`
`Claude Berrou’s latest work was published by OdileJacob Sciences. This book is an accessible introduction to
`
`his latest research interest , co-authored by Vincent Gripon and entitled "Petite mathématique du cerveau.
`Une théorie de l’information mentale“.
`
`Claude Berrou is the co—inventor of turbo codes, a major development
`
`in the history of digital
`
`communications now widely used in satellites and high-speed networks. An internationally-respected
`
`researcher and recipient of numerous prizes, as well as a member ofthe prestigious Académie des Sciences,
`
`since 2011 Professor Berrou has been working on a new challenge: applying his knowledge of information
`
`theory to artificial intelligence and the neocortex.
`
`Tags: Artificial intelligence, Claude Berrou, neocortex, turbo codes
`
`Share this entry
`
`
`
`
`
`tnstitut NliiifiS~TéiéCOlii w Enfold Theme by Kriesi
`
`http://MMM/mihes—telecomfr/en/cl aude- berrowfrom—turbo— codes-lo—the neocortex/
`
`4/4
`
`