To review this article, go to My Profile and then View Saved Stories.
To review this article, go to My Profile and then View Saved Stories.
Anil Ananthaswamy
Despite the resounding good fortune of ChatGPT and other giant language models, the synthetic neural networks (ANNs) that underpin those systems would likely be on their way.
For one, the ANNs are “hungry for superpowers,” said Cornelia Fermüller, a computer scientist at the University of Maryland. “And the other challenge is [their] lack of transparency. “Such systems are so confusing that no one understands what they do, or why they work so well. This, in turn, makes it almost unlikely that they explain why through analogy, what humans do: they employ symbols for objects, ideas. , and the relationships between them.
Original story reprinted with perproject from Quanta Magazine, an editorially independent publication of the Simons Foundation whose project is the public understanding of science through coverage of advances and trends in studies in mathematics and physical and biological sciences.
These gaps stem from the existing design of RNAs and their building blocks: individual synthetic neurons. Each neuron receives inputs, performs calculations, and produces outputs. Modern RNAs are elaborate networks of these computing units, trained to perform fast tasks.
However, the limitations of NAS have been evident for a long time. Consider, for example, an ANN that separates circles and squares. To do this, you must have two neurons in your output layer, one indicating a circle and one indicating a square. If you want your ANN to also discern shape color, for example, blue or red, you will want 4 output neurons: one for the blue circle, the blue square, the red circle, and the red square. More functions means even more neurons.
It can’t be the way our brain perceives the herbal world, with all its variations. “You have to propose that, well, you have one neuron for all combinations,” said Bruno Olshausen, a neuroscientist at the University of California, Berkeley. “So you’d have in your brain, [say,] a purple Volkswagen detector. “
Instead, Olshausen and others argue that data in the brain is constituted through the activity of many neurons. Therefore, the belief of a purple Volkswagen is not encoded as the movements of a single neuron, but as those of thousands of neurons. The same set of neurons, pulling another, can constitute a completely different concept (a pink Cadillac, perhaps).
This is the starting point for a technique radically different from computing, known as hyperdimensional computing. The key is that information, such as the perception of a car or its make, style or color, or all of them together, is represented as a single entity: a hyperdimensional vector.
A vector is simply an ordered array of numbers. A three-dimensional vector, for example, consists of 3 numbers: the x, y, and z coordinates of a point in three-dimensional space. A hyperdimensional vector, or hypervector, can simply be a matrix of 10,000 numbers, for example, representing a point in a space of 10,000 dimensions. These mathematical elements and the algebra to manipulate them are flexible and resilient enough to take fashionable computing beyond some of its existing limitations and foster a new technique for synthetic intelligence.
“That’s what I’ve been most excited about for almost my entire career,” Olshausen said. For him and many others, hyperdimensional computing promises a new world in which computing is effectively and physically powerful and decisions made through machines are transparent.
To perceive how hypervectors make calculation possible, let’s go back to photographs with red circles and blue squares. First, we want the vectors to constitute the variables SHAPE and COLOR. Then, we also want vectors for the values that can be assigned to the variables: CIRCLE, SQUARE, BLUE, and RED.
The vectors will have to be different. This difference can be quantified through an asset called orthogonality, which means being at right angles. In three-dimensional space, there are 3 orthogonal vectors: one in the x direction, one in the y direction, and a third in the z direction. In a space of 10,000 dimensions, there are 10,000 mutually orthogonal vectors.
Justin Pot
Tik root
Parker Room
Alejandro Cristian
But if we allow the vectors to be almost orthogonal, the number of those distinct vectors in a high dimension explodes. In the 10,000-dimensional array there are millions of near-orthogonal vectors.
Now let’s create separate vectors to constitute SHAPE, COLOR, CIRCLE, SQUARE, BLUE and RED. Because there are so many conceivable near-orthogonal vectors in a high-dimensional space, you can assign six random vectors to constitute the six elements; They are almost guaranteed almost orthogonal. “The ease of creating near-orthogonal vectors is a primary explanation for why the hyperdimensional constitution should be employed,” Pentti Kanerva, a researcher at the Redwood Center for Theoretical Neuroscience at the University of California, Berkeley, wrote in a paper. Influential article from 2009.
The article was based on paintings made in the mid-1990s by Kanerva and Tony Plate, then a doctoral student with Geoff Hinton at the University of Toronto. Both independently developed algebra to manipulate hypervectors and alluded to its usefulness for high-dimensional computing.
Given our hypervectors for shapes and colors, the formula developed through Kanerva and Plate shows us how to manipulate them in certain mathematical operations. These movements correspond to tactics of symbolic manipulation of concepts.
The first operation is multiplication. It is a way of combining concepts. For example, multiplying the vector SHAPE by the vector CIRCLE links the two into a single representation of the concept “SHAPE is CIRCLE”. This new “bound” vector is almost orthogonal to SHAPE and CIRCLE. Individual parts are capable, a vital feature if you need to extract data from linked vectors. Given a connected vector representing your Volkswagen, you can dissociate and the vector by its color: PURPLE.
The moment operation, uploadition, creates a new vector that represents what is called a concept overlay. For example, you can take two similar vectors, “SHAPE is CIRCLE” and “COLOR is RED,” and load them in combination to create a vector that represents a circular shape with the color red. Again, the superimposed vector can be decomposed into its constituents.
Justin Pot
Tik root
Parker Room
Alejandro Cristian
The third operation is permutation; This is about rearranging the individual elements of the vectors. For example, if you have a three-dimensional vector with prices rated x, y, and z, the permutation can move the price from x to y, from y to z, and from z to x. “Permutation to build a structure,” Kanerva said. It’s you to manage sequences, things that stick together. “That would destroy the data about the order of events. The mixture of addition and permutation preserves order; Events can be retrieved in order using reverse trades.
Together, those 3 operations proved sufficient to create a formal algebra of hypervectors that allows symbolic reasoning. But many researchers have been slow on the prospect of hyperdimensional computing, Olshausen added. “It just didn’t penetrate,” he said.
In 2015, an Olshausen student named Eric Weiss demonstrated a facet of the functions of hyperdimensional computing. Weiss figured out how to constitute a complex symbol as a single hyperdimensional vector that contains data about all the elements of the symbol, adding their properties, such as colors. , positions and sizes.
“I almost fell off my chair,” Olshausen said. Suddenly, the light bulb went on. “
Soon, other groups began devising hyperdimensional algorithms to reflect undeniable responsibilities that deep neural networks had begun to play about two decades earlier, such as symbol classification.
Consider an annotated knowledge set consisting of handwritten number symbols. A set of rules analyzes the characteristics of each of the symbols in a predetermined pattern. It then creates a hypervector for each of the symbols. The rule set then adds the hypervectors for all symbols of 0 to create a hypervector for the concept of 0. It then does the same for all digits, creating 10 “class” hypervectors, one for each of a single digit.
The rule set now receives an unlabeled symbol. Create a hypervector for this new symbol and then compare the hypervector to the stored elegance hypervectors. This comparison determines the number to which the maximum of the new symbol resembles.
Still, this is just the beginning. The strengths of hyperdimensional computing lie in the ability to compose and decompose hypervectors for reasoning. The most recent demonstration came in March, when Abbas Rahimi and colleagues at IBM Research in Zurich used hyperdimensional computing with neural networks to solve a long-standing summary challenge. Visual reasoning: A significant challenge for typical NAS and even some humans. Known as progressive Raven matrices, the challenge features symbols of geometric elements in, say, a grid from 3 to 3. A position in the grid is empty. The theme must choose, from a set of candidate symbols, the symbol that most productively corresponds to the target.
“We said, ‘It’s really. . . the killer of visual summary reasoning, come on,” Rahimi said.
To solve the challenge using hyperdimensional computing, the team first created a hypervector dictionary to constitute the elements in each symbol; Each hypervector in the dictionary constitutes an object and a mixture of its attributes. The team then trained a neural network to read over a symbol and generate a bipolar hypervector (a detail can be 1 or -1) as close as you can imagine to a superposition of hypervectors in the dictionary; Thus, the generated hypervector comprises data about all the elements and their attributes in the symbol. “You’re guiding the neural network into a meaningful conceptual space,” Rahimi said.
Justin Pot
Tik root
Parker Room
Alejandro Cristian
Once the network has generated hypervectors for the context symbols and for the candidate for the empty location, some other set of rules analyzes the hypervectors to create probability distributions for the number of elements in the symbol, their size, and other characteristics. These probability distributions, which speak of the maximum probable characteristics of the context and candidate symbols, can be remodeled into hypervectors, allowing the use of algebra to wait for the candidate symbol to fill the vacant location.
Their technique was about 88 percent accurate on a set of problems, while neural network responses alone were accurate on less than 61 percent. The team also showed that, for 3-by-3 grids, their formula was almost 250 times faster than a classical technique that uses rules of symbolic logic to reason, as this technique has to pass through a massive rulebook to the next correct step.
Hyperdimensional computing not only gives us the strength to solve disorders symbolically, but it also solves some of the thorny disorders of classical computing. vice versa) cannot be corrected through built-in error correction mechanisms. In addition, those error-correcting mechanisms can impose a functionality penalty of up to 25 percent, said Xun Jiao, a computer scientist at Villanova University.
Hyperdimensional computing is more error-tolerant because even if a hypervector undergoes a large number of random bit changes, it remains close to the original vector. This implies that any reasoning about those vectors is not particularly affected by errors. Jiao’s team showed that those systems are at least 10 times more tolerant of hardware problems than classic ANNs, which are themselves orders of magnitude more resilient than classical IT architectures. “We can leverage any [this] resilience to design effective hardware,” Jiao said.
Another merit of hyperdimensional computing is transparency: algebra clarifies why the formula chose the answer it chose. Not so with classical neural networks. Olshausen, Rahimi and others expand hybrid formulas in which neural networks map elements of physics. global in hypervectors, and then hyperdimensional algebra takes over. “Things like analog reasoning fall on you,” Olshausen said. .
All those benefits over classical computing suggest that hyperdimensional computing is suitable for a new generation of incredibly robust, low-power hardware. retail data (unlike existing von Neumann computers that move data inefficiently between memory and the central processing unit). In von Neumann’s computation, this randomness is “the wall you can’t get through,” Olshausen said. But with hyperdimensional computing, “you can just get through. “
Despite those advantages, hyperdimensional computing is still in its infancy. “There’s a genuine perspective here,” Fermüller said.
“For large-scale problems, this requires a lot of hardware,” Rahimi said. “For example, how do you search for more than a billion items effectively?”
All of this comes with time, Kanerva said. There are other secrets that the giant spaces keep,” he said. “I see this as the beginning of time for vector calculus. “
Original story reprinted with perproject from Quanta Magazine, an editorially independent publication of the Simons Foundation whose project is the public understanding of science through coverage of advances and trends in studies in mathematics and physical and biological sciences.
? Don’t miss our most important stories, delivered to your inbox every day
? Our new podcast for you to have a bright future
The night the army of 17 million went up in smoke
Protest campaigns against AI against human extinction
How your new one follows you
A grid collapse would make a heat wave much more lethal
Wired’s favorite “buy it for life” team
? Our Gear team has diversified with a new consultant for ground mattresses and new selections for refrigerators and binoculars
Alejandro Cristian
Mary McKenna
Chris van Tulleken
Celia Ford
Celia Ford
Raquel Nuwer
Ricardo Forbes
Ramin Skibba
More from CABLING
Contact