No single neuron produces a thought or a behavior; anything the brain accomplishes is a vast collaborative effort between cells. When at work, neurons talk rapidly to one another, forming networks as they communicate. Researchers led by Rockefeller University’s Alipasha Vaziri are developing technology that would make it possible to record brain activity as it plays out across these networks.
In research published October 31 in Nature Methods, they recorded the activity of thousands of neurons layered within three-dimensional sections of brain as they signaled to one another in a living mouse.
“The ultimate goal of our work is to investigate how large numbers of interconnected neurons throughout the brain interact in real time and how their dynamics lead to behavior,” says Vaziri, associate professor and head of the Laboratory of Neurotechnology and Biophysics. “By developing a new method based on ‘light sculpting’ and using it to capture the activity of the majority of the neurons within a large portion of the cortex, a layered brain structure involved amongst others in higher brain function, we have taken a significant step in this direction.”
This type of recording presents a considerable technical challenge because it requires tools capable of capturing short-lived events within individual cells, all while observing large volumes of brain tissue.
Vaziri, who joined Rockefeller last year, began working toward this goal about six years ago while at the Research Institute of Molecular Pathology in Vienna. His group first succeeded in developing a light-microscope–based approach to observing the activity within a whole 302-neuron roundworm brain, before moving on to the 100,000-neuron organ of a larval zebrafish. Their next target, the mouse brain, is more challenging for two reasons: Not only is it more complex, with about 70 million neurons, but the rodent brain is also opaque, unlike the more transparent worm and larval fish brains.
To make the activity of neurons visible, they had to be altered. The researchers engineered the mice so their neurons could emit fluorescent light when they signal to one another. The stronger the signal, the brighter the cells shine.
The microscopy system they developed had to meet competing demands: It needed to generate a spherically shaped spot, slightly smaller than the neurons and capable of efficiently exciting fluorescence from them. Meanwhile, it also had to move quickly enough to scan the activity of thousands of these cells in three dimensions as they fire in real time.
The team accomplished this using a technique called “light sculpting,” in which short pulses of laser light, each lasting only a quadrillionth of a second, are dispersed into their colored components. These are then brought back together to generate the “sculpted” excitation sphere.
This sphere is scanned to illuminate the neurons within a plane, then refocused on another layer of neurons above or below, allowing neural signals to be recorded in three dimensions. (This was done while the mouse’s head was immobilized, but its legs were free to run on a customized treadmill.)
In this way, Vaziri and his colleagues recorded the activity within one-eighth of a cubic millimeter of the cortex, of the animal’s brain, a volume that represents the majority of a unit known as a cortical column. By simultaneously capturing and analyzing the dynamic activity of the neurons within a cortical column, researchers think they might be able to understand brain computation as a whole. In this case, the section of cortex studied is responsible for planning movement.
The researchers are currently working to capture the activity of an entire such unit.
“Progress in neuroscience, and many other areas of biology, is limited by the available tools,” Vaziri says. “By developing increasingly faster, higher-resolution imaging techniques, we hope to be able to push the study of the brain into new frontiers.”
New training technique would reveal the basis for machine-learning systems’ decisions.
In recent years, the best-performing systems in artificial-intelligence research have come courtesy of neural networks, which look for patterns in training data that yield useful predictions or classifications. A neural net might, for instance, be trained to recognize certain objects in digital images or to infer the topics of texts.
But neural nets are black boxes. After training, a network may be very good at classifying data, but even its creators will have no idea why. With visual data, it’s sometimes possible to automate experiments that determine which visual features a neural net is responding to. But text-processing systems tend to be more opaque.
At the Association for Computational Linguistics’ Conference on Empirical Methods in Natural Language Processing, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) will present a new way to train neural networks so that they provide not only predictions and classifications but rationales for their decisions.
“In real-world applications, sometimes people really want to know why the model makes the predictions it does,” says Tao Lei, an MIT graduate student in electrical engineering and computer science and first author on the new paper. “One major reason that doctors don’t trust machine-learning methods is that there’s no evidence.”
“It’s not only the medical domain,” adds Regina Barzilay, the Delta Electronics Professor of Electrical Engineering and Computer Science and Lei’s thesis advisor. “It’s in any domain where the cost of making the wrong prediction is very high. You need to justify why you did it.”
“There’s a broader aspect to this work, as well,” says Tommi Jaakkola, an MIT professor of electrical engineering and computer science and the third coauthor on the paper. “You may not want to just verify that the model is making the prediction in the right way; you might also want to exert some influence in terms of the types of predictions that it should make. How does a layperson communicate with a complex model that’s trained with algorithms that they know nothing about? They might be able to tell you about the rationale for a particular prediction. In that sense it opens up a different way of communicating with the model.”
Neural networks are so called because they mimic — approximately — the structure of the brain. They are composed of a large number of processing nodes that, like individual neurons, are capable of only very simple computations but are connected to each other in dense networks.
In a process referred to as “deep learning,” training data is fed to a network’s input nodes, which modify it and feed it to other nodes, which modify it and feed it to still other nodes, and so on. The values stored in the network’s output nodes are then correlated with the classification category that the network is trying to learn — such as the objects in an image, or the topic of an essay.
Over the course of the network’s training, the operations performed by the individual nodes are continuously modified to yield consistently good results across the whole set of training examples. By the end of the process, the computer scientists who programmed the network often have no idea what the nodes’ settings are. Even if they do, it can be very hard to translate that low-level information back into an intelligible description of the system’s decision-making process.
In the new paper, Lei, Barzilay, and Jaakkola specifically address neural nets trained on textual data. To enable interpretation of a neural net’s decisions, the CSAIL researchers divide the net into two modules. The first module extracts segments of text from the training data, and the segments are scored according to their length and their coherence: The shorter the segment, and the more of it that is drawn from strings of consecutive words, the higher its score.
The segments selected by the first module are then passed to the second module, which performs the prediction or classification task. The modules are trained together, and the goal of training is to maximize both the score of the extracted segments and the accuracy of prediction or classification.
One of the data sets on which the researchers tested their system is a group of reviews from a website where users evaluate different beers. The data set includes the raw text of the reviews and the corresponding ratings, using a five-star system, on each of three attributes: aroma, palate, and appearance.
What makes the data attractive to natural-language-processing researchers is that it’s also been annotated by hand, to indicate which sentences in the reviews correspond to which scores. For example, a review might consist of eight or nine sentences, and the annotator might have highlighted those that refer to the beer’s “tan-colored head about half an inch thick,” “signature Guinness smells,” and “lack of carbonation.” Each sentence is correlated with a different attribute rating.
As such, the data set provides an excellent test of the CSAIL researchers’ system. If the first module has extracted those three phrases, and the second module has correlated them with the correct ratings, then the system has identified the same basis for judgment that the human annotator did.
In experiments, the system’s agreement with the human annotations was 96 percent and 95 percent, respectively, for ratings of appearance and aroma, and 80 percent for the more nebulous concept of palate.
In the paper, the researchers also report testing their system on a database of free-form technical questions and answers, where the task is to determine whether a given question has been answered previously.
In unpublished work, they’ve applied it to thousands of pathology reports on breast biopsies, where it has learned to extract text explaining the bases for the pathologists’ diagnoses. They’re even using it to analyze mammograms, where the first module extracts sections of images rather than segments of text.
“There’s a lot of hype now — and rightly so — around deep learning, and specifically deep learning for natural-language processing,” says Byron Wallace, an assistant professor of computer and information science at Northeastern University. “But a big drawback for these models is that they’re often black boxes. Having a model that not only makes very accurate predictions but can also tell you why it’s making those predictions is a really important aim.”
“As it happens, we have a paper that’s similar in spirit being presented at the same conference,” Wallace adds. “I didn’t know at the time that Regina was working on this, and I actually think hers is better. In our approach, during the training process, while someone is telling us, for example, that a movie review is very positive, we assume that they’ll mark a sentence that gives you the rationale. In this way we train the deep-learning model to extract these rationales. But they don’t make this assumption, so their model works without using direct annotations with rationales, which is a very nice property.”
Learn more: Making computers explain themselves
A research group of the UPV/EHU-University of the Basque Country has made progress in obtaining bio-oils and raw materials from biomass using its patented reactor
The UPV/EHU’s Catalytic Processes for Waste Valorisation research group is working on various lines of research relating to renewable energies, one of which corresponds to the obtaining of bio-oils or synthetic petroleum using biomass. In a paper recently published in the scientific journal Fuel, the researchers have proposed using artificial neural networks to determine the heating power of each type of biomass using its composition as it is a highly irregular material.
Biomass is one of the main sources of energy and heat in the field of renewable energy production: it is any type of non-fossil organic matter, such as living plants, timber, agricultural and livestock waste, wastewater, solid urban organic waste, etc. The three most developed technologies for obtaining energy from biomass are as follows: pyrolysis (decomposition by heating in the absence of oxygen), gasification (reaction with air, oxygen or a blend of both and conversion into gas) and combustion (decomposition through heating with oxygen). The effectiveness and emission levels of these three processes change depending on the composition of the biomass as well as its properties, the experimental conditions and equipment used.
In collaboration with researchers at the University of Sao Carlos in Brazil and within the framework of a European project, members of the UPV/EHU’s Catalytic Processes for Waste Valorisation research group analysed the process to set up a refinery to obtain bio-oils or synthetic petroleum using biomass. Since “afterwards, using the bio-oil produced it is possible to obtain the same products that are obtained from petroleum; hydrogen as well as any other compound,” explained Martin Olazar, project leader and professor of the Department of Chemical Engineering. The reactor developed and patented by this research group, the conical spouted bed reactor, is highly suited to this process because it is suitable for handling irregular, sticky materials —biomass is a highly irregular material and difficult to handle using conventional technologies—.
Artificial neural networks to determine gross calorific value
In the design of the process to obtain bio-oils using biomass, certain variables need to be determined: the temperature that needs to be achieved, how this temperature is to be achieved, how much fuel (in this case how much biomass) needs to be burnt, etc. The gross calorific value is a key parameter in determining all these data: it is the heat (energy) that is released when a certain quantity of fuel is completely burnt. This parameter is essential in the analysis, design and improvement in biomass pyrolysis, gasification and combustion systems. The correlations existing in the literature give highly variable results depending on each type of biomass and its properties. So the researchers in the group are proposing that artificial neural networks be used to calculate this; they have proven empirically that the system gives very good results and they have reported on them in a paper recently published in the scientific journal Fuel.
A team of scientists from the Moscow Institute of Physics and Technology (MIPT) have created prototypes of “electronic synapses” based on ultra-thin films of hafnium oxide (HfO2). These prototypes could potentially be used in fundamentally new computing systems.
The paper has been published in the journal Nanoscale Research Letters.
The group of researchers from MIPT have made HfO2-based memristors measuring just 40×40 nm2. The nanostructures they built exhibit properties similar to biological synapses. Using newly developed technology, the memristors were integrated in matrices: in the future this technology may be used to design computers that function similar to biological neural networks.
Memristors (resistors with memory) are devices that are able to change their state (conductivity) depending on the charge passing through them, and they therefore have a memory of their “history”. In this study, the scientists used devices based on thin-film hafnium oxide, a material that is already used in the production of modern processors. This means that this new lab technology could, if required, easily be used in industrial processes.
“In a simpler version, memristors are promising binary non-volatile memory cells, in which information is written by switching the electric resistance – from high to low and back again. What we are trying to demonstrate are much more complex functions of memristors – that they behave similar to biological synapses,” said Yury Matveyev, the corresponding author of the paper, and senior researcher of MIPT’s Laboratory of Functional Materials and Devices for Nanoelectronics, commenting on the study.
Synapses – the key to learning and memory
A synapse is point of connection between neurons, the main function of which is to transmit a signal (a spike – a particular type of signal, see fig. 2) from one neuron to another. Each neuron may have thousands of synapses, i.e. connect with a large number of other neurons. This means that information can be processed in parallel, rather than sequentially (as in modern computers). This is the reason why “living” neural networks are so immensely effective both in terms of speed and energy consumption in solving large range of tasks, such as image / voice recognition, etc.
Over time, synapses may change their “weight”, i.e. their ability to transmit a signal. This property is believed to be the key to understanding the learning and memory functions of the brain.
From the physical point of view, synaptic “memory” and “learning” in the brain can be interpreted as follows: the neural connection possesses a certain “conductivity”, which is determined by the previous “history” of signals that have passed through the connection. If a synapse transmits a signal from one neuron to another, we can say that it has high “conductivity”, and if it does not, we say it has low “conductivity”. However, synapses do not simply function in on/off mode; they can have any intermediate “weight” (intermediate conductivity value). Accordingly, if we want to simulate them using certain devices, these devices will also have to have analogous characteristics.
The memristor as an analogue of the synapse
As in a biological synapse, the value of the electrical conductivity of a memristor is the result of its previous “life” – from the moment it was made.
There is a number of physical effects that can be exploited to design memristors. In this study, the authors used devices based on ultrathin-film hafnium oxide, which exhibit the effect of soft (reversible) electrical breakdown under an applied external electric field. Most often, these devices use only two different states encoding logic zero and one. However, in order to simulate biological synapses, a continuous spectrum of conductivities had to be used in the devices.
“The detailed physical mechanism behind the function of the memristors in question is still debated. However, the qualitative model is as follows: in the metal–ultrathin oxide–metal structure, charged point defects, such as vacancies of oxygen atoms, are formed and move around in the oxide layer when exposed to an electric field. It is these defects that are responsible for the reversible change in the conductivity of the oxide layer,” says the co-author of the paper and researcher of MIPT’s Laboratory of Functional Materials and Devices for Nanoelectronics, Sergey Zakharchenko.
The authors used the newly developed “analogue” memristors to model various learning mechanisms (“plasticity”) of biological synapses. In particular, this involved functions such as long-term potentiation (LTP) or long-term depression (LTD) of a connection between two neurons. It is generally accepted that these functions are the underlying mechanisms of memory in the brain.
The authors also succeeded in demonstrating a more complex mechanism –spike-timing-dependent plasticity, i.e. the dependence of the value of the connection between neurons on the relative time taken for them to be “triggered”. It had previously been shown that this mechanism is responsible for associative learning – the ability of the brain to find connections between different events.
To demonstrate this function in their memristor devices, the authors purposefully used an electric signal which reproduced, as far as possible, the signals in living neurons, and they obtained a dependency very similar to those observed in living synapses (see fig. 3).
Fig.3. The change in conductivity of memristors depending on the temporal separation between “spikes”(rigth) and thr change in potential of the neuron connections in biological neural networks.
Source: MIPT press office
These results allowed the authors to confirm that the elements that they had developed could be considered a prototype of the “electronic synapse”, which could be used as a basis for the hardware implementation of artificial neural networks.
“We have created a baseline matrix of nanoscale memristors demonstrating the properties of biological synapses. Thanks to this research, we are now one step closer to building an artificial neural network. It may only be the very simplest of networks, but it is nevertheless a hardware prototype,” said the head of MIPT’s Laboratory of Functional Materials and Devices for Nanoelectronics, Andrey Zenkevich.
Energy-friendly chip can perform powerful artificial-intelligence tasks
In recent years, some of the most exciting advances in artificial intelligence have come courtesy of convolutional neural networks, large virtual networks of simple information-processing units, which are loosely modeled on the anatomy of the human brain.
Neural networks are typically implemented using graphics processing units (GPUs), special-purpose graphics chips found in all computing devices with screens. A mobile GPU, of the type found in a cell phone, might have almost 200 cores, or processing units, making it well suited to simulating a network of distributed processors.
At the International Solid State Circuits Conference in San Francisco this week, MIT researchers presented a new chip designed specifically to implement neural networks. It is 10 times as efficient as a mobile GPU, so it could enable mobile devices to run powerful artificial-intelligence algorithms locally, rather than uploading data to the Internet for processing.