The K Computer has been used by researchers from the RIKEN HPCI Program for Computational Life Sciences, the Okinawa Institute of Technology (OIST) in Japan and Forschungszentrum Jülich in Germany to carry out out the largest general neuronal network simulation to date:  1.73 billion nerve cells connected by 10.4 trillion synapses. 

The simulation was made possible by the development of advanced novel data structures for the open-source simulation software NEST.

Using NEST, the team, led by Markus Diesmann and Abigail Morrison of the Institute of Neuroscience and Medicine at Jülich, succeeded in simulating a network consisting of 1.73 billion nerve cells connected by 10.4 trillion synapses. To realize this feat, the program recruited 82,944 processors of the K Computer. The process took 40 minutes, to complete the simulation of 1 second of neuronal network activity in real, biological, time. 

Obviously records are nice but there is a long way to go. The simulated network is huge but only represents 1% of the neuronal network in the brain. The nerve cells were randomly connected and the simulation itself was not supposed to provide new insight into the brain - the purpose of the endeavor was to test the limits of the simulation technology developed in the project and the capabilities of K. In the process, the researchers gathered invaluable experience that will guide them in the construction of novel simulation software. 

But the simulation gives neuroscientists a glimpse of what will be possible in the future, with exa-scale computers.

"If peta-scale computers like the K Computer are capable of representing 1% of the network of a human brain today, then we know that simulating the whole brain at the level of the individual nerve cell and its synapses will be possible with exa-scale computers hopefully available within the next decade," explains Diesmann.

Memory of 250,000 PCs

Simulating a large neuronal network and a process like learning requires large amounts of computing memory. Synapses, the structures at the interface between two neurons, are constantly modified by neuronal interaction and simulators need to allow for these modifications.

More important than the number of neurons in the simulated network is the fact that during the simulation each synapse between excitatory neurons was supplied with 24 bytes of memory. This enabled an accurate mathematical description of the network.

In total, the simulator coordinated the use of about 1 petabyte of mainmemory, which corresponds to the aggregated memory of 250.000 PCs.

NEST

NEST is a widely used, general-purpose neuronal network simulation software available to the community as open source. The team ensured that their optimizations were of general character, independent of a particular hardware or neuroscientific problem. This will enable neuroscientists to use the software to investigate neuronal systems using normal laptops, computer clusters or, for the largest systems, supercomputers, and easily exchange their model descriptions.

The Advanced Institute for Computer Science in Kobe, home of the K computer. Credit: RIKEN

A large, international project

Work on optimizing NEST for the K Computer started in 2009 while the supercomputer was still under construction. Shin Ishii, leader of the brain science projects on K at the time, explains that "Having access to the established supercomputers at Jülich, JUGENE and JUQUEEN, was essential, to prepare for K and cross-check results."

Mitsuhisa Sato, of the RIKEN Advanced Institute for Computer Science, points out that "Many researchers at many different Japanese and European institutions have been involved in this project, but the dedication of Jun Igarashi now at the Okinawa Institute of Technology, Gen Masumoto now at the RIKEN Advanced Center for Computing and Communication, Susanne Kunkel and Moritz Helias now at Forschungszentrum Jülich was key to the success of the endeavor."

Paving the way for future projects

Kenji Doya of OIST, currently leading a project aiming to understand the neural control of movement and the mechanism of Parkinson's disease, says "The new result paves the way for combined simulations of the brain and the musculoskeletal system using the K Computer. These results demonstrate that neuroscience can make full use of the existing peta-scale supercomputers."

The achievement on K provides new technology for brain research in Japan and is encouraging news for the Human Brain Project (HBP) of the European Union, scheduled to start this October. The central supercomputer for this project will be based at Forschungszentrum Jülich.

The researchers in Japan and Germany are planning on continuing their successful collaboration in the upcoming era of exa-scale systems.