Approaching the von Neumann Bottleneck: Neuromorphic Computing & beyond

Must Read

Ed-Tech Companies and the Consumer Protection Act

In the present time when the whole country is getting back to normal after the wrath of the Coronavirus,...

The Right to Information and its Working of 15 years

On 12th October 2020, RTI finished fifteen years since its commencement. The question remains whether the legislation stands up to...

An Insight into Custodial Death in India

“The occurrence of Custodial deaths in the world’s greatest democracy has raised the eyebrows of every citizen and shaken...

Implications in Travel Insurance in Light of the COVID-19 Crisis

As the world, today is crippled by this once in a century pandemic and as of date more than...

Second-Round Effects of Rent Control Laws: The Argentine Case

Introduction In colonial India, a city had an issue with its cobra population, which was a problem clearly in need...

Why Are the Big Techs of Silicon Valley Accused of Anti-Competitive Behaviours?

The big tech giants of the Silicon Valley are facing major challenges with relation to their monopolistic powers after...

Follow us

There are one trillion synapses in a cubic centimeter of the brain. If there is such a thing as General AI, it would probably require one trillion synapses.”

-Dr. Geoffrey Hinton

Digital computing in no less measure has come to compose the societal fabric in today’s world.Its transformative influence, made possible by the pervasive technological evolution and remarkable commercial success leaves no doubts about its legitimacy. However, the hardwaredesign that most computers today are based on has been seemingly unchanged since the von Neumann architecture (named after the famous computer scientist and mathematician John von Neumann) and begs reform to keep up with the new technology in the continuously evolving computer age.

In the current model, there exists a sharp demarcation between computational units and memory. Simply put, during an operation, data is designed to move from the memory to the processor, which then processes the data before transferring it back to the memory. There exists modern technology with microprocessors possessing multiple computation units and 64 bit registers but the inherent model has virtually stayed the same. Von Neumann architecture has seemed to work and survive all these decades since the conception of computational hardware, so where does the issue lie?

Approaching the von Neumann Bottleneck

Conventional computers appear to be reaching their limits with various kinds of super computers and artificial intelligence applications requiring massive computing capacity. These employ deep learning systems that have all but maxed out the hardware they operate on. To put this into perspective, a simple computer chip, the size of a button, constitutes billions of transistors. Every workaday computer in today’s world employs thousands of these button sized chips.

Moore’s law states that the numbers of transistors that can be placed on a chip will double every year, with the expense staying the same. An AI company based out of San Francisco, US called Open AI analyzed current computing trends which are used to train AI systems over the past decades. They concluded that before 2012 it had generally followed Moore’s Law, with computational power doubling every two years. However, since 2012, this exact computation has been doubling every 3.4 months. Meanwhile, memory performance has struggled to proportionately increase, lagging behind the processor performance. Even squeezing millions of micro components into even smaller chips shall raise the cost of the same by record high factors.

Further due to the separation of memory and computing in von Neumann architecture, a large part of the energy consumption gets used up in the delayed transfer of information betweenmemory and computing parts. Thereby, with increased expenditure, limitation in physical hardware, and delays in computing, we seem to be approaching what has been termed as the von Neumann bottleneck. This “von Neumann” bottleneck limits the future development of revolutionary computational systems and overall performance improvementsThis also prevents us from realizing a general level of artificial intelligence.

Whilst a slowdown of Moore’s Law is being felt world over by experts, scientists have been proffering insights into the brain’s behavior that provide inspiration for novel computing solutions more than ever before. Replicating the brain’s reach into computing not only betters computational capacities but also widens applicability of future and current AI. This is where neuromorphic computing steps in.

Neuromorphic Computing: A Panacea?

One of the most favoured approaches to the problem posed by von Neumann’s bottleneck is inspired by biological principles. Our brains distribute computation and memory amongst billions of single processing units called neurons, which are interconnected with hundreds of thousands of connections called synapses. What makes brain wide processing possible is that fact that there’s no specific memory or central computational unit element. Biological brains are vastly parallelized and require an iota of the energy that conventional (which perform computations in a serial and time taking manner) employ.

In employing such principles to computational technologies, the redundancy linked with data traffic can be wholly evaded subject to computational operations and data storage being carried out locally together in the memory itself. Unlike conventional architecture in computers, this energy efficient biologically-inspired approach is known as in-memory computing. It is expected to mitigate issues of computational complexity and memory thrashing with rapid execution, and an innate capability to learn. Capitalising on increased understanding of the human brain,technologies that circumvented the problem of von Neumann’s architecture were fashioned to result in computational principles termed as neuromorphic computing.

Carver Mead, of California Institute of Technology (Caltech) was one of the former researchers who highlighted the extraordinary stinginess of energy usage in biological computing in his visionary paper written in 1990, and coined the term “neuromorphic”. Neuromorphic computing is expected to provide for a tool that understands the dynamic processes of development and learning in the brain and entail this inspiration into cognitive computing. Since, operations in brain are performed asynchronously, in a parallel fashion with memory and processing taking place locally in the neurons and synapses, neuromorphic architecture is expected to imitate the same.

von Neumann v. Neuromorphic Computing

Neuromorphic computational models will allow computers to carry out complex operations faster, in an energy efficient manner, with fewer delays than conventional von Neumann architectures. Neuromorphic chips mimic human brains with interconnected artificial neurons and synapses. Neuromorphic architecture has come to define next-generation AI which constitutes the creation and use of neural networks as analogous electronic circuits, representing innovative non-Turing computational principles. These principles intend to imbibe and reproduce facets of continuing dynamics and computational functionality found in biological brains.

Even though the current scenario seems very dubious in terms of making the switch to neuromorphic computation from the existing von Neumann architecture in the foreseeable future, recent advancement in the development of artificial intelligence technologies with the help of deep learning and algorithms has resulted in an unprecedented exceptional revolution in neuromorphic mechanisms. With the advent of the same, the focus is shifting from paralleling the stimulation of the brain in precise detail to applying the primary organizing principle to practical devices. One recent example is the, “spike”, which are short pulses that carry information between biological neurons that have emerged as the prospective forerunners in the race of neuromorphic technologies. Spikes from hundreds of neurons are transmitted, via synapses, as inputs for another neuron, which coalesce the information to compute and fire off a spike to neurons to which it is connected.

Conclusion

The term artificial general intelligence (AGI) refers to AI that demonstrates intelligence equivalent or congruous to that of humans. Though machines have not yet successfully reached this level of intelligence, neuromorphic computing proffers promising and novel opportunities in transforming the same into a reality. The growing trend of computational heterogeneity and a steady shift towards a data-centric approach calls for more specialized non-von Neumann platforms since the impact of the same has been expected to be colossal. The same can already be observed with application in an array of avenues like speech and image recognition, autonomous vehicles and robotics, medical devices, Internet of Things (IoT), and even artificial body parts.



This Article is written by Dhwani Pandya and Pragya Sharma. Dhwani Pandya is a fifth year law student at Institute of Law, Nirma University. She is interested in International, Human Rights and Media Laws. Pragya Sharma is a fifth year law student at Institute of Law, Nirma University, specialising in Intellectual Property Laws. She has an interest in Telecommunications, Technology, and Data Privacy Laws.


Libertatem.in is now on Telegram. Follow us for regular legal updates and judgments from the court. Follow us on Google News, InstagramLinkedInFacebook & Twitter. You can also subscribe to our Weekly Email Updates. You can also contribute stories like this and help us spread awareness for a better society. Submit Your Post Now.

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Latest News

Bombay High Court Passes Order To Clarify and Modify Previous Order When State of Maharashtra Moved Praecipe

Division Bench of Bombay High Court consisting of Justice S. V. Gangapurwala and Justice Shrikant D. Kulkarni had passed an Order on 25th October...

The European Court of Human Rights Orders Germany To Pay Non-Pecuniary Damages for Prison Strip-Searches 

A serving German prisoner was repeatedly stripped searched for non-legitimate purposes. The European Court of Human Rights (ECHR) found that Germany had violated the...

Lack of Independent Witness Doesn’t Vitiate Conviction: Supreme Court

A three-judge Bench of the Supreme Court in Rajesh Dhiman v State of Himachal Pradesh clarified the law in case of lack of independent...

Madras High Court Observes Unexplained Delay in Procedural Safeguards, Quashes Detention Through Writ Petition

A Writ Petition was filed under Article 226 to issue a writ of Habeas Corpus. The petitioner P. Lakshmi, called for records of the...

UK Court of Appeal Rules Home Department’s Deportation Policy of Immigrants Unlawful

Britain’s Court of Appeal quashed the Home Department’s deportation policy, declaring it unlawful; criticizing it for being too stringent on immigrants to comply with. Background The...

Supreme Court Stays Order Restraining Physical Campaigns in the Madhya Pradesh Bye-Elections

On the 26th of October, a Bench was set up which comprised Justice AM Khanwilkar, Justice Dinesh Maheshwari, and Justice Sanjiv Khanna. They heard...

Inordinate and Unexplained Delay in Considering Representation by Government Renders Detention Order Illegal: Madras High Court

A Petition under Article 226 of the Constitution was filed in the Madras High Court to declare the detention order of the husband of...

Supreme Court Asks Petitioner to Approach Bombay High Court in PIL for CBI Probe in Disha Salian Case

On the 26th of October 2020, the Apex Court heard the PIL praying for a CBI probe into the death of Disha Salian. The...

Privy Council Clarifies Approach To Winding up in “Deadlock” Cases in the Case of Chu v. Lau

The Judicial Committee of the Privy Council clarified several aspects of the law concerning just and equitable winding-up petitions, as well as shareholder disputes...

Madras High Court Directs Hospital To Submit Necessary Medical Reports to Authorization Committee for Approval of Kidney Transplant

A Writ Petition was filed under Article 226 to issue a Writ of Mandamus to K.G. Hospital, Coimbatore by P. Sankar & V. Sobana....

More Articles Like This

- Advertisement -