What I am being paid to do in Switzerland is work as a summer research intern in the Microelectronic Systems Lab. It’s a huge lab with many areas of research, which can be summarized as relating to integrated chip design as well as lower level (less abstraction, more physics) material research and higher level (more abstraction, less physics) circuit research. On the higher end of that scale is the lab’s work relating to neuromorphic (biologically inspired) circuits, and in particular artificial neural nets (ANN) implemented in hardware – the topic that piqued my interest. Perhaps, though really probably not but lets go with it, you are not sure why we should bother with implementing neural net in hardware if we can already use them in software. Well, theoretical curious person, I think we should bother because our brains are computers, albeit of a completely different biological architecture from the computer you are using to read these words right now, and so I can’t help but wonder if we can’t emulate this computer in hardware rather than software. As it is, we are essentially performing drastically inefficient simulations of one computer on another entirely different computer. No wonder neural nets take forever to run. This, as well as the less lofty idea that it would be really cool to have a robot with an embedded ANN chip (a brain on a chip in a robot!), made me apply to the internship and for some reason I got hired.
Before my first day at work, I was vaguely told it’d involve simulating neural nets in C or C++. Then I got to work, and found about the only specific decision made about my summer project is that it will involve simulating memristors. No C or C++ for me. But, at least it turns out memristors are pretty neat. You know those fundamental electrical components, the good old resistor, conductor, and inductor? The resistor relates voltage to current, the inductor relates flux to current, and the conductor relates voltage to charge. Well, a while back Leon Chua published a seminal paper cutely titled “Memristor – The Missing Circuit Element.” This paper postulated that there should be some a fourth basic type of component that relates charge to flux (you don’t want to relate charge to current or voltage to flux, since those are already related by a derivative), because nature should be all nice and symmetric. A couple of decades later, in 2008, some researchers from HP published a paper even more cutely titled “The Missing Memristor Found,” detailing how a range of nano devices exhibit characteristics that appear to match the theoretical components that Chua postulated should exist. According to Google Scholar, the paper has been cited 2210 time since then. Two thousand two hundred and ten citations. In six years.
Why? Well, although we don’t actually know how they work physically and have a host of mathematical models that seem to fit, the big deal is that they have a variable resistance. When current flows through a memristor with a certain voltage, its resistance either increases or decreases (with bounding maximum and minimum resistance) based on the direction of the current, and when the current stops that resistance is stored. Thus, memory resistor. What’s more, this behavior occurs only in nanoscale materials, and we have nice fabrication techniques to arrange a whole bunch of memristors in a dense grid structure called a crossbar. Humorously, the pinched hysteresis loop effect had been observed in materials decades ago and the idea of a crossbar as a nano architecture has existed for at least a decade, but it was not until HP’s landmark 2008 paper that the newly dubbed memristors became a Big Deal. “How We Found The Missing Memristor ” is an accessible and interesting retrospective on how HP got there.
So, what are those 2210 papers actually about? The short version is that memristors can be used to: improve CMOS circuit density, provide dense and low power memory, compute logical functions, and act as synapses in neuromorphic circuits. The long version is all the stuff I have in my head after reading or skimming approximately 100 papers about those topics in the last several weeks. I have the Mendeley library to back that claim by, by the way. In the interest of not trying to summarize approximately 100 papers in this blog post, how about I let you take all that in via the most digestible of formats:
Among the most exciting, and sensationalist, papers covered in the above presentation is “Universal Memcomputing Machines” by Fabio L. Traversa and Massimiliano Di Ventra. The exciting part is that it proposes that computing with the same elements that store data, or memcomputing, is fundamentally different and more powerful than the conventional way our Von Neumann Architecture computers work today. It also proposes the mathematical definition of a Memcomputing Machine akin to the Turing Machine model on which most existing computers are based, and proves the former to be able to solve a class of Hard problems much faster than the latter. It is sensationalist because of the stuff you should have seen and been flabbergasted by in those wonderful slides: the quotation from “Why Memristors Wont Change Anything,” and the brief argument that it is modeling parallel computing in general rather than memcomputing specifically.
So, what am I being paid to do in Switzerland? Let’s review: on my first day, I learned about memristors and that I will need to model them using VerilogA, and do… something with them. No C or C++ for me. I did this within the first week, more or less. On my second day I went to a talk by Professor Di Venta about his exciting new UMM model, and on my forth I sat in on a meeting with Professor Di Ventra about what could be a good focus for my research. The conclusions were vague, but the next day I talked with my postdoc supervisor and settled on trying to actually implement the hypothesized memcomputing hardware that could solve the Hard subset sum problem fast.
Except, that is no longer what I am being to do in Switzerland. The next week I read all existing papers about implementing computation with memristors, and simulated some of the basic blocks to verify they could work. But there was a problem. It is immensely non-trivial to devise an actual computer architecture that is analogous to the UMM model in the way the Von Neuman Architecture is analogous to the Turing Machine. So, I went back to reading in search of relevant papers, and kept my mind spinning about the problem. Two weeks after the meeting with Di Ventra, after several conversations with my supervisor, just as I was almost done writing an email what I had done and about how I was not sure how to proceed, my professor emailed me saying he thought my work could be focused on neural nets.
And the sad part is, just as I walked back home that day and was struck with an idea for a possible approach to designing the architecture for memcomputing, it hit me – the architecture was not, could not be, fundamentally novel. Any separation of computation and memory, even if they are kept very close, is still fundamentally the same as modern processors regardless of how the basic logic is implemented. Memprocessors would certainly have to be distinct, and furthermore have distinct sections for computation and data storage, to implement the proposed subset sum solution. This architecture would certainly be non Von Neumann, but it would be strongly related to a host of older parallel computing concepts: distributed computing with systolic arrays, COMA, and vector processing. There have even been some crazy computers developed based on these concepts. I briefly enjoyed the feeling I had accomplished some insight, before doing yet more reading about memristor as synapses in ANNs.
So, I am being paid to work on simulating memristor-based neuromorphic circuits, with an as of yet undetermined novel end goal. In Switzerland