TUTORIALS

Middleware and Toolkit for Next Generation Analog AI Hardware Acceleration

Abstract

The next step in the evolution of specialized hardware for AI is rooted in addressing the performance efficiency loss from data movement between computational units and memory.  This can be achieved through analog in-memory computing which eliminates the Von Neuman bottleneck and allows highly-parallel computations directly in memory using memristive crossbar arrays. Although Memristive crossbar arrays are a promising future Analog technology for accelerating AI workloads, their inherent noise and non-idealities demand for improved algorithmic solutions.

We introduce the IBM Analog Hardware Acceleration Kit [1], a first of a kind open source toolkit to simulate crossbar arrays from within PyTorch, to conveniently estimate the impact of material properties and non-idealities on the accuracy for arbitrary ANNs (freely available at this URL). This platform allows understanding, evaluating, and experimenting with emerging analog AI accelerators. Our roadmap and capabilities include algorithmic innovations from IBM Research around hardware-aware training, mixed-precision training, advanced analog training optimizers using parallel rank-update in analog, and allowing inference on real research Phase-change memory (PCM)-based analog AI chip prototypes, as well as allowing the research community to extend the toolkit with new devices, analog presets, algorithms, etc.

We will show an interactive demo of how the toolkit can be used online though our web front-end cloud composer. The composer provides a set of templates and a no-code experience to introduce the concepts of analog AI, configure experiments, and launch training experiments. We are actively working to include inference experiments in simulation and a real PCM-based analog AI chip.

[1] Malte J. Rasch, Diego Moreda, Tayfun Gokmen, Manuel Le Gallo, Fabio Carta, Cindy Goldberg, Kaoutar El Maghraoui, Abu Sebastian; Vijay Narayanan, “A Flexible and Fast PyTorch Toolkit for Simulating Training and Inference on Analog Crossbar Arrays” 2021 IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems (AICAS), 2021, pp. 1-4.

Speakers Bios

Dr. Kaoutar El Maghraoui
Dr. Kaoutar El Maghraoui is a principal research staff member at the IBM T.J Watson Research Center where she is focusing on innovations at the intersection of systems and artificial intelligence (AI). She leads the research agenda of the AI testbed of the IBM Research AI Hardware Center, a global research hub focusing on enabling next-generation accelerators and systems for AI workloads. Kaoutar has co-authored several patents, conference, and journal publications in the areas of systems research, distributed systems, high performance computing, and AI. Kaoutar holds a PhD. degree from Rensselaer Polytechnic Institute, USA. She received several awards including the Robert McNaughton Award for best thesis in computer science, Best of IBM award in 2021, IBM’s Eminence and Excellence award for leadership in increasing Women’s presence in science and technology, several IBM outstanding technical accomplishments, and 2021 IEEE TCSVC Women in Service Computing award.

Dr. Fabio Carta
Dr. Fabio Carta is a research staff member at the IBM T.J. Watson Research Center where he is focusing on AI hardware accelerator using analog devices. He has previously worked on device technologies in particular Phase Change Memory development, access device for cross-point memory technology, polysilicon diodes and polysilicon transistor for 3D integration. Fabio holds a PhD in Electrical Engineering from Columbia University, USA. He has co-authored several patents, conferences, and journal publications in the area of device technologies.

Connext DDS for Middleware Researchers: a Gentle Introduction to the Leading Distributed Connectivity Framework for Autonomous Systems

Abstract

Are you working on an edge computing, publish/subscribe, or real-time systems middleware research prototype? Maybe you’re exploring robotics and looking to open up the hood on ROS2 to tinker with the underlying DDS implementation? Or you’re trying to measure distributed communications performance, and you dread the idea of debugging a hand-written distributed application protocol and telemetry code? Well, you might consider a data-centric approach using the open standard Data Distribution Service (DDS). And if you do, RTI is proud to offer FREE SOFTWARE LICENSES for our industry leading DDS-based distributed connectivity framework to academic researchers: https://www.rti.com/free-trial/university-program

Rather than getting bogged down in network programming, cross-platform complexity, and asynchronous handler headaches, come to this tutorial and learn how to employ RTI Connext DDS instead so you can focus more on the research! We’ll show you how to quickly define your data models, topics, application data requirements, and detailed protocol QoS configurations using an intuitive web-based interface. You will learn how to auto-generate compilable example code for your platform of choice or dynamically load the XML data type into your favorite scripting language. Topics covered will range from the basics of DDS to briefly touching on advanced topics that will help you configure the network protocol according to your unique requirements. We will specifically cover some measurement instrumentation features useful for performance studies and visually monitoring live systems. Throughout the tutorial, we will relate the content to one of our hottest real-world customer markets - autonomous vehicles. We will also discuss why the open DDS standard is trusted by mission-critical industries from healthcare to aerospace & defense. For those early in their research careers, we will end with a discussion about ongoing efforts at RTI Research and some open research topics relevant to real-time distributed systems e.g. 5G, SDN, software safety certification, real-time WAN technology, etc.

Biography

Kyle Benson received his PhD in Computer Science from the University of California, Irvine and then joined RTI as a Research Engineer in 2019. His dissertation, entitled “Resilient Communications Middleware for IoT Data Exchange”, explored the use of edge computing and SDN to improve communications between constrained devices during challenging scenarios such as network disruptions. His current areas of research at RTI include semantic data compression, data-centric edge communications leveraging 5G technology, and hybrid LVC (live, virtual, & constructive) technology that leverages the same DDS-based applications as real-world systems for more realistic modeling, simulation, and training systems.