Curated by THEOUTPOST
On Tue, 29 Oct, 12:08 AM UTC
2 Sources
[1]
Magnetic RAM-based architecture could pave way for implementing neural networks on edge IoT devices
There are, without a doubt, two broad technological fields that have been developing at an increasingly fast pace over the past decade: artificial intelligence (AI) and the Internet of Things (IoT). By excelling at tasks such as data analysis, image recognition, and natural language processing, AI systems have become undeniably powerful tools in both academic and industry settings. Meanwhile, miniaturization and advances in electronics have made it possible to massively reduce the size of functional devices capable of connecting to the Internet. Engineers and researchers alike foresee a world where IoT devices are ubiquitous, comprising the foundation of a highly interconnected world. However, bringing AI capabilities to IoT edge devices presents a significant challenge. Artificial neural networks (ANNs) -- one of the most important AI technologies -- require substantial computational resources, and IoT edge devices are inherently small, with limited power, processing speed, and circuit space. Developing ANNs that can efficiently learn, deploy, and operate on edge devices is a major hurdle. In response, Professor Takayuki Kawahara and Yuya Fujiwara from the Tokyo University of Science, are working hard toward finding elegant solutions to this challenge. In their latest study published in IEEE Access on October 08, 2024, they introduced a novel training algorithm for a special type of ANN called binarized neural network (BNN), as well as an innovative implementation of this algorithm in a cutting-edge computing-in-memory (CiM) architecture suitable for IoT devices. "BNNs are ANNs that employ weights and activation values of only -1 and +1, and they can minimize the computing resources required by the network by reducing the smallest unit of information to just one bit," explains Kawahara. "However, although weights and activation values can be stored in a single bit during inference, weights and gradients are real numbers during learning, and most calculations performed during learning are real number calculations as well. For this reason, it has been difficult to provide learning capabilities to BNNs on the IoT edge side." To overcome this, the researchers developed a new training algorithm called ternarized gradient BNN (TGBNN), featuring three key innovations. First, they employed ternary gradients during training, while keeping weights and activations binary. Second, they enhanced the Straight Through Estimator (STE), improving the control of gradient backpropagation to ensure efficient learning. Third, they adopted a probabilistic approach for updating parameters by leveraging the behavior of MRAM cells. Afterwards, the research team implemented this novel TGBNN algorithm in a CiM architecture -- a modern design paradigm where calculations are performed directly in memory, rather than in a dedicated processor, to save circuit space and power. To realize this, they developed a completely new XNOR logic gate as the building block for a Magnetic Random Access Memory (MRAM) array. This gate uses a magnetic tunnel junction to store information in its magnetization state. To change the stored value of an individual MRAM cell, the researchers leveraged two different mechanisms. The first was spin-orbit torque -- the force that occurs when an electron spin current is injected into a material. The second was voltage-controlled magnetic anisotropy, which refers to the manipulation of the energy barrier that exists between different magnetic states in a material. Thanks to these methods, the size of the product-of-sum calculation circuit was reduced to half of that of conventional units. The team tested the performance of their proposed MRAM-based CiM system for BNNs using the MNIST handwriting dataset, which contains images of individual handwritten digits that ANNs have to recognize. "The results showed that our ternarized gradient BNN achieved an accuracy of over 88% using Error-Correcting Output Codes (ECOC)-based learning, while matching the accuracy of regular BNNs with the same structure and achieving faster convergence during training," notes Kawahara. "We believe our design will enable efficient BNNs on edge devices, preserving their ability to learn and adapt." This breakthrough could pave the way to powerful IoT devices capable of leveraging AI to a greater extent. This has notable implications for many rapidly developing fields. For example, wearable health monitoring devices could become more efficient, smaller, and reliable without requiring cloud connectivity at all times to function. Similarly, smart houses would be able to perform more complex tasks and operate in a more responsive way. Across these and all other possible use cases, the proposed design could also reduce energy consumption, thus contributing to sustainability goals.
[2]
Towards implementing neural networks on edge IoT devices
There are, without a doubt, two broad technological fields that have been developing at an increasingly fast pace over the past decade: artificial intelligence (AI) and the Internet of Things (IoT). By excelling at tasks such as data analysis, image recognition, and natural language processing, AI systems have become undeniably powerful tools in both academic and industry settings. Meanwhile, miniaturization and advances in electronics have made it possible to massively reduce the size of functional devices capable of connecting to the Internet. Engineers and researchers alike foresee a world where IoT devices are ubiquitous, comprising the foundation of a highly interconnected world. However, bringing AI capabilities to IoT edge devices presents a significant challenge. Artificial neural networks (ANNs) -- one of the most important AI technologies -- require substantial computational resources. Meanwhile, IoT edge devices are inherently small, with limited power, processing speed, and circuit space. Developing ANNs that can efficiently learn, deploy, and operate on edge devices is a major hurdle. In response, Professor Takayuki Kawahara and Mr. Yuya Fujiwara from the Tokyo University of Science, are working hard towards finding elegant solutions to this challenge. In their latest study published in IEEE Access on October 08, 2024, they introduced a novel training algorithm for a special type of ANN called binarized neural network (BNN), as well as an innovative implementation of this algorithm in a cutting-edge computing-in-memory (CiM) architecture suitable for IoT devices. "BNNs are ANNs that employ weights and activation values of only -1 and +1, and they can minimize the computing resources required by the network by reducing the smallest unit of information to just one bit," explains Kawahara, "However, although weights and activation values can be stored in a single bit during inference, weights and gradients are real numbers during learning, and most calculations performed during learning are real number calculations as well. For this reason, it has been difficult to provide learning capabilities to BNNs on the IoT edge side." To overcome this, the researchers developed a new training algorithm called ternarized gradient BNN (TGBNN), featuring three key innovations. First, it employs ternary gradients during training, while keeping weights and activations binary. Second, they enhanced the Straight Through Estimator (STE), improving the control of gradient backpropagation to ensure efficient learning. Third, they adopted a probabilistic approach for updating parameters by leveraging the behavior of MRAM cells. Afterwards, the research team implemented this novel TGBNN algorithm in a CiM architecture -- a modern design paradigm where calculations are performed directly in memory, rather than in a dedicated processor, to save circuit space and power. To realize this, they developed a completely new XNOR logic gate as the building block for a Magnetic Random Access Memory (MRAM) array. This gate uses a magnetic tunnel junction to store information in its magnetization state. To change the stored value of an individual MRAM cell, the researchers leveraged two different mechanisms. The first was spin-orbit torque -- the force that occurs when an electron spin current is injected into a material. The second was voltage-controlled magnetic anisotropy, which refers to the manipulation of the energy barrier that exists between different magnetic states in a material. Thanks to these methods, the size of the product-of-sum calculation circuit was reduced to half of that of conventional units. The team tested the performance of their proposed MRAM-based CiM system for BNNs using the MNIST handwriting dataset, which contains images of individual handwritten digits that ANNs have to recognize. "The results showed that our ternarized gradient BNN achieved an accuracy of over 88% using Error-Correcting Output Codes (ECOC)-based learning, while matching the accuracy of regular BNNs with the same structure and achieving faster convergence during training," notes Kawahara. "We believe our design will enable efficient BNNs on edge devices, preserving their ability to learn and adapt." This breakthrough could pave the way to powerful IoT devices capable of leveraging AI to a greater extent. This has notable implications for many rapidly developing fields. For example, wearable health monitoring devices could become more efficient, smaller, and reliable without requiring cloud connectivity at all times to function. Similarly, smart houses would be able to perform more complex tasks and operate in a more responsive way. Across these and all other possible use cases, the proposed design could also reduce energy consumption, thus contributing to sustainability goals.
Share
Share
Copy Link
Researchers from Tokyo University of Science develop a new training algorithm and computing-in-memory architecture using Magnetic RAM, potentially enabling efficient implementation of neural networks on IoT edge devices.
As artificial intelligence (AI) and the Internet of Things (IoT) continue to advance rapidly, researchers face a significant challenge: implementing AI capabilities, particularly artificial neural networks (ANNs), on small IoT edge devices with limited resources 12. These devices typically have constraints in power, processing speed, and circuit space, making it difficult to run computationally intensive AI algorithms efficiently.
To address this challenge, Professor Takayuki Kawahara and Yuya Fujiwara from the Tokyo University of Science have developed a novel training algorithm called ternarized gradient binarized neural network (TGBNN) 12. This algorithm builds upon binarized neural networks (BNNs), which use only -1 and +1 for weights and activation values, reducing the smallest unit of information to one bit.
The TGBNN algorithm introduces three key innovations:
The researchers implemented the TGBNN algorithm in a novel computing-in-memory (CiM) architecture, designed specifically for IoT devices 12. This approach performs calculations directly in memory, saving circuit space and power. The team developed a new XNOR logic gate as the building block for a Magnetic Random Access Memory (MRAM) array, using a magnetic tunnel junction to store information.
To change the stored value of individual MRAM cells, the researchers utilized two mechanisms 12:
These methods allowed the team to reduce the size of the product-of-sum calculation circuit to half that of conventional units.
The researchers tested their MRAM-based CiM system for BNNs using the MNIST handwriting dataset 12. The results were impressive:
This breakthrough could lead to more powerful IoT devices with enhanced AI capabilities 12. Potential applications include:
The innovative MRAM-based architecture and TGBNN algorithm represent a significant step towards implementing efficient neural networks on edge IoT devices, potentially revolutionizing the integration of AI and IoT technologies.
Reference
[1]
[2]
Researchers at Tokyo University of Science have developed a groundbreaking dye-sensitized solar cell-based device that mimics human synaptic behavior, offering efficient edge AI processing for various applications while consuming significantly less power.
2 Sources
2 Sources
Researchers at the National University of Singapore have developed a revolutionary silicon transistor that can function like both a neuron and a synapse, potentially transforming the field of neuromorphic computing and AI hardware efficiency.
3 Sources
3 Sources
A comprehensive review published in Nature outlines the path to scale up neuromorphic computing, aiming to rival current computing methods in efficiency and effectiveness for AI applications.
3 Sources
3 Sources
MIT researchers have created a new photonic chip that can perform all key computations of a deep neural network optically, achieving ultrafast speeds and high energy efficiency. This breakthrough could revolutionize AI applications in various fields.
4 Sources
4 Sources
Researchers at the Indian Institute of Science (IISc) Bengaluru have created a groundbreaking 'brain-on-a-chip' technology that mimics human brain functions. This innovation promises to revolutionize computing and artificial intelligence applications.
5 Sources
5 Sources
The Outpost is a comprehensive collection of curated artificial intelligence software tools that cater to the needs of small business owners, bloggers, artists, musicians, entrepreneurs, marketers, writers, and researchers.
© 2025 TheOutpost.AI All rights reserved