Fragmented Neural Networks for Practical Deep Learning

In an article published in the journal Scientific Reports, researchers from China proposed a novel method for constructing deep neural networks using fragmented images and ensemble learning algorithms. They mentioned that their technique could provide an accessible and efficient approach for practical applications by reducing technical difficulty and hardware requirements in deep learning. Moreover, they tested the method on two image classification datasets, achieving comparable or even better accuracy than full models trained with complete data.

Study: Fragmented Neural Networks for Practical Deep Learning. Image credit: Runa0410/Shutterstock
Study: Fragmented Neural Networks for Practical Deep Learning. Image credit: Runa0410/Shutterstock

Background

A deep neural network learns and makes decisions by mimicking the way the human brain works. It uses layers of interconnected nodes to process information, identify patterns, and perform tasks like image recognition or language understanding. Moreover, it evolved rapidly in engineering applications, with models becoming deeper and more specialized. However, developing large models for small and medium enterprises is costly and risky, and most businesses do not need high-level artificial intelligence (AI).

Ensemble learning is an effective technique that combines multiple base learners to obtain a stronger learner. It can improve the accuracy and robustness of the models, as well as the diversity and generalization of the learners. However, many ensemble methods prioritize enhancing test accuracy by designing distinct ensemble architectures without considering the enormous computation and design costs. These performance-oriented ensembles are not well suited for practical business applications.

About the Research

In the present paper, the authors presented a fragmented neural network inspired by the random forest machine learning model. Random forest is a classical ensemble learning algorithm that makes more than one decision tree in parallel based on feature sampling. Its advantages include high accuracy, low complexity, easy parallelism, and natural distributed computing.

The designed model applies the same sampling strategy and randomly samples the samples and features on image data. Images were split into small pieces, and weak neural networks were trained using these images. Using a voting technique, many weak neural networks were then connected to form a strong neural network. In this way, sufficient accuracy is achieved by reducing the complexity and the data volume needed for each base learner.

The authors conducted experiments using two classic image classification datasets, including the modified National Institute of Standards and Technology (MNIST) and the Canadian Institute for Advanced Research 10 (CIFAR-10). The MNIST dataset comprised 70,000 images of handwritten digits from 0 to 9, whereas the CIFAR10 encompassed 60,000 images of 10 classes of objects, such as airplanes, cars, and animals.

The study used feedforward neural networks (FNN), convolutional neural networks (CNN), and deep residual networks (ResNet) as the base learners. For each type of network, 20 to 30 base learners were trained with different sampling window sizes and sample sizes. The sampling window size indicates the number of retained original images, and the sample size shows the number of images used for each base learner. Furthermore, unweighted voting, majority voting, winner-takes-all-based unweighted voting, and winner-takes-all-based majority voting strategies were used to combine the predictions of the base learners.

Research Findings

The outcomes showed that the newly designed method can effectively improve the accuracy and stability of the ensemble model compared to the base learner. The accuracy of the combined model is highly dependent on the base learners, and the complex base learners do not have a significant advantage over the simple base learners. The study highlighted that the new method achieved comparable or even better accuracy compared to the full models trained with full images. The presented model is more sensitive to the sampling window size than the sample size and the number of base learners, and the optimal window size varies depending on the type of neural network.

The authors also analyzed the effects of different parameters, such as the sample size, the sampling window size, and the number of base learners, on the accuracy and time consumption of the ensemble model. They highlighted that increasing the sample size and the number of base learners can improve accuracy but also increase the time and cost. Increasing the sampling window size can improve the accuracy of the CNN and ResNet models but not the FNN model. Moreover, they found that unweighted voting has the highest average accuracy among the four voting methods but also requires more time compared to other methods.

The proposed method has potential applications in various fields that require image classification, such as face recognition, object detection, medical diagnosis, and remote sensing. It can enable parallel and distributed computing to improve the computational speed and scalability of the models. Moreover, it can enhance the robustness and generalizability of the models, which can handle noise and uncertainty better.

Conclusion

In summary, the developed neural network is efficient, effective, robust, and stable for building deep neural networks using fragmented images and ensemble learning. It works on the principle of random forest and aims to reduce the technical difficulty and hardware needed for deep learning. Moreover, it was tested on MNIST and CIFAR10 image classification datasets and achieved comparable or even better accuracy than the full models trained with full data.

The researchers acknowledged limitations and challenges and suggested directions for further research. They recommended future work to develop a control device for dynamic model pools, facilitating operations within the pool. Moreover, they advised applying the proposed method in federated learning scenarios, where each data node can train a local model and transfer it to the central node for ensemble decision-making.

Journal reference:
Muhammad Osama

Written by

Muhammad Osama

Muhammad Osama is a full-time data analytics consultant and freelance technical writer based in Delhi, India. He specializes in transforming complex technical concepts into accessible content. He has a Bachelor of Technology in Mechanical Engineering with specialization in AI & Robotics from Galgotias University, India, and he has extensive experience in technical content writing, data science and analytics, and artificial intelligence.

Citations

Please use one of the following formats to cite this article in your essay, paper or report:

  • APA

    Osama, Muhammad. (2024, February 05). Fragmented Neural Networks for Practical Deep Learning. AZoAi. Retrieved on April 16, 2024 from https://www.azoai.com/news/20240205/Fragmented-Neural-Networks-for-Practical-Deep-Learning.aspx.

  • MLA

    Osama, Muhammad. "Fragmented Neural Networks for Practical Deep Learning". AZoAi. 16 April 2024. <https://www.azoai.com/news/20240205/Fragmented-Neural-Networks-for-Practical-Deep-Learning.aspx>.

  • Chicago

    Osama, Muhammad. "Fragmented Neural Networks for Practical Deep Learning". AZoAi. https://www.azoai.com/news/20240205/Fragmented-Neural-Networks-for-Practical-Deep-Learning.aspx. (accessed April 16, 2024).

  • Harvard

    Osama, Muhammad. 2024. Fragmented Neural Networks for Practical Deep Learning. AZoAi, viewed 16 April 2024, https://www.azoai.com/news/20240205/Fragmented-Neural-Networks-for-Practical-Deep-Learning.aspx.

Comments

The opinions expressed here are the views of the writer and do not necessarily reflect the views and opinions of AZoAi.
Post a new comment
Post

While we only use edited and approved content for Azthena answers, it may on occasions provide incorrect responses. Please confirm any data provided with the related suppliers or authors. We do not provide medical advice, if you search for medical information you must always consult a medical professional before acting on any information provided.

Your questions, but not your email details will be shared with OpenAI and retained for 30 days in accordance with their privacy principles.

Please do not ask questions that use sensitive or confidential information.

Read the full Terms & Conditions.

You might also like...
Deep Learning Cleans Up Noise in X-ray Data Analysis