Abstract
With the growing need of technology into varied fields, dependency is getting directly proportional to ease of user-friendly smart systems. The advent of artificial intelligence in these smart systems has made our lives easier. Several Internet of Things- (IoT-) based smart refrigerator systems are emerging which support self-monitoring of contents, but the systems lack to achieve the optimized run time and data security. Therefore, in this research, a novel design is implemented with the hardware level of integration of equipment with a more sophisticated software design. It was attempted to design a new smart refrigerator system, which has the capability of automatic self-checking and self-purchasing, by integrating smart mobile device applications and IoT technology with minimal human intervention carried through Blynk application on a mobile phone. The proposed system automatically makes periodic checks and then waits for the owner’s decision to either allow the system to repurchase these products via Ethernet or reject the purchase option. The paper also discussed the machine level integration with artificial intelligence by considering several features and implemented state-of-the-art machine learning classifiers to give automatic decisions. The blockchain technology is cohesively combined to store and propagate data for the sake of data security and privacy concerns. In combination with IoT devices, machine learning, and blockchain technology, the proposed model of the paper can provide a more comprehensive and valuable feedback-driven system. The experiments have been performed and evaluated using several information retrieval metrics using visualization tools. Therefore, our proposed intelligent system will save effort, time, and money which helps us to have an easier, faster, and healthier lifestyle.
1. Introduction
Over the past decade, the number of Internet users has increased rapidly [1], and the IoT is one of the latest and newest Internet technologies which is capable of providing great assistance to automated systems utilized daily everywhere around the world [2, 3]. The intelligent innovation that has enabled and connected the intelligent interaction of discrete constructs for elaborating the immense range of services to entities is IoT [4]. Referring to smart homes and sustainability, intelligent devices are different representative emerging technologies in the IoT era. Smart home is one of the leading development which has changed house equipment into being more intelligent [5–7]. Generally, a smart home consists of several smart appliances [8], which are able to collect and exchange data over the Internet and connect to each other through various protocols (like Bluetooth, Wi-Fi, and Ethernet) that can work independently and interactively to perform specific functions [9, 10]. To this end, computers or applications are utilized in order to manage the functions and features of a main appliance remotely via the Internet [11]. At a smart home, the kitchen is the most likely place to house smart devices [12], and the refrigerator is one of the most important appliances in the kitchen [13–15].
In the past years, several works have been proposed to design IoT-based smart refrigerator systems [16–19]. An IoT-based low-cost smart refrigerator is presented in [20] which is designed in order to reduce the food loss and help in providing a more comfortable life. The application given by Android or Mac applications has given a real-time update even though the individual is not physically present at home. In such scheme, the data of the fresh food is inserted in the refrigerator by taking a picture of the food. In [21], a smart refrigerator system is presented which suggests the recipe information as per availability of vegetables which includes inside the refrigerator and processes it to know about vegetable status, temperature, and vegetable items for the recipes that need to be purchased if not available inside the refrigerator. The suggested system is able to determine the weight of the vegetables and predict the dishes which can be cooked in available material. Moreover, the main goal of the system proposed in [18] was to warn the user when a food is over or a week before shelf life of the food. The invention continued to design and develop with features of sensing the quantity and quality of food items inside the fridge [22]. The main aim behind this work was to inform the user about the current status of food items through an android app on a mobile phone. The main achievement of that system was to save money and reduce the food wastage. In addition, the authors focused on developing a low-cost smart refrigerator system in [23] which is aimed at achieving a minimum number of intelligent decision making. Likewise, in order to detect the available items inside the refrigerator and make a decision, deep learning methods are used for training the model in this paper. Such scheme could improve the precision performance with better computational needs. People’s lives are becoming ever busier, and few have time (or perhaps forget) to check the stock of food and beverages inside the household refrigerator. Visits to the supermarket to replenish the refrigerator are also time-consuming. This lack of available time may lead to changes in cooking habits, waste of a lot of food, or an inconvenient lack of a foodstuff when needed. Having a system that can periodically, automatically, and immediately check, order, and purchase depleted products may solve these problems. However, based on the literature review, most of the previous smart refrigerators can only inform the consumers about the available and required stocks in the refrigerator without being able to purchase the depleted products. Moreover, previous studies have not simultaneously considered all criteria of a high-quality smart refrigerator with the intelligent techniques and privacy concerns for security reasons. Even integration of varied techniques including combination of IoT, artificial intelligence (AI), and blockchain is lacking in the previously designed systems. This work attempts to combines the abovementioned technologies as they have the potential to improve supply and usage efficiency, information traceability, smart consumption, and domestic logistics, among other things. On each level of usage and consumption, the item/product will be well linked to the resources and products. AI can assess the conditions for ordering, cooling, etc., and machine level changes using IoT are also considered. This novel design, with the IOT-block, an intelligent refrigerator system, will overcome the previous limitations. Majority of time taken for running a machine learning model is taken in structuring and analysis of the data and features. Using the blockchain technology will ensure that the data is stored in separate blocks and when a particular dataset is required, respective machine learning model will call the data using functions from block chain. This method will significantly reduce the running time.
The proposed system has the capability of automatically self-checking and self-purchasing (after receiving owner confirmation), by integrating smart mobile device applications and IoT technology with minimal human intervention. It is smart enough to check the availability and quantity of foods and beverages stored inside the refrigerator. When the stock of a product reaches a half, a quarter, and then zero, it will automatically notify the owner by sending an SMS message, an email, and an alert to the system’s application. To this end, the proposed smart refrigerator is connected via Ethernet module to Blynk, which is a platform in iOS and Android apps used to connect to Arduino, Raspberry Pi, etc. Blynk mediates all communications between the smartphone and the hardware. An Arduino is connected to the Internet and will be considered as the central server. An application programming interface (API) in the mobile device and website is used for obtaining live updates of the device’s status.
The machine is also integrated with artificial intelligence (AI); by IOT hardware, it collects data from the refrigerator, and this collected data goes into the cloud database. The data is ingested into the preprocessing machine where data preprocess and feature selection are done. The automatic decision related to factors is done by experimenting with different machine learning and deep learning classifiers.
Moreover, our smart fridge uses a load cell sensor to record the weight of a product each time the product is taken, used, and then returned. The system gives the owner control of the purchase operation via an application on a mobile phone from anywhere in the world via the Internet. This electronic system can complete all tasks while people are busy with other activities, thereby saving effort, time, and money; minimizing food waste; and helping us to live an easier, faster, and healthier lifestyle. This will benefit in a number of ways such as periodically checking the availability and quantity of products such as foods and beverage; notifying the consumer immediately once the stock of a product has reached a half, a quarter, and zero via SMS, email, and its own application; and offering the consumer an opportunity to decide and control the purchase operation. This will save customer’s effort, time, and money; minimize waste of food and beverages; and help people to live an easier, faster, and healthier lifestyle.
2. Motivation and Contributions
The development of IOT has given machines a way to communicate. IOT is used to get feedback from hardware and create a channel to make devices communicate among themselves. Here, the use of blockchain is envisaged to create a secure method of data transfer and an effective data retrieval process for our machine learning model. The main contributions of the paper are as follows: (i)To study the in-depth literature related to microprocessors and smart refrigerator systems(ii)To integrate the hardware technology with more sophisticated software using AI for automatic decisions with different opportunities to decide and control the purchase operation(iii)To develop the system architecture underlying the hardware process(iv)To implement a blockchain architecture for storing data. This will work as a middleware between the hardware and ML layer, with integration of blockchain framework(v)To implement the system using machine learning methodologies and comparison is made using accuracy of the classifiers
The structure of this paper is as follows. Section 2 reviews related works. Section 3 illustrates the utilized material in this study. In Section 4, the proposed method is illustrated, followed by Section 5 which presents the system testing of this approach, while the conclusion is discussed in Section 6.
3. Background
In the past years, several smart refrigerator systems have been presented. In this section, current systems and the benefits of the proposed system will be analysed. Also, the criteria of the current systems and our proposed system will be compared, in terms of (i) sending notifications, email, and SMS; (ii) the methodology of knowing the status of foods and liquid products; (iii) checking the stock of products; (iv) the language being used; (v) the percentage of human intervention; (vi) the system platform (control unit), sensors, wireless, and server used; (vii) the ability to send orders to the supermarket; (viii) payment methods; (ix) the ability to control the system remotely; (x) the OS; and (xi) the technique that is used. The blockchain and AI are discussed in the next section.
3.1. Related Work
An I-Fridge using RFID technology is presented in [24] which enables the consumer to efficiently and accurately manage the products stored inside the refrigerator. It uses Radio-Frequency Identification (RFID) technology to collect and identify information about the products stored in the refrigerator, via attached RFID tags. In that system, detailed information and descriptions of the food products can be monitored. Moreover, the customer will be notified when a product is about to expire or to be depleted. The I-Fridge system consists of a normal fridge with an RFID system, a tablet PC, an Alien9900 reader, and two Alien-9611 circular polarized antennas inside the refrigerator. An RFID tag is attached to each item stored in the refrigerator, and this tag communicates with the other elements of the system. In I-Fridge, a tablet PC is placed on the door of the refrigerator and works as the user interface. The RFID reader scans the tags inside the refrigerator via the antennas each time the door is closed, collecting the information from the tags’ IDs. This information is then transmitted from the reader to the tablet PC via the Wi-Fi channel. By making the comparison between the new results of scanned tags and the previous scanned tag set, the system can monitor the items entering and leaving the refrigerator. The applications running on the tablet PC can locate the foods in the fridge effectively and accurately [24]. By investigating the I-Fridge system, we found that one basic problem is that the collected information needs to be checked by the consumer, with no alarm or notification system in place. This markedly reduces the usefulness of the system. That is, since the system only displays the collected information, the consumer needs to manually check the location of food and its expiry date by looking at the tablet PC personally and then must buy the food personally. The system can only be used for foods, and it does not support other languages, like Arabic.
In [25], an IoT-based intelligent home using smart devices (refrigerator) has been introduced which is comprised of different sensors, transceivers, a microcontroller, and a Raspberry Pi module which controls the system. This smart fridge system works on the actual quantity of daily required products in the refrigerator. It uses a load cell sensor to measure the quantity of products in refrigerator, and, when the quantity of current products (e.g., eggs and milk) falls below a set limit, it informs the consumer via an LCD display. When a product has completely run out, the auto mode is on, after which an automatic message of daily needs is submitted to the supermarket by SMS (Short Message Service). This smart device is connected via wireless, and a Raspberry Pi is utilized as a central server. This system can be controlled through a web browser to monitor, control, and manage. The information is submitted to and received from the central server via the Internet or the cloud. An Android application has been developed for remote control of the system (on/off) from anywhere across the globe that has Internet access. This system lacks many of the services provided by ours. For example, the system displays the quantity of a product on the LCD only when it has run out, the customer must check the stock of food and drinks by looking at the LCD personally, the customer must manually check the expiry dates, there is no customer control of purchase action, the system does not alert the user that a product has run out, the system does not inform the user that it is going to send an order to the supermarket, and it does not support other languages, like Arabic.
An IoT-based smart refrigerator system is presented in [22] which uses sensing technology to monitor the quantity and quality (expiry date) of the food products stored inside it. The complete system is controlled by an ARM microcontroller, where a load cell is used as an input to the microcontroller and Wi-Fi transmits all information to an Android phone via IoT. It uses the load cell sensor, to measure the quantity of a product in the refrigerator. When that product is about to run out, an alert notification is sent to the user’s mobile. This system notifies the consumer about the current status of food products via an Android application and will remind the consumer before a product is about to expire. The Android app is used as a GUI for the user to see the status of the food products kept inside the refrigerator. Again, this system is missing many of the services provided by ours. For example, the customer buys the food personally and checks the quantity and quality of food (no liquids), and it does not support other languages, like Arabic.
Authors attempt to present an intelligent refrigerator with monitoring capability through the Internet which is able to automatically keep track items by detecting an empty space in the refrigerator [26]. This smart refrigerator uses three sensors. Each sensor represents a product (e.g., tomatoes, milk, strawberries). Each sensor contains an IR sensor. When the product is about to run out, the sensor sends a signal to the ARM processor which sends an order to the supermarket. The system consists of an LCD screen and a buzzer on the door of the refrigerator. When a product runs out, the LCD screen shows a message and the buzzer sounds to notify residents. Also, the system will automatically inform the owner by sending a Short Message Service (SMS) to the user. After that, the system sends an order to the supermarket via the fast Ethernet network. Once more, this system is missing many of the features provided by ours. The system monitors products but does not weigh them (the milk bottle may be empty), customer must check the expiry dates personally (the system does not monitor expiry dates), there is no user control of purchase action, IR cannot detect products that are transparent (like water), and the system does not inform the user that it is going to send an order to the supermarket.
A smart refrigerator using Internet of Things is presented in [27] which can sense and monitor the products stored inside it. It uses two kinds of sensors, a pressure sensor which triggers a notification to the user as soon as the applied pressure drops below 0.5 kg (so the consumer is notified every time three eggs are used) and light-dependent resistors (LDRs) which monitor and detect the level of milk or soft drink, for example, in a container. Each bottle has its own LDR. Also, LDR sensors are placed alongside a counter, which can be used to sense the number of eggs on a refrigerator shelf. The core functionality of this smart fridge is to provide a list of food products which might want to be purchased once they have run out. This system is also able to remotely notify the user when a product is about to run out, via SMS (Short Message Service) and email. The message contains information about the product which is about to run out and contains a link which facilitates online purchasing of that specific product. This system, too, is missing many of the services provided by ours. The system does monitor expiry dates, the customer must buy the food personally, and the system does not support other languages, like Arabic.
A typical Internet of Things- (IoT-) based refrigerator has been designed in [28] which features an in-built wide-angle camera that takes a picture of its shelves’ contents every time its doors are opened and closed. While visiting a shop, the consumer can then check the latest photo they were sent, to check if they are missing a particular product. In order to keep an updated list of the products in the fridge, RFID is used.
In addition, this system uses “Freshness Tracker” software to track the expiry and by asking the user to enter details of the different features. Observation and monitoring occur when the door of the refrigerator is opened, and information is accessed via monitoring apps on smartphones (mobile devices). Again, this system lacks many of the services provided by ours. The system monitors products but does not weigh them (milk may be empty), the customer buys the food personally, the system does not alert the user that a product has run out, and it does not support other languages, like Arabic. The authors [29] explained the basic fundamentals related to blockchain and IOT; a detailed review on the various aspects of interconnection, interoperability, reliability, and security needed in daily activities has been described. The researchers listed the challenges, future prospects, and the benefits incurred with the IOT by inculcating blockchain nodes in IOT devices [30]. The paper proposes a novel lightweight framework using blockchain that can be used in IOT devices, and experiments claim the effectiveness of the proposed research with significant reduced overhead and processing time and enhanced security aspect [31]. The authors emphasized on the blockchain as a service and how it can be utilized within IOT, and different case studies have been explored for experimentations and simulations with acceptable accuracies reported [32]. The architecture for arbitrating roles and permissions in realistic IOT scenarios has been explained in the paper [25]. The permissions needed in IOT with scalable architecture have been proposed underlying certain advantages.
4. Comparison with Existing Systems
The comparisons of the selected systems against our proposed system are made, in terms of the problems of each system, the technique of each system, the connection type that each system uses, the services that each system provides, what services are lacking in each system, and a full comparison between these systems and our proposed system.
Based on the results demonstrated (see Table 1), some criteria have been considered in all of the systems studied: the status of food and liquids, checking the quantity of products, human intervention, product checking, utilizing sensors, using central server, employing system platform, and using a specific technique. However, using a proper computer or application issue has been taken into consideration in 86% of the studied systems. Moreover, sending notification (alert) buzzer about the status of food or liquids and checking expiry date of products (quality) in the future have been considered only in 57% of the studied systems. Likewise, only 42% of the related systems are able to do payment automatically and send order to the supermarket (refrigerator self-purchasing). Furthermore, only 28% of the studied systems have taken remote controlling issue into account and the Arabic language criterion has not been considered in previous related systems. Moreover, as can be observed (see Table 2), unlike previous works, all criteria are simultaneously considered in our proposed system, which leads to enhance the performance and save effort, time, and money.
5. Hardware Required
This section presents the required hardware and software for completing the proposed refrigerator circuit. Based on the diagram (see Figure 1), in the circuit of our implemented system, a load cell sensor is utilized which calculates the weight of a product and transmits the data as signals. As its signals cannot be read by Arduino, an amplifier HX711 is used. The load cell reads the required information to amplify and further communicate for processing operation. The Liquid Crystal Display (LCD) is used to view the response of the weight. Breadboard is used as a port to collect, as an example, two wires needing the same port. The utilized materials are described in following subsections.
5.1. Arduino UNO
In the proposed system, we utilized Arduino UNO as an open-source microcontroller board, which comes with sets of digital and analogue input-output pins that may be connected to different expansion shields and other circuits. It depends on the Microchip ATmega328P microcontroller and was developed by Arduino.cc [33].
5.2. Arduino Ethernet Shield
An Arduino Ethernet shield is employed in our proposed work. This is a microcontroller board based on the ATmega328, with a reset button [33]. The shield contains two informational LEDs: PWR and RX and TX. The first LED indicates that both the Arduino UNO board and the Arduino Ethernet shield are powered. In the second LED, RX flashes when data is received, while TX flashes when data is sent by the shield.
5.3. Load Cell Sensor
A load cell sensor is utilized to discover the presence of a food product and then to calculate its weight. The load cell weight depends on the application. In our system, we use 20 kg and the straight bar type. The load cell and the HX711 module should be connected.
5.4. Load Cell Amplifier HX711 Module
As the Arduino is not able to read the signal issued from the weight sensor, the HX711 is employed. HX711 is responsible to read the information from the load cell, amplify the signals received from the weight sensor, and then send it to the Arduino processor, based on the ATmega328 microcontroller as digital for processing, which is connected to the Arduino Uno. The GND pins of the module and Arduino should be connected. DT and SCK and digital pins 2 and 3 of the Arduino should be connected. The remaining pins of the Arduino VCC and 3.3V should be connected. Figure 2 shows the diagram.
A breadboard is utilized which is aimed at building an Arduino-compatible circuit [34].
5.5. LCM1602 Module
The LCM1602 module is employed in order to display for the system. It requires a connection to the 5V pin. To divide the Arduino’s 3.3V signal, a breadboard is placed in the middle. Both the SDA and SCL pins of the LCM1602 module are connected by connecting the pins. Moreover, the GND pins and one of the Arduino’s GND pins should be connected, and the VCC pin and the 3.3V signal of the breadboard should be connected. In general, IOS OS is used to run the Blynk application. In addition, this application is used to inform the user of all product situations and for controlling purchase as well. The same has been shown in Figure 3.
5.6. Blynk Application
To control the Arduino, we decided to use Blynk. It is a platform designed for the IoT, with different applications for controlling the Arduino and other devices. The Blynk platform consists of three main components: Blynk app, where you can create a graphical interface using different widgets; the Blynk server, which is responsible for the process of communication between the smartphone and the hardware (Arduino); and the Blynk Library, which allows communication with the server and the processing of all incoming and outgoing commands. To start using the Blynk platform, you need only two things: the hardware (Arduino) and a smartphone. This makes it very easy, as seen (see Figure 4) [10]. Every time a button in the Blynk app is pressed, the message is submitted to the Blynk cloud, as the message knows its path to the Arduino, and in the opposite direction, it works the same [35].
5.7. Proposed Methodology
This section gives the system architecture of the proposed system in detail (see Figure 5); it depicts the architectural design of the proposed system. This system uses a few weight sensors, which monitor the weight of the product stored inside the refrigerator. Weighing the product is performed by interfacing the load cell sensor and the HX711 weight sensor with Arduino. When a product (e.g., milk) is placed on a load cell sensor, pressure (or “load”) is applied. The electrical resistance of the sensor will respond to this pressure and will change to give accurate information of the weight as a signal. As the Arduino is not able to read the signal issued by the weight sensor, the HX711 reads the information from the load cell, then amplifies the signal and sends it to the Arduino processor (based on the ATmega328 microcontroller) for processing.
If the stock of the product has dropped to half, a quarter, or zero, the Arduino sends a notification to the Blynk app, via the Ethernet that is connected to the Arduino processor through the RS232 interface. In addition, in case any temporary problem has affected the application, the Arduino will notify the owner via two methods: email or SMS when the owner does not access to the Internet. The notification includes a message about the current stock of the product, seeking permission from the owner to commence a purchase process with the shop that the owner has already registered with. Once the Arduino receives an acceptance from the owner, it sends the order to the shop via Ethernet.
In the shop’s system, an appropriate software is installed. This system will be connected to refrigerators using generated IDs, unique to each owner for each refrigerator IP, and whenever a request is sent to the shop, a message will pop up, informing the shop that a request has been sent by a refrigerator. The unique code number (ID) will be given to the customer when he/she signs up with the shop online, on provision of home address and card details. This system will use online payment via credit/debit cards.
If the system of the shop successfully receives the order from the refrigerator, it will send a confirmation message to the system of the refrigerator which forwards it directly to the owner, using the three methods of notification. Whether the ordered product is successfully shipped, or it is not in stock, a relevant message is sent to the system of the refrigerator, which is again forwarded directly to the owner, using the three methods of the notification. Additionally, if the ordered product is in stock, its details will be stored in the customer (owner) database. This information includes the product number, product name, refrigerator IP address, shipment date and time, and the price. This information will be sent to the Arduino database to enable recognition of the product. Finally, the product will be shipped to the customer’s address, starting a new process for the refrigerator system.
5.8. Integration of Blockchain
The data captured from various IoT equipment is stored in various blocks smart contracts. The blocks have predefined functional logic written into them to handle and propagate data as per our requirement. Blockchain technology is used as a middleware for integration of IoT devices with added features such as better data retrieval, data security, and data propagation [36]. Each block shall have data from a separate refrigerator feature; when the machine learning model requires a particular data, the system shall call for it using predefined functions. The data flow diagram is shown with the integration of block as shown (see Figure 6). The process flow is described as follows.
The refrigerator has multiple IoT devices that take feedback and give feedback to its internal machinery. Each IoT device gathers information, and that information is stored into different blocks of a blockchain. Now, the machine learning module picks up information functionally from the blockchain and does its analysis. There are certain questions that our system aims to answer; the machine learning module precisely answers these. The answer is then sent back as feedback to the controller, which modifies the physical parameters of the refrigerator.
The refrigerator is fitted with various IoT devices to create an IoT cluster and to capture physical parameters which convey information related to the content inside, cooling levels, etc. The IoT cluster is connected to various blocks which store defined data in them. Each block is again connected with each other on the basis of user-defined rules to create a block chain. The blocks become the primary data source for the ML model. Decisions related to the system functioning are our objection function, and their answers are derived from the machine learning model.
5.9. Mathematical Model
Blockchain has two steps mainly known as hashing which means encryption of the data and second is marketing which states to represent the data of a large dataset so that the user can verify that the base data is not modified by looking as a small representative dataset. In addition to these two steps, the next one is when the blockchain needs to be diamantine for taking the data out, blockchain is mined using very complex machines; however, the underlying mathematics, for mining any blockchain two important factors, needs to be considered.
In the time frame () of the blockchain and the nonce of the dataset, which is the number, both will act as public keys to extract the information from the block chain. The machine learning model will use this technique the get the right information. A hash function turns a numerical input value into another compressed numerical value. The input to the hash function can be any length, but the output is always the same. The other name of Merkle tree is hash tree which helps in verifying and synchronizing the data. In this, every nonleaf node is labelled with the hash of the labels or values of its child nodes. They allow efficient and secure verification of the contents of large data structures. Cryptographic accumulators allow you to compact arbitrarily many pieces of data into a constant amount of space. In other words, a Merkle tree represents arbitrarily many blocks while only transmitting a single hash across the wire. The explanation is given below:
5.9.1. Basics
Hash function: a deterministic function to transform data of any range into a fixed range.
Merkle tree: a way of storing a potentially huge amount of data, while providing the user with a simple way to check that the data has not been modified.
Say, the dataset is
Hashed is where is the hash function.
Merkle tree is a graphical relational tree between the dataset of hashed functions. A block is defined as a vector of entries. Say, we have datasets. So, block will be defined as
Say, we have two blocks, and ; in order to find the proof-of-work of the block chain, we need to find a number called “nonce.”
Mathematically, the relation is where is the concatenation operator and is the hash factor.
The nonce and timestamp explain the complexity and the ease of usage of the blockchain.
5.10. Integration of AI Using Deep Learning Techniques
AI is of great significance from economic, food safety, and public health points of views. Artificial intelligence is the art and science of creating intelligent machines, especially intelligent computer programs [37]. It is like the task of using computers to decode human intelligence and significantly to the economy. In the following phase of the pipeline, the IOT hardware is collecting data from the refrigerator and the collected data goes into the cloud database, like AWS and after the data is ingested into the data preprocessing machine where data cleaning, formatting, and feature selection are done. The data preprocessing is done; thereafter, the answer is needed for several answers are required. Should we reorder item A, is item B’s inventory less or more, cooling is less, and cooling is more? For getting the answers, we run the data into the CNN model. The CNN model is based on the convolution layer, pooling layer, and the softmax layer [38]. The convolution happens in two steps; the first step happens in more extensive number of features, from where we get the larger number of pool of data. Then, it is again max pooled into the smaller convolutional network which will pin point to the answers of the decision which we want. The algorithm of the whole process is shown (see Algorithm 1).
| ||||||||||||||||||||||||||||
The hardware level of integration of equipment is achieved with a more sophisticated software level of integration of system. The microprocessors are gradually graduated to the level of algorithms and computing through cloud. The framework model outlining the process is shown (see Figure 7).
6. Results and Discussions
In this section, five stages of the proposed system are separately tested. The goal of this section is to realize how real users interact with the system and improve system based on the results. The proposed system consists of five stages as follows: (i) the refrigerator system scales the product, (ii) the refrigerator system sends product status notifications (email and alarm to mobile), (iii) the owner of refrigerator controls purchase order of products, (iv) the refrigerator system sends purchase order of product, and (v) the owner of refrigerator receives a confirmation message of sending product order (see Figure 8).
Table 1 shows the real time of execution of five stages. As can be observed, all five stages have been executed in excellent time, which shows high performance of the presented system.
6.1. Experiments and Analysis
This study has been experimented using Python where the different modules such as NumPy for analysis on the dataset, Seaborn for plotting operations, pandas for data ingestion, Matplotlib for basic plotting graphs, and machine learning classifiers sklearn and TensorFlow for performing the training and testing of the data [39] have been used. The results have been computed by suing the Inter i5 8th Gen processor with 16 GB RAM and CentOS operating system. Secondly, the deep learning and machine classifiers are applied considering total 14 features. Blockchain is implemented using Hashlib library and Python 3.0. Table 3 shows the precision, recall, and accuracy of the deep learning classifiers.
Precision is defined as the ratio of recovered words that are appropriate to search [40].
Recall is defined as the ratio of words that are appropriate to that which is successfully recovered.
-measure is defined as the harmonic mean of the above two defined.
In order to train the CNN, KNN, and SVM model, 5000 epochs have been run. The epoch vs. accuracy plot is given (see Figure 9) representing the accuracy at each iteration.
Figure 10 demonstrates the decreased run time with and without the implementation of block chain, which has been validated mathematically as well. The figure shows that the computation time increases by a factor of 25 to 30%.
7. Conclusions
In this paper, a smart refrigerator system using a Blynk platform has been introduced. The proposed system has been created to help users be immediately updated about the quantity of one product (milk) stored in the refrigerator, using IoT technology and Blynk. The AI-enabled system can automatically make the decisions which will help the users in efficient and productive purchasing of the product from a connected shop when it receives permission from the user. This hardware technology can be incorporated with the software level of integration to make unique decisions by cohesively combining cloud database for decreasing the computation complexities involved in the process. Unlike the previous systems, our designed system considers all criteria of a high-quality smart refrigerator, which leads to help us to live an easier, faster, and healthier lifestyle. A single refrigerator has to continuously run for a very long time for the ML module to have sufficient data to make precise predictions. The accuracy with which the system works also depends on the hardware quality of the IoT Devices. Since IoT devices also interact among themselves, the signal attenuation in the process also affects the performance of the system.
Moreover, in the future studies, it is suggested to improve the proposed system in such a way that the customers can be updated about the expiry dates of all products stored inside their refrigerator, immediately and periodically, without human intervention. This paper is better because of an optimized run time and also in terms of providing better data security by using blockchain technology. This research can be extended to agricultural and medical fields.
Data Availability
This is confidential and cannot be discussed at this stage, since the project is ongoing.
Conflicts of Interest
The author declares that there are no conflicts of interest to report regarding the present study.
Acknowledgments
This project was funded by the Deanship of Scientific Research (DSR) at King Abdulaziz University, Jeddah, under Grant No. J: 163-865-1440. The author, therefore, acknowledge with thanks DSR for the technical and financial support.