Abstract
As a large number of mobile terminals are connected to the IoT, the security problem of IoT is a challenge to the IoT technology. Blockchain technology has the characteristics of decentralization, data encryption, smart contract, and so on, especially suitable in the complex heterogeneous network. However, sequential access based on block files in the blockchain hinders efficient query processing. The problem is due to current blockchain solutions do not support temporal data processing. In this paper, we propose two index building methods (TISD and TIF) to address this issue in Hyperledger Fabric System. TISD (temporal index based on state databases) segments the historical data by time interval in the time dimension and indexes events at the same time interval. TIF (temporal index based on files) builds the index of files by the block transaction data, which is arranged in chronological order and is stored at a certain time interval. In the experimental part, we compare the query time on two datasets and analyse the query performance. Experiments demonstrated that our two methods are relatively stable in overall time performance on different datasets in the Hyperledger Fabric System.
1. Introduction
The Internet of Things is made up of devices that generate, process, and exchange large amounts of security-critical data, so IoT devices are often the target of various cyberattacks [1, 2]. Due to cost constraints and harsh application scenarios, most IoT devices have general functions and limited computing and storage capabilities [3]. These devices must use most of their resources for computing and executing core applications, so they cannot consume too many resources in terms of security and privacy [4]. In terms of protecting user privacy, the existing IoT system has obvious defects, which will hinder the IoT application service program from providing normal services. Therefore, the IoT needs a lightweight, scalable, and distributed security privacy-protection mechanism. Blockchain technology supports distributed, secure, and confidential security mechanisms, which are suitable for providing security protection for the IoT.
Blockchain is an accounting technique that is maintained jointly by multiple parties and uses cryptography to ensure the security of transmission and access [5]. It can achieve consistent data storage and difficultly to tamper with and prevent denial. It is also known as a typically distributed ledger technology [6, 7]. The blockchain uses transaction signatures, consensus algorithms, and cross-chain technology to ensure the consistency of the distributed ledgers of both parties to the transaction and realize the automatic disclosure of information [8].
In 2015, Linux Foundation led an open-source blockchain project—Hyperledger-developed Fabric [9, 10], Sawtooth, Burrow, and Iroha multiple blockchain projects. One of the most concerned is Fabric, different from the currency and etheric fang public blockchain projects, such as Hyperledger Fabric as chain alliance specially designed for enterprise applications, introduced the members of the identity authentication [11].
The popularity and engagement of Fabric mainly come from its open-source and rapid iteration. At present, the frame has also undergone a relatively big change in the latest 2.0 version [12]. The main technology is as follows: pluggable consensus mechanism, more flexible intelligent contract, and participant identity-management mechanism [13].
First, Fabric uses a pluggable consensus mechanism [14]. More projects have different requirements on transaction effectiveness, timeliness, and throughput in different scenarios, so the requirements on consensus are different. In Fabric, different consensus algorithms (such as Solo, Kafka, Raft, and PBFT) are provided to adapt to different network and trading environments and meet corresponding performance requirements.
Then, Fabric also expands and improves on the smart contract, eliminating the need for a specific programming language and virtual machine like Ethereum [15, 16]. Fabric can support intelligent contracts written in C, Java, JavaScript, Go, and other languages and complete deployment to reduce the difficulty of learning [17].
Finally, according to the number and roles of the parties involved in different usage scenarios, Fabric defines the rights of users and ensures the security of private data, issues certificates, and controls the rights of users. It makes the deployment of intelligent contracts more secure and flexible.
With the rapid development of blockchain technology, it has a natural advantage in such industries as certificate storage and source tracing. At the same time, it also faces great challenges in the process of system construction and practical application. In the Hyperledger Fabric System, querying historical data can only be done through the GetHistoryForKey interface [18, 19]. In the execution process, the chain code ID needs to get the block location and transaction number from history-DB, then parse out the corresponding value from different block files. This has a great impact on the temporal query in this state. With the continuous growth of transaction time, the corresponding data volume, and block file quantity increase, the speed of temporal query is bound to decline rapidly [20].
If the IoT system uses the traditional temporal indexes of the Hyperledger Fabric System, the speed of the temporal query will reduce in IoT [21, 22]. So we propose two index building methods (TISD and TIF). The TISD method is building the temporal indexes based on state databases. The TIF method is building the temporal indexes based on files. These two methods can solve slow query problems, and service deployment processes complicated problems in the Hyperledger Fabric System. These two methods reduce the query time and improve the efficiency of the query. Meanwhile, these two methods also solve the temporal index problem of the Hyperledger Fabric System in IoT. We compared the query times on two datasets and verified our method.
2. Related Works
In this section, we will introduce the work related to the use of blockchain technology in IoT.
2.1. Blockchain
The blockchain was used as the underlying technology for Bitcoin in the early stage and applied to virtual machines and intelligent contracts in Ethereum [23]. Now, the blockchain as the most widespread technology has been imported into the Hyperledger Fabric System. Each stage represents the different stages of blockchain technology development, but each has the most basic chain structure, organized by block files in order, and each block is linked in chronological order by hash value [24]. In Figure 1, block files include the header section and the block body part. The block header contains the hash value of the last block header, Merkle Tree information, etc. Blocks recorded data related to trading, and each block file links from a chain structure in chronological order by hash value [25]. The blockchain, as a distributed ledger, needs to ensure to reach a consensus between the input and output values of the transaction in the distributed environment. It also maintains the consistency of the data of all parties in the process of bookkeeping [26]. To solve this problem, Bitcoin proposed the concept of consensus algorithm; the algorithm is used to ensure the consistency of the data of each node, so as to achieve the stable development of the network in the presence of malicious nodes to participate in transactions. The current consensus algorithms are mainly divided into three types: Proof of Work (PoW [27]), Proof of Stake (PoS [27]), and traditional distributed consensus algorithm.
On the block data structure, each blockchain platform uses the Merkle Tree to process transaction data [19]. Merkle Tree is normally a full binary tree; as shown in Figure 2, the bottom of each leaf node represents a transaction data on the blockchain. is the hash value of ’s transaction data. is the hash value of , is the hash value of and , and is the hash value of ’s transaction data. is the hash value of , and is the hash value of and . After the hash computation of every layer, it can produce a hash and store in the block header.
2.2. Hyperledger Fabric Framework
The Hyperledger project was established in 2015, led by the Linux Foundation, and attracted projects including IBM, Intel, and other companies contributed several blockchain platforms like Fabric [28]. Fabric is an open-source enterprise blockchain platform, which provides a highly modular and configurable architecture. It can be widely applied to multiple industries [29]. At present, Fabric has been applied in many fields such as banking, financial institutions, and insurance.
Fabric needs to be more flexible in the context of enterprise usage. So far, the following options are provided in different versions of Fabric: Solo, Kafka [30], SBFT [31], and Raft.
Solo mode is the single-node communication mode. In this environment, there is only one sort of service node, and messages sent from Peer point are sorted by this node. The availability and scalability are limited, so it is not suitable for production environment and is generally used for development and test environment.
Kafka mode is the processing of sorting information provided by Kafka service. Kafka is an open-source project of Apache that mainly provides distributed message processing and distribution services. Each Kafka cluster is composed of multiple nodes and supports fault tolerance.
SBFT is a simple Byzantine algorithm. Compared with Kafka, it can tolerate fault nodes and a certain number of malicious nodes.
In Fabric, there are four kinds of nodes participating in the network: Peer node, Orderer node, Certificate Authority node, and Client node [32].
The Peer node is divided into four nodes: accounting node, endorsement node, master node, and anchor node. Each Peer node is a billing node in the channel, responsible for receiving and verifying data blocks sent from the sorting service while maintaining a copy of the ledger.
The Orderer node is received transactions containing endorsement signatures, packages them into blocks, and broadcasts them to Peer nodes. It can ensure the logical order of transactions and ensure that all nodes on the chain receive the same message.
Certificate Authority node is to issue identity information to other nodes in the network, which is the certificate authority in Fabric.
Client node is an entity directly operated by users. It participates in the communication of the blockchain network by connecting to a Peer node or an Orderer node.
Fabric’s data is stored in block file and database, respectively. Block file mainly stores transaction information, including hash value and block packaging time. For the database, from the perspective of database type, it can be divided into Couch DB [33] and LevelDB [34], which can be specified through the configuration file, and LevelDB is selected by default. From the perspective of database usage, there is a ledger index for storing account ID, a block index for storing block, a state database for the current latest state of transactions on the storage chain, and a history database for storing valid parts of the transaction information [35].
2.3. Level DB
LevelDB is a key-value storage engine developed by Google. LevelDB, like most KV systems, also has a simple operation interface and basic operations that include writing record, reading record, and deleting record [36]. LevelDB is widely used in various projects. The general architecture of LevelDB is shown in Figure 3.
Memtable is a storage mode in memory, with the data structure of skip table as the default mode [37]. When performing a data write, it writes to Memtable first, and immutable Memtable changes to read-only Memtable when its data size reaches the threshold; and then, compression threads in the background change immutable Memtable to an SSTable file for storage on disk.
SSTable is a data persistence file, which is an orderly arrangement of keys within the file [38]. The first part of the file is the data, followed by the metadata for the index.
MANIFEST file contains the key range and the hierarchy. The version of MANIFEST file is changed when the database is reopened. If the database is opened, a new file will be generated each time.
CURRENT file is the file name of MANIFEST files [39]. CURRENT file can be used to locate the MANIFEST file to restore the database in the event of a failure, such as a power outage.
LOCK file is used to control multiprocess access to the database [37]. When a process opens the database, an exclusive file lock will be added to the file. If the process finishes, the lock will be automatically released.
2.4. IoT System
The IoT system connects the Internet and information-sensing equipment to achieve intelligent control and management of all objects. It can be divided into three layers: Perception Layer [40], Network Layer, and Application Layer. Perceptual Layer is responsible for object collection and data processing. Network Layer is responsible for transmitting the collected data. Application Layer provides IoT-based applications [41].
The IoT system usually includes embedded devices with resource constraints. As an important part of IoT, sensors will face some technical challenges in practical applications [42]. First, due to the limitations of low cost, low power consumption, and small size of sensors, these limit sensors’ computing, storage, and communication capabilities, thereby affecting the expansion capabilities of the network [43]. Second, most sensors are deployed in unattended areas, so sensor security issues are particularly prominent. Third, since the monitoring and recording capabilities of sensors are often opaque, when sensors share data information, they will cause data information leakage. The IoT system is being integrated with various technologies to solve various problems, such as Liang W who proposed a fast defogging image recognition algorithm based on bilateral hybrid filtering to solve IoT image issues in foggy weather [44].
Blockchain has significant cryptographic security and immutability [8]. With its characteristics of decentralization, consensus mechanism, and nontamper ability, IoT can use the advantages of blockchain to solve its security and scalability issues [45]. The decentralization of blockchain enables IoT devices to directly obtain information without a central server, which reduces the problem of high operating costs for central institutions. But the IoT system uses the traditional temporal indexes that cannot meet IoT’s needs. We propose two index building methods to solve this issue.
3. Efficient Temporal Index
Fabric distinguishes between the historical state and the current state of the data [46]. The data of transactions are stored in LevelDB in the form of key-value pairs. For each key in the transaction, there are multiple states corresponding to it. The latest state is called the current state, and other states are called historical state [47].
The state database stores the current state of the key, while the history database stores only the index location of the historical data on the block file, which is stored on disk [48]. The historical data is inserted in chronological order and distributed in different block files, so the efficiency of processing the historical data is relatively low.
In Fabric, there is no interface for temporal queries in the system. When we need to analyse the historical data, we call the GHFK function to get an iterator. Through the data obtained by iterator that is connected with the corresponding time interval in memory, we realize the temporal query. GHFK function can return all the history data for the committed valid . The calling process is shown in Figure 4.
For example, there is many trading information in the time interval . If we want to query the transaction information in the time interval , we need to call GHFK function to access large amounts of data, deserialize the block file, and connect the data to the time interval. If we want to query the transaction information in the time interval , we should connect the data to the time interval . In other words, the data is useless in the time interval . It is a waste to read and deserialize the data in the time interval .
As time goes on, the block file that needs to be deserialized is increasing by transaction volume and data volume [49]. It will inevitably lead to the decline of query efficiency with the change of query interval.
In this part, we propose two methods which are temporal indexes based on state database and file in Fabric to solve this problem. It can improve the efficiency of temporal query in Hyperledger Fabric.
3.1. Temporal Index Based on State Databases
The reason for the low efficiency of Fabric temporal query is greatly related to the number of reads and writes to the database and the number of accesses to the file; especially, when the query volume is relatively large, the efficiency will significantly decrease. In our method, we divide the transaction into corresponding intervals according to such criteria as time intervals and add indexes to the intervals to speed up the temporal query. In this section, we will introduce how to create a temporal index (TISD: temporal index based on state databases).
In the process of constructing a temporal index, let be the starting time of the temporal index and be the starting time of the next build process. For each entity in the transaction, we divide into many time frames , and every time frame is adjacent. We build temporal index for every in every time frame .
In Figure 5, for entity , let represent a set of events that relate to and occur in time frame . Let the key-value insert the state database, where respresents the composite value and represents the value. It is useful to speed up the temporal index.
When the data is submitted to the state database, we build an index of the time state of the data. The construction process of the index is as shown in Algorithm 1. First, you need to determine the current time interval to verify and the current time interval is valid. If is valid, we use to obtain and update the current status.
As shown in Figure 6, the status of is updated from to ; it also means the current event is added to . At the same time, the value of is updated from to . If satisfies the separation condition, it will generate a new time interval . If is invalid, the status of will update to and generate a new event .
| ||||||||||||||||||||||||||
During index building, we need to divide the time into some time intervals , the time intervals are continuous, and all data events are included. We use three times interval segmentation methods: FI (fixed time interval), FE (fixed event number), and DI (dynamic interval).
FI: In the total length of time, it builds a temporal index in the fixed time interval. Each entity may match a different number of events in each time interval by different trading situations. Therefore, the efficiency may be erratic.
FE: In this model, time intervals are determined only by a fixed event number. The length of temporal index may have a big gap in each entity by different trading situations. For frequently updated entity values, the index is also frequently updated. There may be multiple time intervals in a short period of time, and the database may be accessed multiple times during the query.
DI: The time interval is determined by calculating the time and measuring the number of events. A time interval or several events are fixed, and the time interval must be determined in one of two ways. One is that the number of events must equal or exceed the specified number of events when the time interval is equal to a fixed value. Another is that the interval must be at least fixed when the number of events is equal to a fixed value, so it is able to avoid that events occur too much or too little in a time interval.
3.2. TISD Query Process Analysis
To illustrate the query process, we define four relations: temporal joined relation, temporal connected relation, temporal included relation, and no temporal relation [50]. Letwhere represents temporal joined relation, represents temporal included relation, represents temporal connected relation, and represents no relation between two intervals.
The query process is shown in Algorithm 2, is all events about in the time interval . The algorithm queries and time interval by the iterator returned of and obtains , where represents an interval that has temporal joined relation or temporal included relation between and the target query interval. There is a temporal connected relation between the first and the last in .
| ||||||||||||||||||||||||||||||||||||
For each in , is performed. The block file is parsed through the returned iterator. For the interval with temporal included relation, the data parsed by the iterator is appended to the result sets. For the interval with temporal joined relation, the data returned by the iterator is traversed to remove the data not in the interval .
In the temporal query process, the efficiency of the query is mainly measured by the query time, which depends on the rate of events, the size of the index interval, and the size of the query interval. Therefore, how to choose the appropriate index interval becomes a key problem.
For the temporal query performed by Fabric, is used to represent the query time of the historical state of the keyword in the time interval , which can be expressed aswhere represents events about the keyword and represents the number of events in the query time.
In the normal process of data insertion, we use to represent the insertion rate of events related to the keyword , which can be expressed aswhere represents the total length of time the event inserted.
Since the first transaction in the system needs to be iterated in the Fabric query process, can be expressed as a period of time, starting from the initial transaction point and ending at a larger time point in the target query interval . So
For the FI method, the index construction is determined by a fixed time length. For the total time length and the given fixed time length , letwhere represents the time interval divided, represents the number of partitions for all time intervals.
The query time of the historical state can be defined as
The target query interval must be less than or equal to the total length of time, so
With the increase of , the query time will be reduced correspondingly. In the limit, it only needs one iteration to realize the query, but this situation is not realistic in the practical application.
For the FE method, the division of time intervals is controlled by a fixed number of events. Letwhere is the fixed number of events set. So
When the FE method is used to divide the time interval, within the normal range, the temporal query time is negatively correlated with the number of fixed events and positively correlated with the event occurrence rate.
For the DI method, the division of time interval is determined by the number of fixed events and the length of fixed time. Letwhere represents the number of intervals generated in total time . In the query, represents the query time of the historical state of the keyword in the time interval , and the target time interval of the query is not more than the total time length , so
Therefore, in the DI method, the temporal query time of the keyword is negatively correlated with the fixed time interval and the fixed number of events . It is positively correlated with the event occurrence rate .
During the query, the GHFK function is used in Fabric. When the traditional method is used to query the keyword , the GHFK will generate a lot of access to the ledger index, as well as the access, deserialization, and parsing of files. But using temporal index only needs to deserialize times files, that is to say, using reasonable time intervals to build temporal index can greatly reduce the number of file access.
3.3. Temporal Index Based on Files
When TISD (temporal index based on state databases) builds indexes, it will increase the number of database accesses and the size of block files. It may have an impact on normal transactions on the blockchain as the data volume is extremely large or the disk space is high. Considering the impact of index construction on database read-write and block file growth, we propose TIF (temporal index based on files) that builds indexes without affecting the normal blockchain storage.
In order to reduce the impact on the blockchain storage space, we will explain by two aspects. One is for state database: TIF only saves the latest state of the interval and the corresponding index file information. Another is for block files: TIF will not write temporal index data to the block file.
The TIF structure is shown in Figure 7. For time interval , index files related to are marked such as file name, file path, and file state, and index data is not added to block files in the form of transactions.
The index construction process is as shown in Figure 8 and Algorithm 3. After the verified transaction data is generated, it writes the corresponding data to the status database, updates the database to the latest state, and checks the current existing time interval . If is invalid, it will regenerate , update the latest information to the database, and regenerate the index.
If is valid, it will check enough space allocated in memory. If memory space is enough, it will append the events generated to memory in the agreed format and classify the events by the corresponding time interval. It also updates the latest information.
| ||||||||||||||||||||||||||||||||
3.4. TIF Query Process Analysis
The TIF query process algorithm is shown in Algorithm 4, where represents all events about in the time interval . The iterator returned by GetStateByRange queries the interval partition and divides . Let represent a time interval that is joined and included between the target time interval and by relation function. For each of , it will get the file name, file path, file state, and other information by GetState(). Then, the file state identifies whether the data has been written into memory. If the data is in memory, it will join and calculate the temporal directly, filter the keyword, and return the query results. If the data is not in memory, it will take the data into the memory, parse the data, make temporal connection to the parsed data, filter the results of the temporal connection, and select the keyword as the returned value.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||
In terms of temporal query time, the use of temporal index can reduce the consumption of time to some extent, improve query efficiency, and speed up the query, but in the process of building and maintaining temporal index, there will be some extra costs, and the main costs are time consumption and storage space consumption.
In the storage space consumption, using TIF to build an index requires only one key-value pair associated with the time interval. The value is maintained in the database. When the time interval is updated or the memory capacity reaches the threshold, the value is updated. So TIF can reduce read-write times and space usage on database.
In the time consumption, the generation of indexes is carried out at the same time as the transaction. First, the data is written into memory, the data in memory is classified, the index is established, and then the file is written. Writing to the file can be done asynchronously, so the time consumption of this process is mainly memory reading and writing. Due to the memory bandwidth, the time consumption has little impact on the business logic.
TISD takes advantage of the fast reading and writing of the database, determines the time interval in various ways, and builds the index for the data in chronological order. It reduces the reading and deserialization of the block file and speeds up the query of the historical data. With the minimizing influence of the system efficiency, TIF stores transaction information by using memory as cache and file as a subject. It avoids the state database and historical database many times to read and write and takes up the space of the database.
4. Experiments and Results
In this section, we will introduce the experimental data and experimental results.
4.1. Experimental Data
Since there is no dataset specific to blockchain query in Fabric, we generate two datasets based on usage scenarios: UD (uniform distribution data) and ZD (Zipf distribution data).
UD: There are 520 entities, including 400 cargoes, 100 containers, and 20 truck entities. The events of the transaction include the loading and unloading of cargo, the loading, and unloading of containers and require each entity to produce 2,000 transactions in a total of 100,000 seconds. All events are even inserted into the blockchain in chronological order.
ZD: The number of entities, number of events, and total length of time are the same as UD. But events are inserted into the blockchain in a chronological Zipf distribution.
The Fabric platform is deployed using the following two network architectures:
T1: There are 1 Peer node and 1 Orderer sorting node, and the sorting service adopts solo.
T2: There are 4 Peer nodes and 1 Orderer sorting node, and the sorting service adopts solo.
4.2. Experimental Results
4.2.1. TISD Query
First, we evaluate the query time using TISD on datasets UD and ZD ♦and network structures T1 and T2. Then, the time and space costs during index building are also evaluated.
Figure 9 is TISD query time on dataset UD and network structure T1. Due to the GHFK function, the query time increases with the temporal query interval, while the M1 model and TISD trend basically remain stable with the temporal query interval. Due to the reason of interval division, FI, FE, and DI differ to some extent. However, compared with the original method in M1 and Fabric, they have better performance, which verifies the high efficiency of TISD structure.
Figure 10 is TISD query time on dataset UD and network structure T2. The temporal query trend is similar to that in Figure 9. That is to say, the network structure has little effect on the temporal query.
Figure 11 is TISD query time on dataset ZD and network structure T1. Due to the GHFK function and dataset ZD, the query time increases with the temporal query interval. Figure 12 is TISD query time on dataset ZD and network structure T2. The temporal query trend is similar to that in Figure 11.
When using indexes, there is also a certain amount of time and space overhead in the index building process. In Table 1, the construction of index consumes many times in the process of data writing, but it consumes a small proportion of the total time on network structure T1. The extra time by index building has increased by up to about 2.2% on dataset UD. The extra time by index building has increased by up to about 14.9% in dataset ZD.
In Table 2, it shows the comparison of the disk space occupied on network structure T2. Due to the construction of the index, the size of the block file will increase significantly. Since the size of the transaction data will not be different due to different data distribution and time interval division, the data distribution and time interval division basically have no impact on the size of the generated block file.
4.2.2. TIF Query
In this part, we evaluate the query time using TIF on datasets UD and ZD and network structure T1.
Figure 13 is TIF query time on dataset UD and network structure T1. The query time of Fabric is the most time-consuming, while the other methods take less time to execute and perform more steadily. FI and DI have similar time interval divisions, so the performance of query time is nearly consistent. The temporal query using TIF is superior to Fabric and M1.
Figure 14 is TIF query time on dataset ZD and network structure T1. The query time of Fabric is the most time-consuming. Due to the uneven distribution of data, the execution time of FI, FE, and DI is relatively high in the initial stage. But in general, the query time of FI, FE, and DI is better than that of Fabric and M1.
Tables 3 and 4 show the performance of TIF in query time using FI, FE, and DI time segmentation methods on network structure T2.
Table 3 is TIF query time on dataset UD and network structure T2.
Table 4 is TIF query time on dataset ZD and network structure T2.
All experimental results show that, for different business scenarios, different index building methods and time interval partitioning methods have different performance in temporal query efficiency. In terms of overall performance, TIF has certain advantages compared with other methods under DI.
5. Conclusion
This paper proposes two temporal indexes (TISD and TIF) on the Hyperledger Fabric blockchain platform. The experimental results show that the two temporal indexes and three time interval partitioning methods proposed in this paper can improve the efficiency of temporal query in Fabric to a certain extent on different distributed data while ensuring certain time and space overhead. It can also solve the problem of a low access speed of using temporal indexes of the Hyperledger Fabric System in IoT. Our research can use blockchain technology in IoT system more seamlessly.
Data Availability
The UD (uniform distribution data) and ZD (Zipf distribution data) data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that there is no conflict of interest regarding the publication of this paper.
Acknowledgments
This work is supported by the National Nature Science Foundation of China under grant 61370198 and grant 61300187, the Xiamen Science and Technology Foundation of China under grant 3502Z20183047, and in part by the Liaoning Provincial Natural Science Foundation of China under grant 2019-MS-028.