Abstract
The fusion of emerging technology, means, and music has provided a diversified development direction for the performance form of electronic music. Interactive electronic music has become a hot form of music performance in recent years and is widely used in various art exhibitions. In this paper, we will briefly explain the basic principles and practical applications of wireless wearable sensing devices, sort out the development of interactive platforms and sensors, and study the application value of wireless wearable sensing devices in interactive electronic music works. Near-field communication (NFC), as a short-range radio technology, can realize wireless energy and data transmission simultaneously. By integrating NFC modules, electrochemical sensing circuits, and flexible electrodes, it is possible to ensure complete electrochemical sensing functions while maximizing the miniaturization, flexibility, and integration of the system. Therefore, this paper constructs a series of wireless passive flexible electrochemical sensing systems based on NFC technology, electrochemical sensing technology, and flexible electronics, and uses these systems to realize a series of innovative applications, aiming to guide the creation of higher quality interactive music artworks as well as explore the artistic and esthetic experience brought by interactive music functions, and provide theoretical references for virtual art interactive music creation and practice.
1. Introduction
Digital technology has triggered an unprecedented media revolution, and computer digital technology has controlled and brought about an innovation in the style of virtual media art. Recently, through the high integration of digital technology and the spirit of art, computer virtual reality technology has even broken the boundary between real and virtual spaces, and virtual art has evolved toward a form of cross-fertilization of technology and multiple art media, with virtual reality art as the main representative, and revealed characteristics that are different from traditional virtual art [1]. The immersive, imaginative, interactive, and intelligent nature of virtual art gives creators the possibility to use new technologies to create new interactive experiences. While it continues the immersive perceptual experience brought by the purely visual simulation of traditional images, it also makes interactivity a focus of attention in the creation of virtual art. The immersion, imagination, interactivity, and intelligence brought by virtual art give creators the possibility to use new technologies to create new interactive experiences. While it continues the immersive perceptual experience brought by the purely visual simulation of traditional images, it also makes interactivity a focus of attention in the creation of virtual art. As a new art medium, virtual art is not satisfied with the traditional cognitive model; its technical characteristics give it a multidimensional interactive perceptual approach different from other media, and it also reshapes information with the multifaceted interactive art language accommodated by its technology, broadening the new forms presented by virtual art. Interactivity has gradually become a functional aspect of virtual art, an act that plays a dynamic role with the help of the artistic medium. With the development of interactive technology and the study of cross-disciplinary artistic creation, the form of interactive electronic music creation has shown a diversified development trend [2]. Contemporary interactive music is very flexible in form, and it can be purely a solo or ensemble of electronic instruments, interactive coordination between computers and computers, the interaction between computers and instruments, or electronic instruments. With the support of interactive technology, the creative intent of the creator can be expressed in any form, including emotion.
In this paper, we design and build a wireless, passive, and flexible electronic patch based on NFC technology, electrochemical sensing technology, and printed electronics to realize the impact on the music, while the music is composed or improvised, and the wireless wearable sensing device collects data on-site in real time. The realization of interaction requires an intermediary, and the interactive system built on the wireless wearable sensing device assumes this role, linking the performer, the program controller, and the listener, influencing each other in the same space to produce emotional resonance. In the production of interactive electronic music works, presets and interactions are an important set of concepts in composition [3]. Presets usually refer to predesigned rules or procedural parts of interactive work, either as specific audio files or as specific values or set intervals that control volume, timbre, and tone values. Interactive electronic music can be classified as preset structured, preset combined with real-time structured, and real-time structured according to the level of interaction. This paper focuses on how the smart technology led by wireless wearable sensing devices changes the expression of interactive art from the level of thinking, means, and experience logic, and proposes the paradigm of interactive art expression under the influence of wireless wearable sensing device technology. From the perspective of intelligent technology application, we analyze and study the form, expression logic, and esthetic characteristics of intelligent interactive art expression, explore the future innovation trend of interactive art expression in combination with the current development of artificial energy technology, and try to build a relatively complete set of wireless wearable sensor device interactive art creation application model.
2. Related Works
The concept of virtual art is primarily concerned with both technology and art, due to the strong dependence of the medium itself on technology, and therefore, a study of virtual art inevitably discusses the development and impact of its technological aspects. Based on the former, the focus is on the characteristics of virtual art in the context of art and the interactive experience it offers to its audience. The literature [4] proposes the concept of an “ultimate display,” which describes a head-mounted display that allows the user to interact with objects in the virtual world, making it impossible to distinguish the difference between virtual and real. The literature [5] explores the current state and future development of virtual reality technology and describes virtual reality more comprehensively in terms of seven features such as simulation and interaction. He also argues that the nature of virtual reality is a higher-level artistic medium that arises based on technology. In the literature [6], the concept of the “virtual reality technology triangle” is proposed, and the “3I″” characteristics of virtual reality—immersion, interactivity, and conceptualization—are concisely described. The concept of the “virtual reality technology triangle” is proposed, which concisely describes the “3I″” features of virtual reality—immersion, interactivity, and conceptualization—and describes the basic theory and practical skills of virtual reality from two directions: technology and application. The literature [7] proposes the concept of “virtual art” and discusses the development of virtual reality from both computer and art history perspectives. The literature [8] proposes the term “Aesthetics of Virtuality,” but it is limited to the field of electronic computer virtual technology. From an artistic point of view, the literature [9] has sorted out the lineage and origin of the development of virtual reality art from the perspective of interaction and immersion. The literature [10] points out that the use of virtual reality technology by artists to construct works is a cultural experimental act of thinking and reflecting on the virtual nature of objectivity and the reality of the subjective world from both technical and philosophical perspectives. Literature [11] elaborates on the characteristics of the times and artistic style of digital media art from the perspectives of esthetic characteristics, applicability, and realism. Literature [12] analyzes new artistic expressions from the perspective of media and explores the new artistic experience of new media interactive art in the cultural context constituted by modern new technologies and new media forms, providing theoretical references for the development and creation of new media interactive art. In the literature [13], a lightweight and simple radial pulse sensor was designed using graphene-coated fibers as the core sensing unit and successfully applied to a personalized health state analysis. The flexible sensor used has good linearity and sensitivity to tensile strain, and a structure was designed in combination with the sensor properties to successfully achieve accurate detection of periodic pulse waves and waveform changes due to exercise and disease. In the literature [14], a wearable, multi-SPES sensor integrated data glove was developed using a spiral passive electromagnetic sensor (SPES) as the core sensing unit, which does not require a chip or complex electronic circuitry, thus making the whole device easy to fabricate, lightweight, and comfortable. The glove is capable of encoding data from gestures in the spectral response of the SPES, allowing each specific gesture to be associated with a unique sensor response. The literature [15] studied the haptic patterns of the hand based on the haptic data collected by a flexible data glove to broaden the scope of application of the data glove. They developed a low-cost, stretchable haptic data glove “STAG” using a flexible sensor array that can cover the whole hand. The haptic sensors, which are uniformly distributed on the hand, provide feedback on the amount of force applied to the hand when holding an object so that this data glove can be used to measure external forces between 30 mN and 0.5 N. Using a convolutional neural network to analyze the grayscale map of the data from the sensing array, the mass, and shape of the handheld object can also be estimated. Not only that but also the data glove can also record tactile videos at a frame rate of 7.3 Hz for exploring the typical tactile patterns that occur when a person grasps an object. After developing a tattoo sticker-based wearable potentiometric sensor, the literature [16] reported for the first time a stretchable textile-based potentiometric sensor, the success of this study not only gives the sensor a better stretching effect but also enables the sensor to maintain contact with the wearer itself and reflect the monitoring results in a more timely manner. A flexible smart glove that can be self-powered and can be used for human-computer interaction was fabricated in the literature [17] using a polyvinylidene fluoride (PVDF) sensor as the core sensing unit. Two practical interaction scenarios are shown in the article: one of them realizes the control of five LEDs on/off with five fingers, respectively, while the other one demonstrates the feasibility of using the glove to control the car at rest, stop, forward, backward, right, and left turns, demonstrating that the flexible data glove can be used in human-computer interaction scenarios.
3. Wireless Wearable Devices Based on Flexible Sensors
3.1. Design Architecture of Wireless Wearable Devices Based on Flexible Sensors
Flexible strain sensors are flexible sensors that can regularly convert mechanical deformation into electrical signals after being deformed by external forces to achieve sensing functions. Since these sensors have good sensing performance and have excellent characteristics such as thin, light, simple preparation, and good comfort, they are playing an increasingly important role in wearable devices. For flexible sensors, the selection of suitable materials and preparation methods is the key to the development of such sensors. The substrate material for flexible sensors should have good ductility, flexibility, and biocompatibility, and the conductive material should be able to maintain long-term stability, high sensitivity, and else. The preparation process should be low-cost and easy to implement, thus making large-scale and convenient production possible. Among the wearable sensing units that can be worn on different parts of the human body, the design needs to meet the conditions containing high integration, small size, low power consumption, high measurement accuracy, etc. A variety of sensing units are integrated into a miniature sensing device that is placed on different parts of the body to detect human activity. This research aims to bring the characteristics of flexible sensors into play to help reduce the production cost of wearable devices and make them more lightweight. Therefore, the flexible data gloves made are all made with some traditional and easy to replace electrical components, and some classical filtering methods are chosen to process the data, to better show the sensing performance of flexible sensors in practical applications. Commonly used wearable sensing units include the following: accelerometers—accelerometers are the basic sensing units commonly used in motion monitoring, measuring acceleration rates, and determining whether the device is in a horizontal or vertical position; gyroscope—a gyroscope is a device that detects angular motion and measures the angle of rotation of an object [18]; magnetometer—a magnetometer is a compass that can be used to improve the accuracy of motion tracking. Barometric sensing unit—a barometer is used to measure atmospheric pressure and can be used to determine changes in altitude by atmospheric pressure. One of the wearable sensing units mainly uses inertial sensing units and mobile phone human data, such as accelerometers and spiral meters. Figure 1 shows the design architecture of a wireless wearable device based on flexible sensors.
Miniature sensors are mainly miniaturized by microelectronics and precision machining technology to reduce the size of the sensor from millimeter to micron or even nanometer-level sensitive elements. Miniaturized electrochemical sensors not only have the sensing performance of traditional electrochemical sensors but also have the advantages of small size, lightweight, low energy consumption, and easy diversification, which greatly improve the application range of sensors. In the act of human-computer interaction, flexible wearable microsensors play the role of information transfer hubs.
3.2. rGO Wireless Wearable Sensing Device Implementation Path
rGO has excellent electrochemical and optical properties and is simple and low-cost to obtain. It can be quickly and effectively wrapped around the “I″” substrate material, and its use as a conductive material for sensors can effectively reduce production costs. The double-wrapped yarn consists of two layers of polyester fibers wrapped around a highly elastic polyurethane fiber, which not only has a certain degree of elasticity so that it can be stretched within a certain range, but also has a good affinity with the human body, and does not impede the movement of limb joints. This allows the fibers of the double-coated yarn as a carrier to have a range of tensile strains and to be used in wearable devices to ensure good comfort. Also, the preparation method is extremely simple and the raw materials are cheap and easily available, which can significantly reduce the cost of sensors applied to wearable devices [19].
The conductivity characteristics of a single rGO fiber can be referred to in Equation (1).where is the electrical resistance of the fiber itself, is the electrical conductivity, is the longitudinal length of the fiber, and is the cross-sectional area of the fiber. The cross-sectional area of the prepared rGO fibers varies very little during the tensile deformation process and has relatively little effect on the resistance value. The electrical conductivity will vary somewhat depending on the rGO coating effect, and the preparation process will be influenced by the ratio of GO solution and the number of times the fibers are impregnated in the GO solution. rGO fibers will remain stable in electrical conductivity after forming and can be considered constant. Thus, the magnitude of the resistance of the ego fiber itself is mainly influenced by the longitudinal length , and within a certain range, the resistance value will increase regularly with the production when the fiber is stretched. The above equation only roughly describes some of the factors that cause changes in the resistance of rGO fibers; when the stretching range is too large, the change in resistance does not necessarily show a perfectly linear pattern.
The capacitance is given by the following expression, where is the capacitance in the ion equivalent circuit, is the capacitance per unit area, and is the area directly opposite the ends of the capacitor.
Here, the capacitance of the semiconductor layer at a position in the direction and width is defined, and the expression is
After applying a positive gate voltage to an organic electrochemical transistor, a dedoping mechanism is used to describe the carrier concentration in the semiconductor layer: cations from the electrolyte are injected into the semiconductor film (here, the cations are the charges on this side of the organic semiconductor layer for the equivalent capacitance in the ionic circuit), and each cation injected into the semiconductor layer fills a hole, which is the hole density . According to this mechanism, the expression for the hole density in the organic semiconductor layer is given bya
Therefore, the doping state is different at each part of the semiconductor layer in the direction , so here the hole density at each part of the semiconductor layer is defined as a function of
As in the flow of Figure 2, we designed and assembled a data glove with an rGO-coated fiber sensor as the core sensing unit to explore the potential of the flexible sensor for wearable devices and to investigate some problems in circuit design, data processing, etc., that need to be solved for the application of the flexible sensor in a practical example [20]. Analyzing some performance characteristics of the sensor in use and understanding the distribution of data noise will help to provide directions for the improvement of the sensor itself and the selection of algorithmic models. Compared with static mapping, the dynamic mapping may be less intuitive for the audience to distinguish the correspondence between a performer’s movements and their resulting sound outcomes, as this correspondence changes over time in the work, but the dynamic mapping has another layer of benefit for the performance of real-time interactive electronic music performed by an orchestra. This research aims to bring into play the characteristics of flexible sensors to help reduce the production cost of wearable devices and make them more lightweight, so the flexible data gloves made are using some traditional and easy to replace electrical components, and choose some classical filtering methods to process the data, to better show the sensing performance of flexible sensors in practical applications.
Both the capacitive tactile sensors with traditional parallel electrode plate structure and the capacitive tactile sensors with the same face electrode structure can be reduced to two basic structural layers, namely, the electrode layer and the dielectric layer, so the good or bad performance of the electrode layer and the dielectric layer directly affects the output characteristics of the capacitive tactile sensors [21]. Currently, most of the research on capacitive tactile sensors has been done by improving the electrode material and the dielectric layer material, which in turn improves the mechanical properties and output performance of the tactile sensors. This chapter aims to keep the electrode material and the dielectric layer material unchanged by changing the electrode shape or the dielectric layer shape of the sensor, and then optimizing the output characteristics associated with the sensor. Although capacitive tactile sensors with a homogeneous fork-finger electrode structure have high initial capacitance and sensitivity, and superior advantages over conventional capacitive tactile sensors in terms of large deflection deformation and electrical connection, the extremely uneven distribution of the electric field of the fork-finger electrodes in spatial locations leads to the phenomenon of poor linearity of the sensor capacitive output curve.
To improve the initial capacitance and sensitivity of the sensor, the electrode layer structure is optimized. The serpentine fork-finger electrode, spiral fork-finger electrode, and serrated fork-finger electrode structures were designed, respectively, and the sensor models under these three electrode structures were constructed, and the output characteristics of the three sensors under the three-dimensional force were simulated and experimented using finite element software to analyze the potential shift field distribution under the four-electrode structures. As shown in Figure 3, the results show that the three tactile sensors based on the improved electrode structures have improved 13.6%, 14%, and 24% in initial capacitance, and 23.96%, 18.6%, and 22.9% in sensitivity, respectively, compared with the original ones [21].
3.3. Wireless Wearable Sensing Devices in Interactive Music
The original concept of interactive music referred to a specific form of music that allowed the listener to participate in the process of creating music, influencing the process and transforming the listener from a passive recipient of the music to a part of its composition. Nowadays, along with the development of technology, the concept of interactive music has shifted to refer exclusively to music that is engaged by sensors and generated in real-time. This paper implements the application of wireless wearable sensing devices in interactive music by studying the human-computer interaction behavior of wearable sensors encased in rGO fibers.
Instruments with transducers can also be referred to as “hybrid instruments.” Traditional acoustic instruments are equipped with transducers that provide the player with additional sounds or the ability to control changes in musical parameters. The concentrated distribution of the potential shift field changes the stored charge between the sensor electrodes, thus changing the capacitance between the electrodes, and the more concentrated the potential shift field distribution, the larger the stored charge and the larger the capacitance value. Therefore, for the initial capacitance of the four types of forked-finger electrode tactile sensors, the serrated forked-finger electrode is the largest, the serpentine and spiral forked-finger electrodes are the second largest, and the common forked-finger electrode is the smallest. In this case, the original acoustic instrument maintains all of its default sounds and playing style, but adds additional features that increase its functionality and expressiveness when playing music. It uses several different sensors that can accurately capture the player’s gestures. These sensors include a vibration sensor placed in the yard position with a high-frequency filter (the filter is used to filter out subtle vibrations and prevent accidental touches), a two-axis acceleration sensor located on the bowstring, and a tension sensor. The purpose of creating a controller that mimics an acoustic instrument is to come as close as possible to simulating an existing acoustic instrument. There are two main reasons for creating such controllers: to allow players of traditional instruments to become proficient without any training and to create new sounds using existing playing techniques.
The practical application session of the interactive music controller consists of four important components, which are performance pattern design performance movement design, mapping strategy design, and score design. These four components define the style of playing, the movement, the sound, and the way the piece is transmitted. The components are mainly.
Performance mode design: the design of the form of performance, that is, what form is used to play the controller. For example, solo or repertoire, audience interaction or not, indoor theatre performance, or street outdoor performance, and performance movement design: the type of movement used to play the controller [22]. Good movement can make the performance of work enjoyable as well as lead the audience to understand the meaning of the work more easily.
Mapping strategy design: converting motion information captured by the controller into sound or music control parameters. Score design: interactive music controllers generally use nonpentatonic notation, but instead use textual or graphical notation to record movements. Interactive music controllers are difficult to record their movements in the form of notes, so using graphical or textual notation is more conducive to reducing the learning cost for the player. Figure 4 shows the correspondence between the technical aspects of the controller design and the performance application.
Interactive music controller playing mode refers to the way and means to play the controller. For the same controller, different playing styles can make a big difference. These differences are both musical and considerate in terms of the means of control; for example, in an ensemble formed by multiple players, it is necessary to consider not only the role played by each player but also the coordination and synchronization between the multiple players.
The process of converting motion information captured by a controller into sound or musical control parameters can be referred to as mapping. When composing music for a traditional acoustic instrument, the mapping relationships are fixed and can be changed by the composer in almost no way; as we can see, the strings of a violin are both control and sound generating devices, and the relationship between the two is determined by the laws of physics, independent of the way they are played. When composing music for an interactive music controller, on the other hand, the situation is very different from that of a traditional acoustic instrument, and the composer needs to consider for himself how to establish the mapping. In interactive music, the mapping strategy is of high importance, and by changing the mapping strategy, the resulting music can be highly variable, even if the playing style remains the same. Designing the mapping approach is a complex process because the variety of parameters that can control sound synthesis or musical changes in a computer is very complicated. The concentrated distribution of the potential shift field changes the stored charge between the sensor electrodes, thus changing the capacitance between the electrodes, and the more concentrated the potential shift field distribution, the larger the stored charge and the larger the capacitance value. Therefore, for the initial capacitance of the four types of forked-finger electrode tactile sensors, the serrated forked-finger electrode is the largest, the serpentine and spiral forked-finger electrodes are the second largest, and the common forked-finger electrode is the smallest. In sound synthesis, there are dozens of ways of synthesizing such as additive synthesis, subtractive synthesis, sampling, and becoming, and each involves different types of parameters. For example, in additive synthesis, the variable parameters that can be controlled include amplitude, frequency, and phase, while FM synthesis requires control not only of parameters such as frequency amplitude but also of the scale factor of the carrier and modulating wave; resonant peak synthesis requires control of parameters such as the center frequency, bandwidth, amplitude, and offset of the resonant peak. The controller (two sliders) and the source (single oscillator) are identical in which only the mapping strategy has been changed [23]. The experimental results show that the mapping strategy has a profound psychological impact on the performer. While increasing the complexity of the mapping makes it more difficult for the user to get started, it follows that the duration of the user’s interest in the controller will be longer.
In the development of sensor technology, the correspondence between sensors and human movements has become increasingly clear. By studying human movement patterns and the characteristics of different sensors, sensors have made great strides in their adaptability to the player’s movements. Various sensors have been able to capture all kinds of human movements, including muscle movements, blow sensors, and speech sensors. In this, the force generated by muscle movement can be captured using two different forms of sensors: dynamic and static. In the continued breakdown, the motion provided by the muscles can be captured both in a sensor noncontact situation and in a contact measurement using mechanical linkage. In terms of the design of notation for interactive controllers, fragmented technology makes it possible for controllers to play scores that often do not have a uniform specification. However, there is also a certain convergence in the design of most scores, that is, the use of scores to record performance actions rather than sound notation. Because the mapping possibilities for computer music controllers are so broad, each piece is mapped differently. Therefore, the utility of a notation system centered on the movement of the player is much higher than that of a notation model using sound notation. Notation using movement notation allows players of the controller to play different parts of the music with the same notation. In addition, some pieces use text cues as a form of a score, which is more abstract and difficult to understand than graphical notation scores and are suitable for reminders of playing movements rather than for learning to play new pieces through textual descriptions. However, the advantage of the text cue score is that it is more concise and allows for an overview of the overall structure of the music on a single sheet of paper, which is not the case with the pictorial notation score.
4. Experimental Verification and Conclusion
The experiments are similar to those for static gesture recognition, again with four participants, and will be divided into two parts: model training and testing. The process of collecting the training and test sets is the same, participants will play dynamic gestures sequentially under multiple wears, and a total of 1800 samples are collected as the training set as well as 180 samples as the test set. The dynamic gestures can be represented by a time series of a certain length, and the length of the time series representing the gestures is also uncertain. As shown in Figure 5, the curves have a similar trend but produce a certain offset in the timeline. If we directly calculate the similarity between the gesture to be recognized and the templates with different lengths, the obtained results may be significantly biased. The DTW algorithm is essentially a template matching algorithm, designed based on the classical algorithmic idea of dynamic programming, which can effectively measure the similarity between two time series of different lengths. We first need to construct a gesture template for each gesture from a large amount of gesture data, then match the similarity between the current gesture data to be recognized and the template data, calculate the distance between the two bands using the DTW algorithm, and launch the recognition result according to the calculated distance size 641. Human body sign data collect information from smart device sensing units when the human body carries a smart device that collects human body data. Assuming that the time length of the gesture template is m the length of the gesture to be recognized is m, the distance between the two is calculated with time complexity is 0 (nm). Usually, the lengths of gesture sequences are relatively small and only the distance to each template needs to be calculated to obtain the recognition result. Therefore, when the number of gesture types to be recognized is small, this method is more efficient and easy to extend.
Since the limited recognition range of Leap Motion leads to a limited touch range and cannot perform touch feedback well, this part is mainly based on gesture recognition to realize the human-computer interaction between the audience and the instrument. The gesture interaction needs to ensure the correctness of the gesture recognition. To verify the validity of the gesture recognition and the correctness of the recognition process, the movement changes of the fingers are captured by Leap Motion, and the displacement of the palm, the change of the bit position, and the change of the distance of the fingertip from the palm are recorded, with a total of 11 dimensions of data. The overall trend of the static gesture hand motion is constant and does not need to be verified, while the dynamic gesture has certain gesture changes, according to the changes of its hand data to check whether it is the gesture when performing a certain gesture down command. Its left slide down motion data is as follows.
As seen in the above Figure 6, only the X-displacement change is large in the recognition of the left slip, and the rest is almost constant compared with the X movement, while the rotation change in its positive rotation around the Z-axis (the midaxis of the wrist-elbow) at a certain angle and then counter-rotation, which is consistent with the swinging scenario, verify the correctness of the recognition of the left-slip gesture.
Compared with static mapping, the dynamic mapping may be less intuitive for the audience to distinguish the correspondence between the player’s movements and the resulting sound outcomes, as this correspondence changes from one time to another in the work, but the dynamic mapping has another benefit for the performance form of real-time interactive electronic music performed by an orchestra, as dynamic mapping is very much like constantly changing the player’s “instrument,” and the frequency of the changes can be very fast. This requires that the interactive electronic music orchestra performer’s performance be based on an understanding of how the work works and that he or she is always aware of the relationship between his or her actions and the target is controlled, which is the “new format” brings “new requirements,” which in turn will bring new experiences to the performer and the audience. Wearable sensing units need to be worn for a long time when they are used, and if they are designed to be worn in large numbers, their energy consumption is too high, affecting the length of time they can be worn. To solve the problem of skin color-like interference during dynamic gesture tracking, a multisensor data fusion filter is used to fuse RGB color video streams and depth video streams during dynamic gesture movement and combine with an improved tracking algorithm. In this paper, a wireless wearable sensor data fusion Kalman filter tracking algorithm wrapped with rGO fibers is used to track the RGB color video stream containing skin color-like interference in the dynamic gesture database, and the corresponding depth video stream is used as an auxiliary to provide depth Kan value information for the tracking system, and the tracking results are shown in Figure 7.
The flexibility of interactive music based on wireless wearable devices is mainly reflected in the customization of computer-generated note rules. By controlling the control components such as push coils and buttons given in the software interface, it is easy to change the parameter settings such as timber change information, number of notes, and high pitch range that it sends to the hardware sound source when it receives a trigger message, and by adjusting these parameters, it is possible to form a new sound according to the modifier’s idea. This enhances the potential of the work to be recreated, allowing other performing groups to recreate the work and change the acoustic characteristics of the work as they wish. The simulation results of the potential shift field distributions of the four types of forked-finger electrodes with the same area and different shapes show that the potential shift field distributions of the serpentine, spiral, and serrated forked-finger electrodes are significantly more concentrated than those of the ordinary forked-finger electrodes, and the average value of the potential shift field size between the serrated forked-finger electrodes is the largest in terms of potential shift field size. Among them, both serpentine fork-finger electrodes and spiral fork-finger electrodes are used to enhance the effect of potential shift field by increasing the number of fork-finger electrode bends. For the ordinary fork-finger electrode structure, the individual fork fingers can only interact with each other in one dimension, while in the serrated fork-finger electrode, the horizontal and vertical branching electrodes can interact with each other in two dimensions, thus enhancing the potential shift field between the electrodes. The concentrated distribution of the potential shift field changes the stored charge between the sensor electrodes, thus changing the capacitance between the electrodes, and the more concentrated the potential shift field distribution is, the larger the stored charge is, and the larger the capacitance value is. Therefore, for the initial capacitance of the four fork-finger electrode type tactile sensors, the serrated fork-finger electrode has the largest, the serpentine and spiral fork-finger electrodes have the second largest, and the plain fork-finger electrode has the smallest. The results are shown in Figure 8.
By looking at the specific hardware devices, software devices, and other controlled target carriers from the model and description, it is possible to know how flexible each of them is and how many parameters can be used as controlled targets. The balance between completing the work and ensuring stability can be found.
5. Conclusion
Starting from several technical points (digitization, mapping, and controlled target) of new media real-time interactive electronic music, this paper first introduces the concepts of control source, converter, point, mapping line type, mapping tool, controlled target, model and description, etc., and provides a more detailed overview of the technical issues in new media real-time interactive electronic music. Based on this, the relationship between these technical factors and the stability, convenience, and flexibility of the scheme of the work are discussed. The design of a wearable device based on a flexible sensor is investigated, using a data glove as a specific object of study, in terms of the preparation and sensing performance testing of the flexible sensor, device circuit design, data analysis and processing, and gesture recognition. By sewing the rGO-coated fiber sensor onto the object glove, we designed and fabricated a flexible data glove that is comfortable to wear and less costly. This work focuses on the use of solid-state potential sensing technology, combining novel receptors and nanopolymers as an ion-electric conductive layer for the detection of potential signals. The glove can be used to monitor the flexion state of the ten knuckles of the hand due to the good sensing performance of the sensor. Using the obtained data, the static gesture recognition and dynamic gesture recognition are implemented, respectively, by combining some filtering and noise reduction algorithms. From the viewpoint of the nature of interactive art expression and its flow, the combination of interactive art and wearable sensors is the inevitable result of the development of art science and technology. Only after understanding and accepting the definition, development history, technical application, and singularity outlook of flexible wearable sensors can we discern the future of the development of interactive art at present.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This study was supported by Northern Minzu University Scientific Research Key Project: Northwestern Flower Play Draft (1979–2018), China (Grant no. 2019MYA09).