Abstract
A compact integrated system-on-chip (SoC) architecture solution for robust, real-time, and on-site genetic analysis has been proposed. This microsystem solution is noise-tolerable and suitable for analyzing the weak fluorescence patterns from a PCR prepared dual-labeled DNA microchip assay. In the architecture, a preceding VLSI differential logarithm microchip is designed for effectively computing the logarithm of the normalized input fluorescence signals. A posterior VLSI artificial neural network (ANN) processor chip is used for analyzing the processed signals from the differential logarithm stage. A single-channel logarithmic circuit was fabricated and characterized. A prototype ANN chip with unsupervised winner-take-all (WTA) function was designed, fabricated, and tested. An ANN learning algorithm using a novel sigmoid-logarithmic transfer function based on the supervised backpropagation (BP) algorithm is proposed for robustly recognizing low-intensity patterns. Our results show that the trained new ANN can recognize low-fluorescence patterns better than an ANN using the conventional sigmoid function.
1. Introduction
The development of low-cost portable instruments for rapidly analyzing genetic assays would significantly advance the level of medical services globally. The polymerase chain reaction (PCR) amplification and the capillary electrophoretic (CE) techniques are often adopted for genetic analysis. A complex system that can process full PCR amplification and data analysis tasks usually involves integration of control, optical, thermal, fluid channel, and data acquisition systems. For example, a portable system providing full PCR-CE functions was developed earlier for genetic analysis [1]. The system demonstrated the feasibility of on-site genetic analysis. However, the expense to build such a system is considered relatively high. The entire integrated system consists of multiple PCR chambers, heaters, sensors, solid-state laser excitation light source, fluorescence detection optics, electronics, CE separation microchannels, and power supplies. The data was collected and processed in a portable computer. Recently, a real low-cost (~10US$) pocket-sized PCR thermocycling device has been developed based on a smart technique of simultaneously pseudoisothermally heating multiple zones of a loop channel for PCR amplification [2]. This thermocycler does not contain the CE separation, the fluorescence detection, and the data analysis functions. Theoretically, multiple PCR amplification results can be rapidly generated in parallel and displayed simultaneously by using multiple of these low-cost devices. Therefore, patterns of an array of the PCR resulted samples can potentially be generated similar to the genetic assay patterns on a microchip. Moreover, the integration of PCR and electrochemical (EC) transduction functionality on microfabricated silicon/glass-based devices for DNA amplification and detection was shown successfully [3]. Their microfabricated device needs to operate with external control and data-acquisition systems.
Most of the research efforts for PCR analysis tools were focusing on the development of the PCR microdevices, the associated thermal systems, the optical systems, and the data analysis software tools. However, to our best knowledge, the data acquisition and analysis system for examining PCR samples or assays is usually a computer equipped with specific PCR analysis software but not a compact hardware solution.
Regarding the goal of building a real compact PCR analysis system that can rapidly find and analyze the desired genetic patterns, the existing data acquisition and analysis systems (e.g., portable computers and interfaces) are considered relatively large in size and heavy in weight. In addition, human inspectors cannot recognize the genetic assay patterns as easily as written characters with explicit meanings. Manual massive PCR data analysis can be very time consuming. Therefore, people involved in “the human genome project” have used perceptron-like neural networks for helping to recognize the DNA fragments with specific functions [4]. For gene-recognition purpose, a perceptron was first trained by using the datasets consisting of nucleotide sequences of known functional sites (e.g., transcription initiation sites (promoters), transcription termination sites (terminators), or splice-junction sites). Patterns of fragments of the entire DNA sequence were then fed to the perceptron nucleotide by nucleotide to check if any site of interest appears at a particular position in the fragment. In the protein coding region recognition task by Guan et al. [5], their multisensor /neural network successfully identified 96% of the 17,576 sequence positions as coming from coding or noncoding regions. In the splice junction recognition experiment, the resulting recognition rate was 99% for acceptor junctions and 96% for donor junctions.
Nonetheless, if the genetic analysis task needs to be conducted on a hazardous or dangerous field (e.g., potentially disease-contagious environment), a compact, autonomous, and even disposable PCR data analysis system would be preferred. Therefore, by taking advantages of the VLSI microfabrication technologies and artificial neural network theories, we proposed a microsystem consisting of a unique optical configuration setup, a differential logarithm sensor-processor array chip, and an ANN SoC processor chip for fast recognizing and analyzing the PCR prepared genetic patterns.
In typical PCR amplification procedure, a dual-labeled (i.e., for sample and reference channels) assay design is commonly used for identifying differentially expressed genes. This method also reduces the sources of variability/noise due to aspects of individual spot that affects both specimens similarly [6]. In order to accurately calculate the density of the sample DNA material in a particular dot/well after the PCR amplification, the integral of the total fluorescence intensity (presumably representing the density of the DNA materials inside the dot/well) from the topological profile of the dot/well is usually computed. The logarithmic value of the ratio of the two intensities of the fluorescent-dye-labeled specimens (one for the sample specimen, the other for the reference specimen) measured from the same dot/well is calculated based on the fluorescence assay image. The ratio of the two intensities would provide the normalized population of the genetic material in the dot/well disregarding the initial population density before PCR amplification. The logarithmic operation would amplify the small signals. In most of the commercial available solutions, the fluorescence assay image is usually scanned by a color scanner with high resolution and then transferred to a computer for image analysis. The profile analysis software usually computes the normalized intensity of dot/well after dot/well sequentially.
The intensity of fluorescence light is usually relatively low. Using higher excitation light intensity can lead to brighter fluorescence patterns. Increasing the integral of detection time can enhance the received fluorescence patterns. However, lower power consumption and faster detection are preferred. Furthermore, some fixed-pattern noises in the input pattern may exist (e.g., fixed pattern noises created by scattered lights, nonuniformity of the responsivity of the detector array). These noises may introduce errors to the measurement of the density of the DNA materials.
In order to fast parallel-process the data and resolve the ambiguity induced by the noises in the data analysis task, a trained artificial neural network is considered a solution. The parallel processing capability comes from the nature of the ANN’s multiple input architecture [7–9]. In contrast with the conventional sequential data analysis methods, the data analysis throughput would be increased linearly as the number of dot/well increases. Regarding potential noises, the ANN will automatically take the noises into calculation in the latter recognition phase because the ANN can learn from the training patterns that contain the fixed pattern noises in the learning phase. In addition, because of the natural capability of associating noisy input patterns with output index of a trained ANN, noises introduced by other factors will not significantly affect the ANN’s recognition capability. In conjunction with the natural capabilities of an ANN listed above, a signal amplification stage that can augment the low-fluorescence input before the ANN stage would help the ANN to acquire data more reliably, and thus result in a more robust data analysis capability of the entire system.
2. Biochip Module Architecture
We proposed a hardware microsystem that is suitable for real-time, on-site, robust genetic fluorescence data analysis (Figure 1). This envisioned biochip module architecture consists of an on-chip assay with an array of clusters of dual-labeled genetic dots/wells, a dual-color beams module, an imaging lens, a bioimaging optoelectronic microchip with coated color filters (Figure 2), a parallel analog data-transfer bus (optional depending on the implementation method), and an artificial neural network (ANN) module for image analysis.
The operational function of each module is explained below along the optical and electrical signal pathways. The dual-labeled genetic dots/wells are simultaneously excited by two monocromatic excitation beams (e.g., 532 nm with a bandwidth of 10 nm from a green diode laser pointer source for the cyanine Cy3 dye, and 635 nm with a bandwidth of 10 nm from a red solid-state diode laser source for the cyanine Cy5 dye) according to the receiving bandwidths of the sample and the reference channels. The assay can be either front-side or backside illuminated as long as a clear fluorescence image of the dot/well array is generated. Two fluorescence patterns with different peak wavelengths are produced (e.g., peak value at 570 nm from the Cy3 dye and peak value at 670 nm from the Cy5 dye, the two spectral profiles are highly distinguishable), and imaged onto the bioimaging chip through an imaging lens. Each unit on the bioimaging chip contains two sensor channels. One sensor is coated with a thin-film microfilter for wavelength A (e.g., 580 nm with a narrow transmission bandwidth of approximately 40 nm of a deposited thin-film filter [10]). The other sensor is coated with a thin-film micro-filter for wavelength B (e.g., 675 nm with a transmission bandwidth of 40 nm). The two optical signals are received simultaneously and processed separately through the posterior electrical circuits. A bioimaging chip made of an array of differential logarithm circuitry was designed (Figure 2 (bottom left)). Two analog photoelectric voltage signals produced by the separated logarithmic amplifier circuit channels are fed continuously into the differential pair circuit. The difference of the two input voltages is then represented by the output voltage from the differential pair circuit. The calculation of the logarithm of the ratio of the sample to the reference fluorescence intensities is effectively accomplished in this chip. The analog output from each unit in the bioimaging chip is then sent to the posterior artificial neural network module through the parallel data bus.
The ANN stage is responsible for filtering and recognizing the desired assay cluster patterns. Fixed pattern noises and noises caused by the nonlinear circuitry are expected to be accommodated after the ANN is trained. Either unsupervised or supervised learning algorithm can be adopted to train the ANN. The ANN in the biochip module architecture can be implemented by either hardware or software. In this work, we provide a hardware implementation (i.e., a weight-reconfigurable winner-take-all ANN chip suitable for the Kohonen self-organized filter algorithm [7]) for the unsupervised version, and a computer-simulated ANN (i.e., a feedforward ANN using the back-propagation (BP) learning algorithm [8, 9]) for the supervised version. In the hardware implementation, the weight values are reconfigurable and stored in memory devices. By adopting a massively paralleled neural computing paradigm and a mixed-signal deep submicro fabrication technology, the ANN can be implemented on a single VLSI chip. A row/column parallel data flow architecture is used to connect all on-chip systems, and to reduce data bandwidth limitations due to conventional data bus architectures.
Because the weak fluorescence signals are enhanced by the imaging chip and automatically analyzed by the noise-tolerable neural network module, the entire architecture system is expected to robustly conduct the recognition task.
3. Hardware Module Implementation
3.1. Bioimaging Chip and Its Nonlinear Circuitry
The proposed bioimaging chip consists of an array of differential logarithm processor unit and row/column readout circuit. A prototype layout of an array of differential logarithm unit is shown in Figure 2. Each unit contains two size-matched logarithmic amplifier circuits and a differential pair circuit. As described in the previous section, each unit produces an analog voltage output to represent the difference between the logarithms of the sample (experimental) input and the reference (control) input.
The key logarithmic amplifier circuit is designed after the works of Chamberlain and Lee [11] and Mead [12], but with an additional -well/-sub junction layer for effectively isolating cross-talk noises among the logarithmic amplifier processor unit array.Chamberlain and Lee first adopted the intrinsic vertical - junction (-diffusion/-sub) under the source terminal of an NPN transistor to function as a reverse-biased photodiode. A wide dynamic range silicon photodetector can be implemented [11]. Mead has a similar design by using a vertical parasitic bipolar transistor for photosensing part [12]. This vertical bipolar is a natural byproduct structure in the CMOS process. The -diffusion/-well/-sub structure is the PNP bipolar transistor existing in any PMOS transistor in an isolated section of -well region. This design is the fundamental building block of his silicon retina.
The optoelectronic logarithmic amplifier circuit for each channel in this work was fabricated by using the MOSIS AMI 1.5-μm 5-V ABN BiCMOS -well process as shown in Figure 3 [13]. The photodetector is made of a vertical -well/-base/-emission bipolar detector. Two diode-connected NMOS transistors are connected in series with the bipolar detector. The detecting area collects the fluorescence inputs. Mainly due to the diode configured NMOS transistors, the V-I characteristic of this normally “OFF” and low-power circuit behaves logarithmically while operating in the subthreshold region, and similar to a square root curve (~ ) while operating in the saturation region. This logarithmic amplifier circuit can detect light intensity as low as approximately 10 nW and consumes energy from 100 nW to 2 μW (: 5 Volts) depending on the incident intensity and wavelength.
3.2. Weight-Reconfigurable Artificial Neural Processor Chip
The SoC architecture design of the weight-reconfigurable ANN processor consists of an input neurons array, a programmable synapse weight matrix, an array of output neurons, a winner-take-all module, and a membership encoder [14, 15]. The input neurons array has input neurons that are used to buffer the input vector. Each input vector has analog components (generating from the preceding bioimaging chip). The programmable synapse weight matrix is composed of synapse cells for the -dimensional codevectors. The output neuron array is composed of summing neurons with selectable sigmoid or sigmoid-logarithmic transfer functions. The winner-take-all module consists of competitive circuit cells, which perform parallel comparison among inverted distortion values and choose a single winner. The membership encoder circuit is an -to- decoder that uses binary codes to encode classes.
The ANN processor works as a learning accelerator in the learning phase at a time complexity (1) for processing each learning iteration. Its programmable weight matrix can be either generated by using the on-chip self-organization learning procedure or be uploaded by the BP training subsystem [15]. The ANN processor also realizes a full-search vector quantization process for each input vector at a time complexity (1) in the recognition phase.
This ANN processor can also support the multiple winner-take-all scenario (e.g., more than one classes that the input assay pattern may belong to, or multiple desired patterns that the input assay pattern are similar to). After a winning pattern (the most likelihood) was picked out from the prestored classes/codevectors in the recognition task according to a particular analog input vector, the associated circuitry of this winning pattern will be disabled in the next recognition iteration. Therefore, a second-winner pattern (the second likelihood) can be chosen later according to the same input vector. By repeating the procedure stated above, multiple-winner patterns could be chosen for the current input pattern eventually.
The ANN chip can learn unsupervised if the selforganization learning procedure is adopted. In this case, the ANN chip can perform on-chip learning in the learning phase. For the supervised learning version (e.g., back-propagation algorithm or its variations), the weight update procedure usually involves complex computations that require further signal processing circuits in order to achieve the on-chip learning purpose. Further real estates on chip are then required to accommodate the circuits.
A prototype ANN SoC chip using a scalable 2-μm 5-V CMOS technology was designed, fabricated, and tested. Its chip layout and design features are shown in Figures 4 and 5, respectively. This prototype chip includes 25 input neurons, weight cells, 64 output summing neurons, 64 winner-take-all cells, and a 64-to-8 membership encoder. The estimated power dissipation is 50 mW at 10 MHz. Its equivalent computation power is about 16 giga-operations per second.
An engineering version of the ANN SoC SiP (silicon intellectual property) has been under development to enable the proposed miniaturized PCR system-on-chip design using the TSMC 130-nm 1.2-V CMOS technology. The scalable ANN prototype chip can be converted into a design containing 100 input neurons, weight cells, 256 output summing neurons, 256 WTA cells, and a 256-to-8 membership decoder. The envisioned chip size is approximately . Its estimated power dissipation is about 120 mW at a 100D vector throughput rate of 100 MHz. Its equivalent computation power is about 2.5 tera-operations per second. Because of this ANN SoC SiP design, the feasibility of the proposed low-power, real-time, and on-site PCR assay analysis on an integrated microsystem becomes promising. The proposed microsystem would be useful especially in a scenario of finding desired or suspicious biopatterns in a massive amount of data.
4. Simulations and Empirical Results
4.1. Numerical Simulations of Biosignature and Optical Character Recognition
In this section, two pattern-recognition tasks were computer simulated to demonstrate the feasibility of using an ANN for our proposed biochip module architecture. A novel sigmoid-logarithmic function is also integrated within the learning algorithm (i.e., back-propagation algorithm) to demonstrate the capability of recognizing relatively dim patterns. The study in this section will assist our future circuit design and may contribute to the new techniques for medical image processing.
In most of the fluorescence spectroscopy applications, the fluorescence patterns usually have relatively low intensities and are difficult to analyze. We know that high-excitation intensities and long exposure time can lead to stronger fluorescence signals. However, low-energy consumption and fast detection are the design goals for our biochip module architecture. Therefore, if the posterior ANN of our biochip architecture can analyze dim fluorescence patterns better, we can potentially use relatively lower energy and shorter time to conduct the analysis task.
Regarding the neural network learning algorithm, the simplest transfer function that we can use in the algorithm is a linear ramp function (e.g., linear slope between 1 and −1, flat and continuous outside [1, −1]). However, higher recognition capability can be achieved by using nonlinear transfer function in the neural network learning algorithm.
The nonlinear sigmoid (logistic) transfer function is usually adopted in artificial neural network models because its derivative can be easily obtained algebraically. For example, we define as a sigmoid function:
The first derivative of can easily be calculated by using the identity . Therefore, computation complexity and cost of hardware or software can be reduced.
In addition, a single-layer feedforward network (SLFN) with any bounded continuous nonconstant activation (transfer) function or arbitrarily bounded activation (transfer) function with unequal limits at the infinities can form decision regions with arbitrary shapes [16–18]. A multilayer perceptron architecture naturally consists an SLFN in its structure. As long as a function has unequal upper bond, lower bond, and monotonic behavior, it can be used as a transfer function.
For the above computational advantage and theoretical reasons, we proposed a novel piecewise sigmoid-logarithmic function that also yields similar mathematical identities and computational benefits:
In this piecewise function, and is the net weighted input to the transfer function . The central part (net input ranging from −2 to 2) of the original sigmoid function was replaced by two asymmetric pieces of logarithmic curves.
To demonstrate the capability of recognizing dim patterns by using a feedforward ANN with sigmoid-logarithmic transfer function, a simple pseudo genetic assay analysis task and an optical character pattern-recognition task were simulated. MATLAB programs were created to train an ANN and examine its performance.
A 100-100-2 (100 inputs, 100 hidden neurons, and 2 output neurons) artificial feedforward neural network was chosen to perform both recognition tasks. For the biosignature recognition, 20 patterns/clusters on a microchip genetic assay were prepared (Figure 6). For simplicity, we used the pixellated image of a fluorescence image of the sample material to represent a normalized (i.e., after taking logarithmic value of the ratio of the sample to reference signals) but noisy image for the analog input to the ANN. Additional seven datasets by rescaling the gray level of the original dataset with different rescaling factors were also prepared (factors: 1/3.16, 1/10, 1/31.6, 1/100, 1/316, 1/1000, 1/10000 of the original gray level). Noticing that, both brightness and contrast levels of these new biopatterns were reduced by the rescaling factor. The three desired biopatterns (solid framed in Figure 6 that were randomly picked) are what we were searching in a scenario of finding the designed PCR assay cluster pattern of the subject with certain disease.
(a)
(b)
Similarly, in the optical character-recognition task, a dataset containing 100 different alphanumeric letters 1, 2, 3, and 4 was prepared first (as shown in Figure 7). By using the same rescaling factors (as listed in the biopattern recognition), we generated the other seven datasets that contain dimmer hand script patterns.
In both experiments, the digitized biopattern and character datasets were used for both training and testing the artificial neural network. In contrast to the traditional method of preparing independent training and test datasets, the test datasets were assigned to be identical to the training datasets in order to examine the feasibility of the proposed ANN model with the sigmoid-logarithmic function.
For simplicity, the intensities of the high-resolution pixels of each original fluorescence dot in Figure 6(a) were averaged and replaced by a single super pixel in the pixellated image in Figure 6(a). If the patterns with the lowest resolution (one pixel for one dot/well) can be correctly recognized by the ANN, the patterns with higher resolution should be recognized by the ANN with higher recognition accuracy. To economically implement the bioimaging chip, one neuron unit would be sufficient for receiving all the lights emitting from the fluorescence profile of the imaging dot/well. The results of this simulated ANN model would assist the future design of the proposed architecture. The average of the intensities of the pixels of the original fluorescence patterns is considered simple yet physically reasonable.
The back-propagation training using sigmoid-logarithmic transfer function and gradient descent method was conducted to find a convergent weight configuration (with fixed learning rate ). A criterion is employed to count the percentage of training data that has been learned with an error less than 20%. In order to guide the weight configuration closer to a convergence condition, the BP training using regular sigmoid function was conducted first. The BP training using sigmoid-logarithmic function was conducted afterwards.
The entire procedure of the BP training algorithm using sigmoid-logarithmic transfer function is described as follows.
(i) Prepare the input patterns for the feedforward multilayer perceptron (MLP) neural network.(ii) Assign the target values for the associated input patterns.(iii) Use the input patterns to train the multilayer perceptron with sigmoid transfer function until the criterion value becomes close to one. Now the weight configuration is closer to a convergence condition for latter training.(iv) Use the weight values obtained in the previous step as the initial weight condition for training the multilayer perceptron with the logarithmic-sigmoid transfer function. The regular BP algorithm using the gradient descent method is again adopted. After the criterion becomes one, the training is finished.(v) Use this trained MLP with logarithmic-sigmoid transfer function to recognize the test data set. Examine the recognition accuracy.
The detailed conditions and pseudocodes of the BP algorithm with sigmoid and logarithmic-sigmoid transfer function are provided as the following.
The initial weight values for the first weight matrix and second weight matrix were given randomly. The range of these random weight values was set between −1 and 1. However, the range of the trained weight values was unlimited. The input vector was the vectorized pixellated pattern. The associated target values were given. The learning rate (coefficient in front of the gradient partial derivative) is parameter . The number of iterations is parameter iter. The training data set was used for testing as well to verify the feasibility of this proposed algorithm.
The pseudocode for the regular back-propagation algorithm using sigmoid transfer function is listed in Algorithm 1.
The pseudocode for the back-propagation algorithm using the piecewise sigmoid-logarithmic transfer function is listed in Algorithm 2.
4.2. Simulations Results
The result of recognizing all of the normalized genetic assay datasets by the trained 100-input-100-hidden-neuron-2-output-neuron network is shown in Table 1 (BIO). The network using new transfer function can still find one desired biosignature in the dataset with factor 1/100 of the original gray level. However, the network using conventional sigmoid transfer functions cannot distinguish biopatterns well in the datasets with factor 1/3.16 of the original gray level or below. Consistent recognition accuracy was obtained for the optical character recognition (OCR) tasks (Table 1 (OCR)). The network using new transfer function can recognize 32 characters in the dataset with factor 1/100 of the original gray level but not the network with regular sigmoid function. The recognition accuracies for datasets with a factor below 1/316 become constant because the network tends to recognize one category perfectly according to the converged weight configuration in this particular experiment.
5. Conclusion
A new optoelectronic multichip microsystem for real-time field applicable robust dual-label PCR assay analysis was proposed. This microsystem architecture contains a front-end bioimage chip for analog signal conversion and augmentation, and an artificial neural network for the autonomous data analysis purpose. A differential logarithmic bioimage chip is designed and presented. The typical data analysis procedure of taking logarithm of the ratio of the normalized post-PCR sample intensity is conducted effectively in this differential logarithmic bio-image chip. A single channel logarithmic circuit of the differential logarithmic bioimage chip was designed, fabricated, and characterized. The weak fluorescence signals can be amplified by this logarithmic amplifier circuit for easier data analysis. Regarding the ANN subsystem, an unsupervised hardware version: a weight-reconfigurable winner-take-all ANN SoC chip suitable for selforganized Kohonen filter algorithm, and a supervised software version: a computer-simulated ANN using back-propagation algorithm with a novel sigmoid-logarithmic transfer function is presented. The back-propagation neural network learning algorithm using the sigmoid-logarithmic function was successfully simulated. The simulation results show that a trained ANN using this new transfer function can classify low-fluorescence patterns better than using the conventional sigmoid transfer function. This software model might be applicable to other medical image processing tasks. In summary, by integrating the optical setup, the bioimage chip, and the artificial neural network processor with excellent performances and advantages listed previously, we can envision the success of using this compact microsystem to conduct on-site, real-time, noise-tolerable, and high-throughput dual-labeled genetic expression analysis efficiently.
| ||||||||||||||||||||||||||||||||||
| Algorithm 1 | ||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||||
| Algorithm 2 | ||||||||||||||||||||||||||||||||||||||||
Acknowledgment
The authors would like to thank Dr. Armand R. Tanguay, Jr. at University of Southern California for the kind instructions and funding for implementing the logarithmic amplifier circuitry.