Physical Properties of Hydrocarbon and Non-Hydrocarbon Systems: Application of Machine Learning and Artificial Intelligence ApproachesView this Special Issue
On the Investigation of Effective Factors on Electronic Structure Properties of Transition Metal Complexes: Robust Modeling Using GPR Approach
Materials discovery is usually done using high-throughput computational screening. The use of costly and complex direct density functional theory (DFT) simulation methods has been commonly used to determine subtle trends in spin-state ordering and inorganic bonding of inorganic materials and, in general, to predict the electronic structure properties of transition metal complexes. A Gaussian process regression (GPR) framework consisting of four kernel functions is introduced for spin-state splitting estimation through inorganic chemistry-appropriate empirical inputs. To this end, the present study reviewed an extensive range of data values from earlier works. According to statistical analysis, the GPR model showed very good performance. The coefficients of determination were calculated to be 0.986 for the exponential and Matern kernel functions, suggesting the highest predictive power of these methods. Moreover, the sensitivity of output to inputs was measured. Artificial intelligence (AI) helped accurately predict the target values through various input ranges.
Novel compounds, catalysts , and materials  are routinely discovered via high-throughput computer screening [3, 4]. Numerous screening and recognition experiments still rely on first-principles modeling, but the increased computational expense simulation means that only a narrow subset of the chemical domain can be explored [5, 6]. Lower thresholds of hypothesis, such as machine-learning designs, have emerged as alternatives to traditional methods for efficiently evaluating the latest candidate substances to speed up the exploration . Computational chemists have recently discovered a broad range of uses for artificial neural networks (ANNs) [8–10]. The versatility of machine-learning methods to potential energy surfaces and, therefore, force field simulations were first recognized [9, 11–13]. Molecular or heterogeneous catalyst and substance exploration have lately been studied in exchange-correlation functional advancement [8, 14], common Schrödinger equation strategies , functional hypothesis for orbital-free density [16, 17], numerous body expansions , dynamics velocity [19, 20], and band-gap estimation [21, 22] among others.
The proper identification of widely relevant qualifiers that allow the ANN to be used dynamically beyond particles in the learning collection, e.g., for bigger molecules or those with varied chemical reactions, are essential difficulties for ANNs to substitute direct computation first-principles techniques. ANNs have had the greatest effectiveness thus far beyond proof-of-concept demonstrations developing force fields for well-defined substances, such as water [23, 24]. To make energetic predictions in organic chemistry, compositional qualifiers like the Coulomb matrix  or regional chemical surroundings and adhesive descriptions [26, 27] have been helpful when considering only a small number of mixtures (e.g., C, H, N, and O). Molecular resemblance, force field advancement, numerical structure-activity  correlation, and commutative group hypotheses have all been successfully evaluated using cheminformatics in the past. There are just a handful of force fields  for transition metal combinations covering the whole spectrum of inorganic chemical bonding interactions . More rigorous construction of qualifiers is needed to accurately anticipate the characteristics of open-shell transition metal combinations since spin state and coordination setting influence binding .
In the same way, qualifiers that were effective for organic molecules are ineffective for inorganic crystalline particles . In transition metal combinations, it is well-recognized [33, 34] that the responsiveness of electronic characteristics (such as spin-state separation) correlates strongly with the ligand-atom linkage and ligand-field power [35, 36]. When substituting distantly (e.g., tetraphenyl porphyrin for base porphin), the impact will be restricted because ligands with the identical metal-bonding atom can have vastly distinctive ligand-field powers (for example, C for both weakened field CH3CN and robust field CO). Therefore, the transition metal complex qualifier collection must cautiously balance metal-proximal and metal-distant qualifiers. A second issue pertains to establishing ANN estimations of first-principles characteristics in transition metal chemistry and associated inorganic substances. Transition metal complexes cannot benefit from efficient correlated wave function theory techniques (e.g., MP2) because optimal procedures for transition metal complexes remain mysterious . In transition metal chemistry, while potential paths for ANNs involve projecting lower-level theory findings to a higher-level hypothesis (e.g., from semiempirical assumption) , as has been shown for atomization energies  and more recently reaction obstacles , appropriate degrees of theory for inference are less apparent. The level of precise (Hartree–Fock, HF) transfer to incorporate in the analysis of transition metal combinations is also unclear. Suggestions range from no interchange to alternatively low or large quantities of accurate interchange in a system-dependent way, notwithstanding inordinate delocalization faults in approximation DFT on transition metal combinations [35, 41, 42], with these amounts being determined by the system. It is true that measuring uncertainty about functional choice in energetic forecasts, particularly the responsiveness of projections to include precise interchange, has garnered a lot of attention lately. To get a direct number and understand how the exchange fraction [33, 34] affects spin-state splitting, one must first determine how responsive it is to interchange. To translate empirical forecasts or provide measurements of accuracy on calculated information, a machine-learning system that anticipates spin-state ordering among interchange rates would be helpful.
As a general rule, any presentation of artificial intelligence in inorganic chemistry, such as for the fast identification of novel spin-crossover combination [43, 44], the use of dye-sensitizers throughout solar panels , or the quick assessment of spin-state sequencing to determine the responsiveness of open-shell catalysts, should meet two requirements: (i) qualifiers must integrate metal-proximal and metal-distant properties and (ii) they must also anticipate spin-state sequencing when exchange-correlation blending is taking place. Cheminformatics-inspired transition metal complex structure creation instruments help us make progress toward both of these goals in this study. To educate GPR, as a new method, to anticipate the transition metal complex characteristic, we also developed structure-functional responsiveness correlations in transition metal combinations. In this study, various analyzes have been used to evaluate the proposed models. Our goal is to provide a model with high accuracy in predicting this goal parameter.
2. GPR Model
The present work adopted machine-learning and GPR to handle probabilistic (Bayesian) uncertainties [46, 47]. This approach can simply solve complicated problems. Nonlinear GPR techniques may be employed using small training datasets and integrate new evidence as the data points rise in number . Overfitting is avoided to a great extent as optimization includes fewer hyperparameters in the training phase. The model parameters are determined by the GPR training dataset [49, 50]. Previous data are incorporated into the process along with empirical data to construct the GPR model. GPR operates based on posterior distribution calculations rather than identifying the highest consistency with empirical data, unlike traditional machine-learning algorithms .
Let x be the input and y be the output. Also, denotes a random testing dataset, and is a random training dataset. GPR begins with where XL and YL are the independent variable and target, respectively. Furthermore, denotes the observation noise, is the noise variance, and is the unit array. As a result, the Gaussian noise model connects y values to f(x). f is assumed to be a random function completely definable by the mean functions and covariance . Similarly,where XT and YT are the testing dataset independent variable and target, respectively, f(x) is the Gaussian process distribution whose kernel function is , and mean function is m(x) . Thus,
Explicit basis functions (BFs) could be employed to determine m(x). It should be noted that m(x) is typically assumed to be zero for simplification purposes, since a constant m(x) is difficult to find . Therefore,
Based on the aforementioned parameters ,
The Gaussian conditioning rule is used to obtain the yT distribution (where ΣT is the covariance, and μT is the mean) :
The output estimate of the testing dataset can be obtained by the independent variable and training dataset. The kernel function in the training phase (with asymmetric, invertible matrix) strongly influences GPR predictive performance. The present study implemented the learning technique to identify the most efficient kernel function, manipulating the Matern, exponential, squared exponential, and rational quadratic functions [60, 61].
The Matern kernel is given bywhere α > 0 is the length scale, ℓ > 0 is the scale mixture, σ denotes amplitude, and σ2 is the variance. Moreover, is the modified Bessel function, is a positive variable, and Γ stands for the gamma function. For = 0.5, the Matern kernel converts into the exponential kernel function, whereas = 1.0 transforms the Matern kernel into the squared kernel function (two particular cases of the Matern kernel) [62, 63].
To maximize mode accuracy, 1/5 of the data was employed as the testing dataset to measure model validity, while the remaining data that were exploited was the training dataset for spin-state splitting evaluation. Details of the data are given elsewhere . Performance evaluation was carried out using MSE, R2, STD, MRE, and RMSE. These statistical indices are calculated as [65–68]
3. Accuracy Estimation
A portion of data may show inconsistency with the dataset, with some data being suspected. Such data points majorly imply empirical errors [69, 70]. It is necessary to identify suspected data points since they would diminish predictive performance . To detect suspected (outlier) data, the present study adopted the leverage approach, in which outliers are identified using the hat matrix H and critical leverage limit :where U is an i × j matrix, i denotes the number of parameters, and j stands for the number of training data points [73, 74]. Figure 1 shows William’s plot of the standardized residuals versus the hat value in order to evaluate spin-state splitting data accuracy. The reliable region is represented by a critical leverage limit along with standardized results ranging between −3 and +3. As shown, the dataset is concluded to be satisfactory for the model training and testing phases.
4. Results and Discussion
To measure the performance of the model, the present work utilized statistical parameters to evaluate the consistency between the empirical data and the model estimates. Table 1 provides the comparison between the estimates and empirical data. The coefficient of determination was obtained to be 0.985, 0.984, 0.986, and 0.986 for the rational quadratic, squared exponential, Matern, and exponential kernels, respectively. According to the STD, RMSE, MSE, and MRE values, the GPR models showed satisfactory training performance. Moreover, the models should predict spin-state splitting accurately. Hence, testing data were used to evaluate the models. The GPR models with the exponential and Matern kernels should have the highest spin-state splitting prediction performance.
Figure 2 shows the comparison between the empirical data and model estimates. As can be seen, the model estimates well agreed with the empirical spin-state splitting data, suggesting high accuracy for the proposed models. As a result, the GPR models can be claimed to have excellent performance in spin-state splitting estimation.
Figure 3 shows the comparison between the empirical data and the predictions of the models. The fitting of the predictions to the corresponding empirical data points was calculated to have correlation coefficients above 0.9816. The fit lines significantly cross the bisector line (45°) as the model accuracy measure. However, the model with the exponential and Matern kernel functions showed the largest correlation and thus the highest performance.
The relative deviations of the empirical data and the estimates are shown in Figure 4. According to it, the absolute deviations of the Matern, rational quadratic, and squared exponential kernels were calculated to be below 2000%, whereas the exponential kernel showed an absolute deviation below 1500%.
The GPR models were found to be efficient and effective in the estimation of spin-state splitting. To ensure the spin-state splitting estimation performance of the proposed models with different MOFs, the models were compared to earlier studies. Janet and his colleagues used the RMSE statistical parameter to compare LASSO, KRR, SVR, ANN, and KRR models in predicting this parameter . By comparing their results with the results given in Table 1 of our study, it is proved that our proposed models have a higher ability to predict the target data.
The present study developed GPR models using four kernel functions, i.e., rational quadratic, Matern, exponential, and squared exponential kernels to evaluate spin-state splitting. As they showed good agreement with the empirical spin-state splitting data, the proposed models were concluded to have high performance. However, the GPR model with the exponential and Matern kernels showed the highest performance. Moreover, a comparison of the models to earlier works in the literature revealed that the proposed GPR models outperformed earlier models.
The data used to support the findings of this study are included within the article.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
J. K. Nørskov and T. Bligaard, The Catalyst Genome, Wiley Online Library, Hoboken, NJ, USA, 2013.
R. Gómez-Bombarelli, J. Aguilera-Iparraguirre, T. D. Hirzel et al., “Design of efficient molecular organic light-emitting diodes by a high-throughput virtual screening and experimental approach,” Nature Materials, vol. 15, no. 10, pp. 1120–1127, 2016.View at: Google Scholar
A. M. Virshup, J. Contreras-García, P. Wipf, W. Yang, and D. N. Beratan, “Stochastic voyages into uncharted chemical space produce a representative library of all possible drug-like compounds,” Journal of the American Chemical Society, vol. 135, no. 19, pp. 7296–7303, 2013.View at: Publisher Site | Google Scholar
A. Lekomtsev, A. Keykhosravi, M. B. Moghaddam, R. Daneshfar, and O. Rezvanjou, “On the prediction of filtration volume of drilling fluids containing different types of nanoparticles by ELM and PSO-LSSVM based models,” Petroleum, 2021, https://www.sciencedirect.com/science/article/pii/S2405656121000274.View at: Publisher Site | Google Scholar
R. Setiawan, R. Daneshfar, O. Rezvanjou, S. Ashoori, and M. Naseri, “Surface tension of binary mixtures containing environmentally friendly ionic liquids: insights from artificial intelligence,” Environment, Development and Sustainability, vol. 23, pp. 1–22, 2021.View at: Publisher Site | Google Scholar
P. Atkins and T. Overton, Shriver and Atkins’ Inorganic Chemistry, Oxford University Press, Oxford, UK, 2010.
K. T. Schütt, H. Glawe, F. Brockherde, A. Sanna, K. R. Müller, and E. K. U. Gross, “How to represent crystal structures for machine learning: towards fast prediction of electronic properties,” Physical Review B, vol. 89, no. 20, Article ID 205118, 2014.View at: Google Scholar
J.-F. Létard, P. Guionneau, and L. Goux-Capes, “Towards spin crossover applications,” Spin Crossover in Transition Metal Compounds III, Springer, Berlin, Germany, 2004.View at: Google Scholar
K. Liu, Y. Li, X. Hu, M. Lucu, and W. Dhammika Widanage, “Gaussian process regression with automatic relevance determination kernel for calendar aging prediction of lithium-ion batteries,” IEEE Transactions on Industrial Informatics, vol. 16, no. 6, pp. 3767–3777, 2019.View at: Google Scholar
M. Taki, A. Rohani, F. Soheili-Fard, and A. Abdeshahi, “Assessment of energy consumption and modeling of output energy for wheat production by neural network (MLP and RBF) and Gaussian process regression (GPR) models,” Journal of Cleaner Production, vol. 172, pp. 3028–3041, 2018.View at: Publisher Site | Google Scholar
M. Jamei, I. Ahmadianfar, I. Adewale Olumegbon, M. Karbasi, and A. Asadi, “On the assessment of specific heat capacity of nanofluids for solar energy applications: application of Gaussian process regression (GPR) approach,” Journal of Energy Storage, vol. 33, Article ID 102067, 2021.View at: Publisher Site | Google Scholar
K.-W. Cheng, Y.-T. Chen, and W.-H. Fang, “Video anomaly detection and localization using hierarchical feature representation and Gaussian process regression,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, June 2015.View at: Publisher Site | Google Scholar
A. R. Ghanizadeh, N. Heidarabadizadeh, and F. Heravi, “Gaussian process regression (GPR) for auto-estimation of resilient modulus of stabilized base materials,” Journal of Soft Computing in Civil Engineering, vol. 5, no. 1, pp. 80–94, 2021.View at: Google Scholar
B. Xu, T.-C. Chen, D. Ahangari, S. M. Alizadeh, M. Elveny, and J. Makhdoumi, “Application of a supervised learning machine for accurate prognostication of hydrogen contents of bio-oil,” International Journal of Chemical Engineering, vol. 2021, Article ID 7548251, 8 pages, 2021.View at: Publisher Site | Google Scholar
N. Kardani, A. Bardhan, B. Roy et al., “A novel improved Harris Hawks optimization algorithm coupled with ELM for predicting permeability of tight carbonates,” Engineering with Computers, pp. 1–24, 2021, https://www.springerprofessional.de/en/a-novel-improved-harris-hawks-optimization-algorithm-coupled-wit/19393262.View at: Publisher Site | Google Scholar
N. Kardani, A. Bardhan, P. Samui, M. Nazem, A. Zhou, and D. Jahed Armaghani, “A novel technique based on the improved firefly algorithm coupled with extreme learning machine (ELM-IFF) for predicting the thermal conductivity of soil,” Engineering with Computers, pp. 1–20, 2021, https://www.springerprofessional.de/en/a-novel-improved-harris-hawks-optimization-algorithm-coupled-wit/19393262.View at: Publisher Site | Google Scholar