Mathematics of Machine Learning for Chemical Engineers — Glossary · Professional Certificate in AI for Chemical Engineers

Activation Function – concept #

nonlinear mapping applied to neuron output. Related terms: sigmoid, ReLU, tanh. Explanation: transforms linear combination of inputs into a bounded or unbounded signal, enabling networks to model complex relationships. Example: ReLU(x)=max(0,x). Application: modeling reaction rate saturation in kinetic networks. Challenges: choosing differentiable functions that avoid vanishing gradients.

Adjoint Method – concept #

technique for efficient gradient computation in differential equation models. Related terms: sensitivity analysis, back‑propagation, automatic differentiation. Explanation: solves a backward problem to obtain gradients of a scalar objective with respect to many parameters. Example: computing gradient of conversion with respect to catalyst properties. Application: optimizing reactor design via PDE‑constrained learning. Challenges: handling stiff systems and ensuring numerical stability.

Algorithmic Differentiation – concept #

systematic application of chain rule to computer code. Related terms: forward mode, reverse mode, autodiff libraries. Explanation: provides exact derivatives up to machine precision, unlike finite differences. Example: using JAX to differentiate a loss function involving mass balances. Application: training neural ODEs for process simulation. Challenges: memory consumption in reverse mode for long time horizons.

Bayesian Inference – concept #

updating probability distributions with observed data. Related terms: prior, posterior, likelihood. Explanation: treats model parameters as random variables, yielding a distribution that reflects uncertainty. Example: inferring diffusion coefficients from concentration profiles. Application: quantifying uncertainty in kinetic parameter estimation. Challenges: computational cost of high‑dimensional posterior sampling.

Batch Normalization – concept #

normalizing layer inputs during training. Related terms: internal covariate shift, scaling, shifting. Explanation: reduces training instability by maintaining zero mean and unit variance across mini‑batches. Example: applying BN to hidden layers of a feed‑forward network predicting vapor‑liquid equilibria. Application: accelerating convergence of deep process models. Challenges: dependence on batch size and inference‑time behavior.

Binary Cross‑Entropy – concept #

loss function for binary classification. Related terms: logistic loss, log‑likelihood, sigmoid activation. Explanation: measures discrepancy between predicted probabilities and binary outcomes. Example: classifying safe vs unsafe operating conditions. Application: fault detection in distillation columns. Challenges: class imbalance leading to biased gradients.

Boltzmann Machine – concept #

stochastic neural network with symmetric connections. Related terms: energy‑based model, Gibbs sampling, restricted Boltzmann machine. Explanation: learns probability distribution by minimizing an energy function. Example: modeling joint distribution of temperature and pressure sensors. Application: generative modeling of process data for anomaly synthesis. Challenges: slow convergence of sampling algorithms.

Box‑Cox Transformation – concept #

power transformation to stabilize variance. Related terms: log transform, Yeo‑Johnson, normality. Explanation: converts non‑Gaussian data into approximately normal form, improving linear model assumptions. Example: transforming reaction rate data before regression. Application: preprocessing experimental design data. Challenges: selecting optimal lambda parameter and handling zero or negative values.

Bootstrap Aggregating – concept #

ensemble method that averages predictions from multiple models trained on resampled data. Related terms: bagging, random forest, variance reduction. Explanation: reduces overfitting by diversifying training sets via sampling with replacement. Example: aggregating neural networks trained on different subsets of reactor data. Application: robust prediction of product purity. Challenges: increased computational load and correlation among base learners.

Cartesian Product – concept #

set of all ordered pairs from two sets. Related terms: cross‑product, combinatorial space, design matrix. Explanation: used to generate grid of hyper‑parameter combinations for model tuning. Example: combining learning rates {0.001,0.01} with batch sizes {32,64}. Application: systematic exploration of neural network architectures. Challenges: exponential growth of combinations causing infeasible search spaces.

Catastrophic Forgetting – concept #

loss of previously learned knowledge when training on new data. Related terms: continual learning, rehearsal, elastic weight consolidation. Explanation: occurs because gradient updates overwrite shared weights. Example: updating a catalyst degradation model with new aging data erases earlier performance. Application: online monitoring systems that must retain historic behavior. Challenges: designing regularization strategies to preserve prior knowledge.

Cholesky Decomposition – concept #

factorization of a symmetric positive‑definite matrix into lower‑triangular form. Related terms: LDLᵀ, matrix square root, covariance matrix. Explanation: enables efficient solving of linear systems and sampling from multivariate normals. Example: decomposing Hessian of loss for Newton‑type updates. Application: uncertainty quantification in parameter estimation. Challenges: ensuring matrix positivity and handling near‑singular cases.

Clustering Coefficient – concept #

measure of local connectivity in a graph. Related terms: network analysis, community detection, adjacency matrix. Explanation: quantifies the likelihood that two neighbors of a node are connected. Example: evaluating sensor network topology in a plant. Application: identifying tightly coupled units for reduced‑order modeling. Challenges: interpreting coefficient in sparse process graphs.

Conjugate Gradient – concept #

iterative method for solving large symmetric positive‑definite linear systems. Related terms: Krylov subspace, preconditioning, CG. Explanation: minimizes quadratic form along conjugate directions without forming matrix inverse. Example: solving discretized diffusion equations within a neural PDE solver. Application: accelerating training of physics‑informed neural networks. Challenges: selection of effective preconditioners for ill‑conditioned matrices.

Convex Optimization – concept #

optimization of a convex objective over a convex set. Related terms: global optimum, gradient descent, Lagrangian duality. Explanation: any local minimum is also global, guaranteeing tractable solutions. Example: minimizing mean‑square error of a linear regression model for reaction rates. Application: parameter fitting with regularization. Challenges: ensuring problem formulation remains convex when adding nonlinear constraints.

Cross‑Entropy Loss – concept #

measure of dissimilarity between true distribution and predicted probabilities. Related terms: softmax, classification, KL divergence. Explanation: penalizes confident but incorrect predictions more heavily. Example: predicting product distribution across multiple outlet streams. Application: multi‑class classification of operating regimes. Challenges: numerical stability for extreme logits.

Curse of Dimensionality – concept #

exponential increase in volume associated with adding dimensions to a space. Related terms: feature space, overfitting, manifold learning. Explanation: data become sparse, making distance metrics less meaningful. Example: modeling a catalyst surface with hundreds of descriptors. Application: dimensionality reduction before training regression models. Challenges: selecting informative features without discarding critical chemistry.

Data Augmentation – concept #

artificially expanding training set by applying transformations. Related terms: synthetic data, noise injection, oversampling. Explanation: improves model generalization by exposing it to varied scenarios. Example: adding Gaussian noise to temperature profiles. Application: enhancing robustness of fault detection networks. Challenges: ensuring augmented data remain physically plausible.

Decision Boundary – concept #

hypersurface separating classes in feature space. Related terms: support vector machine, margin, classifier. Explanation: defined by parameters that yield equal class probabilities. Example: separating safe vs unsafe pressure regions. Application: visualizing classifier performance for process safety. Challenges: high‑dimensional boundaries are difficult to interpret.

Derivative #

Free Optimization – concept: methods that do not require gradient information. Related terms: Nelder‑Mead, Bayesian optimization, pattern search. Explanation: useful when objective is noisy, discontinuous, or expensive to differentiate. Example: tuning hyper‑parameters of a black‑box simulator. Application: optimizing catalyst synthesis protocols. Challenges: slower convergence compared to gradient‑based methods.

Dimensionality Reduction – concept #

techniques to project high‑dimensional data onto lower‑dimensional subspaces. Related terms: PCA, t‑SNE, autoencoder. Explanation: retains most variance or structure while simplifying models. Example: using PCA to compress spectroscopic data before regression. Application: faster training of surrogate models. Challenges: loss of interpretability and potential distortion of nonlinear relationships.

Discrete Fourier Transform – concept #

converts discrete time series into frequency domain representation. Related terms: FFT, spectral analysis, signal processing. Explanation: decomposes signal into sinusoidal components. Example: analyzing periodic fluctuations in reactor temperature. Application: feature extraction for time‑series forecasting. Challenges: aliasing and resolution limits for short sequences.

Distribution Shift – concept #

change in data distribution between training and deployment. Related terms: covariate shift, concept drift, domain adaptation. Explanation: degrades model performance if unseen conditions arise. Example: operating a plant at a new feed composition. Application: monitoring model reliability and triggering retraining. Challenges: detecting shift promptly and updating models without excessive downtime.

Dropout – concept #

regularization technique that randomly deactivates neurons during training. Related terms: stochastic regularization, model averaging, overfitting. Explanation: prevents co‑adaptation of features and improves generalization. Example: applying 20 % dropout to hidden layers of a neural network predicting viscosity. Application: robust surrogate modeling of complex flows. Challenges: selecting appropriate dropout rate; inference‑time scaling.

Eigenvalue Decomposition – concept #

factorization of a square matrix into eigenvectors and eigenvalues. Related terms: spectral theorem, diagonalization, principal component analysis. Explanation: reveals intrinsic modes of variation. Example: decomposing Jacobian of a kinetic model to assess stiffness. Application: model reduction for stiff reaction networks. Challenges: handling non‑diagonalizable or defective matrices.

Ensemble Learning – concept #

combining predictions from multiple models to improve accuracy. Related terms: bagging, boosting, stacking. Explanation: leverages diversity among base learners to reduce variance and bias. Example: averaging forecasts from a random forest and a neural network for product yield. Application: reliable prediction under uncertain feed conditions. Challenges: managing correlated errors and computational overhead.

Exponential Moving Average – concept #

weighted average that gives more importance to recent observations. Related terms: smoothing, low‑pass filter, momentum. Explanation: updates recursively with a decay factor. Example: smoothing loss curve during training of a deep network. Application: stabilizing online parameter updates in adaptive control. Challenges: choosing decay rate that balances responsiveness and noise reduction.

Feature Engineering – concept #

creation of informative variables from raw data. Related terms: domain knowledge, transformation, interaction terms. Explanation: improves model performance by embedding physics or chemistry insights. Example: generating dimensionless groups such as Reynolds and Damköhler numbers. Application: enhancing regression models for heat‑exchanger performance. Challenges: avoiding leakage and maintaining reproducibility.

Feature Scaling – concept #

normalizing features to a common range. Related terms: standardization, min‑max scaling, normalization. Explanation: prevents features with large magnitude from dominating gradient updates. Example: scaling temperature (K) and concentration (mol L⁻¹) before neural network training. Application: faster convergence of gradient‑based optimizers. Challenges: handling outliers and ensuring consistent scaling between training and deployment.

Feedforward Neural Network – concept #

directed acyclic graph of layers where information moves forward. Related terms: multilayer perceptron, fully connected, depth. Explanation: maps input vector to output through successive linear transformations and nonlinear activations. Example: predicting azeotrope composition from feed composition and operating pressure. Application: surrogate modeling of equilibrium calculations. Challenges: selecting architecture depth and avoiding overfitting with limited data.

Fisher Information Matrix – concept #

quantifies amount of information that observable data carry about unknown parameters. Related terms: Cramér‑Rao bound, sensitivity, covariance. Explanation: inverse gives lower bound on parameter variance. Example: computing Fisher matrix for kinetic parameters in a batch reactor. Application: experimental design to maximize information gain. Challenges: computational cost for large parameter sets and ill‑conditioned matrices.

Fourier Neural Operator – concept #

learns mappings between function spaces using Fourier transforms. Related terms: operator learning, spectral convolution, PDE surrogate. Explanation: represents integral kernels in frequency domain for efficient learning. Example: predicting temperature fields in a tubular reactor from boundary conditions. Application: rapid simulation of spatially distributed processes. Challenges: handling non‑periodic boundary conditions and aliasing.

Gaussian Process – concept #

non‑parametric Bayesian model defining a distribution over functions. Related terms: kernel, covariance function, kriging. Explanation: provides mean prediction and uncertainty estimate analytically. Example: modeling pressure drop as a function of flow rate with limited data. Application: active learning for experimental campaigns. Challenges: cubic scaling with data size and kernel selection.

Gradient Boosting – concept #

sequential ensemble method that builds models to correct errors of previous ones. Related terms: XGBoost, AdaBoost, additive modeling. Explanation: each new learner fits residuals, improving overall performance. Example: predicting catalyst activity using decision trees boosted on residuals. Application: high‑accuracy regression for process optimization. Challenges: overfitting if learning rate is too high; hyper‑parameter tuning.

Gradient Descent – concept #

iterative optimization algorithm that moves parameters opposite to gradient of loss. Related terms: learning rate, stochastic gradient descent, momentum. Explanation: seeks minima by locally linear approximation. Example: training a neural network to emulate a distillation column model. Application: parameter estimation for kinetic models. Challenges: selecting appropriate step size and avoiding saddle points.

Hessian Matrix – concept #

square matrix of second‑order partial derivatives of a scalar function. Related terms: curvature, Newton’s method, second‑order optimization. Explanation: captures local curvature of loss landscape. Example: computing Hessian of loss for a neural network to assess parameter sensitivity. Application: trust‑region methods for robust training. Challenges: high computational cost and potential indefiniteness.

Hidden Markov Model – concept #

statistical model for sequences with unobserved (hidden) states. Related terms: Viterbi algorithm, Baum‑Welch, state transition matrix. Explanation: models observable outputs as probabilistic functions of underlying Markov chain. Example: detecting phase‑change events from temperature sensor streams. Application: fault diagnosis in batch reactors. Challenges: determining appropriate number of hidden states and handling non‑stationarity.

Hyperparameter Tuning – concept #

process of selecting optimal values for model settings not learned from data. Related terms: grid search, random search, Bayesian optimization. Explanation: influences model capacity, regularization, and training dynamics. Example: choosing learning rate and batch size for a CNN that predicts catalyst morphology. Application: maximizing predictive accuracy of surrogate models. Challenges: costly evaluations and risk of over‑optimizing on validation set.

Independent Component Analysis – concept #

blind source separation technique that extracts statistically independent sources from mixtures. Related terms: ICA, blind deconvolution, non‑Gaussianity. Explanation: maximizes statistical independence rather than variance. Example: separating overlapping spectral signatures of reactants and products. Application: preprocessing of spectroscopic data for kinetic modeling. Challenges: sensitivity to noise and scaling.

Infinity Norm – concept #

maximum absolute value among vector components. Related terms: L∞ norm, Chebyshev distance, sup norm. Explanation: measures worst‑case deviation. Example: assessing maximum error in temperature predictions across reactor length. Application: defining convergence criteria for iterative solvers. Challenges: nondifferentiability at points where multiple components share the maximum.

Kernel Trick – concept #

implicit mapping of data into high‑dimensional feature space using kernel functions. Related terms: support vector machine, radial basis function, Gram matrix. Explanation: enables linear algorithms to capture nonlinear relationships without explicit transformation. Example: using RBF kernel to model nonlinear sorption isotherms. Application: classification of operating regimes with limited data. Challenges: selecting appropriate kernel parameters and computational cost of large Gram matrices.

Kullback‑Leibler Divergence – concept #

measure of how one probability distribution diverges from a reference distribution. Related terms: relative entropy, information loss, variational inference. Explanation: asymmetric; zero only when distributions are identical. Example: comparing predicted and measured concentration distributions. Application: training variational autoencoders for generative process modeling. Challenges: numerical stability when probabilities approach zero.

Latent Variable – concept #

hidden variable that influences observed data but is not directly measured. Related terms: hidden state, factor, embedding. Explanation: captures underlying structure or generative factors. Example: latent variable representing catalyst deactivation level in a degradation model. Application: probabilistic modeling with variational inference. Challenges: identifiability and interpretability of latent dimensions.

Levenberg‑Marquardt Algorithm – concept #

hybrid optimization method combining gradient descent and Gauss‑Newton. Related terms: damped least squares, trust region, nonlinear least squares. Explanation: adapts step size based on curvature, improving convergence for nonlinear regression. Example: fitting Arrhenius parameters to experimental rate data. Application: parameter estimation in kinetic models. Challenges: selecting damping parameter and handling large-scale problems.

Linear Regression – concept #

statistical method modeling relationship between dependent variable and one or more independent variables linearly. Related terms: ordinary least squares, multicollinearity, residual analysis. Explanation: minimizes sum of squared residuals to find best‑fit coefficients. Example: correlating conversion with residence time in a plug‑flow reactor. Application: quick screening of design variables. Challenges: violation of linearity assumptions and heteroscedasticity.

Logistic Regression – concept #

classification model that predicts probability of binary outcome using logistic function. Related terms: sigmoid, odds ratio, maximum likelihood. Explanation: models log‑odds as linear combination of inputs. Example: predicting likelihood of catalyst fouling based on operating temperature and pressure. Application: safety‑critical alarm systems. Challenges: separation of classes and handling imbalanced datasets.

Loss Function – concept #

scalar measure of model error to be minimized during training. Related terms: objective, cost, risk. Explanation: guides optimization by quantifying discrepancy between predictions and targets. Example: mean‑square error for regression of product purity. Application: training surrogate models for process simulation. Challenges: choosing loss aligned with engineering goals and avoiding gradient vanishing.

Markov Chain Monte Carlo – concept #

family of algorithms for sampling from complex probability distributions. Related terms: Metropolis‑Hastings, Gibbs sampling, posterior sampling. Explanation: constructs a Markov chain whose stationary distribution equals target distribution. Example: sampling kinetic parameters conditioned on experimental data. Application: Bayesian calibration of reactor models. Challenges: convergence diagnostics and autocorrelation of samples.

Matrix Factorization – concept #

decomposition of a matrix into product of two or more matrices. Related terms: singular value decomposition, non‑negative matrix factorization, low‑rank approximation. Explanation: extracts latent structure and reduces dimensionality. Example: factorizing a sensor data matrix to identify underlying process modes. Application: denoising and compression of high‑frequency measurements. Challenges: selecting rank and handling missing entries.

Maximum Likelihood Estimation – concept #

statistical method for estimating parameters that maximize probability of observed data. Related terms: log‑likelihood, MLE, frequentist inference. Explanation: finds parameters where model best explains data under assumed distribution. Example: estimating reaction order by maximizing likelihood of concentration measurements. Application: parameter fitting for kinetic models. Challenges: non‑convex likelihood surfaces and sensitivity to outliers.

Mean Absolute Error – concept #

average of absolute differences between predictions and true values. Related terms: MAE, L1 loss, robust metric. Explanation: penalizes all errors linearly, less sensitive to outliers than MSE. Example: evaluating temperature predictions of a surrogate model. Application: performance metric for control‑oriented models. Challenges: lack of differentiability at zero, which may affect gradient‑based training.

Mean Squared Error – concept #

average of squared differences between predicted and actual values. Related terms: MSE, L2 loss, variance. Explanation: emphasizes larger errors due to squaring, facilitating gradient computation. Example: loss function for training a neural network to predict conversion. Application: standard metric for regression tasks. Challenges: sensitivity to outliers and scale dependence.

Monte Carlo Integration – concept #

numerical estimation of integrals using random sampling. Related terms: stochastic integration, importance sampling, variance reduction. Explanation: approximates integral by average of function evaluations at random points. Example: estimating expected product yield over uncertain feed composition. Application: uncertainty propagation in process design. Challenges: slow convergence for high‑dimensional integrals.

Multicollinearity – concept #

situation where predictor variables are highly linearly correlated. Related terms: variance inflation factor, redundancy, ridge regression. Explanation: inflates variance of coefficient estimates, leading to unstable models. Example: temperature and pressure correlated in experimental design. Application: diagnosing regression models for kinetic parameter extraction. Challenges: detecting and mitigating via feature selection or regularization.

Neural Ordinary Differential Equation – concept #

continuous-depth model where hidden state evolves according to learned ODE. Related terms: ODE‑net, continuous normalizing flow, adjoint sensitivity. Explanation: replaces discrete layers with an ODE solver, enabling adaptive computation. Example: modeling concentration profiles along a reactor length as continuous transformation. Application: memory‑efficient deep models for spatiotemporal processes. Challenges: solver selection and handling stiff dynamics.

Newton‑Raphson Method – concept #

iterative root‑finding algorithm using first derivative. Related terms: Newton’s method, quadratic convergence, Jacobian. Explanation: updates estimate by subtracting function value divided by derivative. Example: solving material balance equations for steady‑state reactor. Application: rapid convergence in nonlinear equation systems. Challenges: requirement of good initial guess and Jacobian computation.

Normalization – concept #

scaling of data to have specific statistical properties. Related terms: zero‑mean unit‑variance, L2 norm, data preprocessing. Explanation: improves numerical stability and training speed. Example: normalizing concentration data before feeding into a neural network. Application: consistent model performance across operating ranges. Challenges: preserving physical meaning after scaling.

One‑Hot Encoding – concept #

representation of categorical variables as binary vectors. Related terms: dummy variables, categorical encoding, indicator matrix. Explanation: each category assigned a unique position with value 1, others 0. Example: encoding catalyst type in a regression model. Application: allowing algorithms that require numeric input to handle categorical process choices. Challenges: high dimensionality with many categories.

Optimization Landscape – concept #

geometric representation of loss function over parameter space. Related terms: saddle point, local minimum, basin of attraction. Explanation: shape influences convergence behavior of training algorithms. Example: visualizing loss surface of a small neural network for reactor modeling. Application: diagnosing training difficulties and selecting optimizers. Challenges: high dimensionality prevents full visualization; reliance on projections.

Overfitting – concept #

model captures noise instead of underlying pattern, performing poorly on unseen data. Related terms: regularization, validation error, model complexity. Explanation: occurs when capacity exceeds amount of informative data. Example: a deep network memorizing specific batch experiments. Application: surrogate models for design space exploration. Challenges: detecting via cross‑validation and applying appropriate regularization.

Partial Least Squares – concept #

regression technique that projects predictors and responses onto latent structures maximizing covariance. Related terms: PLS, dimensionality reduction, multivariate calibration. Explanation: handles multicollinearity and high‑dimensional data. Example: relating spectral measurements to concentration of multiple species. Application: building calibration models for online analyzers. Challenges: choosing number of latent components and interpreting loadings.

Patched Convolution – concept #

convolution operation applied to localized regions (patches) of input. Related terms: CNN, receptive field, stride. Explanation: extracts spatial features while preserving locality. Example: processing image of catalyst surface morphology. Application: feature extraction for material property prediction. Challenges: edge effects and computational cost for large images.

Principal Component Analysis – concept #

linear technique that transforms data to orthogonal axes of maximal variance. Related terms: eigenvectors, dimensionality reduction, scree plot. Explanation: first principal component captures most variability, subsequent components capture orthogonal residual variance. Example: reducing dimensionality of FTIR spectra before regression. Application: accelerating training of machine‑learning models. Challenges: linearity assumption and loss of interpretability.

Probabilistic Graphical Model – concept #

representation of joint probability distribution using graph structure. Related terms: Bayesian network, Markov random field, factor graph. Explanation: nodes denote random variables, edges encode conditional dependencies. Example: modeling dependencies among temperature, pressure, and conversion in a reactor network. Application: inference of hidden states such as catalyst activity. Challenges: structure learning and exact inference scalability.

Quadratic Programming – concept #

optimization of a quadratic objective subject to linear constraints. Related terms: convex QP, KKT conditions, interior‑point method. Explanation: widely used for portfolio optimization and control. Example: minimizing deviation from target product composition while respecting mass balance constraints. Application: real‑time optimization of plant operation. Challenges: ensuring positive‑definiteness of Hessian and handling large constraint sets.

Random Forest – concept #

ensemble of decision trees built on bootstrapped samples with random feature selection. Related terms: bagging, feature importance, out‑of‑bag error. Explanation: reduces variance and improves generalization. Example: predicting catalyst lifespan from operating conditions. Application: interpretable surrogate for complex nonlinear behavior. Challenges: bias from shallow trees and memory usage.

Recurrent Neural Network – concept #

neural architecture with cycles allowing information persistence across time steps. Related terms: LSTM, GRU, sequence modeling. Explanation: processes sequential data by maintaining hidden state. Example: forecasting temperature trajectory in a batch reactor. Application: time‑series prediction for process control. Challenges: vanishing/exploding gradients and long‑term dependency capture.

Regularization – concept #

technique that adds penalty to loss to discourage complex models. Related terms: L1 penalty, L2 penalty, dropout. Explanation: balances fit to data against model simplicity. Example: adding L2 term to neural network weights to prevent overfitting. Application: robust kinetic parameter estimation. Challenges: selecting regularization strength and avoiding underfitting.

Residual Neural Network – concept #

deep architecture that learns residual functions relative to layer inputs. Related terms: skip connection, ResNet, identity mapping. Explanation: mitigates degradation problem in very deep networks. Example: modeling multi‑stage separation with many hidden layers. Application: high‑accuracy surrogate for multi‑unit plant. Challenges: increased training complexity and careful initialization.

Ridge Regression – concept #

linear regression with L2 regularization to shrink coefficients. Related terms: Tikhonov regularization, shrinkage, bias‑variance tradeoff. Explanation: improves conditioning of problem and reduces overfitting. Example: fitting Arrhenius parameters with ridge penalty to handle collinearity. Application: stable kinetic model calibration. Challenges: choosing regularization parameter and interpreting biased coefficients.

Robust Statistics – concept #

statistical methods less sensitive to outliers. Related terms: M‑estimator, Huber loss, median absolute deviation. Explanation: provides reliable estimates when data contain anomalies. Example: using Huber loss for regression of conversion data with occasional sensor spikes. Application: resilient model training in noisy environments. Challenges: tuning robustness parameters and potential loss of efficiency.

Sampling Distribution – concept #

probability distribution of a statistic over repeated samples. Related terms: central limit theorem, bootstrap, estimator variance. Explanation: describes variability of estimated parameters. Example: distribution of estimated reaction rate constant from repeated experiments. Application: constructing confidence intervals for kinetic parameters. Challenges: small sample sizes leading to inaccurate approximations.

Scikit‑Learn – concept #

open‑source Python library offering simple tools for data mining and analysis. Related terms: estimator API, pipelines, cross‑validation. Explanation: provides consistent interface for classification, regression, clustering, and preprocessing. Example: using RandomForestRegressor to predict product yield. Application: rapid prototyping of machine‑learning models for chemical processes. Challenges: scalability to very large datasets and limited deep‑learning support.

Sigmoid Function – concept #

smooth S‑shaped function mapping real numbers to (0,1). Related terms: logistic, activation, squashing function. Explanation: commonly used in binary classification outputs. Example: converting neural network logit to probability of catalyst deactivation. Application: probabilistic decision making in safety systems. Challenges: saturation leading to vanishing gradients.

Singular Value Decomposition – concept #

factorization of a matrix into orthogonal matrices and a diagonal of singular values. Related terms: SVD, low‑rank approximation, pseudoinverse. Explanation: reveals intrinsic dimensionality and enables noise reduction. Example: decomposing sensor matrix to isolate dominant process modes. Application: data compression for real‑time monitoring. Challenges: computational cost for large matrices and handling missing entries.

Softmax Function – concept #

generalization of sigmoid to multi‑class probabilities. Related terms: multinomial logistic, normalization, probability vector. Explanation: exponentiates each input and normalizes by sum of exponentials. Example: output layer of a network classifying operating regimes into three categories. Application: multi‑class decision support. Challenges: numerical overflow for large logits; need for stabilization tricks.

Stochastic Gradient Descent – concept #

gradient descent variant that uses a subset (mini‑batch) of data per iteration. Related terms: learning rate schedule, momentum, variance reduction. Explanation: reduces computational load and introduces noise that can escape shallow minima. Example: training a deep network on streaming process data. Application: online model updating in adaptive control. Challenges: tuning batch size and learning rate to balance convergence speed and stability.

Support Vector Machine – concept #

supervised learning method that finds hyperplane maximizing margin between classes. Related terms: kernel trick, soft margin, hinge loss. Explanation: robust to high‑dimensional data and works well with limited samples. Example: classifying safe vs unsafe pressure‑temperature operating points. Application: rapid safety envelope generation. Challenges: selection of kernel parameters and scalability to large datasets.

Tensor Decomposition – concept #

factorization of multi‑dimensional arrays into simpler components. Related terms: CANDECOMP/PARAFAC, Tucker, multi‑way analysis. Explanation: captures interactions across multiple modes (e.g., time, space, species). Example: decomposing a 3‑D concentration‑time‑location dataset from CFD simulations. Application: reduced‑order modeling of spatiotemporal processes. Challenges: determining appropriate rank and handling noise.

Temporal Convolutional Network – concept #

convolutional architecture designed for sequence modeling with causal convolutions. Related terms: dilated convolution, receptive field, sequence-to-sequence. Explanation: captures long‑range dependencies without recurrent connections. Example: forecasting product composition over time in a batch process. Application: real‑time prediction for batch scheduling. Challenges: selecting dilation schedule and managing boundary effects.

Thompson Sampling – concept #

Bayesian algorithm for balancing exploration and exploitation in multi‑armed bandit problems. Related terms: Bayesian optimization, posterior sampling, regret minimization. Explanation: samples a parameter from posterior and selects action with highest sampled reward. Example: choosing experimental conditions to maximize catalyst activity. Application: active learning in experimental design. Challenges: computational overhead for updating posteriors in high dimensions.

Training Set – concept #

subset of data used to fit model parameters. Related terms: validation set, test set, cross‑validation. Explanation: provides information for learning; must be representative of problem space. Example: using 70 % of reactor data for model training. Application: building predictive models for process optimization. Challenges: ensuring no leakage and maintaining diversity.

Transfer Learning – concept #

reusing knowledge from a pretrained model on a related task. Related terms: fine‑tuning, domain adaptation, feature extractor. Explanation: reduces data requirements and accelerates convergence. Example: adapting a network trained on generic fluid dynamics to a specific reactor geometry. Application: rapid development of surrogate models for new units. Challenges: negative transfer when source and target domains differ significantly.

Uniform Distribution – concept #

probability distribution where all outcomes in a range are equally likely. Related terms: random sampling, flat prior, bounded support. Explanation: often used for initializing parameters or generating synthetic data. Example: sampling initial weights of a neural network uniformly in [‑0.1,0.1]. Application: stochastic initialization to break symmetry. Challenges: inappropriate scale can hinder convergence.

Vanishing Gradient – concept #

phenomenon where gradients become extremely small, slowing learning in deep networks. Related terms: exploding gradient, ReLU, residual connections. Explanation: caused by repeated multiplication of derivatives less than one. Example: training a deep feed‑forward network with sigmoid activations for reaction rate prediction. Application: diagnosing training failures in deep process models. Challenges: selecting architectures and activation functions that mitigate effect.

Variational Autoencoder – concept #

generative model that learns latent representation by maximizing evidence lower bound. Related terms: encoder, decoder, KL divergence. Explanation: combines reconstruction loss with regularization to enforce latent distribution. Example: generating plausible concentration profiles for unseen operating conditions. Application: data augmentation for scarce experimental datasets. Challenges: balancing reconstruction fidelity and latent space regularization.

Weighted Least Squares – concept #

regression where each observation is assigned a weight reflecting its variance or importance. Related terms: heteroscedasticity, WLS, confidence weighting. Explanation: minimizes weighted sum of squared residuals, giving more influence to reliable data. Example: fitting kinetic model where measurements at high temperature have lower uncertainty. Application: improving parameter estimates under variable measurement precision. Challenges: determining appropriate weights and handling correlated errors.

Wavelet Transform – concept #

decomposition of signal into localized frequency components. Related terms: multiresolution analysis, Daubechies, time‑frequency. Explanation: captures both temporal and spectral information. Example: denoising pressure sensor data from a distillation column. Application: feature extraction for fault detection. Challenges: choosing mother wavelet and scale levels.

Weight Decay – concept #

L2 regularization applied during gradient descent to shrink model parameters. Related terms: L2 penalty, regularizer, norm penalty. Explanation: adds term proportional to squared magnitude of weights to loss. Example: training a neural network for viscosity prediction with weight decay 1e‑4. Application: preventing over‑parameterization in deep models. Challenges: tuning decay coefficient to avoid underfitting.

Zero‑Mean Unit‑Variance Normalization – concept #

scaling data so that its mean is zero and standard deviation is one. Related terms: standardization, z‑score, data preprocessing. Explanation: centers and scales each feature, facilitating gradient‑based optimization. Example: normalizing inlet composition vectors before feeding into a neural surrogate. Application: improving convergence speed of training algorithms. Challenges: storing scaling parameters for consistent inference.