MACHINE LEARNING ALGORITHM FROM AMATHEMATICAL PERSPECTIVE

Abstract

This study offered a comprehensive mathematical analysis of fundamental machine learning techniques, conducted by the Department of Mathematics and Computer Science of the Islamia University of Bahawalpur. The primary aims of the research included self-contained development, derivation, simulation, and comparison of popular algorithms, which included linear regression, logistic regression, support vector machines (SVM), principal component analysis (PCA), neural networks, Naive Bayes, decision trees, k-means clustering, and minimizations, like gradient descent. Each of these algorithms was formulated using several fundamental areas of mathematics, including calculus, linear algebra, probability, and statistics. Derivations were manually performed symbolically, and the correctness of the derivations was verified symbolically by MATLAB, Python, or Wolfram Mathematica. The algorithms were simulated in Jupyter Notebooks to verify their properties, including algorithm behavior, convergence, and tailored settings to analyze sensitivity. The results showed that linear models were simple and easy to interpret, and that neural networks and SVMs involved far more complexity in terms of math, and computation. PCA showed a good application of eigenvalue decomposition for dimensionality reduction, along with probabilistic models like Naive Bayes being efficient, if the right assumptions were made. The analytical comparison the study conducted assisted in demonstrating the diversity and differences in learning behaviors, stability, and mathematical intensity each model can present. Overall, these research findings require the reader to be aware of the need for mathematical comprehension when selecting, applying, and altering machine learning algorithms. This research has a theoretical contribution in furthering the academic body of knowledge regarding these algorithms and models, and the application of machine learning should be practically aware of the associated mathematical understanding it requires.