SVM

Support Vector Machines

Support Vector Machines (SVM) are a class of machine learning algorithms used for classification and regression tasks. In a classification problem, SVMs aim to find an optimal hyperplane that separates different classes of data points with the largest possible margin. The hyperplane is a decision boundary that maximizes the distance between the nearest data points from each class, known as support vectors. SVMs can handle both linearly separable and non-linearly separable data by employing various kernel functions. The idea behind SVMs is to transform input data into a higher-dimensional feature space using the kernel trick, which allows the algorithm to find a linear separation in this transformed space. This transformation enables SVMs to handle complex decision boundaries that may not be linear in the original feature space. During the training phase, SVMs aim to find the optimal hyperplane by solving a quadratic optimization problem. The objective is to minimize the classification error while maximizing the margin. The margin is defined as the perpendicular distance between the hyperplane and the support vectors. By maximizing the margin, SVMs tend to produce a more robust and generalized model. In addition to binary classification, SVMs can be extended to handle multi-class problems using techniques such as one-vs-one or one-vs-all classification. SVMs also have a formulation for regression tasks, known as Support Vector Regression (SVR), where the objective is to fit the data while limiting the deviations within a certain margin. SVMs have several advantages, such as their ability to handle high-dimensional data, effective generalization, and resistance to overfitting. They can be computationally expensive, particularly with large datasets, and the selection of the appropriate kernel and its parameters can be challenging.

The following are some useful YouTube videos:

Part 1

A run down of what SVMs are and what they can be used for. All of the necessary background to get started with using SVMs.

Support Vector Machines Main Ideas

Part 2

A mathematical description of the Polynomial Kernel and how SVMs use this type of kernel to classify data. This video describes how the Polynomial Kernel classifies data in a set number of dimensions.

The Polynomial Kernel

Part 3

A mathematical description of the Radial Kernel and how SVMs use this type of kernel to classify data. This video describes how the RBF classifies data in infinite dimensions.

The Radial (RBF) Kernel