- Visit http://www.developerstation.org/2011/03/simple-tutorial-on-using-libsvm.html
- libsvm-3.18/README
- Using LibSVM with OpenCV Mat http://kuantinglai.blogspot.kr/2013/07/using-libsvm-with-opencv-mat.html
// No real test is done yet.
LIBSVM is the most popular machine learning tool developed by C. J. Lin at National Taiwan University (
http://www.csie.ntu.edu.tw/~cjlin/libsvm/). This code demonstrates how to load a data matrix in CSV format using OpenCV, and allocate LIBSVM data structure to do SVM predict.
#include "svm.h"
#include <iostream>
#include "opencv2/opencv.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/ml/ml.hpp"
#include <iostream>
using namespace cv;
using namespace std;
const char *CSV_FILE = "/Users/kuanting/libsvm-3.17/heart_scale.csv";
const char *MODEL_FILE = "/Users/kuanting/libsvm-3.17/heart_scale.model";
int main(int argc, char * argv[])
{
CvMLData dataFile;
// Load matrix data in csv format
if (dataFile.read_csv(CSV_FILE) != 0)
{
fprintf(stderr, "Can't read csv file %s\n", CSV_FILE);
return -1;
}
Mat dataMat(dataFile.get_values()); // Default data type is float
struct svm_model *SVMModel;
if ((SVMModel = svm_load_model(MODEL_FILE)) == 0) {
fprintf(stderr, "Can't load SVM model %s", MODEL_FILE);
return -2;
}
struct svm_node *svmVec;
svmVec = (struct svm_node *)malloc((dataMat.cols+1)*sizeof(struct svm_node));
double *predictions = new double[dataMat.rows];
float *dataPtr = dataMat.
ptr<
float>();
// Get data from OpenCV Mat
double prob_est[2]; // Probability estimation
int r, c;
for (r=
0; r<dataMat.
rows; r++)
{
for (c=
0; c<dataMat.
cols; c++)
{
svmVec[c].index = c+1; // Index starts from 1; Pre-computed kernel starts from 0
svmVec[c].
value = dataPtr[r*dataMat.
cols + c];
}
svmVec[c].
index = -
1;
// End of line
if(svm_check_probability_model(SVMModel))
{
predictions[r] =
svm_predict_probability(SVMModel, svmVec, prob_est);
printf(
"%f\t%f\t%f\n", predictions[r], prob_est[
0], prob_est[
1]);
}
else
{
predictions[r] =
svm_predict(SVMModel, svmVec);
printf(
"%f\n", predictions[r]);
}
}
return 0;
}
// EOF //
-----------------------------------------------------
Exerpt from libsvm-3.18/README
Libsvm is a simple, easy-to-use, and efficient software for SVM classification and regression. It solves C-SVM classification, nu-SVM classification, one-class-SVM, epsilon-SVM regression, and nu-SVM
regression. It also provides an automatic model selection tool for C-SVM classification. This document explains the use of libsvm.
Libsvm is available at
Please read the COPYRIGHT file before using libsvm.
Library Usage
=============
These functions and structures are declared in the header file `svm.h'. You need to
#include "svm.h"
in your C/C++ source files and link your program with `svm.cpp'.
You can see `svm-train.c' and `svm-predict.c' for examples showing how to use them. We define LIBSVM_VERSION and declare `extern int libsvm_version; ' in svm.h, so you can check the version number.
Before you classify test data, you need to construct an SVM model (`svm_model') using training data. A model can also be saved in a file for later use. Once an SVM model is available, you can use it to classify new data.
- Function: struct svm_model *svm_train(const struct svm_problem *prob, const struct svm_parameter *param);
This function constructs and returns an SVM model according to the given training data and parameters.
struct svm_problem describes the problem:
struct svm_problem
{
int l;
double *y;
struct svm_node **x;
};
where `l' is the number of training data, and `y' is an array containing their target values. (integers in classification, real numbers in regression) `x' is an array of pointers, each of which points to a sparse representation (array of svm_node) of one training vector.
For example, if we have the following training data:
LABEL ATTR1 ATTR2 ATTR3 ATTR4 ATTR5
----- ----- ----- ----- ----- -----
1 0 0.1 0.2 0 0
2 0 0.1 0.3 -1.2 0
1 0.4 0 0 0 0
2 0 0.1 0 1.4 0.5
3 -0.1 -0.2 0.1 1.1 0.1
then the components of svm_problem are:
l = 5
y -> 1 2 1 2 3
x -> [ ] -> (2,0.1) (3,0.2) (-1,?)
[ ] -> (2,0.1) (3,0.3) (4,-1.2) (-1,?)
[ ] -> (1,0.4) (-1,?)
[ ] -> (2,0.1) (4,1.4) (5,0.5) (-1,?)
[ ] -> (1,-0.1) (2,-0.2) (3,0.1) (4,1.1) (5,0.1) (-1,?)
where (index,value) is stored in the structure `svm_node':
struct svm_node
{
int index;
double value;
};
index = -1 indicates the end of one vector.
Note that indices must be in ASCENDING order.
struct svm_parameter describes the parameters of an SVM model:
struct svm_parameter
{
int svm_type;
int kernel_type;
int degree; /* for poly */
double gamma; /* for poly/rbf/sigmoid */
double coef0; /* for poly/sigmoid */
/* these are for training only */
double cache_size; /* in MB */
double eps; /* stopping criteria */
double C; /* for C_SVC, EPSILON_SVR, and NU_SVR */
int nr_weight; /* for C_SVC */
int *weight_label; /* for C_SVC */
double* weight; /* for C_SVC */
double nu; /* for NU_SVC, ONE_CLASS, and NU_SVR */
double p; /* for EPSILON_SVR */
int shrinking; /* use the shrinking heuristics */
int probability; /* do probability estimates */
};
svm_type can be one of C_SVC, NU_SVC, ONE_CLASS, EPSILON_SVR, NU_SVR.
C_SVC: C-SVM classification
NU_SVC: nu-SVM classification
ONE_CLASS: one-class-SVM
EPSILON_SVR: epsilon-SVM regression
NU_SVR: nu-SVM regression
kernel_type can be one of LINEAR, POLY, RBF, SIGMOID.
LINEAR: u'*v
POLY: (gamma*u'*v + coef0)^degree
RBF: exp(-gamma*|u-v|^2)
SIGMOID: tanh(gamma*u'*v + coef0)
PRECOMPUTED: kernel values in training_set_file
cache_size is the size of the kernel cache, specified in megabytes.
C is the cost of constraints violation.
eps is the stopping criterion. (we usually use 0.00001 in nu-SVC, 0.001 in others). nu is the parameter in nu-SVM, nu-SVR, and one-class-SVM.
p is the epsilon in epsilon-insensitive loss function of epsilon-SVM regression.
shrinking = 1 means shrinking is conducted; = 0 otherwise.
probability = 1 means model with probability information is obtained; = 0 otherwise.
nr_weight, weight_label, and weight are used to change the penalty
for some classes (If the weight for a class is not changed, it is
set to 1). This is useful for training classifier using unbalanced
input data or with asymmetric misclassification cost.
nr_weight is the number of elements in the array weight_label and
weight. Each weight[i] corresponds to weight_label[i], meaning that
the penalty of class weight_label[i] is scaled by a factor of weight[i].
If you do not want to change penalty for any of the classes,
just set nr_weight to 0.
*NOTE* Because svm_model contains pointers to svm_problem, you can not free the memory used by svm_problem if you are still using the svm_model produced by svm_train().
*NOTE* To avoid wrong parameters, svm_check_parameter() should be called before svm_train().
struct svm_model stores the model obtained from the training procedure.
It is not recommended to directly access entries in this structure.
Programmers should use the interface functions to get the values.
struct svm_model
{
struct svm_parameter param; /* parameter */
int nr_class; /* number of classes, = 2 in regression/one class svm */
int l; /* total #SV */
struct svm_node **SV; /* SVs (SV[l]) */
double **sv_coef; /* coefficients for SVs in decision functions (sv_coef[k-1][l]) */
double *rho; /* constants in decision functions (rho[k*(k-1)/2]) */
double *probA; /* pairwise probability information */
double *probB;
int *sv_indices; /* sv_indices[0,...,nSV-1] are values in [1,...,num_traning_data] to indicate SVs in the training set */
/* for classification only */
int *label; /* label of each class (label[k]) */
int *nSV; /* number of SVs for each class (nSV[k]) */
/* nSV[0] + nSV[1] + ... + nSV[k-1] = l */
/* XXX */
int free_sv; /* 1 if svm_model is created by svm_load_model*/
/* 0 if svm_model is created by svm_train */
};
param describes the parameters used to obtain the model.
nr_class is the number of classes. It is 2 for regression and one-class SVM.
l is the number of support vectors.
SV and sv_coef are support vectors and the corresponding coefficients, respectively. Assume there are k classes. For data in class j, the corresponding sv_coef includes (k-1) y*alpha vectors, where alpha's are solutions of the following two class problems:
1 vs j, 2 vs j, ..., j-1 vs j, j vs j+1, j vs j+2, ..., j vs k
and y=1 for the first j-1 vectors, while y=-1 for the remaining k-j vectors. For example, if there are 4 classes, sv_coef and SV are like:
+-+-+-+--------------------+
|1|1|1| |
|v|v|v| SVs from class 1 |
|2|3|4| |
+-+-+-+--------------------+
|1|2|2| |
|v|v|v| SVs from class 2 |
|2|3|4| |
+-+-+-+--------------------+
|1|2|3| |
|v|v|v| SVs from class 3 |
|3|3|4| |
+-+-+-+--------------------+
|1|2|3| |
|v|v|v| SVs from class 4 |
|4|4|4| |
+-+-+-+--------------------+
See svm_train() for an example of assigning values to sv_coef.
rho is the bias term (-b).
probA and probB are parameters used in probability outputs. If there are k classes, there are k*(k-1)/2 binary problems as well as rho, probA, and probB values. They are aligned in the order of binary problems:
1 vs 2, 1 vs 3, ..., 1 vs k, 2 vs 3, ..., 2 vs k, ..., k-1 vs k.
sv_indices[0,...,nSV-1] are values in [1,...,num_traning_data] to indicate support vectors in the training set.
label contains labels in the training data.
nSV is the number of support vectors in each class.
free_sv is a flag used to determine whether the space of SV should be released in free_model_content(struct svm_model*) and free_and_destroy_model(struct svm_model**). If the model is generated by svm_train(), then SV points to data in svm_problem and should not be removed. For example, free_sv is 0 if svm_model is created by svm_train, but is 1 if created by svm_load_model.
- Function: double svm_predict(const struct svm_model *model,
const struct svm_node *x);
This function does classification or regression on a test vector x given a model.
For a classification model, the predicted class for x is returned.
For a regression model, the function value of x calculated using the model is returned.
For an one-class model, +1 or -1 is returned.
- Function: void svm_cross_validation(const struct svm_problem *prob,
const struct svm_parameter *param, int nr_fold, double *target);
This function conducts cross validation. Data are separated to nr_fold folds. Under given parameters, sequentially each fold is validated using the model from training the remaining. Predicted labels (of all prob's instances) in the validation process are stored in the array called target.
The format of svm_prob is the same as that for svm_train().
- Function: int svm_get_svm_type(const struct svm_model *model);
This function gives svm_type of the model. Possible values of svm_type are defined in svm.h.
- Function: int svm_get_nr_class(const svm_model *model);
For a classification model, this function gives the number of classes. For a regression or an one-class model, 2 is returned.
- Function: void svm_get_labels(const svm_model *model, int* label)
For a classification model, this function outputs the name of labels into an array called label. For regression and one-class models, label is unchanged.
- Function: void svm_get_sv_indices(const struct svm_model *model, int *sv_indices)
This function outputs indices of support vectors into an array called sv_indices.
The size of sv_indices is the number of support vectors and can be obtained by calling svm_get_nr_sv. Each sv_indices[i] is in the range of [1, ..., num_traning_data].
- Function: int svm_get_nr_sv(const struct svm_model *model)
This function gives the number of total support vector.
- Function: double svm_get_svr_probability(const struct svm_model *model);
For a regression model with probability information, this function outputs a value sigma > 0. For test data, we consider the probability model: target value = predicted value + z, z: Laplace distribution e^(-|z|/sigma)/(2sigma)
If the model is not for svr or does not contain required information, 0 is returned.
- Function: double svm_predict_values(const svm_model *model,
const svm_node *x, double* dec_values)
This function gives decision values on a test vector x given a model, and return the predicted label (classification) or the function value (regression).
For a classification model with nr_class classes, this function gives nr_class*(nr_class-1)/2 decision values in the array dec_values, where nr_class can be obtained from the function svm_get_nr_class. The order is label[0] vs. label[1], ..., label[0] vs. label[nr_class-1], label[1] vs. label[2], ..., label[nr_class-2] vs. label[nr_class-1], where label can be obtained from the function svm_get_labels.
The returned value is the predicted class for x. Note that when nr_class = 1, this function does not give any decision value.
For a regression model, dec_values[0] and the returned value are both the function value of x calculated using the model. For a one-class model, dec_values[0] is the decision value of x, while the returned value is +1/-1.
- Function: double svm_predict_probability(const struct svm_model *model,
const struct svm_node *x, double* prob_estimates);
This function does classification or regression on a test vector x given a model with probability information.
For a classification model with probability information, this function gives nr_class probability estimates in the array prob_estimates. nr_class can be obtained from the function svm_get_nr_class. The class with the highest probability is returned. For regression/one-class SVM, the array prob_estimates is unchanged and the returned value is the same as that of svm_predict.
- Function: const char *svm_check_parameter(const struct svm_problem *prob,
const struct svm_parameter *param);
This function checks whether the parameters are within the feasible range of the problem. This function should be called before calling svm_train() and svm_cross_validation(). It returns NULL if the parameters are feasible, otherwise an error message is returned.
- Function: int svm_check_probability_model(const struct svm_model *model);
This function checks whether the model contains required information to do probability estimates. If so, it returns +1. Otherwise, 0 is returned. This function should be called before calling svm_get_svr_probability and svm_predict_probability.
- Function: int svm_save_model(const char *model_file_name,
const struct svm_model *model);
This function saves a model to a file; returns 0 on success, or -1 if an error occurs.
- Function: struct svm_model *svm_load_model(const char *model_file_name);
This function returns a pointer to the model read from the file, or a null pointer if the model could not be loaded.
- Function: void svm_free_model_content(struct svm_model *model_ptr);
This function frees the memory used by the entries in a model structure.
- Function: void svm_free_and_destroy_model(struct svm_model **model_ptr_ptr);
This function frees the memory used by a model and destroys the model structure. It is equivalent to svm_destroy_model, which is deprecated after version 3.0.
- Function: void svm_destroy_param(struct svm_parameter *param);
This function frees the memory used by a parameter set.
- Function: void svm_set_print_string_function(void (*print_func)(const char *));
Users can specify their output format by a function. Use
svm_set_print_string_function(NULL);
for default printing to stdout.
End.
