Home

What are variance and bias ? Variance and bias are assertions to evaluate our model based on different datasets. Both of them happen together and we have to find the methodology to trade-off them. A good model is a small variance and bias model.  In wikipedia, we have definitions of variance and bias, formally: The  bias  is an error from erroneous assumptions in the learning  algorithm . The  variance  is an error from sensitivity to small fluctuations in the training set.  (source:  https://en.wikipedia.org/wiki/Bias–variance_tradeoff ) However, I want to make them more normal and practical, so we could understand them as bellowing definitions: Variance asserts fluctuation of precision of model against training data, test data with real data(new data). One model has a hight precision in training and test phase but has low precision with real data, It means this model has hight variance. Hight variance also is called as overfitting: Too...

   Feature selection is one of the most important step to make your model works well. In data mining, feature selection is the first step and it effects to all of process. Feature selection help model on some points: - The model will be trained faster - Reduce overfitting - Simplifying model - Reduce the dimension of data Hence, feature selection is kick-off step and it effects overall, especially in model. There are 3 type of feature selection: Filter methods, wrapper methods and embedded methods. Filter methods: this methods "filter" data based on correlation score. Normally, our data have many features, and a label. We calculate the correlations between features and label. After that, we only retrain the features that have a good (relevant) correlated score and remove others. In this type of method we have some ways to calculate the correlation. - Pearson correlation: this one is based on covariance between 2 continuous variables. $$ p_{X,Y} = \frac {Cov(X, Y)...