In this paper, we deal with the multiclass logistic regression problem, one of the most popular supervised classification method. We aim at developing an efficient method to solve this problem for large-scale datasets, i.e. large number of features and large number of instances. To deal with a large number of features, we consider feature selection method evolving the regularization. The resulting optimization problem is non-convex for which we develop a stochastic version of DCA (Difference of Convex functions Algorithm) to solve. This approach is suitable to handle datasets with very large number of instances. Numerical experiments on several benchmark datasets and synthetic datasets illustrate the efficiency of our algorithm and its superiority over well-known methods, with respect to classification accuracy, sparsity of solution as well as running time.