Multi-label classification is a supervised learning, where one example can belong to several classes. In the case of Support Vector Machine (SVM), One-versus-All (OVA) is the most common approach to tackle this problem. However, the accuracy is very limited due to extremely imbalanced training set. It is interesting that there have been only very few works that applied One-versus-One (OVO) in the multi-label domain even though it has been shown to provide better accuracy than OVA in the multiclass domain. Anyway, OVO requires an extremely high computational cost when there is a large number of labels. This research propose a multi-label classification framework that employs OVO incorporating with the undersampling, technique to alleviate the imbalanced issue. Spark framework along with a mechanism was applied to split a job to a set of small jobs and then processed them in parallel. The framework can induce OVO SVMs very fast, while maintaining the prediction accuracy even though, there is a large number of classes. The experiment was conducted on 6 standard multi-label datasets. The result indicate that our framework can really reduce computing time on Spark environment, while significantly outperforms OVA in terms of F1 on all data.