Liu, T. and Tao, D. and Song, M. and Maybank, Stephen J. (2016) Algorithm-dependent generalization bounds for multi-task learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 39 (2), pp. 227-241. ISSN 0162-8828.
|
Text
14689.pdf - Author's Accepted Manuscript Download (366kB) | Preview |
Abstract
Often, tasks are collected for multi-task learning (MTL) because they share similar feature structures. Based on this observation, in this paper, we present novel algorithm-dependent generalization bounds for MTL by exploiting the notion of algorithmic stability. We focus on the performance of one particular task and the average performance over multiple tasks by analyzing the generalization 1 ability of a common parameter that is shared in MTL. When focusing on one particular task, with the help of a mild assumption on the feature structures, we interpret the function of the other tasks as a regularizer that produces a specific inductive bias. The algorithm for learning the common parameter, as well as the predictor, is thereby uniformly stable with respect to the domain of the particular task and has a generalization bound with a fast convergence rate of order O(1=n), where n is the sample size of the particular task. When focusing on the average performance over multiple tasks, we prove that a similar inductive bias exists under certain conditions on the feature structures. Thus, the corresponding algorithm for learning the common parameter is also uniformly stable with respect to the domains of the multiple tasks, and its generalization bound is of the order O(1=T ), where T is the number of tasks. These theoretical analyses naturally show that the similarity of feature structures in MTL will lead to specific regularizations for predicting, which enables the learning algorithms to generalize fast and correctly from a few examples.
Metadata
| Item Type: | Article |
|---|---|
| Additional Information: | (c) 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works. |
| School: | Birkbeck Faculties and Schools > Faculty of Science > School of Computing and Mathematical Sciences |
| Depositing User: | Administrator |
| Date Deposited: | 20 Apr 2016 13:25 |
| Last Modified: | 09 Aug 2023 12:37 |
| URI: | https://eprints.bbk.ac.uk/id/eprint/14689 |
Statistics
Additional statistics are available via IRStats2.
Archive Staff Only (login required)
