Visual Kinship Understanding

Introduction

There is an urgent need to organize and manage images of people automatically due to the recent explosion of such data on the web as well in social media. Beyond face detection and face recognition, which have been extensively studied over the past decade, the most interesting aspect related to human-centered images is the relationship of people in the image. In this work, we focus on a novel solution to the latter problem, in particular the kin relationships. Our contributions are two fold:

  1. we develop a transfer subspace learning based algorithm in order to reduce the significant differences in the appearance distributions between children and old parents facial images. Moreover, by exploring the semantic relevance of the associated metadata, we propose an algorithm to predict the most likely kin relationships embedded in an image.
  2. Motivated by the lack of a single, unified image dataset available for kinship tasks, our goal is to provide the research community with a dataset that captivates interests for involvement, i.e., large enough in scope to inherently provide platforms for multiple benchmarked tasks. We were able to collect and label the largest set of family images to date with a small team and an efficient labelling tool developed to optimize the process of marking complex hierarchical relationships, attributes, and local label information in family photos.
Related Work
  1. Siyu Xia*, Ming Shao*, and Yun Fu, Kinship Verification through Transfer Learning, International Joint Conferences on Artificial Intelligence (IJCAI), pages 2539–2544, 2011. (* indicates equal contribution) [pdf] [bib]
  2. Ming Shao, Siyu Xia, and Yun Fu, Genealogical Face Recognition based on UB KinFace Database, IEEE CVPR Workshop on Biometrics (CVPR BIOM), 65–70, 2011. (Database is available now) [pdf] [bib]
  3. Siyu Xia*, Ming Shao*, Jiebo Luo, and Yun Fu , Understanding Kin Relationships in a Photo, IEEE Transactions on Multimedia (T-MM), vol. 14, no. 4, pages 1046–1056, 2012. (* indicates equal contributions) [pdf] [bib]
  4. Joseph  Robinson, Ming  Shao, Yue  Wu, and Yun  Fu, Family in the wild (FIW): Large-Scale Kinship Image Database and Benchmarks, ACM Multimedia Conference, pages 242–246, 2016. [pdf] [bib]
  5. Junkang Zhang, Siyu Xia, Ming Shao, and Yun Fu, Family Photo Recognition via Multiple Instance Learning, ACM International Conference on Multimedia Retrieval (ICMR), 2017. [pdf]
  6. Joseph P. Robinson, Ming Shao , Yue Wu, Hongfu Liu, Timothy Gillis, and Yun Fu,, Visual Kinship Recognition of Families in the Wild, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2018 (in press).