• Home
  • Blog
  • People
  • Projects
  • Publications
  • Seminars
  • DSR Expo
  • Courses

Data Science Research

Menu
  • Home
  • Blog
  • People
  • Projects
  • Publications
  • Seminars
  • DSR Expo
  • Courses

VITA: Multimodal Knowledge Extraction and Fusion

VITA: Multimodal Knowledge Extraction and Fusion

In recent years, we have not only observed larger and larger datasets, but also more and more complex datasets with different types of data arte app videos herunterladen. For example, Wikipedia is a huge dataset with unstructured text, semi-structured documents, structured knowledge and images. The VITA project is focused on extracting and fusing knowledge from different data sources email herunterladen android. We introduce a few projects for various applications using multimodal fusion.

Multimodal Ensemble Fusion for Disambiguation and Retrieval
We designed a probabilistic ensemble fusion model to combine results from both images and text for information retrieval and word sense disambiguation download play games. In our ensemble fusion model, text processing and image processing are conducted on text and images separately and a fusion algorithm is used to combine the results library directly. We achieved better performance using multimodal fusion compared to single-modality approaches by exploiting the complementary and correlative relations between different modalities severely disabled card.

Query-driven Knowledge Base Completion with Multimodal Fusion
Over the past few years, massive amounts of world knowledge have been accumulated in publicly available knowledge bases (KBs), such as Freebase, NELL, and YAGO all images of a page. Yet despite their seemingly huge size, these knowledge bases are greatly incomplete. Knowledge base completion (KBC) is the task to fill in missing facts for entities in knowledge bases wii sports gratis downloaden. We design and implement a query-driven knowledge base system with web-based question answering and augmented rule inference. We combine information from the unstructured Web and structured KBs to improve KBC performance herunterladen. We use query-driven techniques to improve the efficiency of the pipeline to provide fast real-time responses to user queries. Experimental results show our system achieves state-of-the-art KBC performance and high efficiency song for free.

Faculty: Daisy Zhe Wang
Students: Yang Peng, Dihong Gong, Xiaofeng Zhou, Ishan Patwa

Publications

  • Multimodal Learning for Web Information Extraction
    Dihong Gong, Daisy Zhe Wang, Yang Peng
    ACM Multimedia, 2017
  • Extracting Visual Knowledge from the Web with Multimodal Learning
    Dihong Gong, Daisy Zhe Wang
    The International Joint Conference on Artificial Intelligence (IJCAI), 2017
  • Multimodal Ensemble Fusion for Disambiguation and Retrieval
    Yang Peng, Xiaofeng Zhou, Daisy Zhe Wang, Ishan Patwa, Dihong Gong, Chunsheng Victor Fang
    Proceedings of the IEEE Multimedia Magazine, 2016
  • Scalable Image Retrieval with Multimodal Fusion
    Yang Peng, Xiaofeng Zhou, Daisy Zhe Wang, Chunsheng Victor Fang
    Proceedings of the 29th International FLAIRS conference, 2016
  • Probabilistic Ensemble Fusion for Multimodal Word Sense Disambiguation
    Yang Peng, Daisy Zhe Wang, Ishan Patwa, Dihong Gong, Victor Chunsheng Fang
    IEEE International Symposium on Multimedia, 2015

Seminar Talks

  • Towards Building Large-Scale Multimodal Knowledge Bases

Data

  • Visual Rules of 42 Categories

Sponsors

NIST

Adobe_Logo

DTCC

pcori uf-clinical

ICHP

Recent Posts

Recent Posts

  • A Brief Overview of Weak Supervision
  • DRUM: End-To-End Differentiable Rule Mining On Knowledge Graphs
  • IDTrees Data Science Challenge: 2017
  • Efficient Conditional Rule Mining over Knowledge Bases
  • Taming The Data Monster To Make Better Decisions

Related Blogs

  • ampLab
  • Data Beta
  • Fast ML

Post Categories

Categories

  • courses
  • ecology
  • NIST and open eval
  • publications
  • research
  • research directions
  • survey
  • Uncategorized

Archives

Archives

  • October 2020
  • December 2019
  • April 2019
  • December 2018
  • August 2018
  • February 2018
  • November 2017
  • June 2017
  • May 2017
  • March 2017
  • December 2016
  • October 2016
  • April 2016
  • March 2016
  • December 2015
  • November 2015
  • October 2015
  • May 2015
  • November 2014
  • October 2014
  • July 2014
  • May 2014
  • March 2014
  • December 2013
  • November 2013
  • October 2013
  • September 2013

Meta

DSR Wiki
Site Admin
WordPress.org
  • Whsmith Lodgers Agreement
  • What To Ask For In A Prenuptial Agreement
  • What Is Department Of State Corporation Bureau Or Business Partnership Agreement
  • What Is A General Security Agreement Nz
  • What Agreement Led To The Establishment Of The Euro A Common European Currency Quizlet
  • Vmware Service Provider License Agreement
  • Validity Of Debt Agreement In India
  • University Of Manitoba Unifor Collective Agreement
  • U.s.-China Trade Agreement 1999
  • Training Agreement Plan Definition
  • Thoroughbred Lease Agreement
  • The Canada-Us-Mexico Agreement Enters Into Force July 1
  • Td Ameritrade Brokerage Agreement
  • Subscription Service Agreements
  • Subject And Verbs Agreement
  • Standard Non Disclosure Agreement Australia
  • Source Code Development Agreement
  • Simple One Page Room Rental Agreement Pdf
  • Shareholders Agreements Sweet Maxwell
  • Service Purchase Agreement Meaning