Multi Modal/Paper Review4 [Survey] VQA 1. VQA: Visual Question Answering (2015) Image는 CNN encoder, Question은 LSTM encoder 거친 vector를 합치는 방식 Pretrained VGG 16, LSTM 사용 2. Hierarchical Question-Image Co-Attention for Visual Question Answering (2016) Image와 Question 사이의 관계를 설명하기 위해 Attention 사용 Image feature 추출은 거의 변한 것이 없음 Question에서 더 semantic한 정보를 뽑기 위해 LSTM 구조를 hierarchical하게 변경 Image와 Question을 attention해서 unified context vector 추.. 2024. 3. 24. 🦩 Flamingo: a Visual Language Model for Few-Shot Learning Flamingo: a Visual Language Model for Few-Shot Learning Building models that can be rapidly adapted to novel tasks using only a handful of annotated examples is an open challenge for multimodal machine learning research. We introduce Flamingo, a family of Visual Language Models (VLM) with this ability. We propo arxiv.org 0. Abstract Flamingo의 주요 아키텍쳐 발전 (1) 사전 학습된 강력한 시각 전용 모델과 언어 전용 모델을 연결 (2) .. 2024. 3. 20. VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text We present a framework for learning multimodal representations from unlabeled data using convolution-free Transformer architectures. Specifically, our Video-Audio-Text Transformer (VATT) takes raw signals as inputs and extracts multimodal representations t arxiv.org 1. Abstract VATT는 트랜스포머 아키텍처를 사용해, 레이블이 없.. 2024. 3. 4. RUBi: Reducing Unimodal Biases for Visual Question Answering RUBi: Reducing Unimodal Biases in Visual Question Answering Visual Question Answering (VQA) is the task of answering questions about an image. Some VQA models often exploit unimodal biases to provide the correct answer without using the image information. As a result, they suffer from a huge drop in performance whe arxiv.org 0. Abstract 일부 VQA 모델은 image 정보를 사용하지 않고, 정답을 도출하기 위해 unimodal bias를 이용.. 2024. 2. 28. 이전 1 다음
