Self-supervised cross modality representation learning

Under guidance of Prof. Preethi Jyothi and Prof. Ganesh Ramakrishnan

The seminar has focused around multi-modal learning and how self-supervised objectives learn better embeddings.

In this work, we learn and critique various self-supervised learning techniques.

Responsibilities :

• Explored self-supervised representation learning for unimodal (like BERT, PASE) and multimodal setting (like ViLBERT, LXMERT, UNITER)
• Understood the nuances of pre-training in feature learning
• Suggested audio-linguistic feature representation, that gave 5.6% improvement on IoU= 0.7 metric
• Technologies: Python and PyTorch