01 — MULTIMODAL DEEP LEARNING
Learning across
vision and language.
SemGrasp combines visual input and natural-language task descriptions to predict grasp quality. A robotics problem becomes a study in multimodal representation learning.
My work compares a cross-modal transformer against a FiLM-conditioned ResNet baseline, exploring how a physics-informed head supports task-conditioned prediction.
Explore SemGraspTWO MODALITIES. ONE PREDICTION.
Concept illustration · Multimodal prediction