Key Responsibilities

• Conduct a literature survey on related work.
• Develop modeling and inference algorithms to infer the operator's intention: what the operator is trying to do now, next, and in overall and predict what is the next.
• Develop algorithms to generate autonomous robotic behaviors to assist the operator with learned robotic policies, task and motion planning (TAMP), and other methods.
• Develop, train, and apply the Large Language Model, Large Vision Model, and Large Multimodal Model to model human behaviors from multimodal input and prior knowledge to infer the operator's intentions for assistance during robot teleoperation.
• Implement and validate the algorithms in simulation and on hardware.
• Publish research results at top-tier conferences and journals in robotics as well as machine learning.

Minimum Qualifications

• Currently pursuing a Ph.D. in machine learning, robotics, computer science or related fields.
• Research experience with Deep Learning, Transfer Learning, or foundational models.
• Research experience with human motion/action/task recognition and robotic policy learning using multimodal data.
• Research experience with task procedure modeling and planning.
• Experience and demonstrable knowledge in integrating large pretrained models, such as LLM (Large Language Models), LVM (Large Vision Models) and LMM (Large Multimodal Models), into applications.
• Excellent programming skills in either C++ or Python

Bonus Qualifications

• Experience in robotic teleoperation and collaborative robotic development
• Research experience with Deep Learning, Foundational Models, Transformers, (e.g. Attention, Encoder, Decoder), Representation Learning, Graphic Neural Network
• Robotic policy learning for long-horizon dexterous manipulation tasks with multi-finger hand (e.g. diffusion policy, imitation learning) and deployment of the learned policy on real robots
• Research experience with computer vision, large models, or foundation models for robotic application.
• Research experience with Task and Motion Planning (TAMP)