| Proj No. | B3061-251 |
| Title | Multimodal Large Language Model for Smart Robotics |
| Summary | This project aims to develop and integrate multimodal large language models into the field of smart robotics. The project will focus on enhancing robots' ability to understand and interact with their environment through the processing of multiple data types (text, visual data, and sensory feedback). By employing advanced machine learning techniques, particularly in natural language processing and computer vision, the robots will develop more intuitive interaction capabilities with human operators and improve task execution in complex environments. Students will engage in the development, testing, and refinement of these models throughout the project. Students will participate actively in innovative research and receive practical training in a range of technologies, including advanced machine learning algorithms, multimodal data processing, natural language understanding, and robotics system integration. The experience will also cover programming in environments like Python, use of deep learning frameworks such as TensorFlow or PyTorch, and exposure to real-world robotic applications. 1. Task-specific finetuning of large language models that integrate textual, visual, and sensory data for robotic systems. 2. Implement and evaluate these models in real-world robotic tasks to assess performance and make iterative improvements. Candidates should possess a solid understanding of Python programming and have experience with or a strong interest in natural language processing and computer vision. Prior exposure to deep learning frameworks like TensorFlow or PyTorch is highly advantageous. |
| Supervisor | A/P Jiang Xudong (Loc:S1 > S1 B1C > S1 B1C 105, Ext: +65 67905018) |
| Co-Supervisor | - |
| RI Co-Supervisor | Dr Zhu Haiyue (RI) |
| Lab | |
| Single/Group: | Single |
| Area: | Intelligent Systems and Control Engineering |
| ISP/RI/SMP/SCP?: | RI: Singapore Institute of Manufacturing Technology (SIMTech) Dr Zhu Haiyue Scientist I Email: ZHU_HAIYUE@SIMTECH.A-STAR.EDU.SG Tel: 65903158; Fax: |
| Available for Phase 2 selection? (for RI projects only) |
No |