| Proj No. | A3221-251 |
| Title | Foundation Model-based Control and Simulation for Robotic Arms |
| Summary | Automated robotic arms are nowadays extensively used in many industries, e.g., manufacturing, biomedicine, warehousing, research and military, etc. Traditionally, they are controlled using designed commands and pre-defined trajectories under a fixed environment, which are hard to program, instructed and updated. The recent advances in AI allow us to control robotic arms using a more interactive way, i.e., directly from human inputs such as visual recording of human motion. In this project, the student will attempt to develop a novel simulation environment for advanced robotic arm control with the help of the recent foundation models, especially CLIP and LLaVA. We have developed some basic robotic arm environments using VREP and set up baseline deep reinforcement learning (DRL) algorithms for realizing imitation learning. The students need to further optimize the system to multi-agent settings to overcome more challenging cases. Interested candidates can email their CVs to me (bihan.wen@ntu.edu.sg). Only qualified candidates will be notified. |
| Supervisor | A/P Wen Bihan (Loc:S2 > S2 B2B > S2 B2B 54, Ext: +65 67904708) |
| Co-Supervisor | - |
| RI Co-Supervisor | - |
| Lab | Satellite Research Centre (Loc: S2.2-B3-06) |
| Single/Group: | Single |
| Area: | Digital Media Processing and Computer Engineering |
| ISP/RI/SMP/SCP?: |