Research Scientist, Generalist Embodied Agent… at NVIDIA · Santa…

Summary: NVIDIA is searching for an outstanding research scientist to build humanoid robot foundation models and systems in the Generalist Embodied Agent Research (GEAR) group. The role involves designing and implementing novel AI algorithms and models for humanoid robots and collaborating with research and engineering teams to transfer research to products and services. Responsibilities: - Design and implement novel AI algorithms and models for general-purpose humanoid robots and embodied agents - Develop large-scale AI training and inference methods for foundation models - Optimize and deploy AI models in physical simulation and on robot hardware - Collaborate with research and engineering teams across all of NVIDIA to transfer research to products and services Required Qualifications: - Ph.D. in Computer Science/Engineering, Electrical Engineering, etc., or equivalent research experience - Outstanding engineering skills in rapid prototyping and model training frameworks (PyTorch, Jax, Tensorflow, etc.) - Python or C++ is required; CUDA or ROS proficiencies are a plus - Excellent knowledge and hands-on experience for training LLMs, multimodal foundation models, or large generative models - Experience in foundation and diffusion models, reinforcement learning, agent learning, and applied robotics - Experience across one or both of these fields: Multimodal Foundation Models or Robotics - Hands-on training experience and publications in at least one of the following topics: LLMs; Large vision-language models; Video generative models and diffusion algorithms; Action-based transformers - Excellent skills in working with large-scale machine learning/AI systems and compute infrastructure - Hands-on training experience and publications in robot learning, such as reinforcement learning, imitation learning, classical control methods, etc - Deep understanding of robot kinematics, dynamics, and sensors - Ability to safely operate robot hardware, lab equipment, and tools - Knowledge of control methods, including PID, model predictive control, and whole-body control - Familiarity with physics simulation frameworks such as MuJoCo and Isaac Sim - Robot hardware design and hands-on building experience Required Skills: Python, C++, PyTorch, Jax, Tensorflow, CUDA, ROS, Training large language models (LLMs), Multimodal foundation models, Large generative models, Foundation, diffusion models, Reinforcement learning, Agent learning, Applied robotics, Robot kinematics, Robot dynamics, Robot sensors, Control methods PID, Control methods model predictive control, Control methods whole-body control, Physics simulation frameworks MuJoCo, Physics simulation frameworks Isaac Sim, Robot hardware operation, Robot hardware design Benefits: Equity, Benefits