16-890: Robot Cognition for Manipulation
Spring 2023
Instructors:
Chris Atkeson, cga@cmu.edu
Mrinal Verghese, mverghes@andrew.cmu.edu
Time:
Tuesday, Thursday 7:00-8:20pm
Location:
Newell-Simon Hall 3002 Zoom
Quick Links:
Piazza
Zoom (See Piazza for password)
Paper Presentation Signups
Example one-page summary
Announcements:
- 1/24: Robot Cognition is going remote! We will still be at the same time, use the Zoom link to attend.
- 1/24: The link to paper presentation signups is posted. Presentation slots highlighted in green are available. Note some slots already have assigned papers, while other slots are for you to present a paper of your choice. See the list of papers for options, or contact us if you would like to present a paper not on the list!
Course Description:
This seminar course will cover a mixture of modern and classical methods for robot cognition. We will review papers related to task planning and control using both symbolic and numeric methods. The goal of this course is to give students an overview of the current state of research on robot cognition.
Areas of Interest:
Click each area to see a list of relevant papers
- How should robots represent their environment?
- How should robots represent their actions and policies?
- What abstractions can robots make?
- This section includes:
- Symbolic environment representations
- Skill-based action representations
- Representation learning
- How can robots plan a series of actions to complete a task?
- This section includes:
- Task planning with skills
- Task planning with large language models
- How can robots leverage cultural knowledge?
- How can robots leverage large offline datasets?
- This section includes :
- Learning from large offline datasets
- Learning from demonstration
- How should robots learn from prior experiences?
- What does robot memory look like?
- This section includes:
- Reinforcement learning
- Memory-based approaches
- How should robots effectively communicate with humans?
- How should robots effectively communicate with other robots?
- This section includes:
- Online robot correction
- Multi-agent coordination
- Preference learning
Topics of Interest:
Here are some of the topics we would like to cover in this class, feel free to suggest further topics for discussion!
- Memory and case-based reasoning
- Parametric function approaches to learning and memory
- Symbolic and numeric perception, reasoning, planning, and control
- Parametric function approaches to symbolic and numeric …
- Cultural knowledge
- Memory systems: representations and indexing
- Avoiding repeating mistakes
- Language as a tool for representation/abstraction/reasoning/planning
- Language tools: WordNet, FrameNet, Large language models, …
- Representing objects (nouns and adjectives)
- Representing relationships (prepositional phrases)
- Representing actions (verbs and adverbs)
- Developing useful abstractions and hierarchies
- Model-based approaches, including learning task-level and more detailed models
- Model-free approaches, including learning reusable skills
- Planning under uncertainty/Planning for failure/Handling errors
- Long horizon task planning
- How do we get robots with human-level and human-expert-level performance?