Toward a Foundation for Evaluating AI Planners

Abstract

There exists a large body of Artificial Intelligence (AI) research on generating plans, i.e., linear or nonlinear sequences of actions, to transform an initial world state to some desired goal state. However, much of the planning research to date has been complicated, ill-understood, and unclear. To make a planner applicable to different planning problems, it should be domain independent. However, one needs to know the circumstances under which a general planner works so that one can determine its suitability for a specific domain. This paper presents criteria for evaluating AI planners; these criteria fall into three categories: (1) performance issues, (2) representational issues, and (3) communication issues. This paper also assesses four nonlinear AI planners (NOAH, NONLIN, SIPE, and TWEAK).