This document contains an in-depth description of the Core Knowledge System (CKS)–an integrative environment for the many functions that must be performed by sensor-based autonomous and semi-autonomous systems. The CKS itself has been designed to support a wide variety of potential applications. However, special attention has been given to assuring its relevance to a particular application–that of an autonomous land vehicle operating in an unconstrained outdoor environment. The functionality provided by the system is described, along with discussions of the various design decisions and their associated trade-offs where applicable. This paper is not intended to serve as a user’s manual, rather its purpose is to describe the CKS in sufficient detail to allow the reader to ascertain its relevance to a particular application and to provide a technical critique of its strengths and weaknesses.