The word "core" in Joomla! pertains to the distributed files which are needed to create and administrate a Joomla CMS powered web site. These files can be downloaded from the Joomla website at http://www.joomla.org/download.html. The Joomla "core" also contains some basic functionality to get new Joomla installations working quickly and easily. Included are the user manager, article manager, weblink manager, category manager, contact manager, and menu manager. There is also a template manager with a few basic templates to power the front-end (website/user) view, a module manager with basic modules, plugin manager with basic plugins, and a few other "out of the box" extensions to extend the functionality of a basic Joomla installation. These core's extensions should not be confused with extensions which are available for downloading from the JED (Joomla! Extension Directory).

Reference: http://docs.joomla.org/Glossary

A Cascading Style Sheet or CSS is used to control the presentation of an XHTML page. For example, a CSS file will often control the font, margins, color, background graphics, and other aspects of a web page's appearance. CSS allows you to separate the content of an XHTML page from it's appearance. In Joomla!, CSS files (for example, template.css) are normally part of the template.

Reference: http://docs.joomla.org/Glossary