Parts of Magento use an EAV database system implemented on top of MySQL. This means querying for a single "thing" often means querying multiple rows

There's a lot of things behind the scenes (application configuration, system config, layout config, etc.) that involve building up giant XML trees in memory and then "querying" those same trees for information. This takes both memory (storing the trees) and CPU (parsing the trees). Some of these (especially the layout tree) are huge. Also, unless caching is on, these tree are built up from files on disk and on each request.

Magento uses its configuration system to allow you to override classes. This is a powerful feature, but it means anytime a model, helper, or controller is instantiated, extra PHP instructions need to run to determine if an original class file or an override class files is needed. This adds up.

Besides the layout system, Magento's template system involves a lot of recursive rendering. This ads up.