I would agree that themes and plugins are the two biggest culprits.

However, using a CDN can make a huge difference with ANY site, not just WordPress. It usually means faster downloads of Susie resources and a higher number of parallel downloads for faster overall rendering (most browsers have per-domain downloading limits) and a higher chance of the resources already being cached. Plus, it frees up the web service from having to serve up those static files.

Nginx and php-fpm can also make a dramatic impact on scalability. Using FPM can reduce the memory footprint of each web server worker and recycle PHP workers faster, leading to the ability to handle more requests per second. Nginx tends to have a lower per-child footprint but also sucks if you rely on htaccess files, since those are Apache-only.

In order of greatest to least opportunity for performance improvement, I'd say:
1. Themes and plugins (any custom code)
2. CDN usage.
3. Opcache/compiling.
4. PHP 7.
5. Non-shared hosting.
6. Tuned MySQL.
7. Tuned web server.
8. Tuned OS.

Of course, any one of these could have a critical misconfiguration that could cause a huge performance loss but that's not as common.

Shared hosting = the most common detriment to page load

I'd generally disagree with that. It depends on the provider, but USUALLY a host isn't so under-powered that it is actually processing PHP at a slow speed. Bear in mind that a normal WordPress install's PHP will usually be processed in under a second. Even if you had a really underpowered or overloaded host, that might double at most (unless they're doing something REALLY wrong).

Plugins are usually where things go wrong. People install plugins that each slow down the page by 50 milliseconds here, or 250 milliseconds there, and it adds up when you install a dozen of them. Suddenly, you have 12 plugins that are cumulatively adding 4-5 seconds to the PHP processing time, even on a dedicated host. Themes CAN have similar impacts, although they're less likely to, since their impact is more on the static asset side (e.g. adding 15 images that have to be downloaded - even if the PHP code takes 1 second to process, it may take 5-10 seconds to download all the theme contents).

If you doubt the difference, use XDebug to generate a cachegrind on a shared host and a dedicated host and compare the differences.