It's been a while for me as well, and I don't know what's original and what's a backronym, but the well-known "PHP Hypertext Preprocessor" name is sort of part of the answer. The "weird \"feature\"" is central to the point of PHP, which you sort of mentioned, just overspecified as dynamic snippets instead of reusable snippets generally. Since a php include() could (and very often did) pull in things like HTML (page templates) or CSS (pre-SASS/LESS/variables) it was necessary, or rather natural, to explicitly invoke PHP (invoke it again, even, separately from the tag preceding the include statement).

This was the tradeoff (or should I say trade-up) for going from

    #!/usr/bin/perl -w
    use strict;
    my $username = <>; 
    chomp($username);
    # blah blah blah blah blah
    print "<html>\n<head>\n[...]<body>\n";
    print "<p>Hello, " . $username . "!</p>";
    print "</body>\n</html>"; 

to

    <html>
    <head>[...] 
    <body>
    <p>Hello, <?php echo $username; ?>!</p>
    </body>
    </html>

(Of course, as you point out, we promptly recreated the former situation in PHP -- but maybe there was some rationale to it? I wouldn't know; I did my best to avoid that kind of overwrought PHP project)

    for$n(1..100){$\=$/;$n3=$n%3;$n5=$n%5;print!$n3&&!$n5?fizzbuzz:!$n5?buzz:!$n3?fizz:$n}