TL;DR; Symfony\Bundle\FrameworkBundle\CacheWarmer\TemplateFinder doesn't find templates inside the templates/ folder.

This is because the TemplateFinder is unaware of the paths setting in the twig config. Changing the default directory structure in Symfony 4 means this setting is used more often now.

Related issue: #21035
Example usage in the wild: Tracker

Steps to reproduce:

composer create-project symfony/skeleton my_project
cd my_project
composer require templating twig frameworkbundle

Add to config/services.yaml

    App\Controller\MyController:
        arguments:
            $template_finder: "@templating.finder"
        public: true

Add to src/Controller/MyController.php

<?php
namespace App\Controller;

use Symfony\Bundle\FrameworkBundle\CacheWarmer\TemplateFinder;
use Symfony\Component\HttpFoundation\Response;

class MyController
{
    private $twig;
    private $template_finder;

    public function __construct(\Twig_Environment $environment, TemplateFinder $template_finder)
    {
        $this->twig = $environment;
        $this->template_finder = $template_finder;
    }

    public function listTemplates()
    {
        return new Response(
            $this->twig->render('templates.html.twig', ['templates' => $this->template_finder->findAllTemplates()])
        );
    }
}

Add to config/routes.yaml

index:
    path: /
    controller: App\Controller\MyController::listTemplates

Finally, add to templates/templates.html.twig

{% extends "base.html.twig" %}
{% block body %}
    {% for template in templates %}
        {{ template }}<br />
    {% endfor %}
{% endblock %}

Expected: base.html.twig and templates.html.twig part of the output
Result: Only files inside the FrameworkBundle and TwigBundle found.