Description
I am working on converting the deal.II library (a large and widely used scientific computing library, see https://www.dealii.org) to use modules. deal.II consists of ~600 header and ~400 source files, and contains about 800,000 lines of code. On the plus side, after converting these to interface and implementation partition units, I can compile all but perhaps a dozen of these files. On the downside, most of the ones that fail run into errors that fundamentally look like this:
/usr/bin/clang++-20 -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_SERIALIZATION_DYN_LINK -DBOOST_SERIALIZATION_NO_LIB -DBOOST_SYSTEM_DYN_LINK -DBOOST_SYSTEM_NO_LIB -DBOOST_THREAD_DYN_LINK -DBOOST_THREAD_NO_LIB -DDEBUG -D_GLIBCXX_ASSERTIONS -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_EXTENSIVE -Ddealii_module_debug_EXPORTS -I/home/bangerth/p/deal.II/2/build/include -I/home/bangerth/p/deal.II/2/build/source -I/home/bangerth/p/deal.II/2/cxx-modules/include -isystem /home/bangerth/p/deal.II/2/cxx-modules/bundled/kokkos-3.7.00/algorithms/src -isystem /home/bangerth/p/deal.II/2/cxx-modules/bundled/kokkos-3.7.00/containers/src -isystem /home/bangerth/p/deal.II/2/cxx-modules/bundled/kokkos-3.7.00/core/src -isystem /home/bangerth/p/deal.II/2/cxx-modules/bundled/kokkos-3.7.00/simd/src -isystem /home/bangerth/p/deal.II/2/cxx-modules/bundled/kokkos-3.7.00/tpls/desul/include -isystem /usr/include/trilinos -std=c++20 -fPIC -pedantic -Wall -Wextra -Wmissing-braces -Woverloaded-virtual -Wpointer-arith -Wsign-compare -Wsuggest-override -Wswitch -Wsynth -Wwrite-strings -Wno-deprecated-declarations -Wno-psabi -Wfloat-conversion -Qunused-arguments -Wno-unsupported-friend -Wno-pass-failed -Wno-unused-local-typedefs -openmp-simd -std=c++20 -ffp-exception-behavior=strict -Og -ggdb -Wa,--compress-debug-sections -MD -MT cxx20-modules/CMakeFiles/dealii_module_debug.dir/interface_module_partitions/matrix_free/portable_matrix_free.templates.ccm.o -MF cxx20-modules/CMakeFiles/dealii_module_debug.dir/interface_module_partitions/matrix_free/portable_matrix_free.templates.ccm.o.d @cxx20-modules/CMakeFiles/dealii_module_debug.dir/interface_module_partitions/matrix_free/portable_matrix_free.templates.ccm.o.modmap -o cxx20-modules/CMakeFiles/dealii_module_debug.dir/interface_module_partitions/matrix_free/portable_matrix_free.templates.ccm.o -c /home/bangerth/p/deal.II/2/build/cxx20-modules/interface_module_partitions/matrix_free/portable_matrix_free.templates.ccm
remark: source manager location address space usage: [-Rsloc-usage]
note: 16065395B (16.07MB) in local locations, 2116191482B (2.12GB) in locations loaded from AST files, for a total of 2132256877B (2.13GB) (99% of available space)
/usr/lib/gcc/x86_64-linux-gnu/14/../../../../include/c++/14/bits/version.h:1997:54: note: file entered 6016 times using 485683712B (485.68MB) of space plus 11974477B (11.97MB) for macro expansions
1997 | # if (__cplusplus > 202302L) && _GLIBCXX_HOSTED && (__glibcxx_to_chars)
| ^
/usr/include/x86_64-linux-gnu/bits/mathcalls.h:433:10: note: file entered 861 times using 12538743B (12.54MB) of space plus 65289138B (65.29MB) for macro expansions
433 | && !__MATH_DECLARING_FLOATN
| ^
/usr/lib/gcc/x86_64-linux-gnu/14/../../../../include/c++/14/cmath:1:1: note: file entered 502 times using 48662876B (48.66MB) of space plus 114636B (114.64kB) for macro expansions
1 | // -*- C++ -*- C forwarding header.
| ^
/home/bangerth/p/deal.II/2/cxx-modules/bundled/kokkos-3.7.00/core/src/impl/KokkosExp_Host_IterateTile.hpp:2393:5: note: file entered 123 times using 14962581B (14.96MB) of space plus 32871012B (32.87MB) for macro expansions
2393 | #if KOKKOS_ENABLE_NEW_LOOP_MACROS
| ^
/home/bangerth/p/deal.II/2/cxx-modules/bundled/kokkos-3.7.00/core/src/Kokkos_CopyViews.hpp:1:1: note: file entered 246 times using 41607456B (41.61MB) of space plus 509097B (509.10kB) for macro expansions
1 | /*
| ^
/home/bangerth/p/deal.II/2/cxx-modules/bundled/kokkos-3.7.00/core/src/Kokkos_View.hpp:1:1: note: file entered 492 times using 39372300B (39.37MB) of space plus 2503173B (2.50MB) for macro expansions
1 | /*
| ^
[...]
I initially though that that must surely be some kind of internal compiler error, but I am no longer sure whether that's the case. /usr/lib/gcc/x86_64-linux-gnu/14/../../../../include/c++/14/bits/version.h is a header file that does not have an include guard against repeated inclusion:
// Copyright (C) 2023-2024 Free Software Foundation, Inc.
// This file is part of the GNU ISO C++ Library. This library is free
// software; you can redistribute it and/or modify it under the
// terms of the GNU General Public License as published by the
// Free Software Foundation; either version 3, or (at your option)
// any later version.
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// Under Section 7 of GPL version 3, you are granted additional
// permissions described in the GCC Runtime Library Exception, version
// 3.1, as published by the Free Software Foundation.
// You should have received a copy of the GNU General Public License and
// a copy of the GCC Runtime Library Exception along with this program;
// see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
// <http://www.gnu.org/licenses/>.
// DO NOT EDIT THIS FILE (version.h)
//
// It has been AutoGen-ed
// From the definitions version.def
// and the template file version.tpl
/** @file bits/version.h
* This is an internal header file, included by other library headers.
* Do not attempt to use it directly. @headername{version}
*/
// Usage guide:
//
// In your usual header, do something like:
//
// #define __glibcxx_want_ranges
// #define __glibcxx_want_concepts
// #include <bits/version.h>
//
// This will generate the FTMs you named, and let you use them in your code as
// if it was user code. All macros are also exposed under __glibcxx_NAME even
// if unwanted, to permit bits and other FTMs to depend on them for condtional
// computation without exposing extra FTMs to user code.
#pragma GCC system_header
#include <bits/c++config.h>
#if !defined(__cpp_lib_incomplete_container_elements)
[...]
In other words, it is credible that via the headers directly included by the source file portable_matrix_free.templates.ccm (an interface partition unit) and by the interface partitions that are imported by portable_matrix_free.templates.ccm, we really do reach 6016 repeated inclusions of this file, using the claimed 485683712B (485.68MB) of space plus 11974477B (11.97MB) for macro expansions. At the same time, I also have no way of actually verifying this.
How would I go about providing you with a stripped down example of this error? I could, of course, create a copy of my source directories (containing only this .ccm file, plus the ones it transitively depends on via partition imports) with a stripped down CMake file that reproduces the issue. That might still require a hundred files or more.
Or do you see a better way in which I can help provide testcases? I've got my fair share of experience in dealing with compiler bugs (on a competitor project: https://gcc.gnu.org/bugzilla/buglist.cgi?bug_status=__all__&content=bangerth&no_redirect=1&order=Importance&query_format=specific) and some interest in making sure Clang can actually compile our project, so quite happy to help provide inputs for these sorts of bugs that only show up on non-trivially large code bases.
Thank you in advance!