When Compilers Disagree on C++ Templates… 2019-11-19

For those unfamiliar with LilyPond, it’s a “music engraving program” and part of the GNU Project. What I really like about its design is the music representation by plain text files: In essence, you have a set of textual input files and use the lilypond executable to produce a PDF file (or other output formats) – very similar to LaTeX. This comes with the usual advantages like the ability to put your files under version control, and so on.

LilyPond itself is written in C++ and Scheme/GUILE, more on that in the documentation. This post describes a particular issue with C++ templates that I’ve come across (and that others had already reported in the past). To make things slightly more complicated, it’s a case where Clang and GCC disagree. As a disclaimer to the following, I’m not super confident with the C++ standard. So I won’t say that the original code is broken or that one compiler is right and the other one is wrong. I just observe that LilyPond didn’t compile with Clang and report what solutions potentially work with both compilers.

The Problem

As with any real application, the code in question is more complex than required to understand the problem. To simplify discussion, I’ve reduced the problem to a single file with only few lines of code:¹

struct Base {
	void f() { }

	template <void (Base::* p)()>
	static void method_finder() {}
};

struct Derived : public Base {
	using Base::method_finder;

	template <void (Derived::* p)()>
	static void method_finder() {}
};

void use() {
	Derived::method_finder<&Derived::f>();
}

This compiles fine with GCC 9.2.0, but fails with Clang 9.0.0:

 $ g++ -c -Wall -Wextra test.cpp
[no output]
 $ clang++ -c -Wall -Wextra test.cpp 
test.cpp:16:2: error: no matching function for call to 'method_finder'
        Derived::method_finder<&Derived::f>();
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test.cpp:12:14: note: candidate template ignored: invalid explicitly-specified argument for template parameter 'p'
        static void method_finder() {}
                    ^
1 error generated.

The warning message about “invalid explicitly-specified argument” is not really helpful (GCC is much better here, explaining which arguments cannot be casted). Nevertheless it’s important to notice that Clang is only considering method_finder in Derived, not the one in Base. The reason is explained in Clang’s test/CXX/dcl.dcl/basic.namespace/namespace.udecl/p12.cpp:

// C++03 [namespace.udecl]p12:
//   When a using-declaration brings names from a base class into a
//   derived class scope, member functions in the derived class
//   override and/or hide member functions with the same name and
//   parameter types in a base class (rather than conflicting).

So Derived::method_finder shadows the base implementation and Clang doesn’t find a matching function to instantiate when called in use.

Two Possible Solutions

I think there are (at least) two possible solutions for this: First we can move method_finder out of the class and make it a (static) template in the global namespace:

struct Base {
	void f() { }
};

template <void (Base::* p)()>
static void method_finder() {}

struct Derived : public Base {
};

template <void (Derived::* p)()>
static void method_finder() {}

void use() {
	method_finder<&Derived::f>();
}

One disadvantage is that now method_finder cannot access non-public methods of Base / Derived. That’s not the case here because the code is very simple, but it would be in real applications. Additionally, LilyPond uses macros to define the method_finder templates. These are currently called in the class scope, so we would need to adapt most of the derived classes.

The other approach would be to re-declare the Base::method_finder in Derived instead of using it:

struct Base {
	void f() { }

	template <void (Base::* p)()>
	static void method_finder() {}
};

struct Derived : public Base {
	// using Base::method_finder;
	template <void (Base::* p)()>
	static void method_finder() {}

	template <void (Derived::* p)()>
	static void method_finder() {}
};

void use() {
	Derived::method_finder<&Derived::f>();
}

At first this seems horrible from a maintenance perspective as it duplicates the original Base::method_finder. However the original method_finder is defined via macros in the file lily/include/translator.hh. It is used (and thereby instantiated) in the file lily/include/translator.icc, via more macros. So for LilyPond, it’s not really a problem to redeclare method_finder because all implementations are contained in macros anyhow.

There’s still one awkward detail about this solution: It doesn’t work out-of-the-box for transitive inheritance. In the case of LilyPond, Ligature_engraver also declares some method_finder implementations that need to be available in a few derived classes. However this only matters for three cases, so it’s better than changing around 100 files when moving method_finder out of the classes. Thus, the commit for this solution is actually pretty small.

You do not need to agree with my opinions expressed in this blog post, and I'm fine with different views on certain topics. However, if there is a technical fault please send me a message so that I can correct it!