Use __builtin_isnan/isinf/isfinite in complex
The libc-provided isnan/isinf/isfinite macro implementations are specifically
designed to function correctly, even in the presence of -ffast-math (or, more
specifically, -ffinite-math-only). As such, on most implementation, these
either always turn into external function calls (e.g. glibc) or are
specifically function calls when FINITE_MATH_ONLY is defined (e.g. Darwin).
Our implementation of complex arithmetic makes heavy use of isnan/isinf/isfinite
to deal with corner cases involving non-finite quantities. This was problematic
in two respects:
1. On systems where these are always function calls (e.g. Linux/glibc), there was a
performance penalty
2. When compiling with -ffast-math, there was a significant performance
penalty (in fact, on Darwin and systems with similar implementations, the code
may in fact be slower than not using -ffast-math, because the inline
definitions provided by libc become unavailable to prevent the checks from
being optimized out).
Eliding these inf/nan checks in -ffast-math mode is consistent with what
happens with libstdc++, and in my experience, what users expect. This is
critical to getting high-performance code when using complex<T>. This change
replaces uses of those functions on basic floating-point types with calls to
__builtin_isnan/isinf/isfinite, which Clang will always expand inline. When
using -ffast-math (or -ffinite-math-only), the optimizer will remove the checks
as expected.
Differential Revision: https://reviews.llvm.org/D18639
llvm-svn: 283051
Cr-Mirrored-From: sso://chromium.googlesource.com/_direct/external/github.com/llvm/llvm-project
Cr-Mirrored-Commit: ae22f0b2423192c919ad3c17627a5d40cfcdc5dc
diff --git a/include/cmath b/include/cmath
index a1d4ee0..ec28e0e 100644
--- a/include/cmath
+++ b/include/cmath
@@ -558,6 +558,66 @@
}
#endif
+template <class _A1>
+_LIBCPP_ALWAYS_INLINE
+typename enable_if<is_floating_point<_A1>::value, bool>::type
+__libcpp_isnan(_A1 __lcpp_x) _NOEXCEPT
+{
+#if __has_builtin(__builtin_isnan)
+ return __builtin_isnan(__lcpp_x);
+#else
+ return isnan(__lcpp_x);
+#endif
+}
+
+template <class _A1>
+_LIBCPP_ALWAYS_INLINE
+typename enable_if<!is_floating_point<_A1>::value, bool>::type
+__libcpp_isnan(_A1 __lcpp_x) _NOEXCEPT
+{
+ return isnan(__lcpp_x);
+}
+
+template <class _A1>
+_LIBCPP_ALWAYS_INLINE
+typename enable_if<is_floating_point<_A1>::value, bool>::type
+__libcpp_isinf(_A1 __lcpp_x) _NOEXCEPT
+{
+#if __has_builtin(__builtin_isinf)
+ return __builtin_isinf(__lcpp_x);
+#else
+ return isinf(__lcpp_x);
+#endif
+}
+
+template <class _A1>
+_LIBCPP_ALWAYS_INLINE
+typename enable_if<!is_floating_point<_A1>::value, bool>::type
+__libcpp_isinf(_A1 __lcpp_x) _NOEXCEPT
+{
+ return isinf(__lcpp_x);
+}
+
+template <class _A1>
+_LIBCPP_ALWAYS_INLINE
+typename enable_if<is_floating_point<_A1>::value, bool>::type
+__libcpp_isfinite(_A1 __lcpp_x) _NOEXCEPT
+{
+#if __has_builtin(__builtin_isfinite)
+ return __builtin_isfinite(__lcpp_x);
+#else
+ return isfinite(__lcpp_x);
+#endif
+}
+
+template <class _A1>
+_LIBCPP_ALWAYS_INLINE
+typename enable_if<!is_floating_point<_A1>::value, bool>::type
+__libcpp_isfinite(_A1 __lcpp_x) _NOEXCEPT
+{
+ return isfinite(__lcpp_x);
+}
+
_LIBCPP_END_NAMESPACE_STD
#endif // _LIBCPP_CMATH