This patch implements the forwarding to frozen C++03 headers as
discussed in
https://discourse.llvm.org/t/rfc-freezing-c-03-headers-in-libc. In the
RFC, we initially proposed selecting the right headers from the Clang
driver, however consensus seemed to steer towards handling this in the
library itself. This patch implements that direction.
At a high level, the changes basically amount to making each public
header look like this:
```
// inside <vector>
#ifdef _LIBCPP_CXX03_LANG
# include <__cxx03/vector>
#else
// normal <vector> content
#endif
```
In most cases, public headers are simple umbrella headers so there isn't
much code in the #else branch. In other cases, the #else branch contains
the actual implementation of the header.
This adds the new std::enable_nonlocking_formatter_optimization trait in
<format>. This trait will be used in std::print to implement the
performance benefits.
Implements parts of
- P3107R5 - Permit an efficient implementation of ``std::print``
Pull request #96032 unconditionall adds the `cwchar` include in the
`format` umbrella header. However support for wchar_t can be disabled in
the build system (LIBCXX_ENABLE_WIDE_CHARACTERS).
This patch guards against inclusion of `cwchar` in `format` by checking
the `_LIBCPP_HAS_NO_WIDE_CHARACTERS` define.
For clarity I've also merged the include header section that `cwchar`
was in with the one above as they were both guarded by the same `#if`
logic.
The previous patch implemented
- P2713R1 Escaping improvements in std::format
- LWG3965 Incorrect example in [format.string.escaped] p3 for formatting
of combining characters
These changes were correct, but had a size and performance penalty. This
patch improves the size and performance of the previous patch. The
performance is still worse than before since the lookups may require two
property lookups instead of one before implementing the paper. The
changes give a tighter coupling between the Unicode data and the
algorithm. Additional tests are added to notify about changes in future
Unicode updates.
Before
```
-----------------------------------------------------------------------
Benchmark Time CPU Iterations
-----------------------------------------------------------------------
BM_ascii_escaped<char> 110704 ns 110696 ns 6206
BM_unicode_escaped<char> 101371 ns 101374 ns 6862
BM_cyrillic_escaped<char> 63329 ns 63327 ns 11013
BM_japanese_escaped<char> 41223 ns 41225 ns 16938
BM_emoji_escaped<char> 111022 ns 111021 ns 6304
BM_ascii_escaped<wchar_t> 112441 ns 112443 ns 6231
BM_unicode_escaped<wchar_t> 102776 ns 102779 ns 6813
BM_cyrillic_escaped<wchar_t> 58977 ns 58975 ns 11868
BM_japanese_escaped<wchar_t> 36885 ns 36886 ns 18975
BM_emoji_escaped<wchar_t> 115885 ns 115881 ns 6051
```
The first change is to manually encode the entire last area and make a
manual exception for the 240 excluded entries. This reduced the table
from 1077 to 729 entries and gave the following benchmark results.
```
-----------------------------------------------------------------------
Benchmark Time CPU Iterations
-----------------------------------------------------------------------
BM_ascii_escaped<char> 104777 ns 104776 ns 6550
BM_unicode_escaped<char> 96980 ns 96982 ns 7238
BM_cyrillic_escaped<char> 60254 ns 60251 ns 11670
BM_japanese_escaped<char> 44452 ns 44452 ns 15734
BM_emoji_escaped<char> 104557 ns 104551 ns 6685
BM_ascii_escaped<wchar_t> 107456 ns 107454 ns 6505
BM_unicode_escaped<wchar_t> 96219 ns 96216 ns 7301
BM_cyrillic_escaped<wchar_t> 56921 ns 56904 ns 12288
BM_japanese_escaped<wchar_t> 39530 ns 39529 ns 17492
BM_emoji_escaped<wchar_t> 108494 ns 108496 ns 6408
```
An entry in the table can only contain 2048 code points. For larger
ranges there are multiple entries split in chunks with a maximum size of
2048 entries. To encode the entire Unicode code point range 21 bits are
required. The manual part starts at 0x323B0 this means all entries in
the table fit in 18 bits. This allows to allocate 3 additional bits for
the range. This allows entries to have 16384 elements. This range always
avoids splitting the range in multiple chunks.
This reduces the number of table elements from 729 to 711 and gives the
following benchmark results.
```
-----------------------------------------------------------------------
Benchmark Time CPU Iterations
-----------------------------------------------------------------------
BM_ascii_escaped<char> 104289 ns 104289 ns 6619
BM_unicode_escaped<char> 96682 ns 96681 ns 7215
BM_cyrillic_escaped<char> 59673 ns 59673 ns 11732
BM_japanese_escaped<char> 41983 ns 41982 ns 16646
BM_emoji_escaped<char> 104119 ns 104120 ns 6683
BM_ascii_escaped<wchar_t> 104503 ns 104505 ns 6693
BM_unicode_escaped<wchar_t> 93426 ns 93423 ns 7489
BM_cyrillic_escaped<wchar_t> 54858 ns 54859 ns 12742
BM_japanese_escaped<wchar_t> 36385 ns 36384 ns 19259
BM_emoji_escaped<wchar_t> 105608 ns 105610 ns 6592
```
Originally, we used __libcpp_verbose_abort to handle assertion failures.
That function was declared from all public headers. Since we don't use
that mechanism anymore, we don't need to declare __libcpp_verbose_abort
from all public headers, and we can clean up a lot of unnecessary
includes.
This patch also moves the definition of the various assertion categories
to the <__assert> header, since we now rely on regular IWYU for these
assertion macros.
rdar://105510916
This reverts commit 7d9b5aa65b09126031e1c2903605a7d34aea4bc1 since
std/utilities/format/format.arguments/format.arg/visit.return_type.pass.cpp
is failing on Windows when building with Clang-cl.
This change requires quite a number of changes in the tests; this is not
code I expect people to use in the wild. So I don't expect breakage for
users.
Implements:
- P2905R2 Runtime format strings, as a Defect Report
The code has been quite ready for a while now and there are no more ABI
breaking papers. So this is a good time to mark the feature as stable.
Reviewed By: #libc, ldionne
Differential Revision: https://reviews.llvm.org/D150802
This adds an incomplete version where the specializations for the
format_kinds are disabled dummy formatters.
Implements part of
- P2585R0 Improving default container formatting
Reviewed By: ldionne, #libc
Differential Revision: https://reviews.llvm.org/D137271
While working on D129964 I noticed some code hadn't been uglyfied, this
rectifies the issue.
Depends on D129964
Reviewed By: #libc, philnik
Differential Revision: https://reviews.llvm.org/D131834
This paper was accepted during the last plenary and is intended to be
backported to LLVM 15. When backporting the release notes in the branch
should be updated too.
Note the feature-test macro isn't updated since this will change; three
papers have updated the same macro in the same plenary.
Implements:
- P2508R1 Exposing std::basic-format-string
Reviewed By: ldionne, #libc
Differential Revision: https://reviews.llvm.org/D130643
This implements the Grapheme clustering as required by
P1868R2 width: clarifying units of width and precision in std::format
This was omitted in the initial patch, but the paper was marked as completed. This really completes the paper.
Reviewed By: ldionne, #libc
Differential Revision: https://reviews.llvm.org/D126971
In particular remove the ability to expel incomplete features from the
library at configure-time, since this can now be done through the
_LIBCPP_ENABLE_EXPERIMENTAL macro.
Also, never provide symbols related to incomplete features inside the
dylib, instead provide them in c++experimental.a (this changes the
symbols list, but not for any configuration that should have shipped).
Differential Revision: https://reviews.llvm.org/D128928
A format string like "{}" is quite common. In this case avoid parsing
the format-spec when it's not present. Before the parsing was always
called, therefore some refactoring is done to make sure the formatters
work properly when their parse member isn't called.
From the wording it's not entirely clear whether this optimization is
allowed
[tab:formatter]
```
and the range [pc.begin(), pc.end()) from the last call to f.parse(pc).
```
Implies there's always a call to `f.parse` even when the format-spec
isn't present. Therefore this optimization isn't done for handle
classes; it's unclear whether that would break user defined formatters.
The improvements give a small reduciton is code size:
719408 12472 488 732368 b2cd0 before
718824 12472 488 731784 b2a88 after
The performance benefits when not using a format-spec are:
```
Comparing ./formatter_int.libcxx.out-baseline to ./formatter_int.libcxx.out
Benchmark Time CPU Time Old Time New CPU Old CPU New
----------------------------------------------------------------------------------------------------------------------------------------------------
BM_Basic<uint32_t> -0.0688 -0.0687 67 62 67 62
BM_Basic<int32_t> -0.1105 -0.1107 73 65 73 65
BM_Basic<uint64_t> -0.1053 -0.1049 95 85 95 85
BM_Basic<int64_t> -0.0889 -0.0888 93 85 93 85
BM_BasicLow<__uint128_t> -0.0655 -0.0655 96 90 96 90
BM_BasicLow<__int128_t> -0.0693 -0.0694 97 90 97 90
BM_Basic<__uint128_t> -0.0359 -0.0359 256 247 256 247
BM_Basic<__int128_t> -0.0414 -0.0414 239 229 239 229
```
For the cases where a format-spec is used the results remain similar,
some are faster some are slower, differing per run.
Reviewed By: ldionne, #libc
Differential Revision: https://reviews.llvm.org/D129426
This implements a not accepted LWG issue. Not doing so would require
integral types to use the handle class instead of being directly stored
in the basic_format_arg.
The previous code used `std::forward` in places where it wasn't required
by the Standard. These are now removed.
Implements:
- P2418R2 Add support for std::generator-like types to std::format
- LWG 3631 basic_format_arg(T&&) should use remove_cvref_t<T> throughout
Reviewed By: ldionne, #libc
Differential Revision: https://reviews.llvm.org/D127570
This changes the implementation of the formatter. Instead of inheriting
from a specialized parser all formatters will use the same generic
parser. This reduces the binary size.
The new parser contains some additional fields only used in the chrono
formatting. Since this doesn't change the size of the parser the fields
are in the generic parser. The parser is designed to fit in 128-bit,
making it cheap to pass by value.
The new format function is a const member function. This isn't required
by the Standard yet, but it will be after LWG-3636 is accepted.
Additionally P2286 adds a formattable concept which requires the member
function to be const qualified in C++23. This paper is likely to be
accepted in the 2022 July plenary.
Depends on D121530
NOTE parts of the code now contains duplicates for the current and new parser.
The intention is to remove the duplication in followup patches. A general
overview of the final code is available in D124620. That review however lacks a
bit of polish.
Most of the new code is based on the same algorithms used in the current code.
The final version of this code reduces the binary size by 17 KB for this example
code
```
int main() {
{
std::string_view sv{"hello world"};
std::format("{}{}|{}{}{}{}{}{}|{}{}{}{}{}{}|{}{}{}|{}{}|{}", true, '*',
(signed char)(42), (short)(42), (int)(42), (long)(42), (long long)(42), (__int128_t)(42),
(unsigned char)(42), (unsigned short)(42), (unsigned int)(42), (unsigned long)(42),
(unsigned long long)(42), (__uint128_t)(42),
(float)(42), (double)(42), (long double)(42),
"hello world", sv,
nullptr);
}
{
std::wstring_view sv{L"hello world"};
std::format(L"{}{}|{}{}{}{}{}{}|{}{}{}{}{}{}|{}{}{}|{}{}|{}", true, L'*',
(signed char)(42), (short)(42), (int)(42), (long)(42), (long long)(42), (__int128_t)(42),
(unsigned char)(42), (unsigned short)(42), (unsigned int)(42), (unsigned long)(42),
(unsigned long long)(42), (__uint128_t)(42),
(float)(42), (double)(42), (long double)(42),
L"hello world", sv,
nullptr);
}
}
```
Reviewed By: #libc, ldionne
Differential Revision: https://reviews.llvm.org/D125606
Use a specialized buffer wrapper to limit the number of insertions in the
buffer. After the limit has been reached the buffer only needs to count
the number of insertions to return the buffer size required to store the
entire output.
Depends on D110498
Reviewed By: #libc, Mordante
Differential Revision: https://reviews.llvm.org/D110499
This optimizes the __format_arg_store type to allow a more efficient
storage of the basic_format_args.
It stores the data in two arrays:
- A struct with the tag of the exposition only variant's type and the
offset of the element in the data array. Since this array only depends
on the type information it's calculated at compile time and can be
shared by different instances of this class.
- The arguments converted to the types used in the exposition only
variant of basic_format_arg. This means the packed data can be
directly copied to an element of this variant.
The new code uses rvalue reference arguments in preparation for P2418.
The handle class also has some changes to prepare for P2418. The real
changed for P2418 will be done separately, but these parts make it
easier to implement that paper.
Some parts of existing test code are removed since they were no longer
valid after the changes, but new tests have been added.
Implements parts of:
- P2418 Add support for std::generator-like types to std::format
Completes:
- LWG3473 Normative encouragement in non-normative note
Depends on D121138
Reviewed By: #libc, vitaut, Mordante
Differential Revision: https://reviews.llvm.org/D121514
The concept is based on P2286R2 Formatting Ranges. It will be used to
optimise the storage of __format_arg_store as required by LWG-3473.
Depends on D120916
Reviewed By: #libc, Mordante
Differential Revision: https://reviews.llvm.org/D120921
Use a specialized "buffer" to count the number of insertions instead of
using a `string` as storage type.
Depends on D110497.
Reviewed By: ldionne, #libc
Differential Revision: https://reviews.llvm.org/D110498
Instead of writing every character directly into the container by using
a `back_insert_iterator` the data is buffered in an `array`. This buffer
is then inserted to the container by calling its `insert` member function.
Since there's no guarantee every container's `insert` behaves properly
containers need to opt-in to this behaviour. The appropriate standard
containers opt-in to this behaviour.
This change improves the performance of the format functions that use a
`back_insert_iterator`.
Depends on D110495
Reviewed By: ldionne, vitaut, #libc
Differential Revision: https://reviews.llvm.org/D110497
This patch changes the requirement for getting the declaration of the
assertion handler from including <__assert> to including any public
C++ header of the library. Note that C compatibility headers are
excluded because we don't implement all the C headers ourselves --
some of them are taken straight from the C library, like assert.h.
It also adds a generated test to check it. Furthermore, this new
generated test is designed in a way that will make it possible to
replace almost all the existing test-generation scripts with this
system in upcoming patches.
Differential Revision: https://reviews.llvm.org/D122506
Instead of using a temporary `string` in `__vformat_to_wrapped` use a new
generic iterator. This aids to reduce the number of template instantions
and avoids using a `string` to buffer the entire formatted output.
This changes the type of `format_context` and `wformat_context`, this can
still be done since the code isn't ABI stable yet.
Several approaches have been evaluated:
- Using a __output_buffer base class with:
- a put function to store the buffer in its internal buffer
- a virtual flush function to copy the internal buffer to the output
- Using a `function` to forward the output operation to the output buffer,
much like the next method.
- Using a type erased function point to store the data in the buffer.
The last version resulted in the best performance. For some cases there's
still a loss of speed over the original method. This loss many becomes
apparent when large strings are copied to a pointer like iterator, before
the compiler optimized this using `memcpy`.
Reviewed By: ldionne, vitaut, #libc
Differential Revision: https://reviews.llvm.org/D110495
All supported compilers that support C++20 now support concepts. So, remove
`_LIB_LIBCPP_HAS_NO_CONCEPTS` in favor of `_LIBCPP_STD_VER > 17`. Similarly in
the tests, remove `// UNSUPPORTED: libcpp-no-concepts`.
Differential Revision: https://reviews.llvm.org/D121528
This adjust the version macro and sets it as completed. All parts of the paper
have been implemented, except for the parts replaced by later papers and
LWG-issues.
Adjusted the synopsis to match the synopsis in the Standard. Not yet
implemented parts of P2216 and P2418 still use the P0645 wording.
Completes:
- P0645 Text Formatting
Depends on D115991
Reviewed By: ldionne, #libc
Differential Revision: https://reviews.llvm.org/D115999
This implements the handler according to P0645. P2418 changes the wording
in the Standard. That isn't implemented and requires changes in more
places. LWG3631 applies modifications to P2418, but is currently
unresolved.
Implements parts of:
* P0645 Text Formatting
Depends on D115989
Reviewed By: ldionne, #libc
Differential Revision: https://reviews.llvm.org/D115991
This implements the last required formatter specialization.
Completes:
- LWG 3251 Are std::format alignment specifiers applied to string arguments?
- LWG 3340 Formatting functions should throw on argument/format string mismatch in §[format.functions]
- LWG 3540 §[format.arg] There should be no const in basic_format_arg(const T* p)
Implements parts of:
- P0645 Text Formatting
Depends on D114001
Reviewed By: ldionne, vitaut, #libc
Differential Revision: https://reviews.llvm.org/D115988
This properly implements the formatter for floating-point types.
Completes:
- P1652R1 Printf corner cases in std::format
- LWG 3250 std::format: # (alternate form) for NaN and inf
- LWG 3243 std::format and negative zeroes
Implements parts of:
- P0645 Text Formatting
Reviewed By: #libc, ldionne, vitaut
Differential Revision: https://reviews.llvm.org/D114001
This removes the `format_args_t` from `<format>` and adjusts the type of
the `format_args` for the `vformat_to` overloads.
The `format_context` uses a `back_insert_iterator<string>` therefore the
new `output_iterator` function uses a `string` as its temporary storage
buffer. This isn't ideal. The next patches in this series will improve
this. These improvements make it easy to also improve `format_to_n` and
`formatted_size`.
This addresses P2216 `6. Binary size`.
P2216 `5. Compile-time checks` are not part of this change.
Implements parts of:
- P2216 std::format improvements
Depends on D103670
Reviewed By: ldionne, #libc
Differential Revision: https://reviews.llvm.org/D110494
