Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Latest commit

 

History

History
History
24 lines (21 loc) · 838 Bytes

File metadata and controls

24 lines (21 loc) · 838 Bytes
Copy raw file
Download raw file
Open symbols panel
Edit and raw actions
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# test formatting floats with large precision, that it doesn't overflow the buffer
def test(num, num_str):
if num == float('inf') or num == 0.0 and num_str != '0.0':
# skip numbers that overflow or underflow the FP precision
return
for kind in ('e', 'f', 'g'):
# check precision either side of the size of the buffer (32 bytes)
for prec in range(23, 36, 2):
fmt = '%.' + '%d' % prec + kind
s = fmt % num
check = abs(float(s) - num)
if num > 1:
check /= num
if check > 1e-6:
print('FAIL', num_str, fmt, s, len(s), check)
# check pure zero
test(0.0, '0.0')
# check most powers of 10, making sure to include exponents with 3 digits
for e in range(-101, 102):
num = pow(10, e)
test(num, '1e%d' % e)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.