bpo-38965: Fix faulthandler._stack_overflow() on GCC 10 #17467

vstinner · 2019-12-04T16:11:14Z

Use the "volatile" keyword to prevent tail call optimization
on any compiler, rather than relying on compiler specific pragma.

https://bugs.python.org/issue38965


        bop-38965: Fix faulthandler._stack_overflow() on GCC 10

Use the "volatile" keyword to prevent tail call optimization on any compiler, rather than relying on compiler specific pragma.

miss-islington · 2019-12-04T20:10:10Z

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.7, 3.8.
🐍🍒⛏🤖

miss-islington · 2019-12-04T20:10:11Z

I'm having trouble backporting to 3.8. Reason: 'Error 110 while writing to socket. Connection timed out.'. Please retry by removing and re-adding the needs backport to 3.8 label.

bedevere-bot · 2019-12-04T20:10:28Z

GH-17468 is a backport of this pull request to the 3.7 branch.


        bpo-38965: Fix faulthandler._stack_overflow() on GCC 10 (pythonGH-17467)

Use the "volatile" keyword to prevent tail call optimization on any compiler, rather than relying on compiler specific pragma. (cherry picked from commit 8b78796) Co-authored-by: Victor Stinner <[email protected]>

vstinner · 2019-12-04T20:10:58Z

@marxin tested the fix for me with GCC 10 and confirmed that my change fix the issue:
#17462 (comment)

So I merged my change.

miss-islington · 2019-12-04T20:11:45Z

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖


        bpo-38965: Fix faulthandler._stack_overflow() on GCC 10 (pythonGH-17467)

Use the "volatile" keyword to prevent tail call optimization on any compiler, rather than relying on compiler specific pragma. (cherry picked from commit 8b78796) Co-authored-by: Victor Stinner <[email protected]>

bedevere-bot · 2019-12-04T20:11:55Z

GH-17469 is a backport of this pull request to the 3.8 branch.

bedevere-bot · 2019-12-04T20:25:52Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Windows7 SP1 3.x has failed when building commit 8b78796.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/81/builds/16) and take a look at the build logs.
Check if the failure is related to this commit (8b78796) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/81/builds/16

Failed tests:

test_asyncio

Failed subtests:

test_pipe_handle - test.test_asyncio.test_windows_utils.PipeTests
test_overwrite_existing - test.test_venv.BasicTest
test_huge_content_recvinto - test.test_asyncio.test_sock_lowlevel.SelectEventLoopTests

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

386 tests OK.

10 slowest tests:

test_multiprocessing_spawn: 2 min 42 sec
test_unparse: 1 min 57 sec
test_tokenize: 1 min 47 sec
test_mmap: 1 min 44 sec
test_concurrent_futures: 1 min 35 sec
test_io: 1 min 23 sec
test_lib2to3: 1 min 22 sec
test_asyncio: 1 min 20 sec
test_capi: 1 min 16 sec
test_largefile: 1 min 12 sec

1 test failed:
test_asyncio

33 tests skipped:
test_curses test_dbm_gnu test_dbm_ndbm test_devpoll test_epoll
test_fcntl test_fork1 test_gdb test_grp test_ioctl test_kqueue
test_multiprocessing_fork test_multiprocessing_forkserver test_nis
test_openpty test_ossaudiodev test_pipes test_poll test_posix
test_pty test_pwd test_readline test_resource test_spwd
test_syslog test_threadsignals test_tix test_tk test_ttk_guionly
test_wait3 test_wait4 test_xxtestfuzz test_zipfile64

2 re-run tests:
test_asyncio test_venv

Total duration: 13 min 1 sec

Click to see traceback logs

Traceback (most recent call last):
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\support\__init__.py", line 380, in _force_run
    return func(*args)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\Buildbot\\AppData\\Local\\Temp\\tmp9y0m6j7z\\Scripts\\activate.bat'


Traceback (most recent call last):
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\asyncio\windows_events.py", line 66, in _cancel_overlapped
    self._ov.cancel()
OSError: [WinError 6] The handle is invalid
k


Traceback (most recent call last):
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_asyncio\test_sock_lowlevel.py", line 225, in test_huge_content_recvinto
    self.loop.run_until_complete(
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\asyncio\base_events.py", line 634, in run_until_complete
    return future.result()
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_asyncio\test_sock_lowlevel.py", line 211, in _basetest_huge_content_recvinto
    nbytes = await self.loop.sock_recv_into(sock, buf)
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\asyncio\selector_events.py", line 400, in sock_recv_into
    return await fut
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\asyncio\selector_events.py", line 409, in _sock_recv_into
    nbytes = sock.recv_into(buf)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host


Traceback (most recent call last):
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\asyncio\windows_events.py", line 66, in _cancel_overlapped
    self._ov.cancel()
OSError: [WinError 6] The handle is invalid


Traceback (most recent call last):
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_asyncio\test_windows_utils.py", line 78, in test_pipe_handle
    raise RuntimeError('expected ERROR_INVALID_HANDLE')
RuntimeError: expected ERROR_INVALID_HANDLE


Traceback (most recent call last):
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\asyncio\windows_events.py", line 453, in finish_recv
    return ov.getresult()
OSError: [WinError 995] The I/O operation has been aborted because of either a thread exit or an application request


Traceback (most recent call last):
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_venv.py", line 71, in tearDown
    rmtree(self.env_dir)
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\support\__init__.py", line 502, in rmtree
    _rmtree(path)
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\support\__init__.py", line 443, in _rmtree
    _waitfor(_rmtree_inner, path, waitall=True)
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\support\__init__.py", line 391, in _waitfor
    func(pathname)
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\support\__init__.py", line 439, in _rmtree_inner
    _waitfor(_rmtree_inner, fullname, waitall=True)
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\support\__init__.py", line 391, in _waitfor
    func(pathname)
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\support\__init__.py", line 442, in _rmtree_inner
    _force_run(fullname, os.unlink, fullname)
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\support\__init__.py", line 385, in _force_run
    os.chmod(path, stat.S_IRWXU)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\Buildbot\\AppData\\Local\\Temp\\tmp9y0m6j7z\\Scripts\\activate.bat'


Traceback (most recent call last):
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\asyncio\proactor_events.py", line 768, in _loop_self_reading
    f.result()  # may raise
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\asyncio\windows_events.py", line 808, in _poll
    value = callback(transferred, key, ov)
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\asyncio\windows_events.py", line 457, in finish_recv
    raise ConnectionResetError(*exc.args)
ConnectionResetError: [WinError 995] The I/O operation has been aborted because of either a thread exit or an application request
k


Traceback (most recent call last):
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\asyncio\proactor_events.py", line 768, in _loop_self_reading
    f.result()  # may raise
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\asyncio\windows_events.py", line 808, in _poll
    value = callback(transferred, key, ov)
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\asyncio\windows_events.py", line 457, in finish_recv
    raise ConnectionResetError(*exc.args)
ConnectionResetError: [WinError 995] The I/O operation has been aborted because of either a thread exit or an application request


Traceback (most recent call last):
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_venv.py", line 225, in test_overwrite_existing
    builder.create(self.env_dir)
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\venv\__init__.py", line 65, in create
    context = self.ensure_directories(env_dir)
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\venv\__init__.py", line 108, in ensure_directories
    self.clear_directory(env_dir)
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\venv\__init__.py", line 91, in clear_directory
    shutil.rmtree(fn)
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\shutil.py", line 731, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\shutil.py", line 613, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\shutil.py", line 611, in _rmtree_unsafe
    os.rmdir(path)
OSError: [WinError 145] The directory is not empty: 'C:\\Users\\Buildbot\\AppData\\Local\\Temp\\tmp9y0m6j7z\\Scripts'


        bpo-38965: Fix faulthandler._stack_overflow() on GCC 10 (GH-17467)

Use the "volatile" keyword to prevent tail call optimization on any compiler, rather than relying on compiler specific pragma. (cherry picked from commit 8b78796) Co-authored-by: Victor Stinner <[email protected]>


        bpo-38965: Fix faulthandler._stack_overflow() on GCC 10 (GH-17467)

Use the "volatile" keyword to prevent tail call optimization on any compiler, rather than relying on compiler specific pragma. (cherry picked from commit 8b78796) Co-authored-by: Victor Stinner <[email protected]>

bedevere-bot · 2019-12-04T20:46:13Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL7 3.7 has failed when building commit 5044c88.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/31/builds/5) and take a look at the build logs.
Check if the failure is related to this commit (5044c88) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/31/builds/5

Failed tests:

test_faulthandler

Failed subtests:

test_dump_traceback_later_file - test.test_faulthandler.FaultHandlerTests

Summary of the results of the build (if available):

== Tests result: FAILURE then ENV CHANGED ==

401 tests OK.

10 slowest tests:

test_concurrent_futures: 2 min 30 sec
test_multiprocessing_spawn: 2 min 22 sec
test_mailbox: 1 min 37 sec
test_multiprocessing_forkserver: 1 min 36 sec
test_tools: 1 min 34 sec
test_lib2to3: 1 min 30 sec
test_tokenize: 1 min 29 sec
test_multiprocessing_fork: 1 min 13 sec
test_asyncio: 1 min 9 sec
test_io: 1 min 6 sec

1 test altered the execution environment:
test_multiprocessing_spawn

14 tests skipped:
test_devpoll test_gdb test_ioctl test_kqueue test_msilib
test_ossaudiodev test_startfile test_tix test_tk test_ttk_guionly
test_winconsoleio test_winreg test_winsound test_zipfile64

1 re-run test:
test_faulthandler

Total duration: 4 min 49 sec

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.7.cstratak-RHEL7-x86_64/build/Lib/test/test_faulthandler.py", line 610, in test_dump_traceback_later_file
    self.check_dump_traceback_later(filename=filename)
  File "/home/buildbot/buildarea/3.7.cstratak-RHEL7-x86_64/build/Lib/test/test_faulthandler.py", line 594, in check_dump_traceback_later
    self.assertRegex(trace, regex)
AssertionError: Regex didn't match: '^Timeout \\(0:00:00.500000\\)!\\nThread 0x[0-9a-f]+ \\(most recent call first\\):\\n  File "<string>", line 17 in func\n  File "<string>", line 26 in <module>$' not found in 'Timeout (0:00:00.500000)!\nThread 0x00007f73ca2d4740 (most recent call first):\n  File "<string>", line 18 in func\n  File "<string>", line 26 in <module>'


        Merge pull request #71 from python/master

bpo-38965: Fix faulthandler._stack_overflow() on GCC 10 (pythonGH-17467)


        bpo-38965: Fix faulthandler._stack_overflow() on GCC 10 (pythonGH-17467)

Use the "volatile" keyword to prevent tail call optimization on any compiler, rather than relying on compiler specific pragma.


        00338: bpo-38965: Fix faulthandler._stack_overflow() on GCC 10 (pytho…

…nGH-17467) Use the "volatile" keyword to prevent tail call optimization on any compiler, rather than relying on compiler specific pragma. (cherry picked from commit 8b78796) Co-authored-by: Victor Stinner <[email protected]>

bop-38965: Fix faulthandler._stack_overflow() on GCC 10

Loading status checks…

6902d83

Use the "volatile" keyword to prevent tail call optimization on any compiler, rather than relying on compiler specific pragma.

the-knights-who-say-ni added the CLA signed label Dec 4, 2019

bedevere-bot added the awaiting core review label Dec 4, 2019

vstinner mentioned this pull request Dec 4, 2019

bpo-38965: Fix stuck in test_stack_overflow tests. #17462

Closed

vstinner changed the title ~~bop-38965: Fix faulthandler._stack_overflow() on GCC 10~~ bpo-38965: Fix faulthandler._stack_overflow() on GCC 10 Dec 4, 2019

vstinner added needs backport to 3.7 needs backport to 3.8 labels Dec 4, 2019

vstinner deleted the vstinner:tail_call branch Dec 4, 2019

bedevere-bot removed the awaiting core review label Dec 4, 2019

bedevere-bot removed the needs backport to 3.7 label Dec 4, 2019

vstinner added needs backport to 3.8 and removed needs backport to 3.8 labels Dec 4, 2019

bedevere-bot removed the needs backport to 3.8 label Dec 4, 2019

python/cpython

bpo-38965: Fix faulthandler._stack_overflow() on GCC 10 #17467

bpo-38965: Fix faulthandler._stack_overflow() on GCC 10 #17467

vstinner commented Dec 4, 2019 •

edited by bedevere-bot

This comment has been minimized.

miss-islington commented Dec 4, 2019

This comment has been minimized.

miss-islington commented Dec 4, 2019

This comment has been minimized.

bedevere-bot commented Dec 4, 2019

This comment has been minimized.

vstinner commented Dec 4, 2019

This comment has been minimized.

miss-islington commented Dec 4, 2019

This comment has been minimized.

bedevere-bot commented Dec 4, 2019

This comment has been minimized.

bedevere-bot commented Dec 4, 2019

This comment has been minimized.

bedevere-bot commented Dec 4, 2019

Sponsor python/cpython

python/cpython

Join GitHub today

bpo-38965: Fix faulthandler._stack_overflow() on GCC 10 #17467

bpo-38965: Fix faulthandler._stack_overflow() on GCC 10 #17467

Conversation

vstinner commented Dec 4, 2019 • edited by bedevere-bot

This comment has been minimized.

miss-islington commented Dec 4, 2019

This comment has been minimized.

miss-islington commented Dec 4, 2019

This comment has been minimized.

bedevere-bot commented Dec 4, 2019

This comment has been minimized.

vstinner commented Dec 4, 2019

This comment has been minimized.

miss-islington commented Dec 4, 2019

This comment has been minimized.

bedevere-bot commented Dec 4, 2019

This comment has been minimized.

bedevere-bot commented Dec 4, 2019

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

This comment has been minimized.

bedevere-bot commented Dec 4, 2019

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

vstinner commented Dec 4, 2019 •

edited by bedevere-bot