Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-43031: Set a timeout when running tests in PGO build #24339

Merged
merged 1 commit into from Jan 27, 2021

Conversation

@vstinner
Copy link
Member

@vstinner vstinner commented Jan 26, 2021

Pass --timeout=$(TESTTIMEOUT) option to the default profile task
"./python -m test --pgo" command.

https://bugs.python.org/issue43031

Pass --timeout=$(TESTTIMEOUT) option to the default profile task
"./python -m test --pgo" command.
@vstinner vstinner force-pushed the vstinner:pgo_timeout branch from eb3c83c to df0ef05 Jan 26, 2021
@vstinner vstinner changed the title bpo-42988: Set a timeout when running tests in PGO build bpo-43031: Set a timeout when running tests in PGO build Jan 26, 2021
Copy link
Member Author

@vstinner vstinner left a comment

cc @methane @pablogsal @zware: Do you think that 20 minutes per test is enough for everyone?

If someone has issues with the new timeout, TESTTIMEOUT can be overriden when running make: make TESTTIMEOUT=3600 (1 hour).

In regrtest --pgo, the slowest test should be asyncio, and it takes around 5 minutes.

If 20 min is an issue, maybe we can increase the default to 30 minutes?


It's the first time that I see a hang while on a PGO build. So another option is to consider that no timeout is better by default (reject this PR).

@pablogsal
Copy link
Member

@pablogsal pablogsal commented Jan 26, 2021

cc @methane @pablogsal @zware: Do you think that 20 minutes per test is enough for everyone?

If someone has issues with the new timeout, TESTTIMEOUT can be overriden when running make: make TESTTIMEOUT=3600 (1 hour).

In regrtest --pgo, the slowest test should be asyncio, and it takes around 5 minutes.

If 20 min is an issue, maybe we can increase the default to 30 minutes?


It's the first time that I see a hang while on a PGO build. So another option is to consider that no timeout is better by default (reject this PR).

I would be fine with that but we should certainly do something about test_asyncio

@vstinner
Copy link
Member Author

@vstinner vstinner commented Jan 26, 2021

I would be fine with that but we should certainly do something about test_asyncio

I don't think that we should do anything for "aarch64 RHEL7 LTO + PGO 3.7", since 3.7 branch no longer accept bug fixes.

@stratakis
Copy link
Contributor

@stratakis stratakis commented Jan 26, 2021

I'm thinking of removing that config from the aarch64 RHEL7 buildbots, as it's the only one that disconnects during a PGO build when test_asyncio runs.

@vstinner
Copy link
Member Author

@vstinner vstinner commented Jan 26, 2021

I'm thinking of removing that config from the aarch64 RHEL7 buildbots, as it's the only one that disconnects during a PGO build when test_asyncio runs.

I created python/buildmaster-config#228 for that.

@methane
Copy link
Member

@methane methane commented Jan 26, 2021

Do you think that 20 minutes per test is enough for everyone?

I think so. And I think tests takes longer than 20min should not be run by default.

@vstinner vstinner merged commit 6790005 into python:master Jan 27, 2021
11 checks passed
11 checks passed
Docs
Details
Check for source changes
Details
Check if generated files are up to date
Details
Windows (x86)
Details
Windows (x64)
Details
macOS
Details
Ubuntu
Details
Azure Pipelines PR #20210126.12 succeeded
Details
Travis CI - Pull Request Build Passed
Details
bedevere/issue-number Issue number 43031 found
Details
bedevere/news News entry found in Misc/NEWS.d
@vstinner vstinner deleted the vstinner:pgo_timeout branch Jan 27, 2021
@vstinner
Copy link
Member Author

@vstinner vstinner commented Jan 27, 2021

The timeout can cause some issues, so I prefer to not backport it. https://bugs.python.org/issue43031 is not a major issue, it only impacted Python 3.7. I added the timeout in Python 3.10, it is enough for now.

Thanks for your review @pablogsal and @methane.

m000 added a commit to m000/cpython that referenced this pull request Jan 29, 2021
)

Pass --timeout=$(TESTTIMEOUT) option to the default profile task
"./python -m test --pgo" command.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

6 participants