[scan] Fix build_for_testing failure to fail - #29718
Open
alvarhansen wants to merge 1 commit into
Open
Conversation
iBotPeaches
requested changes
Aug 15, 2026
iBotPeaches
left a comment
Member
There was a problem hiding this comment.
Sorry for delay on this. CI is failing and we've moved away from CircleCI. So if you rebase/merge-master and take a look at the failures. We can revisit.
Scan's `ErrorHandler.handle_build_error` has explicit logic that checks for `/Executed/` pattern in logs. It seems to be meant for case where tests will be run with same command and so we would want to wait for test results parsing instead. But this is causing scan to not to fail under `build_for_testing` configuration when there is such pattern in logs (eg. some file name contains it). In order to fix this, I propose to explicitly check for exit status code in `build_for_testing` mode and produce an error if status is not ok. Fixes fastlane#20685
alvarhansen
force-pushed
the
alvar/scan/fix-build-for-testing-failure
branch
from
August 16, 2026 07:38
6d587ff to
18078bf
Compare
Contributor
Author
|
@iBotPeaches i've rebased my PR and all checks are now green. |
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes a scan regression where build_for_testing: true could incorrectly succeed when xcodebuild fails but the build log contains the word Executed (e.g., in a filename), by ensuring the xcodebuild exit status is honored in build_for_testing mode.
Changes:
- Update
Scan::Runner#handle_resultsto raise a build failure on any non-zero xcodebuild exit status whenbuild_for_testingis enabled. - Add a regression spec that reproduces the “
Executedin output prevents failure” scenario and asserts aFastlaneBuildFailureis raised.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| scan/spec/runner_spec.rb | Adds regression coverage for the build_for_testing + “Executed” output scenario. |
| scan/lib/scan/runner.rb | Ensures build_for_testing runs fail on non-zero xcodebuild exit status. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+196
to
+200
| allow(FastlaneCore::CommandExecutor).to receive(:execute) do |command:, print_all:, print_command:, prefix:, loading:, suppress_output:, error:| | ||
| system("ruby -e 'exit 65'") | ||
| error.call(error_output) | ||
| "" | ||
| end |
| end | ||
|
|
||
| describe "when :build_for_testing is true" do | ||
| it "fails the run if xcodebuild exits with a non-zero status even when error output contains 'Executed'", requires_xcodebuild: true do |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Checklist
bundle exec rspecfrom the root directory to see all new and existing tests passbundle exec rubocop -ato ensure the code style is validci/circlecibuilds in the "All checks have passed" section of my PR (connect CircleCI to GitHub if not)Motivation and Context
Resolves #20685
Description
scan's
ErrorHandler.handle_build_errorhas explicit logic that checks for/Executed/pattern in logs. It seems to be meant for case where tests will be run with same command and so we would want to wait for test results parsing instead. But this is causing scan to not to fail underbuild_for_testingconfiguration when there is such pattern in logs (eg. some file name contains it).In order to fix this, I propose to explicitly check for exit status code in
build_for_testingmode and produce an error if status is not ok.Testing Steps
build_for_testing: trueExample project https://github.com/alvar-bolt/fastlane-scan-failure-poc