Skip to content

Fix: Stabilize unit tests#1298

Open
JessTello wants to merge 21 commits into
developmentfrom
unitTest-eosu1037-and-eosu1013
Open

Fix: Stabilize unit tests#1298
JessTello wants to merge 21 commits into
developmentfrom
unitTest-eosu1037-and-eosu1013

Conversation

@JessTello

Copy link
Copy Markdown
Contributor

Fix EOS test crashes and stabilize test setup
Fix config path resolution and copy test config to build
Fix prefabs and performance stress test for missing scripts.
Fix asmdef for correct platforms

OswaldoApexSystems and others added 21 commits April 21, 2026 12:27
-Introduced a new CallbackHelper class to centralize notification callback
lifecycle management and prevent stale callback collisions when the EOS
Platform is recreated.

-Updated EOSManager and FriendTest to correctly integrate with the new
callback lifecycle behavior, ensuring callbacks are cleared at the proper
time during test teardown and platform shutdown.

-These changes fix failures encountered when running the full unit test
suite by ensuring isolation between tests and preventing leftover EOS
notification state from leaking across test runs.
…lback disposal

Uncommented EOS_DO_NOT_UNLOAD_SDK_ON_SHUTDOWN to prevent unloading the EOS SDK in the Unity Editor, in line with the official EOS documentation.

This change resolves the unit test failures by avoiding PlatformInterface.Shutdown() during editor test runs, which was corrupting internal SDK state across multiple test suites.
 The shutdown/unload logic is now Editor‑only, runtime behavior is protected, the hardcoded define was removed, and Windows Editor behavior is aligned with macOS/Linux and Epic documentation.
Fix prefabs and performance stress test for missing scripts.
Fix asmdef for correct platforms
- Replaced Directory.GetCurrentDirectory-based logic with a path derived from Application.dataPathto correctly resolve the project root in both Editor and Player.
- Updated TestPlayerModeSetup to persist the build output directory and copy the test config file into the player build.
- Build no longer fails if the config file is missing; a warning is logged instead.
- Prevent multiple EOS initializations when running full test suite
- Make EOS static to ensure single initialization per session
- Avoid teardown issues caused by test-specific EOSManager instances
- Ignore tests that require a second machine instead of crashing
- Add null check in lobby and session cleanup to prevent NPE
@JessTello JessTello requested a review from a team June 11, 2026 21:40
"eos_automated_test_config.json",
FileSystemUtility.CombinePaths(
FileSystemUtility.GetProjectPath(),
Path.GetFullPath(Path.Combine(Application.dataPath, "..")),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be functionally equivalent shouldn't it? The GetProjectPath function will find the Assets folder and step up one directory too

{
Assert.Ignore(
"This test requires a second machine running as a server. " +
"Run only the 'Client' category when a server player is available.");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you run the 'Client' category this test will still be ignored though right?

I'm not sure of an ideal resolution right now, but this seems passable for now:

  • Add a new bool to the test config ("RunLobbyTestsThatRequireHost")
  • Update this code to check that and use Assert.Ignore if it's not enabled
  • Update the Assert.Ignore to refer to using both the category and the option in the test config

{
Assert.Ignore(
"This test requires a second machine running as a server. " +
"Run only the 'Client' category when a server player is available.");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you run the 'Client' category this test will still be ignored though right?

I'm not sure of an ideal resolution right now, but this seems passable for now:

  • Add a new bool to the test config ("RunLobbyTestsThatRequireHost")
  • Update this code to check that and use Assert.Ignore if it's not enabled
  • Update the Assert.Ignore to refer to using both the category and the option in the test config


// This define controls if the EOS SDK should be unloaded in the editor at shutdown to work around DLL unload errors.
//#define EOS_DO_NOT_UNLOAD_SDK_ON_SHUTDOWN

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've copied the pending questions from the original PR here. @OswaldoApexSystems can you respond to these?

string resolvedPath = Path.GetFullPath(playerOptions.locationPathName);
s_TestBuildDirectory = Path.HasExtension(resolvedPath)
? Path.GetDirectoryName(resolvedPath)
: resolvedPath;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FileSystemUtility seems designed to wrap Path and I assume is recommended for consistency.
There's quite a bit of this in this file

/// </summary>
/// <returns></returns>
[UnityTest]
[Category(TestCategories.SoloCategory)]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not suggesting we change this here, but I'm thinking we should get rid of this unless you know why it exists. It isn't going to be attached to every single test that isn't a "Client" one, so it'll result in people not running all the tests they expect. Perhaps there are also some tests that are neither "Solo" or "Client" though, but it would likely make sense to just tag those separately

/// <returns></returns>
[UnityTest]
[Category(TestCategories.SoloCategory)]
public IEnumerator ConnectLogin_WhileAlreadyLoggedIn_ReturnsExpectedResult()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you update the function name to include "DevAuthTool"?

/// <returns></returns>
[UnityTest]
[Category(TestCategories.SoloCategory)]
public IEnumerator AuthLogin_WhileAlreadyLoggedIn_ReturnsExpectedResult()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you update the function name to include "DevAuthTool"?

#if UNITY_EDITOR && (!UNITY_EDITOR_WIN || !EOS_UNLOAD_SDK_ON_SHUTDOWN_IN_WIN_EDITOR)
// This define controls if the EOS SDK should be unloaded in the editor at shutdown to work around DLL unload errors.
#define EOS_DO_NOT_UNLOAD_SDK_ON_SHUTDOWN_IN_EDITOR
#endif

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This docs file covers a couple of points which are a bit of a concern:

  • "It is not the default because if it were, then any edits to the EOSConfig file would require a reboot of the Unity Editor"
    • Can you check if this is accurate?
    • If it is then we would need to think of a solution to this, perhaps when you save the EOSConfig it prompts you to restart the Editor (after checking the new define mentioned in the comment below)
    • I suspect we also need a Unity hook to check if the file was modified outside of Unity (user could've edited it by hand, or pulled latest changes from their repo)
  • "In the past, not unloading the EOS SDK could also cause issues with state from the previous Unity run not being properly cleaned up."
    • This is the opposite of what we're seeing right? I'm wary that there's some edge case issues that we haven't seen with this initial testing. It could definitely be a misunderstanding though
    • The commit message where it was added also says the same, so it doesn't appear to just be a mistake in the docs (It also carries with it the potential for "leaked" EOS State to carry over between play in editor runs.)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Jess' reply from the original PR (link):

I agree the current wording in c_sharp_defines.md is too absolute and also a bit outdated relative to the current config model. The real concern here is not simply that the config file changes, but that EOSManager now conditionally skips full SDK shutdown/unload in the Editor, which means some initialization related SDK state may persist across play sessions. Because of that, I don’t think we should claim that any config edit requires an Editor restart, a more accurate statement is that config changes which affect EOS SDK initialization may require restarting the Editor when the SDK is not fully unloaded. I also agree that the doc should stop referring to EOSConfig here and instead reflect the current ProductConfig / PlatformConfig flow. If we want better UX around this later, we can consider a targeted restart prompt or external change detection as a follow up, but I think the immediate fix is to make the documentation more precise and aligned with the current EOSManager behavior.

// This define controls if the EOS SDK should be unloaded in the editor at shutdown to work around DLL unload errors.
#define EOS_DO_NOT_UNLOAD_SDK_ON_SHUTDOWN_IN_EDITOR
#endif

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With this change to always define EOS_DO_NOT_UNLOAD_SDK_ON_SHUTDOWN in Editor this means that this docs file is out of date.

The change above also removes the ability for developers to control whether it's enabled in Editor or not for Windows.

One option would be to change the above code to this, and update the docs to refer to this define instead:

#if UNITY_EDITOR && (!UNITY_EDITOR_WIN || !EOS_UNLOAD_SDK_ON_SHUTDOWN)
#define EOS_DO_NOT_UNLOAD_SDK_ON_SHUTDOWN
#endif

UnityEngine.Object.DontDestroyOnLoad(s_EOSObject);
var eosManager = s_EOSObject.AddComponent<EOSManager>();
EOSManager.Instance.Init(eosManager);
s_EOSSessionActive = true;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seem like s_EOSSessionActive is never set to false during the test suite and also s_EOSObject is never null after being initialised. They are also private, so no class can change them.
What do we need the s_EOSSessionActive variable for?

EOSManager.Instance?.OnShutdown();
UnityEngine.Object.Destroy(eosObject);
}
public void ShutdownEOS() { }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we still be calling EOSManager.Instance?.OnShutdown() once all tests finish?

s_EOSObject = new GameObject("EOSManager");
UnityEngine.Object.DontDestroyOnLoad(s_EOSObject);
var eosManager = s_EOSObject.AddComponent<EOSManager>();
EOSManager.Instance.Init(eosManager);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This calls EOSManager.Instance.Init and a comment in the restricted PR says not to because it's already called automatically. Awake does call Init, so it seems that the restricted version is correct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants