28 Test flow

Hi folks! Today a more technical blog with a peek into my game production and test flow, with DevOps yml code included!

But first, a gameplay fragment of Gearful. This is somehow connected to ‘Roan’, my new helper!

Gearful

Level 7 of Gearful, played on itch.io. Play it yourself.

This level is created by Roan in March of 2020. He really liked to fantasize about Gearful while I was working on it. So he made a drawing for me, and I recreated it.

Testing by Roan

Last week Roan joined in on the game development team at GameFeelings. With his 9 years old he has a unique look at games. His job will mainly be to test the latest versions and hold me accountable for delivering on my promises.

He is a passionate player of games. He really likes to play games, and when he is not playing games he likes to draw scenes of his favorite games and do some modifications to the gameplay. And then play these out on paper.

Previous week he held a presentation in front of his class about the different types of games available. At the end his classmates could try out a game he created one level for. That is ‘Gearful’, level 7. That is why this bi-weekly shows off level 7 of Gearful in the introduction.

He already does play my games, out of his own curiosity and interest in games. That was really funny for me to discover. Over the past year he played my games a couple of times already, each time getting back to me and talking about how he enjoyed it. On Gearful a few levels are difficult to complete due to Unity Physics not working properly, but he manages to complete them anyway by exploiting the mechanics. And he really likes to tell me this. He does play (the old version of) Find the Gnome too, but less often. Because this game is a bit less appealing to him due to a few quirks in the gameplay. But he can already point those problems out to me, that’s quite remarkable for a 9 years old I think.

So yeah, I think he is a really valuable addition and will increase the quality of the games I deliver.

Testing setup

For those interested in how I do the test part of the production in my game development and the tooling I use, here a peek. If you are interested in more, look me up on my Discord and ask details there.

For me as a solo dev, I build the workflow around my preferred way of working. The testing workflow I am going to describe here is intended more as a signalling function then as an approvement flow. ‘Proper’ testing is still something I need to do myself before checking in my code. That is because in the flow show here there is no coupling between the builds and the stories/bugfixes that are solved in them.

Having said that, lets move on to some actual workflow. So when I check in my code on Azure DevOps the build starts.

I run Unity 2019.4 LTS and have my code on Azure DevOps. There i have a build agent connected to my local dev machine so the builds run locally. This is an evolution of the setup I did in this blog and on Youtube.

I run full yml now, so this is how that looks:

name: 2.0$(rev:.r)

# no PR triggers
pr: none

pool:
  name: Default
  demands: Unity_2019.4

variables:
  gameProjectDir: 'Find the Gnome Revisited'
  subscriptionConnector: 'redacted'


stages:
- stage: WindowsBuild
  displayName: 'Windows build'
  jobs:
  - job: Prepare
    displayName: 'Build FtG for windows'
    steps:
    #cleanup for unity build
    - task: DeleteFiles@1
      displayName: 'Cleanup staging dir leftovers from last build'
      inputs:
        SourceFolder: '$(Build.ArtifactStagingDirectory)'
        Contents: '**/*'
        
    #cleanup for published artifact
    - task: DeleteFiles@1
      displayName: 'Cleanup published artifact target dir'
      inputs:
        SourceFolder: '$(Pipeline.Workspace)\GameBuildPcWin\'
        Contents: '**/*'

    #get the previously secured library (if any) for performance reasons
    - task: DeleteFiles@1
      displayName: 'Delete old libary content'
      inputs:
        SourceFolder: '$(Build.Repository.LocalPath)\$(gameProjectDir)\Library'
        Contents: '**/*'
        
    - task: CmdLine@2
      displayName: 'Restore Unity library from last build for performance reasons'
      inputs:
        script: 'if EXIST "$(Build.BinariesDirectory)\Library\" move "$(Build.BinariesDirectory)\Library" "$(Build.Repository.LocalPath)\$(gameProjectDir)\Library"'
  
    
    # Build -------------------------
    
    #unity PC build
    - task: UnityBuildTask@3
      displayName: 'Unity build'
      inputs:
        buildTarget: 'Win64'
        unityProjectPath: '$(Build.Repository.LocalPath)\$(gameProjectDir)'
        buildScriptType: 'existing'
        scriptExecuteMethod: 'ImprovedBuild.PerformBuild'
        additionalCmdArgs: '-outputPath "$(Build.ArtifactStagingDirectory)\WindowsBuild" -outputFileName "FindTheGnomeRevisited" -buildVersion "$(Build.BuildNumber)" -versionJsonFileName "Assets\Resources\ProjectVersion.json"'
    
    #upload normal game build artifact
    - task: PublishBuildArtifacts@1
      displayName: 'Upload clean build'
      inputs:
        PathtoPublish: '$(Build.ArtifactStagingDirectory)\WindowsBuild'
        ArtifactName: 'GameBuildPcWin'
    
    # Cleanup -------------------------
    
    #secure the library for performance reasons
    - task: DeleteFiles@1
      displayName: 'Clean the temp store library'
      inputs:
        SourceFolder: '$(Build.BinariesDirectory)\Library'
        Contents: '**/*'
        
    - task: CmdLine@2
      displayName: 'Move the files over to be saved for the next build'
      inputs:
        script: 'move "$(Build.Repository.LocalPath)\$(gameProjectDir)\Library" "$(Build.BinariesDirectory)\Library"'

    # upload the setup creator so we don't have to download the whole code base next time we want to build the game installer
    # can be uploaded in a separate concurrent job
  - job: UploadSetup
    displayName: 'Upload windows setup creator'
    steps:
    - task: PublishBuildArtifacts@1
      displayName: 'Upload setup creator'
      inputs:
        pathtoPublish: '"$(Build.Repository.LocalPath)\Installer'
        artifactName: 'SetupCreator'


- stage: CreateQAInstaller
  displayName: 'Create installer for QA'
  jobs:
  - job: BuildInstaller
    displayName: 'Build installer for windows'
    steps:
    - checkout: none

    - download: current
      artifact: 'GameBuildPcWin'
      displayName: 'Download GameBuildPcWin Artifact'

    - download: current
      artifact: 'SetupCreator'
      displayName: 'Download SetupCreator Artifact'

    #create the installer
    - task: BatchScript@1
      displayName: 'Create installer'
      inputs:
        filename: '$(Pipeline.Workspace)/SetupCreator/createsetup.bat'
        arguments: '"$(Pipeline.Workspace)\GameBuildPcWin\*" "$(Build.ArtifactStagingDirectory)\Installer" "$(Build.BuildNumber)"'
        workingFolder: '$(Pipeline.Workspace)\SetupCreator'

    #upload game installer artifact
    - task: PublishBuildArtifacts@1
      displayName: 'Upload installer'
      inputs:
        PathtoPublish: '$(Build.ArtifactStagingDirectory)\Installer'
        ArtifactName: 'GameInstallerPcWin'


- stage: Upload
  displayName: 'Upload to stores'
  jobs:
  - job: QAInstaller
    displayName: 'Installers for QA (early builds)'
    steps:
    - checkout: none

    - download: current
      artifact: 'GameInstallerPcWin'
      displayName: 'Download GameInstallerPcWin Artifact'

    - task: AzureFileCopy@4
      displayName: 'Upload PcWin early build installer for QA to blob store https://redacted.exe'
      inputs:
        SourcePath: '$(Pipeline.Workspace)\GameInstallerPcWin\SetupPc.exe'
        azureSubscription: '$(subscriptionConnector)'
        Destination: 'AzureBlob'
        storage: 'redacted'
        ContainerName: 'earlybuild'

It still needs evolving. 1) I want to get it to build for and release to Steam too in the same script. 2) And I want to introduce 2 different game build types: one for early testing with all kinds of bells and whizzles and one for user release without test tools and shortcuts, probably by using different scriptExecuteMethod depending on my expected build output. 3) And the generated installer isn’t signed, so the browser is very picky on it currently and you have to jump a few big warning hoops to get the game installed this way.

Roan as a tester then has the following workflow:
He has a document on his desktop with a few links in them. First he uses the installer link to download & update the game to the latest version. Then he opens a google form with questions on the game, and answers those.

I then use the form excel document to create new tasks in DevOps. To do this efficiently I track the response and the reports I have processed.

One thing noteworthy is the tracking of the version number. To get a proper connection between reports and fixes, I had to somehow get an automatic build number to display in-game. This was not as easy as it sounds, because Unity hasn’t that great of a support for a build in versioning. TLDR: insert it at build time through a custom unity build script into a json file as a native Unity resource asset and read that asset when running the game.

The custom build script for the versioning part:

using System;
using System.IO;
using UnityEditor;
using UnityEngine;

#if UNITY_2018_1_OR_NEWER
using UnityEditor.Build.Reporting;
#endif

/// <summary>
/// Improved build script over GenericBuild.
/// -Use build version
/// </summary>
public class ImprovedBuild
{
    // Build inputs
    private static string outputFileNameArgName = "outputFileName";
    private static string locationPathNameArgName = "outputPath";

    // Version inputs
    private static string buildVersionArgName = "buildVersion";
    private static string versionJsonFileNameArgName = "versionJsonFileName";

    public static void PerformBuild()
    {
        try
        {
            // Preset versions
            SetVersions();

            // Prepare build
            EditorBuildSettingsScene[] editorConfiguredBuildScenes = EditorBuildSettings.scenes;
            string[] includedScenes = new string[editorConfiguredBuildScenes.Length];

            for (int i = 0; i < editorConfiguredBuildScenes.Length; i++)
            {
                includedScenes[i] = editorConfiguredBuildScenes[i].path;
            }

#if UNITY_2018_1_OR_NEWER
            BuildReport buildReport = default(BuildReport);
#else
                    string buildReport = "ERROR";
#endif

            var useLocationPathName = Path.Combine(FindArg(locationPathNameArgName), GetBuildTargetOutputFileNameAndExtension());
            Debug.Log("Using locationPathName: "+ useLocationPathName);  // Backward compatible with earlier unity editor C# versions


            // Build
            buildReport = BuildPipeline.BuildPlayer(new BuildPlayerOptions
            {
                scenes = includedScenes,
                target = EditorUserBuildSettings.activeBuildTarget,
                locationPathName = useLocationPathName,
                targetGroup = EditorUserBuildSettings.selectedBuildTargetGroup,
                options = BuildOptions.None
            });

            // Log build results

#if UNITY_2018_1_OR_NEWER
            switch (buildReport.summary.result)
            {
                case BuildResult.Succeeded:
                    EditorApplication.Exit(0);
                    break;
                case BuildResult.Unknown:
                case BuildResult.Failed:
                case BuildResult.Cancelled:
                default:
                    EditorApplication.Exit(1);
                    break;
            }
#else
                    if (buildReport.StartsWith("Error"))
                    {
                        EditorApplication.Exit(1);
                    }
                    else
                    {
                        EditorApplication.Exit(0);
                    }
#endif
        }
        catch (Exception ex)
        {
            Debug.Log("BUILD FAILED: " + ex.Message);
            EditorApplication.Exit(1);
        }
    }

    private static string GetBuildTargetOutputFileNameAndExtension()
    {
        var outputFileName = FindArg(outputFileNameArgName);
        switch (EditorUserBuildSettings.activeBuildTarget)
        {
            case BuildTarget.Android:
                return (outputFileName + ".apk");
            case BuildTarget.StandaloneWindows64:
            case BuildTarget.StandaloneWindows:
                return (outputFileName + ".exe");
#if UNITY_2018_1_OR_NEWER
            case BuildTarget.StandaloneOSX:
#endif
#if !UNITY_2017_3_OR_NEWER
                    case BuildTarget.StandaloneOSXIntel:
                    case BuildTarget.StandaloneOSXIntel64:
#endif
                return (outputFileName + ".app");
            case BuildTarget.iOS:
            case BuildTarget.tvOS:
#if !UNITY_2019_2_OR_NEWER
            case BuildTarget.StandaloneLinux:
#endif
            case BuildTarget.WebGL:
            case BuildTarget.WSAPlayer:
            case BuildTarget.StandaloneLinux64:
#if !UNITY_2019_2_OR_NEWER
            case BuildTarget.StandaloneLinuxUniversal:
#endif
#if !UNITY_2018_3_OR_NEWER
                    case BuildTarget.PSP2:    
#endif
            case BuildTarget.PS4:
            case BuildTarget.XboxOne:
#if !UNITY_2017_3_OR_NEWER
                    case BuildTarget.SamsungTV:
#endif
#if !UNITY_2018_1_OR_NEWER
                    case BuildTarget.N3DS:
                    case BuildTarget.WiiU:
#endif
            case BuildTarget.Switch:
            case BuildTarget.NoTarget:
            default:
                return outputFileName;
        }
    }


    private static string FindArg(string argName)
    {
        argName = ("-" + argName).ToLower();  // Backward compatible with earlier unity editor C# versions
        string[] args = Environment.GetCommandLineArgs();
        for (int i = 0; i < args.Length; i++)
        {
            var arg = (args[i] ?? string.Empty);
            
            if (arg.ToLower() != argName || i == args.Length - 1 || (args[i + 1] ?? string.Empty).StartsWith("-"))
                continue;

            return args[i + 1];
        }

        return string.Empty;
    }

    /// <summary>
    /// Set all version numbers alike
    /// </summary>
    private static void SetVersions()
    {
        var buildVersion = FindArg(buildVersionArgName);
        
        PlayerSettings.bundleVersion = buildVersion;
        UpdateVersionAssetFile(FindArg(versionJsonFileNameArgName), buildVersion);
    }

    private static void UpdateVersionAssetFile(string versionJsonFileName, string newVersionNumber)
    {
        if (string.IsNullOrEmpty(versionJsonFileName))
        {
            Debug.LogFormat("Skipping updating version asset file, paramater -{0} is empty", versionJsonFileNameArgName);
            return;
        }

        try
        {
            var fileContent = JsonUtility.ToJson(new BuildVersion { Version = newVersionNumber });
            File.WriteAllText(versionJsonFileName, fileContent);
        }
        catch (Exception exc)
        {
            Debug.LogWarning("Could not update version json file due to error: " + exc.Message);
        }
    }

    private class BuildVersion
    {
        public string Version;
    }
}

This is by the way an improved version of the script the Unity DevOps build task generates itself. One major difference is the original build script works in-line in the DevOps build so they can directly insert parameters into code, while I had to supply the DevOps build parameters into the Unity build startup parameters.

In conclusion: so this is how I do testing in my code. I work on DevOps tasks. When work is finished I test stuff myself, then let the build automatically create an installer for additional testing. Roan, or someone else that I instructed, then installs the updated version and runs a test on all the game that is in there. The test results are send back to me and I create new DevOps tasks if needed.

I like to share more of my ways of working if I improve my dev flow even more. Jump on my Discord if you want some help earlier, or wait for my video’s to come in the coming months.

Published by Erik_de_Roos

Erik de Roos is a Freelance software developer.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: