Skip to content

Stratkit Content Archives

Summary

Builds and downloads content archives. These are an alternative representation of asset bundles specifically created for ECS subscenes. - Builds content archives as part of the build pipeline - Downloads Content Archive and load them into the game on demand

What is Content Archive

Link

Content Archive Builder

  • ContentArchiveBuilder will scan AssetDatabase to find all SubSceneLoader for streaming SubScenes and GameMapConfigurationScriptable for non-streaming SubScenes.
  • All these SubScenes and their dependencies are included in the Content Archive. This includes balancing data, units and buildings prefabs, models, textures, materials and shaders.

Content Archive Builder - The content archive builder comes with several other functionalities - Check Asset Duplication list all assets that are part of Content Archive, and also part of another system (Addressable or Built-in scenes) - Print All Content Archive Assets. List all assets referenced directly or indirectly from Content Archive, which consists of the maps and the SharedSubScene. Noted that these assets are referenced from the Entity Scenes and not the Authoring Scenes. So selecting these scenes and choose Select Dependencies in the context menu won't give the same result. - Print All Built-in Assets. These are assets referenced directly or indirectly from all the scenes in the Build Settings, minus the scenes with "SubSceneMaster" in the name. - If Show Results Visually is checked, a result window will be shown instead of logging.

Asset Duplication Validator

Asset Duplication Result Window - This window is shown after clicking one of the three buttons listed above. - Each column can be sorted. - You can select multiple items. - After item selections, you can click one of the buttons at the top: - Select Assets: this will select the assets in the Project Panel. From here you could move, delete them. Or investigate their dependencies, and reversed dependencies. - Move Assets: this will move all these asset to a folder of your choice. The usual target would be the ContentArchiveAssets folder.

Content Archive Flow

flowchart LR
    subgraph CI
        BalancingData --> Build
        ReferencedAssets --> Build
        Maps --> Build
        Build{Build}
        ContentArchive
        Publish{Publish & Compress}
        Build --> ContentArchive --> Publish
        Publish -- create --> Catalogs.bin
        Publish -- hash --> HashedFiles
    end
    subgraph CDN
        Catalogs2[Catalogs.bin]
        HashedFiles2[HashedFiles]
    end
    Catalogs.bin -- replace --> Catalogs2
    HashedFiles -- add to --> HashedFiles2
    subgraph Device
        Load{Load}
        Catalogs2 -- download --> NewHashFound
        NewHashFound{New Hash Found}
        NewHashFound -- yes --> Downloader
        Downloader --> Uncompress -- add to --> CachedHashedFiles
        HashedFiles2 -- download --> Downloader
        CachedHashedFiles --> Load
    end

Test Content Archive Locally and Remote Update

Build Local Content Archive - Find the scriptable object ContentArchivesModule - Set ShouldUseContentArchive to true - Set UseLocalPath to false - Set DownloadUri to where you host your web server (e.g. http://localhost:8000) - Make your build similar to a Content Archive build on GitHub - Remove all SubSceneMaster scenes in BuildSettings - Add ENABLE_CONTENT_DELIVERY to PlayerSetting - Host your Content Archive web-server - Find ContentArchiveBuilder scriptable object - Click Build Content Archive at Default Location in the inspector - Note down the Default Output Build Path folder - Open terminal and change directory to that folder - Execute python3 -m http.server to start a web-server in the exported Content Archive folder - Testing in the editor: - The downloading phase can be tested - The loading phase can not be tested, the editor will always load from the cached Entity Scenes and ignore the Content Archive - Create your local build (Windows / Mac) to test - The downloading phase can be tested - The loading phase can be tested - Remote update test - Make changes and create a new Content Archive - Change texture, material, shader or model - Regen the map, new config, new props arrangement - The local server should automatically be aware of new catalog and files - Rerun the already built client, it should download the new files and show the new content

Check Content Archive size on Github

Build Local Content Archive

Compression Ratio

  • From the example building Content Archive for Mac platform, 134.8MB was compressed into 24.7MB, the ratio is at 81.6%
  • This means the client will download 24.7MB and unpack it to 134.8MB locally, making the app size on the device grows
  • High compression ratio is nice to save bandwidth and space on our CDN
  • It's bad for device's storage space, we should optimise also the uncompressed size of Content Archive
  • We will examine the uncompressed size only Entity Scenes. ReferencedAssets is out of scope.

Entity Scene composition

  • Entity Scene is composed from chunks. Each chunk:
  • Always has a size of 16KB
  • Can only store entities of the same archetype
  • This leads to the fact that a chunk could be underutilised
  • Cumulatively, chunk always has unused space, the smaller the better

Chunk utilisation

  • Vigilus Map 4 SubScenes Vigilus Map 4 SubScenes

  • Vigilus Map 16 SubScenes Vigilus Map 16 SubScenes

  • Vigilus Map 68 SubScenes Vigilus Map 68 SubScenes

  • Multiple factors could affect this unused space
  • How large is the archetype cost per entity
  • How many entity of that archetype exist in the SubScene
  • How much info in the archetype is derivative
  • How many system archetype exist (such as section info)
  • What can we do to improve the chunk utilisation?
  • Calculate derivative info at runtime. This is a storage-cpu tradeoff.
  • Divide the scene differently to achieve optimal chunk utilization. Beware that very large section size defeat the purpose of streaming. Vigilus Section Streaming 4 SubScenes