Skip to content

Add spark.js support to three.js 3DTilesRenderer #1497

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
castano wants to merge 6 commits into
base: master
Choose a base branch
from
+96 −13
Draft

Add spark.js support to three.js 3DTilesRenderer #1497

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
1de40ff
Add spark support to GLTFExtensionsPlugin. Upgrade three.js dependenc…
castano Mar 4, 2026
601cd19
Actually add spark.js dependency.
castano Mar 4, 2026
08c227f
Add Spark support to googleMaps example.
castano Mar 5, 2026
b96fd14
Track memory footprint of spark textures.
castano Mar 5, 2026
73f9481
Undo changes to GLTFExtensionPlugin.
castano Mar 17, 2026
9a5171e
Merge branch 'master' into spark
castano Mar 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 57 additions & 5 deletions example/three/googleMapsAerial.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import { GeoUtils, WGS84_ELLIPSOID, TilesRenderer } from '3d-tiles-renderer';
import { TilesFadePlugin, TileCompressionPlugin, GLTFExtensionsPlugin, CesiumIonAuthPlugin, ReorientationPlugin } from '3d-tiles-renderer/plugins';
import { CesiumIonAuthPlugin } from '3d-tiles-renderer/core/plugins';
import { TilesFadePlugin, TileCompressionPlugin, GLTFExtensionsPlugin, ReorientationPlugin } from '3d-tiles-renderer/plugins';
import { estimateBytesUsed } from '../../src/three/renderer/utils/MemoryUtils.js';
import {
Scene,
WebGLRenderer,
Expand All @@ -9,14 +11,27 @@ import {
} from 'three';
import { DRACOLoader } from 'three/examples/jsm/loaders/DRACOLoader.js';
import { OrbitControls } from 'three/examples/jsm/controls/OrbitControls.js';
import { SparkGL } from '@ludicon/spark.js';
import Stats from 'three/examples/jsm/libs/stats.module.js';
import { GUI } from 'three/examples/jsm/libs/lil-gui.module.min.js';
import { createSparkPlugins } from '@ludicon/spark.js/three-gltf';

let camera, controls, scene, renderer, tiles;
let spark;
let stats;

const params = {
enableSpark: true,
preferLowQuality: true,
generateMipmaps: false,
errorTarget: 16.0,
textureVRAM: 0,
};
Comment thread
gkjohnson marked this conversation as resolved.

const raycaster = new Raycaster();
raycaster.firstHitOnly = true;

init();
animate();
init().then( () => animate() );

function reinstantiateTiles() {

Expand All @@ -35,7 +50,8 @@ function reinstantiateTiles() {
tiles.registerPlugin( new GLTFExtensionsPlugin( {
// Note the DRACO compression files need to be supplied via an explicit source.
// We use unpkg here but in practice should be provided by the application.
dracoLoader: new DRACOLoader().setDecoderPath( 'https://unpkg.com/three@0.153.0/examples/jsm/libs/draco/gltf/' )
dracoLoader: new DRACOLoader().setDecoderPath( 'https://unpkg.com/three@0.153.0/examples/jsm/libs/draco/gltf/' ),
plugins: params.enableSpark ? createSparkPlugins( spark, { preferLowQuality: params.preferLowQuality, generateMipmaps: params.generateMipmaps } ) : []
} ) );
tiles.registerPlugin( new ReorientationPlugin( { lat: 35.6586 * MathUtils.DEG2RAD, lon: 139.7454 * MathUtils.DEG2RAD } ) );

Expand All @@ -55,7 +71,7 @@ function reinstantiateTiles() {

}

function init() {
async function init() {

scene = new Scene();

Expand All @@ -79,8 +95,35 @@ function init() {
controls.autoRotateSpeed = 0.5;
controls.enablePan = false;

// initialize Spark
const gl = renderer.getContext();
spark = await SparkGL.create( gl, {
preload: [ 'rgb' ],
cacheTempResources: true,
verbose: true,
} );

reinstantiateTiles();

// Initialize stats
stats = new Stats();
stats.showPanel( 0 ); // 0: fps, 1: ms, 2: mb, 3+: custom
document.body.appendChild( stats.dom );

// Create GUI
const gui = new GUI();
gui.add( params, 'enableSpark' ).name( 'Enable Spark' ).onChange( () => {
reinstantiateTiles();
} );
gui.add( params, 'preferLowQuality' ).name( 'BC1 / ETC2' ).onChange( () => {
reinstantiateTiles();
} );
gui.add( params, 'generateMipmaps' ).name( 'Generate Mipmaps' ).onChange( () => {
reinstantiateTiles();
} );
gui.add( params, 'errorTarget', 4, 16, 0.1 ).name( 'Error Target' );
gui.add( params, 'textureVRAM' ).name( 'Texture VRAM (MB)' ).listen().disable();

onWindowResize();
window.addEventListener( 'resize', onWindowResize, false );
window.addEventListener( 'hashchange', initFromHash );
Expand Down Expand Up @@ -120,11 +163,14 @@ function animate() {

requestAnimationFrame( animate );

if ( stats ) stats.begin();

if ( ! tiles ) return;

controls.update();

// update options
tiles.errorTarget = params.errorTarget;
tiles.setResolutionFromRenderer( camera, renderer );
tiles.setCamera( camera );

Expand All @@ -134,6 +180,8 @@ function animate() {

render();

if ( stats ) stats.end();

}

function render() {
Expand All @@ -154,4 +202,8 @@ function render() {

}

// Update GPU memory info
const approximateVRAM = estimateBytesUsed( scene );
params.textureVRAM = ( approximateVRAM / ( 1024 * 1024 ) ).toFixed( 1 );
Comment thread
gkjohnson marked this conversation as resolved.

}
33 changes: 28 additions & 5 deletions package-lock.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 3 additions & 2 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@
"@babylonjs/core": "^8.47.2",
"@babylonjs/loaders": "^8.47.2",
"@eslint/js": "^9.0.0",
"@ludicon/spark.js": "^0.1.3",
"@react-three/drei": "^10.0.0",
"@react-three/fiber": "^9.0.0",
"@types/node": "^24.3.0",
Expand All @@ -108,7 +109,7 @@
"leva": "^0.10.0",
"lil-gui": "^0.21.0",
"postprocessing": "^6.36.4",
"three": "^0.170.0",
"three": "^0.182.0",
"typescript": "^5.6.0",
"typescript-eslint": "^8.48.1",
"vite": "^6.2.2",
Expand All @@ -120,7 +121,7 @@
"@react-three/fiber": "^8.17.9 || ^9.0.0",
"react": "^18.3.1 || ^19.0.0",
"react-dom": "^18.3.1 || ^19.0.0",
"three": ">=0.167.0"
"three": ">=0.182.0"
Comment on lines -123 to +124
Copy link
Copy Markdown
Contributor

@gkjohnson gkjohnson Mar 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't change the peer dependency requirements for the project

Copy link
Copy Markdown
Contributor Author

@castano castano Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll need to double check whether the required three.js features are all present in that older version, and if they don't gracefully handle the error. It may just work, since most of the three.js changes were in the WebGPU backend, and to add better support for normal maps, which these demos don't need.

Copy link
Copy Markdown
Contributor

@gkjohnson gkjohnson Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spark.js is not a dependency or requirement of this project so the peer dependency should not change - users can use and install 3d-tiles-renderer with r167+ just fine. Spark.js will need to specify it's own three.js dependency limit so if users install Spark that peer dependency version will need to be respected.

},
"peerDependenciesMeta": {
"@react-three/fiber": {
Expand Down
9 changes: 8 additions & 1 deletion src/three/renderer/utils/MemoryUtils.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { estimateBytesUsed as _estimateBytesUsed } from 'three/examples/jsm/utils/BufferGeometryUtils.js';
import { TextureUtils } from 'three';
import { TextureUtils, ExternalTexture } from 'three';

export function getTextureByteLength( tex ) {

Expand All @@ -9,6 +9,13 @@ export function getTextureByteLength( tex ) {

}

// IC: The code below assumes tex was created from an ImageBitmap
Comment thread
gkjohnson marked this conversation as resolved.
if ( tex instanceof ExternalTexture && tex.userData?.byteLength ) {

return tex.userData.byteLength;

}
Comment on lines +12 to +17
Copy link
Copy Markdown
Contributor

@gkjohnson gkjohnson Mar 7, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if an "ExternalTexture" is provided without userData.byteLength? It looks like "image" will be "null" on the Texture, then and the follow code will crash, right? Assuming we can't get the actual texture size from the ExternalTexture handle we should figure out a reasonable default behavior when byteLength isn't provided.

Copy link
Copy Markdown
Contributor Author

@castano castano Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code will crash with ExternalTextures generated by other tools. It also won't compute the size of CompressedTextures correctly, as the generateMipmaps parameter is forced to false, but textures may still have mipmaps. I'd be happy to robustify this code. I was aiming for minimal changes here.

Copy link
Copy Markdown
Contributor

@gkjohnson gkjohnson Mar 9, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making it more robust would be great. I expect it's not possible to determine the actual in-memory size of one of these textures from just the WebGLTexture (or WebGPU) handle itself, right? At least not without the gl contenxt. In lieu of any other information being available for calculating the size I was thinking we could check for a custom field (as you are here) and otherwise fallback to a fixed memory size of a 64x64 texture or something just so the value isn't "0" in the cache.

Copy link
Copy Markdown
Contributor Author

@castano castano Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll propose that change in a separate PR. Here I just wanted to keep it simple to demonstrate that the changes required for integration are minimal.

Copy link
Copy Markdown
Contributor

@gkjohnson gkjohnson Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be great. A PR to improve handling of just ExternalTextures is something we could merge immediately. A couple other things that come to mind that need to be handled:

  • How is disposal handled for these external textures? Does calling ExternalTexture.dispose actually dispose of the WebGLTexture / WebGPUTexture handle? Or does disposal need to happen separately, as with an imageBitmap?
  • The UnloadTilesPlugin is designed to delete any GPU memory for tiles that aren't actively visible or being rendered. When the tiles are made visible again the textures and geometry are re-initialized on the GPU. I assume this model wouldn't work for the ExternalTexture since re-initializing it isn't as simple as just reuploading the existing data? The easy thing to do in that case is to just ignore and not dispose ExternalTextures encountered in the plugin.

Copy link
Copy Markdown

@donmccurdy donmccurdy Apr 21, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... I think my primary concern is that as it is we can't notify a user that their texture isn't being cleaned up if they're using a non-Spark ExternalTexture instance...

Hm, my original intention for THREE.ExternalTexture had been that if the user (or a library the user delegates to) creates a GPU resource, they must also manage the GPU resource throughout its lifecycle. From that perspective:

  1. It feels unexpected that the user must store the WebGL context on the texture
  2. It should be up to the user (or delegate library) to decide whether the GPU resource has a 1:1 or 1:many relationship with ExternalTexture container(s), and to manage GPU resources accordingly

If the user is relying on Spark to create these ExternalTextures, it becomes Spark's responsibility to either manage the resource lifecycle, or communicate that responsibility to the user. If the user has created an ExternalTexture manually or with some other dependency, it would be the user and/or dependency's responsibility to do the same.

The proposal Ludicon/spark.js#31 is consistent, I think, in defining a clear contract for the lifecycle of the ExternalTexture and its GPU resource. Arguably, GLTFLoader is not doing so good of a job there, by constructing ImageBitmap textures in unspecified conditions, and leaving it to the user to dispose those resources manually... which I'd guess is often overlooked. I don't have an easy fix for that, but it's not a pattern I'd want to promote.

I think it's not practical for an ExternalTexture to store enough metadata for its GPU texture to be recreated after disposal, would you agree? The original image should no longer exist in memory. Assuming so... maybe a spark.destroyTexture(gpuTexture) utility would be helpful rather than passing around the GL context? The typical pattern is that three.js textures can be reused after disposal, so 3DTilesRendererJS could dispose the texture eagerly, and the GPU resource only after the tile is finally unloaded.

Copy link
Copy Markdown
Contributor

@gkjohnson gkjohnson Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@donmccurdy I guess fundamentally I'm confused as to why one would advocate for his behavior in the spark library but not with ImageBitmap in GLTFLoader - it's just as possible for GLTFLoader to register the same kind of disposal event but I think it's known this causes massive usability issues, which is why it is the way it is.

If the user is relying on Spark to create these ExternalTextures, it becomes Spark's responsibility to either manage the resource lifecycle, or communicate that responsibility to the user.

To be clear I completely agree that symmetry in the life cycle of the texture is both a nicer and safer pattern. The problem is the practical issues it causes. The suggested pattern betrays the expectations by every other Texture instance in the project. Off the top of my head my own real-world use cases this would have broken if it behaved this way:

  • Disposing of a texture to temporarily replace it with another, then swapping the original texture back in later.
  • Disposing of resources to store them in a CPU-side cache to avoid having to reload them again when needed.
  • Cloning a texture to adjust offsets, then disposing the original.
  • Disposing of a cloned instance will not actually dispose of the external texture handle

All of these very reasonable use cases would wind up breaking if textures were managed in the way that is being suggested.

It should be up to the user (or delegate library) to decide whether the GPU resource has a 1:1 or 1:many relationship with ExternalTexture container(s), and to manage GPU resources accordingly

This isn't how three.js' API presents itself though. Perhaps it's just becoming clear that three.js' API surface was not designed for these kinds of use cases if even just because the original source image textures used did not have these issues. Why should "dispose" behave inconsistently for image bitmap or external texture? Perhaps it's just the way I'm thinking about but as I mentioned before, "dispose" has never meant "dispose content forever" in three.js. Perhaps there's are argument to be made for adding a "disposeForever" function to three.js textures to accommodate these types of use cases.

I think it's not practical for an ExternalTexture to store enough metadata for its GPU texture to be recreated after disposal, would you agree

Of course agree with this. But I also think that I should also be able to call "dispose" and expect consistent behavior. These classes are supposed to be reliable abstractions with consistent, usable behavior so I'm not in agreement that the "dispose" function should suddenly be overloaded to completely discard resources and make a texture instance unusable, which again doesn't happen anywhere else.

Assuming so... maybe a spark.destroyTexture(gpuTexture) utility would be helpful rather than passing around the GL context?

My preference is to not add library-specific code throughout the project to accommodate every special case someone may come to the project with.

--

Either way if this is the preference for how to manage the disposal of these textures that's fine. I only intended to point out what will be ergonomic issues down the line (and within this project).

Copy link
Copy Markdown
Contributor Author

@castano castano Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The more I learn about how ExternalTextures work, the more confused I am. The behavior of the WebGL and WebGPU backends are entirely different.

In WebGPU, it appears that the behavior depends on whether the texture has been used for rendering or not. During rendering, three calls Textures.updateTexture and this registers the textureData in the Textures DataMap and enables the shared ownership behavior that you have come to expect.

This is quite confusing. If you call dispose() prior to rendering, nothing happens, if you call it after, the texture is destroyed. If you have multiple textures referencing the same GPUTexture, but some have been used for rendering and others haven't, then the behavior is completely broken.

The WebGL backend doesn't appear to behave like that, and it never takes ownership of the textures objects. This is at least more consistent.

I'm working on a solution that involves the creation of a map to manage the lifetime of spark external textures. Instead of using THREE.ExternalTexture to create a texture, you use a derived class that overrides the copy constructor and the dispose method and tracks the gl object associated with spark and the number of textures that reference it, so that it can dispose the texture object when no texture references it anymore. I'll post an update once I have something solid.

Copy link
Copy Markdown
Contributor Author

@castano castano Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, here's what I have:

Ludicon/spark.js@f2915cd

Usage is as follows:

        const gl = this.spark.gl
        const onRelease = isWebGL ? tex => gl.deleteTexture(tex) : tex => tex.destroy()
        const texture = new SparkThreeExternalTexture(gpuTexture, onRelease)

in the case of the 3DTilesRenderer no changes are necessary. The GLTF loader plugin takes care of creating the external texturs using the SparkThreeExternalTexture class.

It works as expected in my examples, but it needs more testing. Let me know what you think of this approach.

Copy link
Copy Markdown
Contributor

@gkjohnson gkjohnson May 5, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

During rendering, three calls Textures.updateTexture and this registers the textureData in the Textures DataMap and enables the shared ownership behavior that you have come to expect.

This difference in behavior between WebGLRenderer and WebGPURenderer may not be intended or desired. WebGPURenderer is still very much in development and there are still quite a few oversights like this throughout.

OK, here's what I have:

Ludicon/spark.js@f2915cd

This doesn't completely address everything but I think this is fundamentally something that three.js needs to promote a more consistent pattern for if it's going to become more commonly used. I think we can wait for more problems to arise before trying to address this further.


const { format, type, image } = tex;
const { width, height } = image;

Expand Down
Loading