3D with Flutter GPU and flutter_scene

Rendering real meshes inside the widget tree, using Impeller's GPU abstraction directly. This is the only path where 3D content is drawn by Flutter itself rather than by an embedded engine.

:::warning Preview APIs

Flutter GPU and flutter_scene are preview-stage. Names, signatures, and the build-hook mechanism have changed between releases. Treat the code here as the shape of the solution, pin exact versions in pubspec.yaml, and check the package README and example for the version you install before writing against it.

:::

Contents

• The two layers
• Setup
• Importing a model
• Rendering into a widget
• Camera
• Orbit controls
• Materials and lighting
• Animation loop
• Picking
• Overlaying Flutter widgets
• Troubleshooting

The two layers

Layer	What it is	Use when
Flutter GPU (flutter_gpu)	A low-level API over Impeller: device, buffers, textures, render passes, pipelines, shader libraries	You are writing a renderer, or need a pipeline flutter_scene does not offer
flutter_scene	A scene graph built on Flutter GPU: nodes, meshes, materials, cameras, glTF import	You want to display and manipulate models

Almost all application work belongs in flutter_scene. Drop to Flutter GPU when you need custom passes — post-processing, instancing, a bespoke material model.

Setup

Both are gated on Impeller. Add the SDK package and the scene library:

dependencies:
  flutter:
    sdk: flutter
  flutter_gpu:
    sdk: flutter
  flutter_scene: <pinned version>

dev_dependencies:
  flutter_scene_importer: <pinned version>

Run the app with Impeller enabled if it is not the default for your target:

flutter run --enable-impeller

Verify the backend before debugging anything else — every failure mode below looks identical when Impeller is not active.

Importing a model

flutter_scene does not read .glb at runtime; a build step converts it into an engine-ready model asset. Declare the source models for the importer, and the output directory as a Flutter asset:

flutter_scene_importer:
  models:
    - assets/models/product.glb

flutter:
  assets:
    - build/models/

The conversion runs as part of the build. Confirm the exact key names, the output path, and whether a hook/build.dart entry is required for your pinned version — this part of the toolchain has moved more than the runtime API.

Then load it:

import 'package:flutter_scene/scene.dart';
import 'package:vector_math/vector_math_64.dart' as vm;

late final Node _model;

Future<void> _loadModel() async {
  await Scene.initializeStaticResources();
  _model = await Node.fromAsset('build/models/product.model');
  _model.name = 'product';
}

Scene.initializeStaticResources() compiles the built-in shaders and must complete before the first render.

Rendering into a widget

The scene is drawn from a CustomPainter, so it composites with the rest of the widget tree like any other painted content:

class ProductScenePainter extends CustomPainter {
  ProductScenePainter({
    required this.scene,
    required this.camera,
    required Listenable repaint,
  }) : super(repaint: repaint);

  final Scene scene;
  final Camera camera;

  @override
  void paint(Canvas canvas, Size size) {
    scene.render(camera, canvas, viewport: Offset.zero & size);
  }

  @override
  bool shouldRepaint(covariant ProductScenePainter oldDelegate) =>
      oldDelegate.scene != scene || oldDelegate.camera != camera;
}

RepaintBoundary(
  child: CustomPaint(
    size: Size.infinite,
    painter: ProductScenePainter(
      scene: _scene,
      camera: _camera,
      repaint: _ticker,          // a Listenable that fires each frame
    ),
  ),
)

Driving the repaint from a Listenable rather than setState keeps the frame loop out of the build phase — the same technique as in Custom painting.

Assembling the scene:

final scene = Scene();
scene.add(_model);

Camera

final camera = PerspectiveCamera(
  fovYDegrees: 45,
  position: vm.Vector3(0, 1.4, 4),
  target: vm.Vector3(0, 0.8, 0),
);

Three numbers decide whether a model looks right: the field of view (40 to 50 degrees reads as natural; wider exaggerates perspective), the distance, and the target. Frame the model by computing its bounding box and placing the camera at a distance derived from the box radius and the field of view, rather than hard-coding a position that breaks the moment the model changes.

Orbit controls

Convert drag and pinch gestures into spherical coordinates, then into a camera position:

double _yaw = 0;          // radians
double _pitch = 0.3;
double _distance = 4;

void _onPanUpdate(DragUpdateDetails d) {
  _yaw -= d.delta.dx * 0.01;
  _pitch = (_pitch + d.delta.dy * 0.01).clamp(-1.2, 1.2);
  _controller.notify();
}

void _onScaleUpdate(ScaleUpdateDetails d) {
  _distance = (_distance / d.scale).clamp(1.5, 12.0);
  _controller.notify();
}

vm.Vector3 get _cameraPosition {
  final cosPitch = math.cos(_pitch);
  return vm.Vector3(
    _distance * cosPitch * math.sin(_yaw),
    _distance * math.sin(_pitch),
    _distance * cosPitch * math.cos(_yaw),
  ) + _target;
}

Clamp the pitch short of the poles, or the camera flips when the view direction becomes parallel to the up vector. Add inertia by continuing to apply the last gesture velocity with damping after onPanEnd — it is the difference between a control that feels cheap and one that does not.

Materials and lighting

Models imported from glTF arrive with their materials. To adjust one, walk the node tree and replace or tweak the material on the mesh primitives:

void _tint(Node node, vm.Vector4 colour) {
  final mesh = node.mesh;
  if (mesh != null) {
    for (final primitive in mesh.primitives) {
      final material = primitive.material;
      if (material is UnlitMaterial) {
        material.baseColorFactor = colour;
      }
    }
  }
  for (final child in node.children) {
    _tint(child, colour);
  }
}

Material and lighting support in the preview is deliberately limited compared with a full engine. If you need image-based lighting, shadow maps, or advanced physically based materials today, that is the argument for Filament.

Animation loop

late final Ticker _ticker;

@override
void initState() {
  super.initState();
  _ticker = createTicker((elapsed) {
    _model.localTransform = vm.Matrix4.rotationY(
      elapsed.inMicroseconds / 1e6 * 0.5,
    );
    _repaint.notify();
  })..start();
}

@override
void dispose() {
  _ticker.dispose();
  super.dispose();
}

Stop the ticker when the route is not visible or the app is backgrounded — an idle 3D view still burns the GPU at full rate otherwise. See Lifecycle.

Picking

To select a part of the model by tapping, cast a ray from the screen point into the scene:

1. Convert the tap position to normalised device coordinates: x and y in the range -1 to 1, with y flipped.
2. Build the inverse of the view-projection matrix.
3. Unproject two points, at near and far depth, and subtract to get the ray direction.
4. Transform the ray into each candidate node's local space using the inverse of its world transform.
5. Test against the node's bounding box first; only test triangles for the nodes that pass.
6. Keep the nearest hit.

Bounding-box picking is enough for "which part did the user tap" on a model with a handful of named nodes, and it is far cheaper than triangle-accurate picking. Name the nodes meaningfully in the source file so the app can reason about them without depending on index order.

Overlaying Flutter widgets

The reason to choose this path is that the 3D view is a painted widget, so everything else in Flutter still works:

Stack(
  children: [
    Positioned.fill(child: _ProductScene(controller: _controller)),
    Positioned(
      left: 16,
      right: 16,
      bottom: 24,
      child: Card(
        child: ListenableBuilder(
          listenable: _controller,
          builder: (context, _) => Text(
            _controller.selectedPartName ?? 'Tap a part',
            style: Theme.of(context).textTheme.titleMedium,
          ),
        ),
      ),
    ),
  ],
)

No platform view, no message channel, no texture synchronisation — the overlay reads the same Dart objects the renderer uses. That is the concrete advantage over the WebView and Unity approaches in Embedded 3D renderers.

Troubleshooting

Symptom	Likely cause
Blank area where the model should be	Impeller not active, or initializeStaticResources not awaited
Asset not found at runtime	The importer did not run, or build/models/ is not declared under flutter: assets:
Model loads but is invisible	Camera inside the model, model at a different scale, or faces culled because normals are inverted
Model is enormous or microscopic	Unit mismatch on export — glTF is metres
Model is upside down or rotated	Axis convention mismatch; re-export with the correct up axis
Frame rate collapses	Too many triangles or draw calls, oversized textures — see 3D performance
Build fails after upgrading the package	Preview API change; read the changelog before adjusting code

Next: Embedded 3D renderers.