Gestures and interaction

Tap objects, model parts and individual copies; drive the scene from gestures with shared values; build custom drags; and combine scenes with buttons, scroll views and overlays.

Taps#

Every shape, <Mesh>, <Model> (per part) and <Instances> (per copy) can be tapped. A tap is one finger lifted within 10 points and half a second of where it went down. The nearest object under the finger that listens gets it.

Shapes#

TapShape.tsx

import { Scene, Sphere } from "@apricot/react";
import { useState } from "react";

export function TapShape() {
  const [on, setOn] = useState(false);
  return (
    <Scene style={{ flex: 1 }}>
      <Sphere material="light" color={on ? "#ffcc66" : "#444444"} onPress={() => setOn((v) => !v)} />
    </Scene>
  );
}

onPress receives a PressHit:

Field	Meaning
`point`	World-space point that was hit (a three.js `Vector3`)
`distance`	Distance from the camera
`mesh`	The mesh that was hit
`material`, `materialIndex`	The material of the face that was hit
`instanceId`	Which copy, for instanced meshes

Model parts#

onPartPress names the part that was tapped, using your model kit's names. See Loading models.

TapDoor.tsx

import { Scene } from "@apricot/react";
import { defineModelKit, Model } from "@apricot/react/model";
import { useState } from "react";

const car = defineModelKit(require("./car.glb"), {
  doorLeft: { select: "door_left", hinge: "front", axis: "y", range: [0, -65] },
});

export function TapDoor() {
  const [open, setOpen] = useState(false);
  return (
    <Scene style={{ flex: 1 }} look="studio">
      <Model kit={car} joints={{ doorLeft: open ? 1 : 0 }} onPartPress={({ part }) => part === "doorLeft" && setOpen((o) => !o)} />
    </Scene>
  );
}

Copies#

<Instances onPress> tells you which copy was tapped: { index, item, hit }. See Instances.

Gestures with shared values#

For drags, swipes, pinches and flings, use react-native-gesture-handler around the scene and write shared values that the scene reads. Nothing re-renders, and the gesture runs on the UI thread.

DragToTilt.tsx

import { Box, Group, Scene } from "@apricot/react";
import { Gesture, GestureDetector } from "react-native-gesture-handler";
import { useDerivedValue, useSharedValue, withSpring } from "react-native-reanimated";

export function DragToTilt() {
  const x = useSharedValue(0);
  const y = useSharedValue(0);
  const drag = Gesture.Pan()
    .onChange((e) => {
      x.value += e.changeY * 0.01;
      y.value += e.changeX * 0.01;
    })
    .onEnd(() => {
      x.value = withSpring(0);
      y.value = withSpring(0);
    });
  const rotation = useDerivedValue<readonly [number, number, number]>(() => [x.value, y.value, 0]);

  return (
    <GestureDetector gesture={drag}>
      <Scene style={{ flex: 1 }} look="studio">
        <Group rotation={rotation}>
          <Box size={[2, 1.2, 0.2]} radius={0.08} material="aluminium" />
        </Group>
      </Scene>
    </GestureDetector>
  );
}

Ready-made versions: swipe-to-spin (fling with decay), card-tilt (drag plus the phone's tilt sensor), drive-joystick (a thumbstick).

Orbit the camera#

<CameraRig orbit> gives drag-to-turn and pinch-to-zoom without writing a gesture. See Camera.

Custom drags inside the scene: `useSceneTouches`#

To build your own drag interaction that receives the scene's fingers directly (as <CameraRig orbit> does), use useSceneTouches in a component inside the scene. While it is active, the scene claims touches that start on it.

DragToSpin.tsx

import { Box, Group, Scene, type SceneObject, useScene, useSceneTouches } from "@apricot/react";
import { useRef } from "react";

function Spinner() {
  const scene = useScene();
  const group = useRef<SceneObject>(null);
  const last = useRef<number | null>(null);

  useSceneTouches({
    touch: (points) => {
      const x = points[0].x;
      if (last.current !== null && group.current) {
        group.current.rotation.y += (x - last.current) * 0.01;
        scene.invalidate();
      }
      last.current = x;
    },
    release: () => {
      last.current = null;
      return 0;
    },
  });

  return (
    <Group ref={group}>
      <Box size={1.4} />
    </Group>
  );
}

export function DragToSpin() {
  return (
    <Scene style={{ flex: 1 }}>
      <Spinner />
    </Scene>
  );
}

A listener has touch(points, time) (every start and move: fingers in screen points), release(time) (all fingers lifted; return how far they moved, in points, so taps can be told apart from drags) and optionally wheel(deltaY) on the web. Pass false as the second argument to stop listening.

Buttons and UI over a scene#

Views placed over a scene keep their own touches: buttons, sheets and toolbars work as usual. Lay out the scene with StyleSheet.absoluteFill and put UI on top, then frame the subject around it with <CameraRig inset>.

Overlays that let touches through#

A scene drawn over a screen should not block it: transparent plus pointerEvents="none" sends every touch to the views below.

CoinShower.tsx

import { Coin } from "@apricot/assets";
import { Pile, Scene } from "@apricot/react";
import { StyleSheet } from "react-native";

export function CoinShower() {
  return (
    <Scene transparent pointerEvents="none" style={StyleSheet.absoluteFill} camera={{ position: [0, 0, 20], fov: 30 }}>
      <Pile count={30} duration={3}>
        <Coin radius={0.5} />
      </Pile>
    </Scene>
  );
}

Taps#

Shapes#

TapShape.tsx

import { Scene, Sphere } from "@apricot/react";
import { useState } from "react";

export function TapShape() {
  const [on, setOn] = useState(false);
  return (
    <Scene style={{ flex: 1 }}>
      <Sphere material="light" color={on ? "#ffcc66" : "#444444"} onPress={() => setOn((v) => !v)} />
    </Scene>
  );
}

onPress receives a PressHit:

Field	Meaning
`point`	World-space point that was hit (a three.js `Vector3`)
`distance`	Distance from the camera
`mesh`	The mesh that was hit
`material`, `materialIndex`	The material of the face that was hit
`instanceId`	Which copy, for instanced meshes

Model parts#

onPartPress names the part that was tapped, using your model kit's names. See Loading models.

TapDoor.tsx

import { Scene } from "@apricot/react";
import { defineModelKit, Model } from "@apricot/react/model";
import { useState } from "react";

const car = defineModelKit(require("./car.glb"), {
  doorLeft: { select: "door_left", hinge: "front", axis: "y", range: [0, -65] },
});

export function TapDoor() {
  const [open, setOpen] = useState(false);
  return (
    <Scene style={{ flex: 1 }} look="studio">
      <Model kit={car} joints={{ doorLeft: open ? 1 : 0 }} onPartPress={({ part }) => part === "doorLeft" && setOpen((o) => !o)} />
    </Scene>
  );
}

Copies#

<Instances onPress> tells you which copy was tapped: { index, item, hit }. See Instances.

Gestures with shared values#

For drags, swipes, pinches and flings, use react-native-gesture-handler around the scene and write shared values that the scene reads. Nothing re-renders, and the gesture runs on the UI thread.

DragToTilt.tsx

import { Box, Group, Scene } from "@apricot/react";
import { Gesture, GestureDetector } from "react-native-gesture-handler";
import { useDerivedValue, useSharedValue, withSpring } from "react-native-reanimated";

export function DragToTilt() {
  const x = useSharedValue(0);
  const y = useSharedValue(0);
  const drag = Gesture.Pan()
    .onChange((e) => {
      x.value += e.changeY * 0.01;
      y.value += e.changeX * 0.01;
    })
    .onEnd(() => {
      x.value = withSpring(0);
      y.value = withSpring(0);
    });
  const rotation = useDerivedValue<readonly [number, number, number]>(() => [x.value, y.value, 0]);

  return (
    <GestureDetector gesture={drag}>
      <Scene style={{ flex: 1 }} look="studio">
        <Group rotation={rotation}>
          <Box size={[2, 1.2, 0.2]} radius={0.08} material="aluminium" />
        </Group>
      </Scene>
    </GestureDetector>
  );
}

Ready-made versions: swipe-to-spin (fling with decay), card-tilt (drag plus the phone's tilt sensor), drive-joystick (a thumbstick).

Orbit the camera#

<CameraRig orbit> gives drag-to-turn and pinch-to-zoom without writing a gesture. See Camera.

Custom drags inside the scene: `useSceneTouches`#

DragToSpin.tsx

import { Box, Group, Scene, type SceneObject, useScene, useSceneTouches } from "@apricot/react";
import { useRef } from "react";

function Spinner() {
  const scene = useScene();
  const group = useRef<SceneObject>(null);
  const last = useRef<number | null>(null);

  useSceneTouches({
    touch: (points) => {
      const x = points[0].x;
      if (last.current !== null && group.current) {
        group.current.rotation.y += (x - last.current) * 0.01;
        scene.invalidate();
      }
      last.current = x;
    },
    release: () => {
      last.current = null;
      return 0;
    },
  });

  return (
    <Group ref={group}>
      <Box size={1.4} />
    </Group>
  );
}

export function DragToSpin() {
  return (
    <Scene style={{ flex: 1 }}>
      <Spinner />
    </Scene>
  );
}

Buttons and UI over a scene#

Overlays that let touches through#

A scene drawn over a screen should not block it: transparent plus pointerEvents="none" sends every touch to the views below.

CoinShower.tsx

import { Coin } from "@apricot/assets";
import { Pile, Scene } from "@apricot/react";
import { StyleSheet } from "react-native";

export function CoinShower() {
  return (
    <Scene transparent pointerEvents="none" style={StyleSheet.absoluteFill} camera={{ position: [0, 0, 20], fov: 30 }}>
      <Pile count={30} duration={3}>
        <Coin radius={0.5} />
      </Pile>
    </Scene>
  );
}

Gestures and interaction

Taps#

Shapes#

Model parts#

Copies#

Gestures with shared values#

Orbit the camera#

Custom drags inside the scene: useSceneTouches#

Buttons and UI over a scene#

Overlays that let touches through#

Related#

Gestures and interaction

Taps#

Shapes#

Model parts#

Copies#

Gestures with shared values#

Orbit the camera#

Custom drags inside the scene: useSceneTouches#

Buttons and UI over a scene#

Overlays that let touches through#

Related#

Custom drags inside the scene: `useSceneTouches`#

Custom drags inside the scene: `useSceneTouches`#