Learn basics of React-XR

Have you ever considered creating an AR/VR app for Vision Pro—not just for VR headsets, but also for mobile phones? Learn how to do it!

Tue, Apr, 15, 2025
Adam Hitzger's blog - Have you ever considered creating an AR/VR app for  Vision Pro—not just for VR headsets, but also fo

Getting Started with Your Project

First, you'll need to install Node.js and an IDE such as VS Code, Cursor, or Neovim. It's also recommended to install Git for version control and to publish your app on GitHub.

Since we'll be fetching data from the OpenWeatherAPI, create an account and save your API key.

We'll deploy the app to Vercel. Sign in using your GitHub account. When creating a new project, Vercel will prompt you to import code directly from GitHub. A great feature is that whenever you update your code and push it to GitHub, Vercel will automatically deploy your changes. It's free until you reach Vercel's usage limits, which should suffice for most projects—and no credit card is required!

Here are the download links:

Setting Up Next.js with WebXR

Let's learn how to set up a Next.js application and implement WebXR. We'll utilize React Three packages - Drei, Fiber, XR, and uikit from Poimandres.

Open terminal and run this command that will create nextjs app.

example.sh
1npx create-next-app@latest

Configuring Your Next.js Project

When you run the create-next-app command, the terminal will prompt you with several configuration options:​

example.sh
1What is your project named? my-app 2Would you like to use TypeScript? No / Yes 3Would you like to use ESLint? No / Yes 4Would you like to use Tailwind CSS? No / Yes 5Would you like your code inside a `src/` directory? No / Yes 6Would you like to use App Router? (recommended) No / Yes 7Would you like to use Turbopack for `next dev`? No / Yes 8Would you like to customize the import alias (`@/*` by default)? No / Yes 9What import alias would you like configured? @/*

After completing the setup, navigate to your project directory:

example.sh
1cd your-project-name

Open the project in your preferred IDE.​

Next, create a .env.local file in the root of your project directory. This file is used to store environment variables, such as API keys, that should not be committed to version control.​

In .env.local, add your OpenWeather API key as follows:

example.sh
1/.env.local 2OPEN_WEATHER_API=your_api_key

Now, install react-three packages:

example.sh
1npm install three @types/three @react-three/fiber @react-three/drei @react-three/xr @react-three/uikit

And what is react-three? It's a React abstraction of Three.js. Instead of writing it imperatively:

example.javascript
1const scene = new THREE.Scene(); 2const geometry = new THREE.BoxGeometry(); 3const material = new THREE.MeshBasicMaterial({ color: 0x00ff00 }); 4const cube = new THREE.Mesh(geometry, material); 5scene.add(cube);

React is component base:

example.javascript
1<Canvas> 2 <mesh> 3 <boxGeometry args={[1, 1, 1]} /> 4 <meshStandardMaterial color="orange" /> 5 </mesh> 6</Canvas>

Understanding React Three Fiber and Its Ecosystem

React Three Fiber is a React renderer for Three.js, enabling you to build 3D scenes declaratively using reusable, self-contained components that respond to state changes.

To simplify working with React Three Fiber, several libraries have been developed:​

Project Structure Overview

Within the app directory of your Next.js project, you'll find the following key files:​

example.tsx
1/app/layout.tsx 2import type { Metadata } from "next"; 3import localFont from "next/font/local"; 4import "./globals.css"; 5 6const geistSans = localFont({ 7 src: "./fonts/GeistVF.woff", 8 variable: "--font-geist-sans", 9 weight: "100 900", 10}); 11const geistMono = localFont({ 12 src: "./fonts/GeistMonoVF.woff", 13 variable: "--font-geist-mono", 14 weight: "100 900", 15}); 16 17export const metadata: Metadata = { 18 title: "hitzge01", 19 description: "Generated by create next app", 20}; 21 22export default function RootLayout({ 23 children, 24}: Readonly<{ 25 children: React.ReactNode; 26}>) { 27 return ( 28 <html lang="en"> 29 <body 30 className={`${geistSans.variable} ${geistMono.variable} antialiased`} 31 > 32 {/*<Header/> component*/} 33 <main className="flex flex-col min-h-screen"> 34 {children} 35 </main> 36 {/*<Footer/> component*/} 37 </body> 38 </html> 39 ); 40}

In layout.tsx, we will find imports for fonts, which are part of Next.js. Metadata are useful for SEO. I declare only title and description but there's a lot more:

example.typescript
1export const metadata: Metadata = { 2 icons: { 3 icon: "/logo.png" 4 }, 5 applicationName: "name_of_your_app", 6 generator: "Next.ts", 7 title: "tiitle_of_your_app", 8 description: "...", 9 authors: [{name: "..."}, {name: "..."}], 10 keywords: [ 11 "xr", 12 "react", 13 "next.js", 14 "tutorial", 15], 16creator: "Adam Hitzger", 17 publisher: "Adam Hitzger", 18 formatDetection: { 19 email: false, 20 address: false, 21 telephone: false, 22 }, 23openGraph: { 24 title: "...", 25 description: "...", 26 url: "...", 27 siteName: "...", 28 locale: "cs_CZ", 29 type: "website" 30} 31 32};

I have added styles to <main> tag to display content in column with 100% height.

We are done with layout. create a folder called components and file Scene.tsx.

example.tsx
1/components/Scene.tsx 2"use client"; 3 4import { Canvas } from "@react-three/fiber"; 5import { createXRStore, XR } from "@react-three/xr"; 6import { Fullscreen } from "@react-three/uikit"; 7import { Defaults } from "@react-three/uikit-apfel"; 8import { Text } from "@react-three/uikit"; 9import { Card } from "@react-three/uikit-apfel"; 10 11interface WeatherData { 12 weather: { 13 description: string; 14 }[]; 15 main: { 16 temp: number; 17 }; 18 wind: { 19 speed: number; 20 }; 21} 22 23const store = createXRStore({ 24 enterGrantedSession: false, 25}); 26 27export default function Scene({ data }: { data: WeatherData }) { 28 return ( 29 <div className="flex flex-col h-screen"> 30 <button 31 className="relative border-2 border-black-900 rounded-full w-fit p-2 mx-auto" 32 onClick={() => store.enterAR()} 33 > 34 Enter AR 35 </button> 36 <button 37 className="relative border-2 border-black-900 rounded-full w-fit p-2 mx-auto" 38 onClick={() => store.enterVR()} 39 > 40 Enter VR 41 </button> 42 <Canvas className="bg-transparent w-full min-h-screen"> 43 <XR store={store}> 44 <ambientLight intensity={0.5} /> 45 <directionalLight intensity={1} position={[-5, 5, 10]} /> 46 <Defaults> 47 <Fullscreen 48 overflow="scroll" 49 scrollbarColor="black" 50 flexDirection="column" 51 gap={32} 52 paddingX={32} 53 alignItems="center" 54 padding={32} 55 > 56 <Card 57 borderRadius={32} 58 padding={32} 59 gap={8} 60 flexDirection="column" 61 > 62 <Text fontSize={32}>Weather in Jihlava </Text> 63 <Text fontSize={24} opacity={0.7}> 64 {`Desc: ${data.weather[0].description}\nTemp: ${data.main.temp} Celsius\n Wind speed: ${data.wind.speed}`} 65 </Text> 66 <Text fontSize={10}> 67 Created by Adam Hitzger for English Seminar work 68 </Text> 69 </Card> 70 </Fullscreen> 71 </Defaults> 72 </XR> 73 </Canvas> 74 </div> 75 ); 76} 77

Integrating WebXR with React Three Fiber in Next.js

At the top of your component file, include the directive:

example.typescript
1'use client';

This tells Next.js that the component should be rendered on the client side, which is essential when using browser-specific APIs like WebGL, upon which Three.js is built. ​

Next, import the necessary dependencies. The <Canvas> component from React Three Fiber serves as the container for your 3D scene. It sets up the scene and camera, rendering your 3D content within the browser.

To manage the weather data fetched from the OpenWeather API, define an interface that represents the structure of the JSON response. This will help in typing the data passed to your Scene component.​

For initiating an XR session, create an XR store using the createXRStore function. When working with iOS devices, setting the enterGrantedSession parameter can help prevent certain errors related to session initiation. Here's an issue on Github why is it happening.

Pass the fetched weather data to the Scene component as a prop, ensuring it adheres to the defined interface.

Within the return statement of your component, include buttons that allow users to start AR or VR sessions. Inside the <Canvas> component, set up your XR environment, including lighting and 3D objects, to visually represent the weather data.​

Now, open your page.tsx file. Make the component asynchronous to fetch data from the OpenWeather API. Access your API key using process.env.OPEN_WEATHER_API_URL and perform the fetch operation accordingly.

example.tsx
1/app/page.tsx 2import Scene from "@/components/Scene"; 3 4export default async function Home() { 5 const apiKey = process.env.OPEN_WEATHER_API; 6 let data; 7 try{ 8 const res = await fetch(`https://api.openweathermap.org/data/2.5/weather?q=Jihlava&units=metric&appid=${apiKey}`); 9 if(!res.ok){ 10 throw new Error("Chyba při načítání počasí"); 11 } 12 data = await res.json(); 13 console.log(data) 14 } catch(error){ 15 console.error("Error: ",error) 16 17 } 18 return ( 19 <Scene data={data}/> 20 ); 21} 22

Deploying Your WebXR-Enabled Next.js App

After passing the fetched weather data to your Scene component, it's essential to host your application over HTTPS, as WebXR requires a secure context to function correctly. Serving your site over HTTPS ensures that WebXR features are accessible in supported browsers. Serving your site over HTTPS ensures that WebXR features are accessible in supported browsers.

Vercel offers a seamless and efficient way to deploy your Next.js application:​Medium

  1. Sign Up and Create a Project: Visit Vercel and sign up for an account. Once registered, create a new project.​
  2. Import Your GitHub Repository: During project setup, Vercel will prompt you to import an existing GitHub repository. Select the repository containing your Next.js app.​
  3. Automatic Deployment: Vercel automatically detects that you're using Next.js and configures the deployment accordingly. After the initial build, your app will be accessible via a unique Vercel-generated URL.​
  4. Continuous Deployment: Any subsequent commits to your GitHub repository will trigger automatic redeployments, ensuring your live site remains up-to-date.

To test your WebXR application on iOS devices, consider using the XR Browser, an augmented reality viewer designed for iOS.It allows you to run AR experiences built with web technologies.You can download it from the App Store XR browser.

[object Object]

And we're done! I hope you're now able to view the 3D scene and enjoy the new possibilities in the world of the web!

Subscribe to my newsletter !

and be updated with lastest libraries. Enter your email address!