Installation

Expo

1. Install the packages

   npx expo install react-native-mediapipe-posedetection react-native-vision-camera react-native-worklets-core

2. Add the worklets Babel plugin

   Frame processors run as worklets, so babel.config.js needs the worklets plugin:

   babel.config.js

   module.exports = function (api) {
     api.cache(true);
     return {
       presets: ['babel-preset-expo'],
       plugins: [['react-native-worklets-core/plugin']],
     };
   };

3. Configure the plugins in app.json

   app.json

   {
     "expo": {
       "plugins": [
         [
           "react-native-vision-camera",
           {
             "cameraPermissionText": "$(PRODUCT_NAME) uses the camera to track your body pose."
           }
         ],
         [
           "react-native-mediapipe-posedetection",
           {
             "assetsPaths": ["./assets/models/"]
           }
         ]
       ]
     }
   }

   The second plugin copies every file in assets/models/ into the native projects. See Model files for which model to download.

4. Build a development client

   This library contains native code, so it won’t run in Expo Go.

   npx expo prebuild --clean
   npx expo run:ios     # or: npx expo run:android

   Or build with EAS: eas build --profile development.

Bare React Native

1. Install the packages

   npm install react-native-mediapipe-posedetection react-native-vision-camera react-native-worklets-core

2. Add the worklets Babel plugin

   babel.config.js

   module.exports = {
     presets: ['module:@react-native/babel-preset'],
     plugins: [['react-native-worklets-core/plugin']],
   };

3. Make sure the New Architecture is on

   It’s the default from React Native 0.76. On older versions:

   android/gradle.properties

   newArchEnabled=true

   cd ios && RCT_NEW_ARCH_ENABLED=1 pod install

4. Add camera permissions

   ios/YourApp/Info.plist

   <key>NSCameraUsageDescription</key>
   <string>$(PRODUCT_NAME) uses the camera to track your body pose.</string>

   android/app/src/main/AndroidManifest.xml

   <uses-permission android:name="android.permission.CAMERA" />

5. Install pods and add a model file

   cd ios && pod install

   Then add a .task model to both platforms as described in Model files.

6. Rebuild the app

   Native code changed, so do a full rebuild rather than a Metro reload.

Ask for camera permission at runtime

VisionCamera needs permission before the camera will start:

import { useCameraPermission } from 'react-native-vision-camera';

const { hasPermission, requestPermission } = useCameraPermission();

useEffect(() => {
  if (!hasPermission) requestPermission();
}, [hasPermission, requestPermission]);

Next: get a model file.