📌 Introduction
Integrating the image capture feature into a React Native video calling app enhances its functionality by allowing users to capture images alongside recording videos. Image Capture enables users to freeze memorable moments during video playback or instantly capture stills while shooting. This feature enriches user experience, offering versatility and convenience within the app.
By seamlessly merging video recording and image capture functionalities, users can efficiently create multimedia content without switching between multiple applications. Furthermore, it provides additional creative possibilities for users, enabling them to express themselves more dynamically through a combination of videos and images within the same platform.
This article explains how to integrate the image capture feature in the React Native Video Calling App. We will guide you through the steps of installing VideoSDK, integrating it into your project, and adding an image capture feature and improve the video viewing experience in your application.
🚀 Getting Started with VideoSDK
To take advantage of the chat functionality, we must use the capabilities that the VideoSDK offers. Before diving into the implementation steps, let's ensure you complete the necessary prerequisites.
Create a VideoSDK Account
Go to your VideoSDK dashboard and sign up if you don't have an account. This account gives you access to the required Video SDK token, which acts as an authentication key that allows your application to interact with VideoSDK functionality.
Generate your Auth Token
Visit your VideoSDK dashboard and navigate to the "API Key" section to generate your auth token. This token is crucial in authorizing your application to use VideoSDK features. For a more visual understanding of the account creation and token generation process, consider referring to the provided tutorial.
Prerequisites and Setup
Make sure your development environment meets the following requirements:
- Node.js v12+
- NPM v6+ (comes installed with newer Node versions)
- Android Studio or Xcode installed
🛠️ Integrate VideoSDK
It is necessary to set up VideoSDK within your project before going into the details of integrating the Image Capture feature. Installing VideoSDK using NPM or Yarn will depend on the needs of your project.
- For NPM
npm install "@videosdk.live/react-native-sdk" "@videosdk.live/react-native-incallmanager"- For Yarn
yarn add "@videosdk.live/react-native-sdk" "@videosdk.live/react-native-incallmanager"Project Configuration
Before integrating the Image Capture functionality, ensure that your project is correctly prepared to handle the integration. This setup consists of a sequence of steps for configuring rights, dependencies, and platform-specific parameters so that VideoSDK can function seamlessly inside your application context.
Android Setup
- Add the required permissions in the
AndroidManifest.xmlfile.
<manifest
xmlns:android="http://schemas.android.com/apk/res/android"
package="com.cool.app"
>
<!-- Give all the required permissions to app -->
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<!-- Needed to communicate with already-paired Bluetooth devices. (Legacy up to Android 11) -->
<uses-permission
android:name="android.permission.BLUETOOTH"
android:maxSdkVersion="30" />
<uses-permission
android:name="android.permission.BLUETOOTH_ADMIN"
android:maxSdkVersion="30" />
<!-- Needed to communicate with already-paired Bluetooth devices. (Android 12 upwards)-->
<uses-permission android:name="android.permission.BLUETOOTH_CONNECT" />
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.SYSTEM_ALERT_WINDOW" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE"/>
<uses-permission android:name="android.permission.WAKE_LOCK" />
<application>
<meta-data
android:name="live.videosdk.rnfgservice.notification_channel_name"
android:value="Meeting Notification"
/>
<meta-data
android:name="live.videosdk.rnfgservice.notification_channel_description"
android:value="Whenever meeting started notification will appear."
/>
<meta-data
android:name="live.videosdk.rnfgservice.notification_color"
android:resource="@color/red"
/>
<service android:name="live.videosdk.rnfgservice.ForegroundService" android:foregroundServiceType="mediaProjection"></service>
<service android:name="live.videosdk.rnfgservice.ForegroundServiceTask"></service>
</application>
</manifest>- Update your
colors.xmlfile for internal dependencies:
<resources>
<item name="red" type="color">
#FC0303
</item>
<integer-array name="androidcolors">
<item>@color/red</item>
</integer-array>
</resources>- Link the necessary VideoSDK Dependencies.
dependencies {
implementation project(':rnwebrtc')
implementation project(':rnfgservice')
}include ':rnwebrtc'
project(':rnwebrtc').projectDir = new File(rootProject.projectDir, '../node_modules/@videosdk.live/react-native-webrtc/android')
include ':rnfgservice'
project(':rnfgservice').projectDir = new File(rootProject.projectDir, '../node_modules/@videosdk.live/react-native-foreground-service/android')import live.videosdk.rnwebrtc.WebRTCModulePackage;
import live.videosdk.rnfgservice.ForegroundServicePackage;
public class MainApplication extends Application implements ReactApplication {
private static List<ReactPackage> getPackages() {
@SuppressWarnings("UnnecessaryLocalVariable")
List<ReactPackage> packages = new PackageList(this).getPackages();
// Packages that cannot be autolinked yet can be added manually here, for example:
packages.add(new ForegroundServicePackage());
packages.add(new WebRTCModulePackage());
return packages;
}
}/* This one fixes a weird WebRTC runtime problem on some devices. */
android.enableDexingArtifactTransform.desugaring=false- Include the following line in your
proguard-rules.profile (optional: if you are using Proguard)
-keep class org.webrtc.** { *; }- In your
build.gradlefile, update the minimum OS/SDK version to23.
buildscript {
ext {
minSdkVersion = 23
}
}iOS Setup
Ensure that you are using CocoaPods version 1.10 or later.
1. To update CocoaPods, you can reinstall the gem using the following command:
$ sudo gem install cocoapods2. Manually link react-native-incall-manager (if it is not linked automatically).
Select Your_Xcode_Project/TARGETS/BuildSettings, in Header Search Paths, add "$(SRCROOT)/../node_modules/@videosdk.live/react-native-incall-manager/ios/RNInCallManager"
3. Change the path of react-native-webrtc using the following command:
pod ‘react-native-webrtc’, :path => ‘../node_modules/@videosdk.live/react-native-webrtc’4. Change the version of your platform.
You need to change the platform field in the Podfile to 12.0 or above because react-native-webrtc doesn't support iOS versions earlier than 12.0. Update the line: platform: ios, ‘12.0’.
5. Install pods.
After updating the version, you need to install the pods by running the following command:
Pod install6. Add “libreact-native-webrtc.a” binary.
Add the "libreact-native-webrtc.a" binary to the "Link Binary With Libraries" section in the target of your main project folder.
7. Declare permissions in Info.plist :
Add the following lines to your info.plist file located at:
<key>NSCameraUsageDescription</key>
<string>Camera permission description</string>
<key>NSMicrophoneUsageDescription</key>
<string>Microphone permission description</string>Register Service
Register VideoSDK services in your root index.js file for the initialization service.
import { AppRegistry } from "react-native";
import App from "./App";
import { name as appName } from "./app.json";
import { register } from "@videosdk.live/react-native-sdk";
register();
AppRegistry.registerComponent(appName, () => App);🎥 Essential Steps for Building the Video Calling
By following essential steps, you can seamlessly implement video into your applications with VideoSDK, which provides a robust set of tools and APIs to facilitate the integration of video capabilities into applications.
Step 1: Get started with api.js
Before moving on, you must create an API request to generate a unique meetingId. You will need an authentication token, which you can create either through the videosdk-rtc-api-server-examples or directly from the VideoSDK Dashboard for developers.
export const token = "<Generated-from-dashbaord>";
// API call to create meeting
export const createMeeting = async ({ token }) => {
const res = await fetch(`https://api.videosdk.live/v2/rooms`, {
method: "POST",
headers: {
authorization: `${token}`,
"Content-Type": "application/json",
},
body: JSON.stringify({}),
});
const { roomId } = await res.json();
return roomId;
};Step 2: Wireframe App.js with all the components
To build up a wireframe of App.js, you need to use VideoSDK Hooks and Context Providers. VideoSDK provides MeetingProvider, MeetingConsumer, useMeeting, and useParticipant hooks.
First, you need to understand the Context of Provider and Consumer. Context is primarily used when some data needs to be accessible by many components at different nesting levels.
- MeetingProvider: This is the Context Provider. It accepts value
configandtokenas props. The Provider component accepts a value prop to be passed to consuming components that are descendants of this Provider. One Provider can be connected to many consumers. Providers can be nested to override values deeper within the tree. - MeetingConsumer: This is the Context Consumer. All consumers that are descendants of a Provider will re-render whenever the Provider’s value prop changes.
- useMeeting: This is the meeting hook API. It includes all the information related to meeting such as join/leave, enable/disable the mic or webcam, etc.
- useParticipant: This is the participant hook API. It is responsible for handling all the events and props related to one particular participant such as name, webcamStream, micStream, etc.
The Meeting Context provides a way to listen for any changes that occur when a participant joins the meeting or makes modifications to their microphone, camera, and other settings.
Begin by making a few changes to the code in the App.js file.
import React, { useState } from "react";
import {
SafeAreaView,
TouchableOpacity,
Text,
TextInput,
View,
FlatList,
} from "react-native";
import {
MeetingProvider,
useMeeting,
useParticipant,
MediaStream,
RTCView,
} from "@videosdk.live/react-native-sdk";
import { createMeeting, token } from "./api";
function JoinScreen(props) {
return null;
}
function ControlsContainer() {
return null;
}
function MeetingView() {
return null;
}
export default function App() {
const [meetingId, setMeetingId] = useState(null);
const getMeetingId = async (id) => {
const meetingId = id == null ? await createMeeting({ token }) : id;
setMeetingId(meetingId);
};
return meetingId ? (
<SafeAreaView style={{ flex: 1, backgroundColor: "#F6F6FF" }}>
<MeetingProvider
config={{
meetingId,
micEnabled: false,
webcamEnabled: true,
name: "Test User",
}}
token={token}
>
<MeetingView />
</MeetingProvider>
</SafeAreaView>
) : (
<JoinScreen getMeetingId={getMeetingId} />
);
}Step 3: Implement Join Screen
The join screen will serve as a medium to either schedule a new meeting or join an existing one.
function JoinScreen(props) {
const [meetingVal, setMeetingVal] = useState("");
return (
<SafeAreaView
style={{
flex: 1,
backgroundColor: "#F6F6FF",
justifyContent: "center",
paddingHorizontal: 6 * 10,
}}
>
<TouchableOpacity
onPress={() => {
props.getMeetingId();
}}
style={{ backgroundColor: "#1178F8", padding: 12, borderRadius: 6 }}
>
<Text style={{ color: "white", alignSelf: "center", fontSize: 18 }}>
Create Meeting
</Text>
</TouchableOpacity>
<Text
style={{
alignSelf: "center",
fontSize: 22,
marginVertical: 16,
fontStyle: "italic",
color: "grey",
}}
>
---------- OR ----------
</Text>
<TextInput
value={meetingVal}
onChangeText={setMeetingVal}
placeholder={"XXXX-XXXX-XXXX"}
style={{
padding: 12,
borderWidth: 1,
borderRadius: 6,
fontStyle: "italic",
}}
/>
<TouchableOpacity
style={{
backgroundColor: "#1178F8",
padding: 12,
marginTop: 14,
borderRadius: 6,
}}
onPress={() => {
props.getMeetingId(meetingVal);
}}
>
<Text style={{ color: "white", alignSelf: "center", fontSize: 18 }}>
Join Meeting
</Text>
</TouchableOpacity>
</SafeAreaView>
);
}Step 4: Implement Controls
The next step is to create a ControlsContainer component to manage features such as Join or leave a Meeting and Enable or Disable the Webcam/Mic.
In this step, the useMeeting hook is utilized to acquire all the required methods such as join(), leave(), toggleWebcam and toggleMic.
const Button = ({ onPress, buttonText, backgroundColor }) => {
return (
<TouchableOpacity
onPress={onPress}
style={{
backgroundColor: backgroundColor,
justifyContent: "center",
alignItems: "center",
padding: 12,
borderRadius: 4,
}}
>
<Text style={{ color: "white", fontSize: 12 }}>{buttonText}</Text>
</TouchableOpacity>
);
};
function ControlsContainer({ join, leave, toggleWebcam, toggleMic }) {
return (
<View
style={{
padding: 24,
flexDirection: "row",
justifyContent: "space-between",
}}
>
<Button
onPress={() => {
join();
}}
buttonText={"Join"}
backgroundColor={"#1178F8"}
/>
<Button
onPress={() => {
toggleWebcam();
}}
buttonText={"Toggle Webcam"}
backgroundColor={"#1178F8"}
/>
<Button
onPress={() => {
toggleMic();
}}
buttonText={"Toggle Mic"}
backgroundColor={"#1178F8"}
/>
<Button
onPress={() => {
leave();
}}
buttonText={"Leave"}
backgroundColor={"#FF0000"}
/>
</View>
);
}function ParticipantList() {
return null;
}
function MeetingView() {
const { join, leave, toggleWebcam, toggleMic, meetingId } = useMeeting({});
return (
<View style={{ flex: 1 }}>
{meetingId ? (
<Text style={{ fontSize: 18, padding: 12 }}>
Meeting Id :{meetingId}
</Text>
) : null}
<ParticipantList />
<ControlsContainer
join={join}
leave={leave}
toggleWebcam={toggleWebcam}
toggleMic={toggleMic}
/>
</View>
);
}Step 5: Render Participant List
After implementing the controls, the next step is to render the joined participants.
You can get all the joined participants from the useMeeting Hook.
function ParticipantView() {
return null;
}
function ParticipantList({ participants }) {
return participants.length > 0 ? (
<FlatList
data={participants}
renderItem={({ item }) => {
return <ParticipantView participantId={item} />;
}}
/>
) : (
<View
style={{
flex: 1,
backgroundColor: "#F6F6FF",
justifyContent: "center",
alignItems: "center",
}}
>
<Text style={{ fontSize: 20 }}>Press Join button to enter meeting.</Text>
</View>
);
}function MeetingView() {
// Get `participants` from useMeeting Hook
const { join, leave, toggleWebcam, toggleMic, participants } = useMeeting({});
const participantsArrId = [...participants.keys()];
return (
<View style={{ flex: 1 }}>
<ParticipantList participants={participantsArrId} />
<ControlsContainer
join={join}
leave={leave}
toggleWebcam={toggleWebcam}
toggleMic={toggleMic}
/>
</View>
);
}Step 6: Handling Participant's Media
Before Handling the Participant's Media, you need to understand a couple of concepts.
1. useParticipant Hook
The useParticipant hook is responsible for handling all the properties and events of one particular participant who joined the meeting. It will take participantId as argument.
const { webcamStream, webcamOn, displayName } = useParticipant(participantId);2. MediaStream API
The MediaStream API is beneficial for adding a MediaTrack into the RTCView component, enabling the playback of audio or video.
<RTCView
streamURL={new MediaStream([webcamStream.track]).toURL()}
objectFit={"cover"}
style={{
height: 300,
marginVertical: 8,
marginHorizontal: 8,
}}
/>Rendering Participant Media
function ParticipantView({ participantId }) {
const { webcamStream, webcamOn } = useParticipant(participantId);
return webcamOn && webcamStream ? (
<RTCView
streamURL={new MediaStream([webcamStream.track]).toURL()}
objectFit={"cover"}
style={{
height: 300,
marginVertical: 8,
marginHorizontal: 8,
}}
/>
) : (
<View
style={{
backgroundColor: "grey",
height: 300,
justifyContent: "center",
alignItems: "center",
}}
>
<Text style={{ fontSize: 16 }}>NO MEDIA</Text>
</View>
);
}Congratulations! By following these steps, you're on your way to unlocking the video within your application. Now, we are moving forward to integrate the feature that builds immersive video experiences for your users!
📸 Integrate Image Capture Feature
This function proves particularly valuable in Video KYC scenarios, enabling the capture of images where users can hold up their identity for verification.
- By using the
captureImage()function of theuseParticipanthook, you can capture an image of a local participant from their video stream. - You have the option to specify the desired height and width in the
captureImage()function; however, these parameters are optional. If not provided, the VideoSDK will automatically use the dimensions of the local participant's webcamStream. - The
captureImage()function returns the image in the form of abase64string.
import { useMeeting, useParticipant } from '@videosdk.live/react-native-sdk';
const {localParticipant} = useMeeting()
const { webcamStream, webcamOn, captureImage } = useParticipant(localParticipant.id);
async function imageCapture() {
if (webcamOn && webcamStream) {
const base64 = await captureImage({height:400,width:400}); // captureImage will return base64 string
console.log("base64",base64);
} else {
console.error("Camera must be on to capture an image");
}
}You can only capture an image of a local participant. If you call captureImage() the function on a remote participant, you will receive an error. If you want to capture an image of a remote participant, you can follow the below documentation.How to Capture Images of remote participants?
- Before proceeding, it's crucial to understand VideoSDK's temporary file storage system and the underlying pubSub mechanism.
- Here's a breakdown of the steps, using the names Participant A and Participant B for clarity:
Step 1: Initiate Image Capture Request
- In this step, you have to first send a request to Participant B, whose image you want to capture, using pubSub.
- To do that, you have to create a pubSub topic called
IMAGE_CAPTUREin theParticipantViewComponent. - Here, you will be using the
sendOnlyproperty of thepublish()method. Therefore, the request will be sent to that participant only.
import {usePubSub} from '@videosdk.live/react-native-sdk';
import {
TouchableOpacity,
Text
} from 'react-native';
function ParticipantView({ participantId }) {
// create pubsub topic to send Request
const { publish } = usePubSub('IMAGE_CAPTURE');
// send Request to participant
function sendRequest() {
// Pass the participantId of the participant whose image you want to capture
// Here, it will be Participant B's id, as you want to capture the image of Participant B
publish("Sending request to capture image", { persist: false, sendOnly: [participantId] });
};
return <>
// other components
<TouchableOpacity style={{ width: 80, height : 45, backgroundColor: 'red', position: 'absolute', top: 10 }} onPress={() => {
sendRequest()
}}>
<Text style={{ fontSize: 15, color: 'white', left:10 }}>
Capture Image
</Text>
</TouchableOpacity>
</>;
}
Step 2: Capture and Upload File
- To capture images from remote participant [Participant B], we've created the
CaptureImageListenercomponent. When a participant receives an image capture request, this component uses thecaptureImagefunction ofuseParticipanthook to capture the image.
import {
useFile,
usePubSub,
useParticipant
} from '@videosdk.live/react-native-sdk';
const CaptureImageListner = ({ localParticipantId }) => {
const { captureImage } = useParticipant(localParticipantId);
// subscribe to receive request
usePubSub('IMAGE_CAPTURE', {
onMessageReceived: (message) => {
_handleOnImageCaptureMessageReceived(message);
},
});
const _handleOnImageCaptureMessageReceived = (message) => {
try {
if (message.senderId !== localParticipantId) {
// capture and store image when message received
captureAndStoreImage({ senderId: message.senderId });
}
} catch (err) {
console.log("error on image capture", err);
}
};
async function captureAndStoreImage({ senderId }) {
// capture image
const base64Data = await captureImage({height:400,width:400});
console.log('base64Data',base64Data);
}
return <></>;
};
export default CaptureImageListner;- The captured image is then stored in the VideoSDK's temporary file storage system using the
uploadBase64File()function of theuseFilehook. This operation returns a uniquefileUrlof the stored image.
const CaptureImageListner = ({ localParticipantId }) => {
const { uploadBase64File } = useFile();
async function captureAndStoreImage({ senderId }) {
// capture image
const base64Data = await captureImage({height:400,width:400});
const token = "<YOUR-TOKEN>";
const fileName = "myCapture.jpeg"; // specify a name for image file with extension
// upload image to videosdk storage system
const fileUrl = await uploadBase64File({base64Data,token,fileName});
console.log('fileUrl',fileUrl)
}
//...
}- Next, the
fileUrlis sent back to the participant who initiated the request using theIMAGE_TRANSFERtopic.
const CaptureImageListner = ({ localParticipantId }) => {
//...
// publish image Transfer
const { publish: imageTransferPublish } = usePubSub('IMAGE_TRANSFER');
async function captureAndStoreImage({ senderId }) {
//...
const fileUrl = await uploadBase64File({base64Data,token,fileName});
imageTransferPublish(fileUrl, { persist: false , sendOnly: [senderId] });
}
//...
}- Then the
CaptureImageListenercomponent has to be rendered within theMeetingViewcomponent.
import CaptureImageListner from './captureImageListner';
import {useMeeting} from '@videosdk.live/react-native-sdk';
function MeetingView() {
//...
// Get `localParticipant` from useMeeting Hook
const {localParticipant } = useMeeting({});
return (
<View>
// other components
<CaptureImageListner localParticipantId={localParticipant?.id} />
</View>
);
}Step 3: Fetch and Display Image
- To display a captured image, the
ShowImagecomponent is used. Here's how it works: - Within
ShowImage, you need to subscribe to theIMAGE_TRANSFERtopic, receiving thefileUrlassociated with the captured image. Once obtained, leverage thefetchBase64File()function from theuseFilehook to retrieve the file inbase64format from VideoSDK's temporary storage.
import {
useMeeting,
useFile
} from '@videosdk.live/react-native-sdk';
function ShowImage() {
const mMeeting = useMeeting();
const { fetchBase64File } = useFile();
const topicTransfer = "IMAGE_TRANSFER";
const [bitMapImg, setbitMapImg] = useState(null);
usePubSub(topicTransfer, {
onMessageReceived: (message) => {
if (message.senderId !== mMeeting.localParticipant.id) {
fetchFile({ url: message.message }); // pass fileUrl to fetch the file
}
}
});
async function fetchFile({ url }) {
const token = "<YOUR-TOKEN>";
const base64 = await fetchBase64File({ url, token });
console.log("base64",base64); // here is your image in a form of base64
setbitMapImg(base64);
}
}- With the
base64data in hand, you can now display the image in a modal. This seamless image presentation is integrated into theMeetingViewcomponent.
import {
Image,
Modal,
Pressable
} from 'react-native';
function ShowImage() {
//...
return <>
{bitMapImg ? (
<View>
<Modal animationType={"slide"} transparent={false} >
<View style={{
flex: 1,
flexDirection: 'column',
justifyContent: 'center',
alignItems: 'center'
}}>
<View>
<Image
style={{ height: 400, width: 300, objectFit: "contain" }}
source={{ uri: `data:image/jpeg;base64,${bitMapImg}` }}
/>
<Pressable
onPress={() => setbitMapImg(null)}>
<Text style={{color:"black"}}>Close Dialog</Text>
</Pressable>
</View>
</View>
</Modal>
</View>
) : null}
</>;
}function MeetingView() {
// ...
return (
<View>
// other componets
<CaptureImageListner localParticipantId={localParticipant?.id} />
<ShowImage />
</View>
);
}Congratulations! By successfully integrating the Image Capture feature, developers can enhance the immersive video experience for users within their applications.
The file stored in the VideoSDK's temporary file storage system will be automatically deleted once the current room/meeting comes to an end.
🔚 Conclusion
In summary, integrating Image Capture into the React Native video calling app not only enhances its functionality but also opens up various possibilities for users to create, share, and engage with multimedia content in diverse scenarios.
Developers can simply integrate the Image Capture functionality into their apps, whether for identity verification (allowing additional functionality such as Video KYC), content creation, or collaborative experiences, developers can effortlessly integrate VideoSDK into their apps and use the capabilities of this feature to improve user experiences.
To unlock the full potential of VideoSDK and create easy-to-use video experiences, developers are encouraged to Sign up with VideoSDK today and Get 10000 minutes free to take the video app to the next level!
