📌 Introduction
Picture-in-Picture (PiP) is a commonly used feature in video conferencing software, enabling users to simultaneously engage in a video conference and perform other tasks on their devices. With PiP, you can keep the video conference window open, resize it to a smaller size, and continue working on other tasks while still seeing and hearing the other participants in the conference. This feature proves beneficial when you need to take notes, send an email, or look up information during the conference.
PiP mode is very effective in apps that require users to watch a video while engaging with other information or features. For example, in a video conferencing program, users can continue their video call in a tiny window while accessing other elements of the application, such as chat or file sharing. Students on an e-learning platform can view a lecture video while taking notes or accessing other information.
PiP mode is effective because it improves the user experience without interfering with their workflow. PiP mode increases the flexibility and usability of applications by allowing users to multitask and access additional functions while watching a video. This can result in greater user engagement, satisfaction, and, ultimately, success for your application.
In this post, we'll explore how to implement Picture-in-Picture(PIP) into a React Native video calling app. We will guide you through the steps of installing VideoSDK, integrating it into your project, and adding PiP mode to improve the video viewing experience in your application.
🚀 Getting Started with VideoSDK
To take advantage of the chat functionality, we must use the capabilities that the VideoSDK offers. Before diving into the implementation steps, let's ensure you complete the necessary prerequisites.
Create a VideoSDK Account
Go to your VideoSDK dashboard and sign up if you don't have an account. This account gives you access to the required Video SDK token, which acts as an authentication key that allows your application to interact with VideoSDK functionality.
Generate your Auth Token
Visit your VideoSDK dashboard and navigate to the "API Key" section to generate your auth token. This token is crucial in authorizing your application to use VideoSDK features.
For a more visual understanding of the account creation and token generation process, consider referring to the provided tutorial.
Prerequisites and Setup
Make sure your development environment meets the following requirements:
- Node.js v12+
- NPM v6+ (comes installed with newer Node versions)
- Android Studio or Xcode installed
🛠️ Integrate VideoSDK
Install the VideoSDK by using the following command. Ensure that you are in your project directory before running this command.
- For NPM
npm install "@videosdk.live/react-native-sdk" "@videosdk.live/react-native-incallmanager"- For Yarn
yarn add "@videosdk.live/react-native-sdk" "@videosdk.live/react-native-incallmanager"Project Configuration
Setting up your project correctly is the next essential phase in ensuring smooth operation and integration. Setting up permissions, updating required files, and connecting dependencies are all part of project configuration, which helps to make sure your application can make the most out of VideoSDK capabilities.
Android Setup
- Add the required permissions in the
AndroidManifest.xmlfile.
<manifest
xmlns:android="http://schemas.android.com/apk/res/android"
package="com.cool.app"
>
<!-- Give all the required permissions to app -->
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<!-- Needed to communicate with already-paired Bluetooth devices. (Legacy up to Android 11) -->
<uses-permission
android:name="android.permission.BLUETOOTH"
android:maxSdkVersion="30" />
<uses-permission
android:name="android.permission.BLUETOOTH_ADMIN"
android:maxSdkVersion="30" />
<!-- Needed to communicate with already-paired Bluetooth devices. (Android 12 upwards)-->
<uses-permission android:name="android.permission.BLUETOOTH_CONNECT" />
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.SYSTEM_ALERT_WINDOW" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE"/>
<uses-permission android:name="android.permission.WAKE_LOCK" />
<application>
<meta-data
android:name="live.videosdk.rnfgservice.notification_channel_name"
android:value="Meeting Notification"
/>
<meta-data
android:name="live.videosdk.rnfgservice.notification_channel_description"
android:value="Whenever meeting started notification will appear."
/>
<meta-data
android:name="live.videosdk.rnfgservice.notification_color"
android:resource="@color/red"
/>
<service android:name="live.videosdk.rnfgservice.ForegroundService" android:foregroundServiceType="mediaProjection"></service>
<service android:name="live.videosdk.rnfgservice.ForegroundServiceTask"></service>
</application>
</manifest>- Update your
colors.xmlfile for internal dependencies:
<resources>
<item name="red" type="color">
#FC0303
</item>
<integer-array name="androidcolors">
<item>@color/red</item>
</integer-array>
</resources>- Link the necessary VideoSDK Dependencies:
dependencies {
implementation project(':rnwebrtc')
implementation project(':rnfgservice')
}include ':rnwebrtc'
project(':rnwebrtc').projectDir = new File(rootProject.projectDir, '../node_modules/@videosdk.live/react-native-webrtc/android')
include ':rnfgservice'
project(':rnfgservice').projectDir = new File(rootProject.projectDir, '../node_modules/@videosdk.live/react-native-foreground-service/android')import live.videosdk.rnwebrtc.WebRTCModulePackage;
import live.videosdk.rnfgservice.ForegroundServicePackage;
public class MainApplication extends Application implements ReactApplication {
private static List<ReactPackage> getPackages() {
@SuppressWarnings("UnnecessaryLocalVariable")
List<ReactPackage> packages = new PackageList(this).getPackages();
// Packages that cannot be autolinked yet can be added manually here, for example:
packages.add(new ForegroundServicePackage());
packages.add(new WebRTCModulePackage());
return packages;
}
}/* This one fixes a weird WebRTC runtime problem on some devices. */
android.enableDexingArtifactTransform.desugaring=false- Include the following line in your
proguard-rules.profile (optional: if you are using Proguard)
-keep class org.webrtc.** { *; }- In your
build.gradlefile, update the minimum OS/SDK version to23.
buildscript {
ext {
minSdkVersion = 23
}
}Register Service
Register VideoSDK services in your root index.js file for the initialization service.
import { AppRegistry } from "react-native";
import App from "./App";
import { name as appName } from "./app.json";
import { register } from "@videosdk.live/react-native-sdk";
register();
AppRegistry.registerComponent(appName, () => App);🎥 Essential Steps for Building the Video Calling
By following essential steps, you can seamlessly implement video into your applications with VideoSDK, which provides a robust set of tools and APIs to facilitate the integration of video capabilities into applications.
Step 1: Get started with api.js
Before moving on, you must create an API request to generate a unique meetingId. You will need an authentication token, which you can create either through the videosdk-rtc-api-server-examples or directly from the VideoSDK Dashboard for developers.
export const token = "<Generated-from-dashbaord>";
// API call to create meeting
export const createMeeting = async ({ token }) => {
const res = await fetch(`https://api.videosdk.live/v2/rooms`, {
method: "POST",
headers: {
authorization: `${token}`,
"Content-Type": "application/json",
},
body: JSON.stringify({}),
});
const { roomId } = await res.json();
return roomId;
};Step 2: Wireframe App.js with all the components
To build up a wireframe of App.js, you need to use VideoSDK Hooks and Context Providers. VideoSDK provides MeetingProvider, MeetingConsumer, useMeeting, and useParticipant hooks.
First, you need to understand the Context Provider and Consumer. Context is primarily used when some data needs to be accessible by many components at different nesting levels.
- MeetingProvider: This is the Context Provider. It accepts value
configandtokenas props. The Provider component accepts a value prop to be passed to consuming components that are descendants of this Provider. One Provider can be connected to many consumers. Providers can be nested to override values deeper within the tree. - MeetingConsumer: This is the Context Consumer. All consumers that are descendants of a Provider will re-render whenever the Provider’s value prop changes.
- useMeeting: This is the meeting hook API. It includes all the information related to meetings such as join/leave, enable/disable the mic or webcam, etc.
- useParticipant: This is the participant hook API. It is responsible for handling all the events and props related to one particular participant such as name, webcamStream, micStream, etc.
The Meeting Context provides a way to listen for any changes that occur when a participant joins the meeting or makes modifications to their microphone, camera, and other settings.
Begin by making a few changes to the code in the App.js file.
import React, { useState } from "react";
import {
SafeAreaView,
TouchableOpacity,
Text,
TextInput,
View,
FlatList,
} from "react-native";
import {
MeetingProvider,
useMeeting,
useParticipant,
MediaStream,
RTCView,
} from "@videosdk.live/react-native-sdk";
import { createMeeting, token } from "./api";
function JoinScreen(props) {
return null;
}
function ControlsContainer() {
return null;
}
function MeetingView() {
return null;
}
export default function App() {
const [meetingId, setMeetingId] = useState(null);
const getMeetingId = async (id) => {
const meetingId = id == null ? await createMeeting({ token }) : id;
setMeetingId(meetingId);
};
return meetingId ? (
<SafeAreaView style={{ flex: 1, backgroundColor: "#F6F6FF" }}>
<MeetingProvider
config={{
meetingId,
micEnabled: false,
webcamEnabled: true,
name: "Test User",
}}
token={token}
>
<MeetingView />
</MeetingProvider>
</SafeAreaView>
) : (
<JoinScreen getMeetingId={getMeetingId} />
);
}Step 3: Implement Join Screen
The join screen will serve as a medium to either schedule a new meeting or join an existing one.
function JoinScreen(props) {
const [meetingVal, setMeetingVal] = useState("");
return (
<SafeAreaView
style={{
flex: 1,
backgroundColor: "#F6F6FF",
justifyContent: "center",
paddingHorizontal: 6 * 10,
}}
>
<TouchableOpacity
onPress={() => {
props.getMeetingId();
}}
style={{ backgroundColor: "#1178F8", padding: 12, borderRadius: 6 }}
>
<Text style={{ color: "white", alignSelf: "center", fontSize: 18 }}>
Create Meeting
</Text>
</TouchableOpacity>
<Text
style={{
alignSelf: "center",
fontSize: 22,
marginVertical: 16,
fontStyle: "italic",
color: "grey",
}}
>
---------- OR ----------
</Text>
<TextInput
value={meetingVal}
onChangeText={setMeetingVal}
placeholder={"XXXX-XXXX-XXXX"}
style={{
padding: 12,
borderWidth: 1,
borderRadius: 6,
fontStyle: "italic",
}}
/>
<TouchableOpacity
style={{
backgroundColor: "#1178F8",
padding: 12,
marginTop: 14,
borderRadius: 6,
}}
onPress={() => {
props.getMeetingId(meetingVal);
}}
>
<Text style={{ color: "white", alignSelf: "center", fontSize: 18 }}>
Join Meeting
</Text>
</TouchableOpacity>
</SafeAreaView>
);
}Step 4: Implement Controls
The next step is to create a ControlsContainer component to manage features such as Join/leave a Meeting and Enable/Disable the Webcam or Mic.
In this step, the useMeeting hook is utilized to acquire all the required methods such as join(), leave(), toggleWebcam and toggleMic.
const Button = ({ onPress, buttonText, backgroundColor }) => {
return (
<TouchableOpacity
onPress={onPress}
style={{
backgroundColor: backgroundColor,
justifyContent: "center",
alignItems: "center",
padding: 12,
borderRadius: 4,
}}
>
<Text style={{ color: "white", fontSize: 12 }}>{buttonText}</Text>
</TouchableOpacity>
);
};
function ControlsContainer({ join, leave, toggleWebcam, toggleMic }) {
return (
<View
style={{
padding: 24,
flexDirection: "row",
justifyContent: "space-between",
}}
>
<Button
onPress={() => {
join();
}}
buttonText={"Join"}
backgroundColor={"#1178F8"}
/>
<Button
onPress={() => {
toggleWebcam();
}}
buttonText={"Toggle Webcam"}
backgroundColor={"#1178F8"}
/>
<Button
onPress={() => {
toggleMic();
}}
buttonText={"Toggle Mic"}
backgroundColor={"#1178F8"}
/>
<Button
onPress={() => {
leave();
}}
buttonText={"Leave"}
backgroundColor={"#FF0000"}
/>
</View>
);
}function ParticipantList() {
return null;
}
function MeetingView() {
const { join, leave, toggleWebcam, toggleMic, meetingId } = useMeeting({});
return (
<View style={{ flex: 1 }}>
{meetingId ? (
<Text style={{ fontSize: 18, padding: 12 }}>
Meeting Id :{meetingId}
</Text>
) : null}
<ParticipantList />
<ControlsContainer
join={join}
leave={leave}
toggleWebcam={toggleWebcam}
toggleMic={toggleMic}
/>
</View>
);
}Step 5: Render Participant List
After implementing the controls, the next step is to render the joined participants in ParticipantList Component.
You can get all the joined participants from the useMeeting Hook.
function ParticipantView() {
return null;
}
function ParticipantList({ participants }) {
return participants.length > 0 ? (
<FlatList
data={participants}
renderItem={({ item }) => {
return <ParticipantView participantId={item} />;
}}
/>
) : (
<View
style={{
flex: 1,
backgroundColor: "#F6F6FF",
justifyContent: "center",
alignItems: "center",
}}
>
<Text style={{ fontSize: 20 }}>Press Join button to enter meeting.</Text>
</View>
);
}function MeetingView() {
// Get `participants` from useMeeting Hook
const { join, leave, toggleWebcam, toggleMic, participants } = useMeeting({});
const participantsArrId = [...participants.keys()];
return (
<View style={{ flex: 1 }}>
<ParticipantList participants={participantsArrId} />
<ControlsContainer
join={join}
leave={leave}
toggleWebcam={toggleWebcam}
toggleMic={toggleMic}
/>
</View>
);
}Step 6: Handling Participant's Media
Before Handling the Participant's Media, you need to understand a couple of concepts.
1. useParticipant Hook
The useParticipant hook is responsible for handling all the properties and events of one particular participant who joined the meeting. It will take participantId as argument.
const { webcamStream, webcamOn, displayName } = useParticipant(participantId);2. MediaStream API
The MediaStream API is beneficial for adding a MediaTrack into the RTCView component, enabling the playback of audio or video.
<RTCView
streamURL={new MediaStream([webcamStream.track]).toURL()}
objectFit={"cover"}
style={{
height: 300,
marginVertical: 8,
marginHorizontal: 8,
}}
/>Rendering Participant Media
function ParticipantView({ participantId }) {
const { webcamStream, webcamOn } = useParticipant(participantId);
return webcamOn && webcamStream ? (
<RTCView
streamURL={new MediaStream([webcamStream.track]).toURL()}
objectFit={"cover"}
style={{
height: 300,
marginVertical: 8,
marginHorizontal: 8,
}}
/>
) : (
<View
style={{
backgroundColor: "grey",
height: 300,
justifyContent: "center",
alignItems: "center",
}}
>
<Text style={{ fontSize: 16 }}>NO MEDIA</Text>
</View>
);
}Congratulations! By following these steps, you're on your way to unlocking the video within your application. Now, we are moving forward to integrate the feature that builds immersive video experiences for your users!
🔍 Integrate Picture-in-picture (PiP) Feature
Picture-in-picture (PiP) is a commonly used feature in video conferencing software, enabling users to simultaneously engage in a video conference and perform other tasks on their devices. With PiP, you can keep the video conference window open, resize it to a smaller size, and continue working on other tasks while still seeing and hearing the other participants in the conference. This feature proves beneficial when you need to take notes, send an email, or look up information during the conference.
This guide explains the steps to implement the Picture-in-Picture feature using VideoSDK.
Step 1: Install Package
To begin with, you need to install a third-party package react-native-pip-android to achieve PiP mode in Android.
npm install react-native-pip-androidStep 2: Setup
Include the following attribute in /android/app/src/main/AndroidManifest.xml file.
<activity
...
android:supportsPictureInPicture="true"
android:configChanges=
"screenSize|smallestScreenSize|screenLayout|orientation"
...Step 3: Import Activity in MainActivity.java file
MainActivity.java file...
import com.reactnativepipandroid.PipAndroidModule;
public class MainActivity extends ReactActivity {
...
@Override
public void onPictureInPictureModeChanged (boolean isInPictureInPictureMode) {
PipAndroidModule.pipModeChanged(isInPictureInPictureMode);
}Step 4: Setup PiP for Rendering participant media
To set up Picture-in-Picture (PiP) mode for rendering participant media, you can utilize the usePipModeListener hook to control the rendering. The example below demonstrates how to render a participant list.
import PipHandler, { usePipModeListener } from "react-native-pip-android";
function ParticipantView({ participantId, inPipMode }) {
const { webcamStream, webcamOn } = useParticipant(participantId);
return webcamOn && webcamStream ? (
<RTCView
streamURL={new MediaStream([webcamStream.track]).toURL()}
objectFit={"cover"}
style={{
height: inPipMode ? 75 : 300,
marginVertical: 8,
marginHorizontal: 8,
}}
/>
) : null;
}
function MeetingView() {
// Get `participants` from useMeeting Hook
const { participants } = useMeeting({});
const inPipMode = usePipModeListener();
// Use this boolean to show / hide ui when pip mode changes
if (inPipMode) {
// Render the participant in PiP Box
return [...participants.keys()].map((participantId, index) => (
<ParticipantView
key={index}
participantId={participantId}
inPipMode={true}
/>
));
}
return (
<View style={{ flex: 1 }}>
// ..
// ..
</View>
);
}Step 5: Enter PiP mode
To enter Picture-in-Picture (PiP) mode, use the enterPipMode method of PipHandler. Provide the PiP box's height and width as parameters for this method.
function ControlsContainer() {
return (
<View
style={{
padding: 24,
flexDirection: "row",
justifyContent: "space-evenly",
}}
>
<Button
onPress={() => {
PipHandler.enterPipMode(300, 500);
}}
buttonText={"PiP"}
backgroundColor={"blue"}
/>
</View>
);
}By following these essential steps, you can successfully integrate VideoSDK into your application and enhance it with the Picture-in-Picture (PiP) feature.
🔚 Conclusion
Congratulations You’ve now learned how to implement Picture-in-Picture mode in a React Native Android app! By following these systematic guidelines, developers can unlock the full potential of video communication with the Picture-in-Picture (PiP) feature within their applications. This feature can offer a great user experience and enable users to multitask more effectively.
VideoSDK provides a powerful set of tools and APIs that make it easier than ever to create immersive video experiences for your users. Whether you're building a video conferencing app, a live streaming service, or a virtual event platform, VideoSDK has everything you need to succeed.
To unlock the full potential of VideoSDK and create easy-to-use video experiences, developers are encouraged to Sign up with VideoSDK today and Get 10000 minutes free to take the video app to the next level!
