Real Time Face Detection with Android Studio and OpenCV
In this Blog we are going to develop Real time Face Detection Android Application. OpenCV is an open-source library that includes several hundreds of computer vision algorithms. Using OpenCV we can do image processing and other computer vision tasks. For face detection we are going to use Haar Cascade classifier. OpenCV comes up with JNI Wrappers for Java and the Android platform. So Let’s get start for the our Android App.
Step 1 : Start new project in android studio with Empty Activity This will create a project structure as follow.
Step 2 : Now we need to import OpenCV. For this download the OpenCV SDK version and extract it in your desktop. To import the OpenCV SDK navigate through File → New → Import Module…
This will open the pop up to select the Source Directory select the java folder from OpenCV SDK as show in below image.
After import is done and If there are any errors while sync remove those errors and sync again. Make sure the project structure matches the below image. There is possibility of targetsdkversion and compilesdkversion error in AndroidManifest.xml. So if you come across any such error then make necessary changes in that and again sync the project probably this will help to solve those errors.
Step 3 : We have to add OpenCV Library to our App module as a dependency. Go to File → Project Structure. This will open the below pop up.
Select the Dependencies tab → click on Plus icon → Select Module Dependency As above. Then select the OpenCVLibrary343 module and click on ok button as bellow.
Step 4 : Add Native Libraries
On your file explorer, navigate to the folder where you extracted the content of the OpenCV Android library zip file. Open the sdk folder and then the native folder and copy the libs folder as below.
Paste that libs folder to the main folder of project… (ProjectName/app/src/main) and rename it to jniLibs as in below image.
Step 5 : Add Haar Cascade XML file which is use to detect the face from the camera frame.
Again, navigate to the folder where you extracted the content of the OpenCV Android library zip file. Open the folder sdk → etc → haarcascades and copy the haarcascade_frontalface_alt.xml file as below.
Paste that file to android → app → res → raw as below
Hurray we are done with the project set up Now Let’s start to code.
Step 6 : Add camera permission to AndroidManifest.xml
<uses-feature android:name="android.hardware.camera" android:required="false"/>
<uses-feature android:name="android.hardware.camera.autofocus" android:required="false"/>
<uses-feature android:name="android.hardware.camera.front" android:required="false"/>
<uses-feature android:name="android.hardware.camera.front.autofocus" android:required="false"/>Step 7 : Replace the activity_main.xml with below code. In this we are adding JavaCameraView from OpenCV.
<?xml version="1.0" encoding="utf-8"?>
<FrameLayout
xmlns:android="http://schemas.android.com/apk/res/android"
android:layout_width="match_parent"
android:layout_height="match_parent" >
<org.opencv.android.JavaCameraView
android:layout_width="match_parent"
android:layout_height="match_parent"
android:id="@+id/javaCameraView" />
</FrameLayout>Step 8 : Start to code MainActivity.java
Add below variables before onCreate method
JavaCameraView javaCameraView;
File caseFile;
CascadeClassifier faceDetector;
private Mat mRgba,mGrey;In onCreate method checking the OpenCV is connected or not if not then trying to make connection with. If it finds that OpenCV is configured successfully then it calls baseCallback method with Status Success. Below is the onCreate method.
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
javaCameraView = (JavaCameraView)findViewById(R.id.javaCameraView);
if(!OpenCVLoader.initDebug()){
OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_3_0_0, this,baseCallback);
}
else{
try {
baseCallback.onManagerConnected(LoaderCallbackInterface.SUCCESS);
} catch (IOException e) {
e.printStackTrace();
}
}
javaCameraView.setCvCameraViewListener(this);
}Below are the camera Life Cycle methods which will run according to camera ON/OFF and and while camera is working. As the name suggest onCameraViewStarted this will run as the camera started and initialize the mRgba and mGray variables, onCameraViewStopped this will run when the camera stopped. Now onCameraFrame this is the important method which will run at the time when camera is running and return the Mat object by adding rectangle to the face detected by OpenCV. Add below code after the onCreate method in MainActivity.java .
@Override
public void onCameraViewStarted(int width, int height) {
mRgba = new Mat();
mGrey = new Mat();
}
@Override
public void onCameraViewStopped() {
mRgba.release();
mGrey.release();
}
@Override
public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
mRgba = inputFrame.rgba();
mGrey = inputFrame.gray();
//detect Face
MatOfRect facedetections = new MatOfRect();
faceDetector.detectMultiScale(mRgba,facedetections);
for(Rect react: facedetections.toArray()){
Imgproc.rectangle(mRgba, new Point(react.x,react.y),
new Point(react.x + react.width, react.y + react.height),
new Scalar(255,0,0));
}
return mRgba;
}Now below method is baseCallback which will detect the faces from image using haarcascade_frontalface_alt2.xml file. Add below method after the camera life cycle methods.
private BaseLoaderCallback baseCallback = new BaseLoaderCallback(this) {
@Override
public void onManagerConnected(int status) throws IOException {
switch (status)
{
case LoaderCallbackInterface.SUCCESS:
{
InputStream is = getResources().openRawResource(R.raw.haarcascade_frontalface_alt2);
File cascadeDir = getDir("cascade" , Context.MODE_PRIVATE);
caseFile = new File(cascadeDir, "haarcascade_frontalface_alt2.xml");
FileOutputStream fos = new FileOutputStream(caseFile);
byte[] buffer = new byte[4096];
int bytesRead;
while((bytesRead = is.read(buffer)) !=-1 ){
fos.write(buffer,0,bytesRead);
}
is.close();
fos.close();
faceDetector = new CascadeClassifier(caseFile.getAbsolutePath());
if(faceDetector.empty()){
faceDetector = null;
}
else{
cascadeDir.delete();
}
javaCameraView.enableView();
}
break;
default:
super.onManagerConnected(status);
}
}
};And here we are… Run the android app in Android Emulator or Physical Device. You need to give the Camera permission to run this app.
You can clone the complete code from https://github.com/laksh0798/FaceApp.git .
That’s it! With a few steps we have created Face detection android app. Well, there are still some enhancement can be done such as adding flash option, switch the camera from rare to front and vice versa, etc. but the face detection is done!
