Flask OpenCV Face Recognition
A Web-Based Application for Detection of Faces on video using Flask Opencv and face_recognition. by Harmesh Rana, Prateek Sharma, Vivek Kumar Shukla.
Object Detection is one of the most promising avenues of Artificial Intelligence Today. It has been constantly evolving throughout recent years to the point that a good AI can even detect faces and many other details that only human beings were able to detect. Face Recognition is a part of Computer Vision which is a field of computer science that works on enabling computers to see, identify and process images in the same way that human vision does, and then provide appropriate output. It is like imparting human intelligence and instincts to a computer. In reality, though, it is a difficult task to enable computers to recognize images of different objects.
Facial Recognition
Facial recognition scanning systems also use computer vision technology to identify individuals for security purposes. The most common example of computer vision in facial recognition is for securing smartphones. More advanced uses of facial recognition and biometrics include in residential or business security systems that use unique physiological features of individuals to verify their identity. Deep learning algorithms can identify the unique patterns in a person’s fingerprints and use it to control access to high-security areas such as high-confidentiality workplaces, such as nuclear powerplants, research labs, and bank vaults.
Face Recognition by Python
face_recognition is state of art, simplest face detection library built with the deep learning. It has an accuracy of 98.38 % in order to detect faces on images and videos. Not only detection, but face_recogintion also provides face manipulation features.
OpenCV by Python
OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library.
Flask by Python
Flask is a web framework. This means flask provides you with tools, libraries and technologies that allow you to build a web application
Installing Dependencies
Our project requires the following dependencies to be installed.
⚫️ dlib
Dlib is a modern C++ toolkit containing machine learning algorithms and tools for creating complex software in C++ to solve real-world problems.
➡ pip install dlib
Alternatively, if you want to compile dlib yourself then go into the dlib
root folder and run:
➡ python setup.py install
Compiling dlib should work on any operating system so long as you have
CMake installed. On Ubuntu/Mac OSx, this can be done easily by running the command:
➡ sudo apt-get install cmake
Also, note that this example requires Numpy which can be installed
via the command:
➡ pip install numpy
if you are getting errors on installation then there are tons of write-ups available online you guys can refer that.
⚫️ Flask and OpenCV
Before installation make sure your package manager is updated. By executing the following commands on the Command processer you can install Flask and Open cv.
➡pip install flask
➡pip install opencv-python
Directory Structure of Project
Let’s Build The Application
Step 1: Taking the Input Video
An uploader is designed in the flask for taking the input video file. then uploading that file to Upload directory.
UPLOAD_FOLDER = '/Users/prate/upload/'
ALLOWED_EXTENSIONS = set(['txt','mp4'])
app = Flask(__name__)
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDERdef allowed_file(filename):
return '.' in filename and \
filename.rsplit('.', 1)[1] in ALLOWED_EXTENSIONS@app.route("/", methods=['GET', 'POST'])
def index():
if request.method == 'POST':
file = request.files['file']
if file and allowed_file(file.filename):
filename = secure_filename(file.filename)
file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))return """
<!doctype html>
<title>Upload new File</title>
<h1>Upload new File</h1>
<form action="" method=post enctype=multipart/form-data>
<p><input type=file name=file>
<input type=submit value=Upload>
</form>
<p>%s</p>
""" % "<br>".join(os.listdir(app.config['UPLOAD_FOLDER'],))
Step 2: Adding Known Faces
Face1 = face_recognition.load_image_file("face1.jpg")
hfencoding = face_recognition.face_encodings(Face1)[0] face2 = face_recognition.load_image_file("face2.jpeg")
pfencoding = face_recognition.face_encodings(face2)[0] # Create arrays of known face encodings and their names
known_face_encodings = [
hfencoding,
pfencoding
]
known_face_names = [
"Harmesh",
"Prateek"
]
Step 3: Video Preprocessing
video_capture is an object of the video file given as an input through which we read the frames and further, we convert the image from BGR color (which OpenCV uses) to RGB color (which face_recognition uses)
video_capture = cv2.VideoCapture(filename)
for i in range(1,length-1):
ret, frame = video_capture.read()
rgb_frame = frame[:, :, ::-1]Step 4:Feature Extraction
face_locations function locates all the faces in a particular frame and returns the coordinates of the face(s)
As mentioned HERE. It is a face for us. But, for our algorithm, it is only an array of RGB values — that matches a pattern that the it has learnt from the data samples we provided to it.
For face recognition, the algorithm notes certain important measurements on the face — like the color and size and slant of eyes, the gap between eyebrows, etc. All these put together define the face encoding — the information obtained out of the image — that is used to identify the particular face.
face_locations = face_recognition.face_locations(rgb_frame)
face_encodings = face_recognition.face_encodings(rgb_frame, face_locations)Step 5: Comparison of Faces
Comparison of Known Faces with the Faces Detected in the video.
for (top, right, bottom, left), face_encoding in zip(face_locations, face_encodings):
matches = face_recognition.compare_faces(known_face_encodings, face_encoding)Or instead, use the known face with the smallest distance to the new face.
face_distances = face_recognition.face_distance(known_face_encodings, face_encoding)
best_match_index = np.argmin(face_distances)
if matches[best_match_index]:
name = known_face_names[best_match_index]Step 6: Drawing Rectangle on the Faces
cv2.rectangle function is used to draw a rectangle around the faces and cv2.putText to write their names on the locations that we obtained during the feature extraction.
cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)
cv2.rectangle(frame, (left, bottom - 10), (right, bottom+10), (10, 10, 10), cv2.FILLED)
font = cv2.FONT_HERSHEY_DUPLEX
cv2.putText(frame, name, (left + 2, bottom), font, 0.4, (255, 255, 255), 1)Step 7:Writing the Output File
To write in a video file we recommend using the cv2 library. for this application, we used the WebM format with vp80 encoding which ultimately helps to run video files on webpages smoothly.
fps = int(video_capture.get(cv2.CAP_PROP_FPS)
width = int(video_capture.get(3))
height = int(video_capture.get(4))
fourcc = cv2.VideoWriter_fourcc(*'vp80')
PATH = '/Users/prate/static/demo.webm'
out = cv2.VideoWriter(PATH,fourcc, fps, (width,height))we have to write the frames in the output video immediately after drawing rectangle on them so that we get the serialized output.
for (top, right, bottom, left), face_encoding in zip(face_locations, face_encodings):
out.write(frame)Step 8: Display the Output file.
We will show the output video file in a new HTML file which will play the output video. To launch the HTML page write following code in the index() function of flask.
return render_template('video.html')Code of HTML file
<html>
<head>
<title>Video Example</title></head>
<body><h2>Video After Processing</h2>
<video width="560" height="340" controls>
<source src="/static/demo.webm" type="video/webm">
</video>
</body>
</html>Optimization
This process takes a lot of time in between face detection, drawing rectangles, and displaying the result. To reduce the time, we applied the following solutions
- Instead of drawing a rectangle around faces and showing the result simultaneously, we decided to store the frames in a video file because painting an object on the frame and displaying those frames simultaneously hogs up the memory thus delaying the whole process. But, by separating both processes, we allow the processer to work faster.
- We don't actually have to process all the frames of the video. We can skip a few frames in between to save time. The reason behind this is a second of a video has 20–30 frames as we know human faces don't actually move all that much in a second so we can skip alternative frames and the result should still be the same while reducing the time by half.
- Processing the video under an infinite loop is not an efficient way and may end up with an error. Because every video can only have a finite number of frames and while using an infinite loop, we will eventually run out of frames to process and it will result in an error, the infinite loop should only be used in the case of live streaming or taking input from the webcam where we don't know the number of frames prior to processing which is not the goal of this application. To combat this we have limited the process to TOTAL_NUMBER_OF_FRAMES-1 and this change is also not noticeable.
length = int(video_capture.get(cv2.CAP_PROP_FRAME_COUNT))
fps = int(video_capture.get(cv2.CAP_PROP_FPS))
for i in range(1,length-1):
i+=1OUTPUT
Conclusion
This Application Focuses on the identification of the face that is shown in the input in the form of video, this process is known as Face Recognition. In this Application, we can easily identify the Persons using the face encoding of the person available to us and matching them with the encoding extracted from the input.
The entire project code is available in the following Github Repository: Facerecognition.
Thank You.
Open For Reviews.
