A facial recognition system is a technology capable of matching a human face from a digital image or a video frame against a database of faces. Such a system is typically employed to authenticate users through ID verification services, and works by pinpointing and measuring facial features from a given image.
Since their inception, facial recognition systems have seen wider uses in recent times on smartphones and in other forms of technology, such as robotics. Because computerized facial recognition involves the measurement of a human’s physiological characteristics, facial recognition systems are categorized as biometrics. Although the accuracy of facial recognition systems as a biometric technology is lower than iris recognition and fingerprint recognition, it is widely adopted due to its contactless process like:
- Advanced human–computer interaction
- Video surveillance
- Automatic indexing of images
Their effectiveness varies, and some systems have previously been scrapped because of their ineffectiveness. The use of facial recognition systems has also raised controversy, with claims that the systems violate citizens’ privacy, commonly make incorrect identifications, encourage gender norms and racial profiling, and do not protect important biometric data. The appearance of synthetic media such as deepfakes has also raised concerns about its security.
The facial recognition system analyzes a face image. It maps and reads face geometry. Identifies key facial landmarks to distinguish a face from other objects. Facial recognition technology often looks for:
- Distance between eyes
- Distance from forehead to chin
- Distance between nose and mouth
- Depth of eye lenses
- Cheekbones shape
- Lip, ear and chin contour
Then, the system converts the facial recognition data into a sequence of numbers or dots called a face print. Each person has a unique facial print, similar to a fingerprint. The information used by facial recognition can also be used in reverse, to digitally reconstruct someone’s face.
In this tutorial, I am going to deploy an AI model at the edge to detect and recognise faces in a live video feed.
What Will We Need
- Maixduino development board: https://s.click.aliexpress.com/e/_DBd6fTF
Maixduino Face Recognition Algorithm Steps
Step 1 – Detect the face in our video stream
- It is basicly the Face detection algoritm, which uses the same model as the previous Maixduino Face Detection post (Link)
Step 2 – Cut the face out of the video feed to a standart picture format and align it to the front position
- Cut out the face, find the eyes, nose and mouth of the face, to a picture of
128x128 - Rotate the face in the face image to the standard position
Step 3 – Extract information from the face and store it
- Use the feature extraction model to extract the feature values of the face: Position of the eyes, nose and mouth.
- Store the face information to be used after
Step 4 – Then just the algoritm and try to recognize some faces
Maixduino Face Recognition (Extra Step)
If you have had problems trying to get this to work, we can use the method when everything fails, erase the data from the chip and start fresh. Go to this page link to make the procedure: https://www.edgemicrotech.com/maixpy-15-maixduino-erase-the-chip-data-micropython/
Maixduino Face Recognition Files
Just get them from Maixhub or in the links in end of the post page (Maixduino Experiment Materials).
Two ways to make this work (The changes can be made in the main MaixPy code):
- Copy the files to the memory space(s) of the Maixduino chip (I am going to use this method)
- Copy the files to the Sdcard
Upload [FaceDetection.smodel] to the [0x300000] memory address
Upload [FaceLandmarkDetection.smodel] to the [0x400000] memory address
Upload [FeatureExtraction.smodel] to the [0x500000] memory address
Upload the firmware [maixpy_v0.6.2_84_g8fcd84a58_
Maixduino Face Recognition Code
import sensor
import image
import lcd
import KPU as kpu
import time
from Maix import FPIOA, GPIO
import gc
from fpioa_manager import fm
from board import board_info
import utime
fm.register(board_info.LED_R, fm.fpioa.GPIO0, force=True)
fm.register(board_info.LED_G, fm.fpioa.GPIO1, force=True)
led_r = GPIO(GPIO.GPIO0, GPIO.OUT)
led_g = GPIO(GPIO.GPIO1, GPIO.OUT)
led_r.value(1)
led_g.value(1)
#Load the AI models from Memory or SD Card
#Face Detection model
task_fd = kpu.load(0x300000)
#Facial Features model (Left eye, Right eye, Nose, Left mouth corner, Right mouth corner)
task_ld = kpu.load(0x400000)
#Facial Features model 192 Dimensions
task_fe = kpu.load(0x500000)
#Load the AI models from SD Card
#task_fd = kpu.load("/sd/FaceDetection.smodel")
#task_ld = kpu.load("/sd/FaceLandmarkDetection.smodel")
#task_fe = kpu.load("/sd/FeatureExtraction.smodel")
#To get the frame rate
clock = time.clock()
#Setup the Boot Button function to be used to store the face recogniton parameters
fm.register(board_info.BOOT_KEY, fm.fpioa.GPIOHS0)
key_gpio = GPIO(GPIO.GPIOHS0, GPIO.IN)
start_processing = False
BOUNCE_PROTECTION = 50
def set_key_state(*_):
global start_processing
start_processing = True
utime.sleep_ms(BOUNCE_PROTECTION)
key_gpio.irq(set_key_state, GPIO.IRQ_RISING, GPIO.WAKEUP_NOT_SUPPORT)
#Set up the LCD and Camera
lcd.init()
sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QVGA)
sensor.set_hmirror(1)
sensor.set_vflip(1)
sensor.run(1)
#Create variables to store stuff
#Anchor for face detect
anchor = (1.889, 2.5245, 2.9465, 3.94056, 3.99987, 5.3658, 5.155437,
6.92275, 6.718375, 9.01025)
#Standard face key point position
dst_point = [(44, 59), (84, 59), (64, 82), (47, 105),
(81, 105)]
a = kpu.init_yolo2(task_fd, 0.5, 0.3, 5, anchor)
img_lcd = image.Image()
img_face = image.Image(size=(128, 128))
a = img_face.pix_to_ai()
record_ftr = []
record_ftrs = []
names = ['Mr.1', 'Mr.2', 'Mr.3', 'Mr.4', 'Mr.5',
'Mr.6', 'Mr.7', 'Mr.8', 'Mr.9', 'Mr.10']
ACCURACY = 85
#Main loop
while (1):
img = sensor.snapshot()
clock.tick()
code = kpu.run_yolo2(task_fd, img)
if code:
for i in code:
# Cut face and resize to 128x128
a = img.draw_rectangle(i.rect())
face_cut = img.cut(i.x(), i.y(), i.w(), i.h())
face_cut_128 = face_cut.resize(128, 128)
a = face_cut_128.pix_to_ai()
# a = img.draw_image(face_cut_128, (0,0))
# Landmark for face 5 points
fmap = kpu.forward(task_ld, face_cut_128)
plist = fmap[:]
le = (i.x() + int(plist[0] * i.w() - 10), i.y() + int(plist[1] * i.h()))
re = (i.x() + int(plist[2] * i.w()), i.y() + int(plist[3] * i.h()))
nose = (i.x() + int(plist[4] * i.w()), i.y() + int(plist[5] * i.h()))
lm = (i.x() + int(plist[6] * i.w()), i.y() + int(plist[7] * i.h()))
rm = (i.x() + int(plist[8] * i.w()), i.y() + int(plist[9] * i.h()))
a = img.draw_circle(le[0], le[1], 4)
a = img.draw_circle(re[0], re[1], 4)
a = img.draw_circle(nose[0], nose[1], 4)
a = img.draw_circle(lm[0], lm[1], 4)
a = img.draw_circle(rm[0], rm[1], 4)
# align face to standard position
src_point = [le, re, nose, lm, rm]
T = image.get_affine_transform(src_point, dst_point)
a = image.warp_affine_ai(img, img_face, T)
a = img_face.ai_to_pix()
#Display the matched face feature image on the screen
# a = img.draw_image(img_face, (128,0))
del (face_cut_128)
# calculate face feature vector
fmap = kpu.forward(task_fe, img_face)
feature = kpu.face_encode(fmap[:])
reg_flag = False
scores = []
for j in range(len(record_ftrs)):
score = kpu.face_compare(record_ftrs[j], feature)
scores.append(score)
max_score = 0
index = 0
for k in range(len(scores)):
if max_score < scores[k]:
max_score = scores[k]
index = k
if max_score > ACCURACY:
a = img.draw_string(i.x(), i.y(), ("%s :%2.1f" % (
names[index], max_score)), color=(0, 255, 0), scale=2)
led_r.value(1)
led_g.value(0)
else:
a = img.draw_string(i.x(), i.y(), ("X :%2.1f" % (
max_score)), color=(255, 0, 0), scale=2)
led_r.value(0)
led_g.value(1)
#Store the current feature to the face feature list
if start_processing:
record_ftr = feature
record_ftrs.append(record_ftr)
start_processing = False
break
#Calculate the frame rate
fps = clock.fps()
#Print the frame rate
print("%2.1f fps" % fps)
a = lcd.display(img)
gc.collect()
# kpu.memtest()
# a = kpu.deinit(task_fe)
# a = kpu.deinit(task_ld)
# a = kpu.deinit(task_fd)
Maixduino Face Recognition Realtime Steps
- Run the code
The program will try to find a face
When it finds one, it will draw a box around it and starts to get the parameters
Now we have to press the Boot button on the Maixduino board to save the face info
Finally we can start recognizing faces
Maixduino Experiment Materials
- Firmware: maixpy_v0.6.2_84_g8fcd84a58_openmv_kmodel_v4_with_ide_support.bin
- Model File_1: FaceDetection.smodel
- Model File_2: FaceLandmarkDetection.smodel
- Model File_3: FeatureExtraction.smodel
- Code: main_final.py
