Smartphones with a near-infrared camera for 3D face or palm/vein recognition
Portables avec une caméra proche infrarouge pour la reconnaissance 3D du visage ou des veines de la main
This page focuses on smartphones with 3D face recognition, but LG has used a near-infrared ToF
camera for palm vein recognition, so I changed the title.
You will find other modalities in the:
Face recognition for smartphone using the front camera
(the camera for selfies) is not very original, and can be easily spoofed with a photo.
So the idea is to use some 3D information to avoid this problem and get some better
Using 3D information for face recognition is now quite an old technology:
it's a long times that we are using two (or more) cameras
to acquire images from different angles to get the distance of objects.
The novelty for smartphones is to be able to integrate the technology
inside a very small form factor, using some specific sensors under different names
- a camera sometimes called XY+Z camera to show that the Z information=distance is also acquired
- a TOF camera = a "time of flight" camera, where the time of the light going from a source
to the camera is measured (for each pixel), so the distance
- in general, a specific light source is needed: structured light (lines or points), laser (VCSEL),
rarely in the visible domain as it is annoying for the eyes...
As these 3D camera are working in the near-infrared band, we can use them for
And I'm wondering if the 3D information is useful -normally we don't care for veins.
Take a look at 3D Cameras for Consumer Application
/ Markus Rossi, if you want to understand the basics of 3D sensing.
Note: only smartphones with at least a real attempt to get 3D,
I excluded all smartphones using the (simple and unique) front camera.
(2017 Jun) Sony to demo 3D face biometric running on an Xperia smartphone.
Just such a feature is set to be demoed at the MWC tradeshow in Shanghai this week —
the wholly owned Sony subsidiary which makes camera sensor modules,
running on a Sony Xperia smartphone and using facial recognition software
from a Swiss company called KeyLemon. DepthSense time-of-flight (ToF).
(2017 Sep) The iPhone X features Face ID (and no fingerprint sensor).
(More info about Apple and biometrics).
Face ID is using a 7-megapixel camera sensor, an infrared camera, a flood illuminator, and a dot projector.
Aside from taking standard 2D still images, this array projects more than 30,000 dots
to help create a depth map of the user's face, which can be combined with the still image to create a 3D version.
This facial map is then stored in the encrypted Secure Enclave, where it is used for Face ID,
Apple's facial recognition system.
According to Apple, Face ID is capable of generating far fewer false positives than Touch ID,
decreasing the chance of an incorrect authentication from 50,000:1 to 1,000,000:1.
The system is designed to resist spoofing of the owner's face by the use of photographs
and masks, though it was noted during launch that identical twins may find this to be an issue.
(2017 Sep) About face ID advanced technology (Apple web site)
(2017 Sep) Face ID security: details about specifications. But no associated FRR with the announced FAR!
A FAR of 1 to 1 million is incredible. And unbelivable. The very first demo of Face ID failed:
Federighi’s initial attempt to unlock the onstage demo iPhone X was unsuccessful.
The passcode screen popped up. He then picked up a second iPhone X and
the feature worked successfully.
Apple gave an explanation:
people were handling the device for the stage demo ahead of time and didn't realise Face ID
was trying to authenticate their face. After failing a number of times, because they weren't
Craig, the iPhone did what it was designed to do, which was to require his passcode.
While this is explaining why the passcode pop-up, with a 1:1 000 000 chance to get wrong,
this is just stupid to run like this. This is OK to ask for the passcode, but keep the Face ID
working at the same time, this gives a chance to enter and not to fail at the first demo.
I believe that the real world is not like Apple wants: the FAR is likely far lower than
they said, and it is a good practice to limit the number of attempts -as they do with the
fingerprints, so you lower the chances to get a false accept (and virtually, you are unable
to show again a false accept if you get one...).
(2017 Sep) Vivo to Use Dual Pixel Camera for 3D Face Recognition
Vivo is going to use dual pixel camera for 3D face scanning
in its "bi-directional Face Wake" facial recognition technology.
(2017 Nov) Huawei 3D technology
When releasing the Honor V10, Huawei also unveiled a 3D technology (not included
in the Honor V10).
300 000 dots (so ten times the Apple X, the number war is starting),
recognition time is 400 ms, sub-millimeter accuracy.
Likely Sunny optical technology 舜宇光学科技
(2018 May) Xiaomi Mi 8 Explorer Edition
Xiaomi announces Mi 8 flagship phone. The Explorer Edition has
an in-display fingerprint sensor that, for the first time in a phone, uses pressure sensitivity.
Xiaomi also claims that the Explorer Edition is the first Android phone in the world
to use 3D face recognition technology, like the iPhone X’s Face ID —
and there’s an Animoji equivalent on board.
Recognition technology supplied by Mantis Vision
Uses a grating element (diffraction pattern) to display infrared points (not a DOE like Apple)
Module provided by 由欧菲光 (O-Film)
(2018 Jun) Oppo Find X
Oppo Find X :
3D structured light technology.
0.1s fast recognition, 0.03s reconstruction, 0.0001% false recognition rate
(= a stupid copy of Apple numbers)
3D module (with a 15000 dots projector) likely supplied by
Q Technology 丘钛科技.
Recognition technology supplier (unclear):
also claimed by Megvii
so Oppo claims an AI-enhanced 3D Camera :)
(2018 Jun) Vivo prototype
At MWC Shanghai 2018,
shows two prototypes equipped with TOF (time of flight) technology.
300,000 depth pixel resolution.
Likely Panasonic technology for the TOF camera
involved for applications.
(2018 Jun) Huawei Mate 20
(2018 Jun) A render leaks, showing a notch for a 3D camera ?
(2018 Aug) Another leak unveils a possible 3D camera + underdisplay fingerprint sensor.
(2018 Sep) The front panel leak reveals cut-outs for Face ID-like tech.
(2018 Oct) The Huawei Mate 20 Pro has a 3D face recognition, so similar to the Apple iPhone X.
30 000 dots. Infrared. Emojis. But it also have a fingerprint sensor, unlike the iPhone X.
(2018 Oct) Already a false accept: AndroidPit
which is a pity for a 1 to 1 million FAR (Huawei announced the same rate than Apple, what a surprise).
(2018 Nov) Zol teardown.
(2018 Nov) iFixIt teardown.
(2018 Aug) Announced with 3d face recognition.
Star version: The smartphone is being touted to offer face unlock functionality,
which the brand is calling “zero-light face recognition.”
(2018 Nov) Symphony version: infrared face recognition sensor -likely nothing about 3D
(2018 Oct) Apple iPhone XR, XS, XS Max, iPad Pro 12.9', iPad Pro 11'
Same as iPhone X.
(2019 Feb) LG G8ThinQ
The LG G8ThinQ is using the Real3 ToF near-infrared camera from Infineon
to perform palm vein recognition (and not face recognition).
The 3D side of the camera is of no use for vein recognition. LG wanted to perform
gesture recognition, so they needed some 3D information.
The Real3 camera from Infineon
: press release (Feb 25, 2019).
Global Shutter, 100fps, modulation up to 100MHz (for phase detection), IRS1125C has 352x288 pixels,
resolution likely about 1% of the range.
(2019 Jul) Google Pixel 4
The Google Pixel 4 has ... so many sensors! OK about one infrared camera +
the flood illuminator + the dot projector, but why a second infrared camera?
To be better compared to Apple? Or they discovered that the dot solution is not so accurate,
so we need a parallax extraction type? Wait and see.
(2020 Feb) Trinamix
Trinamix (brand of BASF) :
Imaging system for mobile applications.
(2020 Feb) Startup trinamiX GmbH has joined Qualcomm Technology Inc.’s software accelerator program.
Aliveness detection testing. Trinamix's early camera prototypes use a small Raspberry Pi computer that works with the LG phone's USB-C port.
(photo credit: Scott Stein/CNET).
For Apple: here is an interesting page to get all the real competition in one shot, isn't it?
Pour Yole: et voilà encore une autre page pratique pour mes rapports...