Smartphones with a near-infrared camera for 3D face or palm/vein recognition
Portables avec une caméra proche infrarouge pour la reconnaissance 3D du visage ou des veines de la main

This page focuses on smartphones with 3D face recognition, but LG has used a near-infrared ToF camera for palm vein recognition, so I changed the title. You will find other modalities in the:

Face recognition for smartphone using the front camera (the camera for selfies) is not very original, and can be easily spoofed with a photo. So the idea is to use some 3D information to avoid this problem and get some better recognition accuracy.

Using 3D information for face recognition is now quite an old technology: it's a long times that we are using two (or more) cameras to acquire images from different angles to get the distance of objects. The novelty for smartphones is to be able to integrate the technology inside a very small form factor, using some specific sensors under different names

As these 3D camera are working in the near-infrared band, we can use them for vein recognition. And I'm wondering if the 3D information is useful -normally we don't care for veins.

Take a look at 3D Cameras for Consumer Application / Markus Rossi, if you want to understand the basics of 3D sensing.

Note: only smartphones with at least a real attempt to get 3D, I excluded all smartphones using the (simple and unique) front camera.


(2017 Jun) Sony to demo 3D face biometric running on an Xperia smartphone.
  • (via Techcrunch) Just such a feature is set to be demoed at the MWC tradeshow in Shanghai this week — by SoftKinetic, the wholly owned Sony subsidiary which makes camera sensor modules, running on a Sony Xperia smartphone and using facial recognition software from a Swiss company called KeyLemon. DepthSense time-of-flight (ToF).
  • (2017 Sep) The iPhone X features Face ID (and no fingerprint sensor).

    (More info about Apple and biometrics).

  • Face ID is using a 7-megapixel camera sensor, an infrared camera, a flood illuminator, and a dot projector. Aside from taking standard 2D still images, this array projects more than 30,000 dots to help create a depth map of the user's face, which can be combined with the still image to create a 3D version. This facial map is then stored in the encrypted Secure Enclave, where it is used for Face ID, Apple's facial recognition system. According to Apple, Face ID is capable of generating far fewer false positives than Touch ID, decreasing the chance of an incorrect authentication from 50,000:1 to 1,000,000:1. The system is designed to resist spoofing of the owner's face by the use of photographs and masks, though it was noted during launch that identical twins may find this to be an issue.

  • (2017 Sep) About face ID advanced technology (Apple web site)
  • (2017 Sep) Face ID security: details about specifications. But no associated FRR with the announced FAR!

  • A FAR of 1 to 1 million is incredible. And unbelivable. The very first demo of Face ID failed: Federighi’s initial attempt to unlock the onstage demo iPhone X was unsuccessful. The passcode screen popped up. He then picked up a second iPhone X and the feature worked successfully.
    Apple gave an explanation: people were handling the device for the stage demo ahead of time and didn't realise Face ID was trying to authenticate their face. After failing a number of times, because they weren't Craig, the iPhone did what it was designed to do, which was to require his passcode.
    While this is explaining why the passcode pop-up, with a 1:1 000 000 chance to get wrong, this is just stupid to run like this. This is OK to ask for the passcode, but keep the Face ID working at the same time, this gives a chance to enter and not to fail at the first demo.
    I believe that the real world is not like Apple wants: the FAR is likely far lower than they said, and it is a good practice to limit the number of attempts -as they do with the fingerprints, so you lower the chances to get a false accept (and virtually, you are unable to show again a false accept if you get one...).
  • Teardown: TechInsights
  • (2017 Sep) Vivo to Use Dual Pixel Camera for 3D Face Recognition
  • Vivo is going to use dual pixel camera for 3D face scanning in its "bi-directional Face Wake" facial recognition technology.
  • (2017 Nov) Huawei 3D technology
  • When releasing the Honor V10, Huawei also unveiled a 3D technology (not included in the Honor V10).
  • 300 000 dots (so ten times the Apple X, the number war is starting), recognition time is 400 ms, sub-millimeter accuracy. Likely Sunny optical technology 舜宇光学科技
  • 2018

    (2018 May) Xiaomi Mi 8 Explorer Edition
  • Xiaomi announces Mi 8 flagship phone. The Explorer Edition has an in-display fingerprint sensor that, for the first time in a phone, uses pressure sensitivity. Xiaomi also claims that the Explorer Edition is the first Android phone in the world to use 3D face recognition technology, like the iPhone X’s Face ID — and there’s an Animoji equivalent on board.
  • Recognition technology supplied by Mantis Vision
  • Uses a grating element (diffraction pattern) to display infrared points (not a DOE like Apple)
  • Module provided by 由欧菲光 (O-Film)
  • (2018 Jun) Oppo Find X
  • Oppo Find X : 3D structured light technology. 0.1s fast recognition, 0.03s reconstruction, 0.0001% false recognition rate (= a stupid copy of Apple numbers)
  • 3D module (with a 15000 dots projector) likely supplied by Q Technology 丘钛科技.
  • Recognition technology supplier (unclear): Orbbec / 奥比中光 also claimed by Megvii so Oppo claims an AI-enhanced 3D Camera :)
  • (2018 Jun) Vivo prototype
  • At MWC Shanghai 2018, Vivo shows two prototypes equipped with TOF (time of flight) technology. 300,000 depth pixel resolution.
  • Likely Panasonic technology for the TOF camera
  • Megvii involved for applications.
  • (2018 Jun) Huawei Mate 20
  • (2018 Jun) A render leaks, showing a notch for a 3D camera ?
  • (2018 Aug) Another leak unveils a possible 3D camera + underdisplay fingerprint sensor.
  • (2018 Sep) The front panel leak reveals cut-outs for Face ID-like tech.
  • (2018 Oct) The Huawei Mate 20 Pro has a 3D face recognition, so similar to the Apple iPhone X. 30 000 dots. Infrared. Emojis. But it also have a fingerprint sensor, unlike the iPhone X.
  • (2018 Oct) Already a false accept: AndroidPit which is a pity for a 1 to 1 million FAR (Huawei announced the same rate than Apple, what a surprise).
  • (2018 Nov) Zol teardown.
  • (2018 Nov) iFixIt teardown.
  • Vivo X23
  • (2018 Aug) Announced with 3d face recognition.
  • Star version: The smartphone is being touted to offer face unlock functionality, which the brand is calling “zero-light face recognition.”
  • (2018 Nov) Symphony version: infrared face recognition sensor -likely nothing about 3D
  • (2018 Oct) Apple iPhone XR, XS, XS Max, iPad Pro 12.9', iPad Pro 11'
  • Same as iPhone X.
  • (2018 Nov) ams and Qualcomm Technologies
  • ams and Qualcomm Technologies to focus engineering strengths on active stereo camera solution for mobile 3D applications
  • ams’ advanced VCSEL light sources and optical IR pattern technology incorporating mass-production proven wafer level optics
  • ams provides the full range of IR illumination devices specifically designed to address the three 3D technologies Active Stereo Vision, Structured Light and Time-of-Flight (ToF).
  • (2018 Nov) Samsung Galaxy S10 rumours
  • The Galaxy S10 may include a full 3D camera with Mantis Vision + Woodgate.
  • (2019 Feb) it was just a rumour, no 3D camera.
  • 2019

    (2019 Feb) LG G8ThinQ
  • The LG G8ThinQ is using the Real3 ToF near-infrared camera from Infineon to perform palm vein recognition (and not face recognition).
    The 3D side of the camera is of no use for vein recognition. LG wanted to perform gesture recognition, so they needed some 3D information.
  • The Real3 camera from Infineon : press release (Feb 25, 2019).
    Global Shutter, 100fps, modulation up to 100MHz (for phase detection), IRS1125C has 352x288 pixels, resolution likely about 1% of the range.
  • (2019 Jul) Google Pixel 4
  • The Google Pixel 4 has ... so many sensors! OK about one infrared camera + the flood illuminator + the dot projector, but why a second infrared camera? To be better compared to Apple? Or they discovered that the dot solution is not so accurate, so we need a parallax extraction type? Wait and see.
  • 2020

    (2020 Jan) Infineon and PMD (pmdtechnologies)
  • CES 2020: Infineon presents the world’s smallest 3D image sensor for face authentication and photo effects on smartphones and the like. New chip: IRS2877C (VGA, 5um pixel). To use with the illumination driver IRS9100C.
  • (2020 Feb) Trinamix
  • Trinamix (brand of BASF) : Imaging system for mobile applications.
  • (2020 Feb) Startup trinamiX GmbH has joined Qualcomm Technology Inc.’s software accelerator program. Aliveness detection testing. Trinamix's early camera prototypes use a small Raspberry Pi computer that works with the LG phone's USB-C port. (photo credit: Scott Stein/CNET).
    (Trinamix datasheet)
  • (2020) ams
  • ams / 3D sensing for mobile: product list.
  • Dot-Pattern Infrared Illuminator for 3D Stereoscopic Imaging:

    Flood Illuminator:

    Time-of-Flight Sensing (1D):

    Camera NIR (CGSS CMOS Global Shutter Sensor):

    • Mira130 (1.3MP global shutter CMOS image sensor NIR, 120fps @ 10bits) (datasheet)

    For Apple: here is an interesting page to get all the real competition in one shot, isn't it?

    Pour Yole: et voilà encore une autre page pratique pour mes rapports...