July 2012: Apple acquires Authentec for $356 Million
What a surprise! So the big firm from Cupertino is now interested into biometrics? During years, competition has tried to get the killing application using biometrics -just have a look at my website- with some mitigated success.
And Apple has certainly some plans, because buying the first fingerprint sensor manufacturer able to produce millions of pieces, and the only real one since the merge between the number one and number two, Authentec and Upek, is not a small step towards biometrics.
As expected from Apple, access to the Authentec fingerprint sensors has been reduced, not saying closed but almost, which is a difficulty for all companies that were using their sensors, and particularly Apple's competition. So it is somewhere a good news for small fingerprint sensor manufacturers: there is an opportunity to enter this market!
But there is a big mystery, as usual with Apple, about how a fingerprint sensor will be included in their products. So speculations and leaks started... As usual, the only reliable source of information are the patent applications, but you never know if it will be used, and when. But that's better than nothing.
Note: if you don't want to read the original patent applications (something I understand :) you will find also some analysis at Patently Apple, a pretty good website.
I give very few comments hereafter, for sure, there is much more to say about all those patent applications... but this is nearly a fulltime job to do so.
Oct, 2012: Patent application : access to internal devices
The 20120258773 application from Alvarez, Dinh & Myers discloses a device with a window relealing a biometric sensor (or other things).
The image speaks for itself.
April, 2013: Production of the iPhone 5S 'set to be delayed beyond June because of its fingerprint sensor'
The MailOnline article says that:
So there is a fingerprint sensor! Making the design of a new chip is long and difficult: the chip has to work, and also ramping up the production to reach millions of pieces is pretty difficult. And moreover, there will be a packaging problem, I know that the coating is not an easy thing to handle. It makes sense...
June, 2013: Patent application : integrally molded die and bezel structure...
The 20130154031 application from Bond, Kramer (Upek), Gozzini unveils a bezel to encapsulate a swipe fingerprint sensor.
This is making sense, as it is required to protect the sensor, as well as the wired connections on the side.
The DigiTimes article says that:
Second warning, after the April info. TSMC was the usual foundry of Authentec, and having yield problems means this is a new chip.
July, 2013: Patent application : finger sensor, coupling electrodes
The 20130181949 application from Dale Setlak (Authentec R&D).
In a few words, this is about integrating a (more or less) capacitive swipe fingerprint sensor in a touchscreen display. So we learn that it is only a small part of the screen that is used, and it is likely pretty difficult to integrate with the regular capacitive sensor that goes with the touchscreen. Definitively something long to develop, it would be a surprise to be able to integrate this with a product in a close future...
Also, integrating a swipe sensor directly with the touchscreen, where you have to slide to unlock, would be very convenient. But hard to make from a technological point of view.
BTW, you will see that this application is starting with a reference to a patent from Dale Setlak, immediatly followed by the U.S. Pat. No. 6,289,114, from ... myself (and dated September 11, 2001 which is weird).
August, 2013: Patent application: differential measurements
The 20120085822 application from Setlak, Landy, Straub, Schenk is about a sensor making differential measurements.
This patent application is interesting mainly because it adresses the problem of the protective coating thickness above the pixels, a very hard problem to solve. If you want to have a nice external appearance of the sensor, and some protection, it is required to add something above the pixels. But doing this, the resulting signal is decreasing, leading to faint images, or even no image at all if it is too thick. And this is a critical problem for Apple.
Another interesting fact is this differential approach, which gives images that looks like 3D, a typical feature. We got the same kind of 3D images when using the FingerChip, which was a pure differential device as measuring temperature changes in time (not spatially). (To my former collegues that may see that: this is not a FingerChip image, I have been surprised myself also about similarities.)
Sept. 5, 2013: leaks about the iPhone 5S internal
Sonny Dickson unveils some Behold The First Part Leaks of the Grey or ‘Graphite’ iPhone 5S.
So we can see the internal, and maybe the fingerprint sensor? Unfortunaly, there is nothing. There is no hole in the front panel for a sensor, and the button seems to have nothing special. Maybe it is not a 5S, or even this is a version without fingerprint sensor.
Sept. 5, 2013: Patent application : Electronic Device With Shared Near Field Communications and Sensor Structures
Hidden behind an unclear name, the 20130231046 application from Pope, Jarvis, Merz & Myers unveils the shape of the future home button with a fingerprint sensor.
A few days after, it is obvious, see below.
Sept. 5, 2013: leaks about the fingeprint scanner
Sonny Dickson unveils some High-Res Images of the iPhone 5S Fingerprint Scanners.
Sorry guy, there is no sensor, but the square area is intriguing, something is obviously missing. And with flexible circuits, you never know how it is installed in the final device. This circuit is making more sense than what has been unveiled on August, 25, and is not the same...
The wait is over.
Apple unveils its own version:
Let's start with the official statements:
... packed into its beautiful and refined design are breakthrough features that really matter to people, like Touch ID, a simple and secure way to unlock your phone with just a touch of your finger.
iPhone 5s introduces Touch ID, an innovative way to simply and securely unlock your iPhone with just the touch of a finger. Built into the home button, Touch ID uses a laser cut sapphire crystal, together with the capacitive touch sensor, to take a high-resolution image of your fingerprint and intelligently analyze it to provide accurate readings from any angle. Setting up Touch ID to recognize your fingerprint is easy, and every time you use it, it gets better. The Touch ID sensor recognizes the touch of a finger so the sensor is only activated when needed, preserving battery life. All fingerprint information is encrypted and stored securely in the Secure Enclave inside the A7 chip on the iPhone 5s; it’s never stored on Apple servers or backed up to iCloud®.
There is a little more technical information during the presentation from Phil Schiller (senior VP of worldwide marketing, Apple) (click on image to enlarge):
500 dpi: nothing special, this is the usual numbers we knows for years. One pixel is 50 microns.
170micron thick: OK, the chip is thinned, this is because they have not very much space, but this is pretty regular in semiconductor industry.
Reading the "subdermal layer" : this is the usual speech about Authentec sensors. Ah! marketing guys...
"The stainless ring is used as a detector" : OK, this makes sense. But the usual Authentec technology is using a ring to send an electrical signal in the skin, so that the infamous subdermal layer can participate to the reading of the fingerprint: is it still the case, or do they move to a pure capacitive sensor? I would guess no, because they added this sapphire layer which is not a good idea to read fingerprint -the distance between the sensor and the skin increases-, but is a brillant idea to get a nice design. Let's try to get more clues.
The official video gives some more informations:
Opps! there is one more layer? the green one (changing color later in the video...)? That is curious, because adding layers between the skin and the pixels is NOT a good idea, as it reduces the collected signal because of the increased thickness, and you may get some trouble with the air caught between the layers... We don't know if the silicon sensor is really like that, but at least we are sure it is a square sensor. Also, obviously, the button must be flat because the silicon is flat, and to have an homogeneous image, it's better not having a concave area. And it helps the manufacturing.
Maybe it is only some glue layer, so that there is no air caught between the silicon and sapphire. The high relative dielectric permittivity of sapphire (~10 where air is 1) will help a lot the capacitive sensor. Apple should have made their sensor using SOS, silicon on sapphire, with the sensing area on the backside, so the wiring is no more a problem... (I made my first circuit using SOS).
Or just a mistake of the marketing guys :)
The first image below is scaring for someone used to work with fingerprints:
The side of your thumb is certainly not the best place to get a usable fingerprint. Moreover, it is likely that the captured skin area is reduced, lowering the recognition performances. The second image is a much more better situation. At least, enrollment has to be special, because it is required to collect a lot different images. And people must learn that if you enroll like the second image, the first image will not be recognized. It is not clear if the stored template is enhanced AFTER the initial enrollment. That would help the recognition performances, but this may be also dangerous for the false acceptance rate...
Apple is making some efforts to explain how it is working, so we know that it is a minutia-based algorithm, which makes sense with a 500 dpi sensor. The fingerprint logo is nice, but well, anyone working with fingerprints knows that the lines are not broken as shown on the logo. And there is NO bifurcation (excepts the delta, sure), which never happens.
Not showing the real fingerprint image is a good thing from a security point of view. It is not a good thing for someone like me that would like to have an idea of the quality of the sensor, which is questionable because of the added layers over the silicon sensor.
The whole biometric community hopes that the fingerprint sensor of the iPhone 5S will properly work, because Apple devices are so exposed, if something wrong happens, then it could close the consumer market to biometrics...
What could be a nightmare?
Apple is not disclosing anything about false accepts, false rejects: the marketing guys
have chosen to act as if it was obvious, I suppose, so that users will be confident
with the technology. Fingerprint recognition never fails.
Looking at the first videos "hands on the iPhone 5S", it looks like it is working fine, and that's a minimum for a device coming from Apple where convenience is a must. The pretty long enrollment sequence where the user is asked to touch the sensor several times surely helps. Moreover, it is easy to adjust the internal threshold so that it works reasonably well. I didn't see any adjustment available to the user up to now, something that helps to adjust the security threshold, so I believe that it is an arbitrary marketing choice not bothering the user with such adjustment. And maybe the threshold is automatically adjusted, depending on the image quality linked to the fingerprint of this particular user.
We don't know if the system continuously learns, that is each time a positive identification is done, then the template is enhanced with the new data.
About what is commonly called "bad fingers", I expect that the sensor will be sensitive to dry fingers, chalk, moist fingers, mainly because of the thickness of the sapphire coating that does not help. But we may have a good surprise if what is described in the patents is working well, you never know.
What we don't know is the probability to get a false acceptance.
Is it 1%? is it 0.01%? In other words, how many non-enrolled fingerprints should I try to enter the system, from a statistical point of view? At least, there is a clue with the sensor size. Obviously, if the sensor is too small, then it will be easily confused as you can expect. On the other hand, if the sensor is larger than the fingerprint, then it is the best we can do.
Silicon cost is directly linked to the area of silicon. This is why I invented the silicon swipe sensing in the past. The main question is: what is the minimum size to get reasonable results? Then, we will try to know the size of the touchID sensor.
In 2004, I published a paper with Cogent "Reducing Silicon Fingerprint Sensor Area"
at the International Conference on Biometric Authentication ICBA 2004 Hong-Kong July, 15-17 2004
Lecture Notes in Computer Science, Volume 3072 / 2004, pp. 301 - 308
In this paper, we concluded that, with the best algorithm available (Cogent is making fingerprint systems for the police), there was clearly a limit with a 7mm overlap, AND we were speaking of swipe sensor, that is the height was much more. In other words, if you make a sensor using minutiae (as explained by Apple) that is less than 7x7mm, the performance results will be bad. If you want to make better, then you must go to a better resolution to use the third level properties (the pores).
Another way to get an idea of this minimum is the size of devices that have been done in the past, just have a look at this list. We see that all sensor manufacturers did not go below 6.5 mm (conveniently 128 pixels) for swipe sensors, and only Authentec made a 6.5 x 6.5 mm sensor in the past, and it was not the one that gave the best results.
Just to give an idea, the square on the image below shows a 7x7 mm area. Do the matching!
So what is the size of the touchID sensor? It is, at best, the square inside the stainless steel ring. Looking at the nice images from the Apple website, and knowing the width of the device (58.6mm), I estimate the internal size of the ring to 10mm, so containing a 7x7mm square. Considering that making a 128x128 pixel sensor is practical, my guess is that the touchID sensor is a 6.4x6.4mm square for the sensing area.
(A few days after, plans of the iPhone 5S were released (a short time...), and the ring is 10.9mm, external)
We are at the limit. So my guess is that we will quickly see some false accept reports.
Concerning spoofing, it seems that we keep the old marketing way from Authentec, that is: the sensor is reading the subdermal layer of the skin. So the average person will think that a fake cannot work, as the sensor is reading the subdermal layer. But saying that you read the subdermal layer (which is certainly true in the case of the skin) does not mean that other modes are not possible!
At least, there is nothing specific said about aliveness recognition, no specific sensor. IF it is exactly the same technology than Authentec used to made, and it is likely given the short time to create and produce a new sensor, then we'll have the same results about spoofing (and I'm used to spoof fingerprint sensors).
So it is just a matter of waiting till the first hacker will release a video with a fake finger accessing the iPhone 5S. It should be easy, considering that it is a static sensor (spoofing a swipe sensor is more complicated), and that the enrolment sequence will help to see if the sensor acquires something or not...
About all this security side related to cryptography, storage of the templates... there is no information about all this stuff, and obviously, there is nothing else to do at the moment than naively believe what Apple is saying about this. If Apple has properly done the work, then everything is hidden in the processor and never gets out. Proving that is difficult without opening the internal. But if it is not hidden, some leaks will happen.
Now, I'm waiting for more information, once the iPhone 5S will be opened so we can see the silicon. Or the first reported false accept or spoof...
One more rumour about that, after April and July.
TechCrunch article / Darrell Etherington shows that a cat's paw is able to unlock the iPhone 5S. And also some skin areas where fingerprints are not so usable.
So Apple's explanation about fingerprint minutiae is not the only way to recognize. Now we know that some form of pattern recognition is also used, or the cat -without fingerprints- should not work, and even cannot be enrolled.
As a result, it will not be very long till a false accept will be reported. Making pattern recognition over a so small area cannot be very accurate. Especially if you store many small areas in the template, you increase the chance of true recognition, but also false accepts...
The day after, it is a dog... iPhone 5S Fingerprint sensor works for dogs too! CNN iReport.
Apple disclose on their website (support) some information related to false accepts:
Every fingerprint is unique, so it is rare that even a small section of two separate fingerprints are alike enough to register as a match for Touch ID. The probability of this happening is 1 in 50,000 for one enrolled finger. This is much better than the 1 in 10,000 odds of guessing a typical 4-digit passcode. Although some passcodes, like "1234", may be more easily guessed, there is no such thing as an easily guessable fingerprint pattern. Instead, the 1 in 50,000 probability means it requires trying up to 50,000 different fingerprints until potentially finding a random match. But Touch ID only allows five unsuccessful fingerprint match attempts before you must enter your passcode, and you cannot proceed until doing so.
That's interesting. But it creates much more questions that requires answers...
Is this the probability every time I touch the sensor? It is likely this situation, because it gives better numbers. Remember that you are allowed to touch several times, with different areas of your skin, so each try increases your chances to enter... and you can enroll several fingers, so testing against several templates at the same time.
It is possible to enroll 5 fingers, so the final probability is ... 1 in 10,000. And it is limited to 5 attempts, so the probability falls to 1 in 2000.
Finally, comparison with the
4 digit pincode is not good, as described in this page:
Myth: A FAR of 10-4 = a 4 digit PIN code. The situation
is much more better in the case of the fingerprint, and Apple could take advantage of this
in their description... Maybe Apple considered that it was too complicated for their
Well, this is a nice FAR (false accept rate) for such small sensor. There are many biometric modalities that would like to have this kind of performances, together with the good FRR (false reject rate) as observed, likely a few % -there is (not yet) reports about fingers that were not able to be recognized, and it will not be the case as the sensor is already working for difficult prints such as cats and dogs...
Did Apple tried with 50,000 persons, well even 5,000 with ten fingers ? I don't think so, because it is a too long work (and how keeping such secret?), and you cannot rely on only one test to get some probabilities. In general, there is some data collection (collecting raw images), and the recognition algorithm is tested against the database, and you get the raw FRR and FAR. Then, you need to translate this into your system, here it seems that several images are taken and stored, looking at the enrolment procedure. It is also important to acquire your database using a system representative of your final system, especially with the same ergonomics. Well, the basics for any biometric professional guy...
As a reminder, the myth of the 0.0001% FAR and 0.0001% FRR system. Here, it is 0.05% FAR, FRR unknown, likely 2%, maybe even 10%.
Sept. 20, 2013: iPhone 5s teardown
Here is the home button (credit: Ifixit).
Hey guys, remove the sapphire, so we can see the silicon sensor and estimate the thickness! But it is likely very well glued...
Note Suwanchote has posted a video on YouTube, showing it was possible to enroll and recognize 5 different fingers in one slot, so you can enroll up to 25 fingers.
This is a very interesting fact when connected with the 1 to 50,000 probability discussed just above, because this is lowering the announced FAR to 1 to 400 when enrolling 25 fingers and trying 5 times. We are not very far to see the first false accept...
This is also giving another important clue during enrolment: the algorithm does not search to create a larger print from the small raw images, which would be very difficult to make as the recovery would be too low to
Why is it working like that? Just to make sure that the owner of the iPhone is not rejected. Everything has been done to reduce the false rejection rate, because this is the very first feature the regular user sees. The more images you store, the better the recognition will be. Unfortunaly, if you relax too much the system so the good guys are not rejected, this is also the same for the bad guys, and the false reject rate increases.
The Chaos Computer Club, which is used to spoof fingerprint sensors, has fooled the touchID sensor very quickly, with some usual fake finger (see the video on YouTube), even if they did not understand how it happened from a physical point of view (it is only a matter of electrical conductivity with the stainless steel ring, this is why they need moisture...) and said something wrong about the resolution of the sensor which is 500 dpi, as all sensors produced by Authentec excepted the oldest ones, 10 years ago.
There are simpler ways to do the trick, but well, no matter, it tooks less than two days to spoof it. So there is nothing special in this sensor related to aliveness detection, it is working as usual.
Finally, the stack of materials described in the presentation movie was pretty good! We can see this green-blue layer which is a kind of glue between the sapphire and the chip. Unfortunately, Chipworks didn't say a word about the thickness, which is a big problem concerning the resolution as it blurs the images: imagine you have one pixel every 0.05mm, and what is above is more than 0.10mm, twice the pitch!
Concerning the chip itself, Chipworks has an excellent note: An unusual feature of the die is the dark areas at the top and bottom edges. It appears that the silicon has been partially etched away to provide a recessed shelf within the die area for the wire bonds. That allows the top surface of the chip to be bonded straight onto the sapphire disc, minimizing the finger-to-chip distance. That's extremely unusual. Another technique would have to use crossing vias (silicon-thru) to get the bondpads on the back of the chip, but this is likely not yet available -or too expensive- in the foundry they used.
Sensing area size
The chip is 6.6mm x 6.1mm (I guessed 7x7mm), with an array of 88x88=7744 pixels at 500dpi (so the usual 50.8 microns pitch of fingerprint sensors), so the sensing area is as small as 4.5mm x 4.5mm: that's too small to get good recognition performances! I was expecting 128x128 pixels (the purple square), and was not confident, so with 88x88 pixels (the red square), and moreover with a thick layer that blurs images, forget it...
This is mainly described in the 20130307818 application from Pope & als.
So we can guess that Apple will merge the sensor and the display one of these days...
Not a real surprise: increasing the sensing area while keeping the same button will help to enhance the recognition rates.
The resolution seems to be still 508 dpi, but looks like it is not a square chip: 112x96 pixels (I am unsure if I'm still able to properly count pixels, that's weird numbers...), so a little bit better than the previous version.
They are now using TSV to avoid the recess and ease the packaging.
So the rumours about the next iPhone 7 are starting :)
A patent about sound? Why not! Not sure we will see that in a close future. There are already some ultrasonic sensors, Qualcomm for instance.
So the rumours about the next iPhone 8 are starting, as usual.
(2017 Sept) And the winner is... everyone! Finally, there are 3 iPhones:
Apple gave an explanation: people were handling the device for the stage demo ahead of time and didn't realise Face ID was trying to authenticate their face. After failing a number of times, because they weren't Craig, the iPhone did what it was designed to do, which was to require his passcode.
While this is explaining why the passcode pop-up, with a 1:1 000 000 chance to get wrong, this is just stupid to run like this. This is OK to ask for the passcode, but keep the Face ID working at the same time, this gives a chance to enter and not to fail at the first demo.
I believe that the real world is not like Apple wants: the FAR is likely far lower than they said, and it is a good practice to limit the number of attempts -as they do with the fingerprints, so you lower the chances to get a false accept (and virtually, you are unable to show again a false accept if you get one...).
Really? after the Samsung disaster? Coming from Apple, I would expect the sensor will cover the full display, that would be an extraordinary feature and really impressive.
involved: JDI, Qualcomm and GIS
According to Chinese Apple analyst – Guo Mingxuan – from Tianfeng Securities, the iPhone 10 will be featuring a fingerprint scanner on the side when it will come out in the first half of 2021.
- integrating a fingerprint sensor into the Digital Crown
- fingerprint sensor under the screen.
Maybe a Series 7 (2021) feature.
Well, not so new, this feature is existing for years on some mobile phones. Sharp Aquos, Sony Xperia X & XZ Sony Z5, Vernee Mars, Ulephone Future, ZTE V7 & Nubia Z9, Huawei Honor 7i, X3, Turing Phone, Saygus V2, & not speaking of swipe sensors.
Now guess what the next iPhone will have -wearing masks does not help Face ID :-)