Within just a few decades facial recognition has gone from being some staple of highly advanced security systems in movies to existing all around us and even in the palms of our hands. The technology has been implemented in several ways in our society. Some phones use facial recognition to grant access and some governments like China and the US are using facial recognition on databases like driver’s licenses for a variety of reasons.

Then we have got fun things like Snapchat filters that utilize facial detection. Now there is a big difference between facial detection and facial recognition. With facial detection the computer is saying yep that’s a face and with facial recognition the computer is saying yep that’s your face.

Is facial recognition something to fear? In some ways COVID-19 gave us a reprieve on facial recognition – pretty hard to do facial recognition with masks and to be honest, one would probably be happy to use a mask on a regular basis as long as one can socially get away with it.

Unfortunately, majority of us may not know the implications of facial recognition technology and how this is being used/misused. One may not like the Orwellian references but knowledge about how it can be used against us will spring up many surprises.

What is Facial Recognition

Facial recognition is a technology that is capable of extracting a human face from a digital image or a video frame grab and matching it against a database of faces to verify or recognise the individual.

The two main tasks performed by the system are:

• Verification – comparing a new input face to a known identity (one-to-one analysis) – for example, unlocking of smartphones with facial identification
• Recognition – comparing an input face to a database of multiple face identities (one-to-many analysis) – generally done by law enforcement / surveillance systems. (An interesting read is the INTERPOL Face Recognition System).

How Does it Work

The ability to confirm identity is something our brains pick up on pretty quickly when we are young, but it is actually a pretty complex problem for computers. When we think of a human face, we probably think of a pretty basic set of features – a face has eyes, a nose, and a mouth. But obviously there is more to a face than just these features. In fact, to get a good idea of how complex this problem can get one can draw a bunch of different simple faces and you will quickly notice that they differ in a lot of factors like the width of the nose the distance between the eyes the shape in the size of the mouth and so on.

So far most facial recognition is done by looking at the proportions of your face relative to the position of your eye’s ears nose mouth and sometimes eyebrows and chin. There are various algorithms for doing this. We can refer to this technology as 2D facial recognition since the analysis is performed on a flat image acquired from either is picture or a video frame grab. Some facial recognition technologies evaluate up to 80 factors on the face to help identify unique features and ultimately identities. These features get to be pretty detailed observing things like the depth of eye sockets, the height of cheekbones, and the shape of the jawline. Most of it is done by AI machine learning. If we do not want to write code for facial recognition, we can send the images to Amazon AWS and they will process them. So today it is easy for any app to do facial recognition through the use of Amazon AWS.

Another way of doing facial recognition is by doing contour mapping off the face, for example that is done by iOS and Pixel 4 phone facial recognition. This is done by a combination of laser and infrared. We can refer to this kind of facial recognition as 3D facial recognition.

 Accuracy

There have been many complaints in the press about the accuracy of facial recognition. For example, it was tested against the African American members of Congress and some of them were identified as criminals by this facial recognition. The facial recognition system employed was claimed to have flaws with dark skinned faces. For obvious reasons it means that the AI cannot demarcate the position of the eyes versus nose and mouth in dark skinned people. This will be true in the dark and unlit areas well.

Facebook also uses this kind of recognition logic meaning eyes nose and mouth alignment. But because of large volumes of data, the AI is able to capture all the different angles of a face in a much better way than its competitors. Most estimates peg the accuracy of Facebook facial recognition to be around 98%. There is no doubt that Google and Amazon will strive to reach that same level of accuracy for those of us feeding its AI via Google Photos and other apps. 2D facial recognition is pretty accurate these days.

As far as 3D facial recognition is concerned, it does not have a large enough database yet. The technology is slated to overcome the short comings of 2D facial recognition due to is ability to used depth details of the face. In fact, the iOS phones are able to validate faces even with sunglasses and even if the face is partially obscured by inanimate objects.

Exposure to Everyday Facial Recognition

Facial recognition is being done automatically in more places and apps than we actually realize, and this database is growing like crazy. Does this have any major effect on our privacy?

Most people are not aware that even common day-to-day activities are tied to facial recognition and we are not talking about facial recognition for secure entry to your phone. Simple events like going to the supermarket will subject us to multiple instances of facial recognition. That is how bad it is.

Jogging around our neighbourhood, taking selfies on the phone – seemingly innocuous events include facial recognition. This is very bothersome because we expect facial recognition when we are going to the airport or visiting a government facility, but do we expect the same while taking a walk in our own neighbourhood. The problem is that most of us are unwilling participants in the facial recognition infrastructure and many of us who are aware of the technology cannot do anything about it.

In the following paragraphs, I will attempt to inform you where facial recognition occurs including unexpected places, then I will try and explain about how it is evaded and finally I will explain why this is such a big deal. Why facial recognition will affect our privacy and ultimately our freedom.

So, what are these unexpected instances of facial recognition?

In any modern city there are street cameras actively taking facial recognition shots. You will get traffic challans with your face prominently recognized based on your driver’s license. I have witnessed these traffic challans in action and in some geographies, they actually state that they use facial recognition. They also use cameras to actually track both your face and your license plate.

A well-publicized event was when the LAPD used facial recognition for pre-crime tracking of people associating with gang members. It was done through a contract with Palantir, which did the pre-crime analysis of the profiles of people living in gang areas.

Google explicitly states that they do facial recognition on your photos in Google Photos, so, if you upload your photos there, it will behove you to be aware of this. Apple also does facial recognition of photos sent to iCloud. They want to suggest photos to your friends, that is their excuse.

Facebook is the world leader in facial recognition. Just being on Facebook exposes you to constant facial recognition tied to your Facebook account. If you have not been hiding in a cave, then perhaps your face has been tagged in Facebook. Tagging, as you know is suggested by Facebook, when they recognize someone as being a friend. Obviously, they can recognize everyone, even if they are not a friend. If you walked around Disneyland and a family next to you takes a photo, if you are in the shot and they post it to Facebook, then Facebook will know you were in Disneyland at that exact time in with your accurate location. This is pretty insidious.

Here is another new way we do not even think about. Ring cameras (Home Security Cameras). Buying them is a dumb idea if you are a privacy focused person. Do you understand what you signed up for when you got Ring? It means it is okay with you to upload your videos to Amazon for facial recognition and have those videos be available for use by law enforcement or anybody else with a government contract like Palantir. What is worse is when one jogs around the block ring cameras of the neighbours are basically spying and sending information about the location of joggers and passers-by to third parties and you cannot do anything about it.

Do you know that Ring is promoted by law enforcement because of this particular availability of videos that they get access to? The promotion, by the way, is an actual contract Amazon tells them that they can get access to the videos as long as they promote Ring. This keeps getting worse, our lack of ability to deny permission to facial recognition is very frustrating anytime there is a camera. We now have to worry where are the newest cameras being added.

About the autonomous vehicles – new 2021 model cars will be partially self-driving now, not just Tesla, but many models will have this kind of capability to do any kind of autonomous driving. It means, lots of cameras on the car, in fact, a car like a Tesla will be driving around and basically enable the possibility of tracking any person on the street that was spotted by the car camera for safety reason. Tesla will definitely have a record of every person it encounters on the streets. Where does that video go?

Here is another one, that you may not think about. If I can take a picture of you, I can use a Reverse Image Search to find out who you are on the Internet. Can you imagine the access of Google and Facebook employees – they have access to facial recognition from Google Photos and Facebook, which they can reverse image search. Facebook, by the way, refined its facial recognition because many of us have uploaded thousands of pictures on Facebook so its AI keeps learning.

For the moment, we can get away from some aspects of 2D facial recognition because of COVID-19.  If one blocks his/her nose and mouth, the facial recognition will not work or if one wears sunglasses. You can try it yourself; it is difficult even for a human to recognize people entirely from a face alone if they are wearing a mask. Usually if one cannot see the person’s entire body, to add to the input, it is exceedingly difficult to recognise who it is with a masked face.

So, what is the big deal with actually getting facially recognized through these automated means?

Would you like to walk around with your name address and birth date and a permanent id attached to you at all times and big enough to read. Well, that is exactly what happens in facial recognition.

Back in 2013 or so a company in Russia decided to make an app with facial recognition and simply by pointing your phone on anyone you can then look at their VK profile. The VK (short for its original name VKontakte meaning InContact) app is the Russian equivalent of Facebook. At the time with their limited tech, they were already 80 percent accurate finding somebody’s face on VK. So, if you want to stalk some woman just take her picture then you will be able to find out all her info on VK using this app. At some point anyone with a large enough collection of photos either stolen from social media sites or the Internet can accomplish this.

Companies that already have a huge collection of social media data include Spokeo, Intelius and Acxiom to name a few. These companies collect all the identifiable information including your house, your car, your employment records, your legal records, arrests, divorces, lawsuits etc and this can be matched to a face. This is with you not even being on the Internet. You could be hiding on the Internet recently and your past will show up through these third-party databases and if these companies release an app with facial recognition matched to their databases – we will all be in the dock.

Doxing (real life stalking) will blow up big time and there will be no pseudo anonymity on the Internet any longer or creating a different persona on the Internet will be a thing of the past.

The other threat of course is with government mass surveillance and Google/Facebook surveillance – you will not need to be on the Internet – through the sources I mentioned earlier which includes self-driving cars and traffic cameras your every move will be known in profiled. Your participation in protests will be inhibited because your face will pop up in a database and get your profile. This is already an issue in dictatorship and countries with authoritarian governments.

Many places in the world have extremely dense camera systems. Cities like London and New York are already known for having lots of security cameras, but with the use of Ring cameras the governments are getting an expanded infrastructure for free. So, there is the capability now for so many players to know every place you have been to. Every time you visit the market your face is captured at a traffic intersection that you cannot avoid. Privacy focused people may have to map out locations of cameras on their route and will take the longer route to avoid them.

The possibility of misuse of facial recognition technology for mass surveillance by various entities including nation states and big tech is very real. Populations living in countries with a dictator/authoritarian leader can vouch of it, where one could be jailed for having a different idea. So, this is not theory. Imagine, if that dictator had access to this information, he would make sure that his opponents disappear after he learns who they are. If you are a journalist trying to keep your source private like Edward Snowden this is going to be extremely difficult. Your source could end up whacked or suicided.

These are the real impacts of the loss of privacy. When it comes to freedom fighting, a powerful and corrupt government could render it impossible when facial recognition is rampant. Active participation and mobilisation of opinion on the Internet is no longer important. It is a scary thought that there is no place to hide from facial recognition and what would become of some one who simply does not agree with someone powerful.

The newest attack which uses facial recognition is Deepfake someone could take your face and your voice and use the Deepfake app for the video and the Adobe Audio Manipulator (VoCo) app for voice and basically, they can duplicate you saying something else. Someone could come up with a video of people singing praises about Stalin or Hitler. It is scary.

In a future world, this will become worse – privacy will become non-existent in a world with extreme facial recognition. Combine that with your love of Alexa Echo and we are truly in a world without privacy. Facial recognition is the final nail on the coffin. The only way to stop this is for us to realise its impact and force our lawmakers to come up with laws that prevent the misuse of the technology.

It is really sad that back in the days before mobile phones people were extremely sensitive about being clicked without their permission – but the millennial generation has minimized this. Now with phones tracking your location, facial recognition and voice listening by Alexa/Siri happening 24/7, the future is truly scary. It is only a matter of time when some powerful person will misuse the technology like big brother in George Orwell’s 1984.

We have to legislate new laws to prevent the misuse of the facial recognition infrastructure by our law enforcement and business enterprises. We have to come up with detailed policies and procedures to prevent this collection and abuse of facial recognition data because there is no technological way to stop it.