LR6: How I Processed 20,000 Faces

Lightroom 6 offers face detection technology.  What this means, in the ideal case, is that Lightroom can detect and then suggest the names of people in photographs.  When a name is confirmed it is added as a keyword to the photograph.  

For existing Lightroom 5 systems each photograph in the library is scanned for faces.  When something Lightroom thinks is a face is found it puts the face on a list of Unnamed People. The Lightroom user then reviews this list of faces and gives each one the name of a person.  When a face is named it goes to the list of Named People.  The list of Named People can be thought of as a list of buckets where each bucket holds all the faces for an individual. 

In my case 20,286 faces were found in 118,680 photographs for 282 Named People. It took 48 hours to complete the process of finding the faces. That is a rate of 423 faces / hour. It took me another 6-7 hours to name the faces.

Before I describe the approach a few observations:

• Not all faces are always identified in a photograph.  What this means is that you can’t assume there will be 100% coverage. The reasons for this gap is covered below.
• It is not unusual for the process to find a face where none exists (e.g., in a lump of coal).
• When a face is detected, Lightroom 6 will attempt to suggest a name for that face.
• A face may be in one of four states: [1] unknown name [2] suggested wrong name [3] suggested right name [4] confirmed name.

My approach to effectively and efficiently completing the naming process was based on the notion of the “long tail,” coupled with the understanding that Lightroom is able to suggest names once it knows what to look for.  The Long Tail Theory suggests there will be only a few people that make up the bulk of the faces to be named. This coupled with Lightroom’s ability to suggest matches (once it knows what to look for) should reduce the amount of manual work to be done. Put another way, every time Lightroom can suggest the correct name for a face effort is reduced to just one click (confirm) versus typing in a name.

A corollary to the point above is that there is a large number of People with only one face (image) recorded.  How this translates in practice is that family members and close friends will fall into the head while the tail will be comprised of a lot of unknown people.  Why will there be unknown people? Pictures taken at tourist sites; street photography; restaurant, etc. all pick up faces of unknown people.   

There is a third group, say the body, that is made up of friends and acquaintances. One needs to decide whether to name the faces or not. To do so implies more work so it is  judgment call as to whether it is worth while or not.  

Following the 3-step approach summarised above I focused first on close friends and relatives. Here I was able name 8,956 faces leaving 11,330 faces in the Unnamed People list. Obviously a daunting task remained, yet that I had completed nearly half was encouraging.

The second step was to focus on the tail; the faces for which I had no name.  The goal here was to delete them.  Here I was able to reduce the number of faces of Unnamed People to 1,122.  Those that remained were acquaintances that fall into that middle place between head and tail, but also included family members and close friends I missed in the first pass.  

The last step was to go through the remaining faces in the Unnamed People list. Very often this was a picture-by-picture assignment which, needless to say, is time consuming and mind-numbingly tedious.  Hence the trade-off of how many do you want to keep.  

With the overall approach in mind, there a few tactical steps that can be taken as one works through the process:

• It is important to leverage Lightroom’s ability to suggest names in order to reduce typing and thus time.   To make this work, lightroom’s algorithm needs to be “primed” with a small set of faces for a name (a person) and with that it seems quite capable of making good suggestions.  So I started with highly photographed family members and close friends. I reviewed the list of Unnamed People and started to manually assign names.   It seemed that with as few 3 or 4 pictures, Lightroom was able to start making suggestions.  As the list of confirmed faces grew for a name the accuracy seemed to get better, but only to a point.  As suggested above there are limits on what the face detection technology can do:
  • The process works well for faces looking square into the camera.  The further off square (e.g., in profile, tilted) the higher the miss-rate.   
  • Sunglasses impact whether a face is identified
  • Hats, especially those with wide brims can impact detection
• Once I had identified a few big hitters on the Head, I drilled down into their individual panels to review both the list of faces confirmed for the person but importantly the suggestions Lightroom was able to generate.  Correct suggestions can be easily confirmed wit ha single click.  

• The list of Similar Faces often starts with those of the person you are working on.  Lightroom seems to present “best fit” first, however, it is worth going through the first few pages of the similar list to confirm any additional matches.
• I found it necessary to cycle through the list of people a few times as each time I returned the algorithm had identified more correct names.  Although after 3 or 4 revisits no more correct suggestions where offered.  I grew my list of Named People to include all those for whom I had more than just a couple of pictures. 
• It is a good idea to review the list of Confirmed faces for a name and make sure they are all correct.  An incorrectly confirmed face seems to affect Lightroom’s algorithm.  

•  In the second step, where I worked on the tail, the objective was to delete faces for which I had no name.  This involved going through the list of Unnamed People and selecting and deleting accordingly.  The trick however was to try and find a means of deleting large selections in one shot.  Lightroom offers different sorting orders for Unnamed People, including: Suggested Name; Film Strip Order; Stack Size; and Popular Name.   None was obviously better than the other for this purpose and I found myself switching among them. Although I did start with Stack Size on the assumption that it would group like-faces together.  After that I changed the sort to Popular Name and then started reviewing from the end of the list (i.e., the least popular) on the assumption that that is where I would find the most faces to be deleted. It’s hard to tell if this indeed was the case.   
• My guiding principle in this step was to not delete the face of a person I knew the name for.  I would make the decision to keep or reject in the third step.  
• I was often able to delete 30 or more faces at a time; sometimes over 100.  
• I did run several cycles through the Unnamed People list.
• In this second step I did confirm names as well, when they were presented.

• In the last step I was down to 1,122 faces in the Unnamed People list.  While there were a few people that had multiple faces in the list, this phase quickly turned into a picture-by-picture naming processes.  It is at this point where one can choose among a suite of options:
  • Delete them all
  • Work through them and may be pick and choose
  • Defer.  Rather than name the remaining faces in one shot, do a few each day and whittle it down.  This is the approach I’ve taken.
• Lightroom saves the names as keywords and associates them with the photograph.  My Lightroom 5 practice was to manually keyword photographs with the names of the people in the shot.  I decided to follow the same naming convention, last name first name, as done previously.  In some instances existing keywords were used.  In other cases, duplicate keywords were created.  It is possible to differentiate between the two by looking at the Edit Key Word Panel.  Keywords created through Face Detection will have the Person field selected.  The other thing to keep in mind is that the number of entries is likely not to be the same.  This is due largely because face detection is not 100% for the reasons cited above, but manual (pre-LR6) detection is not 100% either.  I merged the two keywords together. I’m not sure if there are any side effects to this decision.