The New York Times has recently published one of their in-depth,
riveting descriptions of how
credit card companies use everything they can learn about us.
Any detail can be meaningful: what time of day you buy things, or the
quality of the objects you choose.
The way credit collectors use psychology reminds me of CIA
interogators (without the physical aspects of pressure). In fact,
they’re probably more effective than CIA interogators because they
stick to the basic insight that kindness elicits more cooperation than
threats.
So who gave them permission to use our purchase information against
us? What law could possibly address this kind of power play?
There’s another disturbing aspect to the data mining: it treats us
all as examples of a pattern rather than as individuals.
Almost eleven years I wrote an
article
criticizing this trend. The New York Times article shows how much
we’ve lost from what we consider essential to our identity–our
individuality.
Update
This article drew six comments in a few hours–thoughtful and valid
comments, which have made me set down attitudes into words. Now we can
look put the attitudes under a light and see what makes sense, or
doesn’t, to readers.
The article contained two levels of criticism: a criticism of data
mining to build up composite pictures of individuals, and a criticism
of the use of data accumulated from routine transactions to manipulate
those individuals.
Building up a composite picture
Of course, a company that reaches out and does any marketing has to
target people. Someone who bought the O’Reilly book
Even Faster Web Sites
(sorry about the plug) might appreciate a notification about our upcoming
Velocity conference,
which was founded by the book’s author and covers the same topics.
Someone who bought a book on a totally different subject wouldn’t want
or respond to the notification. O’Reilly does this kind of targeting,
like most companies, and until everybody participates in truly
frictionless information exchanges, companies will have to continue
doing it.
Aggregated information is useful too. Organizations that mine public
data for evidence of health epidemics can identify likely sites and
investigate them further. The data mining is understood to provide an
approximation of the truth.
Where I see a problem is when the increasing quantity of constant
information refinement shades over into a qualitative change. There’s
a difference between a campaign targeted to 500 likely customers and a
campaign targeted to one.
At some point the composite portrait starts to look so much like a
person that corporate decision makers can begin to believe it
is the person. The portrait becomes like a replicant, or like
the statues that came to life in myths from Pygmalion to Pinocchio.
Joseph Weizenbaum, creator of the classic Eliza program, was shocked
to see that people treated his “doctor” program like a human
interviewer. There were plenty of computer programs that prompted the
user with questions and gave varied responses based on the answers,
but none had imitated a person so realistically.
Nowadays, nobody would be drawn in by Eliza. And perhaps companies and
customers alike will get used to composite portraits. Perhaps the
companies will send their composite to each of us and we can update it
to make it more accurate. That will be a very different world, though.
Now we can turn to the next level, manipulation.
Manipulation
I’ve read numerous accounts in biographies and articles about
interrogations, and talked to a couple people who have undergone
interrogations. I haven’t been on either side of an interrogation, but
I’ve been deposed for a court case. All these situations remind me
vividly of the exchanges reported in the New York Times article.
In these exchanges, a well-armed caller is laying, like a silkscreen,
a composite over the real person and trying to manipulate the result.
It’s not exactly a case of asymmetric knowledge (because at least in
theory, a customer could also learn a lot about a company and use that
knowledge to manipulate it). It’s more insidious: an employee carrying
out a precise initiative on behalf of a company–a machine in the
service of a goal–approaching the targeted customer in an informal
manner that brings out a natural, human, empathetic reaction in
customer.
Interrogation always takes place in the context of an open or implied
threat–there would be no reason for making the contact otherwise–but
as I mentioned in the article, the interrogation goes best when the
threat is raised only rarely and strategically. A feigned sympathy and
heart-to-heart engagement is the path to the most desired outcome.
In a sense, now, the employee has become the replicant. He is using a
careful counterfeit of human responses to induce the behavior he or
she is paid to induce. This is ethical when dealing with a criminal,
although even then US law limits (based on the Fourth Amendment) the
gathering of relevant information by the interrogator beforehand. I
question how ethical it is in a business situation, especially when
exploiting information given by the customer for entirely different
purposes.