3.2) Case study 2: A married couple in their 20s, John and Mary, are
seeking a business loan to help them realise their long-held dream of
owning and operating their own restaurant. Mary is an experienced
accountant, and John is a promising graduate of a prominent cooking
school. They share a strong entrepreneurial desire to be "their own
bosses" and bring something new and exciting to their local restaurant
scene; outside consultants have reviewed their business plan and
assured them that they have a very promising and creative restaurant
concept as well as the skills necessary to successfully implement it.
They should have no trouble receiving a loan to get the business off
the ground, according to the advisors.
John and Mary's local bank loan officer uses an off-the-shelf software
tool to evaluate loan applications, which synthesises a wide range of
data profiles obtained from hundreds of private data sources. As a
result, it has access to details about John and Mary's lives that go well
beyond what was requested on their loan application. Some of this
data is obviously relevant to the application, such as their history of
on-time bill payments. However, much of the information used by the
system's algorithms is information that no human loan officer would
think to look at or have access to, such as inferences about their likely
medical histories based on their pharmacy purchases, data about the
books and movies they watch, and inferences about their racial
background. While much of the material is correct, some of it is not.
A few days after applying, John and Mary receive a call from the loan
officer informing them that their loan has been denied. When they
inquire as to why, they are merely told that they were evaluated as
'moderate-to-high risk' by the loan system. When they seek for further
information, the loan officer says he does not have any and that the
software company that designed their loan system will not tell them
anything about the proprietary algorithm or the data sources it uses,
or even if the data was validated. Even the system's designers do not
know how the data led to any particular outcome, they are told; all
they can say is that the system is 'usually' dependable statistically.
When John and Mary inquire about appealing the judgement, they are
informed that they will be unable to do so since the system will just
process their application again using the same algorithm and data,
yielding the same outcome.
Answer the following questions based on Case study 2. Your answers
must be specific to the case study.
Page8 of 9