The importance
of data quality
for trustworthy
computer vision
Introduction: The growth of computer vision
Computer vision is not a new concept. Since the 1950s
and 1960s, scientists have been working on how to
replicate the human visual system in digital form.
By studying how neurons react to stimuli,
Approximately three-quarters of revenue will
scientists at the time observed that
derive from hardware, with the balance of revenue
human vision is hierarchical and that
coming from computer vision software solutions and
neurons work by detecting simple features
services. More broadly, according to PwC, AI is set
such as edges, before feeding into more
to contribute $15.7 trillion to the global economy -
complex features such as shapes.
exceeding the current output of China and
While early experiments to link a camera to a
India combined.
computer and describe what it saw were far from
Importantly, though, the growth of AI, and in particular
successful, the technology is now experiencing
computer vision, cannot be allowed to go unchecked.
something of a renaissance. This is thanks largely to
We have all read about the problems caused by poor
advances in the field of Artificial Intelligence (AI), as well
datasets or the unconscious or conscious bias of the
as innovations in neural networks and deep learning.
developers behind the algorithms being used. Instead,
Before the advent of deep learning, the tasks that
computer vision could perform were very limited and
required a lot of manual coding - with deep learning
developers no longer needing to manually code every
what’s important is that businesses involved in AI have
an ethical framework in place from the very beginning
to counter any potential problems further down
the line.
single rule in their vision applications. As a result, we are
Only by doing this can they hope to produce accurate
now seeing an explosion in the computer vision market
and fair results when implementing their AI strategy.
as potential new applications are developed including
self-driving cars, cancer-detecting medical scans as well
as content moderation for user-generated images
and video.
In this white paper, we look at the challenges currently
facing those developing computer vision applications
and how they can be overcome. We also look at some
of the industries currently implementing computer
According to a report by the Bit Refine Group, the
vision solutions and the benefits they can bring.
computer vision market is set to increase from $6.6
billion in 2015 to $48.6 billion by the end of this year,
Chris Price
at a compound annual growth rate (CAGR) of 32.9%.
Associate Editor, AI Business
W W W.SHUTTERSTOCK.COM
2
Introduction: The growth of computer vision
The Data Challenge: Tackling AI Bias
We are living in an increasingly visual age.
According to The Conversation, more
than 3.2 billion images and 720,000 hours
of video are shared every day just on
social media alone - making the task of
distinguishing what’s real and what’s fake
more difficult than ever before.
In addition, new applications are being developed
which require advanced visual acuity. For example,
robots moving across a factory floor must be able to
distinguish between objects such as humans and other
machines, while sophisticated artificial intelligence is
required to tell the difference between medical scans
showing patients with cancerous cells and
those without.
In all these scenarios, computer vision plays a vitally
important role. It allows machines to accomplish
a variety of tasks that would have once only been
possible with a human brain. However, it requires
several skills including segmentation (dividing an image
into parts and examining them individually), pattern
recognition (recognizing the repetition of visual stimuli
between images), object classification (classifying
objects found in an image), and object tracking (finding
and tracking moving objects in a video).
Nor are these the only attributes. Computer vision
also requires additional skills including object detection
(looking for and identifying specific objects in an
image) as well as facial recognition: an advanced, and
Publicité
somewhat controversial, object detection system
that can identify human faces. For example, facial
recognition can be used in the healthcare sector
to help elderly people living at home identify their
caregivers. It can also be used within nursing homes to
ensure the elderly have a bespoke care program that
meets their individual needs.
W W W. AIBUSINESS.COM
3
Dangers of inherent bias
There is a popular saying in data science: ‘ garbage in,
garbage out’. This is especially true of computer vision,
where the quality of the images and how they are
labeled can affect the accuracy of the results generated.
of professionals in the AI and data science fields
are women, according to the Alan Turing
Institute). “AI systems are biased because they
are human creations,” said Genevieve Smith and
Ishita Rustagi in When Good Algorithms go
Sexist in the Stanford Social Innovation Review.
For example, it was the release of ImageNet, a
“Who makes the decisions informing AI and who
dataset containing millions of tagged images, in 2010
is on the team developing AI systems shapes their
that helped propel computer vision’s rise. At the same
development,” the article suggests.
time, it also opened the industry’s eyes to many of
the challenges that needed to be addressed for the
technology to become successful – particularly the
need to counter any inherent bias.
Nor is it just gender bias. Often there is a race bias
too. For example, the landmark Gender Shades
project found that datasets comprising mostly
white and male faces resulted in much lower levels
“Many truly offensive and harmful categories hid in
of facial recognition among women, especially those
the depth of ImageNet’s Person categories. Some
of color. Indeed, error rates recognizing female black
classifications were misogynist, racist, ageist, and ableist.
women aged 18 to 30 were 34% higher than lighter-
… Insults, racist slurs, and oral judgments abound,”
skinned males, claimed Harvard University in Racial
wrote AI researcher Kate Crawford in her book Atlas
Discrimination in Facial Recognition
of AI.
Indeed, datasets can reflect either unconscious or
sometimes conscious prejudices of the developers
themselves, many of whom are male (only 22%
Technology. They also found that data sources
(such as photographs) were also not equitable with
default camera settings not optimized to capture
darker skin tones, resulting in lower quality images of
Black Americans being used for AI.
W W W.SHUTTERSTOCK.COM
4
Creating Trustworthy Computer Vision
There’s no doubt that computer vision has
2. Diverse datasets
come a long way since its inception as a
branch of AI in the 1950s. Deep learning has
not only enabled a whole raft of potential
new applications, but it has also helped to
improve accuracy levels considerably.
Despite the advancement and adoption of machine
learning, there is much work to be done related
to bias, diversity, and inclusion within the datasets
themselves. Leaving out specific communities
from datasets results in a lack of representation
1. Audited process
embedded within algorithms.
Machine learning models are commonly trained
One manifestation of this problem is facial
on large amounts of real-world data. This could
recognition being unable to process black faces,
entail the risk of inheriting human biases, resulting in
as highlighted by The Algorithmic Justice
ethical harm against specific communities. Machine
League within their documentary Coded
learning models can be very powerful and - without
Bias. Facial recognition also can misidentify faces,
proper testing - AI can not only recycle these biases
resulting in harm to those communities.
but even enhance them. AI ethics researchers are
pushing for solutions that involve more transparency
in model development and dataset training. However,
regulations on AI are still coming into effect.
Therefore, it’s necessary to establish a rigorous
testing process for biases before these models
reach production. AI models should be tested for
several different biases, from gender to ethnicity and
behavioral bias. Statistical tests must be carried out to
ensure that there is no preference for a certain subset
Publicité
of characteristics in the data that could indicate the
possibility of biased results. If an AI model does show
some bias at the testing stage, then it needs to go
back to the designing stage for retraining.
Therefore, to ensure computer vision is as accurate
as possible, the assets must be obtained from
as diverse a range of sources as possible – truly
representing the diverse world we live in.
For example, Shutterstock sources content from
a network of two million contributors from more
than 150 countries. Says Dr. Alessandra Sala,
Sr. Director of Artificial Intelligence and Data
Science at Shutterstock: “In 2020 Shutterstock
also established The Create Fund to empower
historically excluded artists, help fill content gaps,
and further diversity and inclusion within our
content library and contributor network.”
W W W. AIBUSINESS.COM
5
3. Diverse and inclusive teams
4. Ethical AI training
A diverse AI workforce made up of different races,
Responsible design and use of AI starts with
genders, ethnicities, and ages will decrease the likelihood
training the technical teams to question their
of racial, gender, ethnic, and age discrimination - and
inventions with wider social, economic, and
increase trust in computer vision systems.
cultural perspectives. For example, Shutterstock
According to the World Economic Forum, 78%
of global professionals with AI skills
are male, while a study by the AI Now
Institute of New York University showed
that over 80% of AI professors are men. Women make
up only 15% of AI researchers working
at Facebook and 10% of AI researchers
working at Google. Also, only 2.5% of Google’s
entire workforce and 4% of Facebook’s and
Microsoft’s are black.
Having different viewpoints can help humans
understand why a computer model adopted a
particular choice and represented it in a manner that
people can follow. Research also shows that more
gender and ethnically diverse companies are more likely
to experience higher performance and profit levels -
has partnered with the World Ethical Data
Foundation, a not-for-profit organization that
examines the opportunities and problems arising
from the development of new technologies.
“
World Ethical Data Foundation has
developed a specialized training
program for Shutterstock employees
to raise awareness of the societal
impacts of AI technology and to
give our technical employees the
tools and the knowledge to pursue
responsible AI.
see 5 Business Benefits of a Diverse Team.
Dr. Alessandra Sala, Shutterstock
W W W.SHUTTERSTOCK.COM
6
Computer Vision Applications
Healthcare
Industry 4.0
Computer vision is already playing an important role
When it comes to manufacturing, computer vision is a
in health-tech advances. For example, techniques like
key technology in what’s known as the fourth industrial
Mask-R Convolutional Neural Networks (Mask R-CNN)
revolution – or Industry 4.0. Indeed, it is already widely
can aid the detection of brain tumors, reducing the
used within industries to streamline and automate
possibility of human errors to a considerable extent.
certain processes.
It’s also widely used in various settings to assist medical
For example, using optical character recognition
professionals in making better decisions regarding the
(OCR), devices fitted with computer vision technology
treatment of patients. Medical imaging analysis creates a
such as robots or drones can examine labels on
visualization of particular organs and tissues to enable a
packaging and check them against an online database.
more accurate diagnosis, while deep-learning computer
This procedure helps to identify wrongly labeled
vision models can help with diagnostic tasks such as
products, provide information about expiration dates,
identifying moles from melanomas.
inform the manufacturer about product quantities, and
Research has also identified numerous advantages of
track packages at all stages of product development.
using computer vision and deep learning applications to
Publicité
Camera-based systems can also be used to collect
diagnose breast cancer. Trained with a vast database of
real-time data and leverage computer vision and
images consisting of both healthy and cancerous tissue,
machine learning algorithms to monitor defects in the
the technology can help automate the identification
manufacturing process, and analyze and benchmark the
process and reduce the chances of human error.
results against a predefined set of quality standards.
When it comes to elderly care, the technology will
According to figures from Omdia, the industrial
become widespread, especially in aging populations. For
machine vision market is set to grow by 5.7% from
example, Stanford Medicine has developed an
$5.295 billion in 2022 to $7 billion in 2025.
AI-assisted Care Solution for the remote monitoring,
assessment, and support of seniors living in their own
homes. Using multiple sensors for the detection and
recording of daily activities, it’s able to accurately assess if
an individual needs help – such as in the case of a fall.
Nor is it just on the factory floor where the
technology is being deployed. In agriculture, indoor
hydroponic vertical farming companies such as
Aerofarms, 80 Acres, and Alesca Life are using robots
and drones in conjunction with computer vision to
Using facial recognition technology, it’s possible to ensure
monitor millions of plants and identify potential growth
that a care plan is matched to the individual’s specific
issues, resulting in much less food waste and increasing
needs. It can also be used to increase the security of
crop yields. Ultimately, this form of urban agriculture
seniors living in their own homes by making sure that
could help pave the way to a carbon-negative form of
only authorized caregivers are allowed into the property
food production for cities across the globe.
(see paper here for more information).
W W W. AIBUSINESS.COM
7
In industries such as car manufacturing, computer vision
some people behave maliciously online when
is being used to generate 3D modeling designs and to
they wouldn’t do so in an ‘offline environment’
help guide robot and human workers. In construction,
when faced with the victim. As a result, content
AI-equipped drones can identify issues such as rust and
moderation is becoming increasingly important in
ensure repairs are made before it is too late. Similarly,
the modern world.
water companies are using 5G-enabled drones to
inspect for water leaks remotely before an engineer is
sent to investigate further.
Not only can AI save organizations time and money
in identifying large amounts of offensive content
very quickly, but the technology also reduces the
Content moderation
psychological impact that viewing this content could
Over the last decade, we have seen a proliferation of
User Generated Content (UGC), most of it uploaded
to social media channels. Statistics show that four
have on human moderators (though inevitably
humans will have to look at some of the content
which cannot be determined by machines).
million hours of content are uploaded to YouTube
Simple AI techniques used in content moderation
every day, while Instagram users upload over 100
include hash matching, in which the ‘digital fingerprint’
million photos and videos daily.
of an image is compared with harmful images stored
With so much content being uploaded all the time, it
is virtually impossible for humans to moderate all the
images and videos manually. Instead, computer vision
technology is needed to help identify a large amount
of content that could be in breach of the platform’s
in the organization’s database, as well as keyword
filtering, in which certain harmful words can be
flagged to remove that content. In addition, object
detection and scene understanding can also be used
to flag the harmful content.
content policies, or even the law – for example, videos
More complex is tackling the increasing use of
containing nudity or images depicting criminal acts.
Generative AI techniques, such as ‘generative
According to a Cambridge Consultants report,
Use of AI in Online Content Moderation,
commissioned by UK communications regulator
Ofcom, an ‘online disinhibition effect’ explains why
adversarial networks’ (GANs), where ‘deep fake’
images or videos are uploaded often from fake
profiles or ‘bots.
W W W.SHUTTERSTOCK.COM
8
Conclusion
Publicité
The technological world is changing rapidly. Thanks to
with local laws, examining plants to check they
the widespread adoption of smartphones, consumers
are free of disease or matching elderly faces with
have become used to accessing online content and
individual care packages.
e-commerce services simply by pressing various
onscreen options.
However, to ensure that the technology is truly
effective it first needs to be trusted - not only by
Meanwhile, as part of their digital transformation
the customers using it but by the general public.
strategies, businesses have been rapidly rolling out
At present this does not appear to be the case,
new technologies, particularly since the start of the
especially in the west, where only around 1 in 4 of
COVID-19 pandemic. Legacy systems are being
those interviewed trust the technology.
steadily overhauled and new ways of hybrid working
are being developed which take account of how we
want to live our lives.
So how do companies build this trust in computer
vision? One way is by developing an ethical
framework to ensure there is not any inherent bias
At the same time, new ‘frictionless’ online services
within the AI that can prejudice a particular group of
are being created that promise greater levels of
people based on factors such as their color, gender,
convenience than were previously possible.
or sexuality.
Inevitably, artificial intelligence (AI) is at the forefront
Legal safeguards also need to be put in place, such
of this transformational shift, helping to drive greater
as the Coordinated Plan on Artificial
efficiencies and increase productivity levels for
Intelligence 2021 which has been proposed by
businesses. Not only can intelligent machines often
the EU.
work more quickly than human beings can do, but they
are also able to do so with increasing levels of accuracy.
In many cases, people are now working alongside
AI-equipped devices too, whether that is computers,
robots, or even drones performing tasks that would
have once taken hours in a matter of minutes.
As with all technology, computer vision has the
potential to be used for harm as well as good.
However, by implementing a robust legal and ethical
framework that people can trust, we can help ensure
its use provides maximum benefits for organizations
and individuals alike.
As a branch of AI, computer vision can play a vital role
in automating many tasks - from moderating content
on social media platforms to ensuring it is compliant
Find out more about computer vision at
Shutterstock here.
W W W. AIBUSINESS.COM
9
History of computer vision
1959: Influential paper, Receptive fields of single neurons in the cat’s striate cortex, is published by
neurophysiologists David Hubel and Torsten Wiesel.
1963: Larry Roberts describes the process of deriving 3D information about solid objects from 2D photographs.
1966: Marvin Minksy instructs graduate students to connect a camera to a computer and describe what it sees.
1980: Kunihiko Fukushima builds the ‘neocognitron’, the precursor of modern Convolutional Neural
Networks (CNN).
1982: British neuroscientist David Marr publishes the influential paper, “Vision: A computational investigation
into the human representation and processing of visual information”.
1989: French scientist Yann LeCun releases LeNet- 5, the first modern convnet that introduces some of the
essential ingredients still used in CNNs today.
2001: The first face detection framework that works in real-time is introduced by Paul Viola and Michael Jones.
2003: Shutterstock is founded by programmer and photographer Jon Oringer
2006: Pascal VOC project is launched. It provides a standardized dataset for object classification as well as a
set of tools for accessing the said dataset and annotations.
2010: The release of ImageNet, a dataset containing millions of tagged images, helps to propel computer
vision’s rise.
2010: Google released Goggles, an image recognition app for searches based on pictures taken by mobile devices.
2012: AlexNet competes in the ImageNet Large Scale Visual Recognition Challenge. The network achieves a
top-5 error of 15.3%, 10.8 percentage points lower than that of the runner-up.
2012: Google Brain’s neural network recognizes pictures of cats using a deep learning algorithm.
Shutterstock.AI, a subsidiary of Shutterstock Inc., fuses ingenuity with
insights to power decision-making for creators globally.
Our proprietary creative intelligence platform and collection of over 400 million high-
quality photographs, vectors, illustrations, videos, 3D models and music enable solutions for
computer vision, predictive performance, content recommendations, and more.
Learn how Shutterstock is empowering brands to build faster and smarter computer vision
models every day. For more information please visit www.shutterstock.com.
W W W.SHUTTERSTOCK.COM
10