MEASURING DIGITALIZATION - KEY METRICS

Running enterprises

Fostering enablers

Smart Infrastructure

Internet Users

eGovernment

ICT Investments Cybercrime

Growth of the Internet Digital Technology

Web presence/activity

Mobility

Digital Spending (Assets/HR) Digital Asset Stock Digital transactions Work Digitization

Interaction between firms, customers and suppliers

Internal/External Business Processes

ENTERPRISE

Conversion funnel Digital Traffic Sources Opt In/Out

Cross-sell ratio

Campaign Performance

Social Media Performance Customer Engagement

Digital Revenues/Sales and Assets

CLIENT

(buyer/seller )

On-line Solution Usage

Digital Self -Service Ratio

Application World Performance

Performance of Client Generations an Microsegments

Evaluation of Online Experience

Figure 1. Digitalization dimensions and their primary metrics

(source: own work)

Measuring Digitalization  Key Metrics

125

The purpose of this article is to look at what digitali-

zation metrics are defined and used by various par-

ticipants of the economy and what lessons or

improvement actions can be proposed to enhance

the digitalization measurement systems. The study is

based on literature sources and on-site research with

selected European universal banks.

The following levels of digitalization with samples

of relevant metrics are covered (Fig. 1).

2

Digital Economy Metrics

It is commonly viewed that the term “digital econo-

my” was introduced by Don Tapscott in his publica-

tion: The digital economy: promise and peril in the

age of networked intelligence (Tapscott, 1997).

The term “digital economy” describes an economic

system where the usage of ICTs is widely spread,

embracing the:

1) base infrastructure (e.g., high-speed Internet ac-

cess, computing power, security services),

2) e-business (business models with high utilization

of ICT for front- and back-office functions), and

3) eCommerce (usage of the ICT in business-to-

business (B2B), business-to-consumer (B2C), and

consumer-to-consumer (C2C) transactions.

2.1 Digital Density Index (DDI)

Oxford Economics and Accenture developed jointly

a Digital Density Index (DDI) measuring how digital

technologies impact the economic growth (Macchi,

et al., 2015).

The ultimate aim of DDI is to guide further invest-

ments of both the public and business community in

order to stimulate economic development. The DDI

contains 50 indicators grouped into 4 activity areas

and 18 groups of metrics (Table 1).

Table 1. Measurement framework of the Digital Density Index

(source: Macchi, et al., 2015)

Activity area

Description and metrics

1. Making

markets

Increasing digitalization of existing markets and creation of new digital markets.

Recognition that existing markets are becoming increasingly digital and new markets

are being created through digital means.

Metrics:  Customer activity cycle



Interfirm collaboration

 Digitally contestable markets2

2. Sourcing

inputs

Use of digital technologies to source and/or use factors of production.

Degree to which digital technologies change the lifecycle of sourcing these factors for

the business.

Metrics:  Plant, property, equipment

 Finance (capital, liquidity)

 Labor

3. Running

Enterprises

Business use of digital technologies and activities to execute key business functions.

Metrics:  Technology process

 Strategy process

 Business model



Innovation

 Human capital/talent

 Research and development (R&D) spending

2 Contestable market is a market where new entrants face costs similar to those of established firms and where, on leaving, firms are

able to recoup their capital costs, less depreciation. Consequently, it is not possible for established firms to earn above normal profit as

this will be eroded by the entry of new firms, or, alternatively, the mere threat of such new entry may be sufficient to ensure that estab-

lished firms set prices that yield them only a normal profit return. Collins Dictionary of Economics, 4th ed. Pass C., B. Lowes, B., Da-

vies, L. 2005.

126

Marcin Kotarba

Table 1. Measurement framework of the Digital Density Index, cont.

(source: Macchi, et al., 2015)

Activity area

Description and metrics

4. Fostering

enablers

Changes in institutional and socioeconomic environments to facilitate digitalization.

Metrics:  Organizational flexibility

 Government spending

 Connectivity

 Ease of business

 Attitudes in society

 Long-term regulatory outlook

The metrics are used to compose an overall score

(0–100) for activity areas reflecting the digital pro-

file of a country’s economy in a benchmark view

with other economies (Fig. 2):

Figure 2. Sample Digital Density Score of selected economies  activity areas

(source: Macchi, et al., 2015)

A deep dive on the country level elaborates on the

country’s performance in each group of metrics. An

example of the “Fostering enablers  Ease of busi-

ness” metric shows maturity levels and country per-

formance (Fig. 3).

Figure 3. Digital Density Scores of selected economies: detailed metrics for the “Fostering enablers”

(source: Macchi, et al., 2015)

Measuring Digitalization  Key Metrics

127

2.2 Digital Economy and Society Index (DESI)

Within the framework of ”Europe 2020 Strategy”,

the European Commission introduced a performance

measurement system to track the evolution of the EU

member states in digital competitiveness (European

Commission, 2016).

The Digital Economy and Society Index (DESI) is

a composite index that contains five primary meas-

ure areas (Table 2).

Both the DDI and DESI reflect an attempt to capture

factors of high impact on the competitiveness

of local economies in the global (or internal EU)

context. The results of the benchmarking can be used

to study the best practices in use by the top perform-

ing digital economies, to be qualified for a potential

reuse in other geographies.

The DDI is strongly market/economy oriented, while

the DESI

includes a view on both economic

and social factors, such as human capital potential

and usage of ICT by the population. DESI provides

an important differentiation by analyzing the eGov-

ernment as a separate category and allowing

investment

the efficiency of public

to measure

in the digitalization.

Table 2. Digital Economy and Society Index  activities and metrics

(source: European Commission, 2016)

Activity area

Description and metrics

1. Connectivity

 The deployment of broadband infrastructure and its quality. Access to fast

broadband-enabled services is a necessary condition for competitiveness.

Metrics:

 1a Fixed Broadband

 1a1 Fixed BB Coverage

 1a2 Fixed BB Take-up (usage)

 1b Mobile Broadband

 1b1 Mobile BB Take-up

 1b2 Spectrum

 1c Speed

 1c1 NGA Coverage (>24Mb/s)

 1c2 Subscriptions to Fast BB

 1d Affordability

 1d1 Fixed BB Price

2. Human capital

Skills needed to take advantage of the possibilities offered by a digital society.

Such skills go from basic user skills that enable individuals to interact online

and consume digital goods and services to advanced skills that empower the work-

force to take advantage of technology for enhanced productivity and economic

growth.

Metrics:

 2a Basic Skills and Usage

 2a1 Internet Users

 2a2 Basic Digital Skills

 2b Advanced Skills and Development

 2b1 ICT Specialists

 2b2 STEM (science, technology, engineering, math) graduates

128

Marcin Kotarba

Table 2. Digital Economy and Society Index  activities and metrics, cont.

(source: European Commission, 2016)

Activity area

Description and metrics

3. Use of Internet

Variety of activities performed by citizens already online.

Such activities range from consumption of online content to modern communication

activities or online shopping and banking.

Metrics:  3a Content

 3a1 News

 3a2 Music, Videos and Games

 3a3 Video on Demand

 3b Communication

 3b1 Video Calls

 3b2 Social Networks

 3c Transactions

 3c1 Banking

 3c2 Shopping

4. Integration

Digitization of businesses and their exploitation of the online sales channel.

of information tech-

nology

By adopting digital technology, businesses can enhance efficiency, reduce costs,

and better engage customers, collaborators, and business partners.

Furthermore, the Internet as a sales outlet offers access to wider markets and poten-

tial for growth.

Metrics:  4a Business digitization

 4a1 Electronic Information Sharing

 4a2 RFID

 4a3 Social Media

 4a4 eInvoices

 4a5 Cloud

 4b eCommerce

 4b1 SMEs Selling Online

 4b2 eCommerce Turnover

 4b3 Selling Online Cross-border

5. Digital public

Digitization of public services, focusing on eGovernment.

services

Modernization and digitization of public services can lead to efficiency gains for the

public administration, citizens, and businesses alike as well as to the delivery of

better services for the citizen.

Metrics:  5a eGovernment

 5a1 eGovernment Users

 5a2 Pre-filled Forms

 5a3 Online Service Completion

 5a4 Open Data

Measuring Digitalization  Key Metrics

129

3

Digital Society Metrics

1) reviewing the current set of internationally com-

The digital society can be described as a society

in which the usage of ICT is common across demo-

graphic parameters of the population. Digital citizens

function in the digital economy using the available

digital public and commercial infrastructure for con-

ducting life activities.

The measurement of society digitalization is a part

of the DESI presented in the previous chapter.

A similar approach is reflected in the OECD’s digital

economy measurement system comprising “indica-

tors traditionally used to monitor the information

society” (Macchi, et al., 2015, p.3). In the context

of this paper, it is important to note that OECD aims

at (Macchi, et al., 2015):

parable ICT indicators,

2) exploiting the potential of existing official statis-

tics and experimenting with new metrics,

3) identification of data gaps, and

4) discussing

the data

infrastructure needed

to measure ICT diffusion and impacts, including

tools for analysis of large data sets.

The OECD digital metric system can be generally

mapped onto the DESI areas, providing additional

key performance indicators (KPIs) or more details

for the metrics already indicated in DESI. Proposed

mapping and important additional OECD metrics (vs

DESI) are shown in Table 3.

Table 3. Digital Economy and Society Index  activities and metrics

(source: European Commission, 2016)

OECD activity area  description and selected additional metrics versus DESI

1. Smart infrastructure (DESI: Connectivity):

 availability of fiber optics,

 average download speeds,

 connectivity costs (outside of the primary broadband connection): ICT devices (e.g., laptop, tablet, desk-

top), ICT applications, people using smartphones.

2.

Internet users (DESI: Human capital):

 number of activities performed online per person,

 digital natives: age of first Internet access, Internet usage by students at school and outside,

 usage of parental controls and Web filters,

 ICTs in schools: Internet connectivity, computer use for practicing, drilling, mathematics, and foreign lan-

guages,

 online course usage (e.g., word processor, spreadsheet, programming),

 computer use at work (simple, advanced, no use),

 people who believe that they have sufficient ICT skills to find new jobs.

3. The growth of the Internet (DESI: Use of Internet):

 online purchase analytics (e.g., travel, music, books, food, ticketing),

 usage per type of operation (e.g., e-mail, product information, news, social networks, eBanking, eGovern-

ment, eCommerce (buy and sell), gaming, audio, video, travel, accommodation, Web radio, Web TV, tele-

phone, software download, job search, medical services, content creation),

 average Wikipedia views (fixed and mobile connectivity),

 YouTube views of domestic content,



top Web sites by type (e.g., search engine, social network, media content, portal, reference/encyclopedia,

news, eCommerce, eBanking, blogging),

 search for health information,

 number of domestic Internet domains.

130

Marcin Kotarba

Table 3. Digital Economy and Society Index  activities and metrics, cont.

(source: European Commission, 2016)

OECD activity area  description and selected additional metrics versus DESI

4. Digital technology (DESI: Integration of the digital technology):`

trust in cross-border eCommerce,



 number of applications considered “standard” on the mobile phones,

 overall number of applications,

 usage of data analytics and big data, data-mining-related scientific articles,

 average data storage cost,



innovation (R&D activity and intensity, innovative enterprises in ICT manufacturing and services, en-

gagement in in-house ICT manufacturing),

 patents related to digitalization (ICT related, radicalness of patents, ICT industrial designs, and ICT trade-

marks),



labor productivity in ICT,

 cost of genome sequencing,

 security measures used for authorization and authentication (strong/weak, offsite backups, intrusion detec-

tion systems, tokens, biometrics),

 number of ICT security issues,

 acknowledging security issues  changes to browser settings,

 number of issues on client data protection.

5.

eGovernment services (DESI: Digital public services):

 problems in using eGovernment services (technical issues with portals, outdated information, no support

for digital solutions).

6. Other metrics (not explicitly specified in DESI):

 weight of the digital economy in the overall economy (measured by GDP, number and performance/size of

ICT companies, new entrants on the ICT market, global trade, gross exports),

 ICT investments (capital inflows, investment as a percentage of GDP, value added of ICT, venture capital

investments),

 digital IQ (graduates in ICT, researchers in ICT).

The DESI index is used by the EU members states

for the EU progress reporting and also as a base

framework for more detailed studies of digital devel-

opment on the country level. For example, in Poland,

the Ministry of Digital Affairs3 prepares a compre-

hensive report on the “information society” (Szy-

manek, 2015) by using DESI metrics as well as

additional KPIs that provide very important insights

(Table 4).

A study of DESI and OECD shows that both sets

of metrics are mutually supplementary: a number

of KPIs are shared, while other metrics are distinc-

tive per set. Also, there are differences in definitions

and attributes (analytical dimensions). Both DESI

and OECD cover several aspects of the digital econ-

omy and society, without clear borders between

these dimensions or their strict hierarchy. The digital

citizen is not evaluated separately, but rather as

a part of a larger category of “Internet users.”

3 Ministry of Digital Affairs  Ministerstwo Cyfryzacji i Admi-

nistracji, http://mc.gov.pl

Measuring Digitalization  Key Metrics

131

Table 4. Additional insights  information society metrics in the Polish economy

(source: Szymanek, 2015)

Group of additional insights in the Polish information society reporting:



internet users  new versus regular, Internet illiteracy, Internet skills versus computer skills, competencies

of “digital natives,”

 cybercrime protection,

 client considerations/fear while using digital solutions,



integration services offered by eGovernment (API, open platforms, reporting),

 running of useful eGovernment Web sites/portals (with traffic justifying the investment),

 enterprises with Web presence,



internet advertising versus Internet sales,

 purchasing versus Internet sales,

 ICT usage in public administration:

 promoting ICT usage in public administration,

 implementing ICT in public administration,

 internal process digitalization in public administration,

 external process digitalization in public administration,

 level of user centricity in public administration ICT,

 promotion of citizen and enterprise mobility.

4

Digital Industry Metrics

For the purpose of this paper, the term “digital indus-

try” is defined as the application of digitalization

in any type of industry. It is, therefore, not limited

to the ICT/new technology sector that produces digi-

tal solutions, but it covers all manufacturing or ser-

vice delivery where such digital solutions are used.

One of the most prominent examples of metrics that

are used to measure the digital state of industries is

proposed in the McKinsey Global Institute (MGI)

Industry Digitalization Index (Manyika, et al., 2015).

The index covers three groups of metrics:

1) assets,

2) usage, and

3) labor,

for which detailed KPIs are defined in Table 5.

Another example of a measurement system is the

Industry Digitalization Index (IDI), which is derived

from the data collected in the Eurostat databases

(Friedrich, et al., 2011). The IDI contains data on

company usage of the digital infrastructure, grouped

into four dimensions:

 digital input (procurement process digitalization),

 digital processing (internal and external process

integration, enterprise resource planning (ERP),

customer relationship management (CRM), inter-

nal data sharing, external electronic data ex-

change with business partners and the public

sector, upstream and downstream supply chain

management/SCM),

 digital output (sale process digitalization),



infrastructure (level of advancement of the ICT

function in the industry, Internet connectivity).

In comparison to the economy and society metrics,

the methods used to diagnose the industries empha-

size the process component, with an internal and

external view.

Process digitalization is measured in conjunction

with the ICT system infrastructure, especially with

the usage of established, integrated solutions such

as the ERP or CRM.

132

Marcin Kotarba

Table 5. Metrics included in the MGI Industry Digitalization Index

(source: Manyika, et al., 2015, p.30)

Group

Detailed metrics

Assets

 Digital spending:

 hardware spending (share of total expenditures spent on ICT hardware, e.g., computers

and servers),

 software spending (share of total expenditures spent on software, e.g., enterprise

resource planning (ERP) software),

 telecommunications spending (share of total expenditures spent on telecommunications,

e.g., broadband access and mobile data services),

 IT services spending (share of total expenditures spent on IT services, e.g., IT consulting

and IT architecture and implementation).

 Digital asset stock:

 hardware assets (share of total assets made up of ICT hardware, e.g., computers and

servers),

 software assets (share of total assets made up of software, e.g., purchased software

licenses),

 connected equipment (share of equipment embedded with digital connections, e.g., oil

rigs outfitted to transmit data on yield),

 data storage (data stored per firm, measured in terabytes, for firms with at least 1,000

employees).

 Transactions:

 digital transactions (share of payments and transfers, both from consumers to businesses

(C2B) and from businesses to other businesses (B2B) made through digital means, e.g.,

payments via ACH or wire).

 Interactions between firms, customers, and suppliers:

 digital external communications (composite score based on share of firms reporting

benefits from using social technologies to interface with customers and share of firms

reporting benefits from using social technologies to work with partners),

 digital customer service (composite score based on average number of customer service

chats per month and share of total contact center calls routed by automated systems, i.e.

integrated voice response (IVR) or automated speech recognition (ASR) technology).

 Business processes conducted internally:

 digitized back-office processes (composite score based on the adoption of enterprise re-

source planning (ERP) software (e.g., SAP, Oracle) across the industry, and share

of firms reporting that technology is very integrated into employees’ daily activities),

 digitized front-office processes (composite score based on the adoption of customer re-

lationship management (CRM) software across the industry and digital marketing (e.g.,

e-mail, banner, and search engine advertisements) expenditures, as an estimated share

of total marketing expenditures),

 product development software intensity (intensity of software usage in product devel-

opment process, e.g., for computer-assisted design).

Usage

Measuring Digitalization  Key Metrics

133

Table 5. Metrics included in the MGI Industry Digitalization Index, cont.

(source: Manyika, et al., 2015, p.30)

Group

Detailed metrics

Usage

 Market making:

 digitally enabled markets  extent to which digital platforms are being used to connect

supply with demand, calibrated using the relative size of digital bid-ask or auction-based

markets (in terms of users, transactions, and/or revenues).

Labor

 Digital spending:

 hardware spending on workers’ ICT hardware, for example, computers and server

expenditures per full-time-equivalent employee (FTE),

 software spending per worker, for example, enterprise software license expenditures

per FTE,

 telecommunications spending per worker  telecommunications (e.g., broadband access,

mobile data service) expenditures per FTE,

 IT services spending per worker  IT services (e.g., IT consulting and IT architecture

and implementation) expenditures per FTE.

 Digital capital deepening:

 hardware assets per worker  ICT hardware assets (e.g., servers and computers)

per FTE,

 software assets per worker  software assets (e.g., worker software licenses) per FTE.

 Digitization of work:

 share of tasks that are digital (time-weighted share of worker tasks involving digital

tools or processes, e.g., tasks requiring workers to input information via tablet, conduct

online research, or perform analyses with spreadsheet software). Based on a search

for digital keywords (e.g., data, computer, software) in a publicly available database

of worker tasks,

 share of jobs that are digital (digital jobs, e.g., computer and information systems man-

agers, hardware engineers, telecommunications equipment installers and repairers as

a share of total jobs).

5

Digital Enterprise Metrics

The level of a single enterprise digitalization can be

measured with industry metrics presented in the pre-

vious section. However, there is a large additional

measurement area that is not covered explicitly

by the IDI. Additional KPIs describe the status and

performance of eCommerce and digital customer

dialog in an enterprise. Core KPIs of this type in-

clude:

 conversion funnel (users/visitors  leads  pro-

spects  clients/wins) and cost (cost per lead/

prospect/client),



traffic sources (organic, paid search/affiliate net-

works/referrals, direct, e-mail, social media),

 opt-in/out level and dynamics,

 email/SMS performance(bounce/delivery

rate,

sharing, open rate, click-through-rate (CTR), cost

per lead (CPL), lead quality (LQ), and name

to marketing qualified lead (MQL) conversion),

 public and eCommerce Web sites (bounce/drop-

off rate, page views per visit/total, ad cap-

ture/impressions, CTR,

cost-per-impression

(CPI), cost-per-click (CPC), new sessions, time

on (sub)-site, CPL, LQ),

134

Marcin Kotarba

 content quality (white-book downloads, newslet-

ter/info-service sign-ups),

 subscription service performance (free trial to

subscription rate),

 social media volume and performance (sharing

and tagging, Facebook likes and lead conversion,

Twitter followers, promoted tweet cost/success,

YouTube subscribers, LinkedIn followers, Insta-

gram followers, Google+ circled-by, Pinterest

pinning),

 media performance (social, digital and tradition-

al) number and quality of publications,

 customer engagement (Client Satisfaction Index,

mystery shopper results, Net Promoter Score

(NPS), churn and retention levels),

 campaign and digital channel return on invest-

ment (ROI) and revenue share in total,

 client cross-sell ratio,

 sales and revenue per digital client and revenue

share in total.

The presence of the above enterprise metrics in the

economy/society and industry measurement systems

is limited, despite their purely digital nature.

6

Digital Client Metrics

In the course of research on the topic of enterprise

digitalization metrics, the author collected infor-

mation on the practical application of digital KPIs

in sample financial institutions.

Banks are at the forefront of digital transformation,

engaging in highly competitive struggle internally

within the financial sector and externally with

FinTech/venture capital corporations as well as with

non-financial players making more bold attempts at

capturing financial services’ market share (e.g.,

Bitcoin/Blockchain, Apple Pay, Google Wallet).

From this point of view, digitalization is a key to the

survival and growth of current and future business

models on the banking market.

A case study, sourced via interviews with Chief Dig-

ital Officers and collection of KPIs, was conducted

on five universal banks operating in the European

Union (EU). The goal was to understand what KPIs

and dashboards were used for regular tracking of the

progress of digitalization.

In comparison to all the measurement systems pre-

sented already in the paper, the following conclu-

sions can be drawn from practices in the sample

banks:

 a number of base metrics are in line with the

economy/society/industry/enterprise views, for

example, digital solution users (gross and net),

Web site activity tracking, and social media per-

formance,



there are several groups of KPIs specific to the

banking sector, for example,

 usage of online solutions:

-

logins per day/month (time of day analysis,

session duration),

• number/volume of transactions per ses-

sion/in time series,

• customer login journey (origination pat-

terns),

• most frequently used functionalities,
• hardware technological platform used, with

change history (iOS, Android, Windows),

• software platform used (system, browser

type),

• origination IP for security tracking purpos-

es,

 product sales(volumes), revenues, and profita-

bility of digital clients,

 self-service ratio via digital solutions in sales

and after sales per product and channel,

 business activity generated via the mobile

and stationary channels,

 app-world performance (downloads of appli-

cations per platform, rankings/stars and feed-

back, application updates,

 share of contact campaign records processed

via digital channels,

 banks are highly detailed with respect to digital

performance in client segments via:

 observance of client generations (e.g., Baby

Boomers, X, Y, Z),

 microsegmentation into profiles (e.g., via digi-

tal activity preferences),

Measuring Digitalization  Key Metrics

135

 tracking of con...