Machine Learning Across Cultures: Modeling the Adoption of Financial Services for the Poor

06/16/2016
by   Muhammad Raza Khan, et al.
0

Recently, mobile operators in many developing economies have launched "Mobile Money" platforms that deliver basic financial services over the mobile phone network. While many believe that these services can improve the lives of the poor, a consistent difficulty has been identifying individuals most likely to benefit from access to the new technology. Here, we combine terabyte-scale data from three different mobile phone operators from Ghana, Pakistan, and Zambia, to better understand the behavioral determinants of mobile money adoption. Our supervised learning models provide insight into the best predictors of adoption in three very distinct cultures. We find that models fit on one population fail to generalize to another, and in general are highly context-dependent. These findings highlight the need for a nuanced approach to understanding the role and potential of financial services for the poor.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 3

page 4

12/08/2018

Mobile Money: Understanding and Predicting its Adoption and Use in a Developing Economy

Access to financial institutions is difficult in developing economies an...
11/13/2017

Determinants of Mobile Money Adoption in Pakistan

In this work, we analyze the problem of adoption of mobile money in Paki...
12/09/2017

Behavior Revealed in Mobile Phone Usage Predicts Loan Repayment

Many households in developing countries lack formal financial histories,...
03/13/2018

Understanding Interface Design and Mobile Money Perceptions in Latin America

Mobile money can facilitate financial inclusion in developing countries,...
06/12/2020

Predicting cell phone adoption metrics using satellite imagery

Approximately half of the global population does not have access to the ...
06/02/2016

Mobile phone data for public health: towards data-sharing solutions that protect individual privacy and national security

We outline the constraints faced by operators when deciding to share de-...
04/16/2021

An Analysis of Impact Pathways arising from a Mobile-based Community Media Platform in Rural India

Our research presents the case-study of a mobile phone based, voice-driv...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Billions of people around the world live without access to formal financial institutions. Over the last several years, mobile phone operators across the globe have launched “Mobile Money” platforms, which make it possible for many of the world’s poor to conduct basic financial transactions from inexpensive feature phones. However, outside of Kenya and a few other countries, worldwide adoption of Mobile Money has been extremely anemic. The vast majority of deployments have struggled to promote sustained product adoption, and an industry report from 2014 estimates that 66% of registered customers were inactive

(Scharwatt et al., 2014). An open and important question thus revolves around understanding what drives customers to adopt and use Mobile Money, and whether patterns observed in one country will generalize to another.

Our work builds on several distinct strands in the academic literature. The first strand that is concerned with understanding the determinants of mobile money adoption has historically been the domain of development researchers, and includes both macroeconomic studies (Mas & Radcliffe, 2011; Dermish et al., 2011; Donovan, 2012) as well as qualitative, ethnographic work (Morawczynski, 2009; Medhi et al., 2009; Etim, 2014). A second strand of literature seeks to derive general insights from mobile phone transactions logs. This encompasses a wide array of applications, including predicting the socioeconomic status (Blumenstock et al., 2015), gender (Frias-Martinez et al., 2010), and age (Dong et al., 2014) of individual mobile phone users.

Most relevant to this study, a third area of prior work mines digital transactions logs to model the social and behavioral determinants of product adoption (Ugander et al., 2012; Leskovec et al., 2007), with a few recent papers written in the context of mobile phone data (Khan et al., 2015; Sundsøy et al., 2014; CGAP, 2013). Relative to these studies, our study moves this literature forward by (a) innovating in the method used to generate features, thereby providing a systematic and comprehensive approach to feature engineering; (b) leveraging data from three different contexts to calibrate the external validity and generalizability of our results; and (c) carefully articulating the experiments in a way that will enable other researchers to replicate and extend these methods.

2 Data and Context

The data that we have used in this project consists of anonymized Call Detail Records (CDR) and Mobile Money Transaction Records (MMTR) of all the subscribers from three different operators in Ghana, Pakistan, and Zambia. All three countries rank in the bottom third of the Human Development Index and Financial Inclusion Index as shown in Table 1. The CDR and MMTR contain basic metadata on every event that occurs on the mobile phone network, including phone calls, text messages, and any form of Mobile Money activity. In total, the original data contains billions of transactions conducted by tens of millions of unique individuals. Each dataset spans several months of activity, which we divide into a “training” period and an “evaluation” period. CDR from a 10-day training period was used to engineer features and fit a predictive model, where the target variables (based on Mobile Money activity) were measured in a subsequent 3-month evaluation period. Based on activity during the evaluation period, each subscribers is categorized as a “Voice Only User” (no MM activity), a “Registered Mobile Money User” (one or more MMTR’s), or an “Active Mobile Money User” (at least one MMTR in each month).

Country Ghana Pakistan Zambia
Panel A: National statistics (Source: World Bank)
Population 25.90 Million 185.00 Million 15.72 Million
GDP PPP adjusted $4081.70 $4811.4 $3904.00
Mobile accounts per 100 users 115 73 67
Adults without financial services 59.49% 54.36% 86.97%
Panel B: Mobile phone use (Source: Call Detail Records)
Calls per user per day 6.53 (6.99) 7.76 (10.25) 10.26 (102.86)
SMS per user per day 3.10 (100.36) 38.71 (80.83) 10.88 (262.81)
No. of unique contacts 21.66 (24.91) 46.93 (139.67) 17.63 (328.63)
No. of unique towers 12.98 (16.07) 24.15 (57.30) 7.35 (17.56)
Table 1: Summary statistics by country

3 Feature Engineering

Figure 1: Deterministic Finite Automaton
Figure 2: Tree-based feature classification

The most common approach to constructing interpretable metrics (“features”) from the phone data is to hand-craft a small number of features that correspond to some intuition of the researcher. For instance, (Dong et al., 2014) focus on 5 topological properties of the static social network. In contrast, we employ a deterministic finite automaton (DFA), to formalize the feature generation process (McCulloch & Pitts, 1943; Blumenstock et al., 2015). We use the DFA to specify a set of legal operations that can be recursively applied to raw transactional data in order to produce valid features. The DFA we use to generate features from CDR is shown in Figure 2 (see also Algorithm 1). To interpret the set of features produced by the DFA, we map each feature onto the tree structure as shown in Figure 2, where each branch of the tree partitions the data into features that capture different behavioral characteristics of the subscriber.

Data: cdr (Call Detail Records of all users)
Data: opmap (Dictionary of possible operations)
Result: Features
Step 1: Perform reduce by grouping only on ego foreach , in , do
       .filter(,)
       foreach in do
             .map([] +[combinations(]))
             foreach in [] do
                   .reduce() insert in
                   .map(, ).reduce() insert in
                  
             end foreach
            
       end foreach
      
end foreach
Algorithm 1 Feature Generation Algorithm

4 Results

Using the features generated by DFA 3 our next steps were to (a) build a predictive model that can be used to identify likely adopters, (b) use these features to understand the determinants of Mobile Money use, and (c) determine the extent to which models and features from one context can generalize to another.

4.1 Predicting Mobile Money Use

We use a variety of supervised learning algorithms to attempt two classification tasks: Voice Only vs. Registered User, and Voice Only vs. Active User. For this task, we drew a stratified random sample of 10,000 subscribers from each of the three user types from each of the three countries. Here, we focus on the results from gradient boosting

(Friedman, 2001)

, which marginally outperformed other common classifiers including logistic regression, svm, and so forth. As shown in the Figure

3, the gradient boosting with DFA based features significantly outperforms the naive baseline, and we achieve marginally better success in identifying Active Mobile Money users than Registered Mobile Money users in all of the countries. We will return to the interpretation of these results in Section 5.

Figure 3: Model Accuracy. Left figure shows accuracy in identifying Mobile Money users within each country; Right figure shows Accuracy when model is trained in one country and evaluated in another.

4.2 Determinants of Mobile Money Adoption

To investigate the prominant denominators of Mobile Money adoption, we calculate the AUC

from a bivariate logistic regression of the response variable on each individual feature. We also calculate the

normalized feature importance (NFI) as the relative significance of each feature in the final gradient boosting classifier operating on all of the features.

The distribution of unconditional AUC values for all the features generated through the methods explained in section 3 is shown in Figure 4 (left panel), using Ghana as a test case. Each violin plot shows the distribution of AUC values for all features of a given type - such as all “ego” features, or all “movement” features. While a large number of features have AUC values near 0.5, there are a small number of highly predictive features with AUC0.75. The right panel of Figure 4 shows the distribution of AUC values for the subset of features related to the subscriber’s first degree network (Actor=“Voice Alters” and Type=“All”), a subset that is generally more predictive of Mobile Money use in Ghana. Here, the range of AUC values is significantly higher than in the full set of features, and some sub-classes such as “Network”, which capture the structure of the subscriber’s first degree network, have uniformly high predictive power.

Figure 4: Distribution of AUC values for each feature category. Left figure shows all features in Ghana; Right figure shows the subset of features in Ghana where Actor=‘Voice Alters’ and Type=‘All’

Figure 5 shows the NFI distribution obtained through gradient boosting model operating on all of the features. Just like unconditional ranking, each class of features in this case also contains a mass of features with low predictive power, but perhaps most striking in Figure 5 are the differences between countries in the relative importance of each class of features.

Figure 5: Normalized feature importance

4.3 Learning Across Cultures?

Our standardized methods and analysis performed across all three countries. This makes it possible to answer a question that has been elusive in prior studies of the adoption of new technologies in developing countries: Do the behavioral determinants of adoption identified in one context generalize to another? Based on the analysis we have performed, our short answer to this question appears to be, “No.”

The right panel of Figure 3 shows the performance of a classifier trained in one country and evaluated in another. The first set of six bars shows that the classifiers trained in Ghana perform well in Ghana (the first two grey bars), replicating the results in Figure 3

. However, that same Ghana model does quite poorly when evaluated in Zambia (the next two blue bars) or Pakistan (the final two green bars). While it is almost certain that a more sophisticated approach to transductive transfer learning would perform better

(Pan et al., 2011), the naive application of a model out of context is quite ineffective.

5 Discussion and Conclusions

Taken in the broader context of research into the determinants of Mobile Money adoption, the preceding results uncover several unexpected patterns.It is not surprising that classifiers using CDR-data can predict Mobile Money use. However, we were surprised to find that the supervised model was only marginally better able to identify Active Mobile Money users than Registered Mobile Money users. Since true financial inclusion requires active use, this remains an important topic for future work.

Also interesting are the differences in performance of the same modeling approach applied in different contexts. Most striking here is the relatively poor performance in Pakistan, where the 18% improvement over the baseline is dwarfed by the 55% improvement over the baseline achieved in Ghana. We believe this may also in part be an artifact of the “one size fits all” approach we have taken to standardizing definitions and methods across countries. In particular, there is one type of Mobile Money transaction that is extremely common in Pakistan, which allows a subscriber to add prepaid phone credit to her phone account using Mobile Money. Anecdotally, it is common practice in Pakistan for the retailers of phone credit to perform this Mobile Money transaction on behalf of the subscriber. This potential source of bias highlights the brittle nature of the cross-country analysis, which in its current form does not allow for country-specific adaptation.

Most importantly, our results suggest that across different countries in the developing world, no single set of behavioral features is likely to consistently predict Mobile Money adoption and use. For policymakers interested in promoting the growth of mobile-based financial services, this indicates that the details of the local context will be critical in achieving sustained adoption and use of financial services for the poor.

References

  • Blumenstock et al. (2015) Blumenstock, Joshua, Cadamuro, Gabriel, and On, Robert. Predicting poverty and wealth from mobile phone metadata. Science, 350(6264):1073–1076, 2015.
  • CGAP (2013) CGAP. The Power of Social Networks to Drive Mobile Money Adoption. Technical report, 2013.
  • Dermish et al. (2011) Dermish, Ahmed, Kneiding, Christoph, Leishman, Paul, and Mas, Ignacio. Branchless and mobile banking solutions for the poor: A survey of the literature. Innovations: Technology, Governance, Globalization, 6(4):81–98, October 2011. ISSN 1558-2477. doi: 10.1162/INOV˙a˙00103. URL http://dx.doi.org/10.1162/INOV_a_00103.
  • Dong et al. (2014) Dong, Yuxiao, Yang, Yang, Tang, Jie, Yang, Yang, and Chawla, Nitesh V. Inferring user demographics and social strategies in mobile social networks. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 15–24. ACM, 2014.
  • Donovan (2012) Donovan, Kevin. Mobile money for financial inclusion. Information and Communications for Development, pp. 61–73. The World Bank, 2012. ISBN 978-0-8213-8991-1. URL http://elibrary.worldbank.org/doi/abs/10.1596/9780821389911_ch04.
  • Etim (2014) Etim, Alice S. Mobile banking and mobile money adoption for financial inclusion. Research in Business and Economics Journal, 9:1, 2014.
  • Frias-Martinez et al. (2010) Frias-Martinez, Vanessa, Frias-Martinez, Enrique, and Oliver, Nuria. A gender-centric analysis of calling behavior in a developing economy using call detail records. 2010.
  • Friedman (2001) Friedman, Jerome H. Greedy function approximation: a gradient boosting machine. Annals of statistics, pp. 1189–1232, 2001.
  • Khan et al. (2015) Khan, Muhammad Raza, Manoj, Joshua, Singh, Anikate, and Blumenstock, Joshua. Behavioral modeling for churn prediction: Early indicators and accurate predictors of custom defection and loyalty. In 2015 IEEE International Congress on BigData, pp. 677–680. IEEE, 2015.
  • Leskovec et al. (2007) Leskovec, Jure, Adamic, Lada A, and Huberman, Bernardo A. The dynamics of viral marketing. ACM Transactions on the Web (TWEB), 1(1):5, 2007.
  • Mas & Radcliffe (2011) Mas, Ignacio and Radcliffe, Dan. Scaling mobile money. Journal of Payments Strategy & Systems, 5(3):298–315, 2011. URL http://henrystewart.metapress.com/index/2771530377482470.pdf.
  • McCulloch & Pitts (1943) McCulloch, Warren S. and Pitts, Walter. A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical biophysics, 5(4):115–133, December 1943. ISSN 0007-4985, 1522-9602. doi: 10.1007/BF02478259. URL http://link.springer.com/article/10.1007/BF02478259.
  • Medhi et al. (2009) Medhi, Indrani, Ratan, Aishwarya, and Toyama, Kentaro. Mobile-banking adoption and usage by low-literate, low-income users in the developing world. In Internationalization, Design and Global Development, pp. 485–494. Springer, 2009.
  • Morawczynski (2009) Morawczynski, Olga. Exploring the usage and impact of “transformational” mobile financial services: the case of m-PESA in kenya. Journal of Eastern African Studies, 3(3):509–525, October 2009. ISSN 1753-1055. doi: 10.1080/17531050903273768.
  • Pan et al. (2011) Pan, Sinno Jialin, Tsang, I.W., Kwok, J.T., and Yang, Qiang. Domain Adaptation via Transfer Component Analysis.

    IEEE Transactions on Neural Networks

    , 22(2):199–210, February 2011.
    ISSN 1045-9227. doi: 10.1109/TNN.2010.2091281.
  • Scharwatt et al. (2014) Scharwatt, Claire, Katakam, Arunjay, Frydrych, Jennifer, Murphy, Alix, and Naghavi, Nika. State of the Industry: Mobile Financial Services for the Unbanked. Technical report, GSMA, 2014. URL http://www.gsmamobileeconomy.com/GSMA_ME_Report_2014_R2_WEB.pdf.
  • Sundsøy et al. (2014) Sundsøy, Pål, Bjelland, Johannes, Iqbal, Asif M, de Montjoye, Yves-Alexandre, et al. Big data-driven marketing: How machine learning outperforms marketers’ gut-feeling. In Social Computing, Behavioral-Cultural Modeling and Prediction, pp. 367–374. Springer, 2014.
  • Ugander et al. (2012) Ugander, Johan, Backstrom, Lars, Marlow, Cameron, and Kleinberg, Jon. Structural diversity in social contagion. Proceedings of the National Academy of Sciences, 109(16):5962–5966, 2012.