Universities and other academic institutions increasingly see their presence, visibility and footprint on the Web as central to their reputation and international standing. In this context, the academic web is evolving into more than a vehicle for communicating scientific and cultural achievements; information content is viewed as a reflection of the overall organization and performance of the university (Aguillo et al., 2008). Academic rankings, therefore, play an important role in assessing reputation. With different criteria and disparate methodologies, there can be a significant divergence in the rankings of a particular institution depending upon the list that is surveyed.
Academic excellence is difficult to quantify, yet most ranking organizations start by collecting performance indicators (e.g., Nobel laureates, research volume) about each university which they believe to be independent indicators of quality. After giving each a different, predetermined weight, the indicators are summed to a total score that determines the university's rank. The weighted scoring method is sometimes supplemented with a peer institution survey which is compiled and submitted by academic experts (Enserink, 2007). We propose an alternative metric for ranking universities, University Twitter Engagement (UTE), a score which is the sum of all affiliated users the university promotes on its homepage plus the followers of any Twitter friends who indicate an affiliation with the university in their profile Uniform Resource Identifier (URI). The UTE score is an important metric as it quantifies the potential popularity or prestige of the university without an extensive data collection effort.
This research assumes that (1) universities with higher undergraduate enrollment are likely to have more Twitter followers as students graduate and transition to alumni status, (2) official Twitter accounts will be well advertised on the university's homepage, (3) sports participation is a driver that increases awareness of the university's brand, and (4) the data needed to comprise the ranking criteria is readily available and easy to collect from public data sources on the web. Figure 1 depicts a recent glimpse into the Twitter followers (675K) for Harvard University, a perennially top-ranked school, which represents an approximate 100:1 ratio to its undergraduate enrollment (6,660). On the other hand, the Twitter follower count (1,213) for Virginia Military Institute (VMI), a top 100 school, barely maintains a 1:1 ratio with its undergraduate enrollment (1,717). If we only consider alumni, we would expect that schools with similar enrollment would attract a similar number of Twitter followers. The large disparity between Harvard and VMI presents a first indication that some correlation may exist between rank position and Twitter followers. We propose a novel approach which considers not only the primary Twitter accounts which the university may advertise on its homepage, but secondary accounts which the university informally promotes by following them on Twitter. In order to ensure that a relationship or mutual affiliation exists between the primary and secondary accounts, we enforce the requirement that the top level domain assigned to the university in its URI (e.g., harvard.edu) must be present in the Twitter profile of all affiliated Twitter accounts.
The contributions of this study are as follows:
We aggregate the rankings from multiple expert sources to calculate an adjusted reputation rank (ARR) for each university which allows direct comparison based on position in the list and provides a collective perspective of the individual rankings.
We conduct a web-based analysis to identify and collect a mutually aligned, comprehensive set of primary and secondary Twitter accounts as a measure of social media engagement.
We propose an easily collected proxy measurement, UTE, that achieves comparable rankings as more complex methodologies which rely upon manual compilation.
We produce a social media rich dataset containing Twitter profile data and institutional demographics which will reduce the effort required by other researchers to reproduce our work (Weller and Kinder-Kurlanda, 2016). The complete dataset is posted on GitHub111https://github.com/oduwsdl/University-Twitter-Engagement.
2. Related Work
The relevance of Twitter followers as a means of measuring reputation has been the subject of many previous studies. Our work parallels the studies conducted by Klein et al. (Klein et al., 2009a, b) and Nelson et al. (Nelson et al., 2008) who attempt to find correlations between the rankings of real-world entities (e.g., college football teams, Billboard Hot 100, graduate business schools) and the page rank of their respective home pages. In this paper, we examine something similar, but instead derive the ranking score using social media.
2.1. The Challenge of Ranking Universities
University rankings are subject to normative assumptions about the type of variables used and their associated weightings. Therefore, ranking systems reflect the conceptual framework and the modeling choices used to build them (Goglio, 2016). These systems can potentially give inaccurate indications to university administrators about the activities in which it is better to invest in order to improve the ranking of their institution (Goglio, 2016). And, as predicted by decision-making theory, Bowman and Bastelo (Bowman and Bastedo, 2011) found that anchoring effects exert a substantial influence on future reputational assessments. Once a university reaches the pinnacle of any ranking system, they are anchored and often do not fall very far from their original position. Bowman and Bastelo (Bowman and Bastedo, 2011) observed that academics across the world are influenced in some way by external assessments of their ranking. Further, they concluded it would take an extensive change in academic quality to significantly influence reputation scores in any given year. Nearly always, rankings drive reputation, not the other way around. The notion of reputation largely serves as a feedback loop to maintain the status quo, establishing the credibility of the rankings and ensuring stability in results over time (Bowman and Bastedo, 2011).
Different metrics used by the ranking organizations can make direct comparisons difficult as each list may be intended to convey a distinct purpose. Three of the four ranking systems referenced in this paper determine best colleges based on academic excellence while the fourth, Money Magazine, is focused solely on perceived value and affordability. A particular ranking list may count factors such as external funding, numbers of articles and books authored by faculty members, library resources, proportion of faculty members with advanced degrees, and quality of students based on admissions criteria. With so many heterogeneous metrics, conducting surveys can be time consuming and expensive if the data must be gathered over a long period of time or requires manual input from a university official. These numbers are not easy to obtain and are assumed to be an adequate proxy for quality.
The assumption by the ranking systems is that one set of metrics can be applied to every institution and that the norms of research-based and elite universities are the gold standard that can be applied to everyone (Altbach, 2015). Goglio (Goglio, 2016) showed that the competition to improve ranks among lower ranked universities is different from the competition to do so among higher ranked universities. The rank-localized nature of competition is primarily among those universities that are similarly ranked. Grewal et al.'s (Grewal et al., 2012)
results also showed that a top-ranked university has a 0.965 probability of finishing in the top five the next year. Ultimately, regardless of popularity, universities exhibit very little power to control their rank position and, although almost all aspire to be among the upper echelons, the top positions are perennially dominated by the same institutions(Goglio, 2016).
2.2. Social Media in Higher Education
Even when the ranking systems have the same goal, technical challenges can still hamper data collection; specifically, changes in page names or web domains can affect both the visibility and discoverability of the institution's web presence. An organization can also use different web domains for search engines, aliases and independent domains for some of their subunits or services (Aguillo et al., 2008). For example, in addition to odu.edu which is the expected domain for Old Dominion University, we found odu.trisigma.org and oduwsoccerclub.wixsite.com as domains associated with university-sponsored clubs. As noted by Aguillo (Aguillo et al., 2008), an adequate web presence or lack thereof may not always correlate with the quality or prestige of the institution.
Social networking sites have proven to be an effective vehicle for organizations seeking to implement diverse branding strategies, given that such sites allow consumers to share their experiences and opinions concerning the organization's products and brand in real time (Heller Baird and Parasnis, 2011; Jansen et al., 2009). Many organizations have rapidly adopted social networking services such as Facebook and Twitter, a move that has altered the face of customer relationship management from managing customers to collaborating with customers. While social media interactions in the higher education space are not transactional in the traditional sense, they do provide a way for institutions to continually engage with their constituents. Another form of engagement, or public involvement with a chosen organization that may fall outside of consumer interests is affective commitment which Kang (Kang, 2014) defines as a voluntary bonding between entities; perhaps similar to how a university might maintain contact with its alumni long after graduation. We will focus on engagement at a very basic or minimal level based on familiarity and cognition where one first needs to be familiar with a university's online activity and subsequently start to follow them via social media.
As part of their ongoing research to measure the impact and social media usage in the United States, a 2016 study conducted by the Pew Research Center concluded that while Facebook continues to be the U.S.'s most popular social networking site with nearly 79% of online users using the platform, Twitter usage is holding steady at 24% and is also somewhat more popular among the highly educated (Greenwood et al., 2016). Go et al.'s (Go and You, 2016) 2016 social media benchmarking report also suggests that Twitter is perceived as the most useful application for businesses. At the organizational level Tsimonis et al. (Tsimonis and Dimitriadis, 2014) examined the policies, strategies and outcomes that companies might expect when engaging on social media. One observed outcome related to increased brand awareness theorized that it is possible to use a well-designed webpage to spark additional interest. Further, research findings attest to the value of social media engagement in building communities and nurturing positive public attitudes regarding the reputation of the organization (Men and Tsai, 2015). Through data collected via a large scale survey Dikjmans et al. (Dijkmans et al., 2015) also found that engagement in social media activities is positively related to corporate reputation.
2.3. Influence of Twitter Followers
Measuring influence and social networking potential on Twitter has been discussed in various papers as well as in numerous blogs and online media. Related scientific work on Twitter includes approaches which measure influence by not only taking followers and interactions into account, but also by analyzing topical similarities with the help of a ranking method similar to PageRank (Weng et al., 2010). Other approaches define different types of influence on Twitter, namely indegree, retweet and mention influence (Cha et al., 2010). Accordingly, a question that arises concerns how to determine the Twitter accounts that are most influential and how their influence is subsequently measured (Antoniadis et al., 2016). Measuring Twitter followers is generally considered to be a popular metric as having many followers can indicate a higher level of influence as more people seem to be interested in the user. This metric implies that the more followers a user has, the more impact the user has, as the user seems to be more popular (Leavitt et al., 2009). Preussler (Preussler and Kerres, 2010) contends that the number of followers is an indicator for the social reputation and the number of followers will increase as the user becomes more important. Finally, Kunegis et al. (Kunegis et al., 2013) assert that preferential attachment indicates that people who already have many ties are more likely to receive new ties. In other words, people who are followed by many people (i.e., are popular) are more likely to receive new followers.
An alternative approach for ranking Twitter users undertaken by Saito and Masuda (Saito and Masuda, 2013)
considers the number of others that a user follows, i.e. friends. They concluded that the number of others that a user follows is equally important as the number of followers when estimating the importance of a Twitter user. In previous studies on Twitter, a variety of characteristics, both personal and social, have been used to identify influencers and each study measures influence from different perspectives(Leavitt et al., 2009; Black, 1993; Kwak et al., 2010; Weng et al., 2010). Weng introduced the concept of homophily which implies that a Twitterer follows a friend because she is interested in some topics the friend is publishing, and the friend follows back because she finds they share a similar topical interest. The presence of homophily implies there are Twitter users who are highly selective when choosing friends to follow (Weng et al., 2010). These conclusions are evidenced by super users who are followed by many other users, but do not follow back equally as they only follow a select group of Twitter friends or other super users (e.g., consider the friend-to-follower ratio of Harvard shown in Figure 1).
The following section discusses how we chose the performance indicators to correspond with the entries in the expert lists, the ranking algorithm and other operational details.
3.1. Establishing the Selection Criteria
To select the universities of interest, we begin with the 351 American colleges and universities currently classified as Division I by the National Collegiate Athletic Association222http://www.ncaa.org/about?division=d1 (NCAA). We then consider which of these institutions appear among the rankings of the Academic Rankings of World Universities333http://www.shanghairanking.com/ (ARWU) 2016, the Times Higher Education444https://www.timeshighereducation.com/world-university-rankings/2016/world-ranking (THE) World University rankings 2015-2016, Money's Best Colleges555http://new.time.com/money/best-colleges/rankings/best-colleges/ (MONEY) 2016-2017, and U.S. News (USNEWS) Best Global Universities666http://www.usnews.com/education/best-global-universities 2015 and 2016.
|US News 2015||500||99||0|
|US News 2016||750||137||3|
|Any Two Lists||–||–||22|
|Any Three Lists||–||–||19|
|Any Four Lists||–||–||16|
|All Five Lists||–||–||84|
In Table 1, we identify the overlap between the total number of universities on each list and the NCAA Division I category of interest to our study. While Division I is not necessarily a ranking, participation in Division I is an indicator that the university has a vested interest in engaging with alumni and the general public. A review of the unique appearance of a university on one or more lists demonstrates the diversity or lack thereof between the five rankings under consideration. Only Money Magazine, with its emphasis on perceived value, includes 115 institutions not evaluated elsewhere; while more than 53% of the universities in our dataset appear on at least two of the indicated lists. This anchoring of universities among the ranking lists is consistent with previous research (Bowman and Bastedo, 2011) regarding adherence to the status quo (see Section 2.1).
|University of California–Berkeley||13||5|
|University of California–Los Angeles||16||7|
|University of Pennsylvania||17||8|
|University of San Diego||—||112|
|Old Dominion University||—||112|
3.2. Standardizing the Rank Positions
Two of the ranking systems that contribute to our dataset bin universities alphabetically into groups after a certain threshold has been reached, resulting in tied ranking positions for those universities found lower on the list. After the first 200 individual rankings, THE places the remaining institutions ranked between 201 and 400 into bins of size 50 and then use bins of size 100 for ranks between 401 and 800. The ranking for each binned institution is the lowest number in the bin. All institutions listed alphabetically as ranked between 401 and 500 would be assigned rank 401. The rankings of ARWU are conducted similarly except ARWU starts to bin after the first 100 individual rankings.
One of the problems when comparing two ranked lists is that the items ranked in two particular lists are not identical, meaning items that appear in list A do not necessarily appear in list B. Fagin (Fagin et al., 2003) introduced a new measure which extends Spearman's Footrule by assigning a rank to the non-overlapping elements. For two rankings of size k, each element that appears in list A but does not appear in the list B (either totally missing from B or ranked at position [k]) is assigned rank k+1. For the purpose of our research, application of the footrule essentially places all universities which are not ranked at the end of a respective list. After removing the international entries, if any, the remaining institutions on each ranking list were sequentially ordered by rank as shown in Table 2 using the THE rank as an example. The sequential ordering according to relative position was necessary because of differences in the number of U.S. institutions on each list (see Table 1), and the need to standardize ranking positions to obtain concordance between all lists.
3.3. Computing Adjusted Reputation Rank
One of our research goals is to compute an adjusted reputation rank. Therefore, we must avoid unduly penalizing an institution by including a low, raw ranking on a particular list in our ARR calculation; especially when the institution is referenced on just one or two of the named lists. To ensure that we incorporate different ranking perspectives in our evaluation, we average the ordered positional rankings from all ranking lists in our consolidated dataset to compute a mean reputation score which we then use to sequentially order the listed universities to obtain the adjusted reputation rank shown in Table 4. Upon examination, we discovered that some schools which met the criteria to be ranked by Money Magazine based on value performed differently using the criteria established by the other ranking systems. For example, Columbia University is consistently in the top-15 of the other four ranking systems while Money Magazine ranks the school considerably lower at position 52. As described later in Section 4.1, we computed rank-order correlation for each of the rankings. Table 6 shows that the rankings from Money Magazine are consistently weak to moderately correlated with all other ranking lists we consider. Therefore, we exclude the Money Magazine rankings from our computation of ARR. The 115 schools which appeared only on Money Magazine were placed in a non-ranked position at the end of ARWU, THE, and the lists from U.S. News. A standardized ranking position was then calculated using the methodology described in Section 3.2.
|Ohio State University||40,452||3,633,887||136,966,818||1|
|University of Texas||36,072||3,341,835||152,853,239||2|
|Pennsylvania State University||39,077||3,635,730||117,818,050||3|
|University of Michigan||27,297||9,952,113||131,003,957||3|
|University of Wisconsin–Madison||27,867||2,465,051||122,975,876||5|
|University of Florida||29,577||1,550,000||130,772,416||6|
|Michigan State University||35,038||2,274,813||89,491,630||7|
|University of Washington||27,733||3,076,226||88,580,078||8|
|University of California–Los Angeles||29,027||1,864,605||96,912,767||9|
|University of California–Berkeley||26,320||7,997,099||76,348,304||11|
|University of Illinois||31,312||1,585,807||74,469,976||12|
|University of Southern California||17,898||4,709,511||105,919,366||14|
|University of Georgia||25,259||1,004,987||101,559,307||15|
3.4. Computing the Composite EEE Rank
We identified several candidate attributes in order to determine which combination of quantifiable attributes might provide a good evaluation metric for our ranking system. We empirically selected a combination of web-based and other characteristics which might be calculated or retrieved from the Web: athletic expenditures, undergraduate enrollment, monetary value of the endowment, institution age, primary and secondary Twitter followers. We also combined several of these metrics into a composite ranking consisting of endowment, expenditures, and enrollment (EEE); metrics which are possible to collect from web-based sources. The top-15 universities as ranked by our EEE score are shown in Table3. Due to the broad range of values in the individual components, each of the enrollment, endowment and expenditures was normalized individually across the full dataset of 264 universities to obtain the same scale, from 0 to 1, then aggregated to obtain a sequential EEE ranking of the universities.
We chose to include the total expenditures for men's and women's sports as a measure of the institution's commitment to branding and promoting the university as a whole. Further, we theorize whether the EEE score might serve as a viable proxy measure for a subset of our data, the NCAA Power Five, that we use later in Section 4.3 to assess the strength of UTE as a ranking attribute. The NCAA Power Five Conferences include the Southeastern Conference (SEC), Atlantic Coast Conference (ACC), Big Ten, Pac-12, and Big 12. The chosen conferences are composed of 65 flagship public and private universities who share excellent academic reputations, large endowments, and big budgets allocated for their athletic programs. These schools are representative of institutions that are playing at the highest level of NCAA competition and typically excel in two if not all three of the dimensions of enrollment, expenditures, and endowment.
|University of California–Berkeley||3||5||2||2||3||3||474,901||19|
|University of California–Los Angeles||7||7||4||4||6||6||394,815||28|
|University of Pennsylvania||10||8||10||8||9||8||778,805||5|
|University of Washington||9||13||7||6||9||8||274,674||44|
|University of Michigan||11||11||7||10||10||10||671,277||12|
|University of Minnesota||15||23||16||17||18||16||631,046||13|
3.5. Collecting University Demographic Data
As a starting point for obtaining key institutional and demographic information for each university, we extracted (scraped) the associated website as listed on the university's profile page maintained by the ranking list. We extracted information from multiple websites which included Division I conference membership from the National Collegiate Athletic Association (NCAA), athletic expenditures and endowment value from the National Center for Education Statistics777https://nces.ed.gov/, profile data from Twitter, historical conference data from Sports Reference888http://www.sports-reference.com/, primary and secondary Twitter account names from university homepages, undergraduate enrollment from the Integrated Postsecondary Education Data System999https://nces.ed.gov/ipeds/ (IPEDS) and founding dates from DBpedia101010http://wiki.dbpedia.org/. For endowments that were attributed to a university system (e.g., University of Minnesota Foundation vs. University of Minnesota-Twin Cities), we used DBpedia to obtain the endowment value for the particular university present in the ranking lists to avoid overstating the endowment. Specific institutional data such as the founding date that could not be obtained from another already mentioned source was also resolved using web searches of DBpedia.
3.6. Mining Official Twitter Accounts
One of the proposed performance indicators for our dataset is constructed around a set of primary Twitter seed accounts for each university. For the present study, the presence of Twitter friends is also needed to bootstrap the discovery of affiliated, secondary Twitter accounts. The complete process for identifying these accounts and determining the value for UTE is shown in Algorithm 1 and described here. As illustrated in Figure 2, we start with the URI for the university's homepage obtained from the detailed institutional profile information in the ranking lists. For each URI, we navigated to the associated webpage and searched the HTML source for links to valid Twitter handles. After examining the source anchor link text, we eliminated known false positives which were longer than 15 characters (Twitter limit for a valid screen name) or included /intent, /share, /tweet, /search or /hashtag in the URI which are directives to Twitter queries. Once the Twitter screen name was identified, the Twitter GET users/Show API was used to retrieve the URI from the profile of each user name. If the domain of the URI matched exactly or resolved to the known domain of the institution, we considered the account to be one of the university's official, primary Twitter handles since the user had self-associated with the university via the URI reference. As an example, the user names NBA, DukeAnnualFund, Duke_MBB, and DukeU were extracted from the page source of the Duke University homepage (www.duke.edu). However, only DukeAnnualFund and DukeU are considered official primary accounts because their respective URIs, annualfund.duke.edu and duke.edu, are in the same domain as the university.
As shown in Table 5, ten institutions did not have a Twitter account identified on the homepage as of August 2016, therefore, a primary official account could not be determined via our automated homepage search. For this subset only, we used the Google Custom Search Engine111111https://cse.google.com/cse/ to initiate an X-ray search using the keywords “institution URI” AND “twitter”. We accepted the top ranked result returned by Google, if any, as the official, primary Twitter account for the university. In the event that Google did not render a Twitter account in the search results, we manually searched for any remaining outstanding accounts using the search bar located on http://twitter.com.
|University||Twitter Screen Name|
|University of Louisville||@uofl|
|University of South Carolina||@uofsc|
|University of Missouri||@mizzou|
|University of North Carolina-Greensboro||@uncg|
|University of Evansville||@uevansville|
|Portland State University||@portland_state|
Colleges and universities have a reputation for being decentralized, with many departments operating independently of one another, maintaining a separate social media presence. However, we observed that only 24 of the 264 universities in our dataset promoted multiple, official Twitter accounts on their homepage. For the purpose of computing our UTE score, we want to consider the contribution of all university-affiliated Twitter accounts. Therefore, for each of the identified official, primary accounts, we obtained the full list of their Twitter friends, i.e., users that they follow. Again, we used the Twitter GET users/Show API to determine which of the friends could be included as secondary official Twitter accounts based on the URI in the profile (must have the same domain as the university). These secondary accounts might include the athletic teams, faculty members, and other university organizations. Once the primary and secondary accounts were identified, we used the Twitter GET followers/IDs API to retrieve and accumulate the follower count to form the UTE score for the university.
We launched our crawler to find all of the designated Twitter followers during the time period between June 15, 2016 and August 30, 2016. In total, we collected 1,087,000 user profiles. Approximately 9% of all the user accounts we collected were protected at the profile owner's request; allowing only their friends to view their profiles. Subsequently, we ignored these users in the computation of the UTE score because the underlying profile data is inaccessible using the Twitter API. Once we calculated the UTE score, we then ranked each university, in sequential order, based on the score, as shown in Table 4.
In this section we evaluate our UTE ranking by computing rank-order correlation with the adjusted reputation rank (Section 3.3) and the composite EEE rank (Section 3.4). We also directly compare the rankings of individual universities for the full dataset and discuss the implications for universities in the NCAA Power Five conferences.
4.1. Rank-Order Correlation
Since we know that the potential for tied rankings exists in our data, we used Kendall's Tau-b () rank-order correlation to test for statistically significant (p 0.05), moderate (0.40 0.60) or strong (0.60 0.80) correlations between the individual ranking systems and our adjusted reputation rank. Table 6 shows the respective inter-rank correlation measured in Kendall . With values in the range of 0.3189 to 0.4191, the rankings on Money Magazine are weak to moderately correlated with all other ranking lists including our ARR. This range of values confirms our intuition that the disparate ranking criteria based on value
and the underlying goals of the Money Magazine system appropriately deem it an outlier among the other lists. We note a strong correlation, in the range of 0.7634 to 0.8787, between the remaining four lists which indicates that (1) the criteria traditionally used to rank universities based on academic excellence changes slowly thus resulting in minimal differentiation in the selected universities and (2) the relative ranking position of a particular university is anchored and does not vary significantly from year to year. The strong correlation of 0.8787 between subsequent lists found in the 2015 and 2016 rankings in U.S. News along with the addition of only three new entrants in 2016 (see Table1) confirms this observation. The lack of variety between the U.S. News rankings is also consistent with the conclusions of Grewal et al. (Grewal et al., 2012), noted previously in Section 2.1, which indicated the high probability of a top-ranked university retaining its rank from year to year. Our adjusted reputation rank, with values in the range of 0.8285 to 0.9375, is strongly correlated with the rankings in ARWU, THE, and both years of USNEWS. Therefore, we conclude that ARR can be used as a representative proxy for any traditional ranking system.
4.2. Composite Ranking Correlation with UTE
In order to evaluate our UTE rank against the adjusted reputation rank and EEE rank, we again used Kendall's Tau-b () rank-order correlation to test for statistically significant (p 0.05), moderate (0.40 0.60) or strong (0.60 0.80) correlations. Using ARR as the ranking criteria, we selected the top-50, top-100, top-141 ranked on two or more lists, and all 264 universities in our dataset. As shown in Table 7(a), we found with a value of 0.6691, UTE is most strongly correlated with the ARR for the top-50 institutions followed closely by EEE at 0.5728. We must note the majority of the universities in the top-50 of any ranking list are usually members of the Ivy League or large schools with highly recognizable athletic programs like those in the Power Five (e.g., Ohio State, Penn State) so we might expect similarities in the metrics that comprise EEE. The correlation between UTE and ARR decreases slightly for the top-100, but persists to indicate a strong correlation, = 0.6018, when we examine the full dataset in Table 7(d). Our goal is to maximize the use of web-based metrics, therefore, choosing UTE over EEE should provide similar ranking results regardless of the size of the list. We conclude that primary and secondary Twitter followers, as we have defined for UTE, presents a strong metric for ranking and assessing the reputation of a university.
To further investigate the correlation of ARR, UTE, and EEE, we show scatterplots in Figure 3 of the combinations of the three rankings for all 264 universities. The colors represent bins of the EEE rank, which can be directly seen in Figure 2(a). As discussed in Section 3.3, the 115 schools that appeared exclusively on the Money Magazine list were binned and all assigned a rank of 142 on the ARR. Note that all of the universities in the first bin of EEE (black dots) are ranked below 150 in ARR, suggesting that universities with high enrollments, endowments, and/or athletic budgets also have high academic rank. Figure 2(b) (ARR vs. UTE) shows that there are several universities that have larger Twitter followings than can be explained just by academic rank (i.e., UTE rank is higher than ARR rank). Most of these rankings fall into the first bin of EEE, which could explain the increase in Twitter following. Twitter engagement provides an inexpensive means for smaller schools to reach a large audience, potentially enhancing their reputations. Figure 2(b) also shows that there are several smaller schools (in the last EEE bin, cyan dots) that have larger Twitter followings than their academic rank (not ranked in ARR) or EEE would explain. These schools may be making a concerted effort to enhance their profile and could potentially move into the standard academic rankings in the future. This would be an interesting avenue for future study. Finally, Figure 2(c) shows EEE vs. UTE, which indicates that as expected, universities with more financial resources tend to have larger Twitter followings, though there are still some universities in the lower EEE bins that have significant Twitter followings.
4.3. Correlation Between the NCAA Power Five
We use the fraternity of the schools in the Power Five to more closely examine the collective ranking correlation of these conferences based on their 2016 membership. Within the complete data set, we observed that 55 out of the 65 Power Five member institutions (84.6%) were ranked within the top-100 positions based on the ARR rank. Further, we found that all 65 schools (100%) were ranked within the top-100 positions based on the EEE rank. The latter observation is consistent with the strong correlation between EEE and UTE, = 0.6461, that we determined in Table 7(d) and is consistent with our intuition that large schools with ample financial resources would attract more Twitter followers. Figure 4 highlights the relationships between the Power Five and the various metrics by repeating the same charts from Figure 3 but with members of the Power Five shown in blue.
We noted several similarities which were indicative of the ten schools (15.4%) that were ranked outside of the top tier for ARR. Notably both Texas Christian and Mississippi State are the only schools which were not ranked on two or more of the ranking lists. Both schools also fall significantly below the mean values for the Power Five in terms of undergraduate enrollment ( 21,000), endowment value ( $2.3B), and athletic expenditures ( $90M), placing them at the bottom of the EEE ranking. On the other hand, Wake Forest is the smallest institution in the Power Five, but the school garners an academic reputation (ARR=45) that cannot be readily explained by its comparatively low EEE ranking (EEE=97).
We also note four schools that fall within the bottom 50% of UTE. In particular, the University of Louisville could achieve a considerable boost in UTE ranking ( 107,000 followers) if the Twitter account used by the athletic department (@GoCards) would reference the primary URI of the university rather than its own domain ( http://gocards.com). We discovered 284 primary and secondary accounts followed by Georgia Tech, however only four of these could be considered official, because 150 of 280 secondary accounts did not include a URI in the profile bio. A similar scenario was noted for Oregon State where 271 of the 341 secondary accounts did not include a URI. While we identified 74 official accounts for the University of Pittsburgh, as was the case with Louisville, 140,000 underreported secondary Twitter accounts are associated with university sports. We discovered the Twitter followers of Wake Forest are bolstered significantly by a single celebrity professor, Melissa Harris Perry, who in addition to her faculty position previously hosted a weekly news style program on US television. More than 80% of the Wake Forest UTE score is attributed to the verified MHarrisPerry Twitter account which has more than 600,000 followers.
In Appendix A, we note the diverse, though not exhaustive, spectrum of unique university domains found among secondary Twitter accounts of the NCAA Power Five. Upon visual inspection of the web content of each domain, we find they are related to the university in some capacity (e.g., sports, clubs), but do not conform to our domain association rule. The omission of the UTE for the associated secondary Twitter accounts can, in some cases, significantly lower our calculation of UTE score. For those under performing universities, in terms of Twitter followers, inclusion of more domains would elevate the UTE rank of the university and likely present a stronger correlation of Kendall's Tau-b () than was noted in Table 6. We did not attempt to identify additional secondary domains for the entire set of 264 universities in our dataset. This exercise would be manually intensive and counter to our stated goal of automated data collection.
As noted during our own collection efforts, the quality and availability of the data chosen as performance indicators can impede the efficiency of constructing of a gold standard data set. Manual correction can improve the data collection, but is expensive and is not conducive to reproducible research. We observed that institutions themselves do not maintain a complete listing of all official Twitter accounts as noted by the number of undiscovered and undocumented accounts we extracted during a secondary search. We must also acknowledge the impact of celebrity professors and verified accounts (e.g., Melissa Harris Perry). Given the small number of verified accounts among our official Twitter profiles, we contend that celebrity faculty members might be equated to the influence of Nobel Prize laureates; an indicator which is used by some ranking systems. We did not address known issues with bots and spam accounts which may over inflate the stated number of Twitter followers which is the primary component of our UTE score (e.g., (Davis et al., 2016)) . We also understand that our methodology constrains universities to a single official hostname which can deflate the UTE score as Twitter accounts that reference other university-owned domains are omitted. Based on our research assumptions, we observed that enrollment does not necessarily increase the Twitter followers needed to compute UTE. Universities are not taking the opportunity to advertise their Twitter accounts and are at times promoting other entities on their homepage. This observation necessitated the need to expand the follower network as we have defined. Schools with highly visible sports programs, like those in the Power Five, tend to have more Twitter followers as the public is more aware of the university's overall brand. In general, the perceived reputation of any university is impacted less by metrics which are intrinsic to the institution, but intangibles that translate into more impressions or brand awareness by the public and constituents. This parallels the assertions in prior research (Leavitt et al., 2009; Preussler and Kerres, 2010) which contends that popular entities are more likely to attract more followers (see Section 2.3).
6. Conclusions and Future Work
We examined and ranked a set of 264 U.S. universities extracted from the NCAA Division I membership and lists published in U.S. News, Times Higher Education, Academic Ranking of World Universities and Money Magazine using an adjusted reputation rank which we compared to our University Twitter Engagement score; the friend and extended follower network of primary and affiliated secondary Twitter accounts referenced on a university's home page. When compared to our adjusted reputation rank for all 264 represented universities, we noted a strong correlation, =0.6018, with UTE. We conclude that our UTE rank is comparable to those presented in other academic-based ranking systems, however, we present a low-cost data acquisition methodology using only web-based artifacts. UTE also offers a distinct advantage because (1) it can be calculated on-demand and (2) it promotes diversity in the ranking lists as any university with a Twitter account can be given a UTE rank. These results are highly reproducible as they are derived from social media and obtained using a publicly accessible Twitter API. A similar aggregation strategy might also be applied to other popular social platforms such as Instagram or YouTube. The use of a web-based API allows our results to be calculated on a near-real time basis rather than annually which is the norm for other ranking systems.
The use of web metrics might also provide an incentive for institutions to increase their web presence as way to further engage with constituents and the general public. Social media allows us to measure another proxy for reputation; how the universities and the public engage with one another. The universities themselves have to decide whether this kind of outreach is important and invest in it, and the public needs to be interested enough to follow them.
Our study is subject to a number of limitations that present opportunities for future work. Campbell’s and Goodhart's law suggest that if UTE becomes popular, institutions may seek to artificially increase their Twitter followers in order to increase their ranking. Future work could include only the Twitter accounts of real people. In order to obtain a more complete set of official Twitter accounts, the domain associated with the account URI could be expanded to include all registered domains for the university. Additional research might also broaden the scope of our study to include both U.S. and international universities. It might also be advantageous to subject the observations made in this paper to a temporal analysis to ascertain whether the UTE rankings, at least for those in the upper echelon, persist over time and to look for non-linear spikes in Twitter followers which may indicate artificial manipulation.
- Aguillo et al. (2008) Isidro F Aguillo, Jose Luis Ortega, and Mario Fernández. 2008. Webometric ranking of world universities: Introduction, methodology, and future developments. Higher education in Europe 33, 2-3 (2008), 233–244.
- Altbach (2015) Philip Altbach. 2015. The dilemmas of ranking. International Higher Education 42 (2015).
- Antoniadis et al. (2016) Konstantinos Antoniadis, Kostas Zafiropoulos, and Vasiliki Vrana. 2016. A Method for Assessing the Performance of e-Government Twitter Accounts. Future Internet 8, 2 (2016), 12.
- Black (1993) Thomas R Black. 1993. Evaluating social science research: An introduction. Sage.
- Bowman and Bastedo (2011) Nicholas A Bowman and Michael N Bastedo. 2011. Anchoring effects in world university rankings: exploring biases in reputation scores. Higher Education 61, 4 (2011), 431–444.
- Cha et al. (2010) Meeyoung Cha, Hamed Haddadi, Fabricio Benevenuto, and P Krishna Gummadi. 2010. Measuring User Influence in Twitter: The Million Follower Fallacy. ICWSM 10, 10-17 (2010), 30.
- Davis et al. (2016) Clayton A. Davis, Onur Varol, Emilio Ferrara, Alessandro Flammini, and Filippo Menczer. 2016. BotOrNot: A System to Evaluate Social Bots. Technical Report arXiv:1602.00975.
- Dijkmans et al. (2015) Corné Dijkmans, Peter Kerkhof, and Camiel J Beukeboom. 2015. A stage to engage: Social media use and corporate reputation. Tourism Management 47 (2015), 58–67.
- Enserink (2007) Michael Enserink. 2007. Who ranks the university rankers? Science 317, 5841 (2007), 1026–1028.
- Fagin et al. (2003) Ronald Fagin, Ravi Kumar, and D Sivakumar. 2003. Comparing top k lists. SIAM Journal on Discrete Mathematics 17, 1 (2003), 134–160.
- Go and You (2016) Eun Go and Kyung Han You. 2016. But not all social media are the same: Analyzing organizations’ social media usage patterns. Telematics and Informatics 33, 1 (2016), 176–186.
- Goglio (2016) Valentina Goglio. 2016. One size fits all? A different perspective on university rankings. Journal of Higher Education Policy and Management 38, 2 (2016), 212–226.
- Greenwood et al. (2016) Shannon Greenwood, Andrew Perrin, and Maeve Duggan. 2016. Social Media Update2016. (2016).
- Grewal et al. (2012) Rajdeep Grewal, James A Dearden, and Gary L Llilien. 2012. The university rankings game. The American Statistician (2012).
- Heller Baird and Parasnis (2011) Carolyn Heller Baird and Gautam Parasnis. 2011. From social media to social customer relationship management. Strategy & leadership 39, 5 (2011), 30–37.
- Jansen et al. (2009) Bernard J Jansen, Mimi Zhang, Kate Sobel, and Abdur Chowdury. 2009. Twitter power: Tweets as electronic word of mouth. Journal of the American society for information science and technology 60, 11 (2009), 2169–2188.
- Kang (2014) Minjeong Kang. 2014. Understanding public engagement: Conceptualizing and measuring its influence on supportive behavioral intentions. Journal of Public Relations Research 26, 5 (2014), 399–416.
- Klein et al. (2009a) Martin Klein, Olena Hunsicker, and Michael L Nelson. 2009a. Comparing the Performance of US College Football Teams in the Web and on the Field. In Proceedings of the 20th ACM conference on Hypertext and hypermedia. ACM, 63–72.
- Klein et al. (2009b) Martin Klein, Olena Hunsicker, and Michael L Nelson. 2009b. Correlation of music charts and search engine rankings. In Proceedings of the 9th ACM/IEEE-CS joint conference on Digital libraries. ACM, 415–416.
- Kunegis et al. (2013) Jérôme Kunegis, Marcel Blattner, and Christine Moser. 2013. Preferential Attachment in Online Networks: Measurement and Explanations. In Proceedings of the 5th Annual ACM Web Science Conference. ACM, 205–214.
- Kwak et al. (2010) Haewoon Kwak, Changhyun Lee, Hosung Park, and Sue Moon. 2010. What is Twitter, a social network or a news media?. In Proceedings of the 19th international conference on World wide web. ACM, 591–600.
- Leavitt et al. (2009) Alex Leavitt, Evan Burchard, David Fisher, and Sam Gilbert. 2009. The influentials: New approaches for analyzing influence on twitter. Web Ecology Project 4, 2 (2009), 1–18.
- Men and Tsai (2015) Linjuan Rita Men and Wan-Hsiu Sunny Tsai. 2015. Infusing social media with humanity: Corporate character, public engagement, and relational outcomes. Public Relations Review 41, 3 (2015), 395–403.
- Nelson et al. (2008) Michael L. Nelson, Martin Klein, and Manoranjan Magudamudi. 2008. Correlation of Expert and Search Engine Rankings. Technical Report arXiv:0809.2851. Old Dominion University Department of Computer Science.
- Preussler and Kerres (2010) Annabell Preussler and Michael Kerres. 2010. Managing reputation by generating followers on Twitter. Medien–Wissen–Bildung Explorationen visualisierter und kollaborativer Wissensräume (2010), 129–143.
- Saito and Masuda (2013) Kodai Saito and Naoki Masuda. 2013. Two types of Twitter users with equally many followers. In Proceedings of the 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining. ACM, 1425–1426.
- Tsimonis and Dimitriadis (2014) Georgios Tsimonis and Sergios Dimitriadis. 2014. Brand strategies in social media. Marketing Intelligence & Planning 32, 3 (2014), 328–344.
- Weller and Kinder-Kurlanda (2016) Katrin Weller and Katharina E Kinder-Kurlanda. 2016. A manifesto for data sharing in social media research. In Proceedings of the 8th ACM Conference on Web Science. ACM, 166–172.
- Weng et al. (2010) Jianshu Weng, Ee-Peng Lim, Jing Jiang, and Qi He. 2010. Twitterrank: finding topic-sensitive influential twitterers. In Proceedings of the third ACM international conference on Web search and data mining. ACM, 261–270.
Appendix A Supporting Tables
Arizona State University
|Florida State University||fsu.edu||48||
|Iowa State University||iastate.edu||16||
|Kansas State University||k-state.edu||4||
|Michigan State University||msu.edu||27||
|Mississippi State University||msstate.edu||10||
|North Carolina State University||ncsu.edu||10||
|Ohio State University||osu.edu||28||
|Oklahoma State University||osu.okstate.edu||35||
|Oregon State University||oregonstate.edu||9||
|Pennsylvania State University||psu.edu||71||
|Texas A&M University||tamu.edu||49||
|Texas Christian University||tcu.edu||38||
|Texas Tech University||ttu.edu||66||
|University of Alabama||ua.edu||14||
|University of Arizona||arizona.edu||19||
|University of Arkansas||uark.edu||14||
|University of California–Berkeley||berkeley.edu||42||
|University of California–Los Angeles||ucla.edu||22||
|University of Colorado||colorado.edu||6||
|University of Florida||ufl.edu||11||
|University of Georgia||uga.edu||24||
|University of Illinois||illinois.edu||66||
|University of Iowa||uiowa.edu||22||
|University of Kansas||ku.edu||78||
|University of Kentucky||uky.edu||13||
|University of Louisville||louisville.edu||8||
|University of Maryland||maryland.edu||5||
|University of Miami||miami.edu||6||
|University of Michigan||umich.edu||5||
|University of Minnesota||twin-cities.umn.edu||63||
|University of Mississippi||olemiss.edu||58||
|University of Missouri||missouri.edu||2||mutigers.com||11||23,629,273|
|University of Nebraska||unl.edu||2||
|University of North Carolina||unc.edu||79||
|University of Notre Dame||nd.edu||11||
|University of Oklahoma||ou.edu||9||
|University of Oregon||uoregon.edu||25||
|University of Pittsburgh||pitt.edu||4||
|University of South Carolina||sc.edu||24||gamecocksonline.com libraries.usc.edu||72||1,003,376|
|University of Southern California||usc.edu||110||
|University of Tennessee||utk.edu||36||
|University of Texas||utexas.edu||19||
|University of Utah||utah.edu||22||
|University of Virginia||virginia.edu||8||
|University of Washington||washington.edu||7||
|University of Wisconsin||wisc.edu||72||
|Washington State University||wsu.edu||54||
|West Virginia University||wvu.edu||26||