A Data Science Approach for Honeypot Detection in Ethereum

10/03/2019 ∙ by Ramiro Camino, et al. ∙ University of Luxembourg 0

Ethereum smart contracts have recently drawn a considerable amount of attention from the media, the financial industry and academia. With the increase in popularity, malicious users found new opportunities to profit from deceiving newcomers. Consequently, attackers started luring other attackers into contracts that seem to have exploitable flaws, but that actually contain a complex hidden trap that in the end benefits the contract creator. This kind of contracts are known in the blockchain community as Honeypots. A recent study, proposed to investigate this phenomenon by focusing on the contract bytecode using symbolic analysis. In this paper, we present a data science approach based on the contract transaction behavior. We create a partition of all the possible cases of fund movement between the contract creator, the contract, the sender of the transaction and other participants. We calculate the frequency of every case per contract, and extract as well other contract features and transaction aggregated features. We use the collected information to train machine learning models that classify contracts as honeypot or non-honeypots, and also measure how well they perform when classifying unseen honeypot types. We compare our results with the bytecode analysis method using labels from a previous study, and discuss in which cases each solution has advantages over the other.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Blockchain technology revolutionized the way assets are traded around the world. A blockchain can be seen as an append-only ledger that is maintained by a decentralised peer-to-peer network. The transfer of assets is recorded in form of transactions, which are grouped every few seconds in into so-called blocks. Each block is linked to its previous block via cryptographic means, thereby forming a “blockchain”. This list of blocks is stored across the participants of the blockchain. A consensus protocol dictates the appending of new blocks. Bitcoin [10] and Ethereum [18] are currently the two most prominent implementations of blockchain technology. Blockchain technology is considered as one of today’s major innovations due to the fact that it allows to decentralize the governance of any asset. While the purpose of the Bitcoin blockchain was to decentralize the governance of cash, and thereby the role of banks, the purpose of the Ethereum blockchain is to decentralize the computer as a whole. As of the time of writing, Ethereum has a market capitalisation of more than $22 billion111https://coinmarketcap.com/currencies/ethereum/. Ethereum builds on top of Bitcoin’s principles and offers in addition the execution of so-called smart contracts.

Smart contracts are Turing-complete programs that are executed by the participants of the Ethereum peer-to-peer network. They are identified by an address (a 160-bit identifier), and they get deployed or invoked via transactions. We distinguish between normal and internal transactions. Normal transactions refer to transactions that are initiated by normal user accounts, whereas internal transactions refer to transactions that are initiated by smart contracts and are performed as an reaction to a normal transaction. In other words, every internal transaction has a normal transaction as its origin. Smart contracts may interact with Ethereum’s native cryptocurrency called ether, and receive input from participants. Ether is a unit and can be divided into further subunits, where the smallest subunit is called wei. The blockchain enforces that smart contracts are run exactly as they are programmed, without any possibility of censorship or downtime. Smart contracts are programmed using a high-level programming language, such as Solidity [19]. The execution of smart contracts costs gas. Users can determine how much gas they want to provide to the execution of a smart contract (gas limit), and how much they want to pay per gas unit in therms of wei (gas price). When the total mount of gas assigned to a transaction is consumed, program execution is terminated, an error is thrown and the effects of the transaction are reversed. As with any program, smart contracts may contain bugs [1]. Several smart contracts have already been attacked in the past. The two most prominent hacks, the DAO hack [15] and the Parity wallet hacks [12], caused together a loss of over $400 million. As a result, a number of tools that have been presented in order to scan smart contracts for different types of vulnerabilities [8, 16, 17].

Interestingly, recent works show that attackers started deploying on purposely, vulnerable-looking smart contracts in order to lure other attackers into traps [6, 7]. This new type of fraud is known as “honeypots” [14, 13]. Honeypots are smart contracts, that appear to have a flaw in their design, that allows any arbitrary user on the Ethereum blockchain to drain the funds contained in the contract. However, once the user tries to exploit this apparent vulnerability, a second, yet unknown, trapdoor unfolds which prevents the draining of ether to succeed. The exploitation of honeypots does not come for free. Their exploitation is always linked to the transfer of a certain amount of ether a priori to the contract. The main idea is that the victim solely focuses on the apparent vulnerability and does not consider the possibility that a second vulnerability might be hidden in the contract. Moreover, honeypot creators often upload the source code of their smart contract to Etherscan222https://etherscan.io/ in order to facilitate the detection of their “vulnerable” smart contract. Etherscan is an online platform that allows anyone to navigate through the Ethereum blockchain and to publish the source code of any smart contract. There exist various types of honeypots. They differ on the trapdoor that they use to steal the funds from the victim, and on the vulnerability that they exploit to lure the victim.

Data science is a discipline that draws techniques from different fields like mathematics, statistics, computer science, and information science, with the goal of discovering insights from real world data. The term is closely related to other fields like machine learning, data mining and business intelligence. Transactions from new financial technologies have recently called the attention of data science practitioners. For example, machine learning has been used for trying to predict Bitcoin price [9]. Another study [4]

presented an anomaly detection methodology for transactions from a real-time gross settlement system, remittance and currency exchange network called Ripple

333https://www.ripple.com/. To the best of our knowledge, no previous studies have analysed the behavior of honeypot contracts (or any kind of contract) in the Ethereum blockchain from a data science perspective.

1.0.1 Contributions.

In this paper, we present a step by step methodology to obtain, process and analyze Ethereum contract transactions for the task of honeypot detection. We show how assumptions and hypotheses about honeypot behavior can be contrasted with real data, and we propose how the detection process can be automatized using machine learning. The contributions of this work can also be extended to the detection of other special types of account, both for Ethereum and for other platforms that deal with monetary transactions.

2 Data Acquisition

We scanned the first 6,500,000 blocks from the Ethereum blockchain and extracted data related to 2,019,434 smart contracts with the following properties:

  • address:str ID of the contract.

  • timestamp:int date and time when the contract was created.

  • creator:str ID of the contract creator (user or another contract).

  • block:int ID of the block where the contract was created.

  • transaction:str ID of the transaction that created the contract.

  • byte_code:bytes the contract compiled code.

Using the contract addresses from the previous list, we acquired compilation and source code information from the Etherscan API444https://etherscan.io/apis#contracts:

  • source_code:str the actual source code text.

  • compiler_runs:str amount of compiler optimization runs.

  • compiler_version:str major, minor and patch version of the compiler.

  • library:str ID of the library used (if present).

We also downloaded all the transactions related to our contracts using the Etherscan API555https://etherscan.io/apis#accounts. We gathered around 141M normal and 4.6M internal transactions. For both types, we have the following common properties:

  • from:str ID of the transaction sender.

  • value:str value sent in wei (1 ether = wei).

  • gas:str the gas in wei that the sender expected to use.

  • gas_used:str the gas in wei that the sender ended up using.

  • is_error:str indicates if transaction execution raised an error.

For normal transactions, some properties behave as follows:

  • to:str ID of the contract; when omitted, it is the contract creation transaction (see contract_address).

  • hash:str ID of the transaction.

  • timestamp:int date and time when the transaction was created.

  • block:int ID of the block where the transaction was created.

  • index:int position of the transaction inside the block.

  • contract_address:str the ID of the contract that we used to crawl the transaction, only present in the creation transaction.

But for internal transactions, the previous properties behave in a different way:

  • to:str ID of the transaction receiver; when omitted, another contract was created (see contract_address).

  • hash:str ID of the normal transaction that triggered this transaction.

  • timestamp:int date and time when the normal transaction that triggered this transaction was created.

  • block:int ID of the block where the normal transaction that triggered this transaction was created.

  • contract_address:str when an internal transaction creates another contract, this property contains the ID of the new contract.

We compiled a list of contracts labeled as honeypots from the public repository666https://github.com/christoftorres/HoneyBadger/tree/master/results/evaluation of a previous work [6]. One list is provided for each type of honeypot, only for contracts with source code available in Etherscan. The authors developed a tool that generated the labels, and for each case, they indicated if the labeling was correct after a manual inspection of the contract source code. For groups of honeypot contracts that share the exact same byte code, the authors only included one representative address in their list. In order to obtain a label for every contract, we first calculated a hash for the byte code of each contract, and then grouped them based on the hashes. We then transferred the label of each representative contract found in the original list to each contract in its group. We only labeled as honeypot the contracts that were manually confirmed by the authors, and ignored the false positives generated by their tool.

2.1 Feature Engineering

We computed a set of features that we considered relevant to the problem with two goals in mind: on one side, to obtain a better understanding of the data and to verify if the definition of honeypots matches the behavior of their transactions; and on the other side, to experiment with machine learning models and determine if the honeypot detection can be automatized with this approach.

We divided the features in three categories depending on how we extracted them: from the source code, from the aggregation of transaction properties, and from the flow of funds during the execution of transactions. In the following sections we describe the extraction process for each the features on each category and the reason why we believe that they are well suited for the analysis.

2.1.1 Source Code Features

  • contract_has_byte_code:bool indicates if the contract was created with bytecode. The absence of bytecode indicates that the contract presented errors during the creation and was not properly deployed. It makes no sense to analyze a contract if it has no bytecode, because it has no behavior.

  • contract_has_source_code:bool by definition, honeypots are presented as contracts with flaws that can be exploited, and these flaws are expected to be detected by inspecting the source code of the contract. Hence, having source code available should be an important indicator. Furthermore, the labels we obtained only consider contracts with source code.

  • contract_num_source_code_lines:int we hypothesize that the source code of a honeypot might have certain level of readability and interpretability. Counting the number of lines in the source code is not the best approximation of those characteristics, but it is a decent first approximation that does not require static code analysis or symbolic execution.

  • contract_compiler_runs:int we are interested in determining if a high amount of optimizations during the compilation could lead to an unexpected behavior that can be exploited by honeypots.

  • contract_library_index:int

    we collected all the possible libraries and created a categorical variable of 749 unique values. The index indicates the position in our list of libraries. We are interested in knowing if a library in particular is correlated with the occurrence of honeypots.

And finally, we split the contract_compiler_version:str into three parts: major, minor and patch. We are interested in finding if a major, minor or patch version in particular affected the occurrence of honeypots, caused by some change in the solidity language compilation that can be exploited. For each part, we collected all the possible values and calculated two categorical variables:

  • contract_compiler_minor_version_index:int 4 unique values.

  • contract_compiler_patch_version_index:int 330 unique values.

The compiler major version has the same value for all the examined contracts (major version 0), so we discard the feature for not providing any information.

2.1.2 Transaction Features

First we start with the features we generate both for normal and internal transactions. When referring to a particular case of these features, we use the prefixes normal_ and internal_ respectively:

  • transaction_count:int

    simply the number of transactions of the corresponding type associated with the contract. We expect to filter out contracts with high amount of movement, which are probably not honeypots.

  • transaction_other_sender_ratio:float we calculate this by dividing the amount of unique senders different than the contract creator, divided by the amount of total senders different than the contract creator. The feature should be at maximum 1 when every transaction sent by a non-creator account is unique, close to zero when they tend to be repeated, and we define it as zero when the creator is the only sender. We would like to know if the participation of multiple senders is a characteristic of honeypots or not.

For each of the three properties of transactions related to the movement of funds, we collect all the amounts per contract to compute the average and standard deviation, yielding a total of six features per transaction type:

  • transaction_value_[mean|std]:float

  • transaction_gas_[mean|std]:float

  • transaction_gas_used_[mean|std]:float

The value is only collected when there are no errors in the transaction, because if there is an error, the value is rolled back, but the gas is spent anyways. We expect these feature means to be positive but small for honeypots, but might be big for some popular non-honeypot contracts, and zero for contracts with testing purposes. Furthermore, for the feature standard deviations we have some weak intuitions we might like to test: high standard deviations for gas and gas used might be correlated with contracts with many different execution paths, becoming a proxy indicator for the complexity of the contract code. Additionally, the standard deviation for the value may point out that the contract is dealing with very different scales of negotiations.

We extract features related to time and ordering only for normal transactions. Internal transactions do not carry their own timestamps or block numbers, and we cannot assume the results from Etherscan to be chronologically sorted. Our intuition is that honeypots should have short lives compared to other contracts:

  • normal_transaction_block_span:int derived from the block numbers of the first and last transaction recorded for the contract.

  • normal_transaction_time_span:int computed from the timestamps of the first and last transaction recorded for the contract.

We also computed the difference of block and time between transactions. Both the mean and standard deviation indicate how often the contract receives transactions (only works for contracts with more than one transaction):

  • normal_transaction_block_delta_[mean|std]:float

  • normal_transaction_time_delta_[mean|std]:float

Lastly, features extracted only for internal transactions:

  • internal_transaction_creation_count:int indicates how many contracts this contract created. Our hypothesis is that honeypots are less likely to create other contracts, but discovering the opposite would be interesting.

  • internal_transaction_to_other_ratio:float this is analogous to the feature transaction_other_sender_ratio, but calculated for the receivers, than can be different than the contract in the case of internal transactions.

2.1.3 Fund Flow Features

We would like to learn if the way the balance of different accounts change as a result of normal transactions can be useful to identify honeypots. According to the definition of honeypots, we would expect to find the following events at least once among their transactions:

  1. A valid contract creation.

  2. An initial deposit, that is, the creator sending ether to the contract. This could be interpreted as “setting the trap”, because without funds, no attacker would be interested in exploiting flaws in the contract.

  3. A victim falling for the trap, which means, a transaction sent by an account different than the creator, for which the funds are kept in the contract.

  4. A withdraw from the creator, when the profit is collected.

Most importantly, a honeypot should rarely, if ever, present a withdrawal from an account other than the creator of the contract.

We define different events triggered by normal transactions (and potentially followed by internal transactions), where each possible event falls in exactly one case. We describe these cases with the combination of eight variables in Table 1.

Name Possible Values Restrictions
sender creator, other
creation yes, no sender=creator
error yes, no
balance_creator up, unchanged, down correct balance
balance_contract up, unchanged, down correct balance
balance_sender up, unchanged, down correct balance, sender=other
balance_other_positive yes, no correct balance
balance_other_negative yes, no correct balance
Table 1: Fund Flow Variables.

For each normal transaction we collect the corresponding triggered internal transactions. The sender and creation variable values are computed as described in Section 2.1.2. However, the error variable value in this case is an aggregation: it is set as True if the normal transaction or any of the triggered internal transactions contain an error. For the balance changes, we use the following algorithm: we initialize every account with zero funds moved, and for each transaction (both normal and internal), we subtract the value from the transaction.from account and add it to the transaction.to account (or to the transaction.contract_address account for the contract creation special cases). After iterating the transactions, we check the funds moved for the creator, contract and sender accounts. We mark them as up, unchanged or down if the values are positive, zero or negative respectively. If any account other than the three mentioned above suffer changes, we have two special flags to indicate if at least one other account balance went up or if at least one other account balance went down. This allows us to check whether the balances make sense without adding variables for more accounts. We prefer to keep a small and fixed number of variables, and computing all the possible combinations would yield cases, many of them invalid. For example, creation=yes only makes sense when sender=creator, and we only care about balance_sender when sender=other. Most importantly, at least one balance goes up if and only if at least one balance goes down. After filtering invalid combinations, we obtain 244 cases. We assign an arbitrary index to each case, and assign the corresponding case index to every normal transaction. Finally, for every contract, we extract the frequency of each case, by counting how many times a case appears in the contract divided by the total amount of cases related to the contract.

3 Data Analysis

3.1 Honeypot Types and Block Span

As a first sanity check, we analyze the distribution of honeypot types and the time periods for when each of them remained active. Note that without inspecting the byte code or the execution paths, we cannot infer when a contract was destroyed only from the transaction history, but at least we can see when was the first and the last activity. In Figure 1 we can observe that Hidden State Update is the most common type and the one with the longest block span. Most importantly, we see that most of the types share a common time window, hence, the features related to compiler versions should not have a great impact in the classification. All these findings are consistent with the statistics shown in [6].

Figure 1: Left: Amount of honeypots by type. Right: Block span of honeypots by type.

3.2 Most Relevant Numerical Features

We take all the numerical features and study their distributions for both the cases of honeypots and non-honeypots, to see if there is a clear difference. We only include in this study the results for the features we consider the most relevant. Most of these distributions are very long tailed, so instead of including histograms, we show in Table 2 a description of those distributions:

Feature contract_num_source_code_lines normal_transaction_count normal_transaction_value_mean
Honeypot Yes No Yes No Yes No
count 295.00 158568.00 295.00 158568.00 295.00 158568.00
mean 57.00 279.70 5.13 680.61 0.27 5.46
std 22.80 278.42 4.26 33198.52 0.22 570.30
min 19.00 1.00 1.00 1.00 0.00 0.00
25% 41.00 109.00 3.00 1.00 0.10 0.00
50% 54.00 177.00 4.00 2.00 0.25 0.00
75% 67.00 386.00 6.00 4.00 0.38 0.00
max 185.00 11409.00 32.00 10412943.00 1.67 204365.82
Table 2: Distribution description for the most relevant numerical features. The amounts for normal_transaction_value_mean are shown in ether for readability.
  • contract_num_source_code_lines: Honeypots seems to have a reasonable amount of lines of code in general, with a minimum that could include a fair amount of logic, and a maximum that is still readable. On the other side, non-honeypots present extreme values on both sides, from contracts with only one line of code to a contract with more than 11K lines.

  • normal_transaction_count: We can see that three quarters of the contracts have a small amount of normal transactions. However, the maximum is really big for non-honeypots, while for honeypots is only 32.

  • normal_transaction_value_mean: It seems that all honeypots always move some ether, but most non-honeypots contracts (more than 75%) do not exchange value at all. Nonetheless, non-honeypots have very extreme maximum value movement, and honeypots move on average no more than 2 ether.

Handcrafted rules could be used to cut the extremes of all these features (refer to the appendix for examples). However, one of the problems of those rules, is that they are overfitted to the dataset used to craft them. Even if the boundaries are relaxed, if a new honeypot crosses one of those boundaries (both accidentally or on purpose), it would be automatically classified as non-honeypot disregarding the rest of its properties. In any case, the rules we could craft can capture all the honeypots but do not exclude a big portion of non-honeypots. Of course we could start combining several rules until we find the best filter, but we automatize this kind of task using machine learning in Section 4, with models that can also handle probability measures instead of hard boundaries.

3.3 Categorical Variables

All the categorical features we extracted belong to the source code category, related to the compiler version and the library:

  • The compiler minor version has only four possible values. Most of the contracts in general belong to the minor version 4, and in particular, all but one honeypot among our samples belong to that version too. These statistics agree with a popularity peak in Ethereum described on a previous study [6].

  • The compiler patch version has 330 possible values. We can see that one patch has notoriously more honeypots than others, which corresponds to the compiler version 0.4.19+commit.c4cbbb05. However, checking the right side of Table 11, we see that there is no specific honeypot type that is significantly more common than the others. We repeated the procedure of counting honeypot types for the rest of the compiler versions obtaining the same results. Therefore, we do not think that the honeypot activity raised thanks to a compiler version in particular, and that the high frequency for some versions is rather correlated with the time when they were active, which could be at the same time linked with other factors like Ethereum popularity.

  • Finally, none of the honeypots have a library, and not many contracts in general have them either. Hence, we can presume that no library is linked at all with honeypot development, or that no library contains vulnerabilities that can be exploited for honeypot related purposes.

A detailed description of each of there features can be found in the appendix.

3.4 Fund Flow Cases

We want to study the events related to fund movement mentioned in Section 2.1.3, in order to verify if honeypots behave as we expect. We query our data by partially defining some of the fund flow variables, and then adding the frequencies of all the corresponding fund flow features that match those variable values:

  • Contracts with deposits from the creator: this should be a must for any honeypot to work, since funds are needed to attract attackers. However, we find that 29 honeypots (around 10% of them) do not match this criteria. Inspecting individual cases, we find that these contracts are actually honeypots, but that they rely on initial deposits from victims to attract even more victims. Another explanation could be that the creators of honeypots sometimes use a different account to set their trap. We do not find clear evidence of such practice among our data, but future work could address this topic, for example, using graph techniques to find colluding accounts.

  • Contracts with deposits from others: we find 192 honeypots that do not present this property, which means that only 103 (roughly 35% of them) are effective. This metric is close the statistics presented in [6].

  • Contracts with withdraws from the creator: only 108 honeypot contracts (37% of them) contain this event. It makes sense to expect the creator to withdraw funds after the trap was effective, and this metric is close to the previous one. Yet, the number is not exactly the same, which could mean that the creator sometimes withdraws the bite when the trap is not working.

  • Contracts with withdraws from others: this should never happen for honeypots, but we have 50 cases (17%) matching this criteria across our samples. We can find some interesting cases among this group, where the honeypot creator failed to set the trap properly. For example, the contract 0xbC272B58E7cD0a6002c95aFD1F208898D756C580 belongs to the Hiden State Update category. For this kind of honeypot, a transaction is needed to change the value of one or more variables in the contract. This update mechanism is usually hard to track, because Etherscan does not show internal transactions that carry no value, and victims fall into a trap when they do not know the state of the contract. However, for our contract example, the hidden state change transaction is missing, and the users that were supposed to be victims obtained an actual profit.

More details about the frequencies of these cases can be found in the appendix.

Nevertheless, among the top five most frequent balance movement cases, we still can find creator deposits and withdrawals, deposits from non-creator accounts, and the contract creation (without any fund movement). The unexpected top case, for both honeypots and non-honeypot contracts, are transactions from non-creator accounts that do not generate any kind of fund movement. Deposits from non-creator accounts are also very common for non-honeypot contracts, but the rest of the top cases differ. The details are enumerated in Table 3.

Honeypots
Count Index Description
341 205 sender=other
304 83 sender=creator, creator=negative, contract=positive
292 33 sender=creator, creation=True
168 201 sender=other, contract=positive, sender=negative
136 73 sender=creator, creator=positive, contract=negative
Non-Honeypots
Count Index Description
94475857 205 sender=other
9667149 207 sender=other, sender=negative, other_positive=True
9130582 201 sender=other, contract=positive, sender=negative
8569634 77 sender=creator
5647532 127 sender=other, error=True
Table 3: The five most frequent fund flow cases for honeypots (up) and non-honeypots (down). The index is just an arbitrary identification field for each case. The descriptions where simplified for readability, by ignoring the boolean variables with False values, and removing the balance_ prefix from the balance variable names.

4 Experiments

4.1 Additional Pre-Processing

Before starting any of the machine learning experiments, we need to take further data cleaning steps. Missing values in this setting are mostly a product of our own feature extraction. For any kind of transaction, the value aggregated features cannot be calculated when all the transactions from the contract contain errors, because no funds were actually moved. For normal transactions in particular, the difference of time or block number between consecutive normal transactions cannot be measured when the contract only has one normal transaction. The rest of the normal transaction features are always defined, because there is always at least one normal transaction (the creation, even if it contains errors). Nevertheless, for all these situations we found it reasonable to fill the missing values with zeros. However, a contract may not have any internal transactions and, consequently, all the related aggregated features are not necessarily defined. Even if those features could be filled with zeros, only 18% of the contracts have internal transactions. Hence, for the following experiments we discard the internal transaction features, and in exchange we add a new boolean feature that indicates the presence of internal transactions.

Not every contract is useful for our classification task. We know that contracts without byte code will never be honeypots because they cannot be executed, hence, we discard them. We also discard contracts without source code because:

  • They cannot be analyzed easily, so is very unlikely for them to contain traps.

  • We do not have labels for them.

  • The related features have missing values for them.

After the filtering, the amount of contracts is reduced to only 158,863 samples, but without losing any of the positive case. A more detailed description of the filters applied in this section can be found in the appendix.

Initially, from the 244 valid balance movement cases, we discard 103 of them for having no frequency across the entire dataset. After the filtering by byte code and source code, we discard an additional 68 for not occurring in the filtered dataset, keeping only 74 cases useful for describing transaction behavior. We also check the variance of each individual feature, but none of them present a variance very close to zero, thus we do not discard any further dataset columns.

Our last step is taking any “unbounded” numerical feature (the ones that are neither frequencies nor ratios), and apply min-max scaling to shrink them into the range . We use this technique to have every feature in the same range, to obtain afterwards a better understanding of feature importance.

4.2 Binary Classification

In this section, we train several machine learning models to distinguish honeypots (positive class) from regular contracts (negative class). As a baseline, we start with a simple logistic regression, which should be expected to yield poor results on such a large feature space where classes are not necessarily linearly separable. We continue with Decision Trees, a simple algorithm that tries to find the best way of recursively splitting the feature space until the classes are separated. It can be also interpreted as automatically deriving simple rules similar to those mentioned in section 

3. In our experiments, we prune the maximum depth of the trees to improve generalization. More details about both machine learning models can be found in [2]

. Furthermore, we experiment with Random Forests

[3]

, an ensemble of decision trees. Each tree is trained on different subsets of samples and features, and they vote to decide on a final class. In the same spirit, we try a second, more complex, ensemble of decision trees called XGBoost

[5].

We use k-fold cross-validation to ensure that the models generalize to unseen data. The data is split into separate folds, where one is selected for testing, while the algorithm is trained on all remaining folds. The procedure is repeated times so that each fold is used once for testing. In particular, we use stratified k-fold cross-validation, in which the folds are selected to preserve class proportions as well the fraction of honeypot types among malicious contracts.

Furthermore, we deal with an imbalanced classification problem: our dataset contains many more normal contracts than honeypots. This issue has a negative impact in the classification capabilities of machine learning algorithms. In order to train our models considering the class imbalance, we use class weights that depend on the proportion of positive and negative training samples.

For each fold, the metric we selected is the AUC-ROC (Area Under The Curve - Receiver Operating Characteristics), also known as AUROC (Area Under the Receiver Operating Characteristics). The ROC curve describes the TPR (True Positive Rate) against the FPR (False Positive Rate) for different thresholds over the classification probability. The area under that curve represents how well the model can distinguish the two classes. For any machine learning metric in general, the values obtained from the experiments on the training set indicate how good or how biased the model is, and the values measured on the test set indicate the power of generalization to previously unseen data.

Model Train Test
AUROC AUROC TN FP FN TP
Logistic Regression
Decision Tree
Random Forest
XGBoost
Table 4: Honeypot classification experiments for different machine learning models. We show the mean and standard deviation for every metric collected across all folds.

As we can see in Table 4, Logistic Regression performs poorly, but the other models are very promising. It seems that the increasing complexity of the ensemble models favor the FP in exchange of a few FN. Nevertheless, the balance between FP and FN can be adjusted to match different preferences by changing the classification threshold, which usually is defined at a 0.5 probability.

The tree based algorithms allow us to verify which features are the most important to them, depending on how close to the root the features appear in decision nodes. In Table 5 we see that the most important features for all models include some of the features we expected to be relevant in Section 3.

Feature Decision Tree Random Forest XGBoost
case_83_frequency 1st 3rd 1st
contract_num_source_code_lines 2nd 1st 3rd
normal_transaction_value_mean 4th 2nd 2nd
normal_transaction_gas_mean 3th 5th 4th
case_73_frequency 5th 17th 13th
Table 5: Feature importance ranking for the five most important features in common across different models. The fund flow case 83 corresponds to a deposit from the creator, and the fund flow case 73 corresponds to a withdraw from the creator.

4.3 Classification of Unseen Honeypot Types

We perform additional experiments to study how well the classification behaves when facing a completely new type of honeypot. In order to do so, we change the cross validation by leaving outside the training set only the contracts belonging to a specific honeypot type. This means that the remaining honeypot types are used for training alongside all the non-honeypot contracts. Furthermore, this time we measure accuracy for testing, because only TP and FP can be present.

Model Train Test
AUROC ACCURACY FN TP
Logistic Regression
Decision Tree
Random Forest
XGBoost
Table 6: Classification experiments of unseen honeypot types for different models. We show the mean and standard deviation for every metric collected across all folds.

We can see in Table 6 that the more complex models seem to be overfitting, which means, that they are not generalizing as well as the Decision Trees to completely unseen honeypot types. We are confident that it is possible to overcome this problem with some extra fine tuning of the learning algorithms.

5 Future Work

There are several ways in which this paper can be extended. We could explore the possibility of extracting features from the contract names based on the work presented in [11], by crossing the occurrence of words in the contract names with a list of relevant terms for the Ethereum community. Another alternative could be calculating a global state of account balances after the execution of all the transactions of a contract, instead of only checking fund flows per transaction.

We conceived the fund flow cases with the initial goal of analysing the temporal dependencies between each other. In other words, we wanted to study the behavior of contracts using sequences of discrete symbols, which is equivalent to work with character strings or list of words. However, as we found that most of the honeypots present very short sequences, the approach was not successful in finding useful patterns. In the end, we only used in this paper uni-gram frequencies, and discarded other alternatives like Markov Chains, n-gram frequencies, or Recurrent Neural Networks. Nevertheless, we plan to employ fund flow case sequences in future studies, to classify contracts (or even accounts) with a larger transaction history, into categories that do not involve honeypots.

6 Conclusion

In this paper, we presented a step by step methodology to obtain, process and analyze Ethereum contract transactions for the task of honeypot detection. We showed how assumptions and hypotheses about honeypot behavior can be contrasted with real data, and we proposed how the detection process can be automatized using machine learning, highlighting the case of unseen honeypot types.

We are aware that our methodology can detect honeypots only after they present a minimum amount activity. It can be argued that this is a disadvantage compared to the byte code analysis approach. However, we showed that our technique has the capability of detecting unseen honeypot types, which would be impossible to achieve using byte code analysis without manually crafting new detection rules. We think that both alternatives can complement each other, and that they should be used simultaneously to cover the weaknesses they posses.

References

  • [1] N. Atzei, M. Bartoletti, and T. Cimoli (2017) A Survey of Attacks on Ethereum Smart Contracts (SoK). In Proceedings of the 6th International Conference on Principles of Security and Trust - Volume 10204, pp. 164–186. External Links: ISBN 978-3-662-54454-9 Cited by: §1.
  • [2] C. M. Bishop (2006) Pattern recognition and machine learning. springer. Cited by: §4.2.
  • [3] L. Breiman (2001) Random forests. Machine learning 45 (1), pp. 5–32. Cited by: §4.2.
  • [4] R. D. Camino, R. State, L. Montero, and P. Valtchev (2017) Finding suspicious activities in financial transactions and distributed ledgers. In 2017 IEEE International Conference on Data Mining Workshops (ICDMW), pp. 787–796. Cited by: §1.
  • [5] T. Chen and C. Guestrin (2016) XGBoost: a scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, New York, NY, USA, pp. 785–794. External Links: ISBN 978-1-4503-4232-2, Link, Document Cited by: §4.2.
  • [6] C. Ferreira Torres, M. Steichen, et al. (2019) The art of the scam: demystifying honeypots in ethereum smart contracts. In USENIX Security Symposium, Santa Clara, 14-16 August 2019, Cited by: §1, §2, 1st item, 2nd item, §3.1.
  • [7] D. Luca and B. Mueller (2019-08) The Ether Wars: Exploits, counter-exploits and honeypots on Ethereum. Note: https://infocondb.org/con/def-con/def-con-27/the-ether-wars-exploits-counter-exploits-and-honeypots-on-ethereum Cited by: §1.
  • [8] L. Luu, D. Chu, H. Olickel, P. Saxena, and A. Hobor (2016) Making smart contracts smarter. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, CCS ’16, New York, NY, USA, pp. 254–269. External Links: ISBN 978-1-4503-4139-4, Document Cited by: §1.
  • [9] S. McNally, J. Roche, and S. Caton (2018) Predicting the price of bitcoin using machine learning. In 2018 26th Euromicro International Conference on Parallel, Distributed and Network-based Processing (PDP), pp. 339–343. Cited by: §1.
  • [10] S. Nakamoto (2009-03) Bitcoin: a peer-to-peer electronic cash system. Cryptography Mailing list at https://metzdowd.com, pp. . Cited by: §1.
  • [11] R. Norvill, B. B. F. Pontiveros, R. State, I. Awan, and A. Cullen (2017) Automated labeling of unknown contracts in ethereum. In 2017 26th International Conference on Computer Communication and Networks (ICCCN), pp. 1–6. Cited by: §5.
  • [12] S. Petrov (2017-11) Another parity wallet hack explained. Note: https://medium.com/@Pr0Ger/another-parity-wallet-hack-explained-847ca46a2e1c Cited by: §1.
  • [13] J. Sanjuas (2018-12) An analysis of a couple ethereum honeypot contracts. Note: https://medium.com/coinmonks/an-analysis-of-a-couple-ethereum-honeypot-contracts-5c07c95b0a8d Cited by: §1.
  • [14] A. Sherbachev (2018-12) Hacking the hackers: honeypots on ethereum network. Note: https://hackernoon.com/hacking-the-hackers-honeypots-on-ethereum-network-5baa35a13577 Cited by: §1.
  • [15] D. Siegel (2016-06) Understanding the dao attack. Note: https://www.coindesk.com/understanding-dao-hack-journalists/ Cited by: §1.
  • [16] C. F. Torres, J. Schütte, and R. State (2018) Osiris: hunting for integer bugs in ethereum smart contracts. In Proceedings of the 34th Annual Computer Security Applications Conference, ACSAC ’18, New York, NY, USA, pp. 664–676. External Links: ISBN 978-1-4503-6569-7, Document Cited by: §1.
  • [17] P. Tsankov, A. Dan, D. Drachsler-Cohen, A. Gervais, F. Buenzli, and M. Vechev (2018) Securify: practical security analysis of smart contracts. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pp. 67–82. Cited by: §1.
  • [18] G. Wood (2014) Ethereum: a secure decentralised generalised transaction ledger. Ethereum Project Yellow Paper 151, pp. 1–32. Cited by: §1.
  • [19] G. Wood (2019-09) Solidity 0.5.11 documentation. Note: https://solidity.readthedocs.io/en/v0.5.11/ Cited by: §1.

Appendix 0.A Examples of Handcrafted Rules

contract_is_honeypot False True
15 < contract_num_source_code_lines < 200
False 77641 0
True 80927 295
Table 7: Handcrafted rule for contract_num_source_code_lines.
contract_is_honeypot False True
normal_transaction_count < 40
False 13562 0
True 145006 295
Table 8: Handcrafted rule for normal_transaction_count.
contract_is_honeypot False True
normal_transaction_value_mean < 2
False 3515 0
True 155053 295
Table 9: Handcrafted rule for normal_transaction_value_mean (in ether).

Appendix 0.B Frequencies of Categorical Variables

Minor Version Honeypot Total
Yes No
0.4 294 165266 165560
0.3 1 50787 50788
0.2 0 126 126
0.1 0 29 29
Table 10: Contract count for each compiler minor version.
Version Honeypot Total
Yes No
0.4.19+commit.c4cbbb05 124 9950 9950
0.4.20+commit.3155dd80 62 2937 2937
0.4.24+commit.e67f0147 26 90428 90428
0.4.18+commit.9cf6e910 19 7187 7187
0.4.17+commit.bdeb9e52 12 751 751
Label Count
Hidden State Update 42
Uninitialised Struct 27
Straw Man Contract 25
Hidden Transfer 13
Balance Disorder 10
Inheritance Disorder 5
Type Deduction Overflow 2
Table 11: Left: Contract count for the five most frequent compiler versions for honeypots. Right: Honeypot count by type for compiler version 0.4.19+commit.c4cbbb05.
contract_is_honeypot False True
contract_has_library
False 2013080 295
True 6059 0
Table 12: Presence of library crossed with honeypot binary label.

Appendix 0.C Frequencies of Fund Flow Special Cases

sender=creator
error=False
balance_creator=negative
balance_contract=positive
contract_is_honeypot False True
deposit_creator_frequency > 0
False 150212 29
True 8356 266
Table 13: Contracts with deposits from the creator. Left: definition of the cases. Right: amount of contracts by case frequency crossed with the honeypot binary label.
sender=other
error=False
balance_sender=negative
balance_contract=positive
contract_is_honeypot False True
deposit_other_frequency > 0
False 145816 192
True 12752 103
Table 14: Contracts with deposits from other. Left: definition of the cases. Right: amount of contracts by case frequency crossed with the honeypot binary label.
sender=creator
error=False
balance_creator=positive
balance_contract=negative
contract_is_honeypot False True
withdraw_creator_frequency > 0
False 152601 187
True 5967 108
Table 15: Contracts with withdraws from the creator. Left: definition of the cases. Right: amount of contracts by case frequency crossed with the honeypot binary label.
sender=other
error=False
balance_sender=positive
balance_contract=negative
contract_is_honeypot False True
withdraw_other_frequency > 0
False 155439 245
True 3129 50
Table 16: Contracts with withdraws from other. Left: definition of the cases. Right: amount of contracts by case frequency crossed with the honeypot binary label.

Appendix 0.D Additional Pre-Processing Filters

contract_is_honeypot False True
internal_transaction_count > 0
False 1651206 138
True 367933 157
Table 17: Number of internal transactions crossed with honeypot binary label.
contract_is_honeypot False True
contract_has_byte_code
False 57664 0
True 1961475 295
contract_is_honeypot False True
contract_has_source_code
False 1802931 0
True 216208 295
Table 18: Byte code and source code presence crossed with honeypot binary label.