Writing Solidity Unit Tests for Testing Assert(), Require() and Revert() Conditions Using Truffle

Writing a share of unit tests using Javascript in Truffle, it started to nag at me that I had not been testing whether or not my contracts were “throwing” at conditions in which I did not want functions to execute. As Ron Jeffries pointed out, “I am pretty sure there is a difference between “this has not been proven” and “this is false.” So, I set on a mission to learn how test “throws”, or now asserts, requires, and reverts, using Solidity unit tests. There is a well laid out Truffle tutorial for testing throws, and I have to admit the first time I approached it, it was over my head, and I put it to the side. Revisiting the material again, well, practice pays off, and it’s one more very powerful tool to add the arsenal. I haven’t seen a ton out on this subject yet, so I figured it couldn’t hurt to write supplemental piece on how I learned my way through these Solidity tests. If you feel thrown off, put it down, learn some more advanced methods in Solidity contract design and come back — because what contract couldn’t use a little more testing?

Review of Assert(), Require(), and Revert()

First, let’s go over how assert(), require(), and revert() work as throw has been deprecated after Solidity Compiler Version 0.4.13 — at the time of writing we are currently on version 0.4.19.

In writing functions in Solidity for your smart contracts, one of your first lines of defense against things such as malicious parties, bad input data, and wasted gas expenses is through the use assert(), require() and revert(). Much of the below is from the documents, but for a quick overview…

Assert() should only be used to test for internal errors and to check invariants. Recall that an invariant is something that is assumed to be true during execution. A great example of the use assert() is OpenZeppelin’s SafeMath.sol . Here, in the add() function assert() is used to confirm that any summed integers do not overflow. This add() function is used internally and in a “normal” reality this condition should never be reach as we assume the addition of two numbers should result in their respective sum. If the assert() condition evaluates to false, then all gas is consumed and state conditions are rolled back — like it’s predecessor throw.

Require() should be used to ensure valid conditions (inputs) or contract state variables are met or to validate return calls to external contracts. I think of require() as guarding against transactions that you would not want to execute although the function would allow it if require() was omitted. For example, you might require() that the transaction be initiated from a certain address or require() that a transaction only be allowed to execute prior to a certain time. From a beginner perspective, one of the most important lessons in learning Solidity is how and when to use require() as you shape the appropriate and secure user inputs. Unlike throw, with require(), any remaining gas is returned to the sender, yet similar to throw, state conditions are rolled back as well.

Revert() is similar to require() but can be used to flag an error. I am looking forward to the days of revert() as we will be able to include error messages when the conditions are met. More information here.

Writing the Tests

Let’s take a simple example of a storage contract as below. Here, we start out storing the number 24. Using the function storeNum(), we only want to store numbers above 10.

To get our bearings straight, let’s go over how a Solidity test is put together.

First, we want to import the below:

import "truffle/Assert.sol"
import "truffle/DeployedAddresses.sol"
import "../contracts/MyContract.sol"

Where are Assert.sol and DeployedAddresses.sol? When I first saw these, I thought I had to dig through my files in Truffle to find them because I was sure that, it couldn’t be so easy as for me to import them via simply “truffle/xyz.sol.” However, it is that easy. From Tim Coulter’s, creator of Truffle, answer here, you can see that “DeployedAddresses.sol” is created dynamically at the time of deployment . From what I can tell, “Assert.sol” does exist within your Truffle does exist deep within your Truffle files, but you can pull it into your testing this way as well. And what exactly are these two files? Assert.sol is a library contract that holds functions for all Solidity tests, such as isFalse() or isEqual(). These are largely similar to the ChaiJS assertions in your Javascript tests, but they are executed through a Solidity contract instead. You can see the full details of available tests in Assert.sol here. Again looking back to Coulter’s answer, “DeployedAddresses.sol” is a contract that maintains your contract’s deployments and allows you to access them.

Moving forward, it is important to recognize that your Solidity tests must begin with “Test” — uppercase “T”, to identify it in Truffle as a test. Likewise, all Solidity test functions must also begin with “test.”

Let’s begin with an easy Solidity test case:

When we run truffle test, we see the test passed. As expected, our initial value of mynumber is 24 as laid out in MyContract.sol. And Ganache was busy:

Eight transactions. What all was going on? Digging in, we can see block 1 was a contract creation. Block 2 was a contract call. Block 3 another contract creation. Block 4 another contract call. Block 5 a contract creation. Block 6 another contract creation. Block 7 even another contract creation. And block 8 a contract call. Overall, that’s 5 total contracts created: let’s count we should have TestMyContract.sol, Assert.sol, DeployedAddresses.sol, and MyContract.sol — so what is the extra contract? Pay attention to your console, and you can see that Solidity unit tests make use of your default Migrations.sol for contract deployment as well. That is our “extra” contract.

Now that we have a Solidity test completed, let’s see how we can test our ultimate goal — assert(), require(), and revert(). Truffle tutorials suggest using a proxy contract “to wrap the contract call to a raw call and return whether it succeeded or not.” What does that exactly mean? First, let’s take a look at some full code for testing throws.

You can see from the above that we now have a ThrowProxy contract included in our TestMyContract.sol. What is this doing? Going step by step is easiest, first upon creation, we can see that ThrowProxy is storing the address of it’s target. The target of which is the contract that we would like to test for throws. This coincides with the first two lines of testTheThrow() function as you can see we first create a new MyContract and then feed that address into ThrowProxy.

Alright, so next you can see that the in fall back function ofThrowProxy that the message data is stored as datain the ThrowProxy contract. Recall that the message data is the complete call data of a contract call — and what does that mean? The identifying function signature and the inputs compressed into 32 bytes. A function signature is the first four bytes of the hash of the canonical signature string. In our contract, MyContract, we have a function storeNum with a uint mynum as a input, it’s canonical signature string would be storeNum(uint256), and if it took a string input as well then storeNum(uint256, string). To get the function signature, you can use sha3() or keccak256() to get the hash of which you take the first four bytes for the function signature. Further, the message data combines the input data with the function signature as well.

After the fallback function, we can see that execute() uses the address of the original contract, target, in this case, MyContract to execute .call() using the message data collected from the original fallback function. This is quite brilliant as now execute() returns a boolean true/false if target.call(data) was successfully executed or not — recall that .call() returns a boolean. It does not provide any other returns of function execution. Furthermore, this means if a throw condition was encountered viatarget.call(data) , then the contract would not be able to execute and would return false — instead of breaking our tests as in JavaScript, we can simply carry on to the next test.

Returning the example, in testTheThrow() we see on line 10 that we call MyContract(address(throwProxy)).storeNum(7) , so we are sending a transaction to our throwProxy but throwProxy does not have a function called storeNum()? In this case, throwProxy will execute the fallback function, thus storing the message data for a call using the storeNum() function! On the following line, when we run throwProxy.execute.gas(200000)(), we are in fact calling our original storeNum() on MyContract. After storing the true/false result as r, we can use a function from our Assert.sol library to test whether or not a throw condition was encountered. In our example, since our contract is not supposed to store numbers less than 10, a throw condition will be encountered. So, our result r should be false, and the test will pass — allowing us to test our next conditions.

Now, we have a basis for testing assert(), require() and revert() in a Truffle test environment. The next test testNoThrow() works in exactly the same way as our testTheThrow(), preparing our call data throw the proxy contract and using call() to see whether or not a function completed execution. There is a lot more to Solidity tests to be expanded upon than just testing for throws — they even allow for Before/After Hooks. In a larger test set, we would certainly want to separate out our throwProxy into its own individual contract, and use more aggressive methods to test our code.

I published all code from my examples here if you would like to run the complete set with Truffle.

Thanks for reading. Any corrections needed? Let me know in the comments or at kscarbrough1@gmail.com.