Transaction Lifecycle

Learn about the JavaScript API used for blockchain operations, best practices, and the transaction lifecycle so you can build a scalable system.

Transaction Lifecycle

I have implemented dozens of different kinds of payment methods like PayPal, DotPay, or even credit cards. Every integration seemed a little bit different, but after I created my very first payment integration with the blockchain technology, I realized that it was something new for me; that I cannot just send the user in my application to the third party service and wait for a response with confirmation or rejected status. Using the blockchain makes me responsible for everything I may or may not have been aware of during the payment process.

In this part, I would like to present one way in which you can integrate your application, exchange system, ICO, or e-commerce shop and give your users the ability to pay with, buy or donate Bitcoins. I will start with a general overview of the transaction lifecycle and then implement this process with the JavaScript application.

Why?

The main question is always "why". Why should you integrate your application with Bitcoin payments or even consider integrating with the Blockchain and avoid using ready-to-use services? There are many platforms that provide you with a workflow similar to PayPal’s, but they all have something in common — fees.

Implementing your own solution lets you avoid paying a few percent of fees to other companies (the buyer still needs to pay the Blockchain fees, but she has to do it anyway, in every solution). If you want to keep this money in your pocket, this article is for you.

Goal

After this part, you will know how to integrate the Bitcoin payment method with your application. The purpose you use it for is up to you. You can make a system to exchange crypto and fiat currency, make an investment platform or open an ICO; the possibilities are endless, but all of them include you accepting Bitcoin payments.

For our purposes, we will assume that we have an e-commerce business and we want to give the clients the option to pay with Bitcoins. This is the easiest example that I can imagine because it does not require us to discuss any additional business logic. During the order process, the user can choose between different payment processes including Bitcoin. We will focus only on this one.

Transaction Lifecycle

In the usual payment system like PayPal, your shop redirects the user to a payment processor’s website and sends all required information to process the transaction, like the amount, recipient, and a transaction title which often refers to the order ID.

The user is asked to choose a bank or enter a credit card number to finish the transaction, and the third party system takes care of the validation process, etc.

In the next step, the application just waits for a callback response with the information that the payment is accepted, rejected, or pending. If it is accepted, you can be pretty confident that the money is yours. The advantage of this solution is that the responsibility for the process is taken by the third party. If they give your application the information that the payment has been successful, but it was not, they need to cover the difference. On the other hand, all payment systems I was working with always had a way to withdraw money.

The Blockchain and traditional Internet payment methods are different, and all differences start at the very beginning. Any payment system that works with the Blockchain can deliver the same API ex.bitpay.com. But minus the fees.

User Flow

Let’s start from the beginning when the user is on the page where he chooses the payment method and picks Bitcoin. We cannot redirect the user to any external service because we do not know where he keeps his wallet. He could be managing his wallet on his own computer like we are. Moreover, because all payments have to be anonymous in the Blockchain, the transaction cannot provide or store any information like the sender name, receiver name, or the title. This forces us to take a different approach and recognize the payer not with the order number or ID but rather with the final address in Bitcoin.

The transaction can be presented as a short message that says: "Some amount of BTC is sent from address A to address B and signed by the owner of address A and identified by a hash number C."

Because the Blockchain is decentralized, the payment needs to be done manually, by the user like with a traditional fiat transfer from the bank account. But, the transfer should be completed in approximately 10 to 30 minutes, depending on the fee which is defined by the payer. The system presents a price without this fee, but if the funds should be transferred to the cold wallet, which may happen in ICO-projects or with bigger transactions, you need to be aware that there will be another fee which needs to be paid with the new transfer.

Then, the application needs to synchronize with the Bitcoin Blockchain and check if the transaction is included in the last block. This is the very first confirmation that it has. The last step is to gather as many confirmations from the Blockchain as we need.

The API Client Implementation

I will show you how to connect your JavaScript application with the Blockchain and how to process the full transaction lifecycle from the very beginning, when the user requests Bitcoin payment, to the very end, when the transfer is confirmed. The library that we are going to use is called bitcore-lib and is published at https://github.com/bitpay/bitcore-lib by bitpay.

Prerequisites: bitcore-lib and bitcore-explorers

We will use the node to support the development and the npm or bower package managers, depend on the operating system. At this point, I assume that you are already familiar with those technologies. If not, I strongly recommend one of the Node.js courses that are recommended by X-Team and can be found at https://x-team.com/nodejs-resources/.

Bitcore-lib is a package that allows you to connect to the Bitcoin node and call the API endpoints. Installation is handled by the npm package manager for linux/windows machines:

npm install bitcore-lib

or, bower for mac users:

bower install bitcore-lib

Bitcore-explorers is responsible for making bitcoin transactions and can be installed with:

npm install bitcore-explorers

and for the mac users:

bower install bitcore-explorers




Codebase — Connect To the Bitcoin Node

When the required packages are installed, we can start to write the code. We will start by creating a connection with the Bitcoin node and its API.

First of all, we need to generate a WIF hash key (Wallet Import Format). The WIF hash number is a way of encoding a private ECDSA key to make it easier to copy. To do it, we can use the following script:

var bitcore = require('bitcore-lib')
var Insight = require('bitcore-explorers').Insight

var privateKey = bitcore.PrivateKey('testnet').toWIF()
console.log(privateKey)

The presented WIF number should be saved in the configuration. This is a private key that we will use to authorize the transaction. Now, we will use it to connect to the wallet:

var bitcore = require('bitcore-lib');
var Insight = require('bitcore-explorers').Insight;

var privateWIFkey = 'cUw66adsvt6mrpkzqtPueZ6uQQsPURGcc5iK34DUnGqxsKt3FcKh';

var decodedPrivateKey = bitcore.PrivateKey.fromWIF(privateWIFkey)
var senderBitcoinAddress = decodedPrivateKey.toAddress()

console.log(senderBitcoinAddress)

And finally, we have generated a new address for the wallet that we will use for our application.

Step 1 — Request Payment

When the user wants to pay with Bitcoin, the system should present him an address that he should send the funds to. The best approach is to present the address and the QR code, because many people use mobile applications which can scan the QR codes.

The new address should be generated for each user or even for each order, because all the information that the transaction payload contains about the sender in the Blockchain is limited only to the address that the funds are transferred from, amount and timestamp. Because this information is fully anonymous, the sender cannot be identified by the sender address. But, the Blockchain does not limit the number of addresses assigned to your wallet, so for every new buyer on your website, new address should be generated. This is the application’s responsibility to save this address and create a relationship in the database between it and the order.

Example

  1. A user U1 places an order, the system generates a special address for him — A1 — and presents it.
  2. A user U2 places an order, the system generates a special address for him — A2 — and presents it.
  3. And so on…

If any funds are transferred and confirmed to the address A1, we will know that this is a payment which should be assigned to user U1, or somebody else wants to pay for him.

Generate a New Wallet

Before we generate a new address for the user, we need to create a totally new Wallet for our application. We can do it from the code level, by using the REST API which is delivered with BWS at the http address http://localhost:3232/bws/api.

var Client = require('bitcore-wallet-client');

var fs = require('fs');
var BWS_INSTANCE_URL = 'http://localhost:3232/bws/api'

var client = new Client({
  baseUrl: BWS_INSTANCE_URL,
  verbose: false,
});

  client.createWallet("My Wallet", "MyApplication", 2, 2, {network: 'testnet'}, function(err, secret) {
  if (err) {
    console.log('error: ',err);
    return
  };
  // Handle err
  console.log('Wallet Created. Share this secret with your copayers: ' + secret);
  fs.writeFileSync(wallet.dat', client.export());
});

We will work on this piece of code from now on. I create a Bitcoin-wallet-client object in line 6, which connects to the BWS API. Then, we can execute any call to this API using this client.

The createWallet method accepts a couple of parameters — a wallet name, copayer name, two random numbers, advanced options (in our case we specify that we will use TestNet), and a callback function.

As a result, we will receive a secret hash which we will use to operate our wallet. Also important — we can dump the wallet’s information into one file, which is done in the 18th line. Information about the type of cryptocurrency, network, private and public keys, etc. will be saved in JSON format.

This step can and should be executed only once, and the secret key can be copied to the configuration file in your application or kept in another safe place.

Generate a New Address

Now it is time to generate new random and unique address for our user. We will need to save this address and connect it with a user’s account to be able to recognize if the payment is dedicated to a specific order and which user made the transfer. As I mentioned before, Bitcoin does not allow adding additional fields or information to the transaction. Thus we can recognize the payer only by this unique address. This address will be connected with our Wallet, which we created a moment ago, but this connection will not be visible to anybody else.

var Client = require('bitcore-wallet-client');

var fs = require('fs');
var BWS_INSTANCE_URL = 'http://localhost:3232/bws/api'

//secret saved from the previous exercise
var secret = 'RMAYDCHnV7RoF9FPs8c7qVL3Eny64yFAFG57suLnwFdCjKHQ9uU5efYbDuHxgMEoWtGQtkwRQPTbtc'

var client = new Client({
 baseUrl: BWS_INSTANCE_URL,
 verbose: false,
});

client.joinWallet(secret, "MyApplication", {}, function(err, wallet) {
  if (err) {
    console.log('error: ', err);
    return
  };

  console.log('Joined ' + wallet.name + '!');
  fs.writeFileSync(address.dat', client.export());

  client.openWallet(function(err, ret) {
    if (err) {
      console.log('error: ', err);
      return
    };
    if (ret.wallet.status == 'complete') {
      client.createAddress({}, function(err,addr){
        if (err) {
          console.log('error: ', err);
          return;
        };

        //new address
        console.log('Return:', addr)
      });
    }
  });
});

As in the previous example, I connect to the REST API by creating a Client object. Creating an address is an action that saves something to the blockchain, so we need to unblock our wallet first. The same happens when a transaction is sent.

In the 14th line, we call the joinWallet method, which authenticates the application with the secret key. That is the same key we generated in the previous example.

Another method, openWallet, in line 24, opens this Wallet, and from now on, we will be able to write to the blockchain. As the only argument, it accepts the callback function where we will create a new Address.

Line 30 is responsible for creating the new address. Thus, we will receive an Address object with the field address, which should be presented to the user and saved in the database for future reference. We couldn’t call this method before because it would throw the error that we are not joined to any Wallet, or the Wallet is not opened.

Step 2 - Payment (user site)

The second step is totally on the user’s side, and we cannot automate that process easily. We can just improve the way how or when we present payment information. At this point, the user scans the QR code or copies the address to his wallet and makes the payment.

When that happens, the payment is published to the Blockchain but has not been included in any block yet. It is on the pending list, and we can get information about its status. Until the block with this transaction is mined, the payment is not secure, so we cannot trust it. But, we can give the user the feedback that we noticed the payment.

To do it, we can call an API method that delivers a full list of all pending transactions. Once, the transaction is mined, it will be moved from this list to the block.

At this point, we can check if any new pending transaction for the address generated for our user has appeared. When the Steam platform allowed Bitcoin payments, they did the same. The first step was to present a unique address for the user, and if she did not close the page and transferred coins to this address, the Steam platform displayed the current payment status as Pending.

To approach the same behavior, we can list all transactions for the address, and before being mined, they will have 0 confirmations. At this point, the transaction is Pending and cannot be treated as safe. However, we can always inform the user that we immediately noticed the payment.

We will use the ‘bitcore-explorers’ library and Insights API. This pair of tools allows us to review the blockchain without having a wallet. This is safe. I strongly recommend you connect and open the Wallet only and only when it is necessary.

var explorers = require('bitcore-explorers');
var insight = new explorers.Insight('http://localhost:3001/insight-api/', 'testnet');

const exampleAddress = 'myqfgu5fZKgPBiqv7gghBPETc4yEnCdocU'

insight.requestGet('txs/?address=' + exampleAddress, function(err, response) {
  console.log(response.body)
});

In this piece of code, I used an address where I transferred some amount of coins, but they have not been mined yet. Therefore, I can see that the txs method returns a list of transactions with only one element which has no confirmations.

Step 3 - Get the Transaction From the First Block

There are two ways of getting information about the transaction now — the first one is to save the transaction id from the pending list and track its status with a cronjob or another system, or listen to the last published block and get a list of transactions included in it. Because the second approach requires us to use new API methods, I’ll stick to it. It does not mean it is better, just different.

Get the Latest Block

This is one of the approaches for getting information about the current payment status. If you already got the transaction ID, then you can omit this step and go directly to step four.

But sometimes, it happens that you are not able to observe all addresses. When you run a really successful business, you probably open hundreds of addresses every day, and checking pending transactions for all of them may be a time- and resource-consuming process. That's why I prefer to observe the latest block in the blockchain. The block includes a list of mined transactions. We will review those transactions and compare receivers with our saved addresses. If we find any of the addresses in our database, we have a match; we can save the transaction ID and just wait for confirmations.

To get information about a block, we can use the block method in the Insight’s API. This will return a full set of data including a list of transactions.

var explorers = require('bitcore-explorers');
var insight = new explorers.Insight('http://localhost:3001/insight-api/', 'testnet');

const exampleBlock = '00000000000007435e1e4c72e17ae47fbdd63132950468d3dbd5d732e7fc32d8'

insight.requestGet('block/' + exampleBlock, function(err, response) {
  console.log(response.body)
});

Unfortunately, the list of transactions includes only the IDs, so we need to go through all those IDs and get the transaction data from another method called tx. I prepared the code:

var explorers = require('bitcore-explorers');
var insight = new explorers.Insight('http://localhost:3001/insight-api/', 'testnet');

const exampleBlock = '0000000000000592e07543f0da2c21a93095827c4dc03f178da1557057b9795f'

insight.requestGet('block/' + exampleBlock, function(err, response) {
  JSON.parse(response.body).tx.forEach((transaction) => {
    insight.requestGet('tx/' + transaction, function(err, response) {
      const transactionObject = JSON.parse(response.body)
      console.log(transactionObject)
    });
  })
});

As you can see, the latest block has two transactions, but the receivers are pretty well hidden. This is because they are stored in the vout section. You can notice that the response includes two sections — vin (value in), and vout (value out). Those are arrays because Bitcoin does not make a transfer from one address to another, but from a bunch of addresses in the same wallet to whatever is specified.

TIP: You can create your own transaction and send it to how many receivers you want to. To do so, you need to create a raw transaction, sign it with your private key, and publish to the blockchain. This operation requires three different steps, or four if you want to verify the generated hash. In this case, you can compose your own transaction. You need to specify which addresses you want to use to send money from, and where exactly it will be sent — you can transfer funds to one or many addresses. The funny thing here is that there could be 5 recipient addresses, all of them the same. Bitcoin cannot stop you.

The code below lists all receiver addresses with the amount of funds that were transferred to it. This one list is included into one specific transaction:

var explorers = require('bitcore-explorers');
var insight = new explorers.Insight('http://localhost:3001/insight-api/', 'testnet');

const exampleBlock = '0000000000000592e07543f0da2c21a93095827c4dc03f178da1557057b9795f'

insight.requestGet('block/' + exampleBlock, function(err, response) {
  JSON.parse(response.body).tx.forEach((transaction) => {
    insight.requestGet('tx/' + transaction, function(err, response) {
      const transactionObject = JSON.parse(response.body)
      transactionObject.vout.forEach((vout) => {
        console.log('TRANSFER ' + vout.value + ' BTC, to ' + vout.scriptPubKey.addresses)
      })
    });
  })
});

Get the Block Hash

Now we can get information about the block, but we need to know the block hash, which is pretty hard to guess. That is why blocks also have order numbers. My approach is to save the number of the last block that I checked to the database and increase it every time I run the cronjob. In this case, I can check transactions in many blocks, one after another, not only in the last one.

The API provides a method that returns a block hash for a specific order number. It is called block-index:

var explorers = require('bitcore-explorers');
var insight = new explorers.Insight('http://localhost:3001/insight-api/', 'testnet');

const exampleBlock = 1261415

insight.requestGet('block-index/' + exampleBlock, function(err, response) {
  console.log('BLOCK HASH: ' + JSON.parse(response.body).blockHash)
});

After the transaction is mined in the block, it has the very first confirmation and can be treated with limited trust. I call it limited trust because there is a slight chance that somebody can publish a block that should not be accepted by the Blockchain, but somehow our node accepted it. Some of those blocks may be dropped when the next is mined. Those blocks which have not been accepted after the next block was mined are called orphaned blocks. This is why one confirmation may not be enough to trust the payment.

Step 4 — Confirm the Transaction

The last step is to gather as many confirmations from the blockchain as we need to mark the transaction as secured or trusted. After the transaction is mined for the first time and added to the block, every next block on the stack will add one confirmation to the transaction. The confirmation number is nothing more than a piece of information about how many blocks have been mined after the block with this transaction.

The Blockchain algorithm makes the mining process extremely difficult and mining few blocks in a row by the same miner is almost impossible or impossible. The level of trust can be considered as a good balance between the waiting time for the final confirmation and the certainty level. Because each block is mined in approximately 10 minutes, the more confirmations we need to trust the transaction the more time we need to wait for it. I prefer to use the following thresholds:

  • 1 confirmation (10 minutes) for small transactions, max 10-50 USD, frequent transactions or from trusted users. This limits the risk, and I keep the level of risk to a low level, where nobody will be seriously hurt if the Bitcoins are gone. This threshold can be used for food or game ordering (The Steam platform used a single-confirmation algorithm with BitPay).
  • 3 confirmations (30 minutes) for usual transactions with bigger value. It is quite unusual that one miner will confirm 3 blocks in a row, but that may happen. This kind of threshold is often used with the internet exchange systems that allow purchasing more than 2000 USD.
  • 6 confirmations (60 minutes) for big transactions, mostly used for investing process in ICO or other Blockchain projects, where the users operate with several tens of thousands of dollars. In this case, time is not as important as security and confidence are.

Get Transaction Data

To get information about received confirmation number, we need to know the transaction ID. The tx method is meant to return complete information about the transaction:

var explorers = require('bitcore-explorers');
var insight = new explorers.Insight('http://localhost:3001/insight-api/', 'testnet');

const transactionId  = '84ab8aa62753a29f11adc15b6415c6fc58799e7b6f74b80bc31db2a72f6dd105'

insight.requestGet('tx/' + transactionId, function(err, response) {
  console.log('number of confirmations for ' + transactionId + ' is ' + JSON.parse(response.body).confirmations)
});

After the number of confirmations meets our expectations, we can mark the transaction as done.

Step 5 (extra) — Secure Funds

I did not specify this step at the beginning of this chapter because it is not necessary and the process of the payment confirmation ends with the previous step. However, in some cases, moving funds to the cold wallet or to some other wallet that is not connected to the application may be a good idea for the projects that operate with bigger numbers, like ICOs, investing platforms, or loans systems.

When the investor transfers 200 000 USD to our system in Bitcoins, we do not want to keep it unsecured for too long. Why do I claim that that money is unsecured when everybody is talking about the Blockchain's high security? My previous projects and experience taught me that somebody will always try to break your application. It does not matter if you have a billion dollar wallet or you create a Facebook contest with a 50 USD prize for fuel. If there is any money at stake, there will always be some attempt to take it.

People like to think that a hacker’s job is strictly related to hacking the software and breaking all technical barriers. But it is not. If the code is written right, it is the hardest part to break, thus the attack will be launched somewhere else. And the weakest point is the human factor.

Send a Transfer

After we have received the funds, but before we have collected all the confirmations, we can transfer the funds to a secured cold wallet or at least to some specific address, which is not connected directly to the application or whose secret hash is not saved in the database or the configuration file.

I must admit, sending a transfer with bitcore-wallet-client is a little bit overcomplicated:

var Client = require('bitcore-wallet-client')

var fs = require('fs')
const BWS_INSTANCE_URL = 'http://localhost:3232/bws/api'

//secret saved from the previous exercise
const secret = 'RMAYDCHnV7RoF9FPs8c7qVL3Eny64yFAFG57suLnwFdCjKHQ9uU5efYbDuHxgMEoWtGQtkwRQPTbtc'
const receiverAddress = ''
const amountOfBtc = 1

var client = new Client({
  baseUrl: BWS_INSTANCE_URL,
  verbose: false,
})

client.joinWallet(secret, "MyApplication", {}, function (err, wallet) {
  client.openWallet(function (err, ret) {
    if (ret.wallet.status == 'complete') {
      client.createTxProposal(
        {
          outputs: [{
            toAddress: receiverAddress,
            amount: amountOfBtc,
          }],
          feePerKb: 10000,
          excludeUnconfirmedUtxos: false,
        },
        function (err, tx) {
          wallet.publishTxProposal({txp: tx}, function (err) {
            wallet.getTxProposals({}, function (err, txps) {
              var txp = txps[0]
              wallet.signTxProposal(txp, function (err, txs){
                wallet.broadcastTxProposal(txs, displayend)
              })
            })
          })
        }
      )
    }
  })
})

I omitted all the parts with catching errors so that you can find only the success line in the code. Let’s go from the top and discuss what is going on:

  • joinWallet — we need to connect with the Wallet. This method authorizes us to use it, but it is still not opened.
  • openWallet — from now on, we will be able to use the Wallet. We need to open it to send the transaction.
  • createTxProposal — the blockchain keeps the security at the highest level, so we do not create a transaction, but a transaction proposal. This method returns an object from the specified array. We do not need to enter all the information, like a full list of addresses that the funds are sent from. This will be automatically generated.
  • publishTxProposal — the transaction proposal is published to the blockchain, but it cannot be mined until we sign it with the private key.
  • signTxProposal — we do not need to provide the private key because we have already opened the wallet, so the software knows that we operate on this specific wallet, but we need to pass a specific list of transaction IDs that we want to sign.
  • broadcastTxProposal — after we call this method, the transaction is broadcasted to the internet and can finally be mined.

TIP: If you need to keep some funds for returns, I recommend you check the balance and create an additional address that the application can manage. However, the main part of the money should be secured.

Conclusion

At this point, you know exactly how to handle Bitcoin payments, how to confirm them and secure the funds. Now you can create a payments system with the Bitcoin node. In the next chapter, I will present another Blockchain cryptocurrency — Ethereum. It has some differences in comparison to Bitcoin.

04 Milestones

SCALE YOUR DEVELOPMENT TEAM

We help you execute projects by providing trusted Blockchain developers who can join your team and immediately start delivering high-quality code.

Hire Blockchain Developers