Category Archives: CryptoCurrency

Encryption made easy

Encryption can be a bit of a black box for people working in operations and engineering, but when it’s broken down into basic mathematics, it’s a lot more accessible.

I’ve written a collection of Python scripts that demonstrate the most common implementations of encryption: Diffie-Hellman Key Exchange, RSA Public Private Key Encryption, and Elliptic Curve Encryption.

The anatomy of a Bitcoin transaction

Anatomy of a Bitcoin transaction

Anatomy of a Bitcoin transaction

1. Origination

Every new Bitcoin transaction starts with an existing Bitcoin transaction.

For the purposes of this explanation, let’s assume that a Wallet means a single Public Private Key Pair. The Wallet will have previously scanned the blockchain and know that a particular transaction in a particular block has an output that contains an amount of Bitcoin that has been sent to the Wallet’s address (which is derived from the Wallet’s public key).

The Wallet can then being to construct a new transaction based on the output of the existing transaction.

2. Inputs

The Wallet first creates a list of inputs. In our case, we will assume only 1 input, but there can be more than one.

This input refers to the ID of the existing transaction, and the sequence number of the output in that transaction (there can be multiple outputs in a transaction too).

The input also contains an unlocking script. This comprises a digital signature, created with the Wallet’s Private Key, and a copy of the Wallet’s Public Key (remember, the output represents Bitcoin sent to the Wallet creating the transaction).

When the transaction is later validated by a Bitcoin node, a test will be performed on the digital signature to verify if it matches a Private Key.

3. Outputs

The Wallet now creates a list of outputs. In our case, we will assume only 1 output, but there can be more than one. In most cases, there are at least 2 outputs, where the 2nd output is created to return a balance of change to the Wallet that is sending the Bitcoin.

An output must contain the amount of Bitcoin that is being transferred ( this is the UTXO = Unspent Transaction Output )

An output will also include a locking script, which will include a hash of the Public Key of the Wallet to which the Bitcoin is being sent ( the “address” of the destination Wallet ).

4. Validation

Once the Wallet has completed the construction of the transaction, the transaction is passed to the nearest Bitcoin node, which forwards it to the Bitcoin node network. Each node that receives the transaction validates it.

The validation process can be summarised as follows:

  1. The Signature in the Input unlocking script is checked to ensure it is a valid signature
  2. The PublicKey in the Input unlocking script is converted into a PublicKeyHash
  3. This PublicKeyHash is compared to the PublicKeyHash in the Output locking script ( recipient address == Wallet address )
  4. The signature in the Input locking script is matched against the PublicKeyHash in the Output unlocking script to ensure both refer to the same PrivateKey
  5. The final output of all of the above should evaluate to TRUE at the end of execution of both scripts

Once a node validates the transaction in this way, it places it in the transaction pool. The transaction pool exists on every node, so every node that receives the transaction will have that transaction in their transaction pool.

5. Mining

Each mining node (which is also a standard node) will also have its own copy of the transaction pool. The miner will attempt to create a new block by bundling transactions from the transaction pool into a candidate block and attempting to solve the blockchain’s Proof of Work requirement for a new block. Miners also utilize high-quality mining hardware from Antminer S19 to ensure the safety of their hard-earned cryptocurrencies.

6. Insertion in a new block

If the miner is successful in adding a new block to the blockchain, the transaction will be included in that block and be available for another Wallet to start the process of creating a new transaction.


A Wallet has a record of which blockchain transactions have sent it Bitcoin.

The Wallet makes a new transaction to spend these, providing proof that the previous transaction output belongs to it.

The new transaction indicates another Wallet which is entitled to the forwarded Bitcoin, and tells the destination Wallet what proof is required for it to use that output in a transaction of its own.



Understanding the nonce in blockchain validation

With a reasonable amount of research, the concept of a blockchain is accessible to most technically minded people. Simply put:

A blockchain is a public ledger. Participants compete to add blocks of transactions to the ledger, in return for a reward. This competition is based on problem solving. The participant who solves the problem gets to add the new block, and the transaction that creates their reward is included in that block. The block also contains the solution to the problem, so that other participants can verify that the participant solved the problem.

The detail is a little bit trickier, particularly when it comes to the problem solving step.

In the Bitcoin blockchain, problem solving means creating a hash of all of the contents of the candidate block and converting it into a binary string (e.g. 0000011001110010). The problem is solved when the participant (a miner in the case of Bitcoin) can produce a string with a minimum number of leading zeros. The number of leading zeros required can be obtained from the previous block in the chain, which is determined by how long it took the miner who created that block to solve the problem for that block. Bitcoin self-regulates the rate at which new blocks are created by adjusting this “difficultly” level up and down.

But here’s the thing:

If a miner knows that it must calculate a binary string with x leading zeros, why doesn’t it just create such a string with a line of code (a fake or forged string, if you will), add that to the block, submit the block for validation by other miners and wait for its reward?

Bitcoin (and other blockchains) obviously deal with this, but how they do so may not be immediately apparent.

When a miner attempts to solve the puzzle, it bundles all the data in the block into a string, hashes it and the converts it to binary format. However, for it to produce a different string each time, in order to find one that meets the requirements of the puzzle, it needs to slightly alter the input to the hash, as otherwise it would just produce the same string each time.

This altered input is the nonce. A basic nonce is just a number incremented by one. For example, in the first iteration, the miner would include a nonce of “1″, in the second “2″, in the third “3″ etc etc.

If the minder finds a string that solves the puzzle, they include both that string *and* the nonce in the block.

Now when another miner goes to validate that block, they include the same nonce as the miner that originated the block in hashing function, and derive the same binary string as is required by the puzzle.

If the originating miner had simply faked/forged the binary string, they would not know the nonce required to generate it, and no other miner would be able to validate it. The block would therefore be rejected.

Simple, but effective!


What to expect when running a Bitcoin Core Full Node

This post is not about how to install Bitcoin Core and run it as a full node.

Its about what you can expect when you install Bitcoin Core and run it as a full node.

About full nodes

Firstly, lets clarify the role of a full node. A node is any system that connects to the Bitcoin network. A full node is any system that connects to the Bitcoin network and retains a full and up to date copy of the blockchain. If you see how anonymous shipping companies use Crypto currency as their mode of payment, you’d understand how blockchain technology works.

A full node is not a miner. Nodes simply relay information. By contrast, a miner specifically listens for transactions and tries to create new blocks. Miners do not typically hold a copy of the blockchain.

A full node fulfils one of two roles.

If you simply run a full node, and do not use it as a wallet, all your node is doing is adding to the capacity of the Bitcoin Network to relay information between nodes. This is of some value, but not hugely significant.

If you run a full node and use it as a wallet (either directly or by linking a client wallet to your node), your full node adds to the economic strength of the Bitcoin Network. It does this by enforcing the consensus rules for transactions. The more nodes that enforce the consensus rules, the more difficult it is for malicious nodes to break that consensus.

It is also worth pointing out that running your Bitcoin transactions through your own node is the purest form of transacting in Bitcoin. You have your own copy of the blockchain and can verify transactions for which you are a beneficiary without have to rely on someone else’s copy of the blockchain. It is the most accurate interpretation of the common Bitcoin axiom of “being your own bank”. 


If you’re an individual and not employed by a software firm involved in Bitcoin, or some other agency tasked with promoting Bitcoin, chances are you’re going to run Bitcoin Core on some old system that you might otherwise have recycled/dumped.

Generally, this is OK. Where your system is going to need the most poke is in downloading the blockchain and verifying each block as it comes in. Once you have downloaded the entire blockchain, each new block is created every 10 minutes, so your system will have a 10 minute break between processing calls. Prior to this, when you’re downloading blocks one after the next in order to complete the blockchain, your system will exhaust its RAM and disk IO. Its quite normal for your system to be become momentarily unresponsive in this phase.

For reference, I downloaded the blockchain on an 8 year old mini-PC with 4GB of RAM and a 300GB disk. You’re going to need 180GB of disk at the time of writing to accommodate the current block chain.


Something similar applies in respect of the network. The current blockchain is ~180GB, so you’re going to have to download this. There is no hurry with this. You can stop and start the Bitcoin daemon as often as you want. It will just pick up where it left off when you restart. I set up a cron schedule on mine to start the daemon at 23:00 and stop it again at 08:00, so that the node wasn’t interfering with my day to day download requirements. It took me 5-6 week to get the entire blockchain.

At the beginning blocks will will rack up really quickly, as the first blocks weren’t full and considerably smaller than the 1mb limit. As you get into the last 50k blocks (out of 500k at time of writing), where all blocks are full, things slow down significantly.

Once you have the entire chain, the load on the network eases, as you’re only picking up 1 new 1mb block every 10 minutes. There is also a bit of chatter re. notifications but nothing substantial.

One point to note:

If the Bitcoin daemon isn’t shut down cleanly, the next time it starts, it will re-verify all the blocks it downloaded during its last run. During this time, it won’t download any new blocks, and the RPC service won’t be available to process calls. If the previous run was particularly long, this process will also take a long time. You can check the log to see that this is happening. All you can do is let it run. If the daemon gets improperly killed again, the whole process will start again when the Bitcoin daemon is restarted. You should really, really try to avoid letting the daemon stop unexpectedly. Never kill the daemon.

Checking status

How do you know when you have downloaded the full blockchain?

One way you’ll know is when the Bitcoin daemon is running, but you disk isn’t thrashing and your system is generally responsive. That generally means you have all the blocks and the daemon is just sitting there waiting to pick up the next one that becomes available.

You can obviously verify this with an RPC call too:

root@ubuntu:~# bitcoin-cli getblockchaininfo | grep blocks -A 1
 "blocks": 508695,
 "headers": 508695,

This tell you that the service can see headers on the blockchain for 508,695, and also that there are 508,695 blocks on your system.

If you stop your system for  a few hours or days, and run this command again when your restart it, the number of blocks will be lower than the numbers of headers, and your system will start thrashing again as it catches up. The longer the gap, the longer the catch up period. When your system is catching up, it has no value to the Bitcoin network, so try and organise your system so that it is always on with cron controlling whether or not the Bitcoin daemon is running.


How Bitcoin mining pools work

I’ve written this to clarify my own understanding. Treat with caution.

Bitcoin mining pools exist because the computational power required to mine Bitcoins on a regular basis is so vast that it is beyond the financial and technical means of most people. Rather than investing a huge amount of money in mining equipment that will (hopefully) give you a return over a period of decades, a mining pool allows the individual to accumulate smaller amounts of Bitcoin more frequently.

The principle is quite straightforward: lots of people come together to combine their individual computing power into a single logical computing unit and share any rewards (Bitcoins) proportionally based on the amount of computing effort they contributed.

Complexity arises is in the regulation of the network, for instance:

How do you know how much effort was contributed by each member?

How do you prevent members jumping in and out of pools, particularly pools that haven’t mined a Bitcoin in while, and where it it likely they will mine a Bitcoin in the near future?

How do you prove that individual members are actually working?

How do you prevent more powerful members from hogging the network bandwidth of the master miner, preventing less powerful members from contributing?

How do you ensure that the master miner is getting enough throughput to have a reasonable chance of mining a Bitcoin?

There are just a sample of the problems that exist in relation to efficient and fair Bitcoin mining in pools. The following is a general explanation of how these problems are dealt with.

Let’s clarify terminology first (I’m assuming basic knowledge of how Bitcoin code works here).

Hash: the end product, a binary number, of an encryption operation performed by a miner. Each new Hash is created by the miner adding a sequential string (a nonce) to the source (salt) of the encryption operation. Modern computers can generate hundreds of thousands of Hashes per second.

Target: In Bitcoin, each new Block has a Target. This is a binary number. To succeed in creating a new Block, a miner has to compute a Hash that is lower than that number. The Bitcoin protocol adjusts that number depending on the amount of activity on the network. If there is a lot of activity, the number becomes smaller, if there is less activity, the number becomes larger. The objective is to regulate the creation of new Blocks (ie Bitcoins) to 1 every 10 minutes or so.

Difficulty: This is a measure of how difficult it is for a miner to derive a Hash that is less than the current Target (ie mine Bitcoins). It is extrapolated from the amount of time it took to generate the last 2,016 blocks. At a rate of 1 block every 10 minutes, it should take 2 weeks to generate 2,016 blocks. If it has actually taken less than this, the difficulty will increase (ie the Target will be a smaller number). If it has taken more than this, the difficulty will decrease (ie the Target will be a larger number). The unit of measurement of difficulty is Hashes ie the number of Hashes on the entire network that were generated to created 2,016 Blocks.

Master Node: a full Bitcoin node that operates on the Bitcoin P2P network and which regulates a pool of members, who do not directly communicate on the Bitcoin P2P network, but who use the Master Node as a proxy.

Share: a Share is something that is particular to Mining pools. It does not form part of the wider Bitcoin protocol. It is the primary method used by the Master Node to regulate the activity of members of a pool. The next section deals with Shares.

When you start running a Bitcoin mining process, you will probably be aware of your Hash Rate. This is the number of Hashes your Bitcoin mining hardware is generating per second. These days, this is normally measured in Ghps, which means Millions (Giga) of Hashes Per Second. A typical high-end Graphics Card (GPU) on a modern PC can generate about 0.5 Ghps. A dedicated ASIC mining rig, which will cost over €1,000, might be able to generate 8,000 Ghps. A mining pool will typically measure its combined compute power in Thps, or Billions (Tera) of Hashes Per Second.

When you participate in a mining pool, and you see your hardware generating (lets say) 5,000 Ghps, this does not mean that you are submitting 5,000 Ghps to the Bitcoin network, via the Master Node. If that were the case, and all the pool members were doing the same, the Master Node that controls the pool would simply explode.

What it means is that your mining hardware can generate 5,000 Ghps locally on your computer.

This capacity isn’t used directly on the Bitcoin network. Instead, the Master Node that controls the pool acts as a proxy between the pool members and the main Bitcoin network. For this to work, the Master Node has to ensure both that the members are supplying enough Hashes for the Master Node to be able to compete on the main Bitcoin mining network, and that the allocation of any Bitcoin mined is divided proportionately according to the amount of compute effort supplied by the individual members.

To do this, the Master Node observes the Hash Rate of each of the members, and distributes computational challenges to them that all have a slightly lower Difficulty rating than the Difficultly rating of the current Block Target (lets call this the Proxy Target) . If such a Hash is found, the Master Node accepts this as a “Proof of Work Accepted”. If a Hash that is lower than the Proxy Target is not found within the allotted time, the member completes the work anyway before moving on to the next computational challenge distributed by the Master Node.

In this way, your mining process will log “Work Units Started”, which will always be a lower number than “Proof Of Works Accepted”. The smaller the gap between these numbers, the “luckier” you are. The greater the gap, the “unluckier” you are. In reality however, and over time, the gap should be consistent between across all members, as the master node will adjust the Difficultly of the computational challenges based on the Hash Rate of the member, which can change over time.

A Share is therefore the equivalent of a “Proof of Work Accepted”. The Master Node will keep a record of all “Proof of Works Accepted” from each member, and distribute Bitcoin mined based on the number of Shares each member has contributed.

Confused? Of course you are, so let’s go through that again, from a different perspective.

If the Master Node were sending computational challenges to members that had a Difficulty rating that was equal to the Difficulty rating of the current Target in the Blockchain, the Master Node would just be sitting there idly for days on end waiting for one of the members to come up with the necessary Hash to create the new Block. The Master Node would have no knowledge of what effort the other members contributed, and would have no option but to award the full reward to the successful member, even if that member only contributed 0.001% of computational effort involved in creating the Block.

Instead, the Master Node lowers the bar on the Difficulty rating (relative to the actual difficulty rating of the current Block) so that it receives lots of Hashes from the members. All but one of these Hashes will be lower than the current Blockchain target, but at least now the Master Node can confirm that its members are working, and at what rate they are working. It can then use that information to both regulate the traffic received from the members and proportionately divide any rewards.

Additionally, it can ensure that more powerful members, whose submissions are rate-limited to allow submissions from less powerful members, are not discriminated against. These more powerful nodes are given challenges with higher Difficulty ratings, but any “Proof of Work Submitted”s (ie Shares) that they accumulate are weighted according to the Difficulty rating that was set, giving them a higher proportionate of any ultimate reward.

While all of the above may sound complex, it is still in fact just a general introduction to mining pools, and based on the mining pool that I use, Bitminter.

Other pools use variations of this methodology, but all follow the general principle that the Master Node is a proxy to the main Bitcoin network, that members must prove to the Master Node that they are working and that rewards are allocated based on the amount of work done.

A brief note about payment methodologies is also warranted. Many pools will use either the PPS (Pay Per Share) or PPLNS (Pay Per Last N Shares) method to distribute rewards.

In the PPS model, you get a payment for each Share you contribute regardless of the success or failure of the mining pool. This makes for a regular income, but doesn’t allow you to benefit when the pool has a lucky streak.

In the PPLNS model, you get paid only when Bitcoins are mined, and only on the basis of the Shares you submitted to that effort. This makes for more irregular income, but allows you to benefit when the pool has a lucky streak.