Chapter 8
Difficulty and target

Initial version: 2024-03-16
Last update: 2024-03-26

Target and difficulty are related to each other but doesn't mean the same, as current, voltage and resistance are related by Ohm's Law but means different physical properties of electricity.

In this chapter you will understand why target is indispensable part of blockchain and how it determine an even flow of time. However the topic is very general, to give precise examples I will use the Bitcoin's difficulty and target.

Table of contents


Target


In chapter 3: Data fingerprinting, section: Proof of work – solving a hash puzzle you may find an information that to mine a block (to successfully add it to a blockchain) you have to solve cryptographic puzzle. You have to find a number called nonce such that hash of a header of this block with nonce as one of its fields conforms certain criteria. I said that hash must starts with five leading zeros. It was just an example of many possibilities, simple enough so I could explained related material. From now I will change criterion for nonce: it must be a number such that hash is below a given number. This number is called a target.

For some reasons I explain you in chapter 10: Searching consensus, section: Searching Consensus Part 2: Chainwork – The heaviest chain criterion the target adjusts every 2016 blocks (roughly two weeks) to ensure that blocks are mined once every 10 minutes on average:

$$2016 \cdot 10 [\textrm{min}] = 20160 [\textrm{min}]$$

$$[\textrm{day}] = 60[\textrm{min}] \cdot 24 = 1440[\textrm{min}]$$

$$20160[\textrm{min}]/1440[\textrm{min}] = 14 [\textrm{days}] = 2 [\textrm{weeks}]$$.

It therefore creates a constant time intervals between blocks and a constant frequency of new bitcoins issuance to the network (via the block reward). Whenever I use a term session or block session I will have in mind 2016 blocks mined in the same time slot (which is expected to be equal more or less 10 minutes).

Difficulty


If blocks during two weeks period were being mined faster or slower than assumed 10 minutes, the target should be adjusted. The question is, how to select new value as a target. This is how the difficulty comes to play – it controls the target.

After every 2016 blocks added to a blockchain a time needed for this is calculated. In perfect conditions it should be equal to 20160 but in most cases it is not.

Assume blocks during this period were being mined faster than expected 10 minutes and it took 16000 minutes to add 2016 blocks. The ratio timeExpected / timeActual is equal to:


20160/16000 = 1.26
Conversely, assume blocks during this period were being mined slower than expected 10 minutes, and it took 21000 minutes to add 2016 blocks. The ratio is equal to:


20160/21000 = 0.96
If blocks during session were being mined faster than assumed 10 minutes, the ratio is greater than 1.0 so the target should be adjusted downwards to make it more difficult to get below the target for the next part of blocks.

If blocks were being mined slower than assumed 10 minutes, the ratio is lower than 1.0 so the target should be adjusted upwards to make it less difficult to get above the target for the next part of blocks.

As you can see, the higher is difficulty, the lower is target.

difficulty is the number used to calculate target:


target = targetMax / difficulty
where targetMax is the maximum value for target. Interestingly, targetMax doesn't have to be a maximum number in a given range – it is rather the "first target" set when the blockchain first run. For example in case of Bitcoin the maximum target used by SHA256 mining devices is (SHA-256 is a 256-bit digest, which is equivalent to 32 bytes equivalently 64 hexadecimal digits):


                       1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
0x00000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
instead of possible:


                       1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
Moreover, because Bitcoin stores the target in a compact, floating-point-like, representation (see section Target and difficulty in bitcoin below), this is limited to the value:


                       1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
0x00000000FFFF0000000000000000000000000000000000000000000000000000
The ratio timeExpected / timeActual calculated above is used to adjust difficulty:


newDifficulty  = difficulty x ratio
where initial difficulty is an arbitrary selected number (randomly or somehow precalculated) laying in range [0, targetMax].

In Bitcoin the adjustment is limited by a factor of 4, i.e. a number not greater than 4 or less than 0.25 to prevent massive changes from one target to the next.



Example of adjusting target with difficulty


In this section I present a code you can use to make some experiments how difficulty works in practice. To incorporate an unpredictable component you need to use, as it is in real Bitcoin, proof of work function (you should know it from chapter 3: Data fingerprinting, section: Solving hash puzzle):


def proofOfWork(blockOfData, target):
  targetInt = int(target, 16)
  counter = 1

  start = timer()
  while True:
    blockOfData["nonce"] = format(random.getrandbits(64), "x")
    dataString = json.dumps(blockOfData, sort_keys=True)
    dataBytes = dataString.encode('utf-8')

    hash = hashlib.sha256(dataBytes).hexdigest()
    hashInt = int(hash, 16)

    if hashInt < targetInt:
      blockOfData["hash"] = hash
      blockOfData["counter"] = counter
      end = timer()
      blockOfData["powTime"] = end-start
      break

    counter += 1
    

  return blockOfData
Now you can use above proofOfWork(blockOfData, target) function to implement testing function. My implementation takes three arguments:
  • numberOfBlocks – how many blocks I want to add during one block session.
  • averageTimePerBlock - expected average time function proofOfWork(blockOfData, target) needs to mine (and in consequence add) one block of data. Thus expected time for one block session is numberOfBlocks * averageTimePerBlock.
  • iterations – how many block sessions you want to repeat.


In every block I include the same data – it is not important because of the random nature of proofOfWork(blockOfData, target) function:

  dataJSON = {"name": "Piotr", "country": "Poland"}
  blockOfData = {"data" : dataJSON,
                 "nonce": None}
Before main loop I define all required values:

                  #                     1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3
                  # 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  targetMax = int("FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF", 16)
  difficultyInitial = 1.0
  timeExpected = numberOfBlocks * averageTimePerBlock

  difficulty = difficultyInitial

  target = int(targetMax / difficulty)
  if target>targetMax:
    target = targetMax
  target = hex(target)
  
  print(f"Time expected for one block session: {timeExpected}")
  print(f"Initial difficulty: {difficulty}")
  print(f"Initial target: {target}")
timeActualTotal variable accumulates the total time of computation used till current iteration which will allow it to be averaged and determine whether it converges to the assumed average time:

  timeActualTotal = 0
The first block of code in the main loop is for complete one block session:

    timeActual = 0
    for b in range(numberOfBlocks):
      proofOfWork(blockOfData, target)
      timeActual += blockOfData['powTime']
    timeActualTotal += timeActual
Now you can calculate the ratio of expected time and actual time needed to add assumed number of blocks (actual time needed for one block session):

    ratio = timeExpected/timeActual
    
    if ratio < 0.25:
      ratio = 0.25
    elif ratio > 4:
      ratio = 4
Having ratio you can calculate new difficulty and update target:


    difficulty = difficulty * ratio

    target = int(targetMax / difficulty)
    if target>targetMax:
      target = targetMax
    target = hex(target)
Finally you can print effects of one iteration:

    print(f"Iteration: {(i+1)}, actual time in this block session: {timeActual}, average time from beginning: {timeActualTotal/(i+1)}")
    print(f"  New difficulty: {difficulty}")
    print(f"  New target: {target}")
Below I put the full code:


def test(numberOfBlocks, averageTimePerBlock, iterations):
  dataJSON = {"name": "Piotr", "country": "Poland"}
  blockOfData = {"data" : dataJSON,
                 "nonce": None}
                  #                     1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3
                  # 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  targetMax = int("FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF", 16)
  difficultyInitial = 1.0
  timeExpected = numberOfBlocks * averageTimePerBlock

  difficulty = difficultyInitial

  target = int(targetMax / difficulty)
  if target>targetMax:
    target = targetMax
  target = hex(target)
  
  print(f"Time expected for one block session: {timeExpected}")
  print(f"Initial difficulty: {difficulty}")
  print(f"Initial target: {target}")

  timeActualTotal = 0
  
  for i in range(iterations):
    timeActual = 0
    for b in range(numberOfBlocks):
      proofOfWork(blockOfData, target)
      timeActual += blockOfData['powTime']
    timeActualTotal += timeActual

    ratio = timeExpected/timeActual
    
    if ratio < 0.25:
      ratio = 0.25
    elif ratio > 4:
      ratio = 4

    difficulty = difficulty * ratio

    target = int(targetMax / difficulty)
    if target>targetMax:
      target = targetMax
    target = hex(target)

    print(f"Iteration: {(i+1)}, actual time in this block session: {timeActual}, average time from beginning: {timeActualTotal/(i+1)}")
    print(f"  New difficulty: {difficulty}")
    print(f"  New target: {target}")
You can run it simply by:


test(10, 5, 15)
Result you will see will be similar to mine (get full test result):


Time expected for one block session: 50
Initial difficulty: 1.0
Initial target: 0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
Iteration: 1, actual time in this block session: 0.000549109000000006, average time from beginning: 0.000549109000000006
  New difficulty: 4.0
  New target: 0x4000000000000000000000000000000000000000000000000000000000000000
Iteration: 2, actual time in this block session: 0.0007540369999999852, average time from beginning: 0.0006515729999999956
  New difficulty: 16.0
  New target: 0x1000000000000000000000000000000000000000000000000000000000000000
Iteration: 3, actual time in this block session: 0.00249688999999996, average time from beginning: 0.0012666786666666503
  New difficulty: 64.0
  New target: 0x400000000000000000000000000000000000000000000000000000000000000
Iteration: 4, actual time in this block session: 0.010782393000000001, average time from beginning: 0.003645607249999988
  New difficulty: 256.0
  New target: 0x100000000000000000000000000000000000000000000000000000000000000
Iteration: 5, actual time in this block session: 0.030916355999999978, average time from beginning: 0.009099756999999986
  New difficulty: 1024.0
  New target: 0x40000000000000000000000000000000000000000000000000000000000000
Iteration: 6, actual time in this block session: 0.11693818699999992, average time from beginning: 0.027072828666666642
  New difficulty: 4096.0
  New target: 0x10000000000000000000000000000000000000000000000000000000000000
Iteration: 7, actual time in this block session: 0.5838842140000001, average time from beginning: 0.10661731228571428
  New difficulty: 16384.0
  New target: 0x4000000000000000000000000000000000000000000000000000000000000
Iteration: 8, actual time in this block session: 2.535226811, average time from beginning: 0.41019349962499996
  New difficulty: 65536.0
  New target: 0x1000000000000000000000000000000000000000000000000000000000000
Iteration: 9, actual time in this block session: 11.975442539, average time from beginning: 1.6952211706666667
  New difficulty: 262144.0
  New target: 0x400000000000000000000000000000000000000000000000000000000000
Iteration: 10, actual time in this block session: 35.08085456499999, average time from beginning: 5.0337845100999985
  New difficulty: 373628.29847015743
  New target: 0x2ce74b5f5cae680000000000000000000000000000000000000000000000
Iteration: 11, actual time in this block session: 49.03751475100001, average time from beginning: 9.03412362290909
  New difficulty: 380961.6987803588
  New target: 0x2c0a036e0573540000000000000000000000000000000000000000000000
Iteration: 12, actual time in this block session: 60.67007936599998, average time from beginning: 13.337119934833332
  New difficulty: 313961.7606910969
  New target: 0x356fe7df5daf4a0000000000000000000000000000000000000000000000
Iteration: 13, actual time in this block session: 33.559905265, average time from beginning: 14.892718806384615
  New difficulty: 467763.1808134619
  New target: 0x23dded442bbc560000000000000000000000000000000000000000000000
Iteration: 14, actual time in this block session: 61.357718315, average time from beginning: 18.211647342714283
  New difficulty: 381177.13114106195
  New target: 0x2c03a43f3dede00000000000000000000000000000000000000000000000
Iteration: 15, actual time in this block session: 61.685968162999956, average time from beginning: 21.109935397399997
  New difficulty: 308965.83331061096
  New target: 0x364d1b8c3026a40000000000000000000000000000000000000000000000
Iteration: 16, actual time in this block session: 52.95818687100001, average time from beginning: 23.100451114499997
  New difficulty: 291707.3370196905
  New target: 0x39838c910b7a380000000000000000000000000000000000000000000000
Iteration: 17, actual time in this block session: 30.418668168999886, average time from beginning: 23.53093447064705
  New difficulty: 479487.3585507168
  New target: 0x22fd6a42a983aa0000000000000000000000000000000000000000000000
Iteration: 18, actual time in this block session: 75.17976510800003, average time from beginning: 26.400313950499992
  New difficulty: 318893.8924336261
  New target: 0x349c53c58fec200000000000000000000000000000000000000000000000
[...]
Iteration: 21, actual time in this block session: 39.80206675300019, average time from beginning: 28.84023978647618
  New difficulty: 490483.2248725292
  New target: 0x22349a960289680000000000000000000000000000000000000000000000
Iteration: 22, actual time in this block session: 110.7451272369999, average time from beginning: 32.56318921604544
  New difficulty: 221446.86502678873
  New target: 0x4bc3062ef332900000000000000000000000000000000000000000000000
Iteration: 23, actual time in this block session: 48.87483580000003, average time from beginning: 33.27239124143477
  New difficulty: 226544.86854233954
  New target: 0x4a0e9289eea2e80000000000000000000000000000000000000000000000
Iteration: 24, actual time in this block session: 27.066515722000076, average time from beginning: 33.01381309479166
  New difficulty: 418496.5491479947
  New target: 0x2816d9977f76420000000000000000000000000000000000000000000000
Iteration: 25, actual time in this block session: 38.16597068900012, average time from beginning: 33.219899398559996
  New difficulty: 548258.7519627927
  New target: 0x1e99d526aa312f0000000000000000000000000000000000000000000000
Iteration: 26, actual time in this block session: 86.9994053769999, average time from beginning: 35.2883419361923
  New difficulty: 315093.39034387027
  New target: 0x353ec68d8d0ab20000000000000000000000000000000000000000000000
[...]
Iteration: 47, actual time in this block session: 19.789343521999854, average time from beginning: 42.62506108223402
  New difficulty: 793356.5597691704
  New target: 0x1525aa739e23bd0000000000000000000000000000000000000000000000
Iteration: 48, actual time in this block session: 89.86258693899936, average time from beginning: 43.60917620424997
  New difficulty: 441427.6212122169
  New target: 0x2601b866e75abc0000000000000000000000000000000000000000000000
Iteration: 49, actual time in this block session: 60.31378878299938, average time from beginning: 43.95008666504078
  New difficulty: 365942.53993926005
  New target: 0x2dd8b9d64c72680000000000000000000000000000000000000000000000
Iteration: 50, actual time in this block session: 101.13980477199948, average time from beginning: 45.09388102717995
  New difficulty: 180909.25761830772
  New target: 0x5cbd00dd4c51540000000000000000000000000000000000000000000000
[...]
Iteration: 67, actual time in this block session: 116.74055654799895, average time from beginning: 47.595178161701426
  New difficulty: 272525.29439780925
  New target: 0x3d8fe28e90f79a0000000000000000000000000000000000000000000000
Iteration: 68, actual time in this block session: 32.286549626000124, average time from beginning: 47.37005127147052
  New difficulty: 422041.52743894723
  New target: 0x27c0a56bbd6a7a0000000000000000000000000000000000000000000000
[...]
Iteration: 98, actual time in this block session: 27.878575578998607, average time from beginning: 49.657258545489725
  New difficulty: 583147.6488104425
  New target: 0x1cc525411ee7cb0000000000000000000000000000000000000000000000
Iteration: 99, actual time in this block session: 158.3333488119997, average time from beginning: 50.754996831010025
  New difficulty: 184151.8710953479
  New target: 0x5b1af68e4ecd680000000000000000000000000000000000000000000000
Iteration: 100, actual time in this block session: 33.470196680999834, average time from beginning: 50.582148829509926
  New difficulty: 275098.2804948471
  New target: 0x3cfc7bb63187e20000000000000000000000000000000000000000000000


Target and difficulty in bitcoin


In case of Bitcoin in the block header of every block you can find the difficulty field and the bits field which is a way of storing the target in compact representation.

bits field always takes 4 bytes, 8 hexadecimal digits, splited in to two parts:

  • exponent (1 byte length): this gives you the length of the target in bytes (without leading zeros).
  • coefficient (3 bytes length): this gives you the initial 3 bytes of the target.


For example, if bits are:


bits: 0x051ABCD6
then after spliting it into two parts you obtain:


exponent: 05
coefficient: 1ABCD6
This codes the following target:

target: 0x1ABCD60000 which in full size form (32 bytes length) with leading zeros takes the following value:


                                                    coefficient (1ABCD6)
                                                              ------
target: 0000000000000000000000000000000000000000000000000000001ABCD60000
                                                              <---------
                                               exponent (0x05 = 5 bytes)


The value for targetMax in compact representation is 1D00FFFF. After splitting it into two parts you obtain:


exponent: 1D (16) = 29 (10)
coefficient: 00FFFF
In full size form (32 bytes length) with leading zeros it looks like I have mentioned earlier:


        coefficient (00FFFF)
        ------
0x00000000FFFF0000000000000000000000000000000000000000000000000000
  -----><---------------------------------------------------------
    |   exponent (0x1D = 29 = 3 coefficient bytes + 26 zero bytes)
    |
    leading zeros


Summary


In this chapter you have learned how difficulty controls target which in turn influence the speed at which you add new blocks (session blocks) to the blockchain. This control is indispensable part of blockchain as it limit the number of blocks you can add in time interval. This way bad people can't forge past block as they would have to catch up and overtake the newest block. Because of difficulty they can't do this faster than assumed, constant time which causes them to be unable to outrun the newest block.