Strategies Explained

Why Are Backoff Strategies Useful?

Because now and again, stuff breaks.

Often, when making function calls, something goes wrong. The internet might glitch. The API we’re calling might timeout. Gremlins might eat your packets. Any number of things can go wrong, and Murphy’s law tells us that they will.

Which is why we need backoff strategies. Basically, a backoff strategy is a technique that we can use to retry failing function calls after a given delay - and keep retrying them until either the function call works, or until we’ve tried so many times that we just give up and handle the error.

How Do Strategies Work?

In the Backoff-Utils library, strategies exist to calculate the delay that should be applied between retries. That’s all they do. Everything else is handled by the backoff() function and @apply_backoff decorator.

The library supports five different strategies, each of which inherits from BackoffStrategy.

Caution

BackoffStrategy is itself an abstract base class and cannot be instantiated directly. You can subclass it to create your own custom strategies, or you can supply one of our ready-made strategies as the strategy argument when applying a backoff.

When you apply a backoff strategy, you must supply a strategy argument which can accept either a class that inherits from BackoffStrategy, or an instance of a class that inherits from BackoffStrategy.

Passing a class will use the default configuration for the backoff strategy, while passing an instance will let you modify that configuration. For example:

result = backoff(some_function,
                 args = ['value1', 'value2'],
                 kwargs = { 'kwarg1': 'value3' },
                 max_tries = 3,
                 max_delay = 30,
                 strategy = strategies.Exponential)

will call some_function() with an Exponential strategy applying its default settings, while:

my_strategy = strategies.Polynomial(exponent = 3, scale_factor = 0.5)

result = backoff(some_function,
                 args = ['value1', 'value2'],
                 kwargs = { 'kwarg1': 'value3' },
                 max_tries = 3,
                 max_delay = 30,
                 strategy = my_strategy)

will call some_function() with a Polynomial strategy using an exponent of 3 and a scale factor of 0.5.

Strategy Features

Random Jitter

All strategies support using a random jitter.

You can deactivate the jitter on a strategy by instantiating it with the argument jitter = False. For example:

my_strategy = strategies.Exponential(jitter = False)

will ensure that no jitter is applied.

Hint

By default, all strategies apply a random jitter unless explicitly deactivated.

Minimum Delay

While each strategy calculates its delay based on its own logic, you can ensure that the delay returned is always a certain minimum number of seconds. You can apply a minimum by instantiating a strategy with the minimum argument. For example:

my_strategy = strategies.Exponential(minimum = 5)

will ensure that at least 5 seconds will pass between retry attempts.

Hint

By default, there is no minimum.

Scale Factor

Certain strategies - like the Polynomial strategy - can rapidly lead to very long delays between retry attempts. To offset this, while still retaining the shape of the curve between retry attempts, each strategy has a scale_factor property which is multipled by the “unadjusted” delay. This can be used to reduce (or increase) the size (technically the magnitude) of the delay.

To apply a scale factor, pass it as the scale_factor argument when instantiating the strategy. For example:

my_strategy = strategies.Exponential(scale_factor = 0.5)

will ensure that whatever delay is calculated will always be reduced by 50% before being applied.

Hint

The scale factor defaults to a value of 1.0.

Supported Strategies

The library comes with five commonly-used backoff/retry strategies:

However, you can also create your own custom strategies by inheriting from BackoffStrategy.

Exponential

The base delay time is calculated as:

\[2^a\]

where \(a\) is the number of unsuccessful attempts that have been made.

Fibonacci

The base delay time is returned as the Fibonacci number corresponding to the current attempt.

Fixed

The base delay time is calculated as a fixed value determined by the attempt number.

To configure the sequence, instantiate the strategy passing an iterable to sequence like in the example below:

my_strategy = strategies.Fixed(sequence = [2, 4, 6, 8])

Note

If the number of attempts exceeds the length of the sequence, the last delay in the sequence will be repeated.

Tip

If no sequence is given, by default each base delay will be 1 second long.

Linear

The base delay time is equal to the attempt count.

Polynomial

The base delay time is calculated as:

\[a^e\]

where:

  • \(a\) is the number of unsuccessful attempts that have been made,
  • \(e\) is the exponent configured for the strategy.

To set the exponent, pass exponent as an argument to the class as follows:

my_strategy = strategies.Polynomial(exponent = 2)

will calculate the base delay as

\[a^2\]

where \(a\) is the number of unsuccessful attempts that have been made.

Creating Your Own Strategies

You can create your own custom backoff strategy by subclassing from strategies.BackoffStrategy. When you do so, you will need to define your own time_to_sleep property which returns a float.

For example:

import random
from backoff_utils import strategies

class MyCustomStrategy(strategies.BackoffStrategy):
  """This is a custom strategy that will always wait a random number of
  milliseconds."""

  @property
  def time_to_sleep(self):
    return random.random()

The custom strategy created above will always wait a random number of milliseconds, regardless of anything else. You can make your classes as complicated as they need to be, and use whatever logic you choose.