Asynchronous fault tolerant programming with PHP
In the previous post on fault tolerant programming in PHP we have shown the basics of fault tolerant programming using the circuit breaker pattern. Now we will show you a running demo of the application where the circuit breaker library Phystrix is combined with asynchronous programming. The advantage of this approach is that it allows the querying of multiple backend services asynchronously. A timeout can be set for each of the calls and the circuit breaker will deal with failing services.
In order to get this to work we had to make a few changes to Phystrix. The code for this demo can be found here, also check out a screencast of the demo below.
A quick review
Before we get started, lets recap the example from the previous post. It consists of two internal and one external API:
The problem we showed was that by just programming the happy path an error in one of your dependencies can propagate all the way through your application:
Inventory service app is down the
Public shop API will also be down.
We showed you how to avoid this using the circuit breaker pattern, making the
application more fault-tolerant. The next step is to be able to use the circuit
breaker pattern for asynchronous calls. In order to get asynchronous calls in
php we will use the ReactPHP library. Our demo shows how everything works.
Running the demo
Clone the repository and use composer to install the dependencies:
$ git clone https://github.com/qandidate-labs/phystrix-async-demo.git $ cd phystrix-async-demo $ composer install
You can now run the demo - be sure to execute each command in its own terminal.
$ bin/run_all.sh $ bin/get_catalogue_inventory_status.sh
Both of the commands will show some output.
run_all.sh will output the server
access logs and the Phystrix command log:
The command log shows that the execution of both the
GetInventoryStatusCommand resulted in a successful call.
get_catalogue_inventory_status.sh has the following output:
Since both microservices are fully operational the catalogue and inventory
status are properly displayed.
In order to see the circuit breaker in action we will make the inventory status
API fail. You can do this by modifying
inventory-status/index.php and by
sleep(1). The inventory status API will take too long to respond,
triggering a timeout.
When the API call fails, Phystrix will return fallback values instead of
propagating the failures.The result is that the inventory status of all items
in the catalogue is set to
The really interesting part can be found in the command log:
First, a few successes are shown. After this, a failure is triggered for the
GetInventoryStatusCommand (due to the timeout). This results in a
FAILURE,FALLBACK_SUCCESS. After two failures the circuit is short circuited
SHORT_CIRCUITED,FALLBACK_SUCCESS) - the result is that the fallback is
After some time a
FAILURE,FALLBACK_SUCCESS occurs again. The circuit breaker
decides it is time to test if the service is back up. Since it is not (the
timeout is still hit), the Command fails again, and the circuit remains
short-circuited. Had the retry been a success, the circuit would have been
closed and all new calls would have again be forwarded to the backend service.
You now have seen the demo and everything appears to be working magically, including the asynchronous backend calls. Next time we will have a look at how everything works internally!
This demo shows the concept of having a circuit breaker for asynchronous api calls. This prevents a failure in one of your calls to take down your whole system. This screencast shows the circuit breaker in action.
Come and join us in our quest to build the best development team in the universe...and beyond!