Francesco Cesarini – Erlang Solutions – Operators must embrace change

FrancescoErlangSolutions
Francesco Cesarini, Erlang Solutions

Francesco Cesarini is the Founder and Technical Director of Erlang Solutions, a specialist technology consultancy that has worked with companies as varied as bet365 and Whatsapp. 

Here he delivers his views on betting trends on this year’s European Championships and the necessity for sports betting operators to be open minded to the idea of adopting new technologies. 

This year’s Euro competition could see some of the largest peaks of betting activity in the history of online gambling. Peaks that will likely push traditional technologies to the very limits of their capabilities.

As we build towards the closing stages of this year’s competition, sports betting companies that have not explored alternative approaches to their technical development (or are nervous to do so), could be taking an enormous gamble with their business.

Let’s put that into perspective. The Grand National is typically the biggest day in the online sports betting calendar. This year was no different delivering 30% growth, for one of our clients, on last year’s race.

Despite this, reports suggest that the recent Champions League Final saw even bigger peaks with many times more activity than the Grand National.  We can only assume that this trend will continue.

The rabid appetite for ‘In Play’ and the increase of casual gamers getting involved in high profile sporting events, has seen the volumes of betting activity grow exponentially over the last decade. So it stands to reason that this Euro competition and probably every other major football event to follow, will produce record peaks of activity.

When it comes to meeting that demand, sports betting companies have traditionally taken the vanilla route, building Java applications that report into SQL based systems.

As the transaction curve has gotten steeper over time, the answer has been to bring in additional hosting hardware to take up the slack. For major occasions, like the Champions League Final, it’s not unusual for companies to pay through the nose for massive, monolithic machines to handle throughput.

The spending doesn’t end there. Typically, a business will organise a dedicated team to manage the event.  This means getting more hands on deck for problem solving, and generally increasing alertness across staff for these big events, upon which earnings will significantly depend.

Finally, support procedures are reviewed with all suppliers to ensure they have a disaster recovery plan ready, should the worst happen.

But there comes a time when throwing money at the problem no longer works. Java will only scale so far and is not as predictable under heavy load. As a result, the time it takes the system to cascade data out exceeds the time available. This results in bottlenecks that impact the ability to support markets, push data and most vitally, take and reconcile bets.

Leaving this issue unaddressed is not an option. Online betting is a super competitive industry that is driven by the need to balance a smooth, scalable service with pressures to add new functionality that maintain customer interest.

Add to that the need to offer attractive deals to induce players to place bets at peak times and not addressing the problem starts to resemble commercial suicide.

Some firms have embraced the NoSQL approach, to take the pressure of their transaction systems and ensure it can focus on doing the things it was originally intended for. Those that haven’t will likely start to feel the pinch.

Either, they’ll find themselves in a position where they can’t offer comparable, competitive deals for fear of not being able to process the extra traffic. Or they will strike ahead, hoping to ride any hiccups that occur, only to lose business to firms offering a better, more robust experience.

From what I can tell, the industry is in a bit of turmoil. On the one hand, there is an increasing realisation that a switch is needed to new technologies but there is also a reticence to try something new.

This, then, is not a technical problem. The technology is available to meet the demand. The challenge is risk averse cultural mindsets, that prohibit the investigation of new approaches and adoption of new technologies that deliver true innovation.

Typically, the dominant technological mindset always wins in the innovation argument. If you have a workforce of Java programmers, they will look for a solutions in Java and ignore potential new avenues outside of their comfort zones.

Which is a shame because the upsides can be quite spectacular.

Take Erlang for example. It’s a 25 year old programming language that was developed by Ericsson to improve the scale and resilience of its telephony applications. It is now becoming the ‘go to’ language for online organisations, who experience similar challenges of massive scale, concurrency, reliability and low latency.

It’s a technology that lies at the heart of WhatsApp’s technical infrastructure and enables the company’s 1 billion users to send an average of 42 billion messages a day. That’s double the total of daily SMSs.

This is achieved with a few hundred servers and a backend engineering team of approximately 15 responsible for the development, support and maintenance of their code base.

It is used by Machinezone to ensure that millions of concurrent players have the opportunity to wage war on their neighbours in a shared playing field.

It is also a key technology used by bet365 to ensure that their systems continued to function smoothly in the face of massive peaks during this year’s Grand National and Champion League’s Final.

The Erlang programming language has enabled each of these companies to build a concurrent and fault tolerant applications that behaves predictably under massive load, without sacrificing resilience and reliability.

This means, they are not only able to increase capacity exponentially, but because Erlang uses every core in a multi-core machine, they can utilise less hardware, not more.

And because of Erlang’s real-time properties, you can react to external events as they happen. Should something go wrong, you can respond to any problem, without taking the entire system down.

These things don’t happen overnight. Developing new systems, using new technologies is something that takes time. If you haven’t already, it’s too late to get started on an improvement programme for this Euro competition.

But it isn’t too late to ensure you are prepared for the future growth of the industry.