Dynamoth: Cloud-based Scalable Publish/Subscribe

Cloud-Based Publish/Subscribe systems are very useful when it comes to delivering real-time information quickly to interested parties. Publish/subscribe allows for decoupling content producers (publishers) from content consumers (subscribers).

In topic-based publish/subscribe [1], publishers send publication messages to a given publication server, tagged with a specific topic. All subscribers subscribing to that specific topic then receive the publication.

Large-scale topic-based publish/subscribe have wide practical applications in many areas such as social networks, push notification infrastructures and chat frameworks, amongst others. The flexibility of such systems make them easily adaptable to a various scenarios, such as large-to-epic-scale games,since specific areas of the world can be represented as « topics », and interested players and other in-game entities can publish and receive relevant state updates. A desired property is for these systems to be able to scale in real-time according to the current load. Using the Cloud allows for vertical scalability while paying only for the resources that are used.

Dynamoth_Overview

As part of my research, I developed Dynamoth [2,3], a scalable, Cloud-based publish/subscribe platform that automatically scales in real-time to react to variations in experienced load. Dynamoth is tailored towards latency-constrained applications such as immersive large-scale games. It supports “per-topic”, “per-client”, fine-grained real-time load monitoring and load balancing, in order to make efficient use of Cloud resources and scale up and down whenever needed. Dynamoth also supports different horizontal scalability models in order to support arbitrarily popular topics (for instance, a very popular area in a game or a very popular microblogging feed).

Extensive experiments were run on the Amazon EC2 Cloud, in the context of large-scale massive multiplayer online games, with over 1000 AI-controled players and several thousands publish/subscribe topics, in order to assess the scalability of our approach while respecting the tight constraints of latency-constrained applications.


[1] Eugster, P.T., Felber, P.A., Guerraoui, R., Kermarrec, A.M.: The many faces of publish/subscribe. ACM Comput. Surv. 35(2), 114–131 (2003)

[2] Gascon-Samson, J., Garcia, F.-P., Kemme, B., Kienzle, J. (2015) Dynamoth: A Scalable Pub/Sub Middleware for Latency-Constrained Applications in the Cloud, International Conference on Distributed Computing Systems (ICDCS 2015), Columbus, USA [Acceptance ratio: 13%]

[3] Gascon-Samson J., Kemme, B., Kienzle, J. (2013) Lamoth: A Message Dissemination Middleware for MMOGs in the Cloud, Annual Workshop on Network and Systems Support for Games (NetGames 2013), Denver, USA

Advertisements