Massive Multi-player Networking for Games

The United States, along with 26 other countries, signed the Treaty on Open Skies on 24 March 1992 as part of a wide-ranging international effort to promote the openness of military forces and activities. All signatories are obligated to accept overflights of their entire national territory as defined in the treaty. The gathered data is then made available to all treaty signatories regardless of actual overflight participation.

Inspired by this notion that anyone could overfly anywhere without restriction, Cybernet developed its OpenSkies Massive Multi-Player networking and simulation architecture.

• Imagine a simulation or game environment where there is virtually no limit to where you can play or how many players with whom you can play.
• Imagine network performance good enough for the most demanding first person shooter game.
• Imagine inserting the network into your game without complex network programming -- a simplified object-oriented API is all you need.
• Imagine incorporating network text and audio chat into your game environment as part of the network bundle.
• Imagine supporting each new network enabled game without having to add new servers -- one common network infrastructure supports many games simultaneously -- Alternatively, with an alternate library you can support local networked games or simulations which are fully DMSO HLA interoperable.
• Imagine cutting your multi-player network set-up and operating costs in half!

The OpenSkies model and implementation components can do this and more.

Cybernet has developed a patent pending, distributed network server technology for the real-time data transfer of dynamic game network content. The OpenSkies Real-Time Intelligent Routing technology provides a low-cost, high performance solution that is easy to implement into games and simulations. Supporting this networking technology is a full open source simulation engine with network chat, terrain management, dynamics, and realistic rendering that is integrated fully with the OpenSkies communications model.

OpenSkies networking technology was built for Massive Multi-Player Gaming (>50,000 simultaneous players in applications that require first person shooter levels of performance). The network router/server technology that is the basis of OpenSkies is more broadly applicable to any dynamically changing, many-to-many communications application including:

• Massive applications caching.
• Internet-based classes.
• Stock market interactions.
• Chat rooms.
• Online gambling.
• Audio/video teleconferencing.

The majority of web traffic is composed of one-to-one or one-to-many interactions. However, the growth of the online gaming business and chat rooms has shown that there is an increasing demand for a more interactive experience on the Internet. This demand for interactivity has already grown to the point where the national networks are no longer able to handle the traffic and users are staying away due to poor performance. Handling the demand for an interactive Internet requires novel approaches to increase performance and provide a better quality of service to the end users.

Cybernet's OpenSkies Real Time Intelligent Routing Technology meets this need. The technology provides a solution for increased performance where there are many-to-many interactions in real-time. Cybernet is focused on bringing this technology to the online gaming business. By providing intelligent routing capabilities that service non-cacheable requests across the Internet, we can provide a faster and better quality of service to the end user in many-to-many interactions.

Cybernet's technology takes the concept of caching to the next level. While caching distributes content across the Internet to reduce load on a web server, intelligent routing technology reduces the redundancy of real-time requests across the backbone by putting application-specific information in the router. Game clients connect to one of the distributed OpenSkies routers and the routers then act as traffic cops between the clients. In the case of distributed interactive gaming, users' requests for data from all of the other players' data creates an N-squared problem of data transfer across the Internet. By creating a shared network of intelligent router and providing application-specific message culling, OpenSkies technology removes the N-squared redundancy and reduces the bandwidth load by at least 1/N or 25% to 90%.

The OpenSkies architecture provides the ability to substantially reduce the overall bandwidth to the client by culling out data that the client does not necessarily need based on application-specific rules. Two examples:

• In a distributed computer game, if one player cannot interact with another (because of being out of range or out of visibility), messages from the second player need not be relayed to the first. Since the router system knows the position state of both players and has game-specific mutual interaction rules internalized, it can block and not relay most player-to-player messages which would be discarded if forwarded.
• Interactive TV systems provide the ability to watch a sporting event from multiple cameras. Typically, each of these video streams are sent to the user at once and the user chooses which stream to watch at the client. Router-internalized culling rules can determine which stream a specific user is actually watching and forward from the video server only the data for the user's selected stream. Assuming 10 different camera angles and a client who has selected only one to watch, a 90% bandwidth reduction to the client is possible.

Technology

Cybernet's OpenSkies Real Time Intelligent Routing technology uses a software-based system running across a distributed server network. The components of the system are:

• Client API -- The communications library you use with your client game or simulation. Versions are available for Win32 (Windows 98/ME and Windows NT/2000) and Linux (portable to most Unix operating systems). This API is HLA interoperable with Cybernet extensions.
• Lobby Manager -- This is an application that allows a game/simulation client (through its API) to be connected to the shared routing network.
• Federation Host (FedHost) -- This is an application which accepts client connections and routes messages from clients to each other based on applications-specific routing rules.
• Game Server(s) and Web Sites -- Each game (i.e. federation) has at least one game server that is the first "player" that uses the Lobby Manager to establish a game (i.e. a federation). The Game Server sets the game-specific rules as the federation's creator. While game servers are game-specific, they utilize the same client API as do normal game clients (i.e. the game binary you provide to the players). Cybernet provides example game servers and associated web site programming to jump start your development efforts.
  The intelligent routing software actually recognizes what traffic is going where and who is subscribing to it. So instead of a 100-player game having 10,000 different streams between the players, our network consolidates these streams across the network and can reduce the overall bandwidth by an order of magnitude.

The technology behind this service is based on decades of research by the U.S. military, which has been the predominant researcher in multi-player simulations for the past 20 years. The U.S. military has developed a distributed network software architecture called High Level Architecture that provides for a much more scalable network system than any previous or currently developed system. By adapting this architecture, we have created a patented distributed server system that supports massive multi-player games that would otherwise be impossible. In addition, this technology will allow us to meet the bandwidth needs of many other types of multi-participant interactions such as video teleconferencing.

Supporting Technology: OpenSkies Game Engine Features

The OpenSkies game engine is open for your use and modification, but Cybernet retains ownership rights of the original game engine and any such modifications. You are required, upon receipt, to push all such modifications back to us. We request that you inform us of significant modifications and bugs found. You are required to get a commercial license from us for uses that generate income to you.

Example Code:

• HelloWorld Example.
• OpenSkies Virtual Training Environment System.
• Edge of Extinction Game Client.
• Edge of Extinction Game Server.
• Edge of Extinction Web Site.

Graphics:

• OpenGL hardware accelerated graphics with 16-bit color.
• Screen resolutions of up to 1600x1200 with full graphics and frame rate - up to 30 Hz, depending on graphics hardware.
• Real 3-D instruments created from real photography. Supports HMDs and shutter glasses.
• Multiple view support - You can have multiple out the window views or make a whole room your simulation center.
• Moving map display for tracking that uses real digital maps.
• Open Object Representations: VRML, OpenGL, USGS DLG (roads, land use, etc.), DEM (terrain), SCASM airport data, many existing flight simulation formats. Post your scenery to the OpenSkies community free.
• Dynamic culling of terrain for maximum performance.

Open Source:

• Loadable Simulation Modules: Dynamically loaded libraries (DLLs), based on open C++ source code that describes behavior and appearance of all simulated objects. Join a collaborative environment and post your contributions to the OpenSkies community.
• Real Aircraft Behavior Models: T34 single engine aircraft and TH57 helicopters. Unreal Models: Edge of Extinction vehicle at www.edgeofextinction.com / features / screens.shtml.
• Realistic cause/effect scenarios supporting real military training exercises. You can create your own adventures and share them with the community.
• Real Audio: Supports DirectSound and prerecorded ATC. Vehicle to Vehicle text and audio chat.
• Programmable interactions between objects. You can implement collisions, damage and even more complex AI interactions over the internet between players and vehicles.
• Create your terrain based on USGS data or other terrain formats.
• Cool force feedback that you can't ignore from the real flight models. Works on all DirectX compatible devices including I-Force® controllers such as CH Products®, Logitech®, and ThrustMaster® devices as well as Microsoft® force feedback devices.
• Weather cells with wind layers are supported for realistic conditions.

Multi-Player Networking:

• Communications protocols: Supports the military's High Level Architecture (HLA) standard for simulation networking and OpenSkies massive multi-player capability across the Internet.
• Promotes interoperability between different simulations. The HLA communications system API and binaries are freely distributed - no licensing fees required.

The OpenSkies graphics engine runs on standard PC Platforms using low cost hardware accelerated graphics, specifically under Windows 95/98/ME/NT4/2000.

The OpenSkies network engine runs under Windows 95/98/ME/NT4/2000 and Linux (portable to most Unix systems).
For more information go to www.openskies.net, www.edgeofextinction.com, and www.cybernet.com or contact Cybernet Systems, 727 Airport Boulevard, Ann Arbor, Michigan 48108, 734-668-2567.

References

OpenSkies High Level Architecture Run-Time Infrastructure SDK, Cybernet Systems Inc., 2000.

OpenSkies Virtual Training Environment System, Corporate and Product Background, Cybernet Systems Inc., 1999.

All OpenSkies materials are copyright, Cybernet Systems Corp.
Copyright Notice and License

OpenSkies Basics

Introduction - An introduction to OpenSkies.

Scalability White Paper - A demonstration of OpenSkies' scalability.

OpenSkies Presentation - Web-formatted version of the OpenSkies Presentation that was given at GDC 2002.

OpenSkies Detailed

OpenSkies MMPOG - Description of the OpenSkies Massive-Multiplayer SDK at a high level.

OpenSkies Network Architecture - Description of the communication between the various OpenSkies modules in more detail.

More documentation is available as a registered download.