Running Phaser on Node.js - How I did it and why you shouldn't do it

[Crisu83]

Hello All,

 

I am working on a multiplayer action role playing game and I am using Phaser on the client because I just love this framework.

 

Here is a link to the project on GitHub:

https://github.com/crisu83/dungeon-game/tree/feature/phaser-server

 

I have been experimenting quite a lot with running Phaser in headless mode on the server and I managed to get it to run with a few hacks.

 

I am not sure that it is a good idea to run Phaser on the server, but I am looking into this because I would prefer to have an authoritative server that runs on the same code base as my clients.

 

Here is what I did in order to get Phaser running on the server:

 

First I installed the latest stable version of Phaser through NPM by running the following command:

npm install http://github.com/photonstorm/phaser/tarball/v2.0.5

After that I installed the dependencies for Node Canvas, instructions for that can be found in the project wiki on GitHub:

https://github.com/LearnBoost/node-canvas/wiki

 

Next I installed node-canvas and jsdom through NPM. These modules are required in order to "fake" the document, window, canvas and image objects that Phaser depends on that all are available in all browsers, but not on Node.js.

npm install jsdomnpm install node-canvas

Then I wrote this wrapped module for Phaser:

https://gist.github.com/crisu83/5857c4a638e57308be4f

 

I know that this is a ugly hack, but it at least lets me run Phaser on the server.

 

Here is what the server currently outputs:

 

I am now wondering if I should attempt to make changes to Phaser itself to not rely on the document, window, canvas, image when running in headless mode and create a pull-request for the changes. I am sure it will not be easy to remove all those dependencies, if even possible.

 

Does anyone know if Richard has any plans for this? Does this even make sense to run Phaser on the server?

 

Please share your ideas and feel free to use my code for your own projects.

 

Thank you for reading.

[lewster32]

I'd say Phaser on the server is a bad idea because it ends up being highly wasteful and not efficiently targeted towards doing just what is needed. An authoritative server should complement but not replicate exactly the client code, and should in most cases also not run at 60hz - typically servers have a tick rate of 10-30hz and clients interpolate that information.

 

This is a fantastic article outlining the requirements and pitfalls associated with client/server architecture: https://developer.valvesoftware.com/wiki/Latency_Compensating_Methods_in_Client/Server_In-game_Protocol_Design_and_Optimization

[user123456789]

I guess I add my two cents.

 

My honest opinion is that I think you are over-complicating things - first I would ask what is the purpose that the server needs the phaser in the first place? - It doesn't need canvas or similar because it doesn't need to draw anything visible to the clients, it does need the update loop (for physics) - you can't live without it, but you need to make sure that it runs in sync with the clients, such as 30 times/s. The reason being, that once you start make more complicated things such as "client-side prediction" your client & server should end up to same result of calculations even if they are calculated at different point in time. And ofc. if they dont (because your server has authority) your client needs to accept the game states sent by the server. - Making multiplayer online games are damn hard.

 

I earlier used phaser and node.js (among socket.io & express) to program my first online multiplayer "test" which I released to heroku. I quickly realized that mixing these two together is hard, but possible. For instance, you need to think how you share your model codes, such as player.js with client & the server. In addition, if you are working with TCP - your game wont work properly with wireless devices because their connections has so much packet loss...(TCP will retransmit every lost packet) UNLESS you start using proper compensatory methods: e.g. client-side prediction, entity interpolation... extrapolation(?) etc. or choose to make a multiplayer game which are less sensitive to the latency.

 

All in all, after writing this text I reconsider my statement above, maybe the phaser has its place on the server-side - you could probably use phaser classes, such as sprites in a way that your models which you share. e.g. player.js is similar at both sides of the system. (client & server codes).

 

Finally, I think you should start off without the phaser at server-side... fix client-side phaser update() loop to run physics 30t/s and let render be as it is. After that, make server to do its own physics 30t/s and add network loop which sends packets to clients (e.g. 20times/s). You probably will have more work ahead, because you need to do registering/login/session ... player lobby/join/leave server etc. etc.

 

As a final note, I am currently doing research with a topic of "The effect of the internet on multiplayer game development" which will discuss most of the application-level techniques, architectures etc. related to multiplayer online game development (stuff which you need to be familiar with). I will tweet the final version later - maybe within two weeks.. good luck & follow me at twitter if you are interested to read it. Have spent around last 6 months to teach me this stuff, its fascinating topic 

[Crisu83]

This was pretty much the answer that I was expecting.

 

However, it would be great if we could get a build of phaser that can actually be loaded on Node.js so that we can build my own system on the server using Phaser classes.

 

Right now we cannot require the library because all builds depend on things that are not available on Node.js.

 

Any thoughts on this? Does it sound like an acceptance solution.

[user123456789]

This was pretty much the answer that I was expecting.

 

However, it would be great if we could get a build of phaser that can actually be loaded on Node.js so that we can build my own system on the server using Phaser classes.

 

Right now we cannot require the library because all builds depend on things that are not available on Node.js.

 

Any thoughts on this? Does it sound like an acceptance solution.

 

Yea, it definitely sounds tempting, but regarding the build etc. I think we currently can't get separate build (even at client-side) which opt-out stuff which we are not using. e.g. 75k lines are always everything which needs to be included & minified. Stuff such as boostrap etc. allows to make custom builds right at the browser. its pretty neat. Dunno, if that kind of solution is too far to develop (refactor) at this point. (same goes to server-side builds). (?)

[lewster32]

I don't think using Phaser directly is the answer. I think if we need a server-side library to complement Phaser, it should certainly use parts of Phaser, such as the physics systems, timers, events and so on, but it should be well thought out and doing so would probably also need to introduce some new abstractions to Phaser's client-side.

 

An example is that Phaser is based heavily on display objects. Sprites do not extend abstract entities with a position, velocity and so on - all useful things to have for the server-side library, but in Phaser these are things added to what is first of all a visual object in the scene. If writing this from scratch to be efficient, all objects would start with a bare bones non-display-based abstraction.

 

I see this as a separate but related project, not as a simple case of installing Phaser on the server - that just won't work in pretty much all cases without lots of hacking; and then you'd be left with something that's messy, difficult to maintain and performs poorly.

[Crisu83]

I don't think using Phaser directly is the answer. I think if we need a server-side library to complement Phaser, it should certainly use parts of Phaser, such as the physics systems, timers, events and so on, but it should be well thought out and doing so would probably also need to introduce some new abstractions to Phaser's client-side.

 

An example is that Phaser is based heavily on display objects. Sprites do not extend abstract entities with a position, velocity and so on - all useful things to have for the server-side library, but in Phaser these are things added to what is first of all a visual object in the scene. If writing this from scratch to be efficient, all objects would start with a bare bones non-display-based abstraction.

 

I see this as a separate but related project, not as a simple case of installing Phaser on the server - that just won't work in pretty much all cases without lots of hacking; and then you'd be left with something that's messy, difficult to maintain and performs poorly.

 

I agree on what you are saying but as far as I can tell Phaser's physics systems can already be used with custom objects. Maybe one solution would be using Phaser physics on the server with custom objects?

 

This would be a great start because physics and utility classes are pretty much all we can use from Phaser on the server.

[lewster32]

By all means don't let my nay-saying stop you, but I do strongly feel that a well-thought-out server-side library to complement Phaser would begin not with Phaser itself, but with a bare-bones framework that carefully implements certain Phaser components.

[Crisu83]

Considering everything discussed in this thread, does anyone have any ideas on how to implement the required functionality on the server. The reason why I'm asking is that I still will need to implement this in my project.

 

Or, would it be possible to simplify this and just maintain the state of the game objects (position, velocity, etc.) on the server and communicate that state to all the clients when something changes. This would mean that the server would just track the current state of the objects and not run any game loop itself. Does this make any sense?

[lewster32]

In tests I've done, I've gotten away with simply broadcasting the x and y positions of the objects, and on the client side interpolating the positions (basically using tweens) to hide the jerky motion from the low frequency updates. You could then have the server just watch out for anything weird, such as an object moving too far in a specified period of time and correct that as needed. This makes the calculations far lighter on the server side and allows it to step in if any funny business occurs. You can then build upon this base to start to include basic simulation of physics and expected events.

[Fishrock123]

Does this even make sense to run Phaser on the server?

 

Not really. There is some fancy stuff in phaser that I suppose would be nice, but since you're not dealing with rendering, most of it is pointless.

I run p2.js on the backend of my server for realtime physics and it works just fine.

My server entities inherit from p2 directly, so everything is nice and easy imo.

Here's what my server's entity class looks like: https://gist.github.com/Fishrock123/0ee0bd4cdec69f0822aa

@Crisu83 - are you looking at realtime stuff?

[Crisu83]

Not really. There is some fancy stuff in phaser that I suppose would be nice, but since you're not dealing with rendering, most of it is pointless.

I run p2.js on the backend of my server for realtime physics and it works just fine.

My server entities inherit from p2 directly, so everything is nice and easy imo.

Here's what my server's entity class looks like: https://gist.github.com/Fishrock123/0ee0bd4cdec69f0822aa

@Crisu83 - are you looking at realtime stuff?

 

This is exactly what I am looking for.

 

Thanks for the tip about using p2 directly, makes sense. Did you try other physics frameworks?

 

And yes, I'm looking at realtime updates as I'm working on an action rpg.

[Crisu83]

After reading your feedback I decided to remove phaser from the server (you can see the feature/simple-server branch).

 

I did some more research on this and came to the conclusion that I will write a simple game engine for the server that sends periodic updates to the client and use tweens to counter network latency issues and packageloss.

 

I'm not yet sure which physics engine I will use but I will do some research on that but right now I'm leaning towards P2.

 

I also looked into existing game engines that runs on-top of Node.js but I didn't really like any of them. If I were to use a ready-made engine it would need to be actively maintained and enable me to use Phaser on the client, because that framework is just brilliant.

 

Thanks for all the feedback, you have been a great help.

[lewster32]

Can I ask what it is in particular you need to model physics-wise on the server? To me even running P2 on the server seems overkill given the fact that network latency is going to make synchronisation of physics between the server and clients really difficult. Even simple physics calculations are highly sensitive to timing and initial conditions, and this has traditionally made such stuff very, very difficult indeed to translate into the always unpredictable network environment.

 

It seems to me that all you would need on pretty much any server is position, rotation and velocity, along with some state to handle what the entity is doing and maybe some meta to describe it. It's not then difficult to write your own simple integration methods on the server to derive expected positions from the position and velocity of entities and do some sanity checking. This would be more than adequate for most situations, and would then allow you to specify different types of entities; player entities would be controlled by clients and the server would just synchronise them with other clients, performing checks to ensure they're moving as expected. If you have projectiles, these would be created at a point, given a velocity and then the server would just run a simple velocity-based motion calculation and update all the clients with its position. 

 

All of this really pushes the simplicity aspect - you want your server doing the absolute bare minimum. On the client-side you can have physics calculations filling in the gaps; by all means have client-side only stuff like explosions sending debris bouncing around the world, particle systems creating fancy spell effects and so on, but this would all be happening over the top of a very simple bare-bones physics system running in the background purely driving essential gameplay features.

[CtlAltDel]

I hear people say a lot that P2 physics is heavy etc etc. Yes, it is heavier than using simple arcade physics. But running it on the server for your characters is just fine, in fact you should be running it on both server and client, and have both do the calculations. Just don't do it for particle effects etc. Just do it for gameplay objects.

 

Not p2 physics specific but an excellent list of articles explaining: http://www.gabrielgambetta.com/fast_paced_multiplayer.html

[lewster32]

Yeah that's an excellent article - definitely a must read for anyone interested in client-server stuff!

 

I have to say though I do disagree on the idea of running the same level of physics simulation on the server as on the client. It's just so difficult to keep things in sync, and the more fully featured and accurate the simulation is, the more difficult it gets, as timing and sensitivity to initial conditions become more important to the outcome of every single interaction. In 95% of cases a server should only need to know where entities are, estimate with reasonable accuracy where they'll be a short time in the future, and detect whether they're colliding (or about to collide) - finally stepping in only if the estimates don't closely match what's being sent over. The real magic and heavy lifting should be done on the client to make the relatively sparse but vitally important information from the server translate into something that's as smooth and natural looking as possible, because when it comes to networks, the client cannot rely on the server to be there for it in a consistent, timely manner every step of the way.

[Crisu83]

Can I ask what it is in particular you need to model physics-wise on the server? To me even running P2 on the server seems overkill given the fact that network latency is going to make synchronisation of physics between the server and clients really difficult. Even simple physics calculations are highly sensitive to timing and initial conditions, and this has traditionally made such stuff very, very difficult indeed to translate into the always unpredictable network environment.

 

It seems to me that all you would need on pretty much any server is position, rotation and velocity, along with some state to handle what the entity is doing and maybe some meta to describe it. It's not then difficult to write your own simple integration methods on the server to derive expected positions from the position and velocity of entities and do some sanity checking. This would be more than adequate for most situations, and would then allow you to specify different types of entities; player entities would be controlled by clients and the server would just synchronise them with other clients, performing checks to ensure they're moving as expected. If you have projectiles, these would be created at a point, given a velocity and then the server would just run a simple velocity-based motion calculation and update all the clients with its position. 

 

All of this really pushes the simplicity aspect - you want your server doing the absolute bare minimum. On the client-side you can have physics calculations filling in the gaps; by all means have client-side only stuff like explosions sending debris bouncing around the world, particle systems creating fancy spell effects and so on, but this would all be happening over the top of a very simple bare-bones physics system running in the background purely driving essential gameplay features.

 

You are correct. I also think that doing only what's absolutely necessary on the server is the right approach in my case (any probably also in most cases).

 

I want to prevent all possible networking issues possible before even bumping into them, what you said about synchronization made me re-think my approach.

 

Do you have any example on such a simple game server?

 

Also, how would you handle client-side only changes to position like shockwaves pushing back objects, would you allow the client to actually tell the server the new position of all the objects or?

 

Am I making any sense?

[lewster32]

I'm afraid I don't have any released examples for Phaser as they are works in progress and unfortunately not for public viewing, but I do intend to release some of my tests soon. I've been working on a long-term networked multiplayer project, I've conducted a lot of smaller tests using JavaScript and NodeJS, but my primary experience in this comes from testing in Unity after studying the source code of the Quake and Doom 3 engine releases. There are some very nice indepth source review articles of game engines here, which provide some really great insight into how the professionals approached these kinds of problems: http://fabiensanglard.net/

 

Shockwaves and other forces I see being handled discreetly using the same simple velocity calculations you'll have for predicting movement on the server. If you create a system whereby every entity has a position and a velocity, then a shockwave would be handled by the player telling the server 'I have applied a radial force at this location', then the server would calculate the velocity changes on all of the objects in the radius of that point. Meanwhile on your client-side you'd apply the same force in order for it to be shown instantly, and finally rely on the interpolation to smooth out any discrepancies due to lag etc. Finally to jazz it all up you'd have client-only effects such as debris, smoke, particles etc to emphasise the shockwave effect, and help mask any issues with lag.

[mamwalter]

Some interesting articles on multiplayer:

- http://buildnewgames.com/real-time-multiplayer/

- Glenn Fiedler Blog:

      - http://gafferongames.com/networking-for-game-programmers/what-every-programmer-needs-to-know-about-game-networking/

      - http://gafferongames.com/game-physics/networked-physics/

[Crisu83]

I hear people say a lot that P2 physics is heavy etc etc. Yes, it is heavier than using simple arcade physics. But running it on the server for your characters is just fine, in fact you should be running it on both server and client, and have both do the calculations. Just don't do it for particle effects etc. Just do it for gameplay objects.

 

Not p2 physics specific but an excellent list of articles explaining: http://www.gabrielgambetta.com/fast_paced_multiplayer.html

 

Thanks for this link, especially the example code found in the demo was quite useful and helped me to understand how to structure my logic on both the server as well as on the client.

 

I'm not done yet but I will push it to my project on GItHub once I get everything to work.

 

I'm now working on a bare-bones solution with as little logic on the server as possible just as lewster32 suggested, so no physics on the server for now at least.

[lewster32]

I've noticed this thread has been mentioned on the Phaser homepage - indeed there's some great reading in here, especially in the links that have been posted explaining what is an oft misunderstood part of game development.

 

For years now we've been taking for granted the magical ability for increasingly complex and interactive games to play over the Internet with little or no noticeable hitches - the huge effort that's gone into this has been largely ignored because quite simply no-one wants to think about how genuinely difficult it is even to make two players bump into one another online without everything going nuts. I think if more people understand this and create their own implementations, it makes the ecosystem richer and the problem easier to tackle.

 

I for one am really excited to see what this community with fresh minds, an end-to-end JavaScript architecture and new excellent, easy to use frameworks like Phaser and pixi on the client-side, socket.io on the server-side and even peer-to-peer in the likes of PeerJS (this one really deserves some attention - I've been using this recently and it's excellent) mean we stand at the cutting edge of what's possible, and using absolutely relevant cutting edge technology to do it

[user123456789]

Thanks for this link, especially the example code found in the demo was quite useful and helped me to understand how to structure my logic on both the server as well as on the client.

 

Yea, good links.. I personally liked most gabriel gambettas explanations, (source multiplayer networking - valve articles) and quake source code (client-side / server-side game loops etc.).

 

Moreover, I like to say that the thing which surprised me most when working on my first multiplayer online game (MOG) (tests) - was the fact that developing MOG is as hard as the complexity of your game design. Namely, your MOG sounds rather hard to implement if it is the first MOG you are going to make. By all means, I am not saying that you are not a good programmer - I am saying that some months ago I was being naive and though that I could implement my first MOG within a gamejam which lasted only 48 hours; I failed completely.

 

Therefore, based on stuff which I have read and tried I can say that the implementation of MOG is hardest if you are making a fast-phased MOG (like FPS or action games (maybe even action rpg )) where timeliness matters the most. The complexity is further increased if you have much player-to-player or player-to-npc interactions. On the other hand, I believe that MOG such as Heroes Of Might And Magic (HOMM) is perfect example of a MOG which could be a good starting point for MOG development. The reason being that HOMM doesn't have parallel execution - every player takes turns whereas state of the game can still be sent from the server to other players which doesn't have to look a black screen. (They can follow-up, but not interact).

 

I know that these comments might not be directly related to this topic, but I noticed that multiple people are reading this so wanted to share my own experience and warn people beforehand 

[lewster32]

Can I go on-record as saying that I'm also developing a turn-based game and while theoretically easier, it's still very difficult to get right!

[Crisu83]

First of all I'd like to say that it's awesome that this thread was featured on Phaser.io. Thanks for that Rich!

 

After having read all the resources linked here plus many more I don't think what I'm building is a too big challenge, but it remains to be seen. I'll spend a bit more to see if I can get the infrastructure to work (server -> clients), if not I'll think of something else.

 

Here's a few projects form Bonsai Den which you can use directly or use as a reference:

- https://github.com/BonsaiDen/NodeGame-Shooter

- https://github.com/BonsaiDen/Maple.js

[Fishrock123]

(For the record, I didn't see this 2nd page when I wrote this.) 

 

I have to say though I do disagree on the idea of running the same level of physics simulation on the server as on the client.

 

 

Nope. It must be at least as accurate to be correctly authoritative.
 

It's just so difficult to keep things in sync, and the more fully featured and accurate the simulation is, the more difficult it gets, as timing and sensitivity to initial conditions become more important to the outcome of every single interaction.

 

 

And that is why real-time multi-player is *hard*. You have to sync these things, the server needs to be authoritative to prevent cheating.

 

In 95% of cases a server should only need to know where entities are, estimate with reasonable accuracy where they'll be a short time in the future, and detect whether they're colliding (or about to collide) - finally stepping in only if the estimates don't closely match what's being sent over.

 

 

That is Physics. To properly know where a body will be after it collides you need to do physics on the server.

 

The real magic and heavy lifting should be done on the client to make the relatively sparse but vitally important information from the server translate into something that's as smooth and natural looking as possible ...

 

 

That is your rendering and interpolation. You still need the stuff on the server in realtime multiplayer.

 

... because when it comes to networks, the client cannot rely on the server to be there for it in a consistent, timely manner every step of the way.

 

 

Right, that is another reason this is difficult. Your extrapolation and interpolation needs to take this into account and provide you with a decent guess.