Latency issues with Node JS game and VPS

[TheBoneJarmer]

Hi guys!

For a while now I am building a MMORPG in Node JS and HTML5. So far so good and things run fine on my laptop. But I am facing serious latency issues while running my game from my Virtual Private Server. For example, the ping locally is 0.01ms and the ping to my server is 30ms-40ms. A big difference but that difference is causing my game not to run smoothly at all. I figured I made a mistake with the way I communicate from the client to the server since I see a lot of different techniques. But I am facing the edge right now, I do not know what way to go and how to solve these latency issues. Google brought up some decent results but none that really helped me. I'll add my two most important files down here so you guys can have a look.

To give you an idea how I work, this is how I am doing it right now. On the VPS there are 4 Node JS apps running in the background. These are (in my terminology) a database server, a resource server, an authentication server and of course, the game server. I'll explain their functionality down here:

- The database server is a secured app that holds all game data. Like players, items, monsters, e.t.c. The server has several routes built in like /getallcharacters (fictive bytheway) which will return a JSON object of all characters in the game.

- The authentication server is also secured. It is seperated from the game because it was meant to be the only place where users can send their sign in data to. I based it on OAuth bytheway because it works similar. You send your username and password in a request and you'll get a token in return if your credentials are valid. It is also this token that can be used to return your account id. Which in return is being used by the data server to get specific account information. So with other words, when you request player-specific or even sensitive data from the data server, it will ask for your token instead of some number. This is done for security reasons.

- The resource server is just a place where I gather all images, audio and such. Those can be accessed from anywhere.

I really really hope anyone can help me with this. Oh and before anyone starts saying I should not just jump right into a MMORPG project. I got 10 years of programming experiences and I know my theories about networking and such. My own diagnoses are that I made some code design mistake. I just do not know what I can do to improve it. Any help would be really really appreciated!

-TheBoneJarmer

map.js

server.js

[Arcanorum]

10 years of programming experience, yet you still don't comment your code?

[TheBoneJarmer]

Yes I do. I just do not do it after each line because I see no reason to it. And did you opened up those source files? Because if you did you would notice there are actually comment lines. Just not that many.. And speaking of comments, how is your comment going to help me with this problem? If you cannot say anything useful, just do not say a thing at all please.

[away168]

This kind of question is always hard to answer in public forum as it varies to your server stack, your machine, your distance to your VPS, your VPS machine, your network connection (wifi or cable), your browser, your game, your server code, and more.

I suggest you try another language such as Java (with embedded Jetty for example) or Go language for the server-side, as Node.js is definitely slower than both, then do your own benchmark. You have as much programming experience as I am, and if I were you I'd already test in so many different environment. Unless, I have no choice but using Node.js which I don't see the reason why you have to stick with Node.js that is only good for start up and not performance, since we are talking about performance here.

Also note on your data going back and forth. OAuth is also the kind of "enterprise-trying-to-monopolize-standard" more than security so careful on how your data going back and forth just trying to live a standard, unless you implement your own kind of "Oauth" where you just borrow the simple access token technique, which may still take you some data going back and forth, just to keep up a standard you may not need.

You may also want to check your socket.io version (I see you are using Socket.IO) whether it has supported websocket++ or not as it really makes a difference (or so it says, I haven't tried it).

Of all these, I assume it might be just the VPS. What your VPS machine is or where your VPS located and where you are or where the user is affects things a lot. Remember that if you only use a super micro dang small whatever tiny VPS that has small RAM and processor, Node.js (and Java) needs to run on a virtual machine that eats up your memory by default (and it's not small), and it will make your server slow or killed. Some VPS already eat up your memory by default depending what you rent.

[TheBoneJarmer]

Thanks a lot! Yes, these thoughts crossed my mind as well. As for my VPS, it runs quite well in fact. Got 400GB space for a hard drive and 4 GB ram. I assumed this is enough. And it should be enough. Whether it is a performance issue is something I am not sure about. If I test the game locally on my VPS I got no problem at all. So I assumed it was the ping. Still, you are somehow confirming my fears concerning Node JS. There is nothing that requires me to use it, I just like to use it because of how simple it is. And I use it for all kind of things including games. But games require more than common apps. So the whole reason I stay with Node JS is because I use it for everything else.

That does not mean I am not familiar with other methods like C# and the System.Net namespace. These are common for me too. It is just that I win a lot of time with Node JS so well yeah.. I'll give this a thought. It might be I overestimated the power of Node JS and made some horrible mistakes here. Nevertheless not bad either, than I just learn from it and move on. If it means I got to quit my project than that is ok with me too. I was not planning to put more effort into it than I already am doing for I got not that much time. As for OAuth, I am using it in a system I am developing for my current employer.

[Jammy]

@TheBoneJarmer lets not look at the red herring here. There's nothing wrong with the server and 20-30ms is normal. If nodeJS couldn't do it's job then how are so many .io games running today. The problem is with your client code and this problem will exist until you resolve that. You need more reconciling between your client and servers state of the game. 30ms shouldn't be a problem, trust me - my overwatch ping is 30ms, my google ping is 30ms, just about any server i ping at 30ms+

[TheBoneJarmer]

Thanks for the reply! So what you are saying is that I should do something about the communication between the client and server, right? Where you suggest I start?

[Jammy]

@TheBoneJarmer theres an article here talking about it from a higher level: 

http://www.gamedonia.com/blog/lag-compensation-techniques-for-multiplayer-games-in-realtime

And I tried this google search and it came up with a few ready made libraries:

https://www.google.co.uk/search?q=javascript+lag+compensation&rlz=1C1MSIM_enGB703GB703&oq=javascript+lag+compensation&aqs=chrome..69i57.3154j0j7&sourceid=chrome&ie=UTF-8

[mattstyles]

What do you logs say? Where is it slow? Is it execution? Comms between services? Ping? Network latency?

10 years of experience should have provided you with all the opportunities to know exactly how to measure all of these things. Give us those measures for some typical communications you're having problems with and you'll get better answers of how/why things could be improved.

I disagree with Jammy, 30ms ping is poor for me sat here in London (but would hardly be a problem), the real quickies like google/bing etc return in a few ms, even my cruddy super-cheap digital ocean box hosted in Amsterdam returns in <10ms (and DO aren't super fantastic at treating boxes well). I'd have a look at your host, check the hosts forums for anyone else reporting slow response times. I'm assuming you've already done everything else like making sure the box is minimal etc etc, although 4g during development for a team of 1 sounds ridiculous, particularly as you're just running node, 1g would be more than enough for many node processes (you'll need considerably more for Java to run smoothly if you switched).

[Jammy]

@mattstyles if only we all lived in london, i guess its not normal for 30ms ping then. maybe you could run this test too

 

 

[mattstyles]

 

Makes me want to try this at home too (I live out in the sticks)

[Jammy]

11 hours ago, mattstyles said:

 

Makes me want to try this at home too (I live out in the sticks)

Thats pretty insane, i've never seen a ping lower than <10ms! Even when I'm working in the cities close to me, Preston, Leeds. I think you're even pinging 6ms to a french server. Little bit jelly.

[Langerz82]

The call to updateSessions is 0 so it's trying to run as fast as it possibly can. Try around 50ms or so.

Also just a note about the client, I had allot of packets getting received by the client and I solved it by not sending move packets that are not necessary for example out of the screen. I also implemented on the client a packet pump which basically is an array that pushes packets into it, then setting an interval of how long it approximately takes to process a few packets (24ms or so) by staggering the packet processing the FPS was much higher as it is given an opportunity to render the frame in-between.

Hope this helps.

btw I'm the sole programmer of MaEarth. =D

I'm not sure why you call this so often since it's only timing out session after a day. A better option may be setting a time stamp then comparing it with (1000 * 60 * 24) + Date.now(). I say may because I'm not making any assumptions as to what the purpose of elapsing the time is.

An elegant alternative is here:

https://jsfiddle.net/9fs5z8n7/9/

[TheBoneJarmer]

Thanks a lot! Yes, I see now where I sould go with this. It seems I still have a lot to learn. No matter, I think I will start over again but smaller projects this time so I can take my time learning how to NodeJS my way to an mmorpg one day.

[ForgeableSum]

Yeah, if your game can't handle 20-30MS ping, you've gotta rewrite some things on the client side because 20-30MS is actually a best-case scenario. My game (Strike Tactics) is in beta and even 300+ ping will not be noticeable (got players in Germany playing players in US with no problem). Sure, it's not ideal and things aren't 100% where they should be at 300+ ping, but everything still looks the same. I've learned that the perception of perfectly mirrored simultaneous running environments is the only thing more important than actual perfectly mirrored simultaneous running environments. Also, each player is the authority of his/her units (it's an RTS game), so no matter what the ping is, the game objects which you are looking at most of the time will have zero movement lag (my networking layer is completely non-deterministic). I've not had a single person complain about lag. I'm the only one who can tell the difference between a 50MS latency game and a 300MS latency game, and that's because i know what things to look for. 

So the secret is interpolation and clever client-side networking code.

[caymanbruce]

On 04/07/2017 at 8:55 PM, mattstyles said:

 

Makes me want to try this at home too (I live out in the sticks)

Wow those figures are pretty. I can't even ping google.com and html5gamedevs.com but I can still access the sites. Not sure what is wrong. But I think having a global CDN helps a lot.