How large are your integers and floats in your game?

[martensms]

Hey everyone,

 

OC thread at reddit (if you prefer using reddit ):

http://www.reddit.com/r/gamedev/comments/1bu8y5/how_large_are_your_integers_and_floats_used_in/

 

 

 

I'm currently optimizing the BiSON implementation a former colleague at Zynga did a while ago. (https://github.com/BonsaiDen/BiSON.js) The format is mostly a binary implementation of JSON, so it's optimized for WebSocket (or other TCP socket) usage. I'm rewriting parts of the format because I'm integrating it for my lycheeJS game engine and my V8-based OpenGL runtime behind. (If you are curious: http://martens.ms/lycheeJS)

 

Well, I have three questions for you:

 

1st Question: How large are your integers used in your (2d/3d/iso) games?

The current spec divides between different integer lengths, but that can be optimized - depending on how the scales of integers are. I'm currently thinking about if it makes sense to make a trade-off for large integers being always 32 Bits and therefore having more reserved bits (used as switches) available for smaller integers. I haven't built real 3d games with open worlds yet, so I'm curious how the placement of Entities is done there. I could imagine very large integers being used there (either that or relative integers to "map tiles/patches").

To explain what I mean by tweaking the bit lengths, the leading 3 bits are currently used for 7 different bit lengths of integers (or floats, which have a trailing 4 bit shift, so single precision).

number (size) -> bit length
0 - 1 -> 1 Bit
2 - 16 -> 4 Bits
17 - 256 -> 8 Bits
etc.

The different bit sizes are currently 1, 4, 8, 12, 16, 20, 24, 32 for both integers and floats.

The answers here should give me (in best case) hints about what scale ranges of integers and floats are used.

 

2nd Question: How much precision do you need for floats?

The current precision is only a single precision float, which is very bad. I'm thinking of more scale ranges for precisions to optimize it here (so most likely the trailing bits after the integer will be 3 or 4 bits to say: "that's the precision I need").

 

3rd Question: Do you want Dates and TimeStamps in JSON?

Also, I'm thinking of using more bits for the data types, so that client and server could register custom Objects and settings that are passed through. The goal would be having a snapshot-capable data serialization format where custom data types (like Date, lychee.game.Entity, game.AudioTrack etc.) can be stored efficiently. So you may be able to influence the data types known by the BiSON parser manually (maybe via setDataType(constructor, number) / setDataType(Date, 8))

What do you think of this idea?

[IvanK]

I am just wondering, is it a good idea to do what you do? I see, that you are trying to "compress" JSON, but your method is not fully compatible with JSON. Have you tried just to make ZIP compression? I think compressed data would be much smaller.

By the way, first, you should decide, whether you want to make lossy or lossless compression. At first, it looks like you are making lossless compression, but then you say that you want to round float values, which leads to data loss.

[Chris]

Ivan: We are talking about real time communication for multiplayer networking here. Constantly Zipping up your data before sending it and/or after recieving it comes with a high performance cost with only minor benefits in decreased size.

 

The original BiSON implementation didn't even do any validation of the input data for performance reasons. The author warns about that your game can fail horribly if you hand the wrong data over to the library.

 

Transporting data at a binary level is the most effective way you can go and honestly: nobody cares about a 100% covered JSON spec there.

[Ezelia]

the idea bihind BiSON is really good,
but I don't know if you really gain since native JSON encoding/decoding will be much faster than JS implementation of BiSON.
it depend a lot on the game constraints, if BW is an issue, how much data is exchanged ...etc
I experienced an issue with a multiplayer game latency, but after taking network traces ...etc I found that most loss was because of static content I was serving from nodeJS, so I put my NodeJS server behind varnish and everything is working just fine now


 

[martensms]

Well, my implementation of BiSON (will rename it due to the crappy unserious joke-implementation of Kai Jaeger) is meant to be a streaming format. So it allows being used via UDP sockets (in theory, you will need ordering for support of large objects, so pretty senseless for JSON-like apps) or via TCP sockets, which is a much better use case due to WebSockets.

 

The cool thing is that I switched the format headers yesterday. Now it allows a reset of the parser on lost network connection (if header field is 000 and value is 00, so 5 zero bits are required) and incremental updates of objects. Worst case is here if you have a-million-length-long string and the parser is currently parsing it - so you will have to send at least

 

    (length of string * 8) - (relative pointer position) + 5

 

Bits of 0 over the line. The topic here was more meant to figure out what size ranges of integers and floats are widely spread - so that I can tweak the 7 possible variants to have a better compression for smaller integers, for example.