Stuttering when generating and moving x position of sprites on screen.

[ZYandu]

Greetings! 

I am working on a horizontal scrolling rhythm game in React that uses Pixi for generating sprites. I want to use PIXI.ticker.elapsedMS to determine when a new sprite should be created and how far it should move its x position to stay in time with the song. 

My issue is that ticker.elapsedMS is taking longer than the expected millisecond value for a single frame at 60 fps. Is this expected behavior for ticker.elapsedMS to fall behind in a simple example like this?

import React, { PureComponent } from "react";
import { css } from "react-emotion";
import * as PIXI from "pixi.js";
import "pixi-layers";

export default class PixiHighway extends PureComponent {
  constructor(props) {
    super(props);
    this.pixi_container = null;
    this.expectedFPS = 60;
    //assuming we're at 60 FPS
    this.expectedMSPerFrame = 1/this.expectedFPS*1000 //roughly 16.67

    this.app = new PIXI.Application({
      width: 1200,
      height: 800,
      transparent: true,
      backgroundColor: 0xffffff
    });
    this.app.stop();

    this.app.stage = new PIXI.display.Stage();
    this.app.stage.group.enableSort = true;

    this.startSongTime = 0;
  }

  updatePixiCnt = element => {
    // the element is the DOM object that we will use as container to add pixi stage(canvas)
    this.pixi_container = element;
    //now we are adding the application to the DOM element which we got from the Ref.
    if (this.pixi_container && this.pixi_container.children.length <= 0) {
      this.pixi_container.appendChild(this.app.view);
    }
  };

  update = delta => {
    this.now = new Date().getTime();
    const songElapsedTime = this.now - this.startSongTime;

    if (this.app.ticker.elapsedMS > this.expectedMSPerFrame) {
      console.log(
        "WARNING Under 60fps! elapsedMS:",
        this.app.ticker.elapsedMS,
        "deltaTime:",
        this.app.ticker.deltaTime,
        "Dropped frame this far in ms through the song:",
        songElapsedTime
      );
    }
  };

  startTheTicker = () => {
    this.app.renderer.plugins.prepare.upload(this.app.stage, () => {
      this.app.start();
    });
    this.startSongTime = new Date().getTime();
    this.app.ticker.add(this.update);
  };

  render() {
    return (
      <div>
        <button
          className={css`
            position: absolute;
            top: 900px;
          `}
          onClick={this.startTheTicker}
        >
          Start animation
        </button>

        <div
          className={css`
            position: absolute;
            z-index: 2;
          `}
          ref={element => {
            this.updatePixiCnt(element);
          }}
        />
      </div>
    );
  }
}

 

 

Here's the result of the console.log when this.app.ticker.elapsedMS > this.expectedMSPerFrame. Is the log just slow or is my ticker being run incorrectly? 

Thanks for your time!

[ivan.popelyshev]

Its more about RAF than about pixi ticker. Use https://github.com/pixijs/pixi.js/wiki/v5-Custom-Application-GameLoop if you want to get rid of pixi ticker and try doing those things yourself

[ZYandu]

Ok, thanks Ivan I will try that out! ?

[ZYandu]

@ivan.popelyshev I ended up using the customizable RAF game loop setup and am having some stuttering issues with moving sprites across the screen. I recorded its behavior, please download the video to avoid google drive's bad preview showing extra stuttering. https://drive.google.com/file/d/1vi9H7pePHBTrufGvq4iQnCI7vB4YFqPz/view?usp=sharing (~25Mb file).

In my gameloop, I create arrow sprites and try to decrease their x position by a constant 8px per update call. Since deltaFrame is a bit unstable, I fix the sprites' x position when they need to be pushed forward to sync up to the song again. (All x positions are done in whole integers so I don't draw them at a decimal px position). This is the main code from the RAF gameloop showing the creation and movement of the AnimatedSprites and the console.logs:

 //Check for lost frames, if enough frames have dropped then force their x positions forward a bit to sync them up to the song again.
  update = () => {
    this.now = new Date().getTime();
    const songElapsedTime = this.now-this.startSongTime;

    let deltaTime = this.now - this.oldTime;
    this.oldTime = this.now;

    if (deltaTime < 0){
      deltaTime = 0;
    }
    if (deltaTime > 1000) {
      deltaTime = 1000;
    }

    const deltaFrame = deltaTime * 60 / 1000;
    
    //-1 to just get how far off a frame the update was
    if (deltaFrame < 10){
      this.cumulativeFrameSkip = this.cumulativeFrameSkip + (deltaFrame - 1);
    }

    //console.log out if deltaFrame is having trouble being close to deltaFrame = 1
    if(deltaFrame > 1.4){
      console.log("deltaFRAME was slow: ", deltaFrame, "how far into the song in ms:", songElapsedTime, "deltaTime:", deltaTime);
    }
    if(deltaFrame < 0.8){
      console.log("deltaFRAME was fast: ", deltaFrame, "how far into the song in ms:", songElapsedTime, "deltaTime:", deltaTime);
    }

    let amtOfFramesToSkipXPositionBy = 0;
    if(this.cumulativeFrameSkip >= 2){
      amtOfFramesToSkipXPositionBy = Math.floor(this.cumulativeFrameSkip); //floor to keep x position in whole numbers when multiplying with this later
      console.log("I shifted the arrows on screen so you should see a jump in position. amtOfFramesToSkipXPositionBy:", amtOfFramesToSkipXPositionBy);
    }

  //Generate arrows at the edge of the screen as you go through the song
    Object.keys(this.songData).forEach(key => { 
      if (
        this.now - this.startSongTime >= this.masterArray[key].timingWindow[0] * 1000 - this.msToGetToTimingWindowFromWindowEdge 
          && this.masterArray[key].timingWindow[0] * 1000 - this.msToGetToTimingWindowFromWindowEdge >= 0
      ) {
        this.createNewAnimatedSprite(
          this.props.highwayWidth,
          150,
          arrayOfPngNames,
          0.15,
          key,
          this.masterArray[key].strumType,
        );
        
        delete this.songData[key];
      }
   });
    
  //Move each arrow sprite, delete them far off screen
    this.preJudgedContainer.children.forEach(sprite => {
      if (sprite.x <= this.distanceRightTheTimingWindowIs && this.masterArray[parseInt(sprite.name)] !== undefined && this.masterArray[parseInt(sprite.name)].judged == false) {
        this.masterArray[parseInt(sprite.name)].judged = true;
        sprite.alpha = 0;  
        
        sprite.position.set(
          sprite.x - this.changeInHighwayPixelsPerFrame - (amtOfFramesToSkipXPositionBy * this.changeInHighwayPixelsPerFrame),
          sprite.y
        );
      } 
    
      else if (sprite.x <= this.distanceRightTheTimingWindowIs ){
        sprite.alpha = 0;
      }
      //kill sprites off screen so that it won't show the pixi redraw shifting children
      else if (sprite.x <= -500) {
        sprite.destroy();
        this.preJudgedContainer.removeChild(sprite);
      }
      else{
        sprite.position.set(
          sprite.x - this.changeInHighwayPixelsPerFrame - (amtOfFramesToSkipXPositionBy * this.changeInHighwayPixelsPerFrame),
          sprite.y
        );
      }
    });

...
    
    this.renderer.render(this.stage);
    if(this.cumulativeFrameSkip >= 2){
      this.cumulativeFrameSkip = 0;
    }
    requestAnimationFrame(this.update);
  };

The deltaFrame seems to have a lot of issues right after starting the song for about 8-10 seconds and then stables out to be closer to a value of 1. 

Here is the full code I am using in React to create the sprites and move their position: https://gist.github.com/IsaacVraspir/6679b0329178c555b317c6d99bd56c3e

My main concern is that the sprites stutter even when I'm not trying to correct their position and after deltaFrame has smoothed out. (You can see this around the 1:00 mark of the video recording - the console.log doesn't say that I shift x position but the sprites still stutter). I appreciate any advice on what I'm doing wrong! Thank you!

[ZYandu]

After doing some investigations in the performance tab of the DevTools, I think I can see the calls that are causing the fps drops. 

First there are the flush (SpriteRenderer.js:258), bindTexture (WebGlRenderer.js:590), updateTexture (TextureManager.js:89), Texture.upload (GLTexture.js:76), and textImage2D calls that happen together which takes a long time. These look like they get called a bunch at the beginning of starting the animation.

After those calls stop, some of the other stutters can be caused by composite layers which I think comes from chrome's rendering engine.

Let me know if there are ways to increase performance on these, thanks!

[ivan.popelyshev]

updateTexture, Texture.upload, texImage2D is the texture upload. Try "prepare" plugin to .. prepare all your resources and upload them to videomemory.

http://pixijs.download/dev/docs/PIXI.prepare.html

Also be aware that if any BaseTexture didnt appear on screen for 2 minutes or so it'll be removed from videomemory due to pixi gc. You can configure gc somewhere in "renderer.textureGC" or in pixi settings. That thing exists because its not easy to dispose() textures when you dont need them.

[ZYandu]

Thank you, that advice helped get rid of some of the slowdowns! 

I came across this article which talks about RAF being run at the end of the frame instead of preparing for it at the beginning. I'm not using setTimeouts in my code but does Pixi use setTimeouts for updates? 

"Frameworks or samples may use setTimeout or setInterval to do visual changes like animations, but the problem with this is that the callback will run at some point in the frame, possibly right at the end, and that can often have the effect of causing us to miss a frame, resulting in jank." (https://developers.google.com/web/fundamentals/performance/rendering/optimize-javascript-execution).

I'm experiencing this behavior where it runs the RAF closest to the end of a frame for all updates: