Streaming latency (High AvgFrameMS)

steamopen Streaming

ValveSoftware/steam-for-linux#8415 · opened 2022-02-17 by drogeek · updated 2022-03-09 · 4 comments · github

Ddrogeek 2022-02-17 github

Your system information

Steam client version (build number or date): Feb 15 2022 22:39:48
Distribution (e.g. Ubuntu): Ubuntu 20.04.3 LTS
Opted into Steam client beta?: [Yes/No] Yes
Have you checked for system updates?: [Yes/No] Yes

The games are almost unplayable because of stuttering. I tried a bunch of stuff, such as lowering the resolution, trying with HEVC, using Steam client beta, or Steam link beta on the raspberrypi, … whatever I tried, I still have the problem.

I used to have high CPU usage because of libva not being installed for 32 bits #8413 and I thought that was certainly the problem, but now hardware acceleration is enabled.
I noticed that AvgConvertMS, AvgEncodeMS are high (around 22 ms each), and AvgFrameMS is super high (around 87ms)

AvgFrameMS corresponds to the "time to stream the frame" according to this page https://steamcommunity.com/groups/homestream/discussions/0/540733523404402134/
Does that mean it's a networking problem?
I don't see what to do more to debug this.

Here are the system info and the streaming log

Additionally I don't have the problem when streaming the desktop

Ddrogeek 2022-02-17 github

actually, I get better frame rate when not using hardware encoding.
I thought that could be because the GPU is too busy, and therefore doesn't have the resources to encode the frame in time, but I tried on a very simple game TowerFall Ascension which isn't GPU intensive at all, and I still see hardware encoding being slower than software encoding, which isn't supposed to be the case, right?

{
	"GameNameID"		"TowerFall Ascension"
	"appid"		"251470"
	"TimeSubmitted"		"1645101780"
	"ResolutionX"		"1920"
	"ResolutionY"		"1080"
	"CaptureDescriptionID"		"Game Delayed OpenGL NV12 + libx264 main (4 threads)"
	"DecoderDescriptionID"		"Raspberry Pi MMAL hardware decoding"
	"BandwidthLimit"		"30000"
	"FramerateLimit"		"0"
	"Transport"		"1"
	"SlowGamePercent"		"0"
	"SlowCapturePercent"		"0"
	"SlowConvertPercent"		"0"
	"SlowEncodePercent"		"0"
	"SlowNetworkPercent"		"0"
	"SlowDecodePercent"		"0"
	"SlowDisplayPercent"		"0"
	"AvgClientBitrate"		"105.543342590332031"
	"StdDevClientBitrate"		"3.2127537727355957"
	"AvgServerBitrate"		"11250.4033203125"
	"StdDevServerBitrate"		"0"
	"AvgLinkBandwidth"		"100000.0078125"
	"AvgPingMS"		"0.110626220703125"
	"StdDevPingMS"		"0.0923615396022796631"
	"AvgCaptureMS"		"0.795611798763275146"
	"StdDevCaptureMS"		"0.949921667575836182"
	"AvgConvertMS"		"0.0150469895452260971"
	"StdDevConvertMS"		"0.202697455883026123"
	"AvgEncodeMS"		"7.07765626907348633"
	"StdDevEncodeMS"		"1.64562606811523438"
	"AvgNetworkMS"		"2.27109146118164062"
	"StdDevNetworkMS"		"2.01673507690429688"
	"AvgDecodeMS"		"0.403192967176437378"
	"StdDevDecodeMS"		"0.261025935411453247"
	"AvgDisplayMS"		"6.23334980010986328"
	"StdDevDisplayMS"		"1.25721156597137451"
	"AvgFrameMS"		"17.1819114685058594"
	"StdDevFrameMS"		"3.7683408260345459"
	"AvgFPS"		"61.1482467651367188"
	"StdDevFPS"		"11.3788251876831055"
	"BigPicture"		"1"
	"KeyboardMouseInput"		"0"
	"SteamControllerInput"		"1"
	"TouchControllerInput"		"0"
	"GameControllerInput"		"0"
	"XBox360ControllerInput"		"0"
	"XBoxOneControllerInput"		"0"
	"PS3ControllerInput"		"0"
	"PS4ControllerInput"		"0"
	"OtherControllerInput"		"0"
	"GameControllerVID"		"10462"
	"GameControllerPID"		"4354"
	"WasSpectating"		"0"
	"RemotePlayTogether"		"0"
}

{
	"GameNameID"		"TowerFall Ascension"
	"appid"		"251470"
	"TimeSubmitted"		"1645101722"
	"ResolutionX"		"1920"
	"ResolutionY"		"1080"
	"CaptureDescriptionID"		"Game Delayed OpenGL NV12 + VAAPI H264"
	"DecoderDescriptionID"		"Raspberry Pi MMAL hardware decoding"
	"BandwidthLimit"		"30000"
	"FramerateLimit"		"0"
	"Transport"		"1"
	"SlowGamePercent"		"0"
	"SlowCapturePercent"		"0"
	"SlowConvertPercent"		"0"
	"SlowEncodePercent"		"0"
	"SlowNetworkPercent"		"0"
	"SlowDecodePercent"		"0"
	"SlowDisplayPercent"		"0"
	"AvgClientBitrate"		"102.731941223144531"
	"StdDevClientBitrate"		"2.66320490837097168"
	"AvgServerBitrate"		"3335.197265625"
	"StdDevServerBitrate"		"0"
	"AvgLinkBandwidth"		"100000.0078125"
	"AvgPingMS"		"0.0331878662109375"
	"StdDevPingMS"		"0.0198161005973815918"
	"AvgCaptureMS"		"0.751250267028808594"
	"StdDevCaptureMS"		"0.548702478408813477"
	"AvgConvertMS"		"0.0980933010578155518"
	"StdDevConvertMS"		"1.05397999286651611"
	"AvgEncodeMS"		"17.925048828125"
	"StdDevEncodeMS"		"1.09837198257446289"
	"AvgNetworkMS"		"-0.0639836490154266357"
	"StdDevNetworkMS"		"1.38921761512756348"
	"AvgDecodeMS"		"0.218255296349525452"
	"StdDevDecodeMS"		"0.190050080418586731"
	"AvgDisplayMS"		"6.59519529342651367"
	"StdDevDisplayMS"		"1.30409049987792969"
	"AvgFrameMS"		"35.8760490417480469"
	"StdDevFrameMS"		"5.85877561569213867"
	"AvgFPS"		"53.7252693176269531"
	"StdDevFPS"		"10.8845491409301758"
	"BigPicture"		"1"
	"KeyboardMouseInput"		"0"
	"SteamControllerInput"		"1"
	"TouchControllerInput"		"0"
	"GameControllerInput"		"0"
	"XBox360ControllerInput"		"0"
	"XBoxOneControllerInput"		"0"
	"PS3ControllerInput"		"0"
	"PS4ControllerInput"		"0"
	"OtherControllerInput"		"0"
	"GameControllerVID"		"10462"
	"GameControllerPID"		"4354"
	"WasSpectating"		"0"
	"RemotePlayTogether"		"0"
}

Ddrogeek 2022-02-17 github

I did the experience to stream remotely instead of locally with a friend in another country (France<->Germany)

without hw acceleration

   {
   6     "GameNameID"        "TowerFall Ascension"
   7     "appid"     "251470"
   8     "TimeSubmitted"     "1645122373"
   9     "ResolutionX"       "1920"
  10     "ResolutionY"       "1080"
  11     "CaptureDescriptionID"      "Game Delayed OpenGL NV12 + libx264 main (4 threads)"
  12     "DecoderDescriptionID"      "DXVA2 hardware decoding"
  13     "BandwidthLimit"        "15000"
  14     "FramerateLimit"        "0"
  15     "Transport"     "4"
  16     "SlowGamePercent"       "0"
  17     "SlowCapturePercent"        "0"
  18     "SlowConvertPercent"        "0"
  19     "SlowEncodePercent"     "0"
  20     "SlowNetworkPercent"        "1.87628412246704102"
  21     "SlowDecodePercent"     "0"
  22     "SlowDisplayPercent"        "0"
  23     "AvgClientBitrate"      "177.813446044921875"
  24     "StdDevClientBitrate"       "6.39913129806518555"
  25     "AvgServerBitrate"      "6588.1259765625"
  26     "StdDevServerBitrate"       "0"
  27     "AvgLinkBandwidth"      "2714.254638671875"
  28     "AvgPingMS"     "44.8061599731445312"
  29     "StdDevPingMS"      "1.71839070320129395"
  30     "AvgCaptureMS"      "0.676889419555664062"
  31     "StdDevCaptureMS"       "0.112125121057033539"
  32     "AvgConvertMS"      "0.0381121151149272919"
  33     "StdDevConvertMS"       "0.825083732604980469"
  34     "AvgEncodeMS"       "7.14724397659301758"
  35     "StdDevEncodeMS"        "0.806647062301635742"
  36     "AvgNetworkMS"      "32.0165176391601562"
  37     "StdDevNetworkMS"       "5.93080759048461914"
  38     "AvgDecodeMS"       "0.202188372611999512"
  39     "StdDevDecodeMS"        "0.0298398099839687347"
  40     "AvgDisplayMS"      "0.517972648143768311"
  41     "StdDevDisplayMS"       "0.566476643085479736"
  42     "AvgFrameMS"        "40.4625930786132812"
  43     "StdDevFrameMS"     "6.29761171340942383"
  44     "AvgFPS"        "65.0555419921875"
  45     "StdDevFPS"     "22.7209148406982422"
  46     "BigPicture"        "0"
  47     "KeyboardMouseInput"        "0"
  48     "SteamControllerInput"      "0"
  49     "TouchControllerInput"      "0"
  50     "GameControllerInput"       "1"
  51     "XBox360ControllerInput"        "1"
  52     "XBoxOneControllerInput"        "0"
  53     "PS3ControllerInput"        "0"
  54     "PS4ControllerInput"        "1"
  55     "OtherControllerInput"      "0"
  56     "GameControllerVID"     "1356"
  57     "GameControllerPID"     "2508"
  58     "WasSpectating"     "0"
  59     "RemotePlayTogether"        "1"
  60 }

with hw-acceleration

{
 478     "GameNameID"        "TowerFall Ascension"
 479     "appid"     "251470"
 480     "TimeSubmitted"     "1645122413"
 481     "ResolutionX"       "1920"
 482     "ResolutionY"       "1080"
 483     "CaptureDescriptionID"      "Game Delayed OpenGL NV12 + VAAPI H264"
 484     "DecoderDescriptionID"      "DXVA2 hardware decoding"
 485     "BandwidthLimit"        "15000"
 486     "FramerateLimit"        "0"
 487     "Transport"     "4"
 488     "SlowGamePercent"       "0"
 489     "SlowCapturePercent"        "0"
 490     "SlowConvertPercent"        "0"
 491     "SlowEncodePercent"     "0"
 492     "SlowNetworkPercent"        "1.0721515417098999"
 493     "SlowDecodePercent"     "0"
 494     "SlowDisplayPercent"        "0"
 495     "AvgClientBitrate"      "189.340194702148438"
 496     "StdDevClientBitrate"       "29.4780368804931641"
 497     "AvgServerBitrate"      "5022.1845703125"
 498     "StdDevServerBitrate"       "149.266738891601562"
 499     "AvgLinkBandwidth"      "3842.017578125"
 500     "AvgPingMS"     "57.3839454650878906"
 501     "StdDevPingMS"      "16.263702392578125"
 502     "AvgCaptureMS"      "0.875325441360473633"
 503     "StdDevCaptureMS"       "0.294794082641601562"
 504     "AvgConvertMS"      "0.308913946151733398"
 505     "StdDevConvertMS"       "2.28143119812011719"
 506     "AvgEncodeMS"       "17.9259090423583984"
 507     "StdDevEncodeMS"        "1.12220668792724609"
 508     "AvgNetworkMS"      "37.8667373657226562"
 509     "StdDevNetworkMS"       "27.0522651672363281"
 510     "AvgDecodeMS"       "0.227412819862365723"
 511     "StdDevDecodeMS"        "0.297722339630126953"
 512     "AvgDisplayMS"      "0.568129241466522217"
 513     "StdDevDisplayMS"       "1.39200818538665771"
 514     "AvgFrameMS"        "61.6312904357910156"
 515     "StdDevFrameMS"     "11.8504066467285156"
 516     "AvgFPS"        "55.1290359497070312"
 517     "StdDevFPS"     "16.6929473876953125"
 518     "BigPicture"        "0"
 519     "KeyboardMouseInput"        "0"
 520     "SteamControllerInput"      "0"
 521     "TouchControllerInput"      "0"
 522     "GameControllerInput"       "1"
 523     "XBox360ControllerInput"        "1"
 524     "XBoxOneControllerInput"        "0"
 525     "PS3ControllerInput"        "0"
 526     "PS4ControllerInput"        "1"
 527     "OtherControllerInput"      "0"
 528     "GameControllerVID"     "1356"
 529     "GameControllerPID"     "2508"
 530     "WasSpectating"     "0"
 531     "RemotePlayTogether"        "1"
 532 }

Jjosepapaianni 2022-03-07 github

Your system information
Steam client version (build number or date): Mar 6 2022 00:19:25
Distribution (e.g. Ubuntu): Ubuntu 20.04.4 LTS
Opted into Steam client beta?: [Yes/No] Yes
Have you checked for system updates?: [Yes/No] Yes

Exactly same issue here, no problems when streaming desktop but once a game is running the latency jumps to ~85ms.

777boaz 2022-03-09 github

I had this same issue on the previous beta so I switched to stable and it was resolved. Now I guess stable has the issue because streaming to steam link was laggy with bad latency.. I run Manjaro with NVIDIA etc..

Nothing extracted yet.