protonscr

SteamCMD refuses to install on AWS EFS

steamclosed SteamCMD
ValveSoftware/steam-for-linux#4791 · opened 2017-01-02 by joshuacox · updated 2019-07-18 · 7 comments · github
Jjoshuacox 2017-01-02 github
  • Steam client version:
  • Distribution (e.g. Ubuntu): debian stretch
  • Opted into Steam client beta?: Yes
  • Have you checked for system updates?: Yes

Steps for reproducing this issue:

  1. install steamcmd on amazon EC2 instance
  2. " Steamcmd needs 250MB of free disk space to update."
  3. but there is plenty of space:
us-west-2a.fs-7537f4dc.efs.us-west-2.amazonaws.com:/  8.0E     0  8.0E   0% /home/steam  
Jjoshuacox 2017-01-02 github

My best guess is that this is because whatever way steamcmd determines the amount 'free space' is not working properly because EFS is somehow different from normal disk access.

It does appear to be driven by the NFSv4 mechanism inside of linux, here's the mtab from the mount point:

us-west-2a.fs-blah.efs.us-west-2.amazonaws.com:/ /home/steam nfs4 rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=
600,retrans=2,sec=sys,clientaddr=172.31.17.55,local_lock=none,addr=172.31.20.195 0 0 
Jjoshuacox 2017-01-04 github

this is possibly a perms issue as I have got it working now, but I am uncertain as to what went wrong in the first place. If I can determine a cause, and/or a fix, I will post back. Until then my only suggestion is liberal use of chown/chmod

Jjoshuacox 2017-01-05 github

I am getting issues with this again, it seems to crop up when installing arma3, if only errors if I am using the convoy-nfs or convoy-efs volume drivers. The steamcmd is being called like this:

bash /home/steam/steamcmd.sh +runscript /home/steam/steamer.txt 

with this in the steamer.t xt:

@ShutdownOnFailedCommand 1
@NoPromptForPassword 1
login REPLACEME_USERNAME REPLACEME_PASSWORD
force_install_dir /home/steam/serverfiles/arma3
app_update 233280 validate
quit          

and this is the resulting loop (it repeats continuously until I kill the process):

1/4/2017 8:28:36 PMRedirecting stderr to '/home/steam/Steam/logs/stderr.txt'
1/4/2017 8:28:36 PMILocalize::AddFile() failed to load file "public/steambootstrapper_english.txt".
1/4/2017 8:28:36 PM[  0%] Checking for available update...
1/4/2017 8:28:36 PM[  0%] Download Complete.
1/4/2017 8:28:36 PM[----] Verifying installation...
1/4/2017 8:28:36 PM[  0%] Downloading Update...
1/4/2017 8:28:36 PM[  0%] Checking for available update...
1/4/2017 8:28:36 PM[  0%] Download Complete.
1/4/2017 8:28:36 PM[----] Extracting package...
1/4/2017 8:28:36 PM[----] Extracting package...
1/4/2017 8:28:36 PM[----] Extracting package...
1/4/2017 8:28:36 PM[----] Installing update...
1/4/2017 8:28:36 PM[----] Cleaning up...
1/4/2017 8:28:36 PM[----] Failed to remove temporary unzip folder, continuing anyway
1/4/2017 8:28:36 PM[----] Update complete, launching...

The whole folder is owned by user 1000:

total 48K
drwxr-xr-x  7 1000 1000 4.0K Jan  4 20:29 .
drwxr-xr-x 25 1000 1000 4.0K Jan  3 20:08 ..
-rw-r--r--  1 1000 1000  220 Aug 31  2015 .bash_logout
-rw-r--r--  1 1000 1000 3.7K Aug 31  2015 .bashrc
drwxr-xr-x  2 1000 1000   54 Apr 22  2013 linux32
drwxr-xr-x  3 1000 1000   26 Jan  4 20:23 .local
drwxr-xr-x  3 1000 1000 4.0K Jan  4 20:29 package
-rw-r--r--  1 1000 1000  655 Jun 24  2016 .profile
drwxr-xr-x  3 1000 1000   26 Jan  4 20:23 serverfiles
drwxr-xr-x  2 1000 1000   19 Jan  4 20:29 Steam
-rwxr--r--  1 1000 1000  880 Feb  4  2013 steamcmd.sh
-rw-r--r--  1 1000 1000  170 Jan  4 20:27 steamer.txt
-rwxr--r--  1 1000 1000  14K Feb  5  2013 steam.sh
-rw-r--r--  1 1000 1000    0 Jan  4 20:23 .sudo_as_admin_successful

Any clue why steamcmd refuses to install on NFS and EFS?

Ggalexrt 2017-01-22 github

The issue may be that the filesystem uses 64-bit inodes. The steamcmd is using stat() instead of stat64 in that case making it fail.
@joshuacox Can you try running steamcmd with strace (args -ff -x) so I can see if the error is the same as mine?

Jjoshuacox 2017-01-25 github

I haven't gotten around to grabbing that strace it's in a docker container without all that installed. I can say that I am using XFS on one of the NFS where steamcmd.sh does fail for me in the same fashion as EFS does. My guess is that you are indeed correct and Amazon is using a 64 bit filesystem to back EFS (I would not be surprised to hear it is XFS).

Ggalexrt 2017-01-25 github

Thanks for your reply!
It seems that a 64 bit filesystem is used under the hood of EFS.
I created a separate issue for inode64 related problems, see #4824.

Jjoshuacox 2017-01-25 github

Thank you as well. I will close this issue and watch #4824

Nothing extracted yet.