See related: https://github.com/boostorg/filesystem/issues/184

Thanks for reporting this.

I think this may have been a mistake:

        __syscall__copy_file_range = prototype(('syscall', CDLL(None)))

CDLL provides a use_errno argument, so it probably should've been:

        __syscall__copy_file_range = prototype(('syscall', CDLL(None, use_errno=True)))

Hopefully that'll allow us to get a real errno instead of 0.

If it's EINVAL, as that boostorg issue implies, we should probably treat that the same way as EXDEV.

Ah nevermind, on my python version this uses copy_file_range, not copy_file_range_ctypes. I forced it to use copy_file_range_ctypes, and put some print statements:

ret = -1 get_errno() = 0
Traceback (most recent call last):
  File "/home/kelvie/Temporary/./proton", line 1448, in <module>
    copyfile_reflink(sys.argv[1], sys.argv[2])
  File "/home/kelvie/Temporary/./proton", line 228, in copyfile_reflink
    raise e
  File "/home/kelvie/Temporary/./proton", line 225, in copyfile_reflink
    bytes_to_copy -= copy_file_range(src.fileno(), dst.fileno(), bytes_to_copy)
  File "/home/kelvie/Temporary/./proton", line 213, in copy_file_range_ctypes
    raise OSError(get_errno(), errno.errorcode.get(get_errno(), 'unknown'))
OSError: [Errno 0] unknown

def copy_file_range_ctypes(fd_in, fd_out, count):
    "Copy data using the copy_file_range syscall through ctypes, assuming x86_64 Linux"
    global __syscall__copy_file_range
    __NR_copy_file_range = 326

    if __syscall__copy_file_range is None:
        c_int64_p = POINTER(c_int64)
        prototype = CFUNCTYPE(c_ssize_t, c_long, c_int, c_int64_p,
            c_int, c_int64_p, c_size_t, c_uint)
        __syscall__copy_file_range = prototype(('syscall', CDLL(None, use_errno=True)))

    while True:
        ret = __syscall__copy_file_range(__NR_copy_file_range, fd_in, None, fd_out, None, count, 0)
        print(f'{ret = } {get_errno() = }')
        if ret >= 0 or get_errno() != errno.EINTR:
            break

    if ret < 0:
        raise OSError(get_errno(), errno.errorcode.get(get_errno(), 'unknown'))

    return ret

Note that I've since moved my steam .local outside of ecryptfs, so I haven't been testing from steam.

OK, so that means we should not be getting an errno of 0, and use_errno didn't fix it.

Maybe use_errno needs to also be passed to CFUNCTYPE.

Bingo:

ret = -1 get_errno() = 22
Traceback (most recent call last):
  File "/home/kelvie/Temporary/./proton", line 1448, in <module>
    copyfile_reflink(sys.argv[1], sys.argv[2])
  File "/home/kelvie/Temporary/./proton", line 228, in copyfile_reflink
    raise e
  File "/home/kelvie/Temporary/./proton", line 225, in copyfile_reflink
    bytes_to_copy -= copy_file_range(src.fileno(), dst.fileno(), bytes_to_copy)
  File "/home/kelvie/Temporary/./proton", line 213, in copy_file_range_ctypes
    raise OSError(get_errno(), errno.errorcode.get(get_errno(), 'unknown'))
OSError: [Errno 22] EINVAL

I set it on both:

def copy_file_range_ctypes(fd_in, fd_out, count):
    "Copy data using the copy_file_range syscall through ctypes, assuming x86_64 Linux"
    global __syscall__copy_file_range
    __NR_copy_file_range = 326

    if __syscall__copy_file_range is None:
        c_int64_p = POINTER(c_int64)
        prototype = CFUNCTYPE(c_ssize_t, c_long, c_int, c_int64_p,
                              c_int, c_int64_p, c_size_t, c_uint, use_errno=True)
        __syscall__copy_file_range = prototype(('syscall', CDLL(None, use_errno=True)))

    while True:
        ret = __syscall__copy_file_range(__NR_copy_file_range, fd_in, None, fd_out, None, count, 0)
        print(f'{ret = } {get_errno() = }')
        if ret >= 0 or get_errno() != errno.EINTR:
            break

    if ret < 0:
        raise OSError(get_errno(), errno.errorcode.get(get_errno(), 'unknown'))

    return ret

btw this also affects python's os.copy_file_range -- it also returns an EINVAL, so this'll have to be fixed in copyfile_reflink or even higher up the stack.

Thanks for checking that. I think the rest of the fix will be to change this check in copyfile_reflink:

            if e.errno != errno.EXDEV and e.errno != errno.ENOSYS:

We should just be able to handle EINVAL the same way:

            if e.errno not in (errno.EXDEV, errno.EINVAL, errno.ENOSYS):

I've pushed those changes, and they should appear shortly in bleeding-edge experimental.

Hello, I'm pretty sure this is fixed, although I've since moved steam outside the encrypted fs, so I haven't tested it.