When turning off the "Enable steam overlay while in-game" setting in a non-steam game's right-click > properties menu, nothing actually happens. Launching the game still allows me to shift-tab, and steam overlay notifications show up over the game content.