mingw: work around incorrect standard handles

For some reason, when being called via TortoiseGit the standard handles, or at least what is returned by _get_osfhandle(0) for standard input, can take on the value (HANDLE)-2 (which is not a legal value, according to the documentation). Even if this value is not documented anywhere, CreateProcess() seems to work fine without complaints if hStdInput set to this value. In contrast, the upcoming code to restrict which file handles get inherited by spawned processes would result in `ERROR_INVALID_PARAMETER` when including such handle values in the list. To help this, special-case the value (HANDLE)-2 returned by _get_osfhandle() and replace it with INVALID_HANDLE_VALUE, which will hopefully let the handle inheritance restriction work even when called from TortoiseGit. This fixes https://github.com/git-for-windows/git/issues/1481 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Johannes Schindelin committed Nov 22, 2019 at 14:41 UTC c5a03b1e29c69f3f06c8fabd92493edb73469176

1 file changed +11 -1

compat/winansi.c

+11 -1

@@ -662,10 +662,20 @@ void winansi_init(void)

662 */

663 HANDLE winansi_get_osfhandle(int fd)

664 {

665 + HANDLE ret;

666 +

667 if (fd == 1 && (fd_is_interactive[1] & FD_SWAPPED))

668 return hconsole1;

669 if (fd == 2 && (fd_is_interactive[2] & FD_SWAPPED))

670 return hconsole2;

671

670 - return (HANDLE)_get_osfhandle(fd);

672 + ret = (HANDLE)_get_osfhandle(fd);

673 +

674 + /*

675 + * There are obviously circumstances under which _get_osfhandle()

676 + * returns (HANDLE)-2. This is not documented anywhere, but that is so

677 + * clearly an invalid handle value that we can just work around this

678 + * and return the correct value for invalid handles.

679 + */

680 + return ret == (HANDLE)-2 ? INVALID_HANDLE_VALUE : ret;

681 }