Subprocess Popen stdout is empty if the subprocess is a Python script

February 23, 2023

Here is some code that should print numbers 0 through 4 to the terminal: (adapted from https://stackoverflow.com/a/59291466/4391249)

import os
import time
import subprocess

cmd = 'python', '-c', 'import time; [(print(i), time.sleep(1)) for i in range(5)]'
p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
os.set_blocking(p.stdout.fileno(), False)
start = time.time()
while True:
    # first iteration always produces empty byte string in non-blocking mode
    line = p.stdout.readline()
    if len(line):
        print(line)
    if time.time() > start + 5:
        break
p.terminate()

But it doesn’t for me. For me, nothing is printed.

When I instead set cmd = 'ls', it does produce the expected output (prints the contents of my working directory).

Why doesn’t the Python one work?

I’m on Ubuntu20.04 with Python3.10.

>Solution :

Disable output buffering using -u cli flag:

import os
import time
import subprocess

cmd = [
    '/Users/xyz/.pyenv/shims/python', '-u', '-c',
    'import time; [(print(i), time.sleep(1)) for i in range(5)]'
]
p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
os.set_blocking(p.stdout.fileno(), False)
start = time.time()
while True:
    # first iteration always produces empty byte string in non-blocking mode
    line = p.stdout.readline()
    if len(line):
        print(line)
    if time.time() > start + 5:
        break
p.terminate()

Out: