print! macro should flush stdout

[tanadeau]

As stdout is line-buffered, stdout is not implicitly flushed until a new-line is encountered. This means that the print! macro does not act like a println! without the newline as the documentation suggests. To be equivalent, the user must explicitly flush stdout like the following:

use std::io::prelude::*;                                                           
use std::io;                                                                       

fn main() {
    print!("Type something: ");
    io::stdout().flush().ok().expect("Could not flush stdout");

    // Read stdin, etc.
}

For easy use of the print macros, the user should not need to know about how I/O flushing works. As such, print! should explicitly flush stdout itself.

[sfackler]

As a quick survey, standard output is line buffered in C, C++, Python, and Ruby. Go's Printf buffers output and writes it to an unbuffered stdout. Java doesn't document what it does, but System.out appears to autoflush.

[richo]

I opened #23823. It's not entirely clear to me that the solution isn't to just update the docs to point out that stdout in unbuffered, but I think at the point where you're invoking the print*! machinery you're sufficiently far from the OS that you probably just do want some output to appear in the terminal.

[tanadeau]

@richo The implementation of std::io::Stdout is definitely line-buffered. See

rust/src/libstd/io/stdio.rs

Line 213 in 9f9e251

inner: Arc<Mutex<LineWriter<StdoutRaw>>>,

[richo]

Sure, but that's orthogonal, since the underlying fd is not, and writing to it will not emit anything until it's flushed.

I'm pretty sure that the correct approach is to merge that patch, but figured it was worth pointing out that updating the docs may be a valid approach too.

[alexcrichton]

It's not 100% clear to me that this is the behavior that we want. In #23087 it was explicitly removed due to worries about deadlocking. The specific problem of deadlocking can be papered over, but in general APIs tend to work much better over time if they do precisely what you expect, and it may not always be expected that print! flushes the output buffer.

One downside of flushing on print! would be that any form of "maximal buffering" when redirected to a TTY would be lost. For example C performs much more buffering when stdout is redirected than when it is to a console. This means that simple benchmarks which print information will be much slower in Rust than in comparison with others (due to the frequent flushing).

I would be fine beefing up the documentation in this regard, of course!

[richo]

That's totally reasonable, and mostly what I expected. Do you have a
citation about the c buffering though? That wasn't be belief and I'd love
to read more.

Will update the docs if no one beats me to it.

On Saturday, March 28, 2015, Alex Crichton [email protected] wrote:

It's not 100% clear to me that this is the behavior that we want. In
#23087 #23087 it was explicitly
removed due to worries about deadlocking. The specific problem of
deadlocking can be papered over, but in general APIs tend to work much
better over time if they do precisely what you expect, and it may not
always be expected that print! flushes the output buffer.

One downside of flushing on print! would be that any form of "maximal
buffering" when redirected to a TTY would be lost. For example C performs
much more buffering when stdout is redirected than when it is to a console.
This means that simple benchmarks which print information will be much
slower in Rust than in comparison with others (due to the frequent
flushing).

I would be fine beefing up the documentation in this regard, of course!

—
Reply to this email directly or view it on GitHub
#23818 (comment).

[tanadeau]

While making the documentation clearer would certainly be helpful, I think Rust is losing ease of use by making auto-flushing more difficult than need be. C++ has std::unitbuf to making auto-flushing simple in cases like prompts and progress bars. This at least makes it so that users don't need to call flush() explicitly after every call.

[tanadeau]

Could there be new macros for this use case? I'd hate for Rust I/O to be harder to use correctly than C++'s especially in a common learning case. For example, since C++ does flush cout when cin is read, the following works fine there but the equivalent doesn't for Rust:

#include <iostream>

int main() {
    std::cout << "Type something: ";

    std::string input;
    std::cin >> input;

    std::cout << input << std::endl;
} 

Also according to the C standard:

 At program startup, three text streams are predefined and need not be opened explicitly
— standard input (for reading conventional input), standard output (for writing
conventional output), and standard error (for writing diagnostic output). As initially
opened, the standard error stream is not fully buffered; the standard input and standard
output streams are fully buffered if and only if the stream can be determined not to refer
to an interactive device.

[nagisa]

This means that the print! macro does not act like a println! without the newline as the documentation suggests.

println! macro, on the other hand, does not specify anything about its flushing semantics either. Even the std::io::Stdout doesn’t and totally should.

---

I generally really dislike implicit flushing and prefer to flush explicitly. On the other hand, there’s write{,ln}!, which does not and can’t assume anything about buffering, so I’m ambivalent about this issue.

[alexcrichton]

I think Rust is losing ease of use by making auto-flushing more difficult than need be.

The design here is generally mostly about trade-offs. On one hand you have "ease of use" where you don't have to worry about calling flush(), but on the other hand you have worries about deadlocking, and/or surprising behavior happening behind the scenes. On the other hand you have to write a few more flush calls, but you know precisely what is happening and can avoid situations like deadlocking fairly easily.

For example, what should this code do?

let input = io::stdin();
let mut locked = input.lock();

print!("enter input: ");
let mut input = String::new();
try!(locked.read_line(&mut input));

println!("you entered: {}", input);

If print! were to auto-flush, then it would deadlock the current process because the stdin handle was already locked. The alternative here is adding one call to try!(locked.flush()) right above the call to read_line.

[nagisa]

If print! were to auto-flush, then it would deadlock the current process because the stdin handle was already locked. The alternative here is adding one call to try!(locked.flush()) right above the call to read_line.

I think you misunderstood what the issue is about: it is only proposed to flush std_out_ after print! writes to the std_out_. i.e. change

macro_rules! print(…) {
     …
}

to

macro_rules! print(…) {
    …
     stdout().flush()
}

Since print! already takes a lock to write, this change will not introduce any more new cases where it deadlocks. For example snippet below deadlocks currently and will deadlock in all the same cases if we made it to auto-flush.

let output = io::stdout();
let output = output.lock();
print!("stdout locked"); // deadlock.

I’m starting to believe that stabilising Std{in,out,err}::lock was a wrong and hasty move.

[alexcrichton]

I think you misunderstood what the issue is about: it is only proposed to flush stdout after print! writes to the stdout

Hm yes, I think I have misunderstood! There are definitely performance considerations which affect flush-on-all-print semantics as it's not something that other languages tend to do and can greatly hinder various benchmarks.

[richo]

I think the crux of this is basically "What is the intent of print!". My
intuition is that it's not intended to be in the critical section, and is
instead meant to be a helper for "make this string appear on stdout", I
could really take or leave whether or not it flushes. That said,
considering that it already traverses all of the formatting machinery
anyway, if it can't deadlock my intuition is that the flush is just one
extra syscall to make this do something more user friendly/intuitive.

While I don't believe it's actually on the table, I would be a stoic -1 to
any suggestion of making all stdout writes autoflush.

On Mon, Mar 30, 2015 at 1:50 PM, Alex Crichton [email protected]
wrote:

I think you misunderstood what the issue is about: it is only proposed to
flush stdout after print! writes to the stdout

Hm yes, I think I have misunderstood! There are definitely performance
considerations which affect flush-on-all-print semantics as it's not
something that other languages tend to do and can greatly hinder various
benchmarks.

—
Reply to this email directly or view it on GitHub
#23818 (comment).

[tanadeau]

To address the performance considerations, could the print! macro only flush stdout if stdout is attached to a TTY?

[alexcrichton]

@tanadeau unfortunately it's a performance concern both attached and not attached to a TTY

[mkpankov]

I'm against this.

Implicit flushing will degrade performance of programs doing specifically print!() and printing long lines out of several components.

What, for example, do you use for actual text printing of your main output in console programs? I used print!() in clone of hexdump, for example.

About buffering: this man mentions the defaults
http://manpages.courier-mta.org/htmlman3/setbuf.3.html

[softprops]

Fwiw Ive run into this more than a few times and each time it was confusing as a user. Its the same kind of confusion you'd have if you pushed a print button in your browser and nothing happens. I think part of a compromised solution lies in a more meaningful name. We have buffered readers and writers for the performance mentioned above and do not cause the same kind of confusion because their names reveal their expected behavior. What about a bufprint macro you can call when you want buffering and have print's default behavior be to... print something.

[MutantOctopus]

I just ran into this issue for the first time, and just to give my two cents: I generally agree with what's been said.

println! flushing is fine, because when you're done with a line, you're done with a line. On the other hand, in my experiments across various languages, I've generally reserved print! and its relatives in other languages for times where I need to assemble a single line to print, but don't necessarily need to have it flush at separate times, in which case yes, not having a flush is largely irrelevant. And if someone is familiarizing themselves with how output streams work, and they run into unexpected behavior, they'll search, look up documentation, find threads like this, and largely come out of it with a better understanding of the situation (which I did).

At the same time, I think it would be beneficial to have a separate print macro which flushes - print_flush! seems like the obvious name - and I'm not sure if I see the shortcoming in having a flush! macro as @dtolnay suggested, but at the same time I'm sure I don't know the details well enough to insist on it.

[BartMassey]

I've been quite busy lately, but I have a prototype implementation of flushing print macros and an RFC mostly written. I'll try to get it out for review in the next few weeks. If someone wants to help get it out sooner, let me know.

[anirudhb]

I believe that println! and print! should be kept alone.
For the people that want C-style buffering, why not create a printf! macro?
This time, the f refers to flush/force, meaning you will see the data immediately.
This has the advantage of being backwards-compatible, as all existing code using print! will be fine.
For the people that are okay with a performance hit and just want unbuffered printing, printf! should do fine.

(Also, having a printlnf! macro is a bad idea, because most people will mistake printf! for the C version. Having them mistake it for the C version is good, because it should be like the C version, anyways.)

[BartMassey]

The macro names I am currently using in my draft RFC are prompt!, eprompt! and input!. Bikeshed away… I'll try to post a draft Real Soon Now, I promise.

[satchm0h]

My guess is that a large percentage of folks will (or should) read the second edition book to get started. I would recommend changing the chapter two exercise code to use print and io::stdout().flush().unwrap() for the guess prompt. This way newbies (like myself) will just grok the rustafarian way to handle line buffered stdin. I'll try to put forth a pull request on the docs if someone has not already tried it.

If I were to make a suggestion for a more elegant solution I would go back to @CleanCut's suggestion to add a new macro. However, I would look for something a bit terser. Maybe fl_print! or flprint!? @BartMassey's suggestion of prompt! is a good alternative, but there may be other use cases for "print & flush" other than prompting for user input and 'prompt' can certainly have other connotations.

[BartMassey]

My current partially-finished draft RFC has an input!() macro for the prompt-and-read situation and a prompt!() macro for the just print-and-flush situation. Plus some other stuff. Plus a demo crate. I really need to get this out soon so that people can bikeshed it and hopefully move it forward. Unfortunately, the Rust class that I'm teaching for the first time this quarter is taking a lot of my Rust-related bandwidth. :-)

[Lucretiel]

+1 for a python-style input! macro, which takes care of the typical write-then-read case.

Sounds good! Should we remove the implicit flushing in println!() as well?

It's worth noting that technically println! doesn't flush. You can see in the source code that it simply calls print! with an appended newline. Rather, it's the current implementation of stdout which unconditionally line buffers, regardless of whether stdout is a tty or not. In fact, it's called about explicitly as a #FIXME in that code that the buffering behavior should be smarter; that change would implicitly change the flush behavior of all the macros that use it (print, println, and anything built on them).

[NealEhardt]

[Making print!() flush] would just move the stumbling block to some later point when buffered i/o and flushing inevitably bites them anyway. People writing interactive command-line apps need to understand what flushing means and inserting implicit flushes would not be doing them any favors.

I only have a shallow understanding of how flushing works, so please forgive the question. How will people be bitten? My guess:

• The application will be slower.
• If a message is split between multiple print!() statements, a partial message will appear on screen, quickly followed by the message tail.

Is there anything else? These issues seem trivial when I weigh them against the panic and frustration of "I'm printing stuff but it doesn't appear on-screen." The people writing performance-intensive command-line apps can use non-flushing io::stdout().write_fmt(format_args!(...)). Beginners and people using print!() for debugging will expect implicit flushing.

[x0a]

What about a printf! macro that prints and then flushes? Without modifying the behavior of the print! macro

[Lucretiel]

I'd be okay with a macro that guarantees a flush, but I'd be concerned about naming it printf, since that's such a ubiquitous identifier in many other languages for "formatted print"

[anyputer]

How about flprint!, flprintln! macros? Doing printflln! or printlnfl! would look confusing. Is it a problem if it starts with fl cause that might makes it sound like the flushing happens first?

[dancojocaru2000]

The main problem about print! not flushing is when combining it with read! from the text_io create.

Considering this snippet of code:

use text_io::read;
 
let rect = Rectangle {
    length: {
        print!("L: ");
        read!()
    },
    width: {
        print!("l: ");
        read!()
    }
};

The expected behaviour is this:

L: 5
l: 3

What actually happens is this:

5
3
L: l: 

C++ buffers std::cout, however it ties it to std::cin so that std::cout is flushed before std::cin reads any data, which solves this problem. This is not possible in this case for obvious reasons, so an alternative (perhaps a flprint! macro as suggested above?) should be found.

[frol]

@dancojocaru2000

This is not possible in this case for obvious reasons

Sorry, what are those reasons? It seems that in this particular case a special flush+read function/macro is a good bet, so it can be even part of the text_io crate.

[dancojocaru2000]

This is not possible in this case for obvious reasons

Sorry, what are those reasons? It seems that in this particular case a special flush+read function/macro is a good bet, so it can be even part of the text_io crate.

print! is just a macro for dumping stuff into stdout, and it has no connection to anything else (I would assume). A flush+read fn/macro would unnecessarily do two things in one package, restricting flushing only to the case where a read is required.

Having the option to print and flush in one macro, unbounded from needing to also read, seems like a good addition to the language.

[samclaus]

My guess is that a large percentage of folks will (or should) read the second edition book to get started. I would recommend changing the chapter two exercise code to use print and io::stdout().flush().unwrap() for the guess prompt. This way newbies (like myself) will just grok the rustafarian way to handle line buffered stdin. I'll try to put forth a pull request on the docs if someone has not already tried it.

THIS. I am re-reading the book trying to get into Rust again and right at the beginning of the Guessing Game chapter I recalled running into issues with flushing the first time I read the book and used print! instead of println! (so the guess would be on the same line, after a colon). Curiosity lead me here.

If no changes are going to be made for print!, the book should definitely use it and mention the flushing semantics.

With that said, I personally think println! and print! are used for simple logging or when getting input from the user 90% of the time. Someone also mentioned that the line-flushing is implemented at the writer level and could change under people's feet who expected println! to always flush (like me). Why not cater to the 90% use case and make these both more user-friendly with guaranteed flushing? For the 10% writing performance-sensitive formatting code, explicitness, and usage of low-level APIs is probably already mandatory.

[dancojocaru2000]

Furthermore, most people dislike reading a book (even online) about every detail of the language. I know this is a sensible topic, bringing reactions like "how can you not read the book?" (reactions which, imo, drive people away), but Rust By Example should certainly have a topic discussing this.

Python is pretty much the only other language where the default way of printing doesn't flush at least when input is needed, yet they have the input function which covers this exact purpose of asking questions and waiting for answers.

[dancojocaru2000]

@dtolnay

People writing interactive command-line apps need to understand what flushing means

In no other programming language that I worked with is that a need. It is merely an "oh, that's cool".

Furthermore, Rust breaks an expectation that is fulfilled by other languages: input given without the full output shown is garbage, unwanted input.

Here are two examples:

• it doesn't make sense that a line read contains the name if the "Name: " prompt wasn't flushed and therefore displayed beforehand
• when communicating with a server, assuming the output to the server is buffered, if you make a request but then don't flush the request and then you block waiting the answer, it is safe to assume that the server never received the full request and therefore an answer will never be sent, the block on read being therefore infinite

Another question would be why print! and println! exist in the first place while a read! or readline! macro don't. Furthermore, why is there a need for print! when write! can be used? If their purpose is to make things easier, then why not go the extra step and fulfil the expectation that, when data is read from the standard input, all the data must be flushed to standard output?

[cdecompilador]

This is my personal suggestion to the problem:

print!(shouldflush, "{}", args...);

being shouldflush just an optional token, as far as I know this is possible.

fn main() {
    let a = 5;

    // Normal printing, as is now
    print!("{}", a);

    // Some io borrow

    // Printing with flushing
    print!(shouldflush, "{}", a);
}

Maybe other options might be passing a boolean, or flush, or shouldflush = true

[pawkw]

I am new to Rust and ended up here with this problem. I note that print commands in all languages default to displaying on the screen rather than actually printing anything on the printer. Those days have passed. If I were to name an autoflushing macro, I would call it display!

[LollipopFt]

what is the difference between print! and write!? aren't they basically the same?

[Lucretiel]

what is the difference between print! and write!? aren't they basically the same?

They are basically the same; write! allows you to write to any stream, whereas print! specifically writes to stdout.

[bazzilic]

Just to add my 5 cents, I think print! should not flush because line-buffered output is the expected behavior, but

1. there should be a version of the print macro that explicitly flushes; and
2. stdlib functions that read from stdin should first explicitly flush stdout.