Iterator#take_while consumes first falsy value

[Mark-Simulacrum]

The take_while method on iterators consumes the first falsy value, instead of leaving it in the iterator.

This behavior seems unintuitive to me; and I could not find any documentation of this functionality. Due to this (I think) being a breaking change, it's probably not worth changing the take_while functionality, but documenting this is probably a good idea.

I tried this code:

fn main() {
    let a = [1, 2, 3, 4];
    let mut iter = a.into_iter();

    {
        let new_iter = iter.by_ref().take_while(|n| **n != 3);

        let mut back_to_vec: Vec<u32> = Vec::new();
        back_to_vec.extend(new_iter);
        println!("taken iter: {:?}", back_to_vec);
    }

    let mut back_to_vec: Vec<u32> = Vec::new();
    back_to_vec.extend(iter);
    println!("old iter: {:?}", back_to_vec);
}

I expected to see this output:

taken iter: [1, 2]
old iter: [3, 4]

Instead, this I saw this output:

taken iter: [1, 2]
old iter: [4]

Meta

rustc --version --verbose:

rustc 1.8.0-nightly (4b615854f 2016-01-26)
binary: rustc
commit-hash: 4b615854f00ba17ad704155e1d3196c17a6edb62
commit-date: 2016-01-26
host: x86_64-apple-darwin
release: 1.8.0-nightly

I believe this also occurs on stable (tested in play.rust-lang.org).

[huonw]

Not only would it be a breaking change, but it is impossible for a general iterator: the only way to tell if take_while needs to stop or not is to get the element out of the iterator with .next(), and there's no way to uncall .next/push an element back on to the end. (E.g. consider something like Receiver::iter: once the message has been received and yielded, it is gone.)

[Mark-Simulacrum]

For people reaching here and seeking a way to do this currently, itertools provides a method that will leave the last element in the iterator, but at the cost of cloning the entire iterator every iteration.

[nodakai]

Well, maybe we can put the first falsy value to inside struct TakeWhile and add an accessor to it...

[bluss]

The cost of cloning an iterator is totally depedent on which iterator it is. A slice iterator is implemented using two pointers, so the cloning amounts to storing two pointers during the iteration. When the whole iteration is inlined, and llvm breaking the structs down to their members, it should be able to remove the store of the pointer that doesn't change (the end), so the overhead is down to storing and restoring one pointer.

[bluss]

take_while_ref is a nice hack, but it should be complemented by either adaptors on Peekable and/or itertools' PutBack.