Tips for adopting an STL frame of mind

Go To StackoverFlow.com

1

I've been an almost exclusively C# programmer for the last 6 years. I'm now working on a project where C++ is the language of choice, and STL is our library for collections.

After using C#'s LINQ, I'm having a very hard time getting into an STL frame of mind.

As an example: Write an equivalent of IEnumerator.Select.

C#

public static IEnumerator<Output> Select(this IEnumerator<Input> input, Func<Input, Output> func) {
  while (input.MoveNext) {
    yield return func(input.Current);
  }
}

Super easy.

Now try writing something similar in C++ and STL. (Setting aside the issue of the convenient syntax of of the yield keyword and anonymous functions).

It can't even be done without first answering a number of difficult questions. Because STL enumerators use inter-enumerator comparisons instead of MoveNext, you have to decide what your enumerator's terminal value will be. Then you have to mess with all the iterator_traits nonsense. STL uses compile-time template dispatch instead of runtime dynamic dispatch, so your iterator has to be templated not just on the value_type of the input enumerator, but also the specific kind of input enumerator.

Don't even get me started on the time I tried to write a map-join iterator in STL.

From looking at the code that other people write, I've come to the conclusion that STL, unaugmented with Boost, is rarely used for anything other than collections and sort.

My proximal observations are:

  1. Is there a way to succinctly write a mutation iterator in STL?
  2. How do you succinctly sort-copy one collection into another?

More generally, I've noticed a few things that clash with my accustomed manner of thinking:

  1. STL code does not seem to be succinct. Is my aim of writing succinct code a liability when writing STL code? (By not succinct, I mean, often involves very long type identifiers)
  2. Boost seems to be a near requirement for writing algorithms in STL. What do people who aren't allowed to use Boost do?
2012-04-04 18:35
by Kennet Belenky
Nope, the streams are broken, but the rest is just fine. Boost and QT both extend the STL into much larger frameworks, Boost as a general extension, QT for GUI related stuff. If I understand IEnumerator.Select correctly, we call that boost::transform_iterator. If I understand boost::zip_iterator, that's your map-join iterator - Mooing Duck 2012-04-04 18:38
I should note that my project requirements forbid Boost - Kennet Belenky 2012-04-04 18:40
You should take a look at what the <algorithm> header offer - Attila 2012-04-04 18:41
@KennetBelenky: Fine, copy-paste the Boost code in, and claim it's your own. Still easier than writing it yourself. Though a transform_iterator should be pretty easy - Mooing Duck 2012-04-04 18:42
Hmmm because of compile time dispatch you have something called template metaprogramming in c++ :) just search for it on google and you will not praise dynamic dispatch then : - Yavar 2012-04-04 18:47
Yes, I've looked at algorithm. It's got some decent starters, but it has some glaring omissions. As an example, try sorting one collection into another collection with only one copy. Sort doesn't work because it's in place. Partialsortcopy doesn't work because back_inserter doesn't have an equivalent of .end().

My real difficulty is that STL seems to make it cumbersome to write my own algorithms - Kennet Belenky 2012-04-04 18:48

@Yavar I'm very aware of template metaprogramming. I see code that is 1) very difficult to unit test. 2) clogged with the "typename" keyword all over the place. 3) Bloats like you wouldn't believe at compile time. 4) Often has 80+ character type identifiers - Kennet Belenky 2012-04-04 18:49
This comes off as much more a rant than an real question. Perhaps it would help to actually describe at least one thing you really want to accomplish instead of simply exclaiming over how wonderful C# is and/or how broken you consider STL - Jerry Coffin 2012-04-04 18:52
@Jerry Coffin It is a question borne of frustration and the observation that while I have seen plenty of STL code, I have seen very little that uses anything more than the most basic features.

Here are two things that I have brought up: 1) Is there a way to succinctly write a mutation iterator in STL? 2) How do you succinctly sort-copy one collection into another - Kennet Belenky 2012-04-04 18:54

@KennetBelenky: Okay, but you still haven't given any detail about the problem. For the most part, IEnumerable.select is on the same order as std::transform, except that transform produces results immediately, where select creates an object that produces results on demand. Is that difference what leads to your difficulty? If not, what is it - Jerry Coffin 2012-04-04 18:59
@KennetBelenky: features missing from the STL does not make STL "fundamentally broken". A mutation iterator requires a fair bit of code, the sort-copy less so - Mooing Duck 2012-04-04 18:59
@Mooing Duck Back to the STL frame of mind, and my original question. Is the source of my difficulty that I aim for succinct code - Kennet Belenky 2012-04-04 19:03
Also, can anyone point me to a significant open source project that uses STL for more than just its collection classes? I'd like to see how people who think fluently in STL use it - Kennet Belenky 2012-04-04 19:04
@KennetBelenky: I just realized your sample C# doesn't seem to work, it says it's returning IEnumerator<Output> but appears to return Output. STL aside, you should fix that. (If I'm mistaken, forgive my ignorance, my C# is poor. - Mooing Duck 2012-04-04 19:22
@MooingDuck It's a quirk of C# when using the yield return keyword. Basically it means, "I'm returning one element of the enumeration." Under the wrappers, the compiler transforms the local variables into fields of a compiler-named type that implements IEnumerator, and the body of the method is invoked inside of the class's MoveNext method - Kennet Belenky 2012-04-04 19:26


2

Some of your question doesn't really make a lot of sense to me. Just for example, you talk about having to deal with "enumerator_traits". I'm not quite sure what you're talking about. Maybe you meant iterator_traits? I don't recall having having used anything called an "enumerator_trait", nor can I find any mention of such a thing in the C++ standard.

iterator_traits at least exist, but they're something I rarely "mess with". I'm peripherally aware of their existence, but little more than that. I've written a fair number of iterators and algorithms without ever doing anything specific with iterator_traits in any of it.

Let's get to the specific question of creating a new collection that's a sorted version of another collection. This is fairly easy in a number of different ways. std::partial_sort_copy can certainly do that:

#include <algorithm>
#include <vector>
#include <iterator>
#include <iostream>

int main() { 

    std::vector<int> input;

    // generate some data to sort
    std::generate_n(std::back_inserter(input), 20, rand);

    // a destination for the sorted data:
    std::vector<int> result(input.size());

    // do the sort/copy:
    std::partial_sort_copy(input.begin(), input.end(),
        result.begin(), result.end());

    // show the sorted data:
    std::copy(result.begin(), result.end(), 
        std::ostream_iterator<int>(std::cout, "\n"));

    return 0;
}

For many purposes, however, it's easier to make a copy, then sort:

std::vector<int> result(input.begin(), input.end());
std::sort(result.begin(), result.end());

If you really want it succinct, you can make a copy into a data structure that's innately sorted:

std::multiset<int> result(input.begin(), input.end());

This last, however, usually trades off a little efficiency to make the code shorter. Under many (most?) circumstances that's not a problem, but if you find it too slow, faster alternatives are easily available.

2012-04-04 19:38
by Jerry Coffin
Here's a sort with only a single copy: http://ideone.com/s7IjE. 21 lines. Longest identifier I typed was std::vector<in_iter> - Mooing Duck 2012-04-04 19:44
and a transform iterator: http://ideone.com/PYyvY. 26 lines. More complicated, still not that bad. The normal thing is as Jerry says, just std::transform them all at once - Mooing Duck 2012-04-04 20:17
@Mooing Duck There's an important, non-obvious, generally applicable revelation in your implementation of the transform iterator. Calling make_transform on the endpoint iterator is non-obvious to a LINQ programmer, and seems like a good approach - Kennet Belenky 2012-04-04 20:56
@KennetBelenky: Oh, glad to hear that. That's a common trick in the STL, since (as you say) the type names can be inconveniently long. make_tuple, make_shared, and others - Mooing Duck 2012-04-04 21:12
Ads