I’m interested in offering a technical session focusing on CPU caches and why they are important for people interested in developing performance-sensitive software. My idea is to talk about things like the following:
- Instruction caches vs. data caches vs. TLBs
- Cache lines, cache line prefetching, and false sharing
- Cache hierarchies
- Shared and unshared caches
- Interactions among caches, multithreading, hyperthreading, and speculative execution.
- Interactions among C++ language features and instruction caches (e.g., macros, inlining, templates)
- Cache-friendly algorithms and data structures
Please use the comment feature of this blog to discuss your thoughts on this prospective session topic.
Thanks,
Scott
May 6, 2010 at 6:47 am
I think that would be very interesting and informative.
May 6, 2010 at 7:05 am
I got approval to come to this and will be registering soon. I’m very excited.
This is exactly one of the topics that I was hoping to hear about. I did some Java development for a few years and got to know JSR-133 (the memory model) fairly well. There doesn’t seem to be anything similar in C++ and so my confidence level that I’m doing the right thing isn’t high. Much of my confusion is related to thread/cache interactions. For example, the double-checked locking idiom is a common Java problem (or at least was in the Java 5 days). Is it an anti-pattern in C++?
Other questions:
* When I acquire or release a mutex (or do anything else that forces serialized access to some resource), are the caches flushed? I think it must in order to force a read or write to the main store.
* Alpha had weak memory ordering. Is developing on a chip like this more difficult, or just less forgiving?
* Does the volatile keyword prevent caching?
I have some more questions that I can’t recall right now, but mostly the could be resolved by something like a C++ version of JSR-133.
May 8, 2010 at 9:35 pm
C++0x has a memory model, and I expect it will be a recurring topic at C&B. It will be possible to portably and safely implement double-checked locking in C++0x, in part due to the memory model, in part due to some new semantic constraints imposed on the core language (e.g., in new expressions). Regarding your other questions:
* The language itself has no recognition of caches, so there is nothing in C++ itself that can force them to be flushed. My understanding is that cache-to-cache data transfers can occur on some platforms, but Herb probably knows more about that than I do.
* Trying to program using anything other than sequential consistency is extremely hard, but I believe that a good compiler will make it possible for you to program using a sequential consistency model even if the generated code takes advantage of the underlying hardware’s ability to use a more relaxes model (because you won’t be able to tell).
* The question about the interaction of caching and volatile is an interesting one. I’ll have to do some research on this. All I can say at this point is that by the time of C&B, I promise to have what I believe is an accurate answer.
Scott
May 10, 2010 at 2:48 pm
I would love to see this session. It’s a definite weak spot in my current development awareness.
May 13, 2010 at 4:14 pm
I’d be glad to read about things like that.
May 24, 2010 at 11:01 am
I’ll add another vote in favor of this session. It sounds quite useful to me.
May 25, 2010 at 1:48 pm
I would like to hear such a talk. Some suggestions:
* Discuss how to explicitly measure cache related statistics (cache misses, false sharing, etc.) so that the effect of code changes can be shown explicitly. In other words, how do we prove a code change makes the code run faster because of improved cache utilization and not because of some other subtle effect?
* It would be nice to see multiple implementations of commonly used patterns (pipeline, thread safe queue, etc.) that have different cache utilization characteristics (with explicit measurements).
* Some related issues I’d like discussed: Async I/O (zero-copy), thread affinity, kernel bypass, heap implementations that improve cache utilization, transactional memory
May 26, 2010 at 10:07 am
Thanks very much for the comments and suggestions. There are many topics related to cache utilization we could discuss, and while I’m not going to make any promises about what will go into any of my presentations (at least not this far in advance), certainly I expect that conversations about these topics will take place during the open discussion periods.