Multithreaded programming discusions.

Parallel Computing Releases at Microsoft

Mon, 10 Dec 2007 22:56:59 GMT

From http://herbsutter.spaces.live.com :

Parallel Computing Releases at Microsoft

For those of you who may be interested in concurrency for Microsoft platforms, and .NET in particular, I'm happy to report some fresh announcements:
MSDN has launched a Parallel Computing developer center. This is a section of the MSDN site focused on "providing information, ideas, community, and technology to developers to make it easier to write programs that perform and scale well on parallel hardware."
You can download and try a CTP "preview" of the .NET Parallel Extensions (PFX). These compile and run, but remember they're CTP quality. PFX currently has two major parts:
PLINQ: Parallel Language Integrated Query, a parallel version of LINQ with extensions like .AsParallel.
TPL: Task Parallel Library with extensions like Parallel.For loops.

There's also a PFX Team Blog.

PFX is for .NET concurrency development, but of course we are (and personally I am) working on native development tools, including for C++ specifically, that are at least as exciting. Stay tuned here for future news. In the meantime, enjoy these bits and the resources and discussion on the Parallel Computing developer center and forums.

http://highscalability.com

Thu, 15 Nov 2007 19:06:25 GMT

В статье "What nine of the world’s largest websites are running on" встретил ссылку на http://highscalability.com и картинку:

P.S. Наверное, это offtopic для multithreading (надеюсь, полезный:) - а где такое обсуждают?

Memory Models

Mon, 12 Nov 2007 20:31:45 GMT

"CLR 2.0 memory model" - it was quite interesting to know that CLR's memory model is not specified yet but Java's is.

Google And IBM Partner To Push Cloud Computing

Mon, 08 Oct 2007 22:40:27 GMT

http://news.google.com/?ned=us&ncl=1121846917

Упомянутые в предыдущем посте "Cluster Computing and MapReduce Mini Lecture Series" - часть этой инициативы. Все курсы находятся здесь:
- http://code.google.com/edu/content/parallel.html

Где раздобыть железа не университетам - пока не ясно.
Насколько Amazon EC2 может подойти?

P.S. Не эта ли инициатива помогла акцииям Google подрости на днях?

Obliq: A Language with Distributed Scope - by DEC, 1994

Mon, 08 Oct 2007 20:11:54 GMT

http://gatekeeper.dec.com/pub/DEC/SRC/research-reports/abstracts/src-rr-122.html

Interesting to read in general and about not using reentrant mutexes in particular:

A way to satisfy condition (2) is to use reentrant mutexes, that is, mutexes that do not deadlock when re-locked by the “same” thread (for example, see [Forté 1994]). On one hand, this solution is too liberal, because it allows a method to call an arbitrary method of a different object, which then can call back a method of the present object without deadlocking. This goes well beyond our simple desire that a method should be allowed to call its siblings: it may make objects vulnerable to unexpected activations of their own methods, when other methods have not yet finished reestablishing the object’s invariants. On the other hand, this solution may also be too restrictive because the notion of “same” thread is normally restricted to an address space. If we want to consider control threads as extending across sites, then an implementation of reentrant locks might not behave appropriately.

We solve this dilemma by adopting an intermediate locking strategy, which we call self serialization, based on the notion of self-inflicted operations described in section 3.4. Serialized objects have an implicit associated mutex, called the object mutex. An object mutex serializes the execution of selection, update, cloning, and delegation operations on its host object. Here are the simple rules of acquisition of these object mutexes:
External operations always acquire the mutex of an object, and release it on completion.
Self-inflicted operations never acquire the mutex of their object.

Cluster Computing and MapReduce - video course

Mon, 08 Oct 2007 18:57:30 GMT

http://code.google.com/edu/content/submissions/mapreduce-minilecture/listing.html

Recent video talks (for about an hour each)

Tue, 25 Sep 2007 00:38:04 GMT

"Machine Architecture" by Herb Sutter:
- http://herbsutter.spaces.live.com/Blog/cns!2D4327CC297151BB!304.entry

"(VC++ Safe Libraries and More)" and "Understanding Exceptions and When/How to Handle Them" by VC++ People:
- http://blogs.msdn.com/vcblog/archive/2007/09/21/channel-9-ale-contenti-and-louis-lafreniere-understanding-exceptions-and-when-how-to-handle-them.aspx

Lock recursion is usually a bad idea

Wed, 12 Sep 2007 19:21:22 GMT

http://www.bluebytesoftware.com/blog/2007/09/12/ConditionVariablesAndLockRecursion.aspx

JD> Sadly, I guess I'm only about 10 years too late...
PL> The really sad part is that all this was common knowledge even 10 years ago:
PL> http://groups.google.com/group/comp.programming.threads/msg/9ed3cc6318a8f68d

[Google TechTalks] Concurrency and Message Passing in Newsqueak

Tue, 04 Sep 2007 07:40:22 GMT

"A state is (PC, stack). A concurrent program is (PC, stack)+." - просто и доходчиво излагает Rob Pike. И email у него простой - r@google.com. :)

LockSmith - concurrency-oriented refactorings

Fri, 08 Jun 2007 23:19:14 GMT

From http://binkley.blogspot.com/2007/06/all-dancing-all-singing-locksmith.html

This is a list of features which made me look twice:
Split Lock
Merge Locks
Make Class Thread-Safe
Convert Field to Atomic
Convert Field to ThreadLocal
Lock Call-Sites of Method
Convert Simple Lock to Read-Write Lock
Convert Read-Write Lock to Simple Lock
Convert Synchronization Field to Lock
Split Critical Section
Shrink Critical Section
Merge Critical Sections

P.S. "made me look twice" - how to translate this into Russian?

Multithreading book suggestion

Mon, 23 Apr 2007 06:46:09 GMT

Hello lot,
Can you suggest some comprehensive multi threading reference book in English with more Windows then POSIX orientation?

Thank you in advance.

Thinking Parallel

Thu, 19 Apr 2007 13:42:34 GMT

Here is an interesting blog about parallel programming: Thinking Parallel

Transactions in Windows Vista.

Wed, 14 Mar 2007 10:09:58 GMT

Here is an example of using transactions to deal with registry and NTFS in Windows Vista on codeproject. Article called "Vista KTM: Transaction Management in Vista and Beyond ..." By Warren Stevens.

Transacted API in Windows Vista.

Sat, 03 Mar 2007 20:59:43 GMT

At very long last Microsoft has been added routines to WinAPI to deal with registry and file system with using of transactions. New transaction API requires Windows Vista or Windows Server "Longhorn". For example one can see RegOpenKeyTransacted routine in MSDN. Here is list of transacted functions to work with file system. In my opinion such functions should be added from the beginning at least for registry.

ScalPL

Fri, 15 Dec 2006 19:55:02 GMT

I'll try to write something on it to wake up some interest...

ScalPL is a modern programming language thought for writing readable concurrent programs. It doesn't mater whether it's on shared memory, clusters or grid - the program language only describes the flow of the program and the dependencies between sequential (atomic) processes. It will be the task of a compiler to decide how the synchronization is done (mutex, sockets, ...) or how the data is shared.

As an analogy, its creator, David DiNucci tells that at the beginning the programming was thought to be sequential, and instructions were written from up to down (remember any assembler program...). Not too late, came program languages with specific syntax for loops, breaking the sequential old style. The programmers added a new dimension to the programs: the indentation. David adds even another dimension to the programs: a dependency layout between tasks and shared data.

After reading for months his work, I think I understand quite well ScalPL. But I must admit, that the first weeks it was very difficult to me to understand his article Introduction to ScalPL and L23. Now I think that I cannot think of almost any limit programming with ScalPL - but it requires adapting the programs to its syntax; it's not strange that we are already used to mutexes, semaphores, signals, mixed with sequential programs running on multiple threads.

ScalPL is based on Software Cabling; SC is based on F-Nets. I recommend you reading the following, where each complements each other:
Introduction to ScalPL and L23
Towards a Max Benefit, Min Cost, Parallel/Grid Programming Model - Slides, but very helpful.
Tolerant (Parallel) Programming with F-Nets and Software Cabling
F-Nets and Software Cabling: Deriving a Formal Model and Language for Portable Parallel Programming

Google scholar may help finding more articles, but I think only David DiNucci wrote about that. Having an IEEE account will help finding some few more interesting papers to read.

The bad point of all that is that there isn't any editor for ScalPL, neither is a compiler. Amazed by the possibilities of the language, I decided to start writting a compiler of a text-written scalpl. It's not a big success, but I can almost run a very simple subset of the language. I don't have many time to code, but I think often on it - so I hope I can improve it from time to time. Once it's usable for anyone, I'll make a post at least here.

Ah, a new mail list has been created for discussing on ScalPL: scalpl@yahoogroups.com (scalpl-subscribe@yahoogroups.com iirc for getting in)

Multithreading and "volatile"

Fri, 15 Dec 2006 17:53:57 GMT

I never had in mind that I should use volatile in certain cases...

Here is a nice article from DrDobb's portal on that: http://www.ddj.com/dept/cpp/184403766

Deadlocks handling

Thu, 09 Nov 2006 14:03:41 GMT

Igor Odintcov wrote about deadlocks handling in his book "Professional programming. System approach". Here is my free translation from russian:

There are four necessary conditions for deadlock to appear:

1. Condition of mutal execution (threads require exclusive resource ownership).
2. Condition of waiting (threads hold some owned resource and wait to own other one).
3. Condition of undistributed resources (resource cannot be taken away while it will not be released by owner thread).
4. Condition of cycled waiting (There are circle of threads where one thread holds one or more resources which is required by next one).

So that there are four main methods to operate with deadlocks:
1. Full ignorance of a problem ("straus algorithm"). Real operation systems don't strive against deadlocks because it is quite enough of resources.

2. Deadlock prevention. To prevent deadlock it is enough to break at least one condition.
- First condition is natural for some kind of resources (such as input\output devices) and so that this condition cannot be broken in some cases.
- To broke second condition a thread has to acquire all resources at once. Effectiveness of a system can become much worse.
- To broke third condition thread has to release all held resources if it is refused to acquire some more resources and try to acquire this resources after a time. But the most part of the work can be lost in this case.
- To broke fourth condition one has to introduce linear order of resources, and allow holding resources only in order where no deadlocks possible. This method requires some more additional expenditure to store information about resource types and order of resource instances.

3. Deadlock avoidance. This method provides rational resources acquiring by rational rules. The most known algorithm of deadlock avoidance is algorithm of banker.
- Algorithm imitates banker actions which has source of capital, grants a loans and receive a payments. This algorithm is proposed by Deykstra.
- The system state is stable if it can process all tasks of all users in finite quantity of time. The system will process only those tasks which will leave system in stable state.
- There are two serious limitation of this method: each thread has to know how many resources it need beforehand and in each moment of a time holds only one resource. This algorithm is inapplicable on practice because of inefficiency (thread has constantly recalculate quantity of required resources).

4. Deadlock detection. This method supposes that deadlock can happen, detect deadlocked threads and resources and try to resolve this situation.

lwsync library update.

Mon, 23 Oct 2006 15:27:42 GMT

I have updated my lwsync library. Monitor Waiting Cancellations feature was added.
Also it was reviewed and approwed on Code Project, so new article location is here. Note that it still request for comments or proposals.

New synchronization stuff for Windows Vista.

Fri, 13 Oct 2006 09:10:45 GMT

I have found here in MSDN a new synchronization stuff which will be available in Windows Vista and Windows Server "Longhorn".

There will be added the following functionality:
- Read/Write Lock and Condition Variables (at long last!). Condition variable will have an ability to sleep with critical section or RWLock object.
- Some Init Once stuff.
- Some additional Interlocked functions such as InterlockedBitTestAndSet\Reset and InterlockedCompare64ExchangeAcquire128
- Private Namespaces.

pthread in Win32

Mon, 09 Oct 2006 11:18:34 GMT

There is an implementation of pthread interface for Win32 platform.

It will be very interesting to test if I can cancel some thread blocked with system implementation of accept() or read() with Win32 version of pthread_cancel()

Raw sockets in Win32

Mon, 09 Oct 2006 11:10:49 GMT

Я наконец-то понял в чём прикол с сокетами.
Наконец то до меня дошло кто в чём промахнулся :)

В POSIX-е если сокет выполняет какую нибудь блокирующую операцию то для того что-бы эту операцию прервать, можно воспользоваться из другого потока например pthread_cancel (культурный способ) или на крайний случай pthread_kill (это не аналог TerminateThread, у потока ещё остаются шансы).

В винде первый сокетный интерфейс оставили POSIX-ным, а интерфейс потоков сделали полностью свой. И "закенселить" поток, который блокирован например accept-ом, невозможно.

Можно конечно в винде некультурно закрыть сокет через closesocket - но это приведёт к race condition-ам.

И что получается в Win32 без winsock2 невозможно культурно разблокировать поток работающий с сокетами?

stress-flow

Tue, 26 Sep 2006 08:20:25 GMT

StressFlow concept.

The goal of StressFlow is to provide a new, universal, versatile parallel programming technique for all parallel and distributed platforms. StressFlow allows unprecedented ease of true parallel programming as well as smooth transitioning from control-flow to superior, unprecedented data-flow type ideas.

Applying lock and wait synchronization in C++

Fri, 22 Sep 2006 10:11:33 GMT

RFC.

Here is my article about applying lock and wait synchronization in C++. Two patterns are implemented: critical resource and monitor. Critical resource allows one to automatically synchronize all accesses to some object. Monitor allows one to wait until object becomes in some defined state. State is can be determinated by predicate.

Books

Fri, 22 Sep 2006 09:32:41 GMT

The recommended books to start are: Doug Lea, "Concurrent Programming in Java™: Design Principles and Patterns", (Lea's online supplement page). Edition 3 coming soon. Java Concurrency in Practice by Brian Goetz with Tim Peierls, Joshua Bloch, Joseph Bowbeer, David Holmes and Doug Lea. (sorry, couldn't find better links to the gods' homepages).

Most of the Java principles will work for .NET, you know why. Though the memory models differ, especially after Java 1.5. (if you wish, please read Does this fix the double-checked locking problem?, thanks upstartn for links).

For not-so-easy C++, see Boost.Threads library.

Multithreaded programming community is created.

Fri, 22 Sep 2006 08:45:14 GMT

New community about multithreaded programming is created.
You are welcome to join it.