Monday, March 12, 2007

Dreaming of PLINQ

PLINQ is project that aims to parralellize your LINQ queries so that they can make use of the extra processing power available on multi-core machines. I'm very much hoping that it will be released in the Orcas timeframe because it's such a killer app. It will give .NET a HUGE advantage over most other mainstream languages including Python, Javascript, Ruby, and Java. People tend to believe that LINQ queries are only good for data access because they look so much like SQL. In fact a huge number of general programming problems that have nothing to do with database access can be expressed as queries. For example here is a function that multiplies two matrices:



I wrote the To2DimensionalArray method and here is the implementation if you are interested:



If you wrote this algorithm using nested for loops it would be a set of instructions to build a machine that would give you the answer. On the other hand the query is the answer. The importance of this distinction cannot be overstated. Due to the fact that the query describes the intent of the algorithm the computer is free to determine how to best execute it. The PLINQ query processor will analyze the query at run-time, giving it access to information about available system resources and allowing it to formulate a strategy that makes the best possible use of them. It could conceivably be implemented differently every time your query is run depending on how many processors are idle, how much memory there is, etc! This is yet another good reason to express every algorithm possible as a LINQ query. The more processors are added to desktops in the future, the faster your code will run.

The key to this process is the Expression Tree. PLINQ converts your query code into an expression tree in order to analyze it. I predict that languages without the ability to morph code into data will either adapt or be left in the dust in the future. There are simply too many powerful uses for this technique.

9 comments:

Anonymous said...

I have to say, reading your blog, and one or two others like it, has got me very excited about the future of C#. One shortcoming I have is a solid understanding of functional programming, which seems to be key to implementing solutions in this fashion. Can you recommend any good resources to help me get into this mode of thinking? I'm always trying to add more tools to my tool belt.

Jafar Husain said...

1. Structure and Interpretation of Computer Languages
(http://www.swiss.ai.mit.edu/classes/6.001/abelson-sussman-lectures/)

It's an MIT class taught in LISP with textbook and video all available online! Don't worry about LISP, it's probably the simplest general programming languages there is. You'll pick it up in no time. Almost everything you need to know about functional programming you can learn this way.

2. Why functional programming matters. (http://www.swiss.ai.mit.edu/classes/6.001/abelson-sussman-lectures/)

This is a paper that describes the benefits of functional programming. It uses Haskell for its examples. Haskell is a pure functional language that is very new (98) and is one of the forefathers of C# 3.0.

Anonymous said...

Hi,
In you comment you wrote both the "Structure and Interpretation of Computer Languages" and the "Why functional programming matters" with the same link (URL) - (http://www.swiss.ai.mit.edu/classes/6.001/abelson-sussman-lectures/)

Is it a mistake?
And bisides, aren't those courses a little bit old(1981)?

Anonymous said...

Awesome! Thanks very much. This is better than what I was expecting.

Jafar Husain said...

Vladamir: You're right the link was a mistake. Here's the correct link to Why functional programming matters (http://www.math.chalmers.se/~rjmh/Papers/whyfp.pdf)

As for the courses being old it's not really important. That's because as far as programming language theory goes, most of the important stuff was discovered 40 years ago. :-) There's been some new stuff since, but even that is pretty old.

Haskell is relatively new (1998) and it picks up where SICP leaves off with lazy evaluation, monads, and so on.

Jafar Husain said...

URL got cut off in the comment, here is for the final time:

http://www.math.chalmers.se/~rjmh/Papers/whyfp.pdf

Judah Gabriel Himango said...

Jafar, you'll be happy to know that Joe Duffy, the mind behind PLINQ, has resigned as a CLR program manager to work full-time on PLINQ.

Jafar Husain said...

I heard. I'm dying for some more info on PLINQ. Getting so excited about technology that's not going to come out for so long is exhausting. :-)

In the meantime I'm amusing myself with Haskell.

Anonymous said...

I like Archlord gold very much. Since I entered into this game, I learnt skills to earn Archlord money. Thanks to archlord online Gold let me know a lot of friends. It is my habit to buy Archlord gold, and I get some cheap Archlord gold from my friends and Internet.

I like to play Asda Story, because I like its name, also I like Asda Story gold. My friend told me that she would buy Asda Story money for me, and I was so happy. I do not like to go shopping, because it always spends a lot of money, but I never hesitate to buy Asda Story Gold. You can buy cheap Asda Story gold; it is so easy and convenient.

About Me

My photo
I'm a software developer who started programming at age 16 and never saw any reason to stop. I'm working on the Presentation Platform Controls team at Microsoft. My primary interests are functional programming, and Rich Internet Applications.