Brij Singh's Random Musings

MapReduce

Google team would be presenting in OSDI on the topic pf parallel data processing using large clusters. This abstract throws some light on their much talked about super cluster. -

MapReduce is a programming model and an associated implementation for processing and generating large data sets. Users specify a _map_ function that processes a key/value pair to generate a set of intermediate key/value pairs, and a _reduce_ function that merges all intermediate values associated with the same intermediate key.

Programs written in this functional style are automatically parallelized and executed on a large cluster of commodity machines. The run-time system takes care of the details of partitioning the input data, scheduling the program's execution across a set of machines, handling machine failures, and managing the required inter-machine communication. This allows programmers without any experience with parallel and distributed systems to easily utilize the resources of a large distributed system.

Our implementation of MapReduce runs on a large cluster of commodity machines and is highly scalable: a typical MapReduce computation processes many terabytes of data on thousands of machines. Programmers find the system easy to use: hundreds of MapReduce programs have been implemented and upwards of one thousand MapReduce jobs are executed on Google's clusters every day

Will they open source Navision or Great Plains ?

[Via LinuxInsider] Martin Taylor, global general manager of platform strategy at Microsoft

And so we [ask] what if we could actually build an engine leveraging the assets of Great Plains and Navision to allow ISVs to build on top of to reduce their research and development costs and hopefully increase their time to market with applications, and also have a better integration story within Windows?

Will they open source Navision or Great Plains platform ? It's one of those game changing strategies. Think about it, considering the overall threat they are facing from open source/Linux community giving away few billion dollar worth of acquired technology shouldnt be that painful.

Importance of skepticism

Asahi Times on the failure of US media during run-up to the Iraq war -

Why was coverage contrary to the facts allowed to drag on for so long? Noted media researcher Michael Massing blames the following: ``A failure to manifest skepticism and independent spirit-the core work values of journalists.''

We have entered an uncertain era in which there is great concern about the direction in which our nation is moving. It is precisely at times like these that the media must redouble its efforts to temper and refine its spirit of skepticism and independence.

I doubt there will be any far reaching changes except some piecemeal efforts from the big media empires. They need to go through a period of irrelevancy before this problem gets fixed.

Very slick

Check it out - Imagination at work

Punditry ..

Another proof of the slow and sure death of punditry.

No wonders predictors get it wrong most of the time.

Here is my previous rant on this.

xSource

Some of you remember the days when along highway 237 , there was hardly a company to be found whose name didnt end in some form of "NETWORKS". Redback, Caspian, etc etc. Something to be said about the buzz and the prefix/suffix of the start-up names. Current moniker - Source.

When we started Apptility, we started with the idea of filling in the demand/supply gap of open source assurance services and empowering internal IT organizations with the open source support services. At that time the companies involved in this segment were mostly open source "product companies" such as MySQL, JBoss and Covalent among others. Other set of companies which started within 6 to 8 month time interval were on the compliance side, namely - Blackduck and Palamida.

Recent entry of some heavy hitters such as Spikesource and SourceLabs suggests that the support and certification segment of open source is ready for a nice competition. It's good for everybody, more market awareness for the internal IT-driven open source adoption and faster realization of open source as a vendor of choice.

There will be many more start-ups in this space, good scope for healthy competition out there.

Happy days for the open source and happy days for the customer.

Punditry RIP

Fun part of the white house politics in the age of web2.0 is that you can clearly see who is spinning and how.
Check this subtle spin by Mike Mcnamee (deputy editor chief at BusinessWeek) :

Scorned for his scowls and irritability in the first debate on Sept. 30, President George W. Bush made the most of a friendly format in the second. He changed his attitude, went on the offensive, and squeezed out a victory over Senator John F. Kerry at Washington University in St. Louis on the evening of Oct. 8.

Victory ? Maybe I need to check my dictionary for the right meaning of that word. Are we seeing the final phase in the eventual rotting of the punditry business ?

Trust is a big deal here. Factcheck.org is getting increasingly popular because it provides trust in the world full of spin doctors.

Factcheck should not go away after this election, there should be one for the corporate as well. Where CFO's spin can be cross-checked with the marketing guru's whispers to the street guys. Major cleanup possibilities there.

Story of FUD

First and last lesson for all sales hunters. Learn the FUDs and lob it hard and wide.

The concept of FUD enjoys a venerable history in the computing field. According to The Jargon File, an online dictionary of hacker slang, Gene Amdahl used the term as an attack on IBM after he left in the early '70s to found his own company: "FUD is the fear, uncertainty and doubt that IBM salespeople instill in the minds of potential customers who might be considering (Amdahl) products."

In a 1995 case pitting Addamax against the Open Software Foundation and Hewlett-Packard, Addamax claimed that the defendants used FUD to paralyze the industry and unreasonably raise customers' fears. One internal HP memo cited in that case was titled "Impact of FUD on Sun" and discussed ways to sabotage the AT&T-Sun Microsystems operating system by describing it as "nonstandard."

FUD also was used by cryptographers in the 1990s to scare politicians about criminals using data-scrambling encryption products to cloak their communications. More recently, the term has cropped up in the Microsoft and Linux war, with free software advocates using it to describe disinformation they say SCO Group and Microsoft have spread about the merits of software other than Windows.

For sales folks, there is no better weapon than dropping a doubt in the customer's mind and slowing the procurement process to their advantage. After all when CIOs are getting sacked every other day for screwed-up projects, FUD works like a charm.

It's time some non-profit (may be open source type) project comes up and does factcheck.org version for the software companies. Gartners of the world wont do it, since they are party to this game.

Here is my wishlist for the software world factcheck.org

Publish past, current and future claims as made by the software vendors
Publish successful, barely-made-it and screwup projects as implemented by the software vendors
Technology 101 - cold analysis of secret sauce and fluff
How the same software was sold to customers with pricing varying from free to 2million, price variations in the enterprise software world is scandalous
Companies finances and the possibility of a corporate blow-up

What more we can add to this list ?

Skilled Labor out of favor ?

Asked which assets will be most valuable to the company five years from now, 53 percent of respondents said "products/services." Another 26 percent said "brand/reputation," while "management" and "intellectual property" each were selected by about 10 percent. None of the respondents chose "skilled labor."

How did we come to this outcome?

Once upon a time

Once upon a time working for open source project or believing in open source was more about "having fun". It wasnt very long time ago though. Times have changed and changed very fast. JBoss's childish brag about their win over BEA sounds more like a bloodsport much of the commercial software world is characterized as.

Can these guys just focus on executing and building a better software rather than working extra hard and beating the drum of P in their professional open source.

In a not so distant future, you not only need to worry about the software lock-in from proprietary softwares but also be prepared for lock-in from the most "professional" open source software as well.

Why having fun is so hard ?