Monday, December 28

Dean and Ghemawat Strike Back on MapReduce

Jeff Dean and Sanjay Ghemawat wrote an article for the January edition of CACM, MapReduce: A Flexible Data Processing Tool. In the article, they refute the findings of A Comparison of Approaches to Large-Scale Data Analysis. On their blog, the authors also wrote a post bashing MapReduce: MapReduce, A major step backwards. The post is no longer available, but thankfully Greg had good coverage.

In the article Dean and Ghemawat address the paper and attempt debunk its claims, although they lack the benchmarks to back it up. In the process, they inform you about the right way to run M/R jobs efficiently:
  1. Avoid starting processes for each new job, reuse workers.
  2. Careful data shuffling, avoid O(M*R) disk seeks
  3. Beware of text storage formats.
  4. Use natural indices like timestamps on files.
  5. Do not merge reducer output.
They present some good M/R lessons in their refutation. You should be using a binary serialization system like Avro or Protocol Buffers and storing your data in a format that provides efficient access, using a natural file structure or using a database system like HBase.


  1. The "MapReduce is backwards" article moved after some time and now has completely disappeared. I suspect it has something to do with Vertica recently partnering with Cloudera and supporting Hadoop connectivity within the Vertica product. Still, doesn't make Stonebraker's writings any less unbiased.

  2. I love your job jobs efficiently: Avoid starting processes for each new job, reuse workers.
    Careful data shuffling, avoid O(M*R) disk seeks
    Beware of text storage formats.
    Use natural indices like timestamps on files.
    Do not merge reducer output.
    It will Be very helpful for me.
    Delhi Escorts Service

  3. Faith Fully Sweet Independent Delhi Escorts Provide Best Escorts Services in Delhi city. Meet our Delhi escorts, escorts in Delhi.
    Delhi Escorts Service

  4. Day by day the market of escort is too much popular but just because of money most of the escort company offer cheap class escort services and it regret you from escort service and to make it more advanced Delhi escort added some advanced technique with full of pleasure.
    Delhi escort services

  5. This post is very good for us. It has got a lot of benefit from us.I hope that you will be writing this post again Escorts Service In Mahipalpur

  6. We say that you welcome in Delhi, one of the best cities in the world, which is a fully weird and great city. This is not the clerical center of India, but the people city is the main city of India. Woman from outer space click here for acting as well as for state grits
    Independent Sexy Female Delhi Escorts
    Escorts Service in Delhi
    High Profile Delhi Escorts
    High Class Delhi Escorts Service Independent
    Independent Sexy Call Girls Delhi Escorts Service

  7. Helo,High Class Udaipur Escorts Agency, the luxurious Udaipur Escort Agency where you can find the most beautiful Escorts girls in Udaipur.Female Escorts in Udaipur Independent Escorts in Udaipur

  8. Are you a newbie in love making and don’t know how to start and when to end, than our escort service Delhi could be the best teacher for you.Get in touch with us shortly.
    Escort services in Delhi

  9. This post is very good for us. It has got a lot of benefit from us.I hope that you will be writing this post again.
    Nehru Place Call Girls

  10. Much obliged to you for setting aside the opportunity to distribute this data extremely helpful!
    Escorts in Jaipur
    Chandigarh Escorts

  11. If you have come from the West and you are captivated by the Asian beauty, in that case, we make sure that you get the best Call Girls in Mahipalpur to help you taste the Asian flavor in the best way. At Mahipalpur, we make sure that you are getting one heck of an experience with our girls. So, no matter whether it is official meeting or bachelor party, we are there for all your needs. Visit:- Here

  12. Do you need a Call Girls in Gurgaon to give you the real pleasure of your life, well, if you revert in affirmative, in that case, you are at the right place. We are a stupendous track record of providing the best girls that would completely take the evening of yours by storm, so to avail them all you need to do is just call and we will provide the best at the right price. Visit:- Here

  13. Absolutely wonderful it has lots of thing to choose and very entertaining blog thanks.Send Gifts To pakistan

  14. I was thinking before reading that there is looking less information but at last I read figured out all the info and got shocked. Thank of bunch.

    escorts in Delhi

    Noida escorts
    Call girls in Delhi
    Delhi Russian Escorts
    Aerocity escorts
    Mahipalpur escorts
    Karol Bagh escort

  15. Excellent publish it is . I really like it . Thanks for discuss about about an useful publish . Appreciate it and keep publishing.
    Gurgaon Escorts Service |
    Escorts in Gurgaon |
    Female Escorts in Gurgaon |
    Independent Escorts in Gurgaon |
    Affodable Escorts in Gurgaon |
    Gurgaon Escorts

  16. Mb-9643090422 Female 4 Fun Delhi Escorts Agency Offer hi-profile Escorts in Delhi and Book Delhi Escort Service with top class independent Call Girls in Delhi
    Delhi Escorts ||
    Escorts Girls Delhi ||
    New Delhi Call Girls ||
    Escorts Girls In Aerocity ||
    Connaught Place Girls ||
    Call Girls In Delhi ||

  17. Hello Guys first I can introduce myself I am Angel independent Jaipur escorts. I truly enjoy being an elite independent which is immediately evident when you meet me, in Jaipur escort service not about having just only physical relations but give you fully relaxing satisfaction.
    Jaipur Escort
    Jaipur Escort Service
    Jaipur Escorts
    Escort in Jaipur
    Jaipur Call Girls
    Call Girls in Jaipur
    Jaipur Escorts Service
    Independent Jaipur Escort
    Jaipur Escort Service
    Lucknow Escorts Service
    Ludhiana Escorts
    Escort in Lucknow
    Lucknow Escorts

  18. This article was written by a real thinking writer. I agree many of the with the solid points made by the writer.
    Delhi Escorts
    Delhi Call Girls
    Escorts in Delhi

  19. Very interesting blog. Alot of blogs I see these days don’t really provide anything that I’m interested in, but I’m most definitely interested in this one. Just thought that I would post and let you know.
    Delhi Escorts
    Delhi Escorts
    Delhi Escorts
    Delhi Escorts
    Delhi Escorts
    Delhi Escorts
    Delhi Escorts
    Delhi Escorts

  20. Nice blog a lot of information
    Visit our website and hire Escorts Girls by Jaipur Escorts Service