Presented by Owen O'Malley and Eric Baldeschwieler (Yahoo!)
Owen gave a brief overview of the history of Hadoop and the Hadoop ecosystem.
The big news is that on stage Eric announced the release of the Yahoo! distribution, the distribution that Yahoo! uses internally. It's not new software, but a re-packaging of publicly available releases that have been tested internally on Yahoo!'s clusters. They're starting with the 0.20 release of Hadoop as an Alpha release and this will continue to grow and stabilize.
Yahoo now has dozen's of clusters with 25,000+ nodes. The largest cluster has about 4,000 nodes.
Webmap
Contains content and metadata that powers Yahoo! search.
In 2008
- 70 hours runtime
- 300 TB shuffling
- 200 TB output
- 73 hours
- 490 TB shuffling
- 280 TB output
- 55%+ hardware
Cluster stats
In 2008
- 2000 nodes
- 6 PB raw disk
- 16 TB RAM
- 16k CPUs
In 2009
- 4000 nodes
- 16 PB disk
- 64 TB RAM
- 32k CPUs (40% faster cpus)
Major features coming to Hadoop
Core
- Backwards compatibility (0.21 will make the last big API changes)
- Append, sync, and flush support
- Scheduling - Capacity and Fairshare
- Continuous integration - easier to build and test Hadoop distributions
Pig - "Make pigs fly"
- Support for SQL and metadata
- Column oriented storage access layer (a new column-oriented storage view, not services like HBase)
- Multi-query optimizations
Oozie is a new workflow and scheduling system
There was a question about cluster management. Eric recommend that people use Chukwa and Ganglia as open-source tools for large-scale cluster management.
Hi admin, i went through article. As we all know, .net is most popular programming language and it offer huge career prospects for talented professionals. It’s totally awesome, keep on updating your blog with such awesome information. .Net Training in Chennai
ReplyDeleteI am reading your post from the beginning, it was so interesting to read & I feel thanks to you for posting such a good blog, keep updates regularly.
ReplyDeleteRegards,
Python Training in Chennai|Python Training Institutes in Chennai
Thanks for sharing this informative blog. If you are interested in taking .net in professional carrier visit this website.Dot Net Training in Chennai
ReplyDeleteExcellent post!!!. The strategy you have posted on this technology helped me to get into the next level and had lot of information in it.
ReplyDeletesalesforce training in chennai | salesforce training institute in chennai
Really informative post for the candidates who seeking this precious information.
ReplyDeleteReally,
Salesforce Training in Chennai|Salesforce Training
Thank you for this valuable information. Get your business to the next level in simple steps. erp in chennai | erp for automotive industry chennai
ReplyDeleteThanks for your informative post!!! After completing my graduation, i am confused whether to choose web design as my career. Your article helped me to make a right choice. Web Designing Training Institute in Chennai | Web Designing Training Institute in Velachery.
ReplyDeleteThank you for sharing such a wonderful post.. Software Testing Training Institute in Chennai | Software Testing Training Institute in Velachery.
ReplyDeleteInteresting and informative article.. very useful to me.. thanks for sharing your wonderful ideas.. please keep on updating..
ReplyDeletePHP Training in chennai | Web Designing Training in chennai
ReplyDeleteReally an amazing post..! By reading your blog post i gained more information.
SEO MOBILE MARKETING
Thanks for sharing such a informative guide. Your blog gives me detailed information. Android Training Institute in Chennai | IOS Training Institute in Chennai | Best Java Training in Chennai
ReplyDeleteThanks for sharing informative article. Download Windows 7 ultimate for free from getintopc. It helps you to explore full functionality of windows operating system.
ReplyDeleteThanks For Your valuable posting, it was very informative
ReplyDeleteBest Electrical Project Center in Chennai | Best Electrical Project Center in Velachery
thanks for sharing valuable information from your blog..
ReplyDeleteJava Training Center in Chennai | Best J2EE Training Center in Chennai | No.1 Java Training Institution in Velachery | Core Java Training in Chennai
Needed to compose you a very little word to thank you yet again regarding the nice suggestions you’ve contributed here.selenium training in bangalore
ReplyDeleteGood Post..Thanks for sharing such a wonderful article...
ReplyDeletePLC Training in Chennai | PLC Training Institute in Chennai | PLC Training Center in Chennai | PLC SCADA Training in Chennai | PLC SCADA DCS Training in Chennai | Best PLC Training in Chennai | Best PLC Training Institute in Chennai | PLC Training Centre in Chennai | Automation Training in Chennai | Automation Training Institute in Chennai
I have read your blog it's very interesting and informative. Keep sharing.
ReplyDeletePLC Training in Chennai | PLC Training Institute in Chennai | PLC Training Center in Chennai | PLC SCADA Training in Chennai | PLC SCADA DCS Training in Chennai | Best PLC Training in Chennai | Best PLC Training Institute in Chennai | PLC Training Centre in Chennai
I gain more knowledge from your post..Thanks for updating your post..
ReplyDeleteWeb Designing Training Institute in Chennai | DotNet Training Institute in Chennai | Software Testing Training Institute in Chennai
Thanks for splitting your comprehension with us. It’s really useful to me & I hope it helps the people who in need of this vital information.
ReplyDeleteMCA Project Center in Chennai | MCA Project Center in Velachery
I wish to show thanks to you just for bailing me out of this particular
ReplyDeletetrouble.As a result of checking through the net and meeting
techniques that were not productive, I thought my life was done.
java training in chennai
java training in bangalore
java training in bangalore
Your article was really nice. thanks for sharing..Web Designing Training Institute in Chennai | DotNet Training Institute in Chennai | Software Testing Training Institute in Chennai
ReplyDeleteIt has been simply incredibly generous with you to provide openly what exactly many individuals would’ve marketed for an eBook to end up making some cash for their end, primarily given that you could have tried it in the event you wanted.big data training in Chennai
ReplyDeleteI ‘d mention that most of us visitors are endowed to exist in a fabulous place with very many wonderful individuals with very helpful things.
ReplyDeletedotnet training in bangalore
Thank you a lot for providing individuals with a very spectacular possibility to read critical reviews from this site.
ReplyDeletehadoop-training-in-porur
Very interesting article about Hadoop. Thanks for sharing such a wander full post. Best Cloud Solution Services| Austere Technologies
ReplyDeletethanks for giving good information and our massiltechnologies also providing the best services mobile application development and read more details for visit our site Mobile application development Services
ReplyDeleteHi, Very good informative article. Thanks for sharing such nice article, keep on up dating such good articles.
ReplyDeleteNO.1 APP DEVELOPMENT SERVICES | MASSIL TECHNOLOGIES
VERY HAPPY TO SEE YOUR BLOG. IT'S VERY HELP FULL INFORMATION ,THANKS FOR SHARING.
ReplyDeleteORACLE CLOUD SERVICES FOR APPLICATION DEVELOPMENT | MASSIL TECHNOLOGIES
Awesome informative blog. Thanks for sharing.
ReplyDeleteBest Digital Transformation Services | DM Services | Austere Technologies
Great information on this blog, many useful and new ideas i'm learning here.It's very helpful.
ReplyDeleteBest Photoshop Training Institute in Chennai | No.1 Adobe Photoshop Training Institute in Velachery
Great and interesting blog..Thanks for sharing this informative article..
ReplyDeleteME/M.Tech Project Center in Chennai | ME/M.Tech Project Center in Velachery
Extraordinary blog. you put Good stuff. All the themes were clarified briefly.Thanks for sharing that post.Keep in blogging…
ReplyDeleteWeb Designing Training Institute in Chennai | Web Designing Training Institute in Velachery
Thank you for this information.
ReplyDeleteBest VMware Training Institute in Chennai | Best VMware Training Institute in Velachery
very good article, do not forget to visit other interesting articles...
ReplyDeleteBest CorelDraw Training Institute in Chennai | No.1 CorelDraw Training Institute in Velachery
Nice post.Thanks for your informative post. Your article helped me a lot to understand the future of blog.
ReplyDeleteBest AWS Training Institute in Chennai | Best AWS Training Institute in Velachery
Very good informative article. Thanks for sharing such nice article, keep on up dating such good articles.
ReplyDeleteMCA Project Center in Chennai | MCA Project Center in Velachery
Wow. Great resource. I’m new to blogging and have to agree it’s a journey.
ReplyDeleteBest Photoshop Training Institute in Chennai | CorelDraw Training Institute in Chennai | No.1 Photoshop Training Institute in Velachery
Your article is awesome! How long does it take to complete this article? I have read through other blogs, but they are cumbersome and confusing. I hope you continue to have such quality articles to share with everyone! I believe there will be many people who share my views when they read this article from you!
ReplyDeleteBest CorelDraw Training Institute in Chennai | Photoshop Training Institute in Chennai | No.1 Graphics Designing Training Institute in Velachery
Thanks for sharing such a great blog post. I really like your blog post which is so informative.
ReplyDeletewhite label website builder
mobile website builder
Excellent information. Thanks for sharing.
ReplyDeleteBest Mobility Services | Austere Technologies
wow...Great blog, very help full information. Thanks for sharing, keep updating.
ReplyDeleteBest IT Security Services | Austere Technologies
Interesting article is a very awesome in a better ranking improved.
ReplyDeleteGood Graphics Designing Training Institute in Chennai | Best Multimedia Training Institute in Velachery
Your article is detailed, thanks to it I solved the problem I am entangled. I will regularly follow your writers and visit this site daily.
ReplyDeleteExcellent Summer Courses in Guindy | Best Summer Classes in Chennai
Appreciation for really being thoughtful and also for deciding on
ReplyDeletecertain marvelous guides most people really want to be aware of.
white label website builder
Great article, your blog was really unique... thanks for sharing…
ReplyDeleteBest Summer Courses for professional students in Chennai|Best Summer Courses for professional students in Ashok Nagar