Wednesday, November 3

Yahoo! Open Sources S4 Real-Time MapReduce framework

Today Yahoo! announced the release of a new real-time MapReduce framework written in Java called S4. From the website,
S4 is a general-purpose, distributed, scalable, partially fault-tolerant, pluggable platform that allows programmers to easily develop applications for processing continuous unbounded streams of data.
For more technical details you can read the technical overview or check out the code on github. The example application keeps counts of hash tags in a Twitter stream.

The framework was previously announced at a Y! lab event which discussed processing in Y!'s advertising platform.

1 comment: