Skip to Content

RHadoop updated: improved performance and more control

David Smith's picture

Revolution Analytics' open-source RHadoop project, which provides integration between R and Hadoop, has been updated with the release of version 1.2 of the "rmr" package. New in this version: support for binary I/O formats, which improves on the text-only interfact by allowing use of faster and more space-efficient data formats like R's native serialization format. This version also improves the performance of the reduce step (to get around the fact that list appends in R are not constant-time operations), and provides control to the Hadoop user to do things like set number of reducers on a per-job basis.

Find more details about these and other updates in rmr 1.2 (available now) at the link below.

RHadoop: Overview of rmr v1.2