Skip to Content

Efficient alternatives to merge for larger data.frames R

I am looking for an efficient (both computer resource wise and learning/implementation wise) method to merge two larger (size>1 million / 300 KB RData file) data frames.

"merge" in base R and "join" in plyr appear to use up all my memory effectively crashing my system.

Example

load test data frame

and try

test.merged<-merge(test, test)

or

test.merged<-join(test, test, type="all")  
    -

The following post provides a list of merge and alternatives:

How to join data frames in R (inner, outer, left, right)?

The following allows object size inspection:

https://heuristically.wordpress.com/2010/01/04/r-memory-usage-statistics-variable/

Data produced by anonym