-
Notifications
You must be signed in to change notification settings - Fork 975
Installation
Linux, Mac and Windows
install.packages("data.table") # install it
library(data.table) # attach it
example(data.table) # run the examples section of ?data.table
?data.table # read
?fread # read
update.packages() # keep up to date
Please click and read latest news and keep up to date. That page is updated live as we push changes in development. When we release to CRAN we are saying that we are happy with the changes and in general you can rely on them being there in future. It is safe to simply upgrade to development version and give it a go. If it doesn't work out, you can very easily revert back to the CRAN version by removing the package and reinstalling using the commands above.
A compiled Windows binary .zip
file is published. If you are using a recent R version (your major version / first digit number of R matches the current latest major version of R) you can install the package from from a pre-compiled binary file. If you are using an older version of R, then you will need to compile the development version from source, which will require Rtools.
On Windows, when upgrading any package that uses compiled code, it appears to be important to close all R sessions before upgrading. This releases all locks that Windows holds on dlls. To be really sure, reboot too. Then open a new fresh R session. Some users have an
.Rprofile
file that automatically loadsdata.table
whenever any R sessions starts. You probably do not have an.Rprofile
file but if you do, and it loads any packages (which themselves might loaddata.table
), either temporarily rename.Rprofile
or pass--vanilla
when starting R to prevent it from running.Rprofile
. Once you have a single fresh R session without data.table loaded proceed to installation.
install.packages("data.table", repos="https://Rdatatable.gitlab.io/data.table") ## defaults to binary version and only works if you are using a recent version of R
install.packages("data.table", type="source", repos="https://Rdatatable.gitlab.io/data.table") ## installs from source, which requires Rtools; use this for older versions of R
# or, which install only if newer version (by git commit hash) is available
data.table::update.dev.pkg()
To test your installation
require(data.table)
test.data.table()
The full suite of over 5,000 tests runs locally on your machine and should not take more than 2 minutes.
Linux users usually have all tools already installed by default. Mac users will need the correct version of Xcode Command Line Tools, see OpenMP below.
From a clean session (make sure data.table package isn't loaded):
remove.packages("data.table")
install.packages("data.table", type = "source",
repos = "https://Rdatatable.gitlab.io/data.table")
If you would like to revert to CRAN version, from a clean session, do:
remove.packages("data.table")
install.packages("data.table")
Why install.packages
above rather than devtools::install_github
This way you fetch the last version pushed to master which also passed the full test suite. This protects you from pulling master if it happens to be in a broken state. devtools
always pulls master even if a recent push is failing tests. To do that for some reason (e.g. if we've just pushed a fix and you can't wait 15 mins for tests to complete, or if Travis or AppVeyor themselves are not running) :
library(devtools)
install_github("Rdatatable/data.table", build_vignettes=FALSE)
data.table
not detecting OpenMP on Linux
Sometimes (this has been observed on Amazon Linux AMIs), it might happen that installation of data.table
will be without any issue, but while loading the package, you may be told the OpenMP support does not exist. This could be because data.table
might not be able to make use of the latest version of gcc
installed on these machines to detect OpenMP, so you will need to create a Makevars
file in ~/.R (in your home directory) with the following contents to refer to an older version of gcc
, as so:
CC = /usr/bin/gcc64
CXX = /usr/bin/g++
SHLIB_OPENMP_CFLAGS = -fopenmp
Prior to this, make sure that gcc64
and libgomp
(or equivalent) are present on your system.
The clang compiler included in XCode and Command Line Tools does not include an OpenMP runtime. However, the OpenMP runtime can be installed with homebrew or from https://mac.r-project.org/openmp/. Or, you can install an OpenMP enabled compiler like LLVM's clang or GCC.
The basic steps are:
- Install Command Line Tools
- Install OpenMP support, either:
- install OpenMP runtime for clang (libomp), recommended
- or, install LLVM (clang), GFortran, or GCC
- Update ~/.R/Makevars with flags or the preferred compiler.
First, ensure that you have command line tools installed. Do NOT skip this step. It is essential. See https://github.com/Rdatatable/data.table/issues/1692. From the terminal, type:
xcode-select --install
If you get an error message: xcode-select: error: command line tools are already installed, use "Software Update" to install updates
, then you already have command line tools and can proceed to the next step. Else please follow the onscreen instructions and install it first.
NOTE: Some Big Sur updates may cause a condition where xcode command line tools may need to be updated but are not showing as updateable in Software Updater. To check and correct this condition, see the following from Macports
Tested on Big Sur & Monterey.
Option 1.1 Install libomp with brew
Install homebrew if you have not already. After that install the OpenMP runtime (libomp
):
brew update && brew install libomp
Option 1.2 brew-less install
Follow instructions here: https://mac.r-project.org/openmp/. Option 1 may be preferable if you already use brew.
add the following lines to ~/.R/Makevars
.
On Intel Macs:
CPPFLAGS += -Xclang -fopenmp
LDFLAGS += -lomp
On Apple Silicon Macs Issue 5419 (make sure this goes at the end of the file):
LDFLAGS += -L/opt/homebrew/opt/libomp/lib -lomp
CPPFLAGS += -I/opt/homebrew/opt/libomp/include -Xclang -fopenmp
or simply compile data.table directly from the command line with:
PKG_CPPFLAGS='-Xclang -fopenmp' PKG_LIBS=-lomp R CMD INSTALL path/to/data.table_<VERSION>.tar.gz
Choose one of the following OpenMP enabled compiler installs:
brew LLVM/clang
brew update && brew install llvm
Macports clang (tested on macOS 10.15.6)
sudo port install clang-10 pkgconfig
sudo port select clang mp-clang-10
## if you get: Object `datatable.so` not found
# sudo ln -s /opt/local/bin/pkg-config /usr/local/bin/pkg-config
GNU Fortran complier (tested on macOS Big Sur 11.1)
Download and install GNU Fortran complier from CRAN, which contains full GCC. CRAN recommends official GNU Fortran 8.2 from F.X.Coudert.
Then add GCC version Makevars presented below to the file ~/.R/Makevars
.
This options requires less disk storage (<300MB) than llvm (1.4GB).
GNU Fortran compiler (tested on Apple Silicon M1 with macOS Ventura 13.4.1 and GFortran 12.2)
I had a working data.table with openMP support in macOS Monterey but upgraded to Ventura, needed to install MacPorts for Ventura again, had to install XCode 14.3.1 CLT again.
xcode-select --install
I also had an old version of GCC which caused the error
<built-in>: error: unknown value '13.4.0' of '-mmacosx-version-min'
I downloaded gfortran-12.2-universal.pkg which includes GCC and adjusted my .R/Makevars
to the actual directories and file names in lines 1--3:
LOC = /opt/gfortran
CC=$(LOC)/bin/aarch64-apple-darwin20.0-gcc -fopenmp
CXX=$(LOC)/bin/aarch64-apple-darwin20.0-cpp -fopenmp
CXX11 = $(LOC)/bin/g++ -fopenmp # for fst package
CFLAGS=-g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe
CXXFLAGS=-g -O3 -Wall -pedantic -std=c++11 -mtune=native -pipe
LDFLAGS=-L$(LOC)/lib -Wl,-rpath,$(LOC)/lib
CPPFLAGS=-I$(LOC)/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
This worked to install data.table 1.14.9 from source in RStudio and also in a terminal R session:
install.packages("data.table", type = "source", repos = "https://Rdatatable.gitlab.io/data.table")
GCC
brew install gcc
Add the following lines to the file ~/.R/Makevars
using your favourite text editor. It's likely you need to create the .R
directory and the file Makevars
in it if it doesn't already exist. Note that the path to homebrew can vary depending on your Mac's architecture (see also #5271).
LLVM clang
# Newly installed Homebrew is located in
# - /opt/homebrew for ARM Macs (M1 and its successors)
# - /usr/local for Intel Macs
HOMEBREW_LOC=/opt/homebrew
# If you downloaded llvm manually above, replace with your chosen NEW_PATH/clang
LLVM_LOC=$(HOMEBREW_LOC)/opt/llvm
CC=$(LLVM_LOC)/bin/clang -fopenmp
CXX=$(LLVM_LOC)/bin/clang++ -fopenmp
# -O3 should be faster than -O2 (default) level optimisation ..
CFLAGS=-g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe
CXXFLAGS=-g -O3 -Wall -pedantic -std=c++11 -mtune=native -pipe
LDFLAGS=-L$(HOMEBREW_LOC)/opt/gettext/lib -L$(LLVM_LOC)/lib -Wl,-rpath,$(LLVM_LOC)/lib
CPPFLAGS=-I$(HOMEBREW_LOC)/opt/gettext/include -I$(LLVM_LOC)/include
For macOS 10.14.5+ you may need to supplement the last line as follows (see #3698) :
CPPFLAGS=-I$(HOMEBREW_LOC)/opt/gettext/include -I$(LLVM_LOC)/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
If you also intend to use the fst
package, please refer to this issue.
Variant for Macports users
CC=/opt/local/bin/clang -fopenmp
CXX=/opt/local/bin/clang++ -fopenmp
# -O3 should be faster than -O2 (default) level optimisation ..
CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe
CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -g -O3 -Wall -pedantic -std=c++11 -mtune=native -pipe
LDFLAGS=-L/opt/local/lib -Wl,-rpath,/opt/local/lib
CPPFLAGS=-I/opt/local/include
See also variant for Macports users on Ventura on Apple Silicon above.
GCC (Official GNU fortran) ver
LOC = /usr/local/gfortran
CC=$(LOC)/bin/gcc -fopenmp
CXX=$(LOC)/bin/g++ -fopenmp
CXX11 = $(LOC)/bin/g++ -fopenmp # for fst package
CFLAGS=-g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe
CXXFLAGS=-g -O3 -Wall -pedantic -std=c++11 -mtune=native -pipe
LDFLAGS=-L$(LOC)/lib -Wl,-rpath,$(LOC)/lib
CPPFLAGS=-I$(LOC)/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
GCC Brew (Option 5 above)
VER=9 # set the version of brew gcc. (as of today: 11)
CC=gcc-$(VER) -fopenmp # brew gcc nicely creates gcc-9 as symlink
CXX=g++-$(VER) -fopenmp
CFLAGS=-g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe
CXXFLAGS=-g -O3 -Wall -pedantic -std=c++11 -mtune=native -pipe
Save and close the file. Then you should be all set on Mac. The -fopenmp
tag on CC
and CXX
lines might cause compile problems for other packages, notably stringi
. Simply remove -fopenmp or comment #
after compiling data.table
. RStudio IDE reads those flags thus its definition jump functionality for C or C++ codes may get affected. Remove the above added lines if you need that.
Installing R via homebrew might cause additional problems during installation and is not recommended. However, if you want to install it using Homebrew you can find extended instructions to install R and data.table
in this blog post.
It has been reported that gettext
is needed to be installed too. There are several posts online on how to do that, and it may depend on which version of MacOS you have.
If you encounter a stdio.h file not found
error after following the above instructions, install the package at /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg
so as to create a /usr/include
directory (this no longer works for MacOS 10.15+, see here).
To install this package, you can either double click the file or run
sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /
clang-10: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [data.table.so] Error 1
Check the solution described in this issue.
If you encounter a shared object ‘datatable.so’ not found
error after following the above instructions, you may try installing pkg-config
using homebrew:
brew install pkg-config
and then re-attempting the installation from a clean R-session (see #4574).
On macOS 15 you might encounter this error while compiling stdio.h and/or stdlib.h. This is due to the fact that LLVM/Clang 11 automatically adds -Werror,-Wundef-prefix=TARGET_OS_ as default compiler. Apparently TARGET_OS_MAC isn't defined in this compilation environment. Adding -Wno-undef-prefix to the last line of your Makevars file solves the issue.
On macOS you might encounter this error Library not loaded: /opt/X11/lib/libSM.6.dylib
during installation. You may try installing XQuartz using homebrew (see here):
brew install xquartz --cask