CrashPlan

From Segfault
Jump to navigation Jump to search

Prerequisites

We're installing OpenJDK, removing the older[1] SunJDK[2] packages:

apt-get install openjdk-8-jre-headless
apt-get purge sun-java6-bin sun-java6-jre                # Only if needed

We should have something like this now:

$ dpkg --get-selections | grep java
ca-certificates-java                            install
java-common                                     install
openjdk-8-jre:amd64                             install
openjdk-8-jre-headless:amd64                    install
tzdata-java                                     install

Or, for Fedora:

$ rpm -qa | grep java
java-1.8.0-openjdk-headless-1.8.0.91-2.b14.fc23.i686
tzdata-java-2016d-1.fc23.noarch

Installation

We'll create a user CrashPlan will run as:

DIR=/opt/crashplan                                       # Or wherever CrashPlan has been installed

$ useradd -d $DIR -m -s /bin/false -U crashplan
$ id crashplan 
uid=1002(crashplan) gid=1002(crashplan) groups=1002(crashplan)

Now on to the actual installation:

$ su -s /bin/bash - crashplan 
$ tar -xzf /tmp/CrashPlan_3.0.3_Linux.tgz 
$ cd CrashPlan-install/
$ ./install.sh 
[....]
Would you like to start CrashPlanDesktop? (y/n) [y] n

Now that CrashPlan is installed (notice that we did not start the GUI), we'll fix a few permissions and file ownerships:

DIR=/opt/crashplan                                       # Or wherever CrashPlan has been installed

cd $DIR
rm -rf .bash_history CrashPlan-install
chown -R root:crashplan .
chown -R crashplan:crashplan .crashplan
chmod 0750 .
chmod -R o-rwx .

mkdir -p tmp
chmod -R g+rw       log/ conf/  lang/ tmp/ cache/ manifest/

Almost done. A few quirks are still left:

Modifications

DIR=/opt/crashplan                                       # Or wherever CrashPlan has been installed

mkdir $DIR/tmp
sed "s|TARGETDIR/\${NAME}\.pid|TARGETDIR/log/\${NAME}\.pid|" -i.bak $DIR/bin/CrashPlanEngine
sed "s|\"$| -Djava.io.tmpdir=$DIR/tmp\"|"                    -i.bak $DIR/bin/run.conf
sed "s|FINE|WARN|;s|INFO|WARN|"                              -i.bak $DIR/conf/service.log.properties
  • The first command makes sure the PID file gets written into log/, which is writable for our CrashPlan user.
  • The second one fixes a java exception we got, because /tmp is mounted with noexec here. Make sure tmp is accessible!
  • Finally, we reduce the loglevel, so our logfiles are not getting spammed with debug information. You might want to postpone this change until things are up & running.

With all that in place, we can start the CrashPlan engine:

su -s /bin/sh -c "$DIR/bin/CrashPlanEngine start" crashplan

The engine will listen on 127.0.0.1:4243 - that's where our desktop client has to connect to. We'll use the desktop client only to configure and schedule the backups. The actual backup jobs will be run by the engine on the server and we can just close the desktop client at any time. Enjoy! But make sure your restores are working too! :-)

Try to limit what the crashplan user can delete during an (automatic) upgrade:

chmod -c 0440 $DIR/.bash*
chmod -c 0460 $DIR/.bash_history
chmod -c 0550 $DIR/.ssh/
sudo chown -c root:crashplan $DIR/.bash* $DIR/.ssh/

Or, with chattr:

chown -Rc crashplan:crashplan ~crashplan/.{aliases,bash*,profile,ssh}
chmod -c go-rwx ~crashplan/.{aliases,bash*,profile,ssh}
chattr +i  ~crashplan/.{aliases,bash*,profile,ssh} ~crashplan/.vnc/passwd*
chattr -i  ~crashplan/.bash_history                      # Ease up on .bash_history...
chmod 0600 ~crashplan/.bash_history

When installing CrashPlan for MacOS X, the local CrashPlan engine is not needed and can be disabled:

launchctl unload -w /Library/LaunchDaemons/com.crashplan.engine.plist

Import

To import an already configured CrashPlan installation, copy the following files to the installation directory:

DIR=/opt/crashplan                                       # Or wherever CrashPlan has been installed

cp conf/{my.service.xml,service.log.properties,service.login,service.model} $DIR/conf/
cp -a .crashplan $DIR/                                   # This should contain the .identity file

Headless

While the CrashPlan engine runs in headless mode anyway, the desktop application can be used to connect to the (headless) CrashPlan engine on another computer[3], preferably via SSH port forwarding.

Install CrashPlan on the local computer, but don't start the engine:

$ cd crashplan-install/ && ./install.sh
Welcome to the Code42 CrashPlan installer.
[...]
Your selections:   
CrashPlan will install to: /opt/crashplan
And store data in: /opt/crashplan/manifest

After the installation, the engine may have already been started, so let's terminate it again:

pkill -f /opt/crashplan/jre/bin/java

We'll need the correct .ui_info file from the machine where the Crashplan engine is running:

sudo ln -s /opt/crashplan /var/lib/crashplan
ln -s $HOME/.ui_info /var/lib/crashplan/.ui_info

scp crashplan@server:.crashplan/.ui_info $HOME/

Create a tunnel to the remote engine:

ssh -L 4243:localhost:4243 crashplan@server

Now we can start CrashPlanDesktop, which will in turn connect to our remote backup engine.

Tuning

Sometimes CrashPlan might print the following error message

inotify_add_watch: No space left on device

or:

com.code42.jna.inotify.InotifyManager   ] Unable to add watch for path ... errno: 28

This can be helped by increasing max_user_watches:

$ sysctl fs.inotify.max_user_watches
fs.inotify.max_user_watches = 8192

$ sysctl -w fs.inotify.max_user_watches=1048576          # 1024^2

Also, since CrashPlan somestimes handles a lot of files at once, it might be necessary to increase fs.file-max:

fs.file-max = 128262               # maximum number of file-handles that the Linux kernel will allocate
fs.file-nr  = 2672  0   128262     # number of allocated file handles, number of allocated but unused
                                   # file handles, and the maximum number of file handles.
fs.nr_open  = 1048576              # maximum number of file-handles a process can allocate

Also, keep an eye on the shell's open file limit:[4]

$ ulimit -n
1024

The Java environment can be controlled in bin/run.conf[5] where some more parameters[6] can be set:

  • Use -Xmx2048m to increase the Java memory allocation pool[7]
  • Try setting -Dc42.native.md5.enabled to true and see if it affects performance on that particular platform.
  • Set JAVACOMMON in install.vars to different Java runtime.

Update

CrashPlan Pro

From time to time CrashPlan tries to update itself, this

Be sure to check if the modifications from above are still in place. If e.g. run.conf has been overwritten, CrashPlan might want to start with /tmp as its place for temporary files - but this won't succeed if /tmp is mounted with noexec:

[02.14.14 22:14:42.707 ERROR   main                 backup42.service.backup.BackupController] Exception setting up BackupController! java.lang.UnsatisfiedLinkError: /tmp/jna5144
430084218904500.tmp: /tmp/jna5144430084218904500.tmp: failed to map segment from shared object: Operation not permitted, java.lang.UnsatisfiedLinkError: /tmp/jna5144430084218904
500.tmp: /tmp/jna5144430084218904500.tmp: failed to map segment from shared object: Operation not permitted
java.lang.UnsatisfiedLinkError: /tmp/jna5144430084218904500.tmp: /tmp/jna5144430084218904500.tmp: failed to map segment from shared object: Operation not permitted
        at java.lang.ClassLoader$NativeLibrary.load(Native Method)
        at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1965)
        [...]

NOTE: The following is only needed if the installation directory does not belong to the crashplan user!

We have to temporarily alter the permissions so that the update process succeeds:

DIR=/opt/crashplan                                       # Or wherever CrashPlan has been installed

sudo chgrp -R crashplan $DIR
sudo chmod -R g+w $DIR

Now the update should succeed. After the update is done the permissions need to be reset:

cd $DIR
sudo chmod -R g-w .
sudo chmod -R g+rw log/ conf/ lang/ tmp/ cache/ manifest/

Also, let's make a few files immutable, so that the upgrade routine won't be able to remove them:

cd $DIR
chattr -VR +i .aliases .bash_profile .bashrc .profile .ssh/ .vnc/{config,passwd*,xstartup}
chattr -V +a .bash_history

Crashplan SMB

With the move to CrashPlan for Small Business[8][9] it was noted that the 32-bit x86 architecture was being retired:

> Devices running 32-bit Linux systems remain unsupported. 
> Beginning with version 6.6.0, the CrashPlan for Small Business app
> does not function on 32-bit Linux devices

In fact, their install routine now downloads a 64-bit JRE:

$ tar -xzf CrashPlanSmb_6.6.0_1506661200660_4347_Linux.tgz
$ grep ^JRE crashplan-install/install.defaults
JRE_X64_DOWNLOAD_URL=https://download.code42.com/installs/proserver/jre/jre-8u121-linux-x64.tar.gz

It also should be noted that an existing Crashplan v4.x version will automatically upgrade to 6.x - and then cease to function if it cannot execute the 64-bit Java binary. Starting Crashplan with the system's OpenJDK or even with Java SE did not work, as several Code42-specific classes could not be found.

The only viable workaround so far is to disable disallow an existing Crashplan installation to upgrade. That means, locking down everything except few directories with the Crashplan installation directory:

mkdir -p restore tmp
chmod -Rc go+rX .

sudo chown -R root:root .
sudo chown -R crashplan:crashplan .bash_history cache/ conf/ log/ restore/ tmp/ 

This seems to hold off the upgrades so far:

12.17.17 00:55:33.591 WARN  0105_DwldMgr .service.upgrade.DownloadManager] DOWNLOAD:: Unable to download patch. url=https://download.crashplan.com/installs/linux/upgrade5/crashplansmb/1506661
200660_4347.jar?guid=488943415928029291, FileTransferException [srcPath=/linux/upgrade5/crashplansmb/1506661200660_4347.jar, errors=[DST_IO_ERROR]]; DownloadManager[patchDir = upgrade, patchFi
les = [1506661200660 (2017-09-29T05:00:00:660+0000)], patchProblem = false], FileTransferException [srcPath=/linux/upgrade5/crashplansmb/1506661200660_4347.jar, errors=[DST_IO_ERROR]]
STACKTRACE:: FileTransferException [srcPath=/linux/upgrade5/crashplansmb/1506661200660_4347.jar, errors=[DST_IO_ERROR]]
        at com.backup42.common.filetransfer.FileTransferBlockingRequestor.<init>(FileTransferBlockingRequestor.java:84)
        at com.backup42.common.filetransfer.FileTransferBlockingRequestorFactory.create(FileTransferBlockingRequestorFactory.java:23)
        at com.backup42.common.filetransfer.FileTransfer.transferFile(FileTransfer.java:353)
        at com.backup42.common.filetransfer.FileTransfer.execute(FileTransfer.java:196)
        at com.backup42.common.filetransfer.FileTransferService.transfer(FileTransferService.java:356)
        at com.backup42.common.filetransfer.FileTransferService.transfer(FileTransferService.java:334)
        at com.backup42.service.CPService.downloadByMessaging(CPService.java:4076)
        at com.backup42.service.upgrade.DownloadManager.downloadPatch(DownloadManager.java:237)
        at com.backup42.service.upgrade.DownloadManager.downloadPatches(DownloadManager.java:181)
        at com.backup42.service.upgrade.DownloadManager.doWork(DownloadManager.java:128)
        at com.code42.utils.AWorker.run(AWorker.java:149)
        at java.lang.Thread.run(Thread.java:748)

Links


References