Apt-get

From Segfault
Jump to: navigation, search

apt.conf

$ cat /etc/apt/apt.conf.d/99local
APT::Default-Release       "stable";
APT::Install-Recommends    "false";
APT::Get::Purge            "true";
Acquire::Languages         "en";
DPkg::Pre-Invoke           { "mount -o remount,exec   /tmp" };
DPkg::Post-Invoke          { "mount -o remount,noexec /tmp" };
// DPkg::Post-Install-Pkgs { "xargs debsums --silent --generate=missing" };
// Acquire::http::Proxy    "http://proxy:3128";
// Acquire::http::No-Cache false;                 // Debian #564829, Debian #703146
// DSelect:Clean           always;                // always|auto|prompt|never
// Acquire::Check-Valid-Until false;              // E: Release file expired[1]
// Acquire::ForceIPv4      "true";

mirror://

Apparenty apt-get now supports[2] mirror:// URLs for quite some time[3]:

$ cat /etc/apt/sources.list
deb mirror://mirrors.ubuntu.com/mirrors.txt     trusty          main universe multiverse
deb mirror://mirrors.ubuntu.com/mirrors.txt     trusty-updates  main universe multiverse
deb mirror://mirrors.ubuntu.com/mirrors.txt     trusty-security main universe multiverse

Note: apt-file can't handle mirror:// links[4].

unattended-upgrades

Unattended Upgrades can be activated like this:

apt-get install unattended-upgrades

The /etc/cron.daily/apt cronjob should already be in place. To actually active unattended-upgrades:

$ cat /etc/apt/apt.conf.d/20auto-upgrades 
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";
$ cat /etc/apt/apt.conf.d/50unattended-upgrades
[...]
Unattended-Upgrade::Origins-Pattern {
   "o=Debian,n=jessie";
   "o=Debian,n=jessie-updates";
   "o=Debian,n=jessie-proposed-updates";
   "o=Debian,n=jessie,l=Debian-Security";
};

// Send email to this address for problems or packages upgrades
Unattended-Upgrade::Mail "root";

debconf

Ignore questions with a priority less than...

dpkg-reconfigure -p low debconf
→ Dialog
→ low

dpkg

dpkg.old

Be sure to do the following after every (dist-)upgrade and see if the new config files require attention:

find /etc/ -name "*.dpkg*" -o -name "*.ucf*"

Display (some) obsolete packages:

dpkg -l | grep -i transitional

deborphan might also be helpful to show leftover packages, not really required by any other package:

$ deborphan --guess-all
libsigc++-2.0-dev

Sometimes deborphan complains about a corrupt status file[5]:

deborphan: The status file is in an improper state.
One or more packages are marked as half-installed, half-configured,
unpacked, triggers-awaited or triggers-pending. Exiting.

Check the status file with:

egrep '^Status: |^Package: ' /var/lib/dpkg/status | \
 egrep -B 1 'half-installed|half-configured|unpacked|triggers-awaited|triggers-pending'

...and run dpkg --audit or dpkg --configure --pending afterwards, this should help.

get-selections

And then there's dpkg itself, listing installed packages with their current status:

$ dpkg --get-selections | grep -Pv '\tinstall$'
bsdgames-nonfree                                purge
hfsutils                                        deinstall
unrar                                           hold

dpkg-buildpackage

How to rebuild a Debian package, e.g. sitecopy:

sudo apt-get install fakeroot debhelper dpatch

Maybe install package-specific build dependencies:

sudo apt-get install libexpat1-dev libgnutls-dev libneon27-gnutls-dev libxml2-dev
apt-get source sitecopy
cd sitecopy*
patch -p1 < ~/sitecopy.diff                      # Some custom patch(es) may be applied
dpkg-buildpackage

dpkg-query

With dpkg-query the package database can be queried. The --showformat option accepts over 30 different fieldnames, listed in the man page. To show all known fields for a package, the following can be used:

$ dpkg-query -W -f="$(for i in `man dpkg-query | grep -A32 Architecture | awk '{print $1}'`; do \
       printf "### $i: \${$i}\\\n"; done | sed 's/\\n $//')" cowsay
### Architecture: all
### Bugs:
### Conffiles:
### Config-Version:
[...]

To find out which packages were installed from a different section[6], use:

$ dpkg-query -W --showformat='Section: ${Section}\tPackage: ${Package}\n' | fgrep / | sort -k2
Section: contrib/doc             Package: gcc-doc
Section: contrib/doc             Package: gcc-doc-base
Section: contrib/games           Package: game-data-packager
Section: contrib/games           Package: quake2
Section: contrib/games           Package: quake3

However, that still doesn't show us from which releases a package was installed from. For that, we have to use apt-show-versions:

$ apt-show-versions | grep /unstable
broadcom-sta-dkms:all/unstable 6.30.223.271-7 uptodate
dnsdiag:all/unstable 1.6.3-1 uptodate
enigmail:all/unstable 2:1.9.8.2-1 uptodate

With apt-show-versions we can also find out which (installed) packages can no longer be found in our sources.list:

$ apt-show-versions | grep No\ avail
automake1.4:all 1:1.4-p6-13.1 installed: No available version in archive
automake1.9:all 1.9.6+nogfdl-4 installed: No available version in archive

We could employ apt-cache for this too, but it's ever so ugly:

$ dpkg --get-selections | awk '{print $1}' | while read p; do
     printf "$p\t"
     apt-cache showpkg "$p" | awk '/^[0-9].*Packages\)/ {print $2}' | head -1
done | grep -v main
deborphan       (/var/lib/apt/lists/mirrors.ubuntu.com_mirrors.txt_dists_vivid_universe_binary-amd64_Packages)
debsums (/var/lib/apt/lists/mirrors.ubuntu.com_mirrors.txt_dists_vivid_universe_binary-amd64_Packages)
haveged (/var/lib/apt/lists/mirrors.ubuntu.com_mirrors.txt_dists_vivid_universe_binary-amd64_Packages)

If aptitude[7] is installed:

$ aptitude search ~i -F "%s# %p" | grep /
universe/admin                deborphan
universe/admin                debsums
universe/misc                 haveged
universe/mail                 heirloom-mailx

Listing installed packages and their sizes can be quite the challenge:[8]

$ dpkg-query -Wf '${Installed-Size}\t${Package}\n' | sort -n
[...]
32483   mariadb-server-core-10.3
35266   git
69075   mariadb-server-10.3
258084  linux-image-4.19.0-2-amd64
263504  linux-image-4.19.0-4-amd64

However, that's only the installed size (in KB), the real size of an installed package can be quite different:

$ apt-cache show git | grep Size
Installed-Size: 35266
Size: 5621352

The Size parameter however is almost always NULL when queried with dpkg-query and according to its man page, it's an internal value anyway. So how do we get the package size? One way would be to sum up all the file sizes of each package:

$ dpkg -L git | xargs ls -ld | awk '/^-/ {sum+=$5} END {print sum/1024/1024, "MB"}'
33.9257 MB

And for every package:

$ for p in $(dpkg -l | awk '/^i/ {print $2}'); do printf "$p\t"; dpkg -L $p | grep ^/ | xargs ls -ld | awk '/^-/ {sum+=$5} END {print sum/1024/1024, "MB"}'; done | sort -nk2
[...]
mariadb-server-core-10.3         31.658  MB
git                              33.9257 MB
mariadb-server-10.3              67.3028 MB
linux-image-4.19.0-2-amd64      249.58   MB
linux-image-4.19.0-4-amd64      254.769  MB

...which seems to correlate with the ${Installed-Size} value shown above, so the dpkg-query should be sufficient here after all.

Remove old kernels

Fedora seems to clean up old kernel itself and never keeps more than 3 kernels around[9]. With Debian, one has to do this manually:

apt-get purge $(dpkg -l | awk '/linux-(headers|image)-[0-9]/ {print $2}' | grep -v `uname -r | cut -d- -f1,2`)

dselect

This needs to be reviewed. Is anybody still using "dselect"? I mean, it's still better than aptitude :-)

Some packages appear to be set to install ok not-installed, i.e. marked to be installed, but are not installed yet:

  $ dpkg --get-selections | awk '{print $1}' | sort > /tmp/list.dpkg 
  $ awk '/^Package/ {print $2}' /var/lib/dpkg/status | sort > /tmp/list.status
  $ diff -U0 /tmp/list.status /tmp/list.dpkg  | fgrep -v @
  --- /tmp/list.status  2011-02-25 13:12:04.143751694 -0800
  +++ /tmp/list.dpkg    2011-02-25 13:12:24.254753585 -0800
  -command-not-found
  -command-not-found-data
  -geoip-database
  -gnupg-curl
  -irqbalance
  -libcap-ng0
  -libparted0
  -libxml-sax-expat-perl
  -ntpdate
  -plymouth-theme-ubuntu-text
  -ppp
  -pppconfig
  -pppoeconf
  -ubuntu-minimal
  -ufw

Once dselect is run, /var/lib/dpkg/status gets updated with all the available package information. Now we have:

 $ grep ^Status /var/lib/dpkg/status | sort | uniq -c 
  1060 Status: install ok installed
 28631 Status: purge ok not-installed
 
 $ dpkg --get-selections | wc -l
 1060

A bit more on this, not sure where we're going here:

 $ grep Section /var/lib/dpkg/status | grep / | sed 's/^Section: //;s/\/.*//' | sort | uniq -c
     1 restricted
    38 universe
 $ grep -B3 universe/ /var/lib/dpkg/status | grep ^Package | sort
 Package: chromium-browser
 Package: chromium-browser-inspector
 Package: chromium-codecs-ffmpeg
 [...]

E: trying to overwrite shared package

Sometimes this happens:

$ apt-get -f install
[...]
Unpacking libaudio2:amd64 (1.9.4-1+b1) ...
dpkg: error processing archive /var/cache/apt/archives/libaudio2_1.9.4-1+b1_amd64.deb (--unpack):
  trying to overwrite shared '/usr/share/doc/libaudio2/changelog.Debian.gz', which is different \
  from other instances of package libaudio2:amd64
Errors were encountered while processing:
  /var/cache/apt/archives/libaudio2_1.9.4-1+b1_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

This can be solved[10] by forcing the installation with:

dpkg -i --force-overwrite /var/cache/apt/archives/libaudio2_1.9.4-1+b1_amd64.deb
apt-get -f install

List reverse dependencies

During apt-get upgrade, a package is updated and we'd like to know why this package is installed - i.e. what packages depend on this package:

$ apt-cache rdepends libpq5 | xargs dpkg -s 2>/dev/null | grep ^Package
Package: libpq5
Package: nagios-plugins-standard
Package: nagios-plugins-standard
Package: proftpd-mod-pgsql

APT pinning

For pinning to work, 3 files need to be modified:

/etc/apt/sources.list

Let's say we want to install testing packages on a Debian stable installation:

$ grep ^deb /etc/apt/sources.list
deb http://ftp.us.debian.org/debian/    squeeze         main contrib non-free
deb http://ftp.us.debian.org/debian/    squeeze-updates main contrib non-free
deb http://security.debian.org/         squeeze/updates main contrib non-free
deb http://ftp.us.debian.org/debian     testing         main

apt.conf

Specify our default release:

$ grep Default /etc/apt/apt.conf.d/99local 
APT::Default-Release    "stable";

apt_preferences

The actual pinning takes place in /etc/apt/preferences or a configuration file fragment in /etc/apt/preferences.d:

$ cat /etc/apt/preferences.d/10pinning.pref 
Package: *
Pin: release a=stable
Pin-Priority: 900

Package: *
Pin: release o=Debian
Pin-Priority: -10

Now we can install packages from testing but apt-get upgrade won't upgrade any other packages:

$ apt-get upgrade 
[...]
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

$ apt-cache show cowsay | grep ^Version
Version: 3.03+dfsg1-2                    ← stable
Version: 3.03+dfsg1-3                    ← testing

$ apt-get install cowsay/testing
[...]
Selected version '3.03+dfsg1-3' (Debian:testing [all]) for 'cowsay'

$ dpkg -s cowsay | grep ^Version
Version: 3.03+dfsg1-3

Note that pinning one package does mean that it can be installed right away, as it often depends on other packages from the pinned repository as well:

$ cat /etc/apt/preferences.d/90rtorrent.pref
Package: rtorrent
Pin: release a=testing
Pin-Priority: 990

$ apt-get install rtorrent/testing
[...]
Selected version '0.8.9-2' (Debian:testing [powerpc]) for 'rtorrent'
The following packages have unmet dependencies:
rtorrent : Depends: libstdc++6 (>= 4.6) but 4.4.5-8 is to be installed
           Depends: libtinfo5 but it is not going to be installed
           Depends: libtorrent14 but it is not going to be installed
           Depends: libxmlrpc-core-c3 but it is not installable
E: Broken packages

Installing via -t may give better results:

$ apt-get -t testing install rtorrent
[...]
The following packages will be upgraded:
  cpp-4.4 (4.4.5-8 => 4.4.6-14)
  g++-4.4 (4.4.5-8 => 4.4.6-14)
  gcc-4.4 (4.4.5-8 => 4.4.6-14)
  gcc-4.4-base (4.4.5-8 => 4.4.6-14)
  libc-bin (2.11.3-3 => 2.13-27)
  libc-dev-bin (2.11.3-3 => 2.13-27)
  libc6 (2.11.3-3 => 2.13-27)
  libc6-dbg (2.11.3-3 => 2.13-27)
  libc6-dev (2.11.3-3 => 2.13-27)
  libgcc1 (4.4.5-8 => 4.6.3-1)
  libgfortran3 (4.4.5-8 => 4.6.3-1)
  libgomp1 (4.4.5-8 => 4.6.3-1)
  libmpfr4 (3.0.0-2 => 3.1.0-3)
  libstdc++6 (4.4.5-8 => 4.6.3-1)
  libstdc++6-4.4-dev (4.4.5-8 => 4.4.6-14)
  locales (2.11.3-3 => 2.13-27)
  make (3.81-8 => 3.81-8.1)
  pkg-config (0.25-1.1 => 0.26-1)
  rtorrent (0.8.6-1 => 0.8.9-2)

...at which point we might cancel the request and install from source :-\

update-alternatives

update-alternatives can be used to manage different versions of the same software. As an example, consider this:

$ dpkg -l | awk '/ gcc/ {print $2" "$3}'
gcc 4:4.9.2-3
gcc-4.9 4.9.2-15
gcc-4.9-base:amd64 4.9.2-15
gcc-5 5.1.1-2
gcc-5-base:amd64 5.1.1-2

Let's make GCC-5 the default. For some reason, gcc is not managed by update-alternatives yet:

$ update-alternatives --config gcc
update-alternatives: error: no alternatives for gcc

For completness' sake, let's remove all related existing alternatives:

for p in gcc cc g++ c++ cpp gcov; do update-alternatives --verbose --remove-all "$p"; done

Now we can install alternatives:

V=4.9
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-$V 20 --slave /usr/bin/g++ g++ /usr/bin/g++-$V --slave /usr/bin/cpp cpp /usr/bin/cpp-$V --slave /usr/bin/gcov gcov /usr/bin/gcov-$V --slave /usr/bin/cc cc /usr/bin/gcc-$V --slave /usr/bin/c++ c++ /usr/bin/g++-$V

V=5
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-$V 10 --slave /usr/bin/g++ g++ /usr/bin/g++-$V --slave /usr/bin/cpp cpp /usr/bin/cpp-$V --slave /usr/bin/gcov gcov /usr/bin/gcov-$V --slave /usr/bin/cc cc /usr/bin/gcc-$V --slave /usr/bin/c++ c++ /usr/bin/g++-$V

Let's look at what happened:

$ ls -lgotr /etc/alternatives/ | tail -5
lrwxrwxrwx 1  15 May  1 00:23 gcov -> /usr/bin/gcov-5
lrwxrwxrwx 1  14 May  1 00:23 gcc -> /usr/bin/gcc-5
lrwxrwxrwx 1  14 May  1 00:23 g++ -> /usr/bin/g++-5
lrwxrwxrwx 1  14 May  1 00:23 cpp -> /usr/bin/cpp-5
lrwxrwxrwx 1  14 May  1 00:23 cc -> /usr/bin/gcc-5

And also, in /usr/bin:

$ ls -lgotr /usr/bin | tail -5
lrwxrwxrwx 1      22 May  1 00:23 gcov -> /etc/alternatives/gcov
lrwxrwxrwx 1      21 May  1 00:23 gcc -> /etc/alternatives/gcc
lrwxrwxrwx 1      21 May  1 00:23 g++ -> /etc/alternatives/g++
lrwxrwxrwx 1      21 May  1 00:23 cpp -> /etc/alternatives/cpp
lrwxrwxrwx 1      20 May  1 00:23 cc -> /etc/alternatives/cc

We can also switch versions now:

$ update-alternatives --config gcc
There are 2 choices for the alternative gcc (providing /usr/bin/gcc).

  Selection    Path              Priority   Status
------------------------------------------------------------
  0            /usr/bin/gcc-4.9   20        auto mode
  1            /usr/bin/gcc-4.9   20        manual mode
* 2            /usr/bin/gcc-5     10        manual mode

Press enter to keep the current choice[*], or type selection number: 0
update-alternatives: using /usr/bin/gcc-4.9 to provide /usr/bin/gcc (gcc) in auto mode

$ gcc --version
gcc (Debian 4.9.2-15) 4.9.2
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


APT-Cacher

Apt-Cacher NG is a caching proxy for Debian and Ubuntu packages. Installation and configuration is pretty straightforward:

sudo apt-get install apt-cacher-ng

During setup, a working configuration is generated:

$ cat /etc/apt-cacher-ng/acng.conf 
CacheDir: /var/cache/apt-cacher-ng
LogDir: /var/log/apt-cacher-ng
BindAddress: 192.168.0.106
Port:3142
CacheDir: /var/cache/apt-cacher-ng
[...]

# Here's where the magic happens...
Remap-debrep: file:deb_mirror*.gz /debian ; file:backends_debian # Debian Archives
Remap-uburep: file:ubuntu_mirrors /ubuntu ; file:backends_ubuntu # Ubuntu Archives
Remap-debvol: file:debvol_mirror*.gz /debian-volatile ; file:backends_debvol             # Debian Volatile Archives
Remap-cygwin: file:cygwin_mirrors /cygwin # ; file:backends_cygwin                       # Incomplete
Remap-sfnet:  file:sfnet_mirrors # ; file:backends_sfnet                                 # Incomplete
Remap-alxrep: file:archlx_mirrors /archlinux # ; file:backend_archlx                     # Arch Linux
Remap-fedora: file:fedora_mirrors                                                        # Fedora Linux
Remap-epel:   file:epel_mirrors                                                          # Fedora EPEL
Remap-slrep:  file:sl_mirrors                                                            # Scientific Linux
Remap-gentoo: file:gentoo_mirrors.gz /gentoo ; file:backends_gentoo                      # Gentoo Archives

ReportPage:   acng-report.html
ExTreshold:   120                                                                        # Increased from 4
LocalDirs:    acng-doc /usr/share/doc/apt-cacher-ng

With that in place, apt-get can be pointed to use it as a proxy and apt-cacher-ng will happily do so.

Failover Proxy

As the APT proxy will only be available in our local network, let's fail gently[11] when it's not reachable. We'll use Acquire::http::ProxyAutoDetect[12] for that purpose:

$ grep Detect /etc/apt/apt.conf.d/99local
Acquire::http::ProxyAutoDetect "/usr/local/bin/apt-proxy-detect";

And the apt-proxy-detect helper script should look something like this:

[...]
PROXIES="192.168.0.10:3142  10.0.0.2:8000"

for proxy in $PROXIES; do
       if nc -w1 -z ${proxy/:/ }; then
               echo http://$proxy
               exit
       fi
done
echo "DIRECT"

With that, a proxy is selected only when reachable, otherwise a DIRECT connection will be used.

Hints

Updates

While "apt-get upgrade" upgrades all packages, sometimes one only wants to upgrade some packages[13]:

apt-get install --only-upgrade PACKAGENAME

apt-mark

Show manually installed packages:

apt-mark showmanual

Show automatically installed packages:

apt-mark showauto

Install only security updates:[14]

grep security /etc/apt/sources.list > /etc/apt/sources.list.security
apt-get -V upgrade -o Dir::Etc::SourceList=/etc/apt/sources.list.security

apt

Sometimes apt-get may not have all the needed functionality[15] and we can just use apt to do the job:

$ apt moo 
                 (__) 
                 (oo) 
           /------\/ 
          / |    ||   
         *  /\---/\ 
            ~~   ~~   
..."Have you mooed today?"...

Links

References