SystemAdministration/TheRiseAndFallOfNewNodeManagement

For those who opened this sacred tomb, take a moment to decide if you wish to truly proceed. There are better things to do with your life like walk around the world, or learn to play pinball with your feet. If you truly wish to proceed, remember that their be dragons here.

Initial Setup

(this guide assumes we're using Ubuntu 12.04 and are on Linode, most of this is still relevent in general, but ignore the linode bits)

Once a new node is created on Linode, you need to deploy Ubuntu 12.04, this can "Dashboard" tab easily. Make sure you give 512M of swap, power it up, write down the root password, then open a console. We've got work to do.

Install All Updates

Linode's image is a bit out of date, so a quick upgrade is needed, first you need to update the package index

Last login: Sat Mar 22 22:02:47 2014
root@localhost:~# apt-get update
Get:1 http://mirrors.linode.com precise Release.gpg [198 B]
Get:2 http://mirrors.linode.com precise-updates Release.gpg [198 B]
Get:3 http://mirrors.linode.com precise-backports Release.gpg [198 B]
Get:4 http://mirrors.linode.com precise-security Release.gpg [198 B]
-SNIP-

Then install updates

root@localhost:~# apt-get dist-upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages will be upgraded:
  accountsservice apport apt apt-transport-https apt-utils apt-xapian-index base-files bash-completion bc bind9-host curl dbus dmsetup dnsutils dosfstools
  dpkg file gnupg gpgv grub-common ifupdown initramfs-tools initramfs-tools-bin iproute isc-dhcp-client isc-dhcp-common landscape-common language-pack-en
  language-pack-en-base language-selector-common libaccountsservice0 libapt-inst1.4 libapt-pkg4.12 libasn1-8-heimdal libbind9-80 libc-bin libc6 libcurl3
  libcurl3-gnutls libdbus-1-3 libdevmapper1.02.1 libdns81 libdrm-intel1 libdrm-nouveau1a libdrm-radeon1 libdrm2 libgcrypt11 libglib2.0-0 libgnutls26
  libgssapi3-heimdal libhcrypto4-heimdal libheimbase1-heimdal libheimntlm0-heimdal libhx509-5-heimdal libisc83 libisccc80 libisccfg82 libkrb5-26-heimdal
  libldap-2.4-2 liblockfile-bin liblockfile1 liblwres80 libmagic1 libpci3 libplymouth2 libpolkit-gobject-1-0 libpython2.7 libroken18-heimdal libssl1.0.0
  libudev0 libwind0-heimdal libxcb1 libxml2 lsb-base lsb-release multiarch-support openssl pciutils perl perl-base perl-modules plymouth
  plymouth-theme-ubuntu-text procps python python-apport python-apt python-apt-common python-httplib2 python-lazr.restfulclient python-minimal
  python-openssl python-problem-report python2.7 python2.7-minimal rsyslog sudo tzdata udev unzip update-manager-core w3m xkb-data
103 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 44.5 MB of archives.
After this operation, 19.5 kB of additional disk space will be used.
Do you want to continue [Y/n]? 

This takes about 5-10 minutes. Drink a soda, and compliate life ...

Set Hostname

Hostnames should be setup with the next item on the [HostnamePolicy|Hostname Policy]. On Ubuntu, you need to edit /etc/hostname, and /etc/hosts

root@localhost:~# cat /etc/hostname 
boron
root@localhost:~# cat /etc/hosts
127.0.0.1	localhost
127.0.1.1	boron

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

Load the new hostname with 'hostname -F'

root@localhost:~# hostname -F /etc/hostname 
root@localhost:~# 

Note, prompt won't change until to log out/log back in.

Switch Over To Distro Kernels

Linode uses a customized kernel instead of stock Ubuntu kernels. While this works "well enough" for most people, it lacks AppArmor, and cause unexpected splats as it doesn't have a ramdisk.

Here's Linode's guide on how to fix it: https://library.linode.com/custom-instances/pv-grub-howto

When you're done, uname -a should say something like this

root@boron:~# uname -a
Linux boron 3.2.0-60-virtual #91-Ubuntu SMP Wed Feb 19 04:13:28 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

Setup Networking

On the Linode panel, make sure the node has an internal IP address so that other nodes in the data centre can access it, then note it. You have to setup static IP address configure. Linode has a decent guide for this, but the quick and dirty version is you need to edit /etc/network/interfaces to look like this

# The loopback interface
auto lo
iface lo inet loopback

# Configuration for eth0 and aliases

# This line ensures that the interface will be brought up during boot.
auto eth0 eth0:0 eth0:1

# eth0 - This is the main IP address that will be used for most outbound connections.
# The address, netmask and gateway are all necessary.
iface eth0 inet static
 address PUBLIC-IP-HERE
 netmask 255.255.255.0
 gateway   GATEWAY-HERE
# eth0:0
# This is a second public IP address.
iface eth0:0 inet static
 address INTERNAL-IP-HERE
 netmask 255.255.128.0

You can apply the new IP configuration with this

root@boron:~# ifdown eth0 && ifup eth0 eth0:0
resolvconf: Error: /etc/resolv.conf isn't a symlink, not doing anything.
resolvconf: Error: /etc/resolv.conf isn't a symlink, not doing anything.
ssh stop/waiting
ssh start/running, process 1087
resolvconf: Error: /etc/resolv.conf isn't a symlink, not doing anything.
ssh stop/waiting
ssh start/running, process 1127
root@boron:~#  

FIXME: RESOLVER