Mounting Shared Drive on Linux and Headless Bittorent Daemon

There are a number of different ways to automate the download of bittorrent files.  Many bittorrent clients have the ability to "watch" a directory and if a torrent files get added there then the bittorrent will automatically begin to download.  Some even don't need to download the file as it can subscribe to a feed directly.

GUI Way

What I did before was to log into a machine, connect to a shared drive and then open Transmission which was configured to look at the shared drive for any new torrent files.  I even wrote a program that grab checked and downloaded new torrent files every 24 hours.  Setting this up was extremely simple.  As part of my login items on OSX, I had it connected to my shared drive.  Then I would start Transmission and have it point to the shared drive to automatically download the torrents.  I used watch to execute my torrent downloads:

watch -n 72000 <program to download torrents>

I didn't use cron or anything because this was originally on a OSX laptop.

When I switched to a Linux machine that is kept on, I did the exact same thing.  The problem with this is that you're expected to be logged in to the system and keeping the programs running.  If someone else need to use the computer then everything stops since you'd have to log out.

Terminal Way

This similar to the GUI way described above except you do it in a terminal running in a tmux session so that you can disconnect without loosing your transmission job.  To connect to the shared smb drive without the GUI by using transmission-cli tool.

tmux
dbus-launch bash
gvfs-mount smb://<user>@<host>/<path-to-shared-drive>
sudo yum install transmission-cli
transmission-cli

You'd put in your user name and password as it prompts you when you mount the shared drive.  You'll need to configure transmission.

Headless Way

This doesn't require any GUI or terminal and can automatically starts at each boot.

Install the transmission-daemon

sudo yum install transmission-daemon
sudo systemctl start transmission-daemon.service
sudo systemctl stop transmission-daemon.service

The service file is in /usr/lib/systemd/system/transmission-daemon.service.

I started and stopped the daemon because you need to configure it through the /var/lib/transmission/.config/transmission-daemon/settings.json and this gets created when you first start (for other locations of Transmission files click here).  This will also install the transmission user

Next you need to mount the shared drive in order for transmission to access and ONLY transmission:

sudo mount -t cifs -o -credential=<path-to-credential>,gid=`id -g transmission`,uid=`id -u transmission` <mount point>

The credential value points to a file that contains your username, password and domain for logging into the shared drive.  Make sure wherever you store it that it's not readable by others.

The gid and uid belongs to the user you want to be able to do read-and-writes to the shared drive otherwise it'll only be root that can do it.  For Fedora, the user is "transmission"

If it works and you can read-and-write, you can move this into /etc/fstab:

//<host>/<path-to-file> <mount-point> cifs credentials=<path-to-cred>,uid={user id},gid={group id}

and then reload fstab:

sudo mount -a

Once you configure the /var/lib/transmission/.config/transmission-daemon/settings.json to your liking then fire up:

sudo systemctl start transmission-daemon.service

sudo systemctl enable transmission-daemon.service

This starts up the service now and also at each machine reboot.  The one issue is that transmission-daemon might start before the drive is mounted and then it won't look for the changes.  It needs to be told to wait.

Systemd will look at /etc/fstab and automatically create an unit that can be added to the service file to tell it to wait.  Running the systemctl command will show you an unit that can be used (it'll be something with a .mount suffix).

Add that unit to the After line:

After=network.target <mount unit>

Finally, set up a cron job to regularly check for new torrents:

crontab -u <transmission user> -e

With the value being something like:

0 23 * * * <path-to-program-to-run> >> /var/lib/transmission/download.log 2>&1

which says to execute the program every night at 11pm and store the out put in /var/lib/transmission/download.log.

Since the transmission users is disabled for an interactive shell, to execute commands as the transmission user, use:

sudo -u <transmission user> <command>

Thoughts on setting up Murmur and Mumble.

Following my experiences of compiling and setting up Synergy, I found myself setting up Murmur/Mumble this past weekend.  It was a sharp contrast in experience.  Mumble seems to want folks to be able to build and run their software.  Mumble is an open source VOIP solution used often by gamers to talk to each other while playing games and Murmur is the server portion.  I needed Windows and Linux clients since those are the system that people are on when playing and a Linux version of Murmur since that's what I'm going to be running the server on.

In most cases, setting this up on Linux is as simple as using whatever package manager is used by the distro and installing the Mumble package(s), but I ran into a problem with Fedora 21.  There are no packages for Mumble since it was retired after Fedora 20.  That means compiling from source which is normally not a big deal since this is open source, but to just make things a little more difficult some packages that Mumble depends on were also retired so those needed to be built and installed as well.

In the end, I cheated and grabbed the rpm from Fedora since they still had the RPMs from before they decided to retire it from the repo.  Mumble also requires the Ice package which was also retired so make sure to install it as well.

Now, the next step was getting a Windows client and this was when things got interesting.  The Mumble team provides pre-built binaries that are hosted on Sourceforge (development snapshot are available directly from their snapshop location).   I've been fairly distrustful of Sourceforge following what happened with Filezilla and malware and reluctant to use anything hosted there especially Windows binaries so the alternative was to try to compile the client myself.  I approached with some trepidation as there is a reason that on Windows people distribute binaries more then source code.

Compiling other people's code on Windows is not easy!  In the past, the tooling was not always available but that's gotten a little better when Microsoft decided to make free "Express" editions of Visual Studios available.  Still, the effort required to build something like Mumble wasn't trivial, but I have to give kudos to the Mumble guys who has done a very good job documenting how to build Mumble on Windows.  I went though their instructions and they document a lot of potential gotchas that helped get over issues that I ran into and while it took the entire morning (a lot of time was spent just waiting on the build), I was able to get a working client running that talked to the server running on the Linux box.

Misc Thoughts

I didn't know anything about Mumble before the weekend and came across it while I was looking for something to meet my needs.  Once I found Mumble and started looking at how to get the client, I also found mumble.com which looked and sounded like Mumble (it has the binaries, let's you buy the server software, etc.) but in small fine print at the bottom it says it isn't affiliated with Mumbo.info.

It's these kind of things that always bothered me and where a trademark would actually be very helpful.

Compiling Synergy on OSX 10.9 Maverick (and 10.10 Yosemite)

Synergy is an open source software KVM that allows you to use one keyboard/mouse across multiple computers.  Although Synergy is open source, hosted on Github and under GPL, its main developer(s) decided sometime last year to put the binary of the latest version behind a pay-wall.  There is nothing in GPL that prevents them to do so and I don't necessarily object to developers charging a fee for providing a service such as hosting for downloads (I used to buy Linux install CDs from Walnut Creek without a second thought), but I'm just not a fan on the way they're doing it.

The Synergy website looks and feel like one of those spam sites where they try to trick you into buying or downloading something with big call to action buttons while the actual link to download is kind of hidden.  For example, they have a big "Get Synergy" button which is the one that ask you to pay while links to github and a link for free binary downloads (although not of the most recent version since that's the one they want you to pay for) are tiny and hidden way a bottom of the page.  I guess it's better then those sites with a big button that says "Download Here!" (which actually takes you to some spam page) while next to it as a small font link is the actual download link.  Plus, paying for it doesn't seem to get you additional support or anything other then being able to download a pre-compiled binary.  Basically, their execution of it is the turn-off and I had to check a few times to make sure the site is legit and not a spam company trying to appear legit.

Since this is open source anyone can just download the source and build it themselves, right?  Well, not exactly.  The instructions for how to compile is not great, but worst is that the code itself doesn't build if you just get it from Github.  I'm trying to build on OSX since most Linux distro will have it already in a package to install, but Synergy doesn't have a Homebrew formula and Macports has not maintainer.  So, for OSX, you'll need to compile yourself.

The latest source doesn't work yet (I tried to build it and it errors out on compile), so I thought I get the last stable version and build from that.  Nope.  If you try to build strictly from the command line it fails trying to create the right build files because of problems with the toolchain it uses.  If you try to build with Xcode, you'll need to code sign it.  In order to build it, you'll need to patch the toolchain.  I found that someone in Korea fixed the toolchain to allow him to build Synergy for Yosimite, but his solution is actually also needed to get it to build on 10.9 Maverick

So here goes:

xcode-select --install
brew install cmake
brew install qt

Next you'll need to download the source from http://github.com/synergy/synergy.

When I grabbed the latest and tried their instructions for compiling, it didn't work.  So your choice is to pull from an earlier stable release.  That means either 1.6.2 or 1.4.x.  Note that the 1.6.x client can't speak to 1.4.x and even 1.4.18 client isn't compatible with 1.4.10 server.  Fedora 21, for example, comes with 1.4.10 so if you grabbed the latest 1.4.18 release it wont' work.

To build 1.6.2 on OSX Maveric, you'll need to change ext/toolchain/command1.py

./hm.sh conf -g1 --mac-sdk 10.9 --mac-identity Maveric
./hm.sh build

To build 1.4.18, this is the replacement ext/toolchain/command1.py

I realized after building 1.4.18 client that it won't work with 1.4.10 server, I decided to have every system on 1.6.2 even if I can't use a package manager for it.

Don't just have Synergy client talk directly to the Synergy server directly.  There's a chance that someone can be listening on your communication.  Instead, use ssh tunneling so that the communication is secure.  I also suggest using autossh so that it knows to reconnect if your client gets disconnected (e.g. laptop went to sleep).  You can install autossh through Homebrew.

brew install autossh

After installing Fedora, it's time to install Fedora.

Installing Fedora is easy.  Boot up the Live DVD and let 'er rip.  Once it installs itself on to your drive and reboots you'll likely find yourself in Gnome Shell.  Of course, you're not really done with setting up Fedora because it is now time to update and customize!

First thing you'll ways want to do is to run the update to make sure you get all the latest patches and security fixes.

sudo yum update

Note that to use sudo you'll have to mark the account you're using as an administrator account.

Because Fedora believes very much in being purely open source, you won't find things like VLC or Google Chrome so you'll have to add those yourself.  The simplest way is to add RPM Fusion which just requires you to run a one line command to grab it and install it on your system.

Install VLC

sudo yum install vlc

Install Google Chrome

To install Google Chrome, you'll first want to install the package signing key from the Google Linux Repository:

wget https://dl-ssl.google.com/linux/linux_signing_key.pub
sudo rpm --import linux_signing_key.pub

Then you can download Google Chrome RPM package and install it.  It will add the Chrome repository for yum:

sudo yum localinstall https://dl.google.com/linux/direct/google-chrome-stable_current_x86_64.rpm

Note: this is the 64-bit version of Chrome and not the 32-bit version.

Change Desktop Environment

To switch to another desktop environment such as XCFE, you can use yum as well:

sudo yum group install "Xcfe Desktop" --exclude fedora-release-*

Then at log-in select XCFE as your choice for desktop manager.

If you prefer Cinnamon, then it'd be:

sudo yum group install "Cinnamon Desktop" --exclude fedora-release-*

To see what other environments are available, simply type:

sudo yum group list

Thoughts Fedora 21

Even though my primary home desktop the past few years has been a Macbook Pro, my favorite Operating System is Linux.  It would be my primary OS if it wasn't for a few tasks that aren't as convenient on Linux such as photo management (which any father will know has to be good or Mother will bring down the hurt), so at home it's been Homebrew to fill the void.  Because of this I haven't kept up with all the changes with my favorite distribution, Fedora.

I have tried various distributions here and there, but Fedora has always felt the most comfortable to me.  A lot of it obviously due to familiarity as I've been using Fedora when it was first conceived as Fedora Core and Redhat Linux before it (and Slackware before that).  When the time came to put together a new machine for the family for things like Minecraft and playing videos I naturally was going to go with Linux and Fedora which is now at version 21.

In version 21, Fedora now offers 3 separate versions targeted at 3 different needs:  workstation (what most home and office users will use), server and cloud.  I suppose the idea is to make the installation process easier as each version is preconfigured with the software and settings that they feel meets the most likely needs of the target audience.  I'm not sure if I really agree with this approach.  I imagine that most people who are capable and willing to get and install their own OS will be configuring their environment and this just defers that to after the installation.   At the same time, the install process is very simple.  Just a few clicks and off it goes so if you don't mind claiming every bit of disk space maybe this is perfectly fine.

It's nice that the Live CD is used to start the install because it does make the disc more useful and provides a sanity check that everything works before installing to the hard drive.  What I do miss is at install time it doesn't give you the option of adding other repos (how many Fedora users don't add RPMFusion?).  Instead, you finish installing Fedora and then install RPMFusion and then add the packages from it such as VLC.

I admit that I was a bit caught off guard by the default desktop environment: Gnome Shell.  It is simplified and familiar especially if you come from OSX with the way the settings dialog is laid out and the application dock, but might be an oversimplification for developers.  I suppose if the workstation version is targeted at the enterprise office space that it makes sense from that perspective.  Whether or not that is the right direction, I'll leave that for the market to decide.  It is telling that if you search online that one of the first things people recommend after getting Fedora installed is to add the gnome-tweak-tool to customize Fedora.

Fortunately, Fedora makes it very simple to change to another of the other popular desktop environments such as Mate, Cinnamon, Xcfe, etc.

Fedora comes with Firefox and Yahoo as the default search engine.  I have no objections to Firefox or its choice of Yahoo, but the browser is so fundamental now to anyone using a computer that offering more choices here would be best for the user.  Given that Fedora is very much about open source software, I can understand if Chrome is not a choice but Chrominum can be.

These are all very minor complaints and Fedora is still awesome.   

Asus Vivo Mini

After reading about the Asus Vivo Mini UN62 from CES, I was very excited by the idea of a small (it fits into the palm of your hand) and nearly silent PC that had enough power to be an actual computer but with such low power usage that it can be kept running all the time if need be.

It's a barebones system which means it doesn't come with an operating system, memory, or hard drive.  It does come with the CPU, has integrated video card and ethernet.  Prices for it are $160 (Celeron), $280 (i3) and $370 (i5).  Adding your own memory ($60), disk ($100) and wifi ($20) means that a complete machine will run from $340 to $550.  This machine is also Linux compatible so unless you need to run Windows that is one area for savings.

Note:  Don't just go out and buy any SDD.  This is a small machine and what you need to add in are the small components most often used in notebooks such as SODIMMs (memory) and mini pcie (wifi).

I bought the i3 version since I just needed it to do two primary things:  Minecraft and play videos from our NAS.  So far it hasn't disappointed!  Putting everything in all the components took about 10 minutes and I was able to boot off a USB DVD drive that had Fedora 21.  Everything worked immediately including WiFi and it is super quiet.

If it weren't for the small light on the power button I couldn't even tell it that was on since there was no noise.  This isn't a fan less system so there are fans but even when pushed it was still very quiet.

It comes with HDMI and display port output so I just plugged into the TV, told Linux to play sound through HDMI and pretty soon the family was playing Minecraft with sound and video.  The performance is very good and despite how everything fits into such a little box the temperature never felt hot to the touch (although as a desktop machine its probably not going to be moved around much).  It comes with a VESA mount to attach it to the back of the monitor.

ASUS also provided all the screws needed for the drive, wifi card, mount, etc.

It's a sweet little machine.

Just one note, the ASUS page makes it sound like it also has built-in WIFI.  It doesn't.  It supports it if you add your own wifi card.

Specs:

• Intel Core i3-4030U (2 cores, 4 threads, 1.90GHz, 15W)
• Crucial 16GB (8GBx2) DDR3-1600 204-Pin SODIMM 
• Intel Network 7260.HMWG WiFi Wireless-AC 7260 H/T Dual Band 2x2 AC+Bluetooth HMC
• Samsung 840 EVO Series 120GB mSATA3 SSD
• 65W power supply

Enabling bitmap fonts on Linux

The patched Terminus Fonts for Powerline (renamed to Terminess) comes as bitmap fonts (pcf).  After installing those fonts, you might still not see them appear as an option when choosing fonts.  It could be because bitmap fonts support might be turned off on your system.

To enable:

rm /etc/fonts/conf.d/70-no-bitmaps.conf
sudo fc-cache -fv

or if you don't have admin/root access then

cp /etc/fonts/conf.avail/70-force-bitmaps.conf to ~/.fonts.conf.d
fc-cache -vf ~/.fonts

Powerline compatible Terminus Font for OSX

My preferred programming font (and what I set for my terminals whether it's on Linux or OSX) is Terminus.  While it's not on most Linux distribution by default, it's usually available through the distribution's package management system (yum, apt-get, etc.).

On OSX, it's a little more troublesome.  A search on Google will turn up three different versions in the first three results.  In the past, I've used the one here. However, if you use something like powerline or vim-airline it doesn't have all the symbols so for some things it will just use the characters like '>>' and '>'.

To have those nice symbols, you generally need a 'patched' version of the font that has those symbols (this just means they added the characters to the font collection).  You can find versions of many popular programming fonts here.

You'll notice that Terminus is part of the collection except that the format that is included isn't compatible with OSX.  Thus, I have always had to live with the characters above.  The latest version of Terminus TTF from here now includes the powerline symbols and so I was able to install it and now my status bar looks like all those other pretty ones out there!

OSX load order of your bashrc

If you run bash, OSX doesn't loading your .bashrc file.  The first looks for /etc/profile and then it looks in the following for the first available one:

1. ~/.bash_profile
2. ~/.bash_login
3. ~/.profile

If you keep your file in a .bashrc file then probably the best way is to source it from .bash_profile.

Building Protobuf on OSX

I wanted to play with protocol buffers on my local OSX machine but set up isn't so straight forward so I thought that I'd document the steps.

Step 1:  Get the build tools.

If you have all the build tools installed then skip ahead, but on the particular machine that I was using it didn't.   So I had to do the following:

Install Xcode command line tools

xcode-select --install

Install the GNU build tools (autoconf, automake, libtool):

curl -OL http://ftpmirror.gnu.org/autoconf/autoconf-latest.tar.gz

tar -xzf autoconf-latest.tar.gz 

cd autoconf-2.69./configure && make && sudo make install 

curl -OL http://ftpmirror.gnu.org/libtool/libtool-2.4.2.tar.gz

tar -xzf libtool-2.4.2.tar.gz 

cd libtool-2.4.2./configure && make && sudo make install 

curl -OL http://ftpmirror.gnu.org/automake/automake-1.14.tar.gz

tar -xzf automake-1.14.tar.gz

cd automake-1.14 

./configure && make && sudo make install

Alternatively you can use Homebrew or MacPorts to get the packages.

Step 2: Get the Protocol Buffer Compiler and Build It

git clone git://github.com/google/protobuf 

cd protobuf 

./autogen.sh

configure

make 

sudo make install

Since I wanted to use protocol buffer in Go, I have to install the Go plugin for the compiler:

go get -u github.com/golang/protobuf/proto

go get -u github.com/golang/protobuf/protoc-gen-go 

The system can now compile your proto files to let you use it in your code:

protoc --go_out=.