Tiernans Comms Closet

I have decided to do a post on some of my favourite tools to use for development, administration, etc. It’s kind of like Hanselman’s Ultimate Tools list, but not as popular and about 2 years newer… Anyway, the list is available here, and will be updated over time, much like my Daily Carry and Computers pages. If you are interested, you add links though GitHub by editing the toolslist.yml data file.

[NOTE] This part 5 in a series of posts. The rest can be found here.

This post is going to be an update and theoretical post. probably very little “new” stuff going on here, mostly updates, and what I am planning on doing later on.

This week, I have been OOF sick, so I have not done much work, but I have been surfing the web, watching videos, downloading stuff, etc., so I have an idea of how things are going. First, as mentioned in the previous post I have MPTCP, Squid, Socks Servers, OpenVPN and IPTables doing their magic. 2 OpenVPN tunnels between the house and Digital Ocean. All TCP Traffic (bar port 80) is sent over socks to the box in the cloud using RedSocks. All UDP traffic is sent direct over OpenVPN. Since MPTCP is in the mix, all socks traffic is actually split over the 2 connections. All port 80 traffic, and 443 (if the client is using local Squid as their proxy) is sent round-robin between the 2 upstream IPs to Squid (2 OpenVPN end points).

[NOTE] This part 4 in a series of posts. The rest can be found here.

So, this week I went in a completely different direction that I have been thinking recently.

So, the basic theory is as follows:

• I am still using MPTCP kernels on both upstream and local machine
• now have 2 P2P UDP OpenVPN tunnels between house and cloud. Example config is here
• all TCP traffic (bar port 80) that hits the router in house is redirected to RedSocks
• RedSocks uses a socks server, Dante, as an upstream server on the cloud box
• since the socks traffic is over TCP (inside the UDP OpenVPN tunnel) it uses MPTCP
• having socks running, gives me quite the download speed, turning it off does not, hence the following tweet

• I am also noticing that I am starting to hit the limits of my upstream VM. If downloading or uploading at speed, the processor cores (2 in the case of the box I am currently running) are pegged at pretty much 100% full. Well, 80ish, but that because the other 20% is being used by Dante. I am noticing I can hit a full 72Mbit/s up, but the max currently downloading is about 400, maybe 450. Need a faster box now.
• I mentioned port 80 not being set over socks. That’s because its redirected to Squid. Squid (in house) then uses Squid (in cloud) as a parent. There are 2 round-robin parents for squid, one on each OpenVPN connection IP address.
• all other traffic (UDP, ICMP, etc.) are sent over the OpenVPN connection. currently only one is picked, but I have a cunning plan.

The cunning plan? Well, if I am reading the internet correctly, and I would like to think I am, I think ECMP, or Equal Cost Multi-Path Routing, could help. Again, it’s a fledgling idea currently, and I am still reading the documentation, but if it works. Well. I not sure. let’s see.

At the 2016 Build conference, Microsoft announced that Bash on Ubuntu on Windows was coming. Well, it came out last week, and I installed it as soon as I could! My next challenge was to get Jekyll to run and install on it, so I can build and preview this site on my Surface Book.

So, first, I needed to install version 2.0 of Ruby. There is a bit of messing involved for this, but first

[NOTE] This part 3 in a series of posts. The rest can be found here.

In Part 1 of this series I explained the why and what I wanted to do for this “project”. In Part 2 I did some basic testing of both MPTCP and MLVPN. I also mentioned trying MMPPP using vtund but it has been a while since I did that testing, and it had not been on bare metal. So, this post is a follow up, where I am using bare metal.

Microsoft Build 2016 is on this week, and there were a lot of interesting developments yesterday, but the one that interested me the most is Bash on Ubuntu on Windows. Dustin from Ubuntu has a more details, and Scott Hanselman has posted a technical video about this. This is very interesting, and I CANT WAIT TO GET MY HANDS ON IT! But, I do have some questions, which I thought I would put down in blog format:

[NOTE] This part 2 in a series of posts. The rest can be found here.

In my previous post I explained what i was trying to do… This post explains what i have been working on recently, and performance results…
So, first, what have i tried… There are 3 different things i have tried, and here are some of their details. Some will need to be updated (other parts of this series), and others i will try get back to eventually.

Back in August of 2012, I started this site using Git and Jekyll. I hosted most of it at home, pushing to a server in house. Then, a few years back, I moved to pushing the files to Amazon S3 and had Cloud Front doing distribution. The last moved had me hosting the files in NearlyFreeSpeech.NET and Cloud Flare doing the content distribution… Well, that changed over the last few days… again…

Currently, you are still hitting Cloud Flare when you hit this site, but the backend is back to being hosted on Amazon S3. But the files getting to S3 is more interesting now. All the “code” for this site is up on a GitHub repo and any time something is checked in, Travis CI kicks off, builds the files using Jekyll and pushes to S3 using s3_website. All my “private” keys are hidden in Travis-CI, so no one can access them but me. This makes updating the site a lot easier. I can create a file in GitHub directly, preview it, make changes, etc., and then check in. Once checked in, Travis kicks off, builds and deploys. All Good!

[NOTE] This part 1 in a series of posts. The rest can be found here.

First, a bit of background, and then I will explain what I am currently running in Part 2…

For the last 15 or so years, I have had at least 2 internet connections in to the house… 2 of them have always been Cable Modems from NTL, which became UPC, and now is Virgin Media. When I started, i think the modems where 150/50kbit/s and 600/150kb/s, and have steadily increased in speed, currently at 360/36Mbit/s each… But they have always been somewhat separate, and single thread downloads have always been limited to 1 of the connections… I have been looking for ways around this for years…

I have 2 new side projects to announce on the site today. First has been running for a while (first check-in was December 28th) and it’s called B2Uploader. Its a fairly simple Windows application to upload files to BackBlaze B2. If you are not familiar with BackBlaze, they provide unlimited backup storage for the low price of a fiver a month. They are the guys who design the BackBlaze storage pods (I want one, by the way!) that allow them to provide unlimited storage for the fiver a month (I currently backup over 4Tb to them!), and late last year, they started offing B2 which is a storage platform on their pods, and it has a (somewhat) easy to use API. AND ITS CHEAP! half a cent, up 0.5c, per gig stored per month! That’s crazy cheap!