The Bottom of the Stack

As with all the best things in life, this all started with a crazy idea that was initially dismissed out of hand. While I was in a fervor planning for FOSDEM this year I did a quick check to see if there were any conferences in the UK that I might be able to go to (my student budget didn't allow for another international trip). I worked with PHP for two of my previous internships, and used it as the base for my most used personal projects (pizza-get and music-get), so seeing that PHP had a conference in the UK was exciting. But there was a problem. A problem to the tune of £300 in the form of an entry fee.

Eventually I found that PHP UK were offering diversity scholarships for the conference, to the tune of travel, accommodation, and the entry fee. That sounds great, but I was pretty certain that I wasn't going to get one, especially as a student when the conference is aimed at professional developers. Well, turns out that if you don't ask you don't get.

Fast forward to February, and I'm on the way to a campaign reception at the Chiswell Street dining rooms with the other scholarship awardees and conference presenters - not only was I at the conference, but I had a unique opportunity to network with some of the best minds of the PHP world! After an insightful evening of wine, food, and excellent advice it was time to retire to the apartment I was staying in for the duration of the event with my roommate (another student awarded a scholarship).

Some of the main track conference talks blew me away. Compared to what I had been to before - FOSDEM and an 'unplugged' conference by RSAC - this was a big budget event. This also meant that there were some talks which one might not expect at a tech conference. There were talks about diversity in the industry, impostor syndrome, and mentoring. These are all really important subjects, and ones which nigh impossible to pick up from a book. I think more conferences should have these types of 'soft' talks (In fact, thinking back to RSAC unplugged the best session by far was Alexis Conran of the Real Hustle describing the psychology of scamming and deception, and not the ones by various heads of cyber security/research).

Aside from being harder to learn, these topics are the ones which get left out (in academia anyway, it was better when I was at a startup) in favour of technical skills. The thing is, I know how to find out about tech stuff - I can use Google to help me figure out that I can use Docker to ship my app - but I'm far less likely to google why it is that all my colleagues seem to know more than me and how on earth I got through the hiring process without them seeing through my fraud. Wait, did I just say that out loud?

To wrap up, I'm incredibly grateful to PHP UK for giving me this opportunity. I didn't think it would be possible to make people feel included in a community of 700 where, to be honest, we had very little in common. But it's worked! One of the conditions on receiving the scholarship was that I pay it forward one day, and you know what? I will. After the conference it dawned on me that one day I'll be able to provide the same eye opening experience to someone, and that's incredible. And also thanks to my mentors this time around, so I guess it's self perpetuating.

(I mentioned RSAC Unplugged a bit here, so I had to include the awesome picture they made of all the conference talks)

It's Not Rocket Science

This week SpaceX landed an rocket on a barge, and filmed it with a drone. The next day it occurred to me: my friend has a rocket, and I have a drone. If it's good enough for Elon Musk, it's good enough for me! Keen not to anger campus security any more than we had to (after recently crashing said drone onto the roof of the Zeeman building), Hersall Common was chosen as the launch site instead of one of the fields at the university.

The first obstacle encountered was the weather, with one launch aborted due to high winds and rain. The second came in the form of two important life lessons:

1) Super Glue does not fuck around

2) Do not volunteer to hold rocket parts while someone else Super Glues them

Luckily the abandoned launch gave me some time to reflect on poor life choices and a sudden lack of skin on my fingertips.

The next day brought fairer weather, so we headed back out to the common with our now fully assembled rocket. Cue us realising that we hadn't attached the nose cone and parachute to the body tube and another half hour of shenanigans with superglue. Finally, we could launch. I didn't realise until this point just how much having an electric ignition system adds to the whole experience. It really felt like we were playing with something dangerous.

There was quite a lot of wind on the common which meant we had to angle the guide rail at about 15° in order to have the rocket go something resembling upwards. It also meant that the drone was grounded. Craft #1 was small and light, weighing in at 42.5g with an A8-3 motor weighing 16.2g providing an average of 8 Newtons of thrust (9.7 peak). That's 0-60 mph in 0.2 seconds (not counting air resistance).

Rocket #2 was a bit heavier (62.4g), with the same motor. Evidence suggests that we need something bigger to get a decent launch from it. I think that's an excuse to go out and buy some more explosives.

And finally, here are the videos from the launches!

A Game of Drones

There is one thing that will, without doubt, induce fear in the heart of any undergraduate computer scientist: group project. The problem is that they come with all the responsibility of paid work, but none of the accountability. Maybe I'll revisit the game theory of Warwick group projects in a later post.

The topic we chose for ours was one I'd been wanting an excuse to explore for a while: quadcopters. After some brainstorming we decided that the best approach we could take given that this was an academic project was to devise some framework for reasoning about drone networks, and to devise a way of writing high level scripts that could task them. I think the particular use case we had as an example was monitoring forest fires.

The problem is though, that the FOSS software available for simulating sensor networks is *really bad* from a beginners perspective. The de facto network simulation application is NS3 (yes, as imaginatively named as you think), but the learning curve is really high. The hands on section of the sensor networks module at Warwick for *third years* spends weeks building up to the point of moving some nodes around. Suddenly it began to dawn on us that we had bitten off rather more than we anticipated.

Upholding a long tradition of software engineers everywhere, we took a fresh look at the problem and decided that it would be easier to create our own simulation software than to learn the intricacies and quirks of NS3. To be honest, it's done wonders for my understanding of the domain, as well as the specific problems we are trying to model. The best example of this probably lies in the communications side of things. If we had used NS3, I would have experimented with a few different message routing models before trying to piece together a conclusion based on what the corresponding Wikipedia articles told me was supposed to be going on. As it stands, I've actually taken the RFC for Ad hoc on demand distance vector routing and implemented it in C++.

Credit: David Richardson

We made a conscious effort with the project (named octoDrone) to keep things as modular as possible. To this end, the simulator, communications processing, programs that run on drones, and scripts that define simulations are all separate and configurable. This payed dividends when it came to deploying our software on to real drones instead of simulated ones, as we were able to swap out the code that simulated the operating environment for one that sent the same instructions to real hardware (I'll cover the deployment in a separate post as it's something I could talk a lot about). The other goal during development was to cut out a lot of the bloat that plagued NS3. Instead of it taking half an hour and a handful of tutorials to visualise a simulation, the application *should just do that for you*. Instead of having to access node components by passing strings with the same name (eugh), you should just be able to access them through code the way you normally would.

At the end of a year of work we're left with a lightweight simulation package that does exactly what one would want a drone simulator to do, and nothing more. Being able to run the same programs on real drones as you do in a simulation is a real selling point (and that it works at all on the drones we had to test on is a point of personal pride). It even got to the point where I seriously considered running my coursework assignment in octoDrone instead of NS3, which serves to show just how far we've come.

GitHub: https://github.com/mcnutty26/octoDrone

What's all the FOSS About?

This weekend I joined some friends from the computing society on a trip to the Free and Open-source Software Developers' European Meeting (FOSDEM), an opportunity to learn from and share opinions with ~5,000 other developers from the continent. Having such a broad topic means that FOSDEM is a great place to learn about all sorts of incidental things you wouldn't expect to. I learnt about window managers, DNSSEC, and ?. These weren't things that I went with the express intention of learning about, but it helps make me a more rounded engineer than what my course and personal projects allow.

It's nice to be around others who share the same niche interests as you do. I went up to the stall of one of the projects that I use (ownCloud), expecting to say hello and exchange pleasantries, and instead ending up discussing the project and it's intricacies with one of the developers. It takes a bit of courage, but this is the best opportunity you can get to learn from the best of the best. I finally got to see what all the fuss about this Stallman bloke was about, and learnt more than I thought would about software licenses (from the perspective of a developer and a community).

The other astounding thing about FOSDEM is the sheer amount of beer that gets consumed. The convention takes over the delirium cafe in the centre of Brussels on the Friday before the event starts (complete with bouncers that ask you about the JVM before they let you in), which is packed to the point where you can't get into the street it's in. After everyone's woken up with sore heads the next day, extra shops open up around the ULB campus to serve the mas of developers who are still hungry for booze. A few of the stalls bring free (as in beer) beer, and will happily swig away while talking to you.

This time we decided to write off Monday and get a later train back (the idea that you're going to do anything productive on the day you get back to the UK is a naive one), which let us visit a few sites in Brussels. We didn't do this last year, and it was a really nice way to spend the day. We saw the cathedral, complete with the fanciest pulpit you ever did see. On the way out we spotted the Banque Nationale De Belgique, which turned out to have a temporary exhibit open. There's something about money which makes it intriguing, and the ability to print banknotes with your face on aside, it was an insightful look in to the way the euro is managed, and the role the member states play (and what's left to the central bank. My favourite was the machine used to invalidate old coins, which corrugates them :P We also went around the European Parliament, which was filled with quite a lot of European propaganda, but served as a reminder of how incredible it is that the European Union exists and does all the things it does (especially when the press opinion of it is overwhelmingly negative).

A Nodd in the Right Direction

It's been a long time coming, but we're almost at the end of the ill-fated UWCS migration! After the target machine we were planning to move to developed power supply problems (as well as a worrying amount of bad configuration decisions), we finally gave up and started again. In hindsight, it was a godsend that we had a machine to do what would eventually become a test run on. It served as a stark reminder that we had underestimated the complexity of what we set out to do, and that we didn't know what we were doing. This lead to lots of quirks that weren't quite right, and meant that the communications we send out to users was patchy, confusing, and sometimes downright wrong.

This second attempt allowed us to select more reliable hardware (as opposed to, I shit you not, some server we got cheap second hand and untested) and move away from software raid, which turned out to be more trouble than it was worth. We now have a RAID 1 configuration in hardware, with an additional two hot swap disks. We went with an OS with long term support, and postponed the load balancing plans I mentioned in a previous post. That said, we've set things up in a way that means we can transition to a load balanced setup in the future.

Configuring things for the second time was a lot smoother, and now I've learnt more than I probably ever need to about managing IRC, Apache, and MySQL. Our interactions with the user base were a lot more professional, and it feels like we've actually helped people to navigate the switch over, instead of just letting them know and hoping for the best. We're currently in the grace period before turning off the old server for good, which gives us some time to check that everything is working properly (and fix the things that aren't without being in panic mode). Hopefully we've built the infrastructure that will carry us through the next decade, as the last server has taken us through the previous ten.

Music to My Ears

The first step is admitting that you have a problem.

We have a music server at LAN parties, which acts as the gateway for all of the connected PCs and a place where people can queue music and videos. Anyone at LAN can submit media, and everyone at LAN has to suffer through our collective bad taste in music. Following on from the theme of the last year or so, the current music server software is dated and slowly falling apart, and therefore  it must be time to make a new one. There are currently 11 implementations either developed or in development (see list).

This is a textbook case of a time where the usual approach of "I think I can do better, therefore I should" has clearly failed. There's a horrendous amount of effort being duplicated, each with a slightly different way of doing things, or a different philosophy. Of course, some people are writing new music servers as a way of learning a language, and that's fine. Others are writing one because everyone else is, which is more of a problem. In theory the best approach would be for everyone to contribute to a central project, as was done in the past, but in reality this is likely to lead to arguments and little progress (university deadlines have a habit of exacerbating with this).

Unfortunately I'm a guilty party here, because I wrote a music server *because* everyone else was trying (and failing) to come up with a finished project. Enter music-get, styled after pizza-get,  and designed to be easy to maintain as opposed to what might be considered a 'well engineered application'. To allow for this the front and back ends are split up, and there are extra dependencies (like Apache) that could be eliminated if you were trying to make a compact app. But in reality, we're running this on a fairly modern desktop, not a Raspberry Pi. Maybe there are better languages for this than Java, but everyone can use Java. In general modifying music-get doesn't assume much technical knowledge, as when you need to fix something at 4 in the morning during a LAN party you may not quite be on top form :P

This is the first project that I've come up with proper documentation before (as in, how it works instead of just how to run it), which was interesting at first but quickly became tedious. Hopefully it, or something better, will get used at the next LAN - ultimately the goal here is that the problem gets solved, not who solves it! Still, it would be nice if it were mine...

GitHub: https://github.com/mcnutty26/music-get

Balancing Act

There comes a time with most IT systems where you have to call it a day and upgrade. Reasons often include increased load and equipment failure, and in the case of UWCS we have both.

Our main server, codd (after Edgar F. Codd) has been going for a long time now. A lot of things have changed since it was built, including what users expect their hosting solutions to provide for them. Coupled with the improvements in hardware and software this means that there isn't any excuse for providing a good service. No matter how 'executive series' the motherboard in codd is, it doesn't make up for the fact that it's less powerful than my laptop. Sure, it still works, but we can do better.

As luck would have it, we were gifted some poweredge servers last year that pack a bit more of a punch. Along with some of the others that we've bought we ought to have enough metal for a decent setup. But a racks worth of servers wouldn't be much without somewhere to put them! Some grovelling to the department of computer science prevented us having what would have amounted to a very pricey purchase. It didn't fit the servers though, so we had to take it apart and bolt it back together in a more accommodating configuration.

With the servers racked and imaged came the task of actually figuring out the details of this grand plan. I mean, it would of if we hadn't spent six months procrastinating. Once the shiny new toys had come out of the packaging things slowed right down. Thankfully we've started this year with some fire in our collective belly and actually come up with a plan we might actually be able to implement! The design features a number of application servers behind a load balancer, with each app server containing a copy of user files and databases. It might be better to have dedicated storage/db clusters if we had more traffic, but the focus here is on something that holds up to the next ten years whilst being doable in between lectures and assignments. And being intelligible to the next tech officer who has to inherit all of this :P

Ceph is a distributed file system, and how we've chosen to store the files that users store in their hosting accounts. We'll need some way of letting users interact with the system whilst keeping the app servers in sync, so all SSH sessions will terminate at nodd, which will have the Ceph drive mounted. Any changes made will be propagated to their actual location on disk which will be on all of the cluster nodes. Currently there are two, but in the future we will be able to spin up some more and just add them in to the haproxy configuration. SSL connections are terminated at nodd, which then uses regular http to retrieve files across the internal network. This way we can add a cookie to http sessions ensuring that users are served by the same backend server for the entire session (important for managing things like logins).

It's been a long road, but I think we might just make it!