An update to "What's in my Podcast Roll?"

I blogged previously about the podcasts I regularly listen to, and the topic came up again this week on the SAGE-AU mailing list, so I thought I’d update it with my current thoughts.

Regular listens from the previous list

  • SANS Internet Storm Centre daily podcast [feed] – Still my “must listen” podcast.
  • Risky Business [feeds] – They have done a little cleaning up on the NSFW content, but they could still do with a little more.  The RB2 feed has been expanded to include “Serious Business”, a light-hearted look at general current affairs with Dan Ilic, an Australian comedian based in the U.S.
  • Packet Pushers [feeds] – Getting a bit too frequent for a full-length (60+ minutes) podcast, but still interesting; they’ve diversified content and now include “Network Break”, a shorter, business-/news-focused show, and “Datanauts”, a “silo-busting” show on data centre topics in general.  One disappointment is that Michele “Mrs. Y” Churbirka’s “Healthy Paranoia” is inactive at the moment.
  • Linux Voice [feed] – Desktop/mobile/freedom-focused Linux podcast from the ex-Linux Format/Tuxradar team.  Sometimes not as technical as I would like.
  • DevOps Cafe [feed] – good interviews, not too frequent.
  • The Cloudcast [feed] – Broad coverage of cloud topics, from both business and technical perspectives.  Some of their guest spots are a bit light on content, but overall still pretty good. Sometimes it’s hard to keep up with the amount of content.
  • Andy Stanley – A profound Bible scholar disguised as a catchy communicator. Multiple podcasts:

Added recently

  • Quirks and Quarks [feed] – weekly science show which features interviews with (mostly) doctorate-qualified scientists talking about their studies.  Compelling stuff – I never miss an episode.  They take a break over the Northern Hemisphere summer, which almost gives me withdrawal symptoms.
  • Software Gone Wild [feed] – Software-Defined Networking from Ivan Pepelnjak, a long-time networking expert.  Discusses interesting projects; not afraid to call out traditional vendors for their hype and vapourware.
  • Arrested DevOps [feed] – Deals a lot with of cultural and organisational issues relevant to the DevOps movement, as well as technical topics.

Podcasts I’ve dropped


ntpq: write to ::1 failed: Operation not permitted


The other day I got a bug report about check_ntpmon, which was reporting UNKNOWN status back to Nagios even though everything seemed to be working fine. A bit of debugging revealed that it was receiving the message on standard error:

ntpq: write to ::1 failed: Operation not permitted

This was a bit strange, because various links I found indicated that this message is usually due to firewalls:

But this host was not blocking anything, not to mention that check_ntpmon’s use of ntpq only ever uses the loopback interface, which is rarely ever touched by firewalls. A bit of further digging showed that indeed it was not the firewall, but a full conntrack table, with dmesg showing:

Aug 4 03:04:19 hostname kernel: [5226949.016837] nf_conntrack: table full, dropping packet

Increasing the conntrack limit fixed the problem.

(Just thought I’d document this here for posterity, since none of the links I found suggested this issue.)

Generic IT career/job seeking advice to the young starter

I’ve been asked a few different times to advise young (and sometimes not so young) people trying to get a start in IT.  Every story is different, but there are a few commonalities.  So here’s my generic IT career/job seeking advice in rough order.  Note that some of it is a bit Australia-centric.

  1. Prove to potential employers that you are a lifelong learner:
    1. If you don’t have a tertiary qualification, get one.  If you’re not sure how good a studier you are, start with a Certificate III or IV or Diploma.
    2. Keep going!  Get a Bachelor’s degree from a recognised institution.
      • Free MOOCs are fine if you just want to learn a new area with low financial risk, but they’re not worth listing on a resume.  They’re worth every penny you pay for them.
    3. Look for an employer who has a reputation for training their staff.
    4. Work on relevant vendor certifications as you have the time/money; hopefully you’ll find an employer who will at least pay for your exams & textbooks, if not courses.  Make sure the certifications are from reputable sources (e.g. Cisco’s are very good, Microsoft’s about average, many others quite poor); ask colleagues for recommendations.
    5. Think about a Masters, but not until you have a few years’ experience under your belt.  Don’t do it straight after your Bachelor’s.
  2. Make sure you know enough actual computer science to be a decent coder and scripter, regardless of your area of specialisation.  You should expect to automate yourself out of a job several times over the course of your career.
  3. Be prepared to start in a role that involves menial tasks; you’re not an expert in anything yet.  Sometimes the very best thing you might be able to do for an IT department is organise the storeroom, clear out rubbish, and put labels on things.
  4. If you can afford to do it, look for volunteer opportunities and work experience placements.  Do something useful rather than sitting at home sponging off your parents and playing World of Warcraft.  It’s totally legitimate to list volunteer experience on your resume.  Sometimes volunteer opportunities turn into jobs, or great references, or both.  (I started volunteering for Carinity, testing and cleaning decomissioned PCs, and eventually ended up as the IT Operations Manager.)
  5. Join relevant meetups and user groups; meet real people in the real world and try to learn what you can from them.  (Without being a creepy groupie, of course.)
  6. Follow interesting people on Twitter (ask me for some recommendations), and learn to ask good questions of them; sometimes they actually reply!  (Again, don’t be creepy about it.)
  7. When you apply for jobs, actually read the advertisement, and the selection criteria, and actually address the selection criteria in a structured manner in your application.
  8. Have a really clear, easy-to-read, spacious resume.  Yes, spacious; don’t try to fill it with too much.  Less is more if it’s well-designed.  Fit it on 1-2 pages if you can.  Most people reading resumes are reading 40 of them at a time; make yours jump out at them.  My abbreviated resume is 4 pages covering over 20 years in the industry, and I have an additional document containing a 1 page-summary of each major job that I give to people when things progress to interview stage.  (Some recruiters will like to have this earlier – that’s OK too.)
  9. Many recruiters know almost nothing about what actually makes a good IT professional, and don’t know, for example, that Linux, FreeBSD, HP-UX, Solaris, and AIX are all Unix-like OSes, and skills in one will usually transfer to another.  Make sure your resume contains a skills matrix with the right keywords for them to find you in a text search of their database.
  10. Have your work history and educational qualifications published on your preferred social networks – LinkedIn is the one recruiters seem to like most; Google+ and Facebook also have places for it.
  11. Seek seems to have dominated the online job ad market in Australia.  Register with other job sites if it takes your fancy, but definitely make sure you register for relevant email alerts on Seek.


Default permit still winning the security battle

I was stoked when Patrick Gray took up my suggestion to ask Marcus Ranum to reflect on “The Six Dumbest Ideas in Computer Security“.  I encourage you to listen to the interview for yourself, but my summary of it is that Marcus was mostly discouraged that very little progress has been made in computer security, while Patrick was of the opinion that a number of good lessons had been learned in certain key areas.

Patrick pointed particularly to Apple’s iOS as a commercially-successful example of default-deny execution policy.  Whilst iOS, Windows Vista and later, and even Android (to a lesser extent) have implemented varying levels of default-deny when it comes to execution of programs, I think default-permit policy is still the dominant mindset in our industry.  As I was listening to the interview, a few areas came to mind where it still seems to be true:

  • Outbound connections from client devices.  Despite the fact that client-based exploits have become the dominant method of compromising organisations (the so-called “Advanced Persistent Threat” which compromised RSA was started with a phishing campaign and an Excel-delivered Flash exploit) and security practitioners generally assume that client devices (whether PCs or phones) are routinely compromised, many (most?) networks provide allow outbound connections from client devices by default, often to any destination and sometimes on any protocol.  This is exacerbated by the appalling lack of proxy server support in most iOS and Android applications, which means that administrators of BYOD networks rarely have any choice in the matter if they want to provide a functional service.
  • Compounding the problem is the fact that generally when users browse or client-side apps make connections, all web sites are allowed.  In this area, enumerating badness (Marcus’ stupid idea #2) is still dominant; many (most?) web filtering solutions which attempt to protect clients from malware use a blacklist of known-bad sites.
    I’ve worked in K-12 school IT management, support, and consulting for a number of years, and every now and then the suggestion of whitelisting web sites comes up.  That’s usually all that happens.  Other fields (perhaps banking, industrial control systems, or medical applications?) might also consider it, but I suspect that they end up with similar conclusions (i.e. that it’s impractical to implement).  (I’d love to hear from anyone who has actually tried this in a real network.)
  • Scripting languages are a common exception to the default-deny execution policies of operating systems.  To my knowledge, Windows PowerShell is the only common scripting system which allows for script signing policies.  However, scripts can request that Windows simply turn this feature off, which defeats the purpose.  To my knowledge, no signing system or default deny policy has ever been implemented for Unix/Linux systems (other than the default protection provided by Mandatory Access Control systems like SELinux and AppArmor).
  • The Android application permissions system is one of my pet peeves. Android applications must inform the Google Play store about which security- and privacy-related features they intend to use.  This is good; however, permissions are approved when the application is installed, and users only have the choice of installing or not installing.  Many applications require permissions that are not obviously critical to their operation, but because users typically try to install an application because they want to use it, an informed evaluation of an application’s permissions is rarely performed at installation time.  Most applications are installed regardless of what permissions they request.  So effectively, this becomes a default-permit situation.  (Moxie Marlinspike‘s WhisperSystems seemed to be making progress on this before they were acquired by Twitter, and I hope that Open WhisperSystems takes up this work again in the near future.)

All of this says to me that we’re still living very much in a default-permit world, and there’s a lot of work to be done before we can confidently say that progress has been made in this department.

I’d love to hear any further thoughts on this; you can reach me through the feedback page, or on Twitter.


The tragedy of Vyatta Core's demise

Vyatta Core (VC) is one of my top fanboy loves.  It provides a firewall/router based on Debian Linux but with an elegant configuration system modeled on Junos.  Vyatta previously offered VC as a community edition of their commercial router (with somewhat reduced features and a lag time).  Their strategy seems to have changed when Brocade took them over, and unfortunately, the project has withered on the vine.  I think I can safely pronounce it dead:

The community has already mourned this loss and started seeking alternatives.  And yet, a vibrant community feel still exists in a few places:

  • Vyatta’s tweet stream still seems consist of approximately 50% retweets of users raving about how great the config system is and how the product solves their real world problems.
  • When Ubiquiti‘s EdgeRouter Lite using EdgeOS (a fork of VC6.3 ported to 64-bit embedded MIPS) was released, its forums exploded with user activity.  They grew so much in the first few weeks after release that I unsubscribed due to not being able to keep up with the message volume.  Unfortunately, Ubiquiti have always done their development internally, releasing source code as tarballs rather than the preferred method of hosting public git repositories.
  • Plenty of people (including me) are still deploying it in new environments.  When I first encountered VC, I liked it so much that I recommended to a client to use it for all their virtual routing.  We bought VSE and haven’t even used it, because VC does everything we need.  I would be happy to keep paying for VSE just as a way to support the future development of VC.
  • There are still some interesting threads cropping up in the forums.
  • Contributors to the unofficial Vyatta wiki have already started plans for a fork.

Why has this happened to such a great product?  I think it simply boils down to Brocade (and Vyatta prior to the acquisition) not understanding FLOSS (Free/Libre and Open Source Software) culture and how to run a successful FLOSS project.  Simon Phipps just wrote about how hard it is for companies to migrate to FLOSS and do it well, and VC is the latest casualty in this.  Red Hat is one of the few companies doing it consistently.

I think it’s several months too late for Brocade to fix this situation.  As the above links show, many people have lost faith in Brocade to do the right thing with VC.  But hypothetically speaking, what would be the best possible outcome?

  1. Brocade employs a full-time staff member as community advocate/liaison.  (I hereby volunteer for this job. Contact me for rates. 🙂  They reverse their decision about public git repositories.  They convince Ubiquiti to do likewise and work towards a common shared code base in the public git repos.  Together they make a kick-butt Vyatta Core, which is used as the core of future VSE and EdgeOS releases.  It becomes the Vyatta equivalent of Fedora to Red Hat’s RHEL, the OpenSUSE to SLES.  Brocade accepts contributions from the community, and begins providing per-incident or support contract-based services for VC as well as VSE.
  2. Next best thing: Brocade cultivates some good contacts in the community and begins regular source code drops to enable community respins, forming a relationship akin to that between RHEL and CentOS.  The community takes up the mantle of providing support on whatever the Open Source respin is called.
  3. The community forks from the latest available Vyatta Core, integrating Brocade and Ubiquiti code whenever possible.  This would be the least desirable outcome, but I see it as the most likely, given how hard it is to change networking vendors from the community side.

Outcome 1 is obviously a bit pie in the sky from me.  But there are sound reasons why Brocade should do it.  Most significant of these is that the virtual routing space is a hot area at the moment, and making VC the go-to choice for virtual routing would give Brocade the jump on Juniper’s vSRX and Cisco’s CSR1000v (by undercutting them on price).  Vyatta is modelled on the Junos style of configuration, so the learning curve for Juniper engineers would be trivial.  And if people go to Brocade for their virtual routing, they’re more likely to go to them for their physical network and storage switching.

Furthermore, VC has huge mind share amongst the people who deploy it.  The people I’ve met who use Vyatta love working with Vyatta.  I love working with it.  It has the potential to gain Brocade many extra customers and avocates, both paid and unpaid.  (Hint to Brocade: reasonably-priced support for VC would get you a lot of customers that wouldn’t have otherwise paid for VSE.)  Embracing the community is a great way to ensure that network/systems engineers who have the opportunity to influence decision makers will influence them towards Brocade.

Let’s hope it’s not too late, and that Brocade proves me wrong.  It would be a shame to see such a great project die.  (And I am serious about the offer to become their community advocate.  Call me.  🙂