This post is the story of my first practical look at Junos on Juniper EX-series switches.
One day last December, Skeeve Stevens from eintellego opened a can of worms by offering a deal on Juniper equipment to all network engineers on the AusNOG mailing list. I had been looking for an excuse to try out Juniper switches for a while; my office switch, an HP ProCurve 3400cl (J4905A), was very loud, and the EX2200-C was the only affordable fully-managed Gigabit fanless switch which i could find from a major networking vendor. So i ordered one and waited patiently. I expected it to take 6 weeks or so to arrive (since apparently all the cool kids ordered one too), so i was pleasantly surprised when it arrived the day after Boxing Day. (For those non-Commonwealth inhabitants reading this page, Boxing Day is the day after Christmas Day; it's always a holiday here in Australia.) Over the Christmas/New Year break, i spent three days learning and playing with Junos. I don't claim this as a fair, balanced, or reasonable review - it's simply a recounting of my experiences.
I wanted to get a good general idea of the Junos platform and set it up as my main office switch, moving the HP to my test lab (which is only turned on when i'm using it). So the basic task i set myself was to learn enough Junos to replicate the functions of my ProCurve as completely as possible. I knew ahead of time that it wasn't going to be a 100% fit - the 3400cl was a great value switch for its day (and still is, in many ways), and had a lot of features included out of the box (like OSPF and PIM-DM multicast routing). Junos requires an additional license to implement OSPF, and it wasn't discounted (buying it would nearly have tripled the cost of the switch), so i knew that i'd be eliminating that part of my config. My other OSPF devices are VMs running Debian Linux with Quagga, and Cisco 1700 & 1800 series routers running IOS, so i simply connected them to the same VLANs and phased out the ProCurve as a router.
My first few impressions of the hardware were positive:
- The switch has a solid metal case with a nice heavy feel to it.
- It was well packed, and came with a very well-packed rack mount kit (perhaps a little too well-packed):
- It also came with a rather longer-than-expected, but solid, power cord retainer:
- It sports wall-mounting holes as well as a rack-mount kit (hat tip to Chris Lathem for pointing this out):
The biggest turn-off for me on the hardware side was the realisation that Juniper switches use zero-based port numbers. Seriously? Being an old C programmer, i think of this as "proper" from a programming perspective, but try telling an end-user (troubleshooting an issue in a branch office over the phone with tech support) that port 1 is the second port not the first. I can't think of any reason other than compatibility with "the way they've always done it" on their routing platforms, and i consider this self-indulgent behaviour on an equipment manufacturer's part. Other vendors manage to start unit or chassis numbers at zero without being bloody-minded about starting port numbers at zero (although HP's Comware switches start their port numbers at the bottom left rather than the top left, which is almost as frustrating).
After my inspection of the hardware, i moved on to plugging it in and inspecting the software. Bootup took a bit longer than I expected (the system uses a 1.2 GHz Marvell ARM CPU - the same as in my QNAP TS-219P), but it looked reassuringly Unixy: Junos uses a FreeBSD-based management plane. After seeing the switch booted through to a login prompt, i logged in as root and had a poke around. Typical Unix commands like ls, df, and ps all had their usual functions, and one runs "cli" to get into the Junos configuration system. I worked for the first day or so from the serial console.
I noticed some disconcerting dates in the initial copyright notice:
"This product includes FreeBSD software developed by the University of California, Berkeley, ... 1979, ... 1994. "
Does this mean that 1994 is the most recent date they synchronised with the FreeBSD project? I hope this is not true, and that Juniper are actively engaging with the Open Source project upon which their code is based.
"GateD software copyright © 1995, the Regents of the University. All rights reserved. Gate Daemon was originated and developed through release 3.0 by Cornell University and its collaborators. Gated is based on Kirton’s EGP, UC Berkeley’s routing daemon (routed), and DCN’s HELLO routing protocol. Development of Gated has been supported in part by the National Science Foundation. Portions of the GateD software copyright © 1988, Regents of the University of California. All rights reserved. Portions of the GateD software copyright © 1991, D. L. S. Associates."
It has been a very long time since i've seen a Unix distribution old enough to contain a gated implementation, and i sincerely hope that no vestiges of this code remain.
During boot an alarm light appeared on the front panel. I was afraid i had a hardware fault and would have to return my shiny new switch! After some investigation in the relevant manuals, i found the command to view the active alarms ("show chassis alarms") and found that it was due to the management interface being down. I already had a management VLAN in my network, and i was expecting this switch to host it. I investigated the management LAN (at this point a shout-out to the helpful folks in #juniper is due) and found that it was not VLAN-capable at all. It's more like a NIC for the switch's management plane. So it's a true out-of-band network, and can't be bridged onto any part of the data plane. I quickly decided not to use it (and disabled the alarm with "set chassis alarm management-ethernet link-down ignore"). I can understand why the management port is provided, but i'd much prefer that it could be bridged (in software, if necessary) onto the relevant VLAN for installations that aren't large enough to have a dedicated physical out-of-band management network. I also think enabling alarms for the management LAN being disconnected in the factory default configuration violates the principle of least surprise.
Software Updates and Support
When commissioning a new switch, router, server, storage array, etc., one of the first things i do is look for the latest firmware/software to ensure i'm up-to-date with security patches. This proved to be a bit of an issue for Junos. As a newly-arrived customer, i didn't know what to expect; HP offer all software updates for their switches on their public web and FTP sites; with Cisco it's a bit more involved, requiring a login to their site, and with routers you need to know the right things to say to TAC if you don't have a support contract (with switches they're more generous - could this have anything to do with the existence of GNS3?).
With Juniper, it seems one must have a login on their site that is specifically granted access to register devices. I had already registered a user account on their site in order to download the documentation and view their certification materials, and i was somewhat perplexed and frustrated to find that i couldn't even register my switch until Juniper had changed my account to have "real customer" status. Their support staff were first class, though, and got it sorted out pretty quickly.
My reading of documentation began with Juniper's "Day One" guides, some of which i had downloaded prior to receiving the switch. These guides range from the 66-page "Exploring the Junos CLI" to the massive "Complete Software Guide for Junos OS for EX Series Ethernet Switches, Release 11.1", weighing in at 4628 pages. Suffice it to say i only used the latter as a reference. If you've used Vyatta at all, you'll find the former guide a fairly dull read. A quick skim to see the differences between Junos and Vyatta was all that was necessary for me. (Vyatta is a deliberate clone of the applicable parts of Junos' interface.) "Configuring Junos Basics" was a useful guide to the initial setup of the switch, some parts of which have made it into my Network Engineer's Rosetta Stone - initial setup wiki page. At 84 pages, it's pretty digestible in a short CLI session. I used the free PDF format downloads, but most of the Day One guides are available for purchase in print format also. (Curiously, they cost US$0.99 to purchase in Kindle format, even though Kindle can read PDFs just fine.) The Day One guides (and the follow-on series "This Week") are generally excellent in quality, and formatted for easy comprehension, although i thought that the headings could have been clearer in some cases. ("This Week: Hardening Junos Devices" devices is the one that i made mention of in my notes.)
My checklist for features to convert from ProCurve to Juniper looked like this:
- Initial setup: user logins, hostname, management IP address, default gateway, DNS, NTP, time zone
- Essentials: VLANs, Rapid Spanning Tree Protocol (RSTP), LACP trunk to VM server, port labels, syslog, SNMP, ssh public keys
- Hardening: login inactivity timer, login banner, disable unused services (including telnet and web management), management ACLs, eliminate LLDP data leakage on Internet interfaces, Spanning Tree BPDU protection/BPDU filtering
- Nice-to-have: QoS on VLANs, port mirroring, IGMP querier, DHCP relay
Initial setup was pretty straightforward, especially given the aforementioned Day One guide. Having worked with Vyatta for a few months, i found it easy to adjust to the Junos CLI style. It's a simple and elegant configuration system which made instant sense to me the first time i used it, and Junos worked very much as i expected. When logged in via ssh, responsiveness of the EX2200-C at the CLI (in terms of character echo delay) is noticeably snappier than the ProCurve 3400cl it replaced (and many other similar devices).
Junos has a number of powerful configuration features. The ones i came to appreciate most were configuration groups (i used this to set up SNMP) and interface ranges (which i used to set up my VLAN assignments). The latter is very powerful: a range's name can be used in place of an interface name almost anywhere, and it lives in the configuration as a real entity, as opposed to the temporary nature of the equivalent on IOS. (Recent HP Comware versions allow a port list to be permanently associated with a name, but store the configuration in the individual ports rather than the interface range itself.)
Configuration is very slow to commit on the EX2200-C compared with Vyatta on an Intel server (understandable, given the low-end CPU); "commit | display detail" set my mind at rest that it was doing useful things, so i just got used to waiting for my commits to complete. When working in configuration mode, using "run <operational command>" works similarly to IOS' "do <operational command>" with the added bonus that tab and ? completion works. This was very helpful when i found that "run show interfaces brief" is not very brief, and "run show interfaces terse" is probably more like what i expected of briefness.
There were a number of features in the configuration process that struck me as sub-optimal, or at least unpleasantly surprising:
- There are separate NTP servers during boot and normal operation, with separate configurations for each. (Presumably under the covers the former are used as arguments to ntpdate during boot whereas the latter are used to configure ntpd.) This strikes me as a rather obscure feature which probably resulted from a large government department insisting on its inclusion in a large RFP. The very strange thing about this is that only one boot NTP server is allowed, whereas multiple NTP servers are allowed during normal operation.
- The Junos shell interprets the pipe character (vertical bar) in a counter-intuitive fashion: the pipe is a well-established mechanism on Unix/Linux, Windows, IOS and Comware (amongst others) that sends the output of the first command to the input of the second command, which is a filter. Junos seems to use it as a generic way to affect the display of the output of the first command, which would more regularly be done by flags or additional arguments on other operating systems (e.g. "ls -l" on Unix/Linux, "dir /b" in DOS/Windows, "show ip interfaces brief" on IOS). This is another departure from the principle of least surprise, and i'm glad that Vyatta didn't follow Junos in this respect (thus "show | compare" in Junos is simply "compare" in Vyatta).
- The Junos best practice (i can't remember which guide i read this in) is to set the IP address of lo0.0 to 127.0.0.1/32 rather than 127.0.0.1/8, as it is on Linux and Windows. This is more likely an issue when using Junos for routing rather than switching, but i still couldn't see a reason for it.
- Entering multi-line banners is annoying: one must use \n within the string instead of just pressing enter and finishing the string on the next line.
- VLAN setup is a little tricky. I initially tweeted: "#Juniper #EX2200-C impression #9: #Junos makes more sense as a router OS than a switch OS. VLAN setup is almost as painful as #Mikrotik." But after spending more time setting up VLANs on Mikrotik recently, i can now formally rescind that remark. :-) VLANs are configured in two different ways: under the interface (or interface range) and under the VLAN definition. Juniper recommends using the former for trunk (tagged) interfaces, and the latter for access (untagged only) interfaces. Because i often play around with my VLAN assignments, i ignored this advice and defined them all as if they were trunks.
- Junos includes a concept of an "edge" port, but i couldn't find a clear definition of it in the manuals i was working with, and as soon as i tried to apply BPDU filtering using "bpdu-block-on-edge" in interface context, i discovered its inflexibility, and discarded it in favour of "set ethernet-switching-options bpdu-block interface NAME".
Almost everything i had on my checklist to convert from my 3400cl was done at the end of my three-day lab. The exact mechanisms for each feature (on both the Juniper EX2200-C and the HP 3400cl it replaced) can be found in the config files attached to this post. A few features seemed to be missing or deficient (although this could be due to my poor research skills rather than a feature gap):
- There is seemingly no equivalent of IOS' "switchport trunk allowed vlan remove ID". I couldn't find a way to set up a VLAN trunk such that all VLANs except a particular list were allowed.
- There's also seemingly no equivalent of ProCurve's "lldp admin-status PORTNUMBER RxOnly" (for allowing receive but not send of LLDP info); Julien Goodwin suggested that ACLs could be used to fix this; i simply chose to disable LLDP entirely on my ISP uplinks.
- More perplexingly still, there's seemingly no equivalent of IOS' "no ip routing". Seems very strange that there is a Junos kernel option to drop all TCP packets (i can't think why anyone would want to turn on this option, since it would prevent remote management via ssh), but no option to turn off IP routing. Workarounds suggested by the IRC crowd include:
- blocking traffic between vlans by policy
- using multiple routing instances and putting each VLAN in its own instance
- create only L2 VLANs with no L3 interface (which is the option i chose in this case)
- There's no way to enable all interfaces for sFlow - it appears one must add each interface individually. And further, there's no SNMP write access at all, so i couldn't just configure authentication and leave the heavy lifting to sFlowTrend.
- Probably the feature i spent the most time investigating and discussing on IRC (hat tip especially to Kurt Bales) was VLAN QoS priorities. Junos' QoS capabilities are extensive, but configuring them is not as straightforward (some would say "simplistic") a matter as ProCurve makes it (vlan X; qos priority N). Because my home office switch is never congested, i decided it wasn't worth spending the time to learn this part of Junos until later.
Junos is a mostly-logical network OS with an elegant configuration system and excellent freely-available documentation. The quality of the EX2200-C hardware is first rate, and i suspect this only increases in subsequent models in the series. I'm glad i took the time to get familiar with the basics of Junos, and i'll probably try to be certified to their entry-level standard if time permits. I don't think Juniper's offering is unique enough to cause me to choose them ahead of Cisco or HP (the latter's Comware switches are still my go-to choice), but they're certainly an excellent choice for quality managed switches, especially for those organisations with an investment in Juniper's routing platform.