Categories
Announcements Changes

Temporary Outbound Queue Increase for Hurricane Sandy

Due to widespread outages on the East cost due to hurricane Sandy many mail servers hosted there are down for a wide range of reasons, most without any ETA for restoration. We’ve temporarily increased the maximum queue lifetime for our outbound mail service (SMTP AUTH) queue to 10 days instead of our default of 3 days. This means we’ll keep trying for up to 10 days to deliver SMTP AUTH submitted and smarthosted mail to destinations that are currently unavailable but are expected to eventually recover.

Our incoming mail service (Secondary MX, SMTP Redirection, etc.) offers a queue lifetime of three weeks; we will not be adjusting this value. Customers utilizing our incoming mail services in conjunction with impacted east coast destinations are advised to take advantage of our Mail Mirror feature if they wish to configure emergency mail boxes. Log in to the account control center and see the Mail Mirror section for more information.

Categories
Status

Level 3 Transit Impact Report

On October 25, 2012 beginning at 07:07:26 local time our Level 3 transit circuit was impacted for 21 minutes. During this time we continued to receive a full BGP route table, our route announcements were accepted, and bidirectional forwarding detection (BFD) remained in an “up” state, but traffic was not routing/forwarding properly. In a typical circuit or router fault condition BFD would cause the BGP neighbor to go idle and our autonomous system would recalculate new transit paths. In this case, however, a manual administrative shutdown of the interface (and BGP neighbors to bring the interface up for testing) was required. After ceasing to accept/announce routes via Level 3’s AS, transit through and to Roller Network’s AS was fully restored. At no time was traffic transiting via Sprint impacted.

Official explanation from Level 3: “Initial Blue Major (Cisco 7609) Due to routing issues slot 3 needed an emergency reload. It is believed that since Cisco messed with that card and ran debug commands last night and this morning, that this caused some corruption or instability in the card.”

We initially reported this issue via our Twitter account @rollernetnv.

Categories
Uncategorized

Charter Communications Now Available

Charter Communications is now available as an on-net carrier in our colocation facility. Charter is using a Cisco ME 3600X-24FS for additional site capabilities over of their regular ME 3400 series installations.

Customers or prospective customers wishing to utilize this carrier at our facility should contact Roller Network for our Site ID. This ID can then be used by your Charter account manager to locate our facility.

Categories
Fun Stuff

Inside Google’s Datacenters

Google released some pictures of inside their datacenters. We found two things interesting from this:

  • They’re using tapes for backup. A lot of them: large robotic libraries with what appears to be LTO tapes. Our LTO autoloader is only 8-slots. They work the same though, but you can walk inside of theirs.
  • Fire suppression is “highly-pressurized water” that’s been cleaned and treated in case it ever has to be used. In the past some datacenters started trending towards alternative forms of fire suppression such as CO2 or clean agents like Novec, FM200, Sapphire, etc. However, this trend is reversing with many modern datacenters favoring traditional fire sprinklers. Interlocking systems all but eliminate accidents, and unlike in the movies real world glass bulb sprinkler heads trigger individually. If we had to guess it’s because the cost to benefit – including maintenance – wasn’t worth it. Clean agent systems to cover the large spaces as shown are extremely expensive.

Check out the pictures at google.com/about/datacenters/gallery/ if you haven’t seen them yet.

UPDATE: it turns out at least one of these pictures was faked for whatever reason; others are possibly not as accurate as they claim to be.

Categories
Announcements Changes

Primary DNS Major Updates and New Features

Roller Network is pleased to announce the release of our newly updated Primary DNS service with several new features and an all new look. The new major features are:

  • Templates (Personal Plus and higher).
  • TSIG signed request support.
  • Per-zone AXFR support.
  • Added the SSHFP DNS record type.
  • New layout using jQuery UI.

And enhancements:

  • All record types can now be edited.
  • Any default records can now be edited or deleted (defaults are now a template).
  • Added AAAA records to the default template.
  • Check that CNAME doesn’t exist with other records when adding a record.
  • SOA SERIAL can be date-formatted or simple integer auto increment.
  • SOA MNAME can now be changed.
  • Many UI and functionality refinements.

We also had to switch to a new jQuery plugin-based main menu across the entire account control center because the old one broke when combined with jQuery UI. It’s much cleaner in the back end and requires less code, which is something that should have happened anyway, but unfortunately was one of those unexpected things that took extra time to deal with.

You may also notice a “DNSSEC” tab if you look closely; we haven’t enabled DNSSEC support just yet because we didn’t want to keep delaying these updates, but we’re working on it.

The new layout is based on jQuery and jQuery UI but does not strictly depend on it for core functionality. It should still work even in text-only command line browsers in a pinch. This new Primary DNS interface is a major visual change for us, mainly because it was impractical to cram all of the new features into the old static layout. We already sprinkled some jQuery into the colocation section, but the Primary DNS section will have a wider audience for feedback. Over time we plan to push this style of interface to all sections of the account control center.