This page describes the rack space at the Forrestal Campus as configured for members of the Computer Science department. The facility houses (and is sometimes referred to as) the High Performance Computing Research Center (HPCRC).
Specifications
The CS area of the machine room is configured with 25 racks. Seven racks are used for departmental computing and networking infrastructure including a communication rack ("comm rack") that contains patch-panels back to the other 24 racks. Note that this facility (i.e., the room, power, and cooling) is managed by OIT. CS Staff manages and allocates the rack space as well as the network for the CS area.
Each rack has the following standard features:
- 42 rack-units ("U") of available space. These are 44-U racks of which 2-U is used by the patch-panel leaving 42-U available.
- 208V single-phase power
- Cooling/power per rack of up to 18kW.
- 12 cat-5e ports on the top-of-rack patch-panel that terminates in the comm rack. These ports can be patched to 1Gb/sec network ports that are either "inside" or "outside" the CS firewall. The patch-panel ports also allow direct connections to other racks in the CS area.
Mailing List
Administrators of equipment in the CS racks in this facility (and all CS personnel with access to the facility) must subscribe to the cs-151 mailing list.
Power
All rack power is single-phase at 208 volts. The University provides power distribution units (PDUs) and appropriate power cords.
The power is backed-up by UPS and diesel generator.
The facility is designed to provide power for the racks indefinitely in the event of commercial power failure.
Cooling
The CS designated racks use water-cooled rear doors. The racks are sized to cool equipment that uses up to approximately 18kW of power.
IMPORTANT NOTE: Due to the nature of the water cooled racks, not all equipment is suitable for this facility. For the cooling to "work," the equipment much pull air in from the front and force it out the back. In addition, the fans must be sufficiently powerful to force the air through the rear, water-cooled door at the back of the rack. If the equipment in the rack is not sufficiently sealed and/or the equipment fans are not able to push warm air at sufficient volume, the rack will overheat. In addition, the facility is designed to operate at a higher temperature than previous generation computer rooms. Modern, enterprise-class equipment should be fine. Contact CS Staff if you have any concerns prior to purchasing equipment for this facility.
The facility has backup systems in place for cooling.
The facility is designed to provide cooling for the racks indefinitely in the event of commercial power failure.
Network Connections / Top-of-Rack (ToR) Switches
As with all other locations on the CS network, equipment and their network configuration are subject to CS Staff approval.
Each rack has a 12-port patch-panel that terminates in the communication rack. This provides the uplink and allows racks to be patched together.
The entire CS area of the machine room (25 racks) is currently configured as a network island. The main switch in the CS comm rack has 2 redundant 10 Gb/s Ethernet channels back to our core switch in the CS Building. In normal operation, these 2 channels work together to provide an aggregate 20 Gb/s channel. These channels follow separate paths back to the CS Building; if one channel fails, the link gracefully degrades to a 10 Gb/s channel.
[Note: the remainder of this section is identical for both of the CS co-location facilities.]
Note that all ports exposed to users are "untagged" and are configured to operate on a single VLAN (per port).
The expectation is that the equipment in each rack may require some number of network connections that are "inside" the firewall and some number of network connections that are "outside" the firewall. These connections can be made with the provided patch-panel. If a researcher needs more connections than available in the patch-panel, they will need to use a switch. Note that the CS networking infrastructure maintains physical separation between "inside" and "outside" the firewall. In some cases, you may need more than one switch. Contact CS Staff for details.
If you supply and manage your own switch, its uplink will be to a single VLAN.
If you have more complex networking needs and/or want CS Staff to manage your switch as part of the departmental infrastructure, you will need to purchase the departmental standard switch, an HP E6600-48G-4XG (note: this model is no longer available; CS Staff is in the process of specifying a new departmental standard HP switch), along with a 5-year software support contract.
If two researchers are assigned to the same rack and they need more ports than provided by the patch-panel, they may jointly purchase a switch and share it.
Location
A 3 mile drive from the CS Building. Tiger Transit takes about 1 hours each way due to multiple stops. Precise location details are here (login required).
Physical Access
Faculty, Staff, and Graduate Students can be granted 24/7/365 unescorted access to the whitespace; they need to fill out an online form. Contact CS Staff for the link. Graduate Students and Postdocs should pre-arrange for their adviser to also send an email request to CS Staff verifying their need for access and specifying which racks will be accessed.
Undergraduate Students are not allowed unescorted access to the facility.
Once in the whitespace, one uses their TigerCard at the automated key cabinet to check-out keys for their assigned racks.
Current list of CS people with access is here (login required).
Rules
Basic machine room rules apply:
- No food or drink in the room
- No storage of equipment in the whitespace
- All equipment must be inside the racks
- All equipment must be registered -- Any equipment that has an Ethernet interface must be registered in the CS inventory system. This applies even in the case that the equipment will be behind a private switch.
- Special Rules Apply to After Hours -- When using the facility after hours (outside of 7am-7pm M-F), one must proactively contact public safety when entering and leaving. Details are here (login required).
Costs
Researchers are not charged for basic space, power, cooling, or networking (i.e., those outlined in the Specifications section above). If a researcher needs resources beyond those outlined above, these costs must be picked up by the research project. For example, needing 10 Gb/sec uplinks to a given rack, additional/specialized ports requiring a CS Staff managed switch, or specialized power.
Getting Rack Space
Rack space is assigned to faculty upon request and is subject to availability. If you plan on using rack space in the room, please contact CS Staff well in advance. (Just because there is empty space in one of the CS-assigned racks, it doesn't mean that it is available.) Due to airflow considerations, rack space is not necessarily allocated in specific sized chunks (like the quarter-rack allocation in CS 002).
Current rack assignments are here (login required).
Vacating Rack Space
When vacating rack space, all equipment provided by CS Staff/OIT must be returned. This typically includes power strips, patch panels, cable management, power cords, network cables, and the rack itself. All other equipment must be removed or designated for surplus.